Writing WW1 tweets in advance, need script to spread them out over each day

sneakyimp

As some of you may know, 2014 marks the 100th anniversary of the start of the First World War. Being particularly interested in this war, I have started a twitter account that links to news articles for each day. The idea is to give folks a day-by-day feel for the war much like you might get were it happening right now. You can see it here:
https://twitter.com/oldgreyhorror

My sources for articles provide me with anywhere from 5 tweets a day to 30 or more tweets a day. It varies. As you can imagine, it's fairly tedious work to locate it, read it, summarize it, and tweet it. I'm trying to simplify the process. My plan is to:
1) Acquire the articles in advance and load them in a database. Each tweet/article has a date associated with it. E.g., 1914-08-01. I'll use some nepotistic laborers for this bit I expect.
2) Write the tweet part myself.
3) For some articles, I want to pick an exact time for the tweet to issue forth. E.g., "Franz Ferdinand shot in Sarajevo" would have a specific time. For other articles, having to choose a specific time is unnecessary and would be tedious. For these "whenever" tweets, I would just like to spread them out evenly throughout the day.

My question is whether anyone has a good recommendation for a cron job algorithm that will effectively tweet some of the tweets at precise times while spreading the other tweets out evenly during the day. This is complicated firstly by the fact that some tweets need to be precisely timed and also by the fact that some days have lots of tweets (~30) and some have very few (~5).

I'm thinking that we need to generally distinguish timed tweets from non-timed ones but then when I try to think of a way to spread out the other miscellaneous other tweets throughout the day, things get more confusing. It has occurred to me to count up the remaining "miscellaneous" tweets for the day and calculating a probability based on the time remaining and the number of tweets remaining, but that seems pretty messy. Calculating a time for each tweet in advance may not be feasible because I might be adding tweets on the actual day they are to be tweeted -- when would such a calculation occur?

Any thoughts are welcome.

dalecosp

What about this:

The DB would note if the event was precisely timed, or merely a specific day.
The cron job would run every $x minutes regardless ($x might be determined by the granularity needed for the precisely timed events).
A flag could be set, say, on the file system, informing the system of the time of the last non-precise tweet.
The cron job script would then run any precisely-timed tweets, check the remaining tweets of the day against the time remaining, and, IF enough time had passed since the last one, do a tweet.

I'm sure it could use some refinement ....

johanafm

How about

Pre-store tweets along with either exact datetime or simply date ("random" throughout day)
Create a php script which appends entries to crontab (A)
Create a php script which post tweets (😎
1. Set up (A) to run once per day, say a few minutes before midnight
1.1 reading in all of tomorrow's tweets' datetimes
1.2 append entries of (😎 to crontab for each tweet to be posted

For tweets without exact time, you only need to decide how to distribute them: truly random, evenly throughout day, centered around breakfast, lunch and evenings etc. But once a distribution scheme has been created, they can easily be entered into crontab in the same way.

and in other words leave the actual tweet post time handling to crontab which is pretty much its purpose.

dalecosp

Just my opinion, and we all know that post count doth not a genius make (and I aren't no genius 😃) ... as a server admin-type, I don't like my crontab files getting messy. By putting all the jobs in cron, you're essentially making the crontab into a data-holder, which isn't what it was designed for.

Weedpacket

You could have the cron job to run once each day. That job consults the db for any tweets to be made that day (generating times however for those that don't have them however you choose to distribute them); then generates a bunch of at jobs, one for each tweet - for the rest of the day the timely updates will be made by the at daemon (in other words, remember that there is an alternative to cron for "one-shot" jobs).

sneakyimp

Thanks for the responses, guys.

While I understand that there is certain efficiency to be gained by creating at or cron jobs -- namely that I wouldn't need to run a cron job every minute or five minutes -- the drawback of using this approach would appear to be that I must always compile my tweets in advance of the day that they appear. This is somewhat inflexible -- I couldn't add a tweet to the database for today because it wouldn't necessarily be properly scheduled.

I like dalecosp's idea pretty much. I have no problem running my cron job every five minutes or so. It'll be running on my workstation (a six-core machine) and the number of tweets is hardly a lot of data so I'm not especially concerned about resources or efficiency. Chances are that the script won't do more than 5 tweets in one run ever.

That said, I'm going to try and work out a script that takes into account various considerations and contingencies:
Is there any chance that this algorithm might fail to tweet a record on it's intended date? Like what happens if we have a lot of tweets left and only a little time? We may need to tweet more than one tweet.
what happens when we find not-yet-tweeted records that precede today? E.g., it's 2014-08-01 today and we have as-yet-untweeted records for events on 1913-07-31. Should I just tweet them all at once? Or tweet N of them with each run of the script. NOTE: I would ideally like all of my records to be tweeted in order by date. It would offend my sense of order for them to be out of order date-wise.
* I'm starting to think that if a record should be tweeted on date YYYY-MM-DD that it would be a good idea to tweet it during 'peak hours' rather than waiting until 11pm or midnight. I'm starting to think about distributing tweets across some time window.

Given the nature (and language) of my content, it seems that I would be wise to cluster my tweets during peak usage hours in GMT to increase the likelihood that they will be seen (and retweeted). Any thoughts on how to determine peak tweet hours for folks living around London?

dalecosp

Cron takes so little system resources, I mightn't be too scared to run it every minute. The problem would be if it became active ... tweeting might take a minute and then you'd be up a creek. 5-minute granularity seems about right to me without having attempted something like this before.

I do a lot of cron work though ... the crontab is printed and hung on the cube just left of my workstation, and it is currently two printed pages long ...

sneakyimp

So it occurred to me this morning that I haven't yet sorted how to post a tweet from a PHP script. Anybody done this before?

laserlight

sneakyimp wrote:
So it occurred to me this morning that I haven't yet sorted how to post a tweet from a PHP script. Anybody done this before?

Nope, but a quick check shows that it is a simple matter of say, using curl with the Twitter REST API. Unfortunately, the API does not provide a way to schedule the tweet, otherwise you would not need the cronjob in the first place (or maybe you would use an asynchronous message queue instead).

Derokorian

I have done this before, and I may still have the code (I'm not sure, its something I did for someone else). If you haven't gotten it figured when I get home tonight, I will take a look around for it. I'm sure it'll be ugly but could get ya going if you aren't already 🙂

sneakyimp

Thanks, guys. I don't expect this to solve my cron problem. I've found this and am trying to get it working, but it's a bit confusing. It would seem that you can't store some kind of credentials somewhere that allow tweeting to your account. Rather, the twitter api seems to me to allow you to create an app that can rope in other twitter users to use your app, which is not really what I'm trying to do.

Still, I've tried creating an app as described here, I've got the pages doing the various session checking and redirects. When I try to get a twitter authorization URL, it seems to work and I see a page that says "Authorize EventAutoTweeter to use your account?" and I click "sign in" and it never seems to work. It would appear that things are working locally such that I can forward a potential user of my twitter app to twitter for authorization, but once they arrive at twitter, clicking 'sign in' doesn't seem to work. no error messages or anything.

laserlight

sneakyimp wrote:
I've found this and am trying to get it working, but it's a bit confusing. It would seem that you can't store some kind of credentials somewhere that allow tweeting to your account. Rather, the twitter api seems to me to allow you to create an app that can rope in other twitter users to use your app, which is not really what I'm trying to do.

Not quite: another quick check shows that there are a number of ways to obtain an access token. In particular, you can generate a token and token secret from dev.twitter.com. This should allow you to skip the steps that requires user interaction for oauth 1.0a authentication.

sneakyimp

Thanks for that, laserlight. Between looking through the various ways to get tokens and continually checking the code from that github code I was playing with, it dawned on me that most of the code from github was dedicated to extracting a temporary token/token-secret for some random twitter user so your app can mess with their account. I also was confused by the fact that the credentials listed on dev.twitter.com under "your access token" for my app were read-only. I tried "recreate my access token" which immediately showed no change. checking back a few minutes later, I saw that I had a new access token and that it was read and write.

All that said, this bit of code is working swell for me now and I have auto-tweeted my first tweet:

		//attempt to use locally stored credentials to just tweet to twitter

	// Load required lib files
	require_once("/var/www/public/twitter/twitteroauth/twitteroauth.php");
	require_once("/var/www/public/twitter/config.php"); // this defines all 4 constants used in the constructor in the next command

	// Create a TwitterOauth object with consumer/user tokens.
	$connection = new TwitterOAuth(CONSUMER_KEY, CONSUMER_SECRET, TWITTER_ACCESS_TOKEN, TWITTER_ACCESS_TOKEN_SECRET);


	// if you do not authenticate, you should get a 401 or some HTTP code that is not a 200
	// or at least that seems to be the case
	$content = $connection->get("account/verify_credentials");

	switch ($connection->http_code) {
		case 200:
			// all good!
			break;
		default:
			die("Twitter auth failed! Bad http code returned from twitter auth: " . $connection->http_code);
	}

	// Some example calls
	//$connection->get('users/show', array('screen_name' => 'abraham'));
	$connection->post('statuses/update', array('status' => date(DATE_RFC822))); // i tried this one and it worked
	//$connection->post('statuses/destroy', array('id' => 5437877770));
	//$connection->post('friendships/create', array('id' => 9436992));
	//$connection->post('friendships/destroy', array('id' => 9436992));

	switch($connection->http_code) {
		case 200:
			die("apparently successful?");
		default:
			die("Bad http record returned by tweet attempt:" . $connection->http_code);
	}

Progress!

So I'm wondering if anyone has any thoughts on what might be the best time of day to tweet? I.e., any techniques to sniff out optimal times of day for tweeting for the most exposure.

laserlight

sneakyimp wrote:
So I'm wondering if anyone has any thoughts on what might be the best time of day to tweet? I.e., any techniques to sniff out optimal times of day for tweeting for the most exposure.

This stuff should be quite easily found by searching the Web, e.g., The Science of Social Timing Part 1: Social Networks. However, I wonder if it would be more thematic for you to tweet at those times corresponding to the time of the day that the given event happened in history.