GAE Tools – Google App Engine and Twitter
Tonight I have had a win. It’s a small win, but a win nonetheless. I have uploaded a small application which is making use of my new GAE tools library. The purpose behind this library is to make application development in the google appengine even more enjoyable. At this stage a python version of the library has been released, and I don’t currently have plans to build a java equivalent as I’m finding quite a pleasant language to develop in (not a language I have coded much in before).
The GAE tools library started it’s life with me calling it “Twawler” or basically a twitter search wrapper, but it grew to include other functions inside the library the meant it deserved a more generic name. The high level functions are are described below:
SlicedTasks
With the introduction of CRON jobs into AppEngine when google released their Java support, the possibility of background processing for appengine applications became a reality. The limitation, however, is that a cron job on appengine executes in a similar manner to a web request and thus will terminate after 30 second of activity. The Sliced Task class (very raw at this stage) is designed to maximize your processing that you can do in the context of this request, but not terminate and leave you hanging. It’s proven very effective for my twitter search implementation and would work very nicely with processing any kind of data that can be treated as a queue.
Twitter “Twawler”
A library that wraps the twitter search API. It’s very rudimentary at this stage but will be extended in the future. The major victory here is that it implements oauth authentication instead of basic authentication. The trick here at this point is that as I’ve designed it to run with background tasks, on your first call to the your processing script you will need to pass an “init=True” parameter to the script (eg. http://blah.com/tasks/backgroundprocessor?init=True). You will then be taken through the oauth handshaking process and once complete the access token stored in the database.
Additionally, configuration information is stored in a yaml file under a conf directory for your GAE application (/conf/twitter.yaml). The content of this file is basically:
consumerKey: your-oauth-key consumerSecret: your-oauth-secret
Searching for tweets works quite well at the moment, and I have an application deployed to appengine and as I want to make sure the performance is within tolerances for the free limits on appengine (so far, so good).
If you would like to get a copy of the code, then free to head to either the Ohloh site for the project, or to the Google Code repository
Any feedback on the library would be appreciated, and I’d suggest you subscribe to my RSS feed for updates on the library itself or applications I’m building that use it.