My Python Mechanical Turk API — mturkcore.py

mturkcore.py is now on Github: https://github.com/ctrlcctrlv/mturk-python

The current state of Mechanical Turk API’s for programming languages is somewhere between “horrible” and “confusing”. It is completely astounding to me how something as simple as a REST/SOAP webservice is confused so much in every single SDK I tried.

The Perl SDK is totally broken and has been for quite some time. I first realized this in one of my first experiences as a Mechanical Turk requester when I had to go through a totally convoluted process to install it to use a script that a friend of mine wrote (think, failing every test and compiling an old version of Perl then forcing the install for it to “work”). This version seems to work, but is not easily discoverable.

The PHP SDK is totally confusing. It contains gems like building the SOAP responses by itself using string formatting, and this:

/* URL validation - SORT LATER */
function mtValidHTTP($url)
{
   return TRUE;
}

All the while being totally confusing internally, coming with no example programs, and renaming a ton of functions.

Even when the SDK’s do work, you need to refer to two sets of documentation: to the SDK’s documentation and the official mTurk API documentation. Why? Because many of them come up with their own cutely named methods (get_account_balanceGetAccountBalance).

Python is my favorite language, so after much Google searching I tried to use the boto mTurk connection. It’s a mess, it’s missing a ton of calls (doesn’t seem to have been updated in a long time) and is just overkill for most scripts (all of them).

Download latest mturk.py from Github: https://github.com/ctrlcctrlv/mturk-python

Download mturkcore.zip (2.4KB) (v 1.0)

Download old SOAP version (requires python-suds, not recommended for future projects) (2.8KB) (v 0.2)

Update 01/26: Version 0.2 released. This fixes a critical bug where the mTurk WDSL was NOT cached! Updating is strongly recommended.

Update 03/19: Version 1.0 released. This is the best version yet! mturkcore no longer uses Suds and now uses the mTurk REST API, the only major SDK I know of to do so! This release however breaks backwards compatibility.

Update 02/27/2014: I’m using Github for this instead now. Please refer to the documentation there.

Thanks to all of this, I wrote my own SDK, mturkcore.py. At just 103 lines of code, including documentation, I hope this is a real departure from the past. mturkcore.py is a complete Python Mechanical Turk SDK. It only requires Python 2.7, xmltodict and requests and uses standard Python types for parameters, like str, dict, and list. Some example programs and usage info follows.

Your configuration file, passed as a dict to MechanicalTurk or saved in mturkconfig.json

{
"use_sandbox" : false,
"stdout_log" : false,
"aws_key" : "ACCESSID",
"aws_secret_key" : "PASSWORD",
}

Getting your balance

import mturkcore
m = mturkcore.MechanicalTurk()
m.create_request("GetAccountBalance")
if m.is_valid():
        print m.get_response_element("AvailableBalance")

Assigning a qualification

import mturkcore
m = mturkcore.MechanicalTurk()
workers = ["A1ZZZ","A1QQQ"] # Replace these, of course!
for worker in workers:
    m.create_request("AssignQualification",{"QualificationTypeId":"2MYQUALIFICATION","WorkerId":worker,"IntegerValue":100})

Known bugs

None anymore :)

New blog

Hello everyone,

I decided to move on from the old blog that I contributed to, “Tips For Requesters On Mechanical Turk” after the admin made a few posts that I did not agree with and that some thought I wrote.

I will not remove my posts on that blog, but I will not be contributing there any longer. I hope all of my old followers can find this new blog, which is actually on my domain. :)