Home Amazon Streamlines Mechanical Turk With Automatic Categorization App

Amazon Streamlines Mechanical Turk With Automatic Categorization App

The idea behind Amazon’s Mechanical Turk is pretty simple – break programming work down into bite-sized chunks, and put it in front of a large workforce that can do the work quickly and cheaply. Part of the challenge of that is making it easy for requesters to create the bites that workers are chewing on. The new categorization app from Amazon removes some of the hurdles of creating HITs (Human Intelligence Tasks) that ask workers to pick the best category for items. The result could make the crowdsource coding marketplace even more usable and popular.

Creating a request for Mechanical Turk isn’t overly difficult, but it takes a bit of time editing HTML and answering a lot of questions for which first-time users don’t have a lot of context – for example, deciding the qualifications for workers, or whether they need to be “masters” to take on a task.

Amazon is streamlining all that with the Categorization App by making assumptions about what its users would want, providing suggested pricing and helping to create the form that workers will see.

Right now, Amazon is providing an app only for categorization questions. Mechanical Turk offers templates for many more, including image filtering, image tagging, data extraction, data collection and several others. Expect that Amazon will add apps for most of these types of questions in the near future.

Boosting MT Revenues

Obviously, Amazon is trying to streamline the process for creating MT tasks in order to boost its revenue. More tasks means more money. But there’s a bit more to it than that.

With the Categorization App, Amazon’s assumptions all lead to more money per HIT. Users no longer have to decide what kind of worker to farm tasks out to; rather, Amazon makes that decision, automatically choosing the most expensive (master) workers. To increase accuracy, each HIT will be shown to two users by default, doubling the revenue.

Amazon will also suggest a pay level, to ensure that HITs are “priced attractively” to workers. Requesters can change this, but if a significant number of requesters accept the default pricing, that will probably drive HIT prices up over time.

Better Requests

Another reason that Amazon would want to create an app for MT requests is that a lot of requests are not very good. Amazon is in a position to evaluate the existing requests, and has found them wanting.

One problem? Too many categories. Amazon says that the maximum number of categories that workers can keep in mind is seven to 10. Yet they see requests with more than 150 categories!

The app limits requesters to 10 items. It requires that requesters provide instructions. It’s not foolproof, but it provides better odds that requesters will generate reasonable HITs. If requesters want more categories, they need to start from scratch with a custom template.

If you’re using Mechanical Turk now, what do you think of the Categorization App? Would you prefer to see Amazon simplify the service further, or is it already well-suited for the kind of crowdsourcing that you’re doing?

Illustration titled “Blogging Au Plein Air, Jean-Baptiste-Camille Corot” by Flickr user Mike Licht.

About ReadWrite’s Editorial Process

The ReadWrite Editorial policy involves closely monitoring the tech industry for major developments, new product launches, AI breakthroughs, video game releases and other newsworthy events. Editors assign relevant stories to staff writers or freelance contributors with expertise in each particular topic area. Before publication, articles go through a rigorous round of editing for accuracy, clarity, and to ensure adherence to ReadWrite's style guidelines.

Get the biggest tech headlines of the day delivered to your inbox

    By signing up, you agree to our Terms and Privacy Policy. Unsubscribe anytime.

    Tech News

    Explore the latest in tech with our Tech News. We cut through the noise for concise, relevant updates, keeping you informed about the rapidly evolving tech landscape with curated content that separates signal from noise.

    In-Depth Tech Stories

    Explore tech impact in In-Depth Stories. Narrative data journalism offers comprehensive analyses, revealing stories behind data. Understand industry trends for a deeper perspective on tech's intricate relationships with society.

    Expert Reviews

    Empower decisions with Expert Reviews, merging industry expertise and insightful analysis. Delve into tech intricacies, get the best deals, and stay ahead with our trustworthy guide to navigating the ever-changing tech market.