A recent report found that up to 40% of new work requests on Amazon’s Mechanical Turk are spam. It’s not the first time that there have been reports that the marketplace for micro-tasks has been used to employ folks for spam or fraud, or to prank friends or game The New York Times Most Emailed list.
But as ReadWriteWeb’s Marshall Kirkpatrick has often argued, there are some very interesting things you can do with Mechanical Turk. And Lindsey Harper, founder of Swayable, has found that the service has benefited her greatly as she’s developed and tested several components of her web app.
Using Mechanical Turk to Validate an Idea
We noted back in September when Harper had utilized Mechanical Turk in order to help validate her startup idea, using the service to poll 200 people on their reactions to her concept. Then, she had surveyed people to find whether or not they’d use the service, example of how they might use it. In addition to asking for general feedback on the idea, she also captured gender and age demographics, so she could have a better idea of who her market might be.
Using Mechanical Turk to Test Your App
Since then, Harper has continued to use Mechanical Turk, particularly for testing new features and product development along the way. She notes that the analytics and feedback that come with MTurk aren’t that useful, so she uses the service in conjunction with SurveyMonkey – the “hits” on Mechanical Turk direct workers to the survey.
She’s used MTurk workers to test new features, to find bugs and dead links, asl well as to give feedback on the site’s look and feel. Harper says that the service has provided her with a “mass of people for testing” that she could not have otherwise afforded to hire.
Lessons Learned
She does note that she’s learned several lessons along the way in helping her maintain the quality of the work she’s getting done. She’s restricted the location of MTurk workers to those within the U.S, for example, and she’s adjusted her pricing until she’s found the “sweet spot” – neither the highest nor the lowest. There, she says, she’s found quality workers, although she says she does keep an eye out for people that are simply spamming her responses.
As a single-person and non-technical founder, Harper is outsourcing many of the pieces in building her app, of which the work sent to Mechanical Turk is just one component. Harper says she feels as though outsourcing has a bad rap – “Why not hire?” she’s often asked. And certainly the recent news about Mechanical Turk and spam will do little to improve that reputation. But for her part, Harper says she plans to continue using the service for testing new features, even after she launches next year.
