Psychology researchers have long relied on undergraduate college students as study subjects – so much so that the average American college student is 4,000 times more likely than an average person to be a subject of a study. This situation doesn’t make for the best science, which calls for choosing subjects randomly to avoid bias. Now researchers think they may have a Web-based solution to address the bias problem. Call it crowd-sourced psychology.
Researchers are increasingly turning to sites such as Amazon.com’s Mechanical Turk to find low-cost study subjects, as outlined in an article in The Economist. The site’s 500,000 users help researchers overcome the bias that they get from studies that exclusively use college students, who are described in research jargon as WEIRD: western, educated, industrialized, rich and democratic.
Traditionally, many psychologists have worked under the premise that bodies are different but brains are similar, according to Monika M. Wahi, an independent research consultant and epidemiologist based in Boston. But repeating earlier tests with a broader research base has challenged that notion, showing that some of the earlier studies may have indeed been skewed by using only college students.
College students have been the go-to subjects in such studies because they were in plentiful supply where researchers did their work and they came cheap. In many cases, they could also be incentivized by offering course credit. Sites like Mechanical Turk also keep research costs low, and they may speed up the process in which research can be completed.
Wahi cautioned that it’s too soon to tell whether using Web-based research subjects will bring unanticipated problems or whether crowdsourcing can be as effective in other research fields.
“If you want a lot of people quick and don’t care about bias, it’s a pro. If you want accurate research, it still could be a pro, if it’s used wisely,” she said. “With Mechanical Turk, I see it being difficult to produce results we [epidemiologists] could use, because we can’t know what the composition of the background ‘community’ is – but that is not a con if you are of the ‘brains are all the same’ camp.”
Some research fields require more care than others in selecting their study subjects. Epidemiologists, for example, work with a higher degree of ethical oversight, as they often ask subjects for blood samples and other medical testing; psychologists, by and large, get to rely on surveys. When epidemiologists began to have difficulty recruiting study subjects a decade ago, they turned to a computer-based solution, as well.
“First researchers got mad, then patients got mad [that] they were blocked from research. Consequently, online registries were developed where patients could sign up to be contacted for studies,” Wahi said. “So, like psychologists, our way of recruiting is changing in the [Mechanical Turk] direction as well, and the jury is out on how that is going to affect our estimates.”
Image courtesy of Shutterstock.
