Screening Participants on Mechanical Turk: Techniques and Justifications

Julie S. Downs, Carnegie Mellon University, USA
Mandy B. Holbrook, Carnegie Mellon University, USA
Emily Peel, Carnegie Mellon University, USA
Concerns about the quality of Mechanical Turk participants induce researchers to screen participants. We evaluate screening strategies according to their discriminant ability to identify observations contributing only noise. Our results suggest that omitting participants based on these indicators would likely bias the sample rather than improve data quality.
[ to cite ]:
Julie S. Downs, Mandy B. Holbrook, and Emily Peel (2012) ,"Screening Participants on Mechanical Turk: Techniques and Justifications", in NA - Advances in Consumer Research Volume 40, eds. Zeynep G├╝rhan-Canli, Cele Otnes, and Rui (Juliet) Zhu, Duluth, MN : Association for Consumer Research, Pages: 112-116.