Beyond the Turk: an Empirical Comparison of Alternative Platforms For Crowdsourcing Online Research

Eyal Peer, Bar-Ilan University, Israel
Sonam Samat, Carnegie Mellon University, USA
Laura Brandimarte, Carnegie Mellon University, USA
Alessandro Acquisti, Carnegie Mellon University, USA
We compared several available alternative crowdsourcing services to Amazon Mechanical Turk (MTurk). Whereas some sites proved unusable, we found two (CrowdFlower and MicroWorkers) that showed adequate data quality, although sometimes inferior to MTurk’s. Between the sites, we also found psychometric and demographic differences that could affect research outcomes.
[ to cite ]:
Eyal Peer, Sonam Samat, Laura Brandimarte, and Alessandro Acquisti (2015) ,"Beyond the Turk: an Empirical Comparison of Alternative Platforms For Crowdsourcing Online Research", in NA - Advances in Consumer Research Volume 43, eds. Kristin Diehl and Carolyn Yoon, Duluth, MN : Association for Consumer Research, Pages: 18-22.