May 20

The moment clients realize that revisions are not an all-you-can-eat buffet, suddenly they realize they are not hungry.

-- Lester Beall

 
 

Featured chapter

Read the fascinating history of Wearable Computing, told by its father, Steve Mann

Read Steve's chapter !

 
 

Help us help you!

 
 

Jeffrey M. Rzeszotarski

Add description
Add publication

Publications by Jeffrey M. Rzeszotarski (bibliography)

 what's this?
2011
 
Edit | Del

Rzeszotarski, Jeffrey M. and Kittur, Aniket (2011): Instrumenting the crowd: using implicit behavioral measures to predict task performance. In: Proceedings of the 2011 ACM Symposium on User Interface Software and Technology 2011. pp. 13-22.

Detecting and correcting low quality submissions in crowdsourcing tasks is an important challenge. Prior work has primarily focused on worker outcomes or reputation, using approaches such as agreement across workers or with a gold standard to evaluate quality. We propose an alternative and complementary technique that focuses on the way workers work rather than the products they produce. Our technique captures behavioral traces from online crowd workers and uses them to predict outcome measures such quality, errors, and the likelihood of cheating. We evaluate the effectiveness of the approach across three contexts including classification, generation, and comprehension tasks. The results indicate that we can build predictive models of task performance based on behavioral traces alone, and that these models generalize to related tasks. Finally, we discuss limitations and extensions of the approach.

© All rights reserved Rzeszotarski and Kittur and/or ACM Press

 
Add publication
Show this list on your homepage
 
 

Join the technology elite and advance:

 
1.

Your career

 
2.

Your network

 
 3.

Your skills

 
 
 
 
 
 

Changes to this page (author)

05 Apr 2012: Added

Page Information

Page maintainer: The Editorial Team
URL: http://www.interaction-design.org/references/authors/jeffrey_m__rzeszotarski.html
May 20

The moment clients realize that revisions are not an all-you-can-eat buffet, suddenly they realize they are not hungry.

-- Lester Beall

 
 

Featured chapter

Read the fascinating history of Wearable Computing, told by its father, Steve Mann

Read Steve's chapter !

 
 

Help us help you!