Human computation and playing games

I very much enjoyed watching this fun, exciting, and light UW CS talk, "Human Computation", by Luis von Ahn from CMU.

The talk mostly focuses on how you can get people to do useful work for free by making it fun, by turning it into a game. His motivation comes, at least in part, from noticing the millions of hours people waste playing solitaire on the computer and wondering if that effort could produce useful output instead. His games are carefully constructed to be fun, produce useful data, and avoid cheating.

Luis spent most of his time talking about the ESP Game, a popular Web-based game that, as a side effect, gets people to label images with keywords. In aggregate, this kind of data is useful for image search. People enjoy playing the game for many hours. The researchers have collected over 10M image labels as a side effect of the game play.

Luis then discussed a newer game, Peekaboom, that takes the ESP Game one step further, from just labeling images to identifying the region of the image that corresponds to the label (e.g. the "man" is here). Peekaboom has already collected millions of data points. This data set could be quite useful for computer vision research.

Luis mentioned a new game they are about to release called Verbosity (PDF). The game resembles many types popular word guessing games and, as a side effect, produces a stream of common sense knowledge (e.g. "milk is a liquid" and "milk is often found near cereal"). This data set that would be very useful for AI researchers.

By the way, Luis was one of the inventors of captchas, those twisted images of text that you are sometimes asked to read and type in on websites. Captchas are designed to distinguish humans from computers by asking you to do a task computers cannot do very well. The ESP Game and other projects are similar in that they get humans to solve problems that computers cannot easily solve.

I really like some of the lessons from Luis' work. If you have a problem that is very hard for computers to solve but easy for people to solve, see if you can make it a game. If it is fun, a lot of people will do it for free.

However, as Luis said, you do have to expect people to cheat; features to increase data quality and hinder cheating and spam were built-in as part of their early game design.

This is a great talk, well worth watching. A lot of fun and many good ideas here.