Predicts America's Political Winners

Data scientists at Yahoo are using prediction markets—along with polls, sentiment analysis on Twitter, and trends in search queries—to create the mother of all political prediction engines. The project involves Web-based prediction markets like Intrade, in which large numbers of people bet on the outcomes of elections.

The researchers behind this effort, David Rothschild, an economist at Yahoo Research, and Dave Pennock, a computer scientist at Yahoo Research, call their effort the Signal. They plan to produce data visualizations that best convey probability to a lay audience, and to publish work on machine learning and fundamental economic models based on the effort.

They'll get the public involved in all this political and mathematical wonkiness with fun and games. Drawing on Yahoo's success with fantasy sports leagues—for which the company is the biggest community on the planet—Rothschild and Pennock have created "Fantasy Politics," in which users can bet on the outcomes of pretty much anything.

"We're going to let people [bet on simple predictions] like 'the Democrats will win California,' " says Pennock. "But if they want to geek out, they could bet on the odds that 'the Democrats will win both Ohio and Florida,' or 'the Republicans will win Florida but lose the election.' "

Such bets will take Yahoo's political prediction markets, which will roll out in spring, to a level of complexity and predictive power not seen elsewhere, says Pennock.

The Signal will use these markets and other real-time data streams. The prediction markets run by the Signal are polled constantly, as will be the results of analysis of Yahoo search queries and Twitter.

Sentiment analysis, or the effort to automatically determine how people feel about something based on how they communicate about it, is "at an infant stage," says Rothschild, but it can provide insight that no poll can match.