Tuesday, September 22, 2009

Need almost free expert advice? Run a public contest - "$1 Million Research Bargain for Netflix, and Maybe a Model for Others" - NYTimes.com

A $1 Million Research Bargain for Netflix, and Maybe a Model for Others - NYTimes.com

Need almost free expert advice? Run a public contest on the internet.

By STEVE LOHR
Published: September 21, 2009

Excerpts:

Netflix, the movie rental company, announced on Monday that a seven-man team was the winner of its closely watched three-year contest to improve its Web site’s movie recommendation system. That was expected, but the surprise was in the nail-biter finish.

Yet the scientists and engineers on the second-place team, and the employers who gave many of them the time and freedom to compete in the contest, were hardly despairing.

Arnab Gupta, chief executive of Opera Solutions, a data analytics company based in New York, took a small group of his leading researchers off other work for two years. “We’ve already had a $10 million payoff internally from what we’ve learned,” Mr. Gupta said.

Working on the contest helped the researchers come up with improved statistical analysis and predictive modeling techniques that his firm has used with clients in fields like marketing, retailing and finance, he said. “So for us, the $1 million prize was secondary, almost trivial.”

Indeed, since it began in October 2006, the Netflix contest was significant less for the prize money than as a test case for new ideas about how to efficiently foster innovation in the Internet era — notably, offering prizes as an incentive and encouraging online collaboration to tap minds worldwide.

The lessons of the Netflix contest could extend well beyond improving movie picks. The researchers from around the world were grappling with a huge data set — 100 million movie ratings — and the challenges of large-scale modeling, which can be applied across the fields of science, commerce and politics.

Thousands of teams from more than 100 nations competed in the Netflix prize contest. And it was a good deal for Netflix. “You look at the cumulative hours and you’re getting Ph.D.’s for a dollar an hour,” Mr. Hastings said in an interview.

The prize winner was a team of statisticians, machine-learning experts and computer engineers from the United States, Austria, Canada and Israel, calling itself BellKor’s Pragmatic Chaos. The group was actually a merger of teams that came together late in the contest.


Read Full Article: http://www.nytimes.com/2009/09/22/technology/internet/22netflix.html?_r=1&ref=business

***********************************************************************
http://dreamlearndobecome.blogspot.com This posting was made my Jim Jacobs, President & CEO of Jacobs Executive Advisors. Jim also serves as Leader of Jacobs Advisors' Insurance Practice.

No comments: