When David Kirkby and Daniel Margala entered a contest to find out who could most accurately map dark matter in the universe, the first algorithm they submitted failed to crack the top 10 among a field of more than 100 engineers and data scientists taking part in the nearly three-month competition.
“It’s quite intimidating to see a long list of competitors,” said Kirkby, a cosmology professor at the University of California at Irvine. “It’s a good motivator to see how you’re stacking up.”
Kirkby and Margala, a grad student at UC Irvine, kept tweaking and re-submitting their statistical model — 15 more times in fact — before emerging as the winner of the Mapping Dark Matter competition sponsored by NASA and the Royal Astronomical Society and hosted by Kaggle.com, a crowd-sourcing platform for data modeling and prediction competitions that is in the process of raising venture capital.
The competition worked like this: Contestants were provided with a data set to analyze — simulated images of 100,000 galaxies — and asked to create statistical models that could be used to measure the tiny distortions in galaxy images as their light is bent by dark matter. Dark matter is the still-mysterious substance scientists believe makes up about a quarter of the universe but which is largely invisible.
The results were tabulated by comparing the predictive models to actual data that had been withheld from the participants. In essence, contestants were trying to fill in the missing pieces of the puzzle as accurately as possible. Contestants submitted a total of 760 entries, with most of them submitting numerous times as they tried to make their models more accurate.
Kirkby and Margala’s team named themselves “DeepZot” — a merger of Deep Thought, the fictional computer from Douglas Adams’ Hitchhiker’s Guide to the Galaxy, and “Zot!” the battle cry of the UC Irvine mascot Peter the Anteater.
Their key insight was that they could come up with increasingly accurate statistical models by developing an artificial neural network — basically, a computer brain — and teaching it to discern patterns in the galaxy images. Without that neural network, the best the team could have done was eighth place, said Margala.
The final days of the contest period got the competitive juices flowing, thanks to the way it’s structured. Participants were able to watch a real-time leaderboard that would change with each new entry, so that the team in the top spot constantly checked to make sure they weren’t supplanted by a better predictive model.
Another participant, Martin O’Leary, a glaciologist at Cambridge University in the U.K., used his experience analyzing satellite imagery of glaciers to develop an algorithm that put him atop the leaderboard early in the competition — and merited a post on the White House’s Office of Science and Technology Policy blog that said O’Leary had “crafted an algorithm that outperformed the state-of-the-art algorithms most commonly used in astronomy for mapping dark matter.”
“One morning I checked and saw someone had beaten me,” he said. “Your competitive instincts definitely kick in.”
As the hours until the contest deadline wound down, Kirkby, with DeepZot in first place, was on a backpacking trip in the Sierra Mountains. The first thing he checked when he got back to his car was whether their submission still held down the top spot. It had.
The prize? An all-expenses-paid trip to a conference at the California Institute of Technology (Caltech) in nearby Pasadena, where the winners will present their solutions to NASA and other agencies, who will work to implement them.
Though some competitions hosted by Kaggle have sizable cash prizes, for many participants the prize is secondary to the chance to solve interesting problems that are structured in a clear, well-defined way.
“It’s something that people can easily compete on,” said O’Leary of the Kaggle contest. “There’s lots of work that goes into taking a big problem and turning it into something manageable.”
While Big Data has become a buzzword in the tech world, Kirkby says the most significant thing about such competitions isn’t raw computing power. The real significance, he says, is that they attract people with a wide variety of interests and skill sets — cosmologists who teach artificial brains to recognize patterns, glaciologists who look at galaxies and see similarities to huge chunks of ice — to chip away at problems outside their immediate expertise.
“Analysis of large data sets is a science that transcends any one problem,” Kirkby said. “More brainpower is what’s being leveraged by competitions like this. It’s not just about access to the best computers.”
UPDATE: For a look at another contest involving NASA, check out this video from the Wall Street Journal Digital Network.