Political pundits, mostly Republican, went into a frenzy when Nate Silver, a New York Times pollster and stats blogger, predicted that Barack Obama would win reelection.
But Silver was right and the pundits were wrong – and the impact of this goes way beyond politics.
Silver won because, um, science. As ReadWrite’s own Dan Rowinski noted, Silver’s methodology is all based on data. He “takes deep data sets and applies logical analytical methods” to them. It’s all just numbers.
Silver runs a blog called FiveThirtyEight, which is licensed by the Times. In 2008 he called the presidential election with incredible accuracy, getting 49 out of 50 states right. But this year he rolled a perfect score, 50 out of 50, even nailing the margins in many cases. His uncanny accuracy on this year’s election represents what Rowinski calls a victory of “logic over punditry.”
In fact it’s bigger than that. Bear in mind that before turning his attention to politics in 2007 and 2008, Silver was using computer models to make predictions about baseball. What does it mean when some punk kid baseball nerd can just wade into politics and start kicking butt on all these long-time “experts” who have spent their entire lives covering politics?
It means something big is happening.
Man Versus Machine
This is about the triumph of machines and software over gut instinct.
The age of voodoo is over. The era of talking about something as a “dark art” is done. In a world with big computers and big data, there are no dark arts.
And thank God for that. One by one, computers and the people who know how to use them are knocking off these crazy notions about gut instinct and intuition that humans like to cling to. For far too long we’ve applied this kind of fuzzy thinking to everything, from silly stuff like sports to important stuff like medicine.
Someday, and I hope it’s soon, we will enter the age of intelligent machines, when true artificial intellgence becomes a reality, and when we look back on the late 20th and early 21st century it will seem medieval in its simplicity and reliance on superstition.
What most amazes me is the backlash and freak-out that occurs every time some “dark art” gets knocked over in a particular domain. Watch Moneyball (or read the book) and you’ll see the old guard (in that case, baseball scouts) grow furious as they realize that computers can do their job better than they can. (Of course it’s not computers; it’s people who know how to use computers.)
We saw the same thing when IBM’s Deep Blue defeated Garry Kasparov in 1997. We saw it when Watson beat humans at Jeopardy.
It’s happening in advertising, which used to be a dark art but is increasingly a computer-driven numbers game. It’s also happening in my business, the news media, prompting the same kind of furor as happened with the baseball scouts in Moneyball.
Who wants to believe that machines can tell us which stories to write, or which stories people want to read? Who wants to believe that machines can actually write stories? But they do. Forbes, my former home, started running computer-generated stories earlier this year.
Backlash
Each time this happens, there’s always lots of sputtering and outrage. When you strip away the rhetoric, at the core is always the same fear – that machines will take away jobs from humans. Baseball scouts want to keep working. So do journalists and chess masters.
So do pundits, but after this election it’s getting harder to see what role they should play.
Why listen to Joe Scarborough and his crew on MSNBC’s Morning Joe show bloviate about who’s going to win and why, when Nate Silver and his computers can just give you the correct answer?
Scarborough bet Silver $2,000 on the outcome of the election after accusing Silver of being an “ideologue” who was predicting an Obama win simply because that’s what Silver wanted to happen.
Scarborough has lost a lot more than the two grand. He’s lost his reason for being, and he has taken a lot of others down with him.
Scarborough, not Siliver, turns out to be the wishful-thinking ideologue. Nate Silver and his computers may not put Scarborough and his ilk out of business – there’s loads of airtime to fill, and windbags are still needed for that.
But Silver has exposed those guys for what they are, which is propagandists and entertainers.
And that’s fine. We still need entertainers. Computers haven’t learned to do that yet.
For now, anyway.
Lead image courtesy of Shutterstock. Nate Silver image by Randy Stewart.
