Tesla And The Fallacy Of Data-Driven Decisions

We like to pride ourselves on being increasingly data-driven. In fact, we've created a giant new industry frenetically panning for Big Data gold. A healthy $4.5 billion market in 2010, according to IDC, Big Data is set to explode to $23.8 billion in 2016, fueled by our need to be more data-driven in everything from how we do business to how we eat.

I suspect, however, that we're fooling ourselves, as the recent Tesla debacle suggests. As much as we'd like to smugly pat ourselves on the back for being data-driven, the truth is that data is always messy, and never really tells any particular story.

(See also Would You Buy A Tesla Model S?)

Bigger Data ≠ Bigger Answers

New York Times columnist David Brooks nails this in an op-ed piece, wherein he argues that Big Data, while very useful for guiding our intuitions, gets some things very wrong. Like the value of social connections. Or the context for answering a question. In fact, he speculates, Big Data might actually obscure Big Answers by complicating decisions and making it even harder to determine which statistically signifiant correlations between data are informative and not simply spurious.

Such thinking won't be surprising to anyone that has read Nassim Taleb's book The Black Swan, which posits that the more data we analyze, the more likely our conclusions will be wrong. Taleb writes:

In business and economic decision-making, data causes severe side effects - data is now plentiful thanks to connectivity; and the share of spuriousness in the data increases as one gets more immersed into it. A not well-discussed property of data: it is toxic in large quantities - even in moderate quantities.

In other words, the more data you collect, the harder it can become to interpret that data. And even if you can interpret your data correctly, are you actually going to listen to that interpretation?

Which brings us to Tesla. 

Tesla and "Truth"

In case you've been hiding under a rock, a New York Times reporter, John Broder, wrote an unflattering review of Tesla's new Model S. Tesla founder and CEO Elon Musk got the knives out and slammed the reporter using a pile of data (from the reporter's test drive, which is a little bit creepy). Broder responded with his own view of the data, and finally Margaret Sullivan, public editor of the Times, waded in. Her conclusion?

People will go on contesting these points – and insisting that they know what they prove — and that’s understandable. In the matter of the Tesla Model S and its now infamous test drive, there is still plenty to argue about and few conclusions that are unassailable.

But wait! What about all that data Musk collected? Doesn't it prove his point? Or what about Broder's own data? Doesn't it prove his? In both cases the answer is "Yes," leaving would-be Tesla buyers like ReadWrite's Dan Lyons stymied as to what they should do. Which is why being "data-driven" is the start of a solution, not the end. 

The Human Side of Big Data

As David Brooks notes, reviewing Malcolm Gladwell's book, Blink, "We have the capacity to sift huge amounts of information, blend data, isolate telling details and come to astonishingly rapid conclusions, even in the first two seconds of seeing something." This is not to suggest that we shouldn't collect data, but that we perhaps need to be smarter about how we analyze it, and how much we trust it.

As I've argued (see Big Data And The Landfills Of The Digital Enterprise), I don't think this is a matter of hiring expensive data scientists to interpret our data. Rather, I imagine it's a matter of guiding our decisions - even those split-second "hunches" that Gladwell talks about in Blink - through data, without becoming consumed with data. Data kicks off the right questions; data doesn't resolve disputes.

Just ask Musk and Broder: both absolutely convinced they're right, and both with ample data on their respective sides to prove it.

Image courtesy of Shutterstock.