And they’re mostly failing.
Humans are key to understanding Big Data.
Turns out, what they do is really hard.
Wanelo, an e-commerce startup, built its own tools to analyze a flood of data.
R remains popular with the PhDs of data science, but as data moves mainstream, Python is taking over.
The bigger the data set, the bigger the chance for completely misreading the data, particularly with regards to so-called Black Swan events.
The bigger our data sets, the harder it becomes to sift through them to find objective truth, and the higher the costs of getting it wrong. By lowering the bar to data science, open-source tools like Hadoop have actually increased our ability to misread our data, with serious consequences.