Hadoop is a big deal, but its adoption is still stuck in first gear, despite what the media or vendors are telling you.
The reason? As much as enterprises would like to make better use of their data to improve their businesses, they’re still trying to decipher the complexity of Hadoop and other Big Data tools. More fundamentally, according to a recent Gartner webinar, Hadoop’s biggest roadblock may well be that people can’t figure out what they’re supposed to do with it.
Hadoop proponents that make the platform out to be a magical, dancing unicorn, aren’t helping.
Pass The Instruction Manual, Please
It’s not like enterprises are sitting on their hands when it comes to Big Data. According to Gartner, while 31% of enterprises still have no plans to roll out a Big Data project (the same percentage as in 2012), 30% have invested in Big Data technology, including Hadoop and NoSQL databases, up from 27% in 2012. Fewer people are on the fence, too, with the percentage of those who “don’t know” if they’re going to invest in Big Data dropping from 11% in 2012 to 5% in 2013.
That’s progress. But the news is being hobbled by confusion as to just what Big Data, and particularly Hadoop, is good for.
See more: Why Proprietary Big Data Technologies Have No Hope Of Competing With Hadoop
In the webinar, Gartner analysts Merv Adrian and Nick Heudecker walked through a few things needed in this next wave of Hadoop adoption, suggesting we’re entering an era when “adult supervision” is needed for proper governance and security in Hadoop and NoSQL. The analysts pointed to security, governance and enterprise data warehouse integration as priorities for enterprise integration, making new Big Data technologies fit within existing IT infrastructure.
But when the analysts asked webinar attendees the biggest barriers to Hadoop adoption, the results were surprising.
Nearly 50% of attendees cited Hadoop’s lack of a clear value proposition as its biggest barrier to adoption. After this came its lack of integration with existing infrastructure, then the difficult acquisition of necessary Hadoop skills. Security, which normally tops the enterprise wish list, was cited by a mere 1% of attendees, which “amazed” Gartner.
Security As A Secondary Concern
Security matters when a technology has moved beyond the evaluation phase and is being embraced in earnest. Clearly Hadoop isn’t there yet. This isn’t to say that it won’t be, but rather Hadoop still has a range of questions to answer before it can go mainstream.
See more: Hadoop: What It Is And How It Works
Like, where’s the “on” button?
As I’ve written before, we’re still in a vortex of Big Data hype and hope, and Hadoop plays a leading role in this. Enterprises are still trying to figure out what to do with their data:
Hadoop vendors, in particular, may not be helping with this. Hadoop vendors often fall into the trap of speaking optimistically of what Hadoop will be able to do as if it already can. For example, Cloudera executive and Hadoop founder Doug Cutting insists batch-oriented Hadoop will be able to do transactions and anything else an enterprise could want.
“My belief is the sky is the limit,” Cutting said. “It’s hard to imagine a kind of a workload that you can’t move to this platform.”
Actually, it’s easy to imagine all sorts of workloads that don’t fit Hadoop. In a call with IDC this week, the intelligence firm expressed serious doubts as to the ability to turn Hadoop into an online transaction processing (OLTP) system to facilitate data entry and commercial transactions. It’s not a thneed, in the Suessian sense:
A Thneed’s a Fine-Something-That-All-People-Need!
It’s a shirt. It’s a sock. It’s a glove. It’s a hat.
But it has OTHER uses. Yes, far beyond that.
You can use it for carpets. For pillows! For sheets!
Or curtains! Or covers for bicycle seats!
Frankly, the Hadoop advocates may not be doing themselves—or their customers—any favors by spinning the Hadoop story in this way. As revealed by Gartner’s poll, enterprises are confused by the mysticism surrounding Hadoop. They’re concerned with its basic value proposition, not its ability to be everything to everyone. Indeed, that sort of marketing is almost certainly complicit in the confusion.
No Need To Hype Hadoop
Hadoop is extremely elaborate, but that doesn’t mean it’s not incredibly important technology. It is. For enterprises rolling out Big Data projects, Hadoop should definitely be one of the technologies they consider.
See more: Hadoop’s Biggest Vendors May Contribute Least To Its Development
But Hadoop today is hard, and its proponents need to spend more time focusing on making it easy, not necessarily blowing it up into some mythical technology. As Jonathan Gray, founder and CEO of Continuuity, highlights, “Hadoop is a distributed system, and exposes itself as such, but most developers/operators are not distributed systems experts.” All the more reason, then, for companies like Cloudera and Hortonworks to help guide newbies, as called out by Justin Kestelyn, developer community advocate at Cloudera: “Hadoop is complex and the knowledge gaps are huge. Hence Cloudera!” (Or another, preferred vendor.)
This is the real, short-term opportunity for Hadoop vendors: To make it easier to understand. That’s far more important than making it magical. It also happens to be a huge opportunity.
