For those of you still scratching your head over the ins-and-outs of Hive, Spark and Pig, don’t bother. There’s a race on to make complex Big Data technology like Hadoop easy for the lay user, and it has the added benefit of making you rich.
Yes, you.
After all, the closer you are to the end user of Big Data technology, the bigger the paycheck. Former Cowen & Co. analyst Peter Goldmacher nailed this years ago in a research brief, arguing that the “biggest winners in the Big Data world aren’t the Big Data technology vendors, but rather the companies that will leverage Big Data technology to create entirely new businesses or disrupt legacy businesses.”
As prescient as Goldmacher was in 2012, his dictum is more obviously true with each passing day. We rightly laud the builders of Big Data infrastructure, but the companies that will profit most richly from it are those that are closest to technical marketing and sales professionals who don’t know MapReduce from a pivot table.
Selling Solutions, Not Technology
We’re already seeing this with companies like John Deere building incredibly powerful, data-driven applications using Hadoop and NoSQL database technology. While Silicon Valley may like to think of itself as the center of the universe, it’s that wider universe that is putting Big Data to best use.
Not that this should surprise us. As Goldmacher writes, this has always been true of technology:
Past as prologue, if we look at the history of ERP, there were over 200 companies created to capitalize on the automation of standard business processes. This means that investors in 1990 had less than a 0.5% chance of picking either SAP or ORCL as the ultimate winners in the space. However, if an investor had purchased stock in the 30 components of the Dow in 1990 that were all deploying ERP, that investor would have benefited from a 35% decline in General and Administrative costs as a percentage of revenues, a five-fold increase in revenues as automation enabled massive scale, and an almost eight-fold increase in market cap.
Of course, Big Data infrastructure vendors like Cloudera will clean up. Cloudera is already worth several billion dollars, and a few other Big Data vendors like DataStax and MongoDB (my employer) are also worth in excess of $1 billion.
But there are a few reasons that Big Data infrastructure vendors won’t ultimately capture the bulk of the value from their software:
- Most Big Data technology is open source, which makes it easy to adopt and harder to monetize;
- The immediate user of technologies like Hadoop is the developer. Developers are essential to driving adoption but tend not to spend a lot of money;
- Companies that are closer to consumers or others with open wallets are most likely to be able to monetize Big Data.
Related to #1 above, Cloudera co-founder Mike Olson insists, “You can no longer win with a closed-source platform, and you can’t build a successful stand-alone company purely on open source.” This leads vendors to blend proprietary and open-source licensing to optimize monetization of adoption, but companies higher up the software stack don’t have to bother with such licensing gymnastics.
So who are these companies?
And The Winners Are…
Most obviously, they are application vendors that simply charge for the value they provide to the end user while burying all the complexity of the infrastructure. Workday co-founder Aneel Bhusri captured this thought years ago:
.@mikeolson makes a great point: #hadoop value will be delivered through cloud apps vendors. ISV opportunity huge for #hadoop and @cloudera
— aneel bhusri (@aneelb) June 13, 2012
McKinsey & Co. details the different industries and the impact Big Data should have on them:
Such companies include the John Deeres of the world, as I mentioned, but closer to home in technology, who wins?
The answer, as ever, is the company that does most to obscure the complexities of difficult technology and make it actionable by mere mortals.
Microsoft, for example, fits this mold. Look what it’s doing with machine learning on Azure. Azure ML promises to remove nearly all “of the startup costs associated with authoring, developing and scaling machine learning solutions,” adding “Visual workflows and startup templates [to] make common machine learning tasks simple and easy.”
While it’s easy to criticize Microsoft (I’ve certainly had lots of practice), it’s also true that Microsoft has done more than any other company to lower the bar on otherwise complex computing. Azure ML follows in the footsteps of Windows, Visual Studio and a host of other technologies that made it possible for mainstream sysadmins and developers to be productive.
Geeks Be Gone!
However, we may need to go a step further. After all, while it’s nice to serve the less technical developer or sysadmin, what is really needed is to make Big Data easy enough for you or me. Wikibon analyst Dave Vellante gets this:
Challenge to the tech industry: Put #bigdataanalytics in hands of "the people" else we'll repeat failures of traditional BI #ibmstginsights
— Dave Vellante (@dvellante) September 9, 2014
Business Intelligence created a special class of analyst but never truly went mainstream. We need Big Data to go mainstream.
One company that seems well-positioned to do this is Adobe. While historically focused on the creative professional, with the purchase of Omniture years ago Adobe jumped firmly into the Big Data world, but with a focus on assisting marketing professionals make informed decisions about how to reach prospective customers.
Harnessing Big Data is less about out-of-control data volumes and more about a proliferating variety of data sources and types. For a company like Adobe, this involves ingesting and analyzing data from social media streams, cash register receipts and more as it seeks to understand consumer behavior and enable a marketing professional to make a decision within milliseconds as to the right advertisement, graphic or other content to display.
Time To Get Out Of The Weeds
Microsoft and Adobe are but two examples of likely Big Data winners. There are, of course, others, hopefully including your own company.
But to get there, we need to stop diving into the weeds of Big Data technologies and instead focus on the business value they can provide. Such value will be expressed through applications that you or I can use, and not necessarily Mary the rocket-scientist-developer-down-the-hall.
As Olson said in an interview with Bosch’s Dirk Slama recently, he talks to a lot of people who obsess over “Big Data as Big Data,” acknowledging that “Those are bad people for us to work with, because they are not fundamentally driven by a business problem.” The real winners in Big Data will be those completely focused on solving real business problems.
Image courtesy of Shutterstock.
