Home How Open Source Can Fix 2015’s Data Entropy

How Open Source Can Fix 2015’s Data Entropy

ReadWritePredict is a look ahead at the technology trends and companies that will shape the coming year.

This will not be a good year for those that believe in human progress. At least, not if enterprise software is any indication.

Or, more specifically, enterprise data.

As much as we keep coming up with technologies to harness data and put it to use, things continue to fall apart, as Chinua Achebe might say. From data lakes to Internet of Things (IoT) to private clouds, expect to see a lot more entropy in 2015. But also expect to see open source increasingly help us tame it all.

Data: Getting Messier

Nothing is messier than Big Data and, within that category, data lakes get first prize in messiness. 

Vendors have been trying to sell us a rosy view of “data lakes” for some time. As PwC puts it, “Hadoop allows [an enterprise’s] disparate records to be stored in their native formats for later parsing, rather than forcing all-or-nothing integration up front as in a data warehousing scenario.” In other words, just dump all your data in a data lake and use it at your leisure.

See also: Applications Drive The Biggest Money In Big Data

Hortonworks went public on the promise of data lakes. It’s just unfortunate that they don’t work.

By Gartner’s estimation, for example, “Through 2018, 90% of deployed data lakes will be useless as they are overwhelmed with information assets captured for uncertain use cases.” This may be optimistic. 

After all, as Gartner’s Nick Heudecker highlights in a separate post, “Since data lakes lack semantic consistency and governed metadata, [data lake] positioning assumes those audiences are highly skilled at data manipulation and analysis.” But one overriding theme of 2014 was that enterprises lack these skills and are desperately, mostly futilely, trying to hire them, as McKinsey & Co. analysis demonstrates.

But it’s not just data lakes. 

IoT remains a morass of conflicting “standards,” something that won’t get any better in 2015.  In fact, Gartner’s Al Velosa estimates that “Through 2020, there will be no dominant IoT ecosystem platform,” leaving “IT leaders to compose solutions from multiple providers.” 

I’ve written before about the importance of standards in IoT, and how open source could help to foster a dominant one. But if 2014 was the year that enterprises realized they needed to get serious about IoT then 2015 is the year that they grow frustrated that they can’t because of a lack of standardization.

Over in the cloud, don’t expect much better. 

Cloud: Even Cloudier

The good news is that Amazon Web Services is getting serious competition from Google and especially Microsoft, which will make life even better for enterprises looking to push workloads to the public cloud.

But for those that want to keep a toe dipped in the private cloud, things are still murky. 

For example, OpenStack has yet to really take off. While some criticisms can be dismissed as overly negative (“a software particle-board designed by committee, consisting of floor sweepings and a lot of toxic glue”), it’s telling that its major proponents like Red Hat still aren’t making much money from it. 

See also: Big Companies Are Still Struggling To Buy A Big Data Clue

As Microsoft’s Mark Russinovich laments, OpenStack is “not mature enough, scalable enough, or stable enough,” such that it’s “incredibly hard to set up and it is incredibly hard to maintain.” This has translated into persistently low numbers of significant deployments

Other than that, it’s great.

To Hell In A Handbasket

Which is not to say that everything is terrible and we’re all doomed. In fact, I view the increasing entropy in so many areas of computing as a sign that we’re in massive experimentation mode, trying to figure out the future. That’s always going to be messy.

The thing that gives me hope, more than anything else, is that we’re increasingly trying to resolve these problems with open source. 

See also: Big Data Will Get Even Bigger In 2015

No, not always. AWS is built on open source but it’s hardly open. All major mobile applications come completely locked down, as the free software movement’s prophet Richard Stallman opines about Uber.

But these are exceptions to the rule that virtually essential infrastructure built in the last 10 years has been open source. Openness, over time, tends toward collaboration, as Linux, Hadoop and many other projects illustrate. 

So while 2015 promises to be even messier than 2014, open source offers the industry a long-term silver lining. 

Image courtesy of Shutterstock.

About ReadWrite’s Editorial Process

The ReadWrite Editorial policy involves closely monitoring the tech industry for major developments, new product launches, AI breakthroughs, video game releases and other newsworthy events. Editors assign relevant stories to staff writers or freelance contributors with expertise in each particular topic area. Before publication, articles go through a rigorous round of editing for accuracy, clarity, and to ensure adherence to ReadWrite's style guidelines.

Get the biggest tech headlines of the day delivered to your inbox

    By signing up, you agree to our Terms and Privacy Policy. Unsubscribe anytime.

    Tech News

    Explore the latest in tech with our Tech News. We cut through the noise for concise, relevant updates, keeping you informed about the rapidly evolving tech landscape with curated content that separates signal from noise.

    In-Depth Tech Stories

    Explore tech impact in In-Depth Stories. Narrative data journalism offers comprehensive analyses, revealing stories behind data. Understand industry trends for a deeper perspective on tech's intricate relationships with society.

    Expert Reviews

    Empower decisions with Expert Reviews, merging industry expertise and insightful analysis. Delve into tech intricacies, get the best deals, and stay ahead with our trustworthy guide to navigating the ever-changing tech market.