It’s no longer in doubt: Facebook is the world’s largest open source company. Facebook could already have been considered to be in the open source lead through the release of its datacenter designs, databases and more. This week’s open-sourcing of a network switch (Wedge) and its operating system (FBOSS) have cemented Facebook’s place atop the podium of open source contributors.
But have they also sounded the death knell for open source companies?
After all, with so much incredible open source code emerging from Facebook, Google (which released PDFium this week), LinkedIn and others, is there still room for companies to sell open-source software?
Inside The New Software Factory
It has always been true that most software is developed for use, not sale, as author and open source advocate Eric Raymond famously pointed out in his 2001 book The Cathedral And The Bazaar. Yet most of our attention has been on software vendors as we place bets on Oracle or IBM, SAP or Microsoft et al.
These are billion-dollar software companies, but the overall value of software being developed by open-source communities and behind the enterprise firewall is in the trillions of dollars, Red Hat’s Michael Tiemann highlighted in 2009.
While nearly all software has been constrained by myopic, proprietary licensing, that’s starting to change. Facebook and other Web giants recognize great value in releasing their code. The best developers want to work on the best code and increasingly the best code is open source. So Netflix, ostensibly a streaming video company, regularly hosts open source engineering days at its headquarters and makes a point of releasing its software in order to attract developers. That being said, Netflix did announce this week that it will be retiring its public application programming interface last week.
All of which is great: more open source, more awesome. But for all the open-source software being developed and released by such companies, precious little is being supported in a way that makes it easy for mainstream enterprises to embrace.
Spit, Polish And Commitment
A good case in point is the Apache Cassandra project, which Facebook built and open sourced in 2008 to support its messaging system. Two years later, Facebook had moved on. As the company iterated on its messaging system, Facebook switched from Cassandra to Hbase:
In 2008 we open-sourced Cassandra, an eventual-consistency key-value store that was already in production serving traffic for Inbox Search. Our Operations and Databases teams have extensive knowledge in managing and running MySQL, so switching off of either technology was a serious concern. We either had to move away from our investment in Cassandra or train our Operations teams to support a new, large system.
We spent a few weeks setting up a test framework to evaluate clusters of MySQL, Apache Cassandra, Apache HBase, and a couple of other systems. We ultimately chose HBase. MySQL proved to not handle the long tail of data well; as indexes and data sets grew large, performance suffered. We found Cassandra’s eventual consistency model to be a difficult pattern to reconcile for our new Messages infrastructure.
Facebook, in other words, doesn’t necessarily stick with a technology just because Facebook created it. As former Facebook engineer Steven Grimm notes, “Facebook is very pragmatic about which tools we use; the fact that Cassandra was originally developed in-house doesn’t give it a significant leg up when we’re deciding how to store the data for some new site feature, except in that the fact that we already have in-house expertise would factor into decisions about maintainability and so forth.”
Which is great for Facebook, but not necessarily for the many companies that have since embraced Cassandra like Netflix and Cisco. Especially for mainstream enterprises, they want an element of stability that Facebook simply can’t deliver, because Facebook isn’t in the business of supporting the software it open sources.
A Continued Need For Open Source Companies
Which is why we’re just at the beginning of the “open source company” era, not the end. For every significant open-source project released by Facebook or its peers, there will be companies set up to provide ongoing maintenance and support for the code.
While it has become trite to declare that “every company must be a software company,” the reality is most companies are a long way off from this goal. Even for companies with core competency in developing software, they don’t necessarily have bandwidth to take on the maintenance of every open-source project that they use. As such, there will always be a need for open source companies who specialize in supporting software, not simply writing it.
Cassandra offers a great example, with DataStax providing commercial support and ongoing development of the code, now residing with the Apache Software Foundation. But there are many other examples, like Hadoop, launched at Yahoo! and now supported by Hortonworks and Cloudera (among others), or Lucene, initially developed by Doug Cutting at Yahoo! but now supported by Elasticsearch, Lucidworks and others.
We’re entering the golden age of open source. All the most interesting, biggest trends in computing are being driven by open source, whether it’s a mobile operating system running most of the world’s smartphones, the cloud infrastructure supporting enterprise applications or the Big Data infrastructure that undergirds our ravenous desire for more data, better understood. And while much of this open-source software will be released by companies that aren’t in the software business, per se, the best of it will be supported by those that are.
Photo of Facebook CEO Mark Zuckerberg by Selena Larson for ReadWrite
