Amazon may well be the most important infrastructure company in the industry today, given its dominant role in public cloud infrastructure. But the company making that infrastructure sing, regularly releasing code that enables companies to optimize their cloud applications, is Netflix. While Netflix isn’t a software vendor, it’s quickly establishing itself as one of the industry’s most important software companies.
Why Not Google?
Given the industry’s shift to cloud infrastructures and Big Data-driven applications, the first place to look for an industry mover and shaker is Google. But Google, while a significant contributor to open source, has been relatively reticent to share its innovations. MapReduce, which became part of Hadoop in Yahoo’s hands after Google outlined the internal project in a research paper, was more the exception than the rule (albeit an industry-changing exception).
In contradistinction to Google, much of Netflix’s influence on the industry has less to do with the quality of its software, which has been generally been good (though not everyone agrees), than it does with its willingness to share liberally. Netflix’s GitHub page is filled with an array of interesting open-source projects. The company also regularly speaks at industry events and stages its own. The Netflix tech blog is also highly informative, explaining how it leverages Hadoop, among other things.
In each case, Netflix has its own self-interest at heart, as well it should. Netflix’s cloud architect, Adrian Cockroft, makes this clear in a presentation detailing Netflix’s open source strategy. The company is looking for the best and brightest developers, and knows that the best developers generally want to be involved in open source. By contributing its own code and publicizing it through events, Netflix ensures itself a steady stream of recruits.
More Than A Recruiting Tool
But Netflix’s open-source work is more than a recruiting tool. It’s also a way to share its code and improve it through others’ contributions, be they in the form of code or simply feedback. Granted, most people working on Netflix’s code won’t have the same level of expertise with running on AWS at Netflix scale or workloads, and most aren’t going to need something like Netflix’s OpenConnect content delivery network appliances, but different perspectives informed by varying levels of experience can greatly help any open-source project.
Regardless, many of Netflix’s projects have wide applicability, even if you’re not streaming movies at massive scale:
- The Simian Army – My favorite of Netflix’s growing suite of tools, the Simian Army is a suite of tools for keeping your cloud operating at maximum efficiency. Chaos Monkey, the first member, is a resiliency tool that helps ensure that your applications can tolerate random instance failures (basically, it randomly kills your servers, forcing you to build resiliency into your infrastructure);
- Ice – Provides a birds-eye view of a company’s cloud landscape from a usage and cost perspective;
- Genie – A Hadoop platform-as-a-service tool that provides REST-ful APIs to run Hadoop, Hive and Pig jobs, and to manage multiple Hadoop resources and perform job submissions across them. Netflix has detailed its Genie work on its tech blog.
These are just a few of the many Netflix open-source projects designed to help others operate on AWS at maximum productivity. In each case, Netflix shares not only code but also commentary that describes why the company is releasing the code, and how to get the most from it.
Leading The Sharing Economy
For these reasons, Netflix is positioning itself to play a major role as the world moves to the cloud. Netflix is on the cutting edge of cloud, and given its willingness to share its experiences and code with others, it more than other companies will be the one leading mainstream adopters to the cloud.
But more than providing a beacon to would-be cloud adopters, Netflix is driving the entire cloud market forward, as Red Hat’s Krishnan Subramanian posits:
Netflix sees open source as “a strategic weapon,” in addition to being the new default for software development. The company is establishing a path for others to follow. I suspect many will follow.