Home How to Scale Code Deployment Like Twitter Does

How to Scale Code Deployment Like Twitter Does

There is a fundamental problem that Internet startups face when they begin to grow: scalability. It is imperative to be able to not only handle large flows of data from usage, but also to deploy changes and upgrades to your software across multiple servers as fast as possible. Twitter faces this challenge every day as its farm of servers grows, and today the San Francisco-based company is open sourcing its solution to the deployment problem for all to use.

Named “Murder” after the collective noun used to describe a group of crows (I guess “Gaggle,” “Flock” or any other bird plural didn’t sound as nice), this part-Python, part-Ruby tool allows lighting fast code deployments across large groups of servers. Twitter engineer Larry Gadea says it took what once was a 40 minute deploy and shortened it just 12 seconds – an impressive feat, no doubt.

So how did Twitter accomplish this?

Murder is built on top of BitTornado, a version of BitTorrent modified to include advanced functionality. Gadea says they have tweaked the app to better function in the unique environment that exists in a datacenter, including “low-latency access to servers, high bandwidth, no NAT/Firewall issues, no ISP traffic shaping, only trusted peers, etc.”

By integrating this tool with their existing deployment app, Capistrano, Twitter was able to shorten code deployments by 99.5%. Anyone can checkout Murder on GitHub right now and start thinking of ways to use it with their servers. It is important for young companies – not just those with several servers – to think about scalability before it becomes an issue. Projects like Murder are making that process much easier for first time entrepreneurs and system administrators.

Photo by Flickr user rebeccakoconnor.

About ReadWrite’s Editorial Process

The ReadWrite Editorial policy involves closely monitoring the tech industry for major developments, new product launches, AI breakthroughs, video game releases and other newsworthy events. Editors assign relevant stories to staff writers or freelance contributors with expertise in each particular topic area. Before publication, articles go through a rigorous round of editing for accuracy, clarity, and to ensure adherence to ReadWrite's style guidelines.

Get the biggest tech headlines of the day delivered to your inbox

    By signing up, you agree to our Terms and Privacy Policy. Unsubscribe anytime.

    Tech News

    Explore the latest in tech with our Tech News. We cut through the noise for concise, relevant updates, keeping you informed about the rapidly evolving tech landscape with curated content that separates signal from noise.

    In-Depth Tech Stories

    Explore tech impact in In-Depth Stories. Narrative data journalism offers comprehensive analyses, revealing stories behind data. Understand industry trends for a deeper perspective on tech's intricate relationships with society.

    Expert Reviews

    Empower decisions with Expert Reviews, merging industry expertise and insightful analysis. Delve into tech intricacies, get the best deals, and stay ahead with our trustworthy guide to navigating the ever-changing tech market.