Home Here’s Airlock, Facebook’s A/B Testing Platform That Helps Build The Perfect App

Here’s Airlock, Facebook’s A/B Testing Platform That Helps Build The Perfect App

Facebook’s mobile apps are built for specific operating systems. No longer solely dependent on the Web, Facebook has finer control over mobile app development specific to your iPhone or Android. But by switching to native development two years ago, the company could no longer run A/B tests. So developers built Airlock. 

Facebook just released new details about Airlock, the testing framework that allows its developers to compare data from various app implementations and decide which version of the application works best. 

The narrative describes challenges many developers face when it comes to fixing the inner workings of applications to best perform for customers. But by going under the hood with its own A/B testing platform, Facebook shows its users how, exactly, it builds a UI people Like.

“We’re one of the biggest app developers and encountering new challenges,” a spokesperson for Facebook told us. “So we’re sharing our experiences so that when other companies hit these challenges they can learn from our process and understand how we did it.”

Even Facebook Makes Mistakes

Deployment graph for a Facebook experiment. Courtesy of Facebook.

Things are always changing at Facebook HQ. Every four weeks, iOS and Android teams at Facebook coordinate and produce a new executable file with updated features and bug fixes. And with each update, the teams track how new features perform, fix performance and reliability of the app, and make adjustments the user interface that change how and where people spend their time on Facebook.

The company exposes users to two different versions of the app—an A version and B version—and uses Airlock to compare data collected from both to decide which version to ship to all users. 

In its post, Facebook noted that, for instance, one of the first experiments the company ran was changing a chat icon to the word “chat.” Using the A/B system, some people saw the word chat while others still saw the icon, and Facebook determined which version of chat users were most engaged with.

Implementing the word “chat” seemed like a massive success as more users were engaged with that version of the app. Unfortunately what developers found was a pile of bugs and a component incorrectly caching data. 

As the experiment wore on, developers discovered an incongruence between the information the device was told and a delay during which the UI was incorrect. The company eventually added a “two-way handshake,” meaning the device requests data, and the server logs the response that sends it out. 

Growing Pains

At first only able to support two experiments, the Airlock project evolved to support 10 or 15 variations of a single experiment that affects millions of people. 

There are a variety of Facebook designs and interfaces lying on the developers’ cutting room floor that were never implemented to the masses, as A/B testing proved users didn’t like it. Though as the testing platform continues to scale, developers have to make sure different tests don’t pollute one another, and keep some Airlock experiments exclusive in order to deter bugs or incorrect logging in the control group. 

Airlock started as a project with the goal of improving the navigation within its applications, and eventually, after constant iterations, we have the left-hand navigation drawer that enables us to use Facebook with one hand. 

Although Facebook released details of how and why they use Airlock, it is not available to the open source community just yet. But a Facebook spokesperson told us the company “is always evaluating if it will provide value to the open source community.”

About ReadWrite’s Editorial Process

The ReadWrite Editorial policy involves closely monitoring the tech industry for major developments, new product launches, AI breakthroughs, video game releases and other newsworthy events. Editors assign relevant stories to staff writers or freelance contributors with expertise in each particular topic area. Before publication, articles go through a rigorous round of editing for accuracy, clarity, and to ensure adherence to ReadWrite's style guidelines.

Get the biggest tech headlines of the day delivered to your inbox

    By signing up, you agree to our Terms and Privacy Policy. Unsubscribe anytime.

    Tech News

    Explore the latest in tech with our Tech News. We cut through the noise for concise, relevant updates, keeping you informed about the rapidly evolving tech landscape with curated content that separates signal from noise.

    In-Depth Tech Stories

    Explore tech impact in In-Depth Stories. Narrative data journalism offers comprehensive analyses, revealing stories behind data. Understand industry trends for a deeper perspective on tech's intricate relationships with society.

    Expert Reviews

    Empower decisions with Expert Reviews, merging industry expertise and insightful analysis. Delve into tech intricacies, get the best deals, and stay ahead with our trustworthy guide to navigating the ever-changing tech market.