ReadWriteBuilders is a series of interviews with developers, designers and other architects of the programmable future.
When Apple launched a little thing called the App Store in 2008, few people predicted the resulting explosion of mobile apps over the last five years. Raj Aggarwal is one of the few.
Aggarwal co-founded Boston-based mobile analytics company Localytics that same year to build the tools developers and marketers would later need to make sense of how people use apps. Sophisticated analytics engines have long been the bread and butter of the Web. Data produced by these tools helps developers and publishers make money, better serve their users and understand how people fundamentally interact with software.
The new era of mobile computing needed granular analytics centered not around the Web, but around the app. These days, the Apple App Store and Google Play store for Android feature roughly a million apps apiece. Every one of those apps generate data, from how many times a user opens it to what he or she does once it’s running. The consequent data explosion demands interpretation and analysis.
And that’s where Aggarwal and his co-founders, Henry Cipolla and Andrew Rollins, saw an opportunity—in a future in which the mobile app was everything. By providing an analytics engine for the mobile era, companies like Localytics and its rival Flurry laid down the infrastructure that help developers refine their work and produce new innovations. People like Aggarwal, Cipolla and Rollins are the builders that help other builders build.
I sat down with Aggarwal to discuss the explosion of the app economy, challenges in Big Data and whether to build your own software tools or to buy them.
Forecasting The App Explosion
ReadWrite: In 2009, the Apple App Store was a year old, the Android Market was just coming into being, and you happened to see an opportunity. So what drove you to start a mobile analytics company when it wasn’t clear that we would have this app explosion?
See also: A Blueprint For The Internet Of Things: Bump’s David Lieb
Raj Aggarwal: In some ways we saw the app explosion coming, before the App Store or the very earliest days in the app store. I’ll give you a little bit of background. I was consulting to mobile companies for many years and worked directly with Apple and Steve Jobs to build the business model of the iPhone, built Disney Mobile in Japan and Virgin Mobile Canada.
So I was bullish on the space. I knew the smartphone would have a huge impact on the world, and apps provide so much stronger of a user experience that they would be the way that people would interact with their services on smart phones.
[W]e got very networked into the Boston mobile developer community and realized the tools and platforms that app developers and publishers would need didn’t yet exist. Analytics was something that was important to all of us, either as consumers or creators. We believed that the app space would be significant so we decided to go after building the axe and shovels for the space rather than going after the gold rush by building the next big app.Servers Or The Cloud?
RW: So let’s go into the evolution of the product. What were the technical decisions you had to make at the beginning to create an engine that could track hundreds of thousands of apps? You guys tackle a lot of big data problems.
RA: One of the big decisions was, knowing we were going to be collecting a lot of data, what do we host this on? Do we buy our own hardware? Do we do it on the cloud? We ultimately decided, like many startups in that timeframe, to use Amazon Web Services because of the scalability and flexibility primarily.
As we have grown, our needs have become more sophisticated. So has Amazon Web Services. We still are primarily hosted on Amazon. That was an important decision made early on that has boded well for us.
Another decision related to that was, what is our core database going to be? When we had hardly any data, we started off with just a MySQL database and that worked great until one day after about a few hundred apps on board it started to seize up and we realized we needed to do something else because the way to scale MySQL is just to put on more and more boxes. That started to get very expensive. So we started to look at other database technologies. We decided to make our primary data store a column-based database, which was a newer concept, but they were much more optimized towards analytics purposes.
The column-based database aggregates data in a way that helps you access analytics data really quickly. We were able to do everything in real time and we still do. We were able to collect and maintain granular data and some of the things we are able to do we weren’t able to do a few years earlier.
Because of the technology available and the price of computing and the brains that [Localytics] has, we were able to collect and maintain granular user event level detail so that our customers could do any kind of ad hoc query and get answers to any kind of question they had. And they didn’t have to pre-define their questions well in advance.
RW:These questions are looking at how people are using these apps via the events [when somebody pushes a button or makes a swipe] within an app.
RA: These are unique questions that couldn’t otherwise be answered. Sometimes you don’t know what the questions are in advance. You’re suddenly able to ask any question of your users and get insightful data back.
So that was the data store piece that had technical decisions. As we grew in scale—now we’re collecting data on a billion unique devices—other things would break or bottleneck. The rate of data collection is enormous and we start to see thousands of data points a second. How do we scale that? We end up using different technologies on that side. For example, NoSQL databases for their ability to quickly process data and present to our primary data store. So those are some of the key technical decisions.
What To Build, What To Buy
RW: You described how the platform was built and how you could build your own proprietary software or use tools that were already available. How did you choose what to build and what to buy?
RA: There are certain things that we may not consider our key strand—that other people are amazing at—that we decide to buy. But there are many elements of this platform that we built. Stitching it all together is a huge building job.
For example, our dashboard. We could have gotten some standard off-the-shelf dashboard and thrown our data into it, but that wasn’t going to work. Our customers have very unique needs, and mobile was new and nobody built analytics for smartphone apps, so we needed to build our own dashboard and elements of that and interactions. But then for simple charts, other people have built simple charting libraries, so we don’t need to build our own charting libraries. So we’ll buy again on that.
It’s a process of understanding what’s out there that’s already good enough that you aren’t going to do better at. If that’s the case, why waste valuable engineering resources? Engineers are your most valuable resource as a startup. You have limited money to keep that focus, but then there are things (features and products) that are your differentiator. If it’s been done before, if you’re just repackaging the stuff, there’s not much differentiation there.
There are things that you’re doing that are unique and highly differentiated. You need to own those so that you can control them. You need to evolve those quickly and likely they haven’t been built before. It’s a process of selectively figuring out what you should build or what you should buy.
RW: As the platform grows and evolves, how often do you make that decision: buy and integrate or build yourselves?
RA: Very regularly. Sometimes small things, sometimes bigger things. Sometimes it’s not even a “buy,” per se, but there is an open source library that does this and there’s a trade off. We can get it done, we can integrate in a day if we do this, or we can take three days building something that will be a little more flexible for what we need to do. It’s just a trade off.
RW: I wrote a piece earlier this year that kind of pissed off the developer community that said, basically, developers are cheap bastards. They don’t like buying software, they don’t like buying tools. They’ll take a week to build their own tools as opposed to buying something they can integrate. Do you find that to be true?
RA: Engineers build. If they can build something and own it, then why pay for it?
So yes, regardless of the reasons, developers tend to look at the world a little differently. Not in a negative way, they are more focused on cost. They’re just conscious of that, that might make them seem like cheap bastards in some scenarios and it might encourage them to build in some scenarios. Part of that is cost, part of that is desire to control.
I know if that if I build this, I build it the way I want it. Given that leeway to choose, they’ll do it. I probably encourage our development team to look at ways to buy, because these people are a valuable resource and if someone else has done the work, we don’t want to be reinventing the wheel.
Apps Are Everywhere
RW: You were on the cusp of the app explosion. We are five years into that. What‘s next in the mobile world?
RA: So the world is becoming more “appified.” The user experience is so much greater in an app that the percent of people using apps is massively higher than people who access their service through the mobile Web.
Modern websites are built in that fashion more like an app. With Windows 8 apps, and OS X apps, there’s the return to apps. Apps are taking over another much bigger realm of things beyond what you typically think of your computer and your phone and tablet—any other devices that have any digital interface.