Facebook has the most downloaded native application of all time. It also has perhaps the most visited mobile website of all time with nearly 350 million users and growing, using everything from basic feature phones to the smartest smartphones. It is available everywhere. The company started working on mobile solutions in 2006 and since then has grown with the times, using the tools available to them as they went along, from m.sites and WebKit touch interfaces to HTML5. Facebook’s creed, really just a way to make their developers’ lives easier, is to write once, run everywhere. This has been next to impossible.
Facebook mobile is predicated on browser technology. As Facebook’s engineering manager Dave Fetterman says in the transcript below, the browser is what Facebook is good at, how it got where it is now and how it will iterate for the future of mobile. We will touch on the future tomorrow, but be sure to read Fetterman’s presentation at Facebook’s f8 developer conference below because it will inform what we are going to explore tomorrow morning. Really, how did Facebook design for all those platforms and devices?
Editor’s note: This story is part of a series we call Redux, where we’re re-publishing some of our best posts of 2011. As we look back at the year – and ahead to what next year holds – we think these are the stories that deserve a second glance. It’s not just a best-of list, it’s also a collection of posts that examine the fundamental issues that continue to shape the Web. We hope you enjoy reading them again and we look forward to bringing you more Web products and trends analysis in 2012. Happy holidays from Team ReadWriteWeb!
What is below is a direct transcript with photos from Fetterman’s f8 presentation. A few things to note:
Facebook mobile has its backbone in its mobile website. Everything that is built into the native applications actually comes from the mobile Web. Think of the way PhoneGap wraps a browser-based website and that is how Facebook approached the problem. And then some.
HTML5 is the future. The fourth page gets into how all of this history is leading Facebook to a precipice of change with HTML5 and the so-called Project Spartan.
Also note that Fetterman talks fast and occasionally swears. He is the classic Facebook engineer: kind of young, pretty brash and supremely confident. The transcript is as true to his actual words as possible.
Changing Mobile Standards Through The Past Five Years
We took an extreme HTML-based approach to this. So we will go into how we do this so you can learn how HTML5 is the way out of a lot of these problems.
Because, it wasn’t really always this way for us. We have had the same mobile problems that you guys have. We are following the same mobile ecosystem that you guys are following to develop for your users.
So, we have the same problems of cross-platform development that you have and we are hoping that you can learn a little bit from us. So, we have been learning to deal with these issues with what we call “FaceWeb” and learning a new opportunity to get out of this that is emerging as we speak called HTML5.
So, in 2006, building a mobile presence meant that you had a WAP deck that was based on an HTML application with SMS and all of that. But, as you all know, mobile changed fundamentally in 2007. What happened then?
[Crowd] – The iPhone.The iPhone! Great. What else happened in 2007, perhaps unveiling in the room that you are sitting in right now?
[Crowd] – The Platform.Yes, the Facebook Platform API. So, what changed for us is that we had to develop a second user experience for the iPhone. A computer in your pocket that no longer sucked. So, it could have Javascript, a CSS and a really rich interaction model. In addition there was Facebook for BlackBerry, Facebook for Windows phone, for Nokia, for Samsung, for everyone now available through the Facebook API.
How about 2008? What was the big thing that happened in 2008?
[Crowd] – Ummm … Android?I will pretend that I heard the iPhone App Store. What most developers don’t realize is that the first version of the iPhone, you could build websites but the App Store was not available to later. So, in 2008, the App Store enables us to build Facebook for iPhone. The flagship, the vanguard, the best substantiation of Facebook. Based off the API, the same way that you guys are building apps off the API now.
In 2009, what changed in 2009?
[Crowd] Ummm … Android?Android, yes. I will pretend that I heard Android. Android was the new player in 2009 and really started taking off. So, all of a sudden we have all of these users on all these devices using Facebook mobile in the wide rainbow of lovely different experiences across Android, iPhone, Windows, the Web. That was great from a user perspective. What sucks? The environment for my developers, essentially. You have the bad old days. You have four different platforms to build for something essentially. You want to build for all of those groups? You are going to have to build the sucker four times. Then there are all of the features – groups, deals, the new profile. All of this stuff and the matrix got really bad. So, we have to build things four times which means that the code gets slow. The code gets old. There are different versions of parity and things just don’t work together which makes it extremely difficult for a fast moving company like Facebook.
Next page: Fetterman describes how Facebook reconciled M.Sites and Touch
How To Not Write The Same Functionality Four Times
So, how do we solve this problem? If we have any shot at solving this problem of building things four times … if you want to build it once, the Web is probably your best shot. So, back in the day we said that we have two websites, right? We have touch.facebook.com and we have m.facebook.com. Different complete docket roots and who decided what is was going to be, where you are going to go based on your phone? Well, it wasn’t that complicated. If you were a Webkit phone, you basically redirected towards touch, if you didn’t, you got M. If you had CSS, hey, we could throw that in there, no charge, no problem. So, have had this guess and redirect sort of approach to the Web.
That doesn’t really solve the problem of everyone having and optimized experience for their phone. Even things like inline images inside the screen, across CSS and versions, that is really problematic when you that those to my right are saved, those to my left are damned. That is the way it is going to work. So, if we had any chance of unifying these two groups and building something once for the mobile Web, we had to solve this problem.
So, what do we need to do? WebKit wasn’t enough, we had to have a better level of granularity. What is the difference between a Javascript enabled mobile website from a non-Javascript enabled mobile website? Really, Javascript is there to enable certain types of display and certain type of interaction, primarily AJAX. But, when you think about the controller — what is Groups? What is the news feed? What is Message? Those are kind of the same thing and they really don’t have to be written twice. So back in the day you had to use inefficient solutions like NetBiscuits which were opaque. You could do some XSL post-processing but you can’t post-process your way to good Javascript, it doesn’t work. With Webkit you started to get poly-fills and modernizer and all this great stuff. But, you actually needed a way to write your code once.
Progressive Enhancement
Here is what we did.
The cornerstone of this is detecting what your phone is going to be able to do. Capabilities, then you can start the right experience. You guys heard of WURFL at all? Wireless Universal Resource File? This is one of the projects out there that attempts to map a user to a user-agent set of capabilities. You know, what is your screen size? What is your JS? Can it do cookies? These are all pernicious, nasty problems that need to be solved. And the use-agent, as you guys can tell, doesn’t do the job. So, you need an open database for manufacturers and concerned citizens to be able to tell you what is up. We sponsored this project and this project is continuing to evolve as an open source data site.
So, once you have these capabilities you actually have to figure out what to do with these. A button on Facebook isn’t just like it is HTML or this block of Javascript or something. What you want for the homepage is to have your composers render a button that does something. So, you guys say what we really want is a button. You guys figure internally what should be rendering. If it is a low-end phone maybe it is just straight up post form. It if it is a mid-range phone with CSS, maybe you could layer CSS on there. But if it is a high-end phone you really want an AJAX style experience. So, this technique was pioneered by somebody like Yahoo blueprint. So, instead of saying at the top level that this is going to be a good site and this is going to be a low-end site, each component inside that declarative markup that renders the display will decide what it is able to do. And they compose together to form the ideal experience for that phone. This technique is called progressive enhancement.
So, this actually got us to the point where we were able to write once and run anywhere across the Web. And, you know, the Web seems doable. You can do that on your desktop browsers already. Mobile is getting to the point where you actually can do this using a system like this. But, you guys are probably bored about hearing about the Web, despite the fact that an iPhone user and Android user, whoever, can go use a mobile website competently because they all have good browsers, everyone wants an iPhone feature.
So, we were able to eliminate on of our four stacks and get to the point where we had three and that is great. But, of course, everyone wants and iPhone feature. They want their stuff to look like and iPhone, to perform like an iPhone, to feel like an iPhone, all of that. So, we could write it in Objective-C, which we had to do, or we could think about this. Right, so, what about the Facebook news feed is really iPhone-y? You have the top, you have the fixed header, you have it to render directly from cache, all that great stuff. But really, if you look at the content of the publication, that looks a whole lot like a m.site news feed. There is not really that much that is different, there is nothing that special about any particular stack. So, that means that you have to write it three times to do exactly the same shit.
Crazier Than You Guys Can Imagine
So, what if we put the actual browser inside the app? That is the crazy idea. The way we did this was even crazier than you guys might imagine. So, there are a few phones, what browser is going to be able to work for all these phones? Well, three of them have pretty good WebKit implementations. So, we thought, what if we put a Webkit enabled site inside the app and just straight up make that thing work? Just, forget it, we will figure out how to make that thing work.
So, this is actually pretty crazy. This isn’t just building Javascript that runs on the client and uses the back end as a data portal. We are actually straight up injecting the site inside these apps. Not quite as simple as getting a UI Web view or whatever. You have to have some interactivity between that Web view and the container but what if we could just straight up use the site? It is almost the same thing, right? So, this is what we did and it is kind of insane.
So, I want to tell you what we learned from this. Of course, unifying the site into the content containers is difficult and something that we would have to overcome but, betting on the browser is just kind of something that we do at Facebook. It is what we are good at and it is our ultimate opportunity to move fast and we are betting hard on the Web because that is what got us here (f8) today. So, we basically want to put the browser inside the Facebook for iPhone app.
Iterate Daily
So, what doe this get us? It means that you can ship daily. You do not have to wait for an app store approval, which sucks, and you don’t have to wait for every user to download this version on the binary which has to be shipped over into their phone. That sucks. You can ship every day. You can break it every day, but you can ship every day.
You can ship it in easy language too. Your Web development team, you iPhone team and your Android team don’t have to be separate. They are all the same people, they are all solving the same problem. And you can ship once instead of having to ship four times.
So, how does this work? Project FaceWeb is an extension of this progressive enhancement idea. So, instead of the phone saying I am rendering for a WebKit browser, we send an agent that says you are going to be rendering for a WebKit UI WebKit view inside the iPhone app. So, what you have to do is detect that, style a Web code to make that work, build a bridge between the things that you want to write to interact natively with the Objective-C, say in Javascript, then build HTML pages for Facebok in the iPhone. So, you build much smaller native goop instead of having to build over and over again.
The Answer: Wrap Facebook Mobile Inside the Native App
So, how does this technically work? Essentially the user hits the news feed icon on their iPhone. If this thing is enabled for this FaceWeb technology, it won’t map to a native Web controller, it will instead map to some end point on our server that literally says serves up m.facebook.com/home.php. It can’t just jam that thing in there. It can’t just have the Web view. You actually have to know how it is going to be interacting with the Objective-C. So, you have to send back a JS skeleton which specifies the “platform extraction calls layer” required for that frame. Then you make another trip and load the home.php and the Javascript.
It is a bit of extra work and that does affect some things. But, what that means is that we can write a basic m.facebook.com site and use Javascript to make it work on great phones and use this abstraction layer to interpret Javascript when applicable to do native things like say open the camera for something like the iPhone application.
So, what kinds of things have to go across this bridge? Obvious things like sensors and data. If you want to get your GPS, your accelerometer or orientation lock or some of these things, some are available in HTML5 but a lot of them aren’t. So, you have to be able to make some of these sensor-esque answer queries. Depending on the display and how aggressive you want to be, you have to be able to render things look like iPhone. Look like Android. And, you can’t do those things just with the browser. You could in theory do this with JS but to look exactly the same you are going to have to use a native call. And additionally, there are elements to Facebook for iPhone that we don’t really want to move to HTML. For instance, the camera flow. You open up the camera, you take a picture, you put it in your album or you select something from your gallery and caption it and upload it. That is a lot to do in a browser but it is easy to do in native. So, if you want to deep dive into any existing native flows, that is a bridge we are going to have to cross as well.
Sticky Points: Web v. Native Design and Design
PhoneGap has already solved this first problem actually pretty well across even more than just the iPhone. The common problems that you will run into if you do this is; one — objection, it doesn’t look native. Second one is that it is not going to feel native and third is that it isn’t fast enough. Here is what we found for each of these things.
The initial mandate was, hey, you guys want to do this thing in HTML, fine, I don’t want to be able to tell the difference between Objective-C rendered news feed and HTML rendered news feed. This actually didn’t end up being a big problem in terms of visuals. The Web and the app actually work pretty well together. When we shipped this to users there wasn’t any complaint except for one that I thought was pretty funny.
There was an iPad “app” for Facebook that was friendly for Facebook, It just simply wrapped the m.site. Some users said, hey, Facebook is copying the app that wraps its m.site as part of its Facebook for iPhone application. Which we were because we were actually just using our m.site and that was great. So, the fact that it doesn’t look native, we don’t actually have a big problem with that one.
Still, there are some things that you have to look out for. Position fix and overflow scroll were obviously a big problem. Android 2.1 and below, no chance. (Android) 2.2, it is starting to work and iOS 5, thank god, has promised to implement all this stuff. So, you guys are in a better position now to start down this road than you would have been 10 months ago. CSS regradients are a problem, images are a problem and [there are a couple of more gotchas].
Web v. Native Feel: Scrolling
Another problem is that it doesn’t feel native and almost always this means scrolling. It doesn’t really mean, hey, I want to swipe to delete. That is like, pfft, who cares? Almost always it is going to be something about interactivity and scrolling. Something that the native code does really well by using the GPU and like pipelining all that shit but it is something that it really hard to do in HTML. So, when people say that it doesn’t feel native, they are going to be talking about scrolling.
Sometimes you do have implement scrolling. iScroll works pretty well, Hewitts is working on a library that is pretty good. So, you kind of have to defer to a good JS scrolling library. And, make sure you do things like resize your images because was once the responsibility of the GPU is now the responsibility of the browser. And, there are all these little things that just aren’t documented, like the keyboard showing up and not closing on Android and iPhone. There are just a lot of little things that are baked into the system that are getting fixed.
The Issue Of Speed
The final one and this is the real bad one is that it just isn’t fast enough. If you see a slowdown in your Facebook for iPhone application, it means that we are working through just how to make an HTML-based application work for Facebook for iPhone and it has been pretty hard.
So, tactic one is obviously pre-cache the hell out of this thing. If you render a news feed, start and going to get the content for all of those stories because when the user hits that, you want to have the content ready for them. Cache your Web views. If that doesn’t work, the best thing that you can do is to just trick your users. At least have content appear to be ready. Fill out the rendered skeleton and then put in that cached content and then beneath, go get your new content. At least users will feel that there is something there because that is usually what their initial response is. A white screen is just not going to cut it.
So, gotchas in PERF means that you are going to use a lot of JS. You are no longer just loading data, you are loading pre-rendered markup, which is a lot more expensive. And things like HTTP cache and the app cache, they say they work within UI Web views but often they don’t. So, you have to be really careful about promises that are made here.
Taking The Approach Everywhere
But, this worked well enough that we said we are going to put the Android app on this party train too. So, we were actually able to write something like the m.site news feed once. Any time you change a news feed story or add a news feed story or add commenting and Liking or whatever, that is going to show up on low-end m.site, high-end m.site, Facebook for iPhone and Facebook for Android the very next day, or whenever we push, which is actually pretty awesome.
So, we did do this. Now we have elements of the Facebook experience, the browser running inside everything. Especially the most volatile stuff, the stuff that relies least on a certain type of display, the stuff that is going to be evolving and changing so much, like the news feed or your profile timeline, we are doing this straight in HTML. That is where the maximum benefit is.
Next Page: How HTML5 Is Leading Facebook In A New Direction
How It Has Evolved: Heading to HTML5
So, what is the history of this? So, for iOS 3.4 and 3.5 we actually did this. Your news feed and your profile feed are actually being sourced from the m.site. So, that means if the m.site changes and adds a feature, you guys get it. If the m.site is broken, it breaks everything. There are positives and negatives there. To be launched in iOS 4.0, requests, notifications and search, this list-style content that doesn’t require very specific native access, that is going to be based on FaceWeb as well. Android 1.6 and 1.7 follow the same suit. The feedback and the design have been pretty great but the bugs have been pretty crappy though. Because when you version your Javascript and things go wrong, you are tying your future to the actual Web working perfectly all the time which is something that we obviously move fast on very often.
So, the mission is to operate from one code base and adapt to all the browsers in the world. This sounds like something that Facebook would say, this is actually part of the HTML5 battle flag.
HTML5 is probably the way that we should have done it. This is the way we get to do it now because HTML5 has changed so much under our feet. The initial attempt at building a hybrid application, there were certain things in HTML5 that weren’t ready yet and we said forget it, we are going to keep moving forward. The initial attempt to defer certain things to native rendering and native handling that really could be better handled by something like HTML5 and with in-browser technology – device access, good native frameworks and application and display code.
What has changed between then and now that we can take advantage of and that you guys can, who are going to be starting down this hybrid path, might be able to take advantage of in the near term?
Device Access
So, device access. Simple things like getting things that are privileged inside the device. Geo-location, network connectivity, obviously very big for caching, things like your camera and your contacts, what has changed? Well, geo-location has actually been pretty good for a while. This could certainly be HTML5 and that is what we use inside out m.facebook.com based Places and Deals stuff. Network connectivity is emerging and that is getting to the point where you can rely on newer browsers to do it via HTML5. Camera and contacts, because those are so wildly different, a lot of those things are still specified. But, if you want to look at a general list and start by asking the system for a response and getting an answer back, PhoneGap.com actually has a lot of this stuff and they have been able to this poly-fill to make a lot of this work. So, this is the easiest problem but it is the furthest along towards getting solved, especially since we have started working on this hybrid application.
History
There are certain development frameworks that are native. Being able to manage your history, it’s weird. Because your history inside a native stack is very different from a browser history. Browsers are very idiosyncratic about how they handle history and it is a very different model. History management, client-side storage is obviously the big one and threading. With threading, if you guys have developed in iOS, you know there are certain things that you have to respect. That is true in Javascript as well. So, you need a good way to figure out how to get other threads to do your work and get it in a cross-platform way.
HTML5 is emerging standards to make all of this go away as well. History management, there is some good stuff in HTML5 coming. IndexDB and client-side storage is getting much better as well. Things like WebWorkers, if we did these things again, we would be able to do it in HTML5.
Can An App Be Built Now Using Only Web Technology?
Finally, I think the important thing is, could you build your app building straight up Web technologies? The actual Web technologies they are using, you are using. Scrolling, refresh, fast stuff, these are things that we had to defer to a native handler in an awkward way. But, scrolling, thank you to Apple and iOS 5 for promising that they are going to make scrolling, position fixed overflow scroll, this might be the problem of the past. Fixed position headers and good scrolling is a hallmark of what people have come to think of as a native application but it is a problem that can be solved by good JS now thanks to these new frameworks.
I think the most important takeaway is that the JS rendering engines have just gotten so much better in the past year and a half. V8 and Nitro have shown that you can build full-fledged, client-side JS applications instead of using Objective-C or Android Java and they will run and users will love them. Things like Netflix have done this very productively already.
So, the emergence and the investment of big browser manufacturers that are making really, really good JS stuff, which is finally paying off. And if you guys are just starting down this path, you will be surprised at just how much success I think that you could really have.
Move Things, Break Things, Fix Things Fast
So, we have been following what was available to us along the road for us since 2006 in Facebook mobile. If the best that we can do is XHTML, fine. If we have a really good browser that uses JS and CSS fine. If we can build and iPhone app and an Android app, that is great. But, what is changing beneath us now is actually the emergence of this HTML5 standards and browsers that implement these standards. Especially around device access and the ability to write apps entirely in Javascript. We have seen a lot of benefit to writing something in HTML. All of our developers are good at HTML. Only a few of them are really good at Objective-C and Android. We are able to make our Web developers the same as our client-side developers in some respects.
Being able to write it once today and ship it tomorrow? That is something that Facebook is really good at and that we love doing and that is at the center of being able to move fast. Move fast has an implicit third clause – move fast, break things, and fix things fast. That is very difficult to do if you have already shipped your binary to Apple or Android and they have to download another version of it.