It’s a fantastic time to be a web developer. Society and the software economy that serves it are experiencing a dramatic change in the way successful Web applications are built. Previous shifts in software development have had massive, far-reaching consequences: the toppling of old technology giants, crowning of new industry kings, and the very transformation of how people communicate and how work gets done.
At the same time, these shifts also ripple through the software development ecosystem, changing everything from the processes and tools developers use down to the programming languages that are taught in schools. This time, the change is the real-time Web.
Guest author Greg Slovacek is an engineer at Asana, a real-time collaborative Web application that’s reimagining the way teams work together.
Understanding Real Time
The advances in computing leading up to now have had a major impact on the software experience: richer and more powerful clients, access to more information and greater connectivity to other users. We now have the opportunity to get a desktop software-quality interaction experience combined with a Web-quality network experience, empowering us to build apps that are far better than the best of either of the two experiences alone. The first wave of these applications – highlighted by Google Docs – brought us the power and speed of desktop word-processing applications with the ability to jointly edit in near-real time from any browser.
The real-time Web takes this initial step and extends it with two essential attributes: responsiveness and freshness.
Responsiveness
Anyone who has used Apple’s FaceTime on a bad connection understands how difficult – and frustrating – it is to have a conversation when the other party isn’t responsive. If you attempt your natural conversational style in such a situation, the delays throw off your rhythm and cause cross-talk and miscues. You must become much more conscious of what you’re saying and how you say it, stripping your dialogue of nuance and quality, just to ensure that your basic message is getting across. The medium gets in the way.
But when a software is at peak responsiveness, it can have an equally dramatic (yet overwhelmingly positive) effect: The software itself disappears. Truly responsive software should feel as if it doesn’t even exist – it’s natural, fluid and nonintrusive – becoming an extension of the user and creating a sense of flow when in use.
A real-time experience isn’t just faster – it’s way, way better.
Freshness
When you track a package, it’s frustrating if the only available information is that it “left the origin.” Stale information seems almost worse than no information at all because it creates anxiety: Has my package really not moved in three days, or am I just not getting the whole story? You suspect that someone must know exactly which truck it’s on and exactly where that truck is. A better system would propagate that information to you, giving you confidence in its authenticity.
Without freshness, information is simply not trustworthy or accurate. This is increasingly unacceptable in collaborative Web applications. The benefits of fresh information go far beyond organizational agility, which requires users to respond quickly to new information. When you can trust that information is truly authentic, you feel more in control; you’re observing the information directly rather than via some proxy, which might not be filtering it in your best interest. Timely information lets users collaborate with each other more intimately.
Fresh information isn’t just more valuable – it’s indispensable.
Architecture of a Real-Time Web Application
If responsiveness and freshness are so important, what does that mean for software developers? Maximizing these qualities in a Web application requires architectural changes in fundamental ways.
While the advancements and benefits of software in the Web 2.0 era are indisputable, outdated client-server models still persist, forcing developers to heavily invest time and engineering resources to achieve the semblance of real-time. The current software development model is not sustainable. Indeed, to build a truly functional real-time Web application, software must be built from the ground up.
If developers are going to start rethinking the way software is built, there are obviously a number of things they’ll need to consider. But there are three architectural components that are especially important: the application’s data model, sync protocol and the publication-subscription system. Ultimately, developers will need to optimize the way data is stored and accessed, make sure the server is publishing information as it’s received, and make use of a balanced pub-sub system.
Let’s take a deeper dive into each of these components.
Data Model
Your application’s data model itself is primarily affected by two factors: the number of users in the system who may be viewing or updating a document/file/page, and the number of objects a user is viewing. Rich, real-time Web applications crank up both of these numbers.
Many systems treat an object as a monolithic blob, transmitted and stored wholesale in at least some layers each time it’s changed (with no way to change its aspects independently). This is a suboptimal strategy for a real-time Web app, for two reasons. First, the volume of changes means transmitting more data than necessary would bog down the system. Second, unnecessary data collisions would be more frequent, complicating application logic or impacting the user experience.
Instead, each object needs to be divided into a collection of aspects – information with individual and identifiable properties, each of which can be stored, modified and transmitted independently. Clients will want access to many objects, but not all of their data; they are automatically subscribed to only the aspects they need to display or operate on, given the view they are presenting at that moment.
Sync Protocol
The means of synchronizing data between client and server – and from server to server – is of paramount importance. In a Web 2.0 application, data is rarely pushed to a user. More modern applications may poll for new data in the background and show some indicator that users should update their displays (for example, Twitter’s “You have new tweets” message). This behavior is often limited to one aspect of the application, such as a feed; however, it is not achieved or delivered across the entire application.
A real-time application receives information relevant to any part of the user’s view at any time, and minimizes the delay between when it is available on the server and when the user can actually see it in the client view. In this situation, the comet model shines brighter than Ajax – the server must be effectively pushing information as it receives it.
As the client view may change with each keystroke, a proper sync protocol must be able to handle a lot of data churning in and out of view. The client needs to cache some data, but manage that cache carefully. In some contexts, it may be unacceptable to show cached, stale data, and the app should be able to choose where to take advantage of a cache and where to require fresh data from the server.
Publication-Subscription System
Any system that provides notifications of changes must employ some kind of publication and subscription (pub-sub) model. In a real-time application, the units of subscription must be balanced carefully. If they are too granular (e.g., mirroring every single data dependency in the system), then the pub-sub system can become overwhelmed with bookkeeping. If they are too coarse (e.g., subscribing to any change of a particular property on any object), then some operations can cause explosions of notifications.
Developers must make use of pub-sub both on object properties and query results; they must find the right balance of granularity in a hybrid approach. This allows for object-level subscriptions, as well as scoping queries to at least one collection or index.
Development Implications
It comes as no surprise that to keep up with the complexity of the real-time Web, software engineers will have to write more sophisticated applications. But it’s not going to be easy to achieve that sophistication, especially since the way developers build and maintain apps is evolving rapidly. To keep up with the pace of change, it’s time to rethink the way apps are built. The effort will pay off, and the results will be incredible.
Let’s Start Building
We have massive computing power on our desktops and in our pockets. We have a mind-boggling amount of data in the cloud representing the contributions of millions of individuals to collective knowledge. And we have fast conduits to move necessary information between any user or device that needs it. This bountiful technology beckons us to make the next big leap in software. It’s time to realize the promise of the real-time Web.
– Greg Slovacek
