Technologists have spent nearly 20 years now predicting the future of the Web. And while the Web is not dead yet, how we use it and our expectations of it are surely changing. We want what we want exactly when and where we want it. And when we don’t get it, we don’t hesitate taking our business – or eyeballs – elsewhere.

This has led more technologists including myself to start thinking about how the Web needs to evolve to keep up with user expectations. People hold companies to impossibly high expectations to deliver extremely personalized experiences as they browse, shop, learn and play on the Web.

The Adaptive Web: The Future of the Online Experience

Guest author Dr. Scott Brave is a founder and CTO of Baynote, a provider of personalization and digital marketing optimization technology.

Previous attempts at personalizing and optimizing online experiences have struggled to give consumers what they want. That’s because they’ve been focused on drawing conclusions about our intent based on our past behaviors or purchases or compiling more data on peoples’ social graphs. And while user profiles can tell us a lot of things, 1) they’re quickly outdated, and 2) they do a very poor job in helping us figure out what people really want and need in the moment.

What we need is to build a smarter approach that allows companies to adapt to their customers’ needs in real-time.

The concept of collective intelligence, which I’ll address a little further down below, will be critical to achieving this vision–something I like to think of as an “adaptive Web.” That is, a digital experience that is always relevant and based on users’ current intent and interests. It also must be device-agnostic, especially important given the increased mobility of the online experience — a challenge analyst firm Forrester calls “the Splinternet.”

In an adaptive Web scenario, the Web will truly come to life and become self-learning. We’ll see the Google way of determining linkages between sites and content go by the wayside in favor of an approach that’s based entirely on what the Web community at large and like-minded users within it found useful.

An adaptive Web experience would represent a dramatic shift in how we interact with the Web (or more accurately, how the Web interacts with us). Let me explain.

“This is just the tip of the iceberg. At some point, we could see a time when not only your Web experience is adaptive, but also your social connections – based merely on your current intent and interests.”

What An Adaptive Web Would Look Like:

Let’s say you’re planning a trip to Vail with some old college buddies. So you hop on to Expedia to buy plane and hotel tickets. On checkout, you’re suddenly alerted to a winter jacket sale at based on the Web community’s affinities between Vail and buying ski jackets. No hard coding necessary.

After you get your jacket you then visit the Vail Ski Resort homepage, which is automatically optimized to show you the best trails because it knows you’re already headed there (and thus need no persuading why Vail is the best place on Earth to visit!). Later that night you search for “Sushi in Vail” on your iPhone and the top search results are all restaurants by your Vail hotel that are open on the days of your trip.

This is just the tip of the iceberg. At some point, we could see a time when not only your Web experience is adaptive, but also your social connections – based merely on your current intent and interests. For example, in the scenario above you could also be temporally linked with people who are going to Vail at the same time or who live in your area. This would have numerous benefits: you could share travel trips, ask them to carpool with you to save gas, or meet up for sushi at that restaurant you just discovered.

But creating an experience like this requires much more than machine learning and advanced algorithms. It requires a deeper understanding of how the human brain works.

Changing the Status Quo: No Easy Task

To no surprise, there are numerous technical and psychological challenges for building an adaptive Web. Namely, I see three primary roadblocks:

“The third obstacle is the biggest obstacle of all: pure science. It’s not a trivial problem to automatically pinpoint and serve up an experience based on a user’s current intent and context.”

  1. Privacy Issues: To deliver adaptive experiences, we have to pay attention to what people are doing online in the first place. Different users have varying levels of comfort. We’ll have to find some sort of middle ground where the value of an adaptive experience greatly outweighs users’ privacy concerns.
  2. Deciding on the Method: Second, there’s determining the approach itself. Do we need a “metalayer” over the web? Some sort of toolbar or plugin that could connect users’ entire Web experiences across devices? Do ISPs need to get involved at the network level to watch every site users’ visit and how they engage with it? These are all options to consider – some more realistic than others – but the path is murky at best at this point.
  3. Determining Users’ Intents: The third obstacle is the biggest obstacle of all: pure science. It’s not a trivial problem to automatically pinpoint and serve up an experience based on a user’s current intent and context. As someone who has devoted his life’s work to studying human/computer interaction, I can’t emphasize this enough. Predicting what people want and need, and adapting their Web experience in real-time is perhaps one of the remaining “big picture” challenges facing technologists.

The good news is that we’re not starting from scratch. In many ways, Amazon and recommendation engines were really the beginning of the adaptive Web, but because they were completely relegated to sidebar widgets on individual sites and leveraged only a small subset of users’ implicit behavioral signals, they stopped short of delivering a truly end-to-end personalized and relevant Web experience.

The DiggBar hinted at the creation of browser/machine-agnostic Web “metalayer” but as we know that did not end well. The behavioral targeting guys of today have the right spirit, but their focus is on individuals’ previous behaviors and the often-hardcoded needs of the seller rather than the dynamic needs of the buyer, which make it unlikely they’ll be the ones rising to the challenge.

The Power of the Collective

Let’s revisit collective intelligence and its role in making the adaptive Web a reality. Collective intelligence refers to the process of gathering insight from a group of like-minded individuals online, often implicitly based on their shared navigation and engagement patterns. A central concept of collective intelligence is to aggregate behaviors of the silent majority of website visitors, augment that information with the expertise of super-users and provide the most relevant information that meets every individual user’s goals.

An obvious benefit to using collective intelligence to power the adaptive Web is one of mere scale: it enables machines to draw conclusions about an individuals’ current intent based on the knowledge and experiences of the larger community. It also gives us the power to efficiently deliver automated and real-time experiences to users. This would be very difficult within any user-by-user scenario, which again poses enormous difficulties in matters of scale.

The Adaptive Web: Not If But When

Despite the obstacles I discussed earlier, I would argue that we’re on the path to moving toward an adaptive Web – and at a rapid rate. The days of programmers organizing the Web and creating linkages and associations for us are ending, and the age of the self-organizing, adaptive Web is quickly approaching.

In many ways, it’s in step with the democratization of the Internet: In the first phase of the web, only people “in the know” could publish content. In web. 2.0, everyone could explicitly contribute to the community via social networking and self-publishing tools.

Naturally, the next step is for all users to be able to contribute to building and organizing the Web in its entirety, organically shaping our individual relationship with it based on our collective experiences.

Do you agree? What do you think will be the biggest technical, if not cultural challenges in realizing an adaptive web?

Photo by waterlamd