If you've ever used the Internet — and you know who you are — you've undoubtedly had apps or various services stop working unexpectedly. For ordinary users, this usually just means no access to Twitter or Gmail for a while. But for developers, whose apps and services rely increasingly heavily on hooks into popular Web services, the problem can be far more complicated.

That's because modern Web services (and the apps that facilitate them) can fail for a variety of reasons. One of the most common problems arises when some other service has gone down — more specifically, when the application programming interface (API) that lets your app tap into that other service stops working. Trouble is, until recently there hasn't been an easy way to confirm or rule out API failures.

You Don't Need A Weatherman...

And that's where API status dashboards, the weather reports of the Web-service world, come in. Dashboards enable developers and administrators to quickly check to see what's going on with the API itself. If the API is slowed or offline, then at least you, the developer, know the problem isn't in your own code. So you can start working with (translate: yelling at) the API vendor to fix the situation.

Zapier, a startup that helps developers integrate APIs into applications, was already using just such a dashboard for its internal purposes. It's now opened up that API weather report for the world at large. Zapier's tool is unique in that it covers a lot of APIs for smaller but still useful services out there, not just their mega-service cousins. It should be a stopping point for anyone who is using one of these smaller APIs.

...To Tell You Whether APIs Are Up Or Down

It might seem a little obsessive to be so concerned about an API's status that we build "weather reports," but it makes good business sense. Like the air around us, APIs are a type of environment, too. They have to work and be available at any given moment in order to enable connectivity to a given web application and service. When they fail, data exchange can slow down or completely halt.

Of course, API failures aren't the only things that can bring down a Web service. The service itself could have bad code, or one of the servers might be on the way to failure. Tracking down the exact failure, though, can take a lot of time, especially if hardware failure is ruled out. That leaves the code itself, precipitating a search that could take hours.

So it's definitely helpful to know right away whether you've got an API problem... or something else. "When a call to an API breaks," says Zapier CEO and co-founder Wade Foster, "you don't always know where the problem is."

But Weather Reports Help

Zapier isn't the only status board around. Watchmouse has an API Status board that monitors the larger API services, such as Google, Twitter, Dropbox and the like. Its technology was so attractive that CA bought the company in 2011 and incorporated the monitoring service into its Nimsoft Cloud Monitoring tools.

Unfortunately, it's not entirely clear that the public Nimsoft page is up to date. The page is currently reporting disruptions for Digg, Dropbox and some Google services. The latter seems inaccurate, since Google itself isn't reporting any issues today.

Of course, if you have an app that depends on Google services, you can always check out Google's API status page. Amazon Web Services has its own API and service reporting dashboard, too.

If you're building an API for your own service, you can provide your users with a quick status dashboard of your own, using the Stashboard code that was open sourced by Twilio a few years ago. Developers can use the code to create a dashboard that can be hosted on Google Apps Engine.

Lead image courtesy of NOAA

The proliferation of mobile devices has created a firestorm of demand for Application Programming Interfaces (API) to act as data gateways between devices and services. But fire can also be a destructive force, and mis-managed APIs can hurt application performance, alienate developers and even lead to costly and damaging data breaches.

API Security Is Critical

Among other things, APIs serve as gateways to Web-based services like Twitter or Facebook. They are the specifications that let developers build applications that communicate directly with those services. You can think of APIs as doors; they let data in and out of a Web service. Just like physical doors, leaving APIs open can let anyone wander in, for whatever purpose.  

APIs are only as secure as they are written to be, explained Alistair Farquharson, chief technology officer for API-management vendor SOA Software. Smart developers make sure their APIs are open only for those people who have the authorized key.

What Problems Can APIs Cause?

The threat assessment for an API that isn't locked down isn't a pretty thing. Insecure APIs can fold under the artificial pressures of distributed denial of service (DDOS) attacks (which attempt to overwhelm a site or service with spurious requests in order to block legitimate access) , blocking the door through which data from a Web service is supposed to flow - perhaps bringing down the entire site. SQL script injections (which attempt to insert malicious code into a database), Farquharson added, could be used to re-route or copy data to outside servers operated by people who have no business looking at your data or your customers' information.

Because APIs enable very deep leveraging of a web service's features, they can be misused by hackers to spoof services, or even pretend to be entire websites, as web designer Feross Aboukhadijeh detailed last Autumn, when he discovered how the HTML5 Fullscreen API could be abused to appear like any legitimate site, such as a banking transaction web site. Aboukhadijeh works through how the fake web site could be created and fool many unsuspecting users, even down to a citation of a study on "change blindness," a psychological event where people can miss obvious changes.

And then there are the less subtle attacks, such as the 2008 security breach that took advantage of a bad Myspace-Yahoo services API and ended up gaining access to celebrity photos that were supposed to be privately stored.

These are the obvious malicious outcomes of APIs that aren't secured properly. But hacked APIs can also create perceptions of poor quality of service, which could erode customer confidence in a Web service.

The importance of getting APIs under control can't be overemphasized, contended identity-management vendor Xceedium's VP of Product Management, John Suit. "If the web interface is the front door to a company," Suit said, "then the API is the side door." And any door that lets in the wrong person - or the wrong code - can result in the same disastrous results.

Building A Better API Lock

Locking down APIs can tricky business. In these early days of the API boom, there are many different API standards being used by vendors to create the APIs through which applications will leverage Web services. Complicating that is the fact that there are a lot of different security standards, too.

This is a rich recipe for problems, since an effective API management system must allow authorized developers in to use the API, but not let anyone gain so much access they can subvert the API or use it as a doorway to the host service's internal data. Oh, and add to that mix the problem you have if APIs have to reside in a public cloud environment, outside your firewall.

Most security experts recommend using some sort of the strong authentication process in place when working with APIs. You need to make sure that the absolute correct person is accessing the API.

SOASoft's approach is a just-launched API Gateway virtual appliance that uses an OAuth server to work with many different existing security protocols. Playing to its strengths, Xceedium uses role-based identity systems to not only make sure the right person is connecting to the API, but that person should be accessing that API in the first place.

Things To Do Right Now

Even if you don't want to implement a formal identity and security management system for APIs, there are steps to take right now that will at least help mitigate potential problems.

If you want to prevent SQL injection attacks, then by all means sanitize the inputs in the API that connect to your internal databases. This will reduce the risk of a successful attack of this kind: 

API developers should also make sure that everything is transmitted through the Secure Socket Layer (SSL) - encrypted and transmitted by HTTPS - so that information like usernames and passwords are not captured in-process and then used to gain access to users' accounts or worse, the host organization's account. 

APIs are becoming increasingly important as so many new devices on the Internet generate and consume data via an ever-expanding list of Web services. While essential, those APIs also creating tempting targets for hackers. The need to lock down this growing vulnerability has never been a higher priority.

Lead image courtesy of Shutterstock, comic courtesy of XKCD.

A technology born of the Web and accelerated by mobile is now blossoming inside businesses. Ignored for years, application programming interfaces—a key layer of connectivity between disparate software—are undergoing a renaissance.

The trickiness of managing these connections, and their importance to the way businesses run their operations today, explains why we see vendors like Intel buying Mashery. Or CA Technologies snapping up Layer 7 Technologies. Or MuleSoft picking up ProgrammableWeb. Or, this morning, a startup contender, 3scale, raising $4 million from investors.

And that's just in the last seven days.

Of course, this raises the question: what the heck is an application programming interface?

First, The APIs

In the simplest terms, an application programming interface, or API, is a set of requirements that enables one application to talk to another application.

On your desktop, an API is what lets some applications talk to others (like Word to Excel and vice versa), or access features of the operating system. Such APIs are familiar ground to any programmer who has built an application that needs to share features or data directly.

This is the API with which I am familiar: steady sets of code and requirements that lived on the operating system. But there's a whole other class of APIs, built for Web services, that has kicked open the field of API management.

Web APIs are analogous to their older counterparts, but they serve as gateways to Web-based services, like Twitter or Facebook or Foursquare or Amazon. They are what enables developers to build applications to communicate directly with those services.

If you have a third-party app that connects to, say, Twitter, that app communicates with Twitter's API to handle the actual connection. You, as the user, never see this API. As far as you are concerned, the whole thing is seamless. You post a tweet in the app and it shows up in the Twitter feed. But it's the API that handles the job.

It is easy to think of APIs in this context as doors; they let data in and out of a Web service. But they are rarely indiscriminate doors. Like any door, they only swing in a certain way. And they are typically open for only the people who have keys to the lock. They have rules.

And rules have to managed.

Here Come The Managers

It turns out, explained Ed Anuff, a vice president at API management vendor Apigee, there are actually a lot of things that need to be managed about Web service APIs.

There's the sign-up process for developers who express interest in using an API. There's the documentation for the API, so they can write code that accesses it. There are credentials to be issued to both developers and users—these are all just part of the scope of information that has to be managed when a service releases an API for developers.

"All of that stuff is part of what an API management tool does," Anuff said.

A critical function of API management tools is handing out the keys that let authorized developers unlock the door to a Web service's data and functionality. Some APIs charge for access; API management tools handle billing. Sometimes there are limitations on access; those must be enforced.

API management began as a way for popular consumer Web services to open up to the creativity of independent developers. But what fueled the rise of API management as a cottage industry was enterprise IT managers who saw the success these household Web names were having with their APIs and who wanted to adapt the same model for their internal infrastructure.

"Lots more companies looked at these Web services and saw things they needed," Anuff said. "Internal APIs didn't have this self-service stuff."

What really kicked the industry in the pants, however, was the tidal wave of mobile computing. Rather than building two separate versions of software for a desktop website and a mobile app, it's far more efficient to build an API for the underlying service that holds user data and business logic, and then build desktop and Web versions of software that talk to that same API.

Add up all the different mobile platforms out there—iPhone, Android, Windows Phone, and so on—and an API rapidly becomes the only sensible architectural approach. Suddenly enterprise developers needed much better API management to handle all of the apps they wanted to build for their own employees on a variety of platforms.

It's All About The Data

Anuff gave two big reasons why enterprises are seeking API management tools.

The first was operational. If a developer produced a poorly written app that made a burst of requests to an API, one right after the other, for instance, an API management tool would enable the IT staff to throttle the requests hitting the company's Web service to something approaching a sane level until the app could be fixed.

The second example is very likely the reason why there's been so much interest in this sector of late.

Recall that when an API is in use to connect to a service, then all of the data shared by the third-party app and the service passes through the API and, therefore, through the API management tool. This means that API management tools can be one-stop shops for rich and valuable data.

Larger vendors who want to keep their skin in the big-data game are going to be very interested in startups in the API management space. The analytics API management tools can provide for the requests they handle are a rich gold mine of information, and a new source of data is bound to attract attention.

Which explains a lot of the hubbub.

The Machine-to-Machine Future

Today, the APIs we think about most often - like Twitter's and Facebook's - typically handle requests generated by people clicking on a website or swiping and tapping on an app. But another, far more interesting potential for APIs lies in processing requests generated by machines - a market that could hit $18 billion in spending by 2014.

Think of the smart, Internet-connected energy meters being adding to homes. Or diagnostic sensors in your car that report back to the manufacturer when there are signs of an incipient engine failure. Or systems that detect atypical network traffic and reroute it on the fly to avoid slowdowns or outages. These all need defined rules for how one machine talks to another. And those rules are found in - you guessed it - APIs. APIs that need to be managed.

That's the real growth market for APIs. And it suggests that what we've seen in the past week is only the first glimmer of a vein of gold that smart people will mine for decades to come.

Image courtesy of Shutterstock.

Intel, the chip giant, is buying Mashery, a seven-year-old company in San Francisco that specializes in linking together Web-based software and services, a company spokesperson confirmed to ReadWrite.

Mashery's 125 employees are learning of the acquisition through a companywide email sent this morning. Intel expects to offer the "majority" of employees jobs when the deal closes, which is expected to happen in the second quarter. The group will remain in its current location and join Intel's two-year-old Services Division. Terms were not disclosed, but the deal is not material to Intel's financial results.

Big Implications For Intel's Core Business

The implications of the deal are huge: it signals Intel's recognition that the central processing unit is no longer a silicon chip. It is the network.

Mashery, a company ReadWrite has long covered closely, specializes in managing application programming interfaces, or APIs. APIs are the lingua franca of the Internet, the systems through which machines communicate with other machines according to preset rules. For example, Facebook's platform, which websites and apps rely on to add social features, is a set of APIs. Foursquare uses APIs to let other apps access its location database and other features, allowing Instagram and Evernote users to add a place to a photo or a note.

The same techniques that connect consumer apps, it turns out, also work well within large businesses. Comcast, for example, uses Mashery's API management service to allow programmers to access internal systems. That's a far more sensible way to create internal software than the alternative, which involves doing a lot of one-off integrations at considerable time and expense.

Smaller companies often find it difficult to set up systems that grant developers access to these software interfaces. Likewise, enterprises don't generally want to build their own API-management systems. Recently, that has become a bigger and bigger business for Mashery.

Moving Beyond Chips

Intel is in the midst of a shift away from just selling chips to selling software and services. This change, while little-noticed, has been long in the making. Intel bought McAfee for $7.7 billion in 2010, putting it into the security-software business. In 2005, Intel bought a smaller company, Sarvega, which specialized in XML gateways. (XML, or extensible markup language, is a broad descriptor of a file format commonly used in APIs; an XML gateway transports files to make APIs possible.)

Intel first partnered with Mashery in November of 2012, pairing Mashery's API-management tools with its own security offerings. By bringing Mashery in-house, Intel has a more complete and credible offering in cloud-computing infrastructure. (Most cloud-software services communicate with other services via APIs.)

Ideally, Intel might sell the chips inside the servers running the software programs that communicate via these APIs, too. (It has a substantial business selling such chips.) But what's more important is the notion that Intel has a product offering that speaks to innovative startups, not just struggling PC manufacturers.

Mashery has raised a total of $35 million from investors, most recently $10 million last year in a deal that valued the company at $60 million. An experienced startup founder familiar with the terms of the deal says that investors are "happy" with the outcome.

The deal's not expected to be material to Intel's results, but industry norms suggest that Intel likely paid two to three times Mashery's most recent valuation—a range of $120 million to $180 million. (A Mashery spokesperson declined to comment on the company's fundraising or the deal's terms.)

Trying to break your addiction to Facebook's non-stop party of social micro-happenings? If you use the Firefox Web browser, that uphill battle might just have just reached Sisyphean heights. Tuesday, Mozilla cracked open its social API with native support for Facebook's chat feature, Facebook Messenger.  

I spoke with Jonathan Nightingale, VP of Firefox Engineering, and Gavin Sharp, Firefox Engineering manager, about what inspired Firefox's new support for social integration:. "We're trying to recognize that social is a different kind of thing," said Nightingale. "People don't use social the way they use the rest of the Web." Since users aggressively tab-hop to stay plugged into sites like Facebook, the team wanted to develop a way to let social notifications "float above" the browsing experience in a more persistent, less cumbersome way.

So much for productivity.

The new Facebook integration goes above and beyond the existing add-ons that thrive in the Firefox ecosystem, draping Facebook's social sidebar and chat boxes right over the browsing experience. Facebook Messenger for Firefox also adds a toolbar that lets users check notifications and friend requests, and easily toggle the the appearance of the sidebar on or off.

Nightingale and Sharp drew a distinction between Firefox's flexible new social tendencies and the class of social-by-definition browsers like Flock (R.I.P) and RockMelt, which haven't exactly caught on among Web users. Nightingale notes that in true Mozilla fashion, users can opt into new features like Messenger for Firefox or ignore them altogether. "We want to be sure that, at the end of the day, users have choice."  

The built-in social tool is just the first in Firefox 17, Mozilla's latest build of its famously open source browser. The feature has been floating around in the beta since October 22, but Tuesday marks its official debut in a final Firefox release. While Facebook and Firefox worked together to develop the integrated social sidebar, Mozilla's new Social API will open the doors for other social products. "We see potential for Social API integrations beyond traditional social sites, too – imagine using the sidebar as an easy way to keep up with group projects, email or new music," the company writes on its blog.

To install Facebook Messenger for Firefox, you'll want to be sure Firefox is up to date, then click the big green "Turn On" button over at Facebook's hub for the new feature. As for turning it off? Firefox makes that easy enough in theory, but after that first dopaminergic jolt of Facebook notifications right in your browser, you might find it harder than expected to live without. 

The fledgling social media platform, which has increasingly been seen as more of a threat to Facebook and Twitter, just took another step towards social media maturity with the release of some new hooks to let other social media management tools connect directly to Google's social network.

Businesses and organizations have been chomping at the bit for these APIs (Application Programming Interfaces), so that social media management tools like HootSuite and Buddy Media can fully connect with Google+ pages, ever since Google introduced its Pages feature to the social platform back in November.

Some conections were available, but only to a select few software partners, according to Google developer Eduardo Thuler.

"In recent months we've been been previewing a new set of Google+ management APIs with some of these companies (including Buddy Media, Context Optional, Hearsay Social, HootSuite, Involver, and Vitrue)," Thuler wrote on his Google+ page.

But even that access to Google+ Pages was limited. HootSuite, for instance, was not allowed to let too many users access the social media management app to control a Page account. So only paid users of the app were given this access.

Now that the APIs are available to all, no such restrictions are in place, and social media management is a wide-open field for Google+.

Businesses Only

The APIs only let third-party tools manage a brand or business page, not a personal Google+ account. But this is a huge help for businesses that want to capitalize on Google+, because now they can use third-party tools to schedule, monitor and track posts as needed.

This represents a big change for Google+, if only because businesses will start to take it much more seriously. Google+ was slammed at its initial launch for having little to no tools for businesses to make their presence known on the platform, and even after Pages were introduced, Google still guarded its APIs strongly.

If anything, today's change means that Google+ has shaken off any "lab" reputation it might have had, so Google's annoying tendency to drop services that people still like probably won't apply to Google+.

Google+ is growing fast, thanks to all the new Android users who get signed on when they activate their phones. That's a big audience to tap into, and the new APIs make it easier for businessesto climb on board to be a part of the experience.

People search engine Pipl's people search API is now out of beta and open to all developers. The API can be used to bring a variety of information about an individual, from social networks like Facebook and LinkedIn to government resources like the United States Patent and Trademark Office and county clerk offices, into any application.

Customer ID, Social Context and CRM

Pipl’s Chief Revenue Officer Jonathan N. Schreiber says the API could be used for customer identification or verification, or to add social context to applications, such as customer relation management (CRM) systems. In this way, Pipl will compete with people-data providers like Rapleaf, which powers services like Rapportive. But it’s a flexible API and could be used in a number of unforeseen ways.

The API, built using Mashery’s platform, is designed to let developers use the GET command to retrieve Pipl’s search results in JSON on XML format, says Schreiber. Developers can specify a specific type of info they want, such as a mailing address, or just the full search results. The results are time-stamped and include a score that reflects how confident the Pipl service is that it has the right person.

Much of the data that Pipl finds is not normally accessible through Web searches but is still publicly available. According to Schreiber, Pipl uses only publicly available, non-logged-in information and respects robots.txt instructions to avoid the collection of information that’s not meant to be crawlable.

Schreiber describes Pipl as a “real search engine,” not a metasearch engine. It doesn’t query each source for each search - it has already crawled and indexed all of this information. Pipl also performs recursive searches: Once it finds an email address associated with a person, it goes back and uses it to find more profiles and information associated with that email address.

Creepy or Not, Here It Comes

This may sound a bit creepy, but Schreiber also says that although Pipl will try to determine an individual’s email addresses, the search engine will never return email address results to users. In fact, Pipl may actually prove useful in cleaning up online profiles by exposing old profiles and information.

Nevertheless, services like Pipl and other people data search engines like Spokeo have proven controversial. See ReadWriteWeb’s recent post: Here Are 20 Companies Who Sell Your Data (& How To Stop Them). While there are plenty of innocent uses for an API like this, it’s not hard to imagine developers using this API to do some unsettling things.

Lead image courtesy of Shutterstock.

In case you missed it the first time around, the OAuth Playground is a Google-sponsored site that allows developers to work with the OAuth 2 protocol.

Since the release, Google has added support for the client-side flow, which allows developers to test out client applications using the playground. (See also, Facebook's developer docs on client-side flow.)

They've also added support for newer OAuth 2.0 drafts. This gives developers access to drafts up to the March 8th draft (25). Since the Internet does not move at a uniform pace, developers still have access to earlier drafts, as well.

Spending a long time working with an application? The playground now has support for auto-refreshing access tokens – so you don't have to worry about timeouts.

The playground can be used for any OAuth apps, not just Google's services. However, if you're working with applications specific to Google services, you'll now find support for two parameters (access_type, approval_prompt) that are Google-specific. The playground also has support now for Google's API discovery service.

Whether you're working with Google services or just testing applications that need to use OAuth in general, the playground should be an excellent resource.

We have written about prior efforts by the AP to help modernize their archives, such as this project to provide non-profits with free information feeds.

The AP didn't start out by using the MarkLogic solution, but tried to implement a more traditional relational database structure only to run into problems. Their archives are in XML, which was difficult to design the right kind of data structures. Plus, they didn't have a consistent metadata collection across the archives. The MarkLogic implementation took 16 weeks from start to finish and was the first time that the AP had made use of their services. "With this new tool, we are able to run complex, Boolean searches across millions of articles in our content archive and get back precise returns in seconds or minutes instead of days or weeks," said Amy Sweigert, AP's vice president of information management. This much quicker response time is already transforming their B2B product offerings. For example, they are able to tackle other Big Data issue and bring their content to the 21st century and further enrich their news content by managing both structured and unstructured data in real time.

MarkLogic has a free license that can be used for testing and development. Deployment can be expensive, in the tens or hundreds of thousands of dollars.

The company decided to give SendHub.com a facelift because "the site looked a bit dated, and it just didn't feel as snappy as we wanted it to. All pages were rendered almost entirely on the server, and only a couple of places used Ajax to pull in data asynchronously." The company also wanted to test out its developer API>.

Because Backbone.js has a dependency on Underscore.js, they decided to use it over Mustache, Handlebars and other rendering libraries. This turned out to be a good choice because it not only saved them from having to load an additional library, they also didn't need the extra features provided by the other libraries.

Killing Zombies

Another "big learning opportunity" in building the new front-end, was "in understanding the importance of killing zombies." In this case, we're talking about objects that "you can't access directly, but live inside the application and take up memory resources." Basically, SendHub says these crop up "when you forget to clean up event listeners after objects are removed from the page (usually they are view objects)."

The best zombie-killing resource, according to SendHub, is Derick Bailey's "Zombies! Run! (Managing Page Transitions In Backbone Apps)", which explains how to use a close method to avoid zombies.

SendHub also talks about how to use a global dispatch object to get Backbone.js views talking to one another, "without tightly coupling them to each other."

If you're new to Backbone.js, SendHub's post is worth reading.

Or perhaps I just know my audience. The first rule of communication, as I have taught and been taught (both quite repeatedly, and often) is, "Know your audience." The API has become the principal communications tool of any company that does business digitally. Therefore, professes Netflix Director of Engineering Daniel Jacobson, when designing your API, you should identify, evaluate, and serve its audience just like with any other communications tool.

So naturally, like anyone who's heard the "Audience" axiom for the very first time, I asked the principal builder of Netflix' new, corporate-wide API, the veteran developer of NPR's API used in Infinite Radio, and the co-author of APIs: A Strategy Guide, what in the world he possibly means by that. In the third and final part of our continuing interview, part 2 of which was published in RWW last week, Jacobson defines two audiences: the public who uses a company's servers, and the company itself. And in contrast to many developers on the public stage today, he believes the evolution of your API begins internally.

"The most important decision you can make about an API program is, who is or are your key audience that you need to focus on?" states Jacobson. "Because once you've focused on the problem there, you get all kinds of other decisions that fall into place."

One-Third of the Traffic Isn't Half the Picture

The more common API project nowadays is based around a Web app or Web site, for what he calls the public use case. Oftentimes this is built as a shell around the business' existing services, effectively codifying the interactions that already take place every day by way of ordinary Web forms and the Submit button. A much more powerful case for API development - the foundation of Jacobson's entire philosophy - is the internal use case. This is where the business codifies the transactions that constitute everyday business.

Here is where Jacobson makes one of the most shocking revelations of all: Even though his service has been estimated to comprise as much as one-third of all downstream traffic in North America, the volume of API calls to Netflix servers whose source is the general public - someone out there with an Xbox 360 or a Windows Media Server or a Web browser connected to Netflix streaming video - is only about 10%.

"Most companies are exposed to these concepts of open APIs, and that's what gets them excited," he says. "It's about use cases, Twitter, Facebook, and whatever else changes business focus around the public case. So a lot of companies will take that concept of a public API, create a program around that, start thinking about office APIs, and then realize the opportunity that the private APIs gives them - multiple platforms, local apps... And once that happens, there's a transition point where a lot of these companies will say, 'Wow, I can really see a lot more power out of the internal use case.' And just like at Netflix, it will change your thinking about how you want to strategize your office."

As the APIs book states in Chapter 4, "The value chain starts with business assets, something that a business wants to allow others to use. Business assets can range from a product catalog to geospatial maps to Twitter posts to airline status information to services that allow products in the physical world to be controlled by virtual services such as payment systems. If there is nothing of value in the business assets, the API won't succeed."

The API's job, the book goes on to explain, is to expose these business assets in a secure and manageable fashion. Once that's feasible, developers are enabled to build applications around the API that makes these assets available. In the end, the app is developed around the assets, not the API. And if the API doesn't efficiently represent these assets, the app will likely fall apart, and the business' API strategy will fail.

Can an API Be Made 'Turnkey?'

But let's face it, isn't there some type of cloud-based service or outsourcing operation or a set of templates you can just plug in and build an API in three easy steps? You know, an "API wizard?" After all, aren't most RESTful architectures basically two-thirds the same?

You could almost hear Dan Jacobson's grimace through the telephone wire. "A lot of people think about APIs as open APIs," he responded. "And we [the authors of the APIs book] really wanted to put an emphasis on thinking about this as a business opportunity, and the internal, private use cases of interacting with your partners or your internal development teams. When you think about it in that context, outsourcing it to a company and just flipping a switch and turning something on isn't going to have all that business sensitivity baked into it.

"I've said it before, you need to bake your business DNA into your APIs," he emphasized. "If you don't do that, then you have this service that's not really filled with what your company is going for."

Netflix, he admits, is actually among the companies that have had to shift their API strategy. It's redeveloping the current Netflix Public API for a greater emphasis on the use cases that the general public will never see. "In Netflix' case, the business opportunity is independent from our ability to reach tens of millions of users, and get them into streaming on their devices and consuming our content," the lead engineer tells RWW. "That's the opportunity we are trying to pursue, and the API helps facilitate that, makes it faster, easier. And to the extent we can improve our strategy on building and improving these APIs, we can better improve our ability to satisfy our business goals."

The book projects a metaphor of APIs in the form of an iceberg. "A small part of what you see - which represents the public API - is above water, highly visible, but is also such a small percentage of the total mass. This huge mass underneath the water that you can't see, the private API, is the biggest part of the whole opportunity. I think it's a great visual around the way things are evolving in this space."

The extent to which this changes everything is a topic that no one, not even ReadWriteWeb, has fully fathomed. The Web as we have come to know it will be affected significantly. What users have come to know as Web sites will be willingly and eagerly substituted with Web apps. In Part 2 of our interview with the co-author of APIs: A Strategy Guide, Netflix lead API engineer Daniel Jacobson tells us the one huge difference between an app and a site involves the extent to which they rely on an API. It is part of every app's DNA.

The First, Painful Steps Toward Multi-Platform

In 2002, as you learned from part 1 of our RWW interview last week, when Jacobson was with NPR, he helped make a critical decision about its information infrastructure, the implications of which his team had not foreseen: "Literally the first thing that we did," he tells RWW, "is, we built the API and we put the Web site on top of it. So the Web site runs off the API. It's a little bit of a different interaction model; it doesn't have to go through the authentication and whatever else, in the same way that external apps do."

That API later gave NPR the freedom to build apps that run outside the browser, and that use that same API in different ways. So when mobile apps were invented, NPR was among the first publishers to be ready for them. When Netflix saw it needed an architecture that enabled it to reach all its users without it being dependent upon the usage model for any one device, including the Web browser, it hired Jacobson to build it.

A 2005 Netflix demo at a Microsoft convention featured one of that company's program managers at the time, Darryn Dieken, showing then-President Jim Allchin the prospects of using one underlying technology as the foundation for developing a unified product line across different devices. The technology at the time was code-named "Avalon," and evolved into what we now call Silverlight.

After showing how a Netflix product selector ran outside the browser but through the Web, in a way people had never seen before at that time, Dieken showed essentially the same selector running inside Windows Vista on a tablet PC. From there, he proceeded to show where else folks would eventually find Netflix.

The demo took the audience inside Windows Media Center, which had just been released for Windows XP and was being vastly updated for Vista. The Media Center plug-in used many of the same presentation techniques and concepts as the stand-alone version, demonstrating the benefits of code reuse.

But when the demo turned its attention to Netflix on a Windows Mobile phone, it became painfully obvious that the benefits of client-side code reuse could only go so far. Yes, there was communication taking place between all these different clients and the server. But the way these interactions were happening were based on leveraging Web site-oriented, forms-based submissions that at one level could be described as an API, but failed to be uniform - one API for many platforms.

The goal of any modern API, Dan Jacobson emphasizes, is "to treat any presentation layer the same. So if you have multiple Web sites, like NPR does (they have NPR Music as well as NPR.org), both of those sites run off of the same interaction model through the API. They're just presentation layers, the same way as mobile app or Google TV or [NPR] Infinite Radio. Users are going to consume new material in any way that they want to, wherever, whenever; and your goal as publisher is to make sure that you have a presentation layer that serves them wherever that is. And in doing so, the easiest way, the most effective way to date is to leverage APIs, and invest a little bit on having the right talent surrounding it."

"Publish Everywhere" Doesn't Have to Be Homogenous

Because presentation layers are so different from one another, he goes on, a business can and should nurture teams of developers with the exclusive skillsets that each of those layers needs - for example, Objective-C developers for iPhone apps. There's no reason why certain teams can't specialize. Having a single API that addresses each layer in a standard way, he says, provides all your teams with the flexibility they require to take advantage of the platforms on which they're focused.

This allowance for specialization tends to work itself away from the "one Web" way of thinking, the belief that everything will inevitably merge into HTML5. In professing that API design should not be centered around any one single mode of presentation, lest it eventually become obsolete (among other reasons), Jacobson advises that API designers focus on finding ways to symbolize and encode business interactions, the things that businesses do, not the things that Web sites do. Your goal is not to make the browser more efficient or the user experience more immersive. Leave that to the UX designers. As the API engineer, your goal is to enable business.

"That kind of thinking is fundamentally different than, 'How do I want to structure my content? Do I need to think about what resources can be broken up in which ways and made available in different ways?'" says Jacobson. "For NPR, for example, there are stories, there are assets, different kinds of things in that system. For Netflix, there are users, catalog items. How do you want to structure that material, both in terms of the resource level as well as items underneath it? What are the rights management concerns that go into this, legal constraints internally about what can be published? For Netflix, what can I show users in Latin America that I can't show to people in Canada? For NPR, it's, I'm publishing AP photos; whom can't I present that to, and whom can I? Those kinds of things are really business-oriented decisions that you can't just flip a switch and say, 'Make it happen.' You need to be very thoughtful about what you're exposing and to whom, and how you're going to do it so you can get the maximum effectiveness out of it."

It is this concept which may outmode, or render obsolete, the traditional notion of the Web site, the notion that something that's created once and published everywhere (COPE) must always be the same thing. Done properly, Jacobson says, it can and should be integrated with the uniqueness of each device.

"When Web APIs started out, they tended to be more about publishing on all kinds of different platforms. Now I think it's very much about aggregation, and merging others' API experiences," says the Netflix engineer. "One of the interesting things with Netflix, for example: We have branded apps on a wide range of platforms, and if you look at something like AppleTV or Roku or Xbox, or any of these other devices, we're not the only ones there. There is an aggregation of services where Netflix creates an experience on that platform. We actually integrate with their systems, we're creating an experience on that site, and then people can access our experience in the way they expect it to be presented."

Next Time: A Lesson From the Entertainment Industry, "Know Your Audience"

It would be wrong to call the concept "revolutionary," unless you measure time in units of eons. Nearly three decades after its introduction, only recently have businesses come to realize how widely this architectural principle could be applied. No longer do complex processes have to be bound to precise, policy-intrinsic procedures. If teams can work independently, and computer resources devised to suit each team individually, then all that needs to be specified is the exchange of information between them.

So it is that a software designer ends up becoming one of the public faces of the ideal of API architecture as a business tool. Daniel Jacobson is the lead API engineer for Netflix - arguably the largest single consumer of bandwidth on the entire Internet. His O'Reilly book, APIs: A Strategy Guide, co-authored with Apigee CTO Greg Brail and research editor Dan Wood, deals with the implementation of APIs not so much for software's own exclusive purposes, but moreover as a means of realigning and renovating business' resources overall.

"APIs should not be geeky 'science projects,'" reads the first paragraph of Chapter 4. "They are critical business tools. Successful APIs need clear objectives that relate directly to business objectives and track closely to key performance indicators for the business at large."

More open on the inside

We've written here in ReadWriteWeb in the past about the value of APIs in providing transparency and accessibility to businesses, mainly through enabling them to develop mobile apps that connect more directly to their customers. Jacobson has a different perspective, which derives from his experience with Netflix, and earlier as the creator of the API for NPR. It was in 2002 that Jacobson and his NPR team made discoveries that he describes as part logic, part luck.

"At that time, a lot of publishers would be buying these CMSes, off-the-shelf products like Interwoven or Vignette," Jacobson relates to RWW. "And the flexibility, and the opportunity for thinking in these [new] kinds of ways, was somewhat limited."

Subsisting sometimes from month to month on public and government funding, NPR didn't have the budget to go big and invest in a colossal, support-intensive CMS like Vignette - an investment which, at that time, often cost bigger businesses tens if not hundreds of thousands per year, after including maintenance costs. Faced with no other obvious option, NPR was forced to go it alone, building its own CMS. And in recognizing the need to maximize its efficiency, Jacobson and his colleagues decided that their system had to be designed from the beginning to be flexible enough to publish to any platform, including those that had not yet been created.

So NPR adopted a design philosophy called COPE: Create Once, Publish Everywhere.
"That was the really fortunate decision that we made... We didn't think about iPhones and tablets, and things like that, in 2002. But we were thinking that we could imagine a case somewhere down the road where the Web site would need to change again, or we're going to do another redesign... It was really important for us to have this COPE model, so we can actually capture all the metadata that's important to us in a very modularized way so that, regardless of what the display is going to look like, we can publish to it very easily. So conceptually, we separated the idea of capturing the data from presenting the data."

It was NPR's first abstraction layer. But it was not yet an API, mainly because the CMS and the database were still tightly bound. To this day, businesses that invested in content management systems around the year 2000 are wrestling with the headaches of data portability, because their CMS is too tightly bound to its database, and the database has become a rusty, misbehaving vault.

The interface as publishing

It was 2007. While NPR had a system that could publish anywhere, the create-once part was giving it problems. The creation was becoming a frightful mess.

"It was that moment in 2007, I think, when we said, we'll need another abstraction layer to separate out the direct access from the presentation layer to the database, even though we had conceptualized them as being different, that binding to the database was still there. That's when we created this new abstraction layer of the API, and shortly after that, [we realized] we could open this thing up quickly."

The process of integrating the abstraction layer was entirely internal, and its goals were focused on how NPR could retool itself. But in making that change, the organization realized it could effectively publish the benefits of that abstraction in a way that was entirely in keeping with the goals of its COPE methodology. Dan Jacobson tells us that, in this phase of the project, he incorporated another important ethic, this time straight from the world of broadcasting: Know your audience. More specifically, build each component of the system in tune with the needs of its consumer.

Jacobson's API project enabled NPR to publish stories and excerpts through its own cross-platform app, entitled "Infinite Radio."

Jacobson's book suggests that more businesses either nurture or hire someone who can serve as the technologist for their company, and make it this person's job to know the audience - to understand how data is being consumed and who is doing the consuming. "And then understand what abstraction layer, like an API, needs to be put in place," he explains, "to basically be the glue between the capturing of the data and the presentation to its users."

One term Jacobson often borrows from the software development world and applies to the business world is context. He uses it to mean the breadth of a person's influence in the company, and there are reasons that influence may be limited. But only through understanding the different contexts of business units, he feels, can a developer build an API that enables them to interoperate.

"Publishers are thinking about how can they create an organization that will put them in a position for this kind of rapid growth," Dan Jacobson continues. "At Netflix now, we have several hundred devices running off our API. Many publishers of various kinds would love to have that kind of distribution. You need your technologists in a position to have the context and the trust of the superiors, and basically everybody on board with making smart decisions and allowing them to execute. The larger the company sometimes, the more bureaucracy there is, and the more they need to have these discussions. They're basically, potentially, shackling their people... Here, you're putting them in a position to make decisions for you."

Fate has an interesting way of making itself appear coincidental. Had NPR not been so constrained by its own budget limitations, it might never have hired the team that designed their CMS and that implemented COPE in the first place. And it might still be bound by the same tight, complex information architecture that binds so many bigger commercial enterprises to this day.

"I think it's the confluence of a range of things - the financial restrictions, having good people, good context, good control of the situation, and making smart decisions - and a little bit of luck," says Jacobson. "We could have made some smart decisions at the time that weren't quite as lucky down the road. We were very fortunate."

Next in Part 2: Can the Web app outmode the Web site?

Late last year on its company blog, Usergrid stated its intent to build out to a cloud-delivered service - essentially, transferring its intelligence to the cloud to help devs build intelligence on client devices. This morning, the firm got its wish, though maybe in a way not everyone expected: Usergrid has been acquired by cloud-based API modeling tool provider Apigee. And as both companies' CEOs tell ReadWriteWeb, the fruit of their new relationship is a little ways down the road.

"The goal for Usergrid is to be able to provide the developer with an instant mobile back-end stack," explains Ed Anuff, who up until today was Usergrid's CEO. "Most mobile apps need a cloud back-end that provides them with a lot of their key capabilities. Mobile apps are user-centric, they're data-driven, and they need to be powered by a highly scalable back end." For example, mobile apps need standardized connections with social networks, not just to share content and compare social graphs but to authenticate identities.

"When a developer looks to use a mobile back end, it saves maybe 80% of the cost of developing an application," Anuff remarks. "Otherwise they'd have to do that all themselves."

What will a pairing of Apigee and Usergrid accomplish that Usergrid wasn't just about ready to do on its own? "First, it's going to allow us to get this to market in a faster way," responds Anuff. "We're going to be able to deliver it as a cloud service, which was our goal in the launch of Usergrid. Mobile app developers want this to be a seamless, one-click experience. With Apigee, there was a common vision in the way that APIs should be built and consumed."

During Usergrid's last round of beta testing, Anuff reports, the amount of API traffic generated by mobile apps was staggering, even compared against the firm's own expectations. Building on Apigee's existing platform could provide Usergrid with the more robust service performance its testers have been looking for.

But does this acquisition lead to a single Apigee tool, going beyond simply accessing and modeling API functions to the point where developers could instantiate them, populate them with user data, and modify them until they're in proper working order to include with their own mobile apps' source code? This is a question that Apigee CEO Chet Kapoor deferred for a while, though the tool for his deferment looks curiously like a velvet curtain wrapped around a gift-wrapped box tied with a bow.

"Stay tuned," Kapoor responds to RWW. "We understand the problem really well, and it's something that we're working on. Having Ed join us is going to accelerate those plans in a big way."

Some of the problems Apigee had been working to address, even before this morning's Usergrid acquisition, deal with the rapidly expanding and splintering nature of APIs themselves. They're changing almost too fast for any one service to keep up with, which in a sense mandates the need for a kind of Apigee cloud app that works in reverse - absorbing new API functions from their own developers, so app devs can work with them right away. That's an infrastructure problem that Kapoor confirms to us his company has been working with, saying, "We're going to make that infrastructure available very soon."

As KORE Telematics President and COO Alex Brisbourne told us in Part 1 of this three-part discussion with ReadWriteWeb, the engineering of such specialized devices may actually work against the creation of such an API. But commonality and generalization of functions are what drive costs down for device manufacturers - the more specialized something is, the smaller its potential market. Is there anything that "intelligent device" managers could learn from the design of existing, specialized, narrowcast (to use Brisbourne's term) devices that developers may apply to the architecture of the devices that real-world apps will remotely monitor and manage?

Alex Brisbourne, KORE Telematics: I think the broader question is, what are the lessons to be learned? And I think there are three. Number one: When you can utilize standardized interface APIs, whether they be truly industry-standard or broadly-available APIs, that are utilized and re-utilized by a number of people, you can bring efficiencies of getting products to market. Case in point: If I go back and look five or six years, there was a genuine belief that there was a business in and of itself [around] the capability of simply being able to put a dot on a map for every location, a pin on a map - where your car was driving down the road. And that could be measured in man-weeks or months of development, it was specific to a mapping provider, specific to a processor architecture, and probably specific if, let's say, you had location capabilities from the [tracking device].

Today, that capability is writing to an API that's delivered from Google, or somebody like that. And in 15 or 20 minutes, they may be putting location into their application utilizing that standardized interface, and they may or may not be paying some form of royalty to click through and get that data. So certainly, as the industry matures, I think there are broader and broader levels, ranges of building blocks that are available. And I think it's important that those are encouraged.

Second, what is the in-service lifecycle of the device handler application? The right choice of technologies - 2G, 3G, 4G, satellite - [needs to] have been thought through: where they're going to be sold and serviced, so that for example, if you sell devices in Europe, you don't build them with CDMA radios (a slightly obvious example). It surprises me how few people do think through the use cases.

And the third is ultimately the overall cost of ownership, including support and recurring costs, so that you think through the optimization of applications. Recurring costs, which is oftentimes the network, over a five-year lifecycle may represent a third of the TCO of that application. So the guys who win and make a lot of money are the ones who have really tuned their applications and user experience very effectively, and I think that increasingly people are getting that message, but it's certainly one that we feel consistently we need to continue to underscore in people's design thinking.

Later in our discussion, Alex Brisbourne and I touched on the varying projections that manufacturers with a stake in Internet of Things (IoT) have made with regard to the potential size and scope of the project - IBM has projected as many as 24 billion simultaneous devices, while others claim even more.

AB: I'm perfectly confident that you've read about the 50-billion-device story. There's a balloon to be pricked, quite frankly, before anybody ruins their career on staking out their share of it.

Scott M. Fulton, III, ReadWriteWeb: I worry about the chatter that takes place. At least up until the advent of Netflix, the thing that really constituted the majority of Internet traffic was all the signals that routers send to one another. They're basically telling each other, "I'm fine, I'm still fine, I'm still fine." It's what Cisco calls "weather reports." It used to constitute a huge majority of the stuff that browsers send, in absolute quantity. It's M2M communication. And imagine blowing that up to a 50-billion-device level, in the way that some people perceive it. We're not going to be able to handle that.

AB: That's why I mentioned the early days of working with Novell and IPX. Even over a piece of standard copper, in those days, they were sending keep-alives every 15, 20 seconds, which really didn't work terribly well, but it had suddenly gone to wide-area and had a V.56 modem attached to it.

You actually bring a very good point up; it actually brings to one other area that is of interest: Much is talked about with regard to capacity concerns in the networks. If you really believe even a tenth of the 50 billion devices - 5 billion sounds like quite a lot - are they going to create enormous traffic problems in today's cellular [atmosphere] on the basis that all the propaganda would have us believe that there's [not enough spectrum to satisfy our data demands]. And I must be completely honest and say that, many of the research, I think, are a bit mischievous in terms of how strapped we are for spectrum. We certainly have an issue if we, as individuals, continue to use the wireless network as prolifically as we are doing on our iPhones. But much of what happens in the machine-to-machine world is actually microscopically small amounts of data, in the big scheme of events. A hundred thousand home alarm systems, probably between them, generate maybe two gigabytes of data in a month. Which is maybe all that one iPad generates.

The average [bandwidth] usage on M2M devices is probably, at the very low end, probably 30 kilobytes per month, and maybe at the high end - excluding digital signage, the fringe areas - maybe five or six megabytes per month. You can push a heck of a lot of the traffic, particularly when some of it comes off-peak, into today's network.

So I'm left worried about the spectrum and the capacity, but perhaps more worried about policy management and prioritization, which are going to be perhaps the interesting talking points over the next two to three years.

SF3: I've talked to some analysts who projected the level of data usage on wireless in years 2015, 2020, and how many terabytes of data will that be; and when I've asked them what they base their conclusions on, they say, "Scott, just do the math. You put the numbers into a calculator." They stuck 2008 into a spreadsheet, 2009, they drew a line, and they said, "Okay, in 2020 it's going to be up here at this rate of growth." Instead of looking at the factors and taking into account the possibility - maybe even the probability - that someone will solve the problem of why data is ballooning to that size in the first place.

And most of the data that's communicated now between people, on the Web, is junk. It's not even pertinent to what they're saying to one another. It's not actionable information. And better applications - and there could very well be some - will reduce that.

AB: They will. And you look at what's happening in the world, the next generation of 4G networks - which have in the past been more traditional systems for provisioning - are being taken over by policy managers. So the ability to classify your traffic in differing ways, will become very, very important. We're not quite there yet, I think. We haven't had the push yet, bluntly, [where implementers say], "We have to utilize the technology available to us to actually control the resources which are at our disposal." But certainly I do think we'll start to see these initiatives primarily because it's an opportunity to start to differentiate tariffing to applications, and frankly, monetize the value of the network in some of these vertical markets.

A patient alarm system for a guy walking around with a pacemaker, arguably is a little bit more critical than somebody else who's simply trying to repossess a sub-prime auto lease contract. They both want to use the same 50 kilobytes of network capacity, but there's a somewhat different value proposition.

If Bidelman's name is familiar, it's because he wrote the book on the HTML5 Filesystem API, literally. Bidelman also worked on the Python Client Library for Google Docs.

The idea, says Bidelman, is to make the HTML5 API "more approachable for developers that have done file I/O in other languages" as well as making common operations like renaming, moving and duplicating files easier. To that end, the library uses familiar UNIX-type commands (cd, cp, mkdir, mv, rm and others) and accepts entries in several formats.

Bidelman has also included sample code, tests and a sample app for testing out filer.js.

Currently, the HTML5 Filesystem API is only implemented in Chrome, so you won't be able to make use of filer.js in other browsers. If you're not familiar with the HTML5 Filesystem API yet, there's Bidelman's book as well as a tutorial on exploring the filesystem APIs on HTML5 Rocks.

Though the interview was limited to only 24 companies (leaving some doubt as to whether the sample size is adequate enough), the Web API study published by Hurwitz & Associates shows only 12% (3 firms) registering "expanding to social networking sites" as an important motivating factor for adopting APIs in applications.

Some 82% (20) of the companies interviewed for the report said application integration was among the most important driving factors in their API adoption processes, while 78% (19) cited collaboration with partners as most important, and 61% (15) cited connecting to more devices.

The trend indicated here suggests that although Facebook may be "eating the Web," as suggested on multiple occasions this year by Salesforce.com CEO Marc Benioff, it may have gotten too full to eat businesses.

Or, as the Hurwitz team puts it, "these companies recognize that providing a quick and easy way for developers to integrate applications leads to a stronger application ecosystem and increased value to customers." Maybe. Intense study on the subject over the past five years or more from firms such as Forrester confirms that application integration is a key driver for API adoption, but not necessarily for such high and lofty purposes.

As Forrester analysts Ken Vollmer and Noel Yuhanna put it more directly last April, "Enterprises are seeking a lean, mean, and more holistic approach to integration, doing more real-time integration and planning increased usage of enterprise service buses (ESBs) and data services platforms. The need to integrate on-premises apps with software-as-a-service (SaaS) apps is also starting to affect requirements. These trends will affect a wide range of Forrester clients this year and should shape key objectives for planned upgrades or modifications to integration infrastructure and skills."

In other words, it's not the need for a nice, shiny ecosystem that's the problem: It's the fact that SaaS applications are typically self-contained, and for many businesses, the most expedient way to make them useful with respect to businesses' long history of existing data, is to build quick-and-dirty scripts and tools for forcing square pegs into rounder holes. Or as Mae West put it, "Goodness had nothing to do with it."

In a separate question in the Hurwitz survey dealing with the business motivations for adopting APIs, as opposed to the technical factors, 75% (18) of respondents cited the need to connect to more partners as critical, while 65% (16) cited the need to expand their channel strategies. Only 12% (3) said it had anything to do with improving the recognition of their brand - which, if you think about it, is usually one of the factors behind building an ecosystem. That's another indicator that there's grittier, more pressing business needs at work here.

Demis Bellot launched the project on GitHub earlier this month. The goal for jQuip? According to the GitHub page, "to kickstart jquery.com into re-organizing its code-base so it's more modular since we believe we've proved the most useful parts of jQuery is a fraction of its code-base."

The What and Why of jQuip

So what is jQuip? It's a "smaller, lighter, faster, more modular jQuery" that allows developers to include the parts that they want. According to the GitHub page, jQuip includes "90% of the good parts" in jQuery and is broken into several components that developers can include if they need them. A big chunk (three modules) are only needed if you plan to support older browsers. Developers can even use a simple page to generate a jQuip distribution of their own with the features they want.

Bellot says that he started on jQuip while working on a consultancy project, using jQuery on an "interactive datagrid to manage a large dataset in their CRM systems."

Performance problems in the application led Bellot to start whittling jQuery down to size. The client required IE6 support, but the combination of the large dataset and jQuery's size were causing the UI to lock up and throw JavaScript timeout errors. So Bellot made a "midnight ditch effort" that was the seed of jQuip.

He says more recent projects led to making jQuip an official GitHub project. "I started developing a '3rd party' drop in widget for work (Careers 2.0) where forcing a jQuery download (even on a CDN) would've been too heavy and lead to a degraded experience we wouldn't be able to completely control (esp. considering the fraction of jQuery we needed)."

Bellot is also working on the ServiceStack Web services framework, and says that embedding jQuery "has a significant effect on the size of the binaries." jQuip is designed to slim down the size while retaining the jQuery API.

The Future of jQuip

Given Bellot's dissatisfaction with jQuery, why not work with the project to solve the problems he has? Bellot says that he's made "no attempt" to coordinate with the jQuery project. "I believe they were heading in another direction (i.e. they get bigger with every release) and I needed a working solution today and couldn't wait for a future release (if it was coming at all)."

But, he says that he hopes that jQuery will respond to jQuip by restructuring their library "in a more modular architecture so we can only include the parts we want."

Bellot says that the core functionality in jQuip is already there, though he plans to restore bits needed to work well with Underscore.js and Backbone.js. Both are libraries for developing Single Page Applications (SPA). Anything else, says Bellot, can be included via a plugin. Some features, like abstracted events, won't be implemented in jQuip at all. Bellot says that it's rare that they're needed.

jQuip will also have some features, via plugins, that jQuery doesn't have. Bellot says that he plans to contribute plugins to jQuip that will make it easier to develop SPAs. He also says he'll "happily include anyone else's contributions into the library service," though developers should make sure they also work with jQuery.

One thing that Bellot is clear on, he's not looking to compete with jQuery. "I just want to access to a modular jQuery using only the parts I need, whether jQuery.com provides it or I have to maintain it myself. As soon as jQuery provides an option to do this, either by a more modular architecture or some functionality to strip out dead-code I'm happy to maintain it. When they do, the jQuip core will be replaceable but the plugins should be easily portable as they'll rely on the same jQuery API."

Bellot is looking for contributors to fill out missing pieces of jQuery and tests. Is jQuip likely to catch on? If you're using jQuery and looking at jQuip, or taking a first look at jQuip, let us know.

Basically, Google's playground is a way for developers to walk through OAuth interactions from start to finish.

Pick the APIs that you want to access (everything from Google Analytics to YouTube), and then start by going through the full flow. You can even set up custom endpoints to test out non-Google APIs as long as they support OAuth 2.0 draft 10. The Playground will show you the full HTTP requests and responses for each step. If you need to come back later you can generate a link that will reset the interaction to the current state, no doubt very helpful in debugging.

Google also has a forum for people who want to discuss the playground, specifically. What do you think? Is this something you'll be using?

You might also want to read Justin Smith's piece on changes to Oauth 2.0 endpoints that are due on November 15th. You'll see changes to the error responses and more. See the OAuth2-dev Google group for questions on those issues.

So while they tinker with their code, you might want to explore other alternatives that can help you measure your social media effectiveness. We have come up with 17 different services, some free, some fairly expensive. I have tried most of them and will give you my impressions so you can have a head start with your own explorations.

Before I run through the services, let's discuss eight different issues with social media metrics and how the ideal metric should be constructed.

1. There is no single number that can really be universally useful. It isn't like wining the World Series, where you have to score more runs by the end of the game. There are a variety of actions that you want to examine, and you can win in one area and be off elsewhere. My impression is that we place too much emphasis on the final number without really understanding the reasons for its calculation, as the recent changes in Klout have shown.

2. You are also measuring two grossly different activities: giving and taking. This is more than just what you post and what you consume, and there are many subtleties to both. Just because you have tons of followers and friends doesn't mean that you listen to any of them, nor they listen to you. And some of us, such as myself, are more givers (in that we are focused on outbound actions) than takers (collecting information from our networks). Or vice versa. The ideal social media metric should understand both directions and make appropriate adjustments.

3. How transparent is their algorithm, really? By that I mean can you understand how they get the results that you see, and does the scoring make sense to you? Of course, one issue is having something so transparent that the service can be easily gamed or fooled.

4. Can you examine any time-series? Klout has time series data but doesn't label its axes very well, which can be very annoying. The others don't have as much here as I would like. Sometimes you can understand the algorithms better if you can see how they track you over time.

5. How much does the service care if your content is original vs. copied? If you most of your Tweets are retweeted content, is that as good as someone who comes up with original thoughts? The ideal metric should take this into account, and most of them have focused in this area, generally because it is easier to measure than some other things.

6. How many different social networks should be scanned to derive your total score, and how should they be weighed? Klout has done a decent job of expanding their sources beyond Facebook and Twitter, but some of the other services haven't gotten much beyond these two networks yet. Obviously, the wider the reach the better the view into how you are interacting across many networks.

7. Does the tool provide qualitative suggestions in addition to just scores? The ideal metric should provide insight and suggestions for how to improve your engagement and increase your value to your chosen community. Some of them have overly general suggestions that don't really tell you what you really need to do to improve your use of social media.

8. Does your audience really, really like you? Often called sentiment analysis, it isn't enough just to retweet your bon mots but approve of your point of view. There are tools that are beginning to measure this too.
   

Twitter-only metrics

• Twitter Score gives you a single score (I got a 2 out of 10, which seems somewhat low).
• TwitterGrader is another service that gives you a single simple score. I don't think the score is very meaningful: I got 97.5 out of 100, and I know I am not that good.
• Tweetlevel was built by the Edelman PR firm and it gives some good explanations of its assessments and recommendations, although they could be more fine-grained. It tries to provide historical information but there is no way to manipulate the charts timelines.
• Tweetreach shows who retweeted you and some summary stats, and is useful to search across trending topic areas and not just specific Twitter accounts.
• TeraMetric Optimizer for Twitter. This gives you qualitative recommendations on what and how to Tweet. It costs $99/month and has a free trial but requires your credit card info up front.

Facebook-only metrics

Google-owned metrics

• SocialGrapple has paid accounts starting at $6 a month and going up to $125 a month and is used for really deep dives into Twitter.
• Postrank. We have written extensively about them here, which is used to analyze RSS feeds.

Multiple site focus

• PeerIndex is probably the closest competitor to Klout and examines three areas: Activity, Authority, and Audience. They cover multiple sites but are slow to update their scores and don't have much in the way of time-series data.
• Proliphiq which we wrote about earlier in the month has a wide array of measurements and explanations, trending hot topics and more.
• Twitalyzer shows Klout and Peerindex values and costs $5 a month.
• How Sociable is more a general search tool across many sites, and it isn't very accurate since it doesn't tie the search to a particular Twitter username.
• Empire Avenue has lots of games and points for various activities, but underneath all this frilly stuff is some interesting analysis of multiple social network sites.

Sentiment Analysis tools

• We wrote about Viralheat's sentiment analysis for Facebook and Twitter here.
• mBlast mPact can monitor multiple networks and provide some sentiment analysis.
• Kred.ly is still in limited beta but offers some promise in terms of looking at sentiment for Twitter initially.
• Traackr is another sentiment analyzer and at $500 a month is one of the more expensive tools in this list.

Really, all of these tools are somewhat flawed, and we are just beginning to see some consolidation and improvements, such as what Klout is trying to do. And certainly, Google will help here, as they have purchased two companies this year alone in this space. If any of these tools can help improve your social media methods and increase your influence, then stick with what works and what will motivate you to become a better participant in this genre.