New Analytics Service Applause Ranks App Quality

How do you know when a mobile app is any good? Quality is difficult to quantify. What is my cup of tea might be your wretched horror show.

Businesses with several popular apps on Google Play and Apple’s App Store need to know what people think of their apps without having to slave over the notoriously skewed star rankings and comments. They want to be able to parse that data to know where they need to get better.

A Fool’s Errand?

Mobile testing company uTest set out to do the extremely difficult: quantify app quality. Wednesday morning it released a new product called Applause designed to do just that.

“It started out almost as a fool's errand - to see if we could find causal relationships in the data from app stores, both ratings and reviews,” said uTest’s Matt Johnston.

Applause is simple on its surface. It uses an algorithm to crawl the millions of comments and star ratings of all apps live in Google Play and the iOS App Store. It then breaks everything down into 10 Applause Attributes:

  1. Content
  2. Elegance
  3. Interoperability
  4. Performance
  5. Pricing
  6. Privacy
  7. Satisfaction
  8. Security
  9. Stability
  10. Usability

Essentially, uTest is looking to automate explicit user data and parse it into something useful. This is a different approach than that of other mobile analytics companies like Flurry, Localytics or Kontangent, whichtrack implicit data (user device, screen size, operating system, time of use, etc.) and generate business intelligence reports on it. 

The end result is what uTest calls an “Applause Score” ranging from 1-100. For instance, the social tablet magazine app Zite scores a 62 on Android and a 66 on iOS. 

Applause Attribute scores for CBS Fantasy Baseball on Android Applause Attribute scores for CBS Fantasy Baseball on Android

Applause gives marketers and product managers the ability to make objective decisions based on the aggregation of qualitative data. The Applause algorithm can make decisions based on looking for keywords to determine how “cool” or functional an app is. For instance, if multiple users complain that an app “hangs” or “crashes,” its performance attribute will be low.

“There is a lot of opinion-based debate between the [Chief Marketing Officer's] CMO's organization and the [Chief Technology Officer's] CTO's organization and even the CEO about what should we do next?” Johnston said.“ I think, in a lot of cases, the president or CEO is sitting there in the middle and saying, OK, I know what your opinions are, but there is really no data behind it.”

1.1 Million “Live” Apps

Johnston said that between iOS and Android, there are 1.1 million “live” applications. By “live” that means that they are in the app store and able to be reviewed. Both Google and Apple claim that their stores have around 800,000 apps apiece. To a certain extent, that is a pleasant fiction from the companies’ marketing departments. 

“You see a lot of apps that get discontinued either because the app store shuts them down or because the publisher pulled it or replaced it with a different version. We actually see the number at 1.1 million combined,” Johnston said. “You wade into the marketing from the app stores. They want to be the first to say that they are up to a million, obviously. But they might have different definitions than what is available to users.”

To be clear, when Johnston says iOS and Android, he means Google Play. The Amazon Appstore for Android is currently not being crawled by uTest’s Applause algorithm. 

Applause solves a significant problem for businesses that have a lot of apps and those that wonder how their apps stack up against the competition. Large organizations dedicate full-time employees to aggregating rankings and user reviews into spreadsheets so that they can be analyzed.

The approach is costly and prone to error. It also does not help an app publisher determine overall qualitative performance against its competitors (unless it wants to aggregate its opponents' app rankings as well). With a few queries, Applause can look up your app and your competitors and give you a snapshot of how users perceive your product. 

Apple vs. Android

One more thing. uTest has accomplished the seemingly impossible: It can now quantify the quality of the apps for iOS and Android. 

“We started off fairly skeptical that there would be any strong correlations. But as we started to refine the hypothesis we found that there is really good overlay that we could really decode the genome when it comes to mobile app quality for iOS and Android across different devices and as it relates to different categories,” Johnston said. 

Top image courtesy of Shutterstock.