We all know that where the data is, the money is. What ownCloud is doing, then, is sort of surprising. The project (and the company behind it) is all about helping users and companies keep control of their data. That means giving up control of the software, and hoping that money comes from services and support.

Understanding ownCloud

Like Dropbox and others, ownCloud has a client piece that synchronizes data from your desktop to a server. The big difference here is that ownCloud also provides a server that's free software (under the Affero GPL), and ownCloud isn't in the business of storing user data at all.

Instead, it's up to third-party providers to offer hosting, or for companies to provide hosting for their employees.

The project provides a server and clients for Windows, Mac OS X, Linux, Android and (eventually) iOS. You can also access ownCloud via the Web to get to files and use its collaboration features.

What's New in Milestone 4?

The project is growing by leaps and bounds. The fourth milestone release includes versioning, encryption and drag-and-drop uploading from the Web client. Versioning and encryption are a big deal for business users, and something that the competition has had for a while.

The v4 release also includes useful collaboration features. ownCloud now has a tasks application, and this release also improves its calendaring features. For individuals, the release includes improvements to the gallery features, so users can not only sync photos - they can also create a Web-based gallery via ownCloud.

Perhaps most importantly, this release includes publicly defined APIs - stabilizing the server side should make it much easier for third-party developers to create applications against ownCloud. Now the company just needs a compelling developer program.

Finally, the Milestone 4 release offers migration and backup features so organizations that are deploying ownCloud can develop an effective strategy for their users' backups.

Not Quite There Yet

The ownCloud folks are making impressive progress, but there's still a few rough edges around the project. If you ask the ownCloud folks, they'll say that they're not a Dropbox competitor. But Dropbox is still the gold standard for users when it comes to easy file sharing and syncing.

The lack of a LAN sync option, which Dropbox has had for years, is a problem. The ownCloud clients are also a bit primitive compared to Dropbox and not entirely stable. Testing the ownCloud client on Linux, the client kept shutting down due to a segfault.

The opportunity is large, and ownCloud is something the market really needs - an open-source set of tools that allow users and companies to keep full control of their data and the ability to modify and extend the tools as needed. The question now is whether the ownCloud team can build a sufficient community and do the necessary development to get ownCloud to the stage where it's ready for adoption.

Here's hoping.

The one-letter Ruby gem, t, is written by Erik Michaels-Ober. The source is available on GitHub, or you can just grab it using Ruby's gem install command. Ruby is on most *nix type systems by default (Linux distributions, Mac OS X) but Windows users will need to download the installer to get started.

After you've installed the t gem, you'll need to authorize it. You can use it without getting API access, but then you're limited to 150 requests per hour, and you can't use t to post a Twitter message or make changes to your account.

Using t

To see all of the commands available via t, just run t help. You'll find that t supports more than 40 commands, or tasks as its documentation calls them. The syntax is simple: t followed by the task name, and any options and arguments.

For example, you can use t to get all of the followers for an account by running t followers username. Note that you don't need to include the @ in front of a username.

Many of the commands are mundane operations that you can do with any Twitter client. For example, posting a status update or following another user. Running t update "message goes here" will post an update, running t follow username will follow the user that you specify, and so on.

If you're a command-line diehard, then you might prefer using t for those operations. But its real value is in doing things you can't do with most Twitter clients. Want to see a list of users who follow you but whom you don't follow? Run t groupies. You can also use that command to check on other users, t groupies username.

The most interesting thing about t is that it works well with other *nix commands. For example, you can create a list (t list create listname) and add all users who are followed by another user (t followings user | xargs t list add listname. That's just a small sample. You can use any of the standard *nix utilities with t, like uniq, which might be useful in generating lists of users to follow by examining users that other people follow (and much more).

You may bump up against arbitrary limits when adding people to lists and such. According to the Twitter API docs, you can only add 100 users at a time to a list, and a list can only have 500 users. When I tested the limit, though, I was able to add 195 users to a list with one shot. The operation failed (sort of) when I tried to add another large chunk of users (more than 400), leaving my list with 582 members. To put it another way, trial and error is called for.

Some of the t commands also support a --csv option, which you can use to dump output to a comma-separated value file and then import it into a spreadsheet.

The Bad

While t has a lot to offer, it also has a few problems. First and foremost, a few commands don't quite work. For instance, the t stream timeline command is supposed to provide an on-going stream of your timeline. It provides the first 20, and then errors out. The "matrix" command, likewise, returns an error. The "trends_locations" task doesn't seem to be implemented at all, as it returns a "could not find task 'trends_locations'" error.

The error messages, in general, are pretty minimal. You may be notified "too many terms specified in query" for instance - but no information on what part(s) of a command succeeded or failed. I've seen a few operations that appeared to mostly succeed, like adding users to a list, but received an error notice. When that happens, it's not trivial to figure out what worked.

Sparse documentation is another problem. Many of the commands have options that are not well-explained. You may need to go spelunking through the code to see what options are available.

Finally, it's pretty easy to hit Twitter's rate limit. Twitter has an hourly limit of 350 requests per hour with an authenticated client. You can bump up against the limit if, for example, you try to request a list of all users for an account with tens or hundreds of thousands of followers. Note: If you wish to check on your current rate limit, use t rate_limit. You'll see the current limit (should be 350) and how many remaining hits you have, and the time your limit will be reset.

For the most part, though, t works quite well and provides a great way to do more with Twitter than the usual clients allow. If you're doing more than idle chit-chat on Twitter, take a look at t.

Creating a request for Mechanical Turk isn't overly difficult, but it takes a bit of time editing HTML and answering a lot of questions for which first-time users don't have a lot of context - for example, deciding the qualifications for workers, or whether they need to be "masters" to take on a task.

Amazon is streamlining all that with the Categorization App by making assumptions about what its users would want, providing suggested pricing and helping to create the form that workers will see.

Right now, Amazon is providing an app only for categorization questions. Mechanical Turk offers templates for many more, including image filtering, image tagging, data extraction, data collection and several others. Expect that Amazon will add apps for most of these types of questions in the near future.

Boosting MT Revenues

Obviously, Amazon is trying to streamline the process for creating MT tasks in order to boost its revenue. More tasks means more money. But there's a bit more to it than that.

With the Categorization App, Amazon's assumptions all lead to more money per HIT. Users no longer have to decide what kind of worker to farm tasks out to; rather, Amazon makes that decision, automatically choosing the most expensive (master) workers. To increase accuracy, each HIT will be shown to two users by default, doubling the revenue.

Amazon will also suggest a pay level, to ensure that HITs are "priced attractively" to workers. Requesters can change this, but if a significant number of requesters accept the default pricing, that will probably drive HIT prices up over time.

Better Requests

Another reason that Amazon would want to create an app for MT requests is that a lot of requests are not very good. Amazon is in a position to evaluate the existing requests, and has found them wanting.

One problem? Too many categories. Amazon says that the maximum number of categories that workers can keep in mind is seven to 10. Yet they see requests with more than 150 categories!

The app limits requesters to 10 items. It requires that requesters provide instructions. It's not foolproof, but it provides better odds that requesters will generate reasonable HITs. If requesters want more categories, they need to start from scratch with a custom template.

If you're using Mechanical Turk now, what do you think of the Categorization App? Would you prefer to see Amazon simplify the service further, or is it already well-suited for the kind of crowdsourcing that you're doing?

Illustration titled "Blogging Au Plein Air, Jean-Baptiste-Camille Corot" by Flickr user Mike Licht.

HP has been trying to make the case that Oracle is acting against customer interests in dropping support for Itanium. Publicly, HP has been accusing Oracle of breaching contract in order to boost sales of SPARC servers.

Won't Somebody Please Think of the Customers?

No doubt, Oracle is strongly motivated to lure customers away from HP-UX and Itanium to its Sun hardware and Linux or Solaris. But reading through the documents made public by Oracle shows that HP hasn't been straight with its customers, and its own motivations are less than charitable. Along the way, HP has felt free to spread fear, uncertainty and doubt (FUD) about the Sun/Oracle deal to try to move customers the other way.

Last year, when HP filed suit against Oracle, Oracle claimed that HP had been lying to customers. According to Oracle's statement, "HP issued numerous public statements in an attempt to mislead and deceive their customers and shareholders into believing that these plans to end-of-life Itanium do not exist. But they do. Intel's plans to end-of-life Itanium will be revealed in court now that HP has filed this utterly malicious and meritless lawsuit against Oracle."

In fact, the documents brought forward by Oracle show HP execs in a panic over Intel's "bombshell" in 2007 (PDF) that it wanted to cancel Poulsen (an Itanium processor scheduled for release in 2012).

In 2009, the documents show, HP was considering buying Sun (PDF) to take over the Solaris "franchise" and deal with the fact that "HP-UX is on a death march due to inevitable Itanium trajectory." More documents from 2009 discuss the "impending end of life" of Itanium, while HP hoped to keep the "Itanium situation" as "one of our most closely guarded secrets." (PDF)

To keep Itanium afloat, HP worked out a deal with Intel to pay for the development of Itanium (PDF) and fork over money to ensure that Intel wouldn't lose money producing Itanium chips. There's nothing inherently wrong in HP paying Intel to make the Itanium, by the way. Trying to drag other vendors along for the ride, and being dishonest with customers about what was going on, is another story.

An email in March 2011 from Martin Fink, senior VP and GM of HP business critical systems (BCS) (PDF), complained that HP could not say that Intel "at no time communicated to Oracle a change in commitment to the future of the Itanium processor family." In April 2011, an email from Dong Wei to HP's Kirk Bresniker said Intel "specifically told them [Huawei] that the Itanium line is at end of life with 2 more generations to go."

Oracle may be happy to lure customers away from HP-UX and Itanium to Solaris and SPARC (or Linux and x86), but it seems it had plenty of good reasons to abandon the Itanic sooner rather than later.

Lessons Learned

Aside from the corporate drama, what does all this add up to? The short of it is that companies need to be very careful when they're committing to expensive platforms like HP-UX and Itanium.

All of HP's bluster about sticking with Itanium for the customers is belied by the fact that the company has gone to great lengths to obscure from customers the dim future for Itanium and how much HP has had to prop up the ailing platform.

Despite obvious signs to the contrary, HP has spent years pushing Itanium and trying to convince customers that the platform has got a long and healthy life ahead of it. Remember that Itanium was supposed to be the next generation for Intel, and there wasn't supposed to be a 64-bit line for x86 systems. Intel was forced to jump into the 64-bit race with x86 after AMD led the way and demonstrated that, yes, customers wanted to stay on x86.

Red Hat announced it would drop Itanium support in 2009. Microsoft announced the same in 2010. Intel evidently wanted to abandon Itanium back in 2007. Companies that made new or additional investments in Itanium and HP-UX after that should be rethinking their IT practices - and how much they trust what their vendors tell them.

It also, once again, demonstrates why companies should seek commodity and open source systems. Companies that have adopted HP-UX on Itanium have paid a premium for those systems, and now find themselves at a dead end. They'll get support from Oracle on current products, but will have to deal with expensive migrations (one way or another) when Oracle's support commitment ends or when they need features in later releases. Meanwhile, customers that chose Xeon-based x86 systems and commodity operating systems are ticking along just fine.

Email, instant messaging, forums, code forges and other collaboration tools make it possible for distributed teams to get work done - but they're not great tools for making decisions. The team behind Loomio wants to solve that with a new Web-based tool for focused, concise discussions that allow all team members to be heard. 

If you've ever worked with a distributed team, you know how difficult it can be to make decisions as a group. Discussions are unstructured, rambling affairs with dozens of messages flying about and no good way to track consensus. Even worse, requests for feedback can go without comment entirely, or with only a few stakeholders raising a voice.

Agree, Disagree, Abstain, Block

Discussion in Loomio starts with a discussion and specific proposal, and members have the option of voting on the proposal. A group can define the options (defaults are yes/no, abstain and block), and each member can give their view summary. As votes are tallied, everyone can see get a chart that shows how many folks are in agreement, how many aren't, how many have abstained, etc.

This sounds pretty simple, but most of today's collaboration tools don't provide a good way to focus a discussion. The key to Loomio is that it provides a central tool for discussions and (if used properly) narrows things down to decisions that are easy to vote on. Central is key here. It helps a lot to confine activity to one tool rather than making users look all over for information.

A lot of online teams communicate in several ways, including email, IM, IRC, over the phone and face to face. Stakeholders who prefer one medium (like email) lose out if discussions are held in IRC, or vice-versa. Even worse, stakeholders may be totally unaware a decision is being made at all. If a group settles on Loomio, it would enable the group to say "decisions are made here and nowhere else." If something isn't put up in Loomio (or another approved tool), then a decision wouldn't be legitimate.

Settling on a decision tool like Loomio should also help cut down on noise in other communication channels. It's popular to have discussions in email and CC everyone who might have an opinion or might need to vote on something. An active team can inspire email fatigue pretty quickly with discussions that are neverending. Loomio would allow users to visit, vote and get back to work.

Actually, Loomio isn't only for distributed teams. There's no reason it couldn't be used in any organization, but its especially appropriate for situations where team members or stakeholders are far-flung.

Can Loomio Solve the Problem?

Like any tool, Loomio would only be effective if used properly. The early design could probably do with some modification - a more obvious start and end date for votes, for example - but the initial design is solid. The Loomio team says it's already in use by some organizations. New Zealand companies or organizations like Enspiral and BuckyBox are among the first adopters - though no one seems to be providing a public instance that we can point to.

If you want to help, the group is looking for contributions from Ruby on Rails developers, as well as a little extra cash (NZ $5,000) to help the volunteer team devote more time to Loomio development. The project is sort-of open source and already on GitHub. It's "sort-of" open source because the site says it's open source, but if you look at the license text on GitHub it's basically a stump saying: "We need to add the license. GPLv2?" The pledge drive (through the Pledge Me platform) ends on May 18th. The developers have already raised more than their target, but more money might mean more time spent on development.

If adopted a bit more widely, Loomio might help take distributed teams to a new level - much like GitHub has helped with development. It is a simple concept, but bringing order to decision-making could help teams communicate better and make better decisions, no matter where they happen to be located. 

The WP2Cloud plugin was developed by OblakSoft as a solution for Yet Another Picture Sharing Site (Yapixx). Yapixx is provided as a preconfigured Amazon Machine Image (AMI) for EC2 that uses WordPress and several extensions to provide an S3-hosted picture-sharing site.

But you don't have to run Yapixx or use Amazon EC2 at all if you prefer to use hosting elsewhere. All you need is the WP2Cloud plugin and the Cloud Storage Engine for MySQL (ClouSE). Note that ClouSE is mandatory. The plugin will error out if you try to install it without ClouSE available. Naturally, you need an AWS account and an S3 bucket to put files in, too. The full instructions are on the OblakSoft site.

Once it's installed, you can decide whether to go full monty or store only a portion of your content on S3. The benefit of using WP2Cloud is that you take a load off your Web server and let S3 serve up some or all of your content. That includes full posts, if you decide to use ClouSE to put MySQL data in S3 as well. As far as I know, WP2Cloud is the only plugin that puts post data in S3 rather than media only.

While Amazon is the only service that's supported right now, the WP2Cloud documentation indicates that support for other services may be on the horizon. It would be excellent if you could tap other cloud providers or open-source cloud stacks as well.

Other Options

The WP2Cloud plugin might not work well for some users. For example, it requires MySQL 5.5.19 or higher, but plenty of sites have older releases of MySQL. And it might be overkill if you only want to store large media, like videos, in S3 and leave the rest on the WordPress host.

The Amazon S3 for WordPress with CloudFront plugin stores files in S3 transparently and offers the option of using CloudFront. CloudFront is a content distribution network (CDN) that can be used to distribute content more quickly and mitigate traffic spikes.

If you're looking to offload video only to S3, you can use the S3 Video Plugin. It does what it says on the tin, though you may need to tweak some PHP parameters to upload large files to S3.

For sites with minimal traffic (like my personal blog), WP2Cloud is not necessary. But if you're trying to scale WordPress for a lot of traffic, particularly bursty traffic, then you should take a look at some of the cloud storage options to see if they'll help you reduce site load times and server load.

Google's Cloud SQL is MySQL-based and is intended to be used with Google App Engine (GAE). Google's pricing structure is very simple, though not as comprehensive or as expandable as Amazon or others.

Google has two billing plans: a package plan and a per-use plan. The package plan has four tiers, each of which includes a set amount of RAM, storage and I/O per day. For instance, Google charges $1.46 per day for the D1 tier, which has .5GB of RAM, 1GB of storage and 850,000 I/O requests. The top package (D8) includes 4GB of RAM, 10GB storage and 8 million I/O requests for $11.71 per day.

The same instances are available on an on-demand basis, starting at $0.10 per hour, with storage and I/O extra.

The cheapest package from Google, then, runs about $45 a month and the most expensive runs about $357. That doesn't count any overages for I/O or storage.

Sizing Up Google's Pricing

Trying to compare Google pricing with Amazon, Azure or databases offered with PaaS services such as Heroku and Engine Yard is tricky, at best. Heroku's database offerings start at $50 per month, but the specs for its database differ considerably from the other providers. For example, Heroku features data clips for developers, and the hstore extension for key/value data storage.

Amazon's DB instances seem to be a bit more powerful than Google Cloud SQL instances, and Amazon has features that Google Cloud SQL doesn't. For instance, Amazon's Small DB instance has 1.7 GB of RAM and has the equivalent of a single CPU. You're also limited to Google App Engine supported languages, Python and Java.

Developers can choose between 5GB and 1TB of storage (the max for Google is 10GB storage). The Small DB instance runs about $77 a month, if it's on-demand. But, choosing a one-year reserved instance brings that down to about $45 a month. The pricing, then, seems to line up for the "small" instances for Amazon RDS and Google Cloud SQL, but Google has fewer features and what looks to be less compute power.

But if you're using GAE, then Cloud SQL is the natural choice - so it's nice to see Google finally getting this into developers' hands. If you're using GAE and Cloud SQL, we'd love to hear what you think.

Infographics are still a thing with a lot of companies, if my inbox is any indication. Easel.ly, a service that recently debuted in beta, is making it easy to create infographics online. Whether it should is another question.

Easel.ly is a Web-based tool for creating infographics. It's still in its early stages, so you'll find some rough edges, but it does live up to the "easy to use" promise. Select a theme, your objects and shapes, plop in some text, and you can have a passable-looking infographic in a very short amount of time.

Right now it seems to be missing a way to actually develop charts inside the tool - there's just a placeholder for charts that drops in one static chart image. The color palette is "coming soon," and the SVG export contains errors - at least as far as Chrome and Firefox are concerned. But, this is a beta product. Assuming the Easel.ly folks get the kinks worked out and fill out its features, it should be able to generate decent-looking infographics pretty soon.

Easel.ly doesn't seem to be the only game in town for quick-and-dirty infographics, either. There's visual.ly, which seems to have a few stock infographics you can create. I haven't tried that one, though, because it requires authorizing via Twitter or Facebook. (Sorry, kids, I am not willing to give you access to my social media accounts just to create a lame infographic.)

Does the World Need More Infographics?

Judging by the number of infographics that are pitched to ReadWriteWeb, there's a lot of demand for creating infographics. Unfortunately, there's a lot more pink slime in infographics than actual beef these days.

To put it another way, most infographics suck. It was true when we wrote that last November, and it hasn't gotten any better; if anything, it's gotten worse, as companies keep churning out infographics in the hopes of a "viral" campaign. When infographics started to become popular, many were just thinly disguised promotional vehicles with dodgy data and a lot of self-promotion. Lately, they've dropped the pretense and just gone whole-hog on the self-promotion.

Want an example? I'll pick on one of my former employers, who pitched me an infographic on 20 years of SUSE history (PDF). Now, there's nothing wrong with SUSE promoting its 20th anniversary. There's really nothing wrong with creating a nifty graphic that illustrates SUSE's achievements over the years. But calling this an "infographic" is stretching the term to near breaking. At best, it's a timeline with a few numbers thrown in. (Also, somebody needs to get the chameleon to the vet, pronto. It's not looking very good.)

There's this one from RewardLoop that is entirely self-promotional. Domo has one about "the incredible data explosion" that is as confusing as it is light on actual data. In the "Competitive Edge" section, it is entirely unclear whether individual companies are being compared, or industries as a whole.

The point isn't to pick on these companies, though, but to illustrate a point: Infographics are apparently not that difficult to create. The Web is littered with them. But good infographics are difficult to create, because it means having worthwhile information and putting it in context - not just slapping some pixels together in a semi-pleasing manner to help with your branding.

When there's a Web-based app that does that, I'll be very interested.

Prior to the library, getting things out of the Google Analytics API wasn't a trivial process. According to the post announcing the feature, developers had to learn the API, then figure out how to handle authorization, and then meld the data with another visualization library. Fun for some developers, but definitely not speedy. And when has management ever said "we'd really like a better dashboard for our Web traffic, but take as long as you like to come up with it"? Right, so something better was obviously needed.

The Easy Dashboard Library

Google worked with students at the University of California, Irvine to come up with something better. The Easy Dashboard Library has three basic steps: Set up API access with an OAuth 2.0 client ID; copy and paste some code; and configure the code to query data and select your chart type.

The post from Google demonstrates how to create a quick-and-dirty chart for pageviews, visits and visitors over the last 30 days. You can test out a demo on Google Code, where the library lives.

Developers have the option of using line, bar, pie, table or column charts. The demo shows a pretty simple query, but developers can set queries using all kinds of dimensions and metrics supported by the Core Reporting API. It looks like developers can also use the data with another chart API if they prefer.

If you still don't think it's quite easy enough to use or would like to see additional features, you may be in luck. Google is planning to work with another group of students at the university for the next three quarters. The main goal is simplifying the library, but Google is also encouraging feedback via the Google Group for the Easy Dashboard Library.

Given the prevalence of Google Analytics, this should be good news for a lot of developers. It should also make custom dashboards more accessible even to more casual users, who might not have been eager to spend the time needed to get up to speed with the Analytics API but can ramp up pretty quickly with the Easy Dashboard Library. If you've taken a stab at using it, or have any recommendations for working with Analytics data, let us know in the comments.

Mozilla's Asa Dotzler touched on this issue yesterday, saying that Microsoft "is trying to lock out competing browsers when it comes to Windows running on ARM chips." But it actually goes farther than that.

Microsoft is restricting access to some APIs on ARM-architecture devices that are, as Dotzler says, "absolutely necessary for building a modern browser that it won't give to other browsers so there's no way another browser can possibly compete with IE in terms of features or performance."

Dotzler is focused on the implications of Microsoft's win32 API restrictions on ARM because they affect Firefox. This makes sense because Dotzler works for Mozilla and focuses on Firefox in general, not to mention Microsoft's long history of anticompetitive behavior towards third-party browsers. Make no mistake, though: Limiting access to the win32 APIs is likely to impact many other applications as well. How can LibreOffice or Apache OpenOffice compete with Microsoft Office if they're shut out of the win32 APIs?

In the Name of Malware

Microsoft is getting cut a lot of slack for its anticompetitive stance, because it is casting the anti-features for developers in the name of "protecting users from malware." It's OK if Microsoft cuts off competing applications at the knees, because it's trying to prevent malware.

Leaving aside Microsoft's intentions - perhaps it truly is motivated only by the best interests of users - this argument fails on a number of levels. First, it assumes that Microsoft's own applications won't be exploitable. Given Microsoft's history with security, this isn't likely. Why does Microsoft get the assumption of secure applications, while third parties do not?

And let's not forget who got us to this juncture in the first place. Microsoft users have been worn down by more than a decade of security issues that trace back to Microsoft itself. Microsoft is essentially using its own failings to excuse its blocking of third-party apps that may well have better security than its own applications.

Sandboxing third-party apps into limited parts of the machine does nothing to ensure that Microsoft's own browser won't be ownable by malware. Since Internet Explorer code isn't open source, security researchers can't audit the code directly. Firefox, which can be independently audited, won't be available on the new ARM tablets.

Why Not Complain About Apple?

Some folks have tried to dismiss complaints about Microsoft's ARM policies by pointing at Apple. Since Apple also discriminates against developers on iOS, why shouldn't Microsoft?

Yes, Apple's iOS developer policies suck, but they've sucked since the operating system's inception. What's more, there's little chance that Apple is going to change its policies unless users start abandoning iOS, or there's some sort of legal interference. Given that it'd be hard to make a case that Apple has a monopoly, legal interference seems unlikely.

That doesn't mean that third parties should just shrug their shoulders and accept the same treatment from Microsoft. If Microsoft is successful in the tablet market, ceding the Windows 8 ARM tablets is going to be a big loss for third parties. Loss of one platform is difficult, but being shut out of two tablet platforms in a three-horse race is going to spell major problems for Mozilla.

Dotzler distinquishes between a tablet OS and a general-purpose OS, though. Right now, at least, iOS is just for phones and tablets. Firefox can still compete with Safari on Mac OS X. Whether the distinction really makes sense, I'm not sure, given iOS' dominance on tablets so far.

But Windows 8 is not tablet-only. As Dotzler points out, tablets may be a "tiny sliver" of the PC universe now, but if you're looking ahead, that's not going to be the case in a few years. "ARM will be migrating to laptop PCs and all-in-one PCs very quickly," he says. "If you read Microsoft's blog posts about Windows on ARM, you'll see that they expect ARM PCs to cover the whole spectrum. ARM chips are already being used in servers. This is not a tablet-only concern."

Tablets and Tightening

If trends continue, users will do much more computing on their tablets and phones in the future. Even if this was a tablet-only issue, it'd be worth standing against. The amount of lockdown being exhibited on tablets is troubling, to say the least.

Giving Microsoft (or Apple) so much control over what applications run on their platforms is not good for developers or users. It should be assumed that users have control over their computing devices, and that means having the option to choose their own applications for Web browsing and everything else.

It's not at all puzzling that Mozilla is complaining about being shut out of Windows 8 tablets. What's puzzling is how many developers and industry pundits are willing to give Microsoft a pass.

The addition of .Net to Beanstalk follows PHP support added in March. (Beanstalk's first language was Java.) Amazon is letting developers use Elastic Beanstalk for .Net applications, using Windows Server 2008 R2 and IIS 7.5. Amazon says that Beanstalk will support "any Visual Studio Web project... including ASP.NET MVC projects and ASP.NET Web Forms." This makes it simpler to deploy .Net apps across Amazon Web Services (AWS), and (as with PHP and Java) developers pay nothing for Elastic Beanstalk itself - just for the metered services they use running the applications.

Speaking of Visual Studio, Amazon provides a toolkit for Visual Studio that has been updated to allow developers to publish straight to AWS.

Naturally, if you're deploying .Net apps, you probably want to have Microsoft's SQL Server as well. Amazon has added support for multiple editions of SQL Server 2008 R2. Support for SQL Server 2012 should be coming later this year.

Microsoft's version of cloud SQL is "based on SQL Server," but does not require users to pick a specific version of SQL Server. Whether that's a good or bad thing depends on whether developers are looking for features specific to, say, SQL Server 2008 versus SQL Server 2012.

Ever-Expanding AWS

Whether or not Beanstalk constitutes a Platform as a Service, or PaaS (and I still say it does), Amazon continues to branch out at an impressive pace.

This expansion puts AWS even more squarely in competition with Microsoft's Azure and SQL Azure services, though there's some notable differences in how they're set up.

Amazon's pricing is a lot more complex than Microsoft's for SQL Azure. The SQL Azure pricing is based on size of the database. Microsoft offers two types of databases, Web and Business. The price is based on the size of the database, and starts at a flat $4.95 (just call it $5, Microsoft!) per month up to 100MB. After databases reach 1GB in size, it's billed by size, per day. Microsoft does make it a bit complicated by doing a per-day variable calculation. Azure also has bandwidth costs, but they're not specific to SQL Azure. See Microsoft's pricing page for all the details.

Amazon, on the other hand, charges by the size of the database and the size of the SQL Server instance in use, plus data storage, plus I/O rate, plus data transfer out. Amazon's storage rate is much lower ($0.10/GB per month), but you're also paying for the SQL Server instance and more.

If your shop already has SQL Server licenses, Amazon offers Bring Your Own License (BYOL) pricing, too. There's also the option of backup storage, and developers can choose reserved instances at lower prices if the services they're using will run long-term.

Playing with the pricing calculators for both services, it looks like Amazon does well if you're using Express or Web but gets pricey if you add the Standard SQL Server license. A large DB instance runs $0.59 per hour with the Web license but $1.07 per hour using the Standard license.

For developers interested only in Microsoft services, Azure is probably the way to go. But for organizations using .Net and SQL Server as just a portion of their stack, AWS is now the obvious choice.

Data transfer speeds have been getting faster and faster, but that doesn't mean that we're actually reaping the full benefits. A few years ago, Jim Gettys put his finger on the "criminal mastermind" behind poor networking performance. Dubbed Bufferbloat, the problem was not a simple one to solve. Not simple, but Controlled Delay (CoDel) active queue management (AQM) may provide serious progress towards a solution.

The problem, in a nutshell, is that TCP was not designed with today's bandwidth in mind. As Gettys wrote in "the criminal mastermind" post, the problem lies "end-to-end" in applications, operating systems and home networks. Buffering is necessary, but too much buffering is a problem. And today's devices and operating systems are doing too much buffering - which is degrading performance. Says Gettys, "TCP attempts to run a link as fast as it can, any bulk data transfer will cause a modern TCP to open its window continually, and the standing queue grows the longer a connection runs at full bandwidth, continually adding delay unless a AQM is present."

The Linux Tips page on the Bufferbloat wiki highlights the scope of the problem. According to the page, buffers can "hide" in the operating system layer (Linux transmit queue), device driver, hardware (which has buffers of its own), and on and on. One of the long-term solutions to bufferbloat is active queue management (AQM), and the Controlled Delay (CoDel) AQM proposed by Kathleen Nichols and Van Jacobson might be a big piece of the puzzle.

CoDel To the Rescue?

CoDel (pronounced "coddle"), is being called a "no-knobs" AQM. That means users and admins aren't expected to tweak any parameters to get best performance out of CoDel. According to the paper published on ACM Queue, "CoDel’s algorithm is not based on queue size, queue-size averages, queue-size thresholds, rate measurements, link utilization, drop rate or queue occupancy time. Starting from Van Jacobson's 2006 insight (PDF), we used the local minimum queue as a more accurate and robust measure of standing queue."

More importantly, CoDel promises to distinguish between "good" queues and "bad" queues. It's supposed to minimize delay without hampering bursts of traffic. "The core of the bufferbloat-detection problem is separating good from bad ... good queue is occupancy that goes away in about one RTT (round-trip time); bad queue persists for several RTTs. An easy, robust way to separate the two is to take the minimum of the queue length over a sliding time window that's longer than the nominal RTT."

Finally, CoDel is supposed to be suitable for deployment in a wide range of devices. The paper says CoDel is "simple and efficient," and can be deployed in low-end devices or "high-end commercial router silicon."

Proof is in the Pudding, er, Deployment

A fair amount of testing has been done on CoDel, but the proof has to come via real-world deployments. According to the paper, Nichols and Jacobson performed "several thousand simulation runs" that showed CoDel "performed very well" with results "compelling enough to move on to the next step of extensive real-world testing in Linux-based routers."

Note that deploying this in a home-based router may not be enough to rid yourself of bufferbloat. The researchers point out "a savvy user could be tempted to deploy CoDel through a CeroWrt-enabled edge router to make bufferbloat disappear. Unfortunately, large buffers are not always located where they can be managed but can be ubiquitous and hidden. Examples include consumer-edge routers connected to cable modems and wireless access points with ring buffers. Many users access the Internet through a cable modem with varying upstream link speeds. ... The modem's buffers are at the fast-to-slow transition, and that's where queues will build up: inside a sealed device outside of user control."

So, don't expect CoDel to solve the bufferbloat problem overnight, or even by the end of the year. Gettys says that "work to integrate adaptive AQM algorithms into wireless systems work will take months or years, rather than the week that initial CoDel prototype implementation for Ethernet took. But at least much testing of the CoDel algorithm, experimentation, and refinement can now take place."

There's also the matter of tackling wireless, which Gettys says "may be much more difficult, both because queuing is sometimes much more complex than Ethernet, but also since packet aggregation has resulted in OS/driver boundaries hiding information that is necessary for proper functioning."

But CoDel is still very good news, and shows that the community that's come together around the bufferbloat problem is making progress. Bufferbloat won't be solved in one fell swoop by a single breakthrough, but with a number of technology improvements over time.

(Lead image provided courtesy of Jeremiah Ro via Flickr, under the Attribution-ShareAlike 2.0 Generic (CC BY-SA 2.0) license.)

Having data available electronically is not the same thing as the data being useful. Campaign finance disclosures provided electronically by the Federal Elections Commission (FEC), are a good example of that. The New York Times's Fech (not "fetch") is a RubyGem - a packaged application - designed to help journalists and public interest organizations access and make sense of FEC filings.

 Here's the NY Times' description of Fech from its first release last year:

Journalists who work with these filings need to extract their data from complex text files that can reach hundreds of megabytes. Turning a new set into usable data involves using the F.E.C.'s data dictionaries to match all the fields to their positions in the data. But the available fields have changed over time, and subsequent versions don't always match up. For example, finding a committee's total operating expenses in version 7 means knowing to look in column 52 of the “F3P” line. It used to be found at column 50 in version 6, and at column 44 in version 5. To make this process faster, my co-intern Evan Carmi and I created a library to do that matching automatically.

Fech (think “F.E.C.h,” say “fetch”), is a Ruby gem that abstracts away any need to map data points to their meanings by hand. When you give Fech a filing, it checks to see which version of the F.E.C.'s software generated it. Then, when you ask for a field like “total operating expenses,” Fech knows how to retrieve the proper value, no matter where in the filing that particular software version stores it.

Derek Willis of the NY Times announced the 1.0 release of Fech last month. This release covers "all of the current form types that candidates and committees submit." Perhaps most importantly, this release allows comparing two filings against one another.

Why Fech Matters

Fech is already being used by the NYT for its reporting and interactive visualizations of campaign spending. But that's just one editorial team. Putting this tool in the hands of any developer or reporter that wants to work with the data opens a lot more possibilities.

For example, there's ProPublica, which is using Fech and the NY Times' APIs for its reporting and interactive graphics. ProPublica is able to show not just what campaigns are spending, but how much and with whom. (So far the biggest winner is Mentzer Media Services, an ad agency that specializes in GOP campaigns - including the Swift Boaters. Fech doesn't automatically point that out, of course, but it helps journalists uncover it.

Data without context is useless. By helping developers and journalists work with the filings in a more structured way, Fech helps newsrooms (or any other group) put the data in context to find the story behind the data. It's a long way from being simple to use, but it represents a significant improvement over the raw data. It's Apache-licensed, so it might find its way into all kinds of data analysis tools over time.

With Fech maturing well before the elections this fall, it could help all kinds of organizations follow the money trails much more efficiently. Here's hoping that happens.

If you've wondered why so many companies are eager to control data storage, the answer can be summed up in a simple term: data gravity. Ultimately, where data is determines where the money is. Services and applications are nothing without it.

Data gravity is a term coined in a blog post by Dave McCrory. Basically, McCrory says to consider data as if it were an object:

As Data accumulates (builds mass) there is a greater likelihood that additional Services and Applications will be attracted to this data. This is the same effect Gravity has on objects around a planet. As the mass or density increases, so does the strength of gravitational pull. As things get closer to the mass, they accelerate toward the mass at an increasingly faster velocity.

...Services and applications can have their own gravity, but data is the most massive and dense, therefore it has the most gravity. Data if large enough can be virtually impossible to move.

Later, McCrory's post went on to talk about artificial influences on data gravity, such as costs, data throttling, legislation and more. Basically, factors that influence the movement of data in ways that wouldn't happen "naturally." For instance, Amazon allows free inbound data transfer, but charges for outbound data transfer. Another "artificial" influence is legislation, telling companies where they may or may not store data, or dictating terms of its storage.

Data Gravity in Action

You don't have to look very far to see data gravity in action. Consider Dropbox, Amazon S3, iTunes or just about any CMS migration ever.

Lots of companies want to emulate Dropbox, but few have managed to attract the same kind of user base as Dropbox. None are as ubiquitous as Dropbox. And that presence is paying off for Dropbox, which has now attracted quite a few third-party apps to its orbit, like Wappwolf and Ifttt. Perhaps that's why Apple is trying to disrupt Dropbox's gravitational pull and rejecting some iOS apps that use Dropbox.

You'll note that Amazon S3 and other Amazon AWS services make it very easy to get data in, but getting data out gets spendy. No shocker here - Amazon wants to encourage as many developers and companies to toss data into AWS, and then tie them to the service.

Apple's iTunes is all about keeping data in Apple's services. Aside from Apple's now-defunct DRM on music, there's no using iTunes to transfer music or movies to other devices. It's Apple devices or nothing. Getting the entire library out of iTunes is non-trivial for many users, so in many cases it's like a digital roach motel: data checks in, but it doesn't check out.

If you've ever worked with content management systems, you already know all about the concept data gravity - even if you've never heard the term. Getting all the data out of one CMS to another is, well, painful at best. Often impossible. This is one reason why companies often stick with aging CMSes rather than go through the pain of migration. 

Consider Gravity Before Deploying

Whether it's a single-user application like iTunes, or a company wide project: You need to consider the implications of data gravity - once your data is in, how hard will it be to break the gravitational field?

The stronger the data gravity involved, the more cautious you should be when you choose your data storage solution. It's likely that once you have a sufficient amount of data wrapped up in a solution, it's going to be very difficult (if not impossible) to justify the costs of moving it away.

(Lead image courtesy of Flickr User Juan Ramon Rodriguez Sosa under the Attribution-ShareAlike 2.0 Generic (CC BY-SA 2.0) license.)

With all the cloud storage competition coming out of the woodwork, how does Ceph distinguish itself? It started as a doctoral project by Sage Weil, at UC Santa Cruz back in 2004. The question is whether Weil's academic project has evolved enough to take a chunk of the storage market?

All About Ceph

Ross Turk, Inktank's VP of community, says that Ceph is designed for "multiple data storage needs with a unified storage platform." Some storage offerings, for example, provide object storage, but not block storage. Some solutions might provide object and block storage, but aren't POSIX-compatible.

Turk says that Ceph does all of the above. "It provides object storage, similar to Amazon's S3 and compatible with apps written for Amazon S3. It provides the kind of block device storage necessary for VM images, thinly provisioned and striped across the entire storage cluster. Finally, it provides a bottleneck-free, POSIX-compliant network filesystem. It does all of that on top of a single storage cluster, so it's not necessary to have separate clusters for each different storage need."

Just as importantly, Turk says that Ceph's distributed capabilities make it a compelling option for companies looking to fill big storage needs. First, Ceph's placement algorithm ("CRUSH") "is far more intelligent than anything out there today," Turk says. "It allows storage clients to calculate the location of data within the cluster, instead of having to look it up somewhere, and it does it while allowing for robust placement rules."

Ceph CRUSH!

CRUSH, says Turk, lets admins tell a Ceph storage system just how many replicas of data are required, and how they should be distributed "across the various nodes, racks, rows and rooms in their data center."

The second part of Ceph's distributed features is that it's "self-managing" and "self-healing." This means that Ceph's daemons "constantly work with each other to balance data throughout the cluster as the conditions within the cluster change."

This makes Ceph particularly attractive for companies that want to use commodity hardware, where failure is the norm not the exception. What type of companies does Turk see adopting Ceph? He says Inktank is seeing service providers, enterprises and organizations doing High Performance Computing (HPC). "We are also partnering with the Big Data community, various Linux distributions, and a number of platforms, including all of the Cloud stacks and hosting automation platforms."

It'll be interesting to see who Ceph's Linux partners are. Red Hat is busy pushing GlusterFS since it acquired Gluster last year. Ceph is also competing with S3-compatible storage layers provided by open source IaaS stacks, like Eucalyptus' Walrus and OpenStack's Swift, at least in some use cases. There's also RiakCS, which is a proprietary add-on to the open source Riak.

With all the money flowing into storage right now, there's room for plenty of players - especially since they each have features that appeal to specific use cases. But it's a lot of homework for IT managers looking to pick the right solution to standardize on. If you're working with cloud deployments or putting together storage solutions, which options are you looking at?

The inexpensive Kindle Fire took off when it was introduced in late 2011, giving Amazon 16.8% of the tablet market with 4.8 million shipments. Amazon's 7-inch tablet was the right product at the right price at the right time, that being the all-important holiday season. The Fire is an inexpensive tablet that offered many of the features that people want for less than half the price of an iPad. But the Fire didn't knock people's socks off, and many of the reviews were lukewarm, at best.

Amazon Still Trounces B&N

Apparently, the bloom is off the rose. Amazon's Q1 sales put it behind Samsung sales of Android tablets, but still comfortably ahead of Barnes & Noble's Nook tablets. Lenovo took the fourth slot, while B&N grabbed fifth place.

IDC predicts that Amazon will try to win back market share with the introduction of a "new larger-screened device... at a typically aggressive price point." IDC's Tom Mainelli, research director for IDC's Mobile Connected Devices group, also predicts Google will debut a tablet co-branded with ASUS.

Lessons Learned: Price Matters, to a Point

The lesson that Apple's tablet competitors should take from IDC's research is that price does drive sales - up to a point. The drop-off from the last quarter of 2011 to the first quarter of 2012 is far steeper than is easily explained by the end of holiday shopping. IDC had predicted overall tablet sales to be 1.2 million units higher than they were this quarter, with the shortfall mostly attributed to Amazon's slip.

Even though Apple introduced a new iPad this year, it's continuing to sell iPad 2s at a reduced price, fending off the cheap Android tablets and defending its market share while maintaining high margins on the rest of the iPad line. Apple owned 54.7% of the market in Q4 2011, and has bumped that figure back up to 68% in Q1 2012.

Tablet sales have grown 120% from last year, but were still lower than IDC's predictions. Whether tablet sales continue to slow in Q4 will be interesting to see.

Android Still Lags

Android vendor sales have not been able to catch up to iOS in the tablet market in the same way that they've been able to catch up with iOS phones. Despite two years and a slew of vendors chasing Apple's tail, Android has been able to glom on to only 32% of tablet sales in the last quarter. Android's fragmentation, poor customer reviews and pricing missteps have held it back.

It had looked like Amazon's Kindle Fire would be the breakout device for Android. And it was - briefly. But even Amazon's muscle and cut-rate pricing hasn't been enough to overtake the iPad.

POPVOX: Bridging the Public and Congress

Tracking bills through Congress can be complicated, to say the least. Giving elected officials feedback, and making sure it's heard, is even more so. POPVOX was founded in 2010 as an attempt to help voters and Congress by making it easy to find bills, voice support or opposition to legislation, and share opinions. But don't look to POPVOX for its opinions - one of the site's goals is to be free of editorializing.

POPVOX tracks all of the bills in Congress, and how members vote. If you sign up and give POPVOX your information, it will help you track how your representative and senators vote on bills before Congress. You also get to see whether other POPVOX users support or oppose the bills, with handy little pie charts that show support and opposition, as well as how many users have spoken.

The bill summary pages also list organizations that endorse and oppose the bill, as well as the administration's stance on a bill. Naturally, the site also includes the text of the bill and its status before Congress.

POPVOX is supposed to provide a more effective way to read public sentiment on bills and get feedback on them. If POPVOX takes off, maybe it can counter the influence of paid lobbyists in favor of the public.

OpenCongress

While OpenCongress does not avoid editorializing, it's still a fantastic tool for paying attention to Congress. The site is a project between the Sunlight Foundation and Participatory Politics.

One tool that you'll find on OpenCongress that's not available via POPVOX is a way to track your representative and senators specifically. OpenCongress shows how often they vote with their party, their votes and their money trail. OpenCongress even lets you pit legislators against one another by comparing their voting records.

OpenCongress also lets you follow the money trails by industry sector, so you can track things like pro-gun and gun-control spending, how the entertainment industry spends money, and so on. If you want to track specific issues, there's an index of broader issues as well. This shows "hot bills" by the issue area, key votes, the latest bills and enacted bills.

The committee view has potential, although it looks like this is an underloved section of the site. Several committees have no membership data, though the site promises that "it's coming soon in August 2009." Keeping up with Congress isn't easy, though, and on the whole, the site provides a fantastic resource.

Poligraft

The concept behind Poligraft is simple, but extremely complicated to pull off. Give it the text to an article, press release or post from a blog, and it will give you an "enhanced view" of the people, organizations and their relationships.

Give it the URL to a political story, and it will filter the story for points of influence, campaign donations and individuals in the story. It tries to show where money goes and where it comes from, in relation to any given story. For example, this story from Politico points out when donors and recipients are mentioned in the same story - like Goldman Sachs and James Walsh. It helps give context, for instance, when politicians are talking about organizations that may be giving them or their opponents money.

Follow the Money

Money talks in Congress - loudly. Finding out who's spending what, and how, can be pretty difficult, especially with the explosion of super PACs. The Sunlight Foundation's Reporting Group provides a handy site called Follow the Unlimited Money for all super PACs that have raised at least $10,000 since the beginning of 2011.

Unsatisfied with the views of super PAC data provided by the site? No problem: You can download the data and work with it yourself. The data is provided as CSV files.

OpenSecrets.org

The OpenSecrets.org site is a treasure trove of information for tracking the influence of money on U.S. politics. The use of big data in tracking government is trendy now, but OpenSecrets.org was well ahead of the curve. The Center for Responsive Politics has been publishing since 1983, and the Web site has been up since 1996.

OpenSecrets.org goes a bit beyond reading the tea leaves of big data. It also does good old-fashioned reporting and finds a lot of information you might otherwise miss. One of my favorite projects on the site is the revolving door, which tracks former members of Congress and staffers, so you can see where they go when they leave Congress. OpenSecrets.org also lets you check the top lobbying firms and see who hires folks who used to work on the Hill. See also the Sunlight Foundation's lobbying tracker if you're into paying attention to lobbyists.

There's much, much more. OpenSecrets.org will help you find out about the finances of members of Congress, see how earmarks are distributed or let you drill down into local contributions.

Federal Register

Want to see what executive orders are coming from the White House, or rules being proposed by federal agencies? Then you'll want to take a look at the Federal Register. The U.S. government posts notices, proposed rules, rules taking effect and "significant documents" for public inspection.

There's a lot of information on the site, but it's not as easy to use or friendly as some of the sites that are provided by organizations like the Sunlight Foundation. You really have to know what you're looking for here to be able to find out what's going on. However, the Federal Register does have an API, and code for the site is provided on GitHub, so everything is there for third parties to take on examining and taming the data.

MuckRock

Ever thought about filing a Freedom of Information Act (FOIA) request? The folks over at MuckRock have. In fact, they've filed more than 1,000 requests and received more than 30,000 pages of government documents. Out of all of those requests, only 273 have been "successfully completed" and 85 have been denied - meaning there's still some wiggle room for the government to just ignore requests or delay them significantly. Take, for example, this request for FOIA filings in Boston. It's gone unanswered for nearly two years.

Still, the MuckRock folks are turning up interesting information and showing others how it's done.

If you know where to look, you can find out much more about what's going on in government these days - thanks to the series of tubes we call the Internet. But there's always room for more information and better efforts to put that information in context. Have a favorite open government site? Let us know in the comments. And yes, we know this is U.S.-centric. Think we should try to pull together a list of international open government resources? Let us know.

The Data Journalism Handbook effort started at a workshop at the London MozFest 2011 last November. From there, the handbook represents the work of "an international, collaborative effort involving dozens of data journalism's leading advocates and best practitioners." This includes folks from ProPublica, The Washington Post, the BBC, The New York Times and many others.

The result, so far, is an online book that's just now in beta. Eventually it will also be published in dead tree and e-book form by O'Reilly. However, given the nature of the tome, it's most useful online. As you'd expect from a title that was born at a Mozilla conference, the text is full of links to online resources. I suspect trying to read the title as an e-book - or especially on paper - would be a little frustrating.

Inside the Handbook

The handbook offers a glimpse into the practice of data journalism, with some guidance on how to get started. You'll find a slew of case studies, along with sections on getting data, understanding data and delivering data to the public.

The handbook covers topics like open data, data use rights, scraping and crowd-sourcing data, and community engagement. You'll also find some high-level discussion of tools to work with open data, and how to get that data. 

Most importantly, the book offers a resounding case for data-driven journalism. The case studies demonstrate the utility of data-driven journalism and the service that it offers the public. For instance, the OpenSpending.org example should inspire any journalist that covers politics and public funds. The Price of Water case study shows not only the service to the public, but the service of the public in gathering data.

The handbook is not a comprehensive guide to all of the concepts and skills that a journalist needs to practice data journalism. It doesn't teach the skills necessary for data literacy, though it does provide some links to resources. It also, of course, explains the importance of data literacy. But it certainly doesn't try to teach journalists how to program and make use of APIs, or how to use tools to create data visualizations.

In short, it's not Big Data for Journalists or even Programming 101 for Journalists, and more's the pity. Programming and working with data sets is a skill set that many journalists would do well to have, but most don't. To be fair, the handbook doesn't necessarily advocate that journalists be programmers. It does emphasize being able to work well with programmers, but it would probably be a very good idea to have at least a fair grasp of basic programming. 

Tips and Ideas

If you read just part of the handbook, I'd recommend skipping the case studies and going straight to the meat of the book. Specifically, the sections on getting data, understanding data and delivering data. 

For example, "Become Data Literate in 3 Simple Steps." This piece advises journalists, at a high level, how to approach data. Ask yourself how the data was collected and if it can be tested. Don't assume that data handed to you by a source is going to be valid. (And if the data is not valid, it may be a story, or it may defeat the premise of the story.)  Question the data, how it was gathered and whether it's a reliable sample. You see, for instance, many "trend" stories about technology based on a single data set. You may not have a large enough sample size to rely on.

The section on visualizing data is also useful. The handbook recommends that reporters working with data find a way to visualize it, even if that's just pulling numbers into a spreadsheet. Visualizing data allows you to find patterns that you might otherwise miss.

In the enthusiasm for working with data, scraping websites or gathering data in other ways, there's also the small matter of legal restrictions. Whose data is it, and do you have the right to distribute it? The "Using and Sharing Data" section advises reporters to consider the ownership and licensing of data, and when "database rights" might mean that you can't distribute a data set in its entirety. It also covers various open-data licenses and recommends that news organizations apply those when distributing homegrown data sets.

An Unevenly Distributed Future

What the handbook also does, sadly, is provide a tantalizing picture of what is, and what should be. As William Gibson said, "the future is already here - it's just not very evenly distributed." The same can be said for data journalism. We have marvelous tools for doing data journalism, and they're getting better all the time. In some newsrooms, journalists are producing solid work with in-house or open-source tools, examining everything from public data sets to data curated in-house.

In most newsrooms, however, reporting has not yet been significatnly affected by data journalism. In an era of continual layoffs and cutbacks, there's no budget for training or tools to help reporters get up to speed with the necessary tools and practices. Most of the case studies describe projects that take weeks or months, a depressing concept for journalists tasked with writing several stories per day.

There's a deep need for the handbook, and a sequel or two that dive deep into the actual practice of data-driven journalism. (To my friends at O'Reilly, a "programming for journalists" book would be a nifty title.) It's inspiring and educational material, if less focused on "how-to" than one might like.

Data-driven journalism is in its infancy right now, despite the amount of discussion it's generating. I suspect that it's going to be five to 10 years before we'll see the practices in the handbook becoming mainstream.

Image from the Data Journalism Handbook, which is available under the Creative Commons Attribution-ShareAlike 3.0 Unported (CC BY-SA 3.0) in its entirety.

The hostname decline, according to Netcraft, is due to more than 28 million .info hostnames that were controlled by SoftLayer going into oblivion. The drop was enough to offset new growth, in a month in which Apache lost more than 17 million domains.

Netcraft looks at more than just the total domains, of course. It also measures the million busiest sites and the active sites - which helps to get a view into the Web servers that are actually being used for live sites, as opposed to the parked domains that make up the bulk of the Internet.

SPDY and IIS 8.0

Netcraft's survey has also picked up on some cutting-edge tech out there, in very small numbers. Netcraft spotted 654 hostnames being powered by Microsoft IIS 8.0, which is the Web server in Microsoft's Windows Server 2012. It'd be interesting to know how that compares with servers running Apache 2.4x, which was released in February but is still in the early stages of adoption.

Even fewer servers are running SPDY. Netcraft spotted a whopping 339 servers running SPDY, which is mostly Google and a handful of other sites. SPDY usage is likely to increase if and when Apache and Nginx have bundled support for it. You can get a module to use SPDY with Apache now, but it's not distributed with the official project. Nginx isn't expected to have SPDY support until later this month.

Nginx Still on the Rise

Once again, Nginx increased its share of the million busiest sites, but only by a hair. In April, the up-and-coming Web server had 100,394 domains responding to the Netcraft survey. In May, it nudged up to 100,417, maintaining its 10.09% share of the market.

Nginx's share of the active sites actually dropped a bit. In April, it had about 24.3 million. In May, Nginx only had about 23.9 million, which gave it a -0.27% drop in share. Apache increased here, from about 107.7 million to about 109.3 million, or a 0.36% boost to 57.02% of the active servers.

Microsoft also lost servers in May. Microsoft IIS now has 11.9% of the active servers counted by Netcraft, and 14.76% of the top million domains. It might not be long before Microsoft IIS slips to number three behind Nginx and Apache. But it doesn't look like Microsoft is losing a lot of sites because customers are switching; rather, it appears that IIS is falling behind because it's not being deployed on new servers. 

What remains to be seen is whether Nginx can put a serious dent in Apache, or if it's always going to be a distant second. Apache still powers the majority of Web servers, and it has managed to beat back IIS pretty handily.

Google's BigQuery is a Platform-as-a-Service (PaaS) for working with "massive datasets" that can be in the billions of rows. It has a SQL-like query language, and promises to analyze large data sets "in seconds." Note that organizations that want a Google-hosted SQL database can tap the Cloud SQL offering.

Commodity Big Data

What's most interesting about BigQuery is the fact that it provides big-data analytics in a completely hosted offering. Organization's don't have to build out the hardware for a big-data infrastructure. They don't need to worry about setting up Hadoop or any other software. It's big data available instantly, and at a fairly affordable price.

Google is charging by storage and by the queries processed. The storage is priced at $0.12 per GB per month, up to 2TB. This is, more or less, the same as you get from Google Cloud Storage, except there's no drop in price after the 1TB tier.

Queries are priced at $0.035/GB processed, with a limit of 1,000 queries per day and 20TB of data processed per day. Note that this is after the 100GB/month free tier. And you're charged only for the data processed in a column of the data, not for the entire table.

To work with data, you have three options: a browser-based query tool, a Python command-line tool, and a REST API.

As an off-the-shelf service, it's going to be a bit less flexible than what developers could get out of a tool built with Hadoop, Hive, etc. However, it is likely to be effective for quite a few organizations and developers who need big-data tools quickly and can work within the limitations of BigQuery. Data journalists, for example, might find BigQuery quite useful in working with home-grown data sets rather than having to build out their own query tools.

There's also the matter of data privacy. The Terms of Service (ToS) give each party full control of their own intellectual property - so Google should have no rights to the data being studied using BigQuery. Nevertheless, any organization that wants to keep its data private is going to think twice before putting it into Google's BigQuery.

Flexibility and privacy aside, this is going to fill a niche very handily. There's a lot of nonsensitive data that developers might want to crunch, without having to create their own big-data toolset. BigQuery looks like a decent solution for situations when a commodity tool will fit the bill.