As cloud computing services become ever more popular, you might begin to wonder how much you can really trust them to perform when you need them? I decided to find out - by testing the top file-transfer/file-storage/file-backup services.

In many ways, getting a file from one computer to multiple computers is the most challenging task for the cloud. And because I like to use multiple computers running multiple operating systems, including Linux, Windows and the Mac, that function is particularly important to me.

Cloud Services Can Lag

I am pretty agnostic when it comes to cloud providers - as long as they are free or close to it. However, as I was moving files around while preparing my most recent book A Week at the Beach The 2013 Emerald Isle Travel Guide I was a little surprised at the lags I sometimes experienced using the big-name cloud-based file-transfer services.

More than once when I wanted to use a file from one computer to another, I was disappointed by my cloud services. There were a few times that I got so tired of waiting for a file to show up on my other computer’s cloud drive that I resorted to sneakernet using a USB thumb drive.

After my book was published, I decided to go back and run some simple tests to see just how long the four best-known file-transfer/backup services actually take to put the files where you want them.

To compare Dropbox, Google Drive, Amazon Cloud, and Microsoft’s SkyDrive I started by exporting a 500K JPEG test image from Lightroom on my Windows 8 computer directly to each of the four services.

Fighting The Randomization Factor

After running the tests a few times, I noticed what can only be described as random operating system differences. Sometimes the file would pop up first on my Mac and other times it showed up first on my Windows 7 laptop.

In order to eliminate the operating system differences, I restarted the tests and this time stopped the timer when the file showed up on either my Mac running Mountain Lion or my Windows 7 laptop. I also reran my tests with a variety of sizes and types of files. In all I ran twenty-five sets of tests.

The differences were significant, if not overwhelmingly huge. The fastest synchs took less than 3 seconds, while a few others took several minutes. The biggest chunk of tests clocked in between 10 seconds and one minute. A few synchs never completed. But which service recorded the best times with the fewest problems?

Dropbox FTW!

Dropbox ended up being fastest 56% of the time. Even more importantly, it was slowest only 4% of the time.

Skydrive brought up the rear. It was fastest on 12% of the tests, but but slowest on a whopping 80% of the tests. It also had two files that never showed up on the Mac and one that never showed on the Windows 7 laptop.

The Amazon Cloud slightly outpaced Google Drive - which had one file that never showed up on the Mac and another that took a very long time to complete.

If my tests convinced me of anything, it is that Skydrive is a work in progress and has a long way to go. I even had trouble setting up the tests on Skydrive.

My tests also revealed a number of odd results. When testing files saved from Word, strange extra files sometimes showed up on all the cloud drives except Dropbox. The file names always began with the characters “~$”. Sometimes the mystery files disappeared and sometimes they hung around.

Cloud Drive Recommendations

So here are some quick recommendations:

• First, do not treat your cloud drive as one huge dumping ground. Create folders and try to force a little organization on yourself.
• If you save a file to the cloud in order to work on it from another computer, quit the application or close the file on the first computer after you have saved the file to the cloud drive.
• Make sure you have a local copy of important files in your documents folder - not just the replicated cloud folder on your computer. Interesting things sometimes happen when cloud files get updated or deleted from another computer. When you come back to the computer where you first created a file, you could be in for a nasty surprise.
• If you cannot get a cloud folder on your computer to update, trying quitting the cloud application or rebooting your system.

Dropbox and Amazon appear to be the most reliable solutions with only occasional delays. Google isn't far behind, and I can't imagine that Microsoft won't work hard to improve Skydrive - the company's subscription model depends on it.

Even so, I have no plans to throw away my USB thumb drives.

The world of online services may be convenient, but there is always a risk of such services doing something your locally installed software won't: drop off the face of the Earth at a moment's notice.

That is the scenario OfficeDrop users are now facing, after receiving word last Friday that the five-year-old online collaboration and storage service would no longer be around after May 5.

"We know that many of you have come to depend on OfficeDrop to store and share your files in the cloud," a notice to users from OfficeDrop CEO Prasad Thammineni read, "Regrettably, we will not be able to offer OfficeDrop as a standalone service anymore and will be discontinuing it permanently as of May 5th, 2013."

The wording of the notice was specific: OfficeDrop as a company is not going away, just the service to its users. After reaching out to Thammineni over the weekend, he explained the situation.

"Recently we signed an agreement to sell OfficeDrop to another Cloud Storage player. We will be closing soon and will announce more details as to who the acquirer is and other related information," Thammineni responded to inquiries about the nature of the service's ending.

"Unfortunately, [the] acquirer has decided to not continue offering the OfficeDrop service and instead will integrate the product, technology and team with theirs and market it through their channels," he added.

According to the notice sent to OfficeDrop users, billing for OfficeDrop has stopped, and refunds are being issues when applicable. The notice urges all files be downloaded as soon as possible, because "they will be securely and permanently deleted as of May 6th, 2013."

What OfficeDrop Users Will Be Missing

OfficeDrop has seen some critical success over the years, with many reviewers appreciating the capability of iOS and Android apps that could take a picture of a document and then save a file in the cloud as a searchable PDF using optical character recognition. Think Evernote, with big side of Dropbox, and you get the idea.

The service has been aimed at small- to medium-sized businesses (SMBs), but this experience may leave a poor taste in OfficeDrop users' mouths. Given the just 16-day notice they received and the hard stop for their files' existence on the cloud-based service, (a hard stop that falls on a Sunday, which will probably make for a blown weekend for quite a few people), this could put SMBs with potentially limited IT resources in a bind.

After all, downloading the documents will be relatively simple, but finding a place to store them and enable users the same kind of access could be tricky for a local platform. After all, it's the very availability of files that makes the cloud so attractive to customers in the first place. Storage is easy - access is hard.

Vanishing Like A Cloud

In the days before cloud, acquisitions and consolidations happened between software companies all the time. But even when Company A swallowed Company B and discontinued B's software product line, devout users of B's software could still keep using that software - albeit unsupported - until they could figure out what move to make next.

For cloud services, the timing of service shutdowns is no longer decided by the users… it's the service provider who calls the shots.

As online services continue to pop up with some new feature or convenience, it is not unexpected that we will see more of these kinds of consolidations happen over the coming months. And end users will have to bear the brunt of the inevitable service shutdowns that will follow.

Image courtesy of Shutterstock.

Last week, ReadWrite Editor-in-in-Chief Dan Lyons sat down with 27-year-old Box CEO Aaron Levie to discuss the complex market of enterprise cloud technology in the third ReadWrite Mix event in San Francisco.

(See also Aaron Levie On The Uncertain Landscape Of Enterprise Software and Aaron Levie On Finding Mentors & Mixing Enterprise/Consumer Cultures)

In their wide-ranging, hour-long conversation, Lyons and Levie touched on just about every topic of interest in the enterprise startup space. But perhaps the most interesting aspects concerned Levie's own unique story. This 1:53 clip touches on how growing up in Seattle helped inspire Levie to begin starting companies as a teenager (publishing HTML webpages to the Internet was all it took, he says), and how he had to try his hand at countless projects on the way to his dream of starting a successful company. Levie did acknowledge an alternate dream that included playing professional sports: "It's either MLB, or start an Internet company," he joked. 

The enterprise is potentially months away from a significant crisis, a slow-moving malaise that will weaken the efficiency of accessing a company's data and intellectual property. In the worst case scenario, companies could lose information forever. And yet few enterprise IT vendors are paying any attention.

The problem is that employees are busily stashing data all over the place, in all sorts of cloud services, which is leading to what Huddle CEO Alastair Mitchell calls a "fracturing of the enterprise brain."

"This is my massive prediction for 12 months from now," Mitchell says. "When employees are storing stuff in the cloud, and using something like 15 different cloud storage tools to do it, their corporate knowledge, their brain, is destroyed."

To be sure, Mitchell has some skin in the game, since Huddle is an enterprise content-collaboration platform that leverages content to provide collaboration and storage services. It's doing pretty well at it, it seems, landing some big enterprise customers and demonstrating enough security to add the UK Ministry of Justice and the U.S. Department of Homeland Security to the user rolls.

Despite the self-interest, though, Mitchell raises some good points. The problem he highlights is very real.

Who Knows What?

Employees are the keepers of knowledge within a company. Want to run the monthly payroll? The 20-year-veteran in accounting knows how to manage that. Building the new company logo? The superstar designer down in the art department is your gal. When such employees leave the company, it can be a bumpy transition, but usually not impossible, because the data they've been using lies on the corporate file server and can be used to piece together the work that's been done.

Of course, that's based on the premise that, for the past couple of decades or so, data has essentially been stored in one of two places: on the file servers or the employee's local computer.

Today, though, people store data in a variety of places, not all of it under the direct control of IT. Gmail, Dropbox, Google Drive or a company's cloud on Amazon Web Services… all ubiquitous and seemingly innocuous services that make it all too easy for an employee to stick the data for that Q4 fiscal report out in the cloud. Or leave the travel expense report from HR attached to the original email message so you can find it later.

But what happens a couple of years later, when an employee has left and a company audit needs those Q4 2012 report data files? That data is out of IT's reach. A quick call may be able to get that data back, but what happened to that data when it was sitting out in some storage account for those two years? And what if the former employee is unreachable?

The Daily Time-Suck

The problem crops up even without employees leaving the enterprise. On a day-to-day basis, workers themselves increasingly struggle with the problems of finding data. Where did I put that latest template for the expense report? My document folder? In the project folder? In Dropbox? Oh well, I'll just email HR for another copy.

Time wasted, again, and redundant effort from the worker and the people that have to help workers replace data they already have. Yes, this might take just five minutes, but how many times does this happen every day?

Services like Huddle, or on-premise cloud storage services like ownCloud, attempt to avoid the problem by enabling IT to set up easy-to-use storage and collaboration that enterprise IT can keep tabs on at all times. Microsoft SharePoint and Alfresco also offer centralized storage and collaboration, but enterprise social can have problems of its own.

But here's the rub: With the introduction of every new separate social, collaborative or storage service, the problem actually gets worse, because it sets up yet another location for data to live. Without a truly universal service-bridging search tool or the potential to integrate these services, the fracturing of the enterprise brain will only continue.

Image courtesy of Shutterstock.

Last Wednesday, ReadWrite Editor-in-in-Chief Dan Lyons sat down with 27-year-old Box CEO Aaron Levie to discuss the complex market of enterprise cloud technology in the third ReadWrite Mix event in San Francisco.

(See also Box CEO Aaron Levie On The Uncertain Landscape Of Enterprise Software [Video])

In the one-hour sit-down, Lyons and Levie covered surprisingly diverse topics, from Levie's personal life as a notorious workaholic to what it means to sit in the middle of the enterprise and consumer software landscape that has steadily begun to merge. Levie also showcased his sense of humor with descriptions of the companies he started as a teenager (Zazap, the fastest search engine on the Internet - except for Google), and spot-on caricatures of stuffy enterprise types to illustrate how he keeps his company loose and fast-moving. 

This 2:28 video touches on Levie's thoughts on having grown up in the startup landscape with his high school friends, the laughable stereotype of enterprise software culture and seeking out mentors in Silicon Valley (Hint: There's no risk to to sending an email. So what if you don't get a response, he says). 

Last Wednesday, ReadWrite Editor-in-in-Chief Dan Lyons sat down with 27-year-old Box CEO Aaron Levie to discuss the complex market of enterprise cloud technology in the third ReadWrite Mix event in San Francisco.

Lyons and Levie touched on a wide array of subjects, from sly stabs at Microsoft to how Box grew from a dorm room project to the forefront of cloud storage and collaboration. Levie also shared some of his wisdom on how to outmaneuver the competition, understand the early mover conundrum and keep delivering better technology. 

This 2:48 video touches on the uncertainties facing enterprise software companies like Box, as well as the opportunities that lay ahead in a market where, as Levie says, it's not clear where one service starts and another service stops. 

Guest author Ed Lee is lead architect for virtualized storage vendor Tintri.

Virtualization and flash memory are disrupting the staid storage industry. First embraced in slick consumer products like the iPod, flash memory is now the new darling of enterprise IT. One reason is speed.  Flash is more than 400 times faster than rotating disks.

The other reasons are virtualization and cloud computing. Combined, these technology trends have strained the capabilities of traditional storage products. Those big metal boxes of enterprise storage, called arrays, were originally designed in the 1980s - before MC Hammer rocked parachute pants. Today’s input/output storage requirements are heavier and far more random than storage designers could have predicted 30 years ago. And with new technology comes new problems. IT systems that once hummed, “Can’t Touch This,” became hung up by traffic jams. With the highly random I/O of modern virtualization, flash trumps even the best spinning disks.

Well, nothing gets venture capitalists writing checks faster than IT managers with large budgets wringing their hands in frustration. So, enter the flash solutions from enterprise storage startups. Even the incumbent dinosaurs are showing some hustle. Just last spring, storage industry giant EMC's dropped an impressive $400 million to snap up XtremIO, a three-year-old flash company that has yet to ring up a single sale.

Too Much Flash?

So has flash been over-hyped? Yes, of course it has. Most promising new technologies are overhyped, but the best survive the disappointment that is sure to follow. The problem with most of the flash crowd is that they tout it as a cure-all for IT's performance woes – like a magic medicine show, it puts the performance back in your data center. But it's not magic. It does one thing really well - it eliminates contention for disk spindles. But flash by itself does nothing to ease storage management burdens, and in fact may actually contribute to increased infrastructure complexity. 

Like all new technologies, flash-based storage systems need to be designed into complete solutions rather than just point products.   

Speed Isn't Enough

Today, flash is central to nearly every next-generation storage solution coming to market. But smart IT managers know that speed alone doesn’t solve all of their storage problems, especially in a world of virtualization and cloud computing. In fact, flash-based vendors that offer systems that simplify storage management provide a much greater boost to IT performance than just making I/Os go faster.

Like many incremental improvements in component technologies – even order of magnitude performance boosts like flash – are too often hijacked by legacy vendors to create incrementally smaller and faster versions of the same old products. Don’t be blinded by flash. It’s cool, but it needs to be integrated with new approaches to building complete solutions.

Expect the enterprise storage conversation, pushed by the new demands of virtualization and cloud computing, to move beyond flash. It’s not just about the speed; it's about the solution.

Image courtesy of Shutterstock.

One morning this week, at 4:25am Eastern time, I got an email from Amazon:

“Great news! You can now enjoy photos on your Kindle Fire using the Cloud Drive Photos app. Store your photos online and have them at your fingertips on your Fire.”

I had to laugh.

I Was Late To The Party

My first thought was "better late than never." Then I noticed the software had been released back in November, so the joke was actually on me.

Still, Amazon taking a year to release a decent cloud-based photo app had sent me right into the arms of a Nexus 7.

The Kindle Fire I got for my Christmas 2011 gift had been my first adventure into tablets. (I’m hooked on photos - which you might know if you read my post on Photo Finishing Your Memories Old and New.)

As an Android user (see Seniors Should Be Dialing In On Smartphones), I was expecting an App on the Kindle Fire like the “Gallery App” on my LG Spectrum smartphone. After all Amazon is an expert with the cloud so it should know how to store my JPEGs. There was no cloud photo app at the time.

Amazon Cloud Drive Couldn't Figure Out Photos

So I experimented by putting some images on my Amazon Cloud Drive. My Kindle and/or Amazon didn’t seem to know what to do with them. Inevitably when viewing the photos, some were always oriented wrong. Often it was the ones that I really wanted to show.

This was surprising, since Amazon seemed to know exactly what to do with my Word documents. It turned them into books that I could easily page through on my Kindle Fire.

Portfolio: A Third-Party Alternative

It wasn’t long before I discovered Portfolio, a neat $2.99 app for Android that worked with Microsoft’s Sky Drive among other things.

I didn’t really care whose cloud had my photos, I just wanted to show a few images using my Kindle Fire when I visited with friends. Portfolio worked really well for me. Just dropping the photos in a folder on my SkyDrive automatically created a slide show.

When I bought a Nexus 7 this holiday season, I was pleased to get my Gallery App back. Ease of access to my photos, more consistent sharing options and the speed of Chrome on the Nexus 7 relegated my Kindle Fire into a backup tablet that I keep in my upstairs office.

Amazon Cloud Drive Photos Rocks

But when I got the note from Amazon this morning, I had to give its Photo Cloud a try. I installed it on my Windows 8 tower, Windows 7 laptop and my now famous Mac Mini running Mountain Lion. Installation was simple and it workeds well on all of them - although, annoyingly, it did install Java Runtime on the Mac.

Uploading photos is as easy as right clicking or dragging and dropping. Unfortunately there is no Linux client yet.

Just to put it to the test, I used the software on Windows 8 to put photos from my Google Drive on the Amazon Cloud. On Windows 7, I uploaded pictures via my Lenovo laptop from the shared media library of my wife’s Windows 7 laptop. I also uploaded photos from Dropbox and the desktop on the Mac to the Amazon cloud. There is a nice option in the Mac version to import from iPhoto. You access it from the cute cloud icon in your menu bar. It is easy to choose what you want to import.

The pictures all show up in a nice slide show on my Kindle Fire. The only challenge so far is that I cannot seem to find the original handful of pictures that I uploaded last year. Of course I might have copies in a few other places.

The Internet has evolved from a virtual buffet of content to a hyper-connected network of people and things creating massive amounts of information. Businesses continue to struggle finding ways to store all this data in a way that makes it available anywhere and anytime it’s needed.

Historically, adding storage space was measured in the number of devices and the capacity of the media. From tape, to floppies, to optical media, to hard drives, to USB drives... storage technology continues to improve and the prices continue to drop. The market for storage is expected to rise to $37.3 billion by 2015, according to forecasts by analyst firm IDC. But there never seems to be enough.

The modern trend of what IDC calls the “slow and deliberate,” use of cloud-based storage has created a different paradigm - one that challenges those managing information on the Internet. People around the world create 2.5 quintillion bytes of data every day. And the company estimates that 90% of all data in the world was created in the last two years.

To keep up, businesses have been forced to act like squirrels preparing for winter, putting bits of personal and public customer information hither and thither, loosely tied together as a patchwork of devices and services. Too often, companies complain that storing information online is limited by the boundaries between providers, an inability to react to growing demands and services that are complex to manage and slow to install.

This is why companies like Hewlett-Packard have adopted a multi-layered approach that simplifies the process of storage for business. HP’s Converged Storage Systems and Services takes advantage of the trends of storage technologies to provide an easy method for application integration, virtualized infrastructure and cloud services.

The suite’s current lineup includes

• HP 3PAR StoreServ Storage
• HP StoreOnce Backup with StoreOnce Catalyst
• HP StoreAll Storage
• HP StoreVirtual Storage
• HP StoreEasy Storage
• HP StoreVirtual VSA

HP Storage Addresses Big Data

Even before the inclusion of magnetic-card storage in the HP 9100A desktop calculator (personal computer) in 1968, Hewlett-Packard used storage media to address the data being created by its business customers. The latest technology trend of so-called Big Sata has put unprecedented pressure on storage strategies and technologies. It's also delivering unprecedented benefits to the companies of all sizes that are able to leverage Big Data with systems on site or in the cloud.

For example, the HoneyBaked Ham company uses a consolidated suite of HP storage systems to track customer data and transactions. The company reports improved performance in creating reports and providing information to executives.

Other sectors taking advantage of HP’s storage suite include healthcare, media and entertainment, financial services and public sector industries.

HP Storage Taps Into The Cloud

As cloud computing has emerged as a basic networking practice, more and more content is stored in virtualized, interconnected storage devices. Not only does this make it possible to access massive files from anywhere at any time, it also makes storage more affordable, efficient and easier to manage.

One customer that knows about accessing massive files is Hostworks, which used HP’s cloud computing storage products to assist Australia's Special Broadcasting Service (SBS) in its tracking of fan traffic during the 2010 FIFA World Cup. Using HP's cloud service, Hostworks recorded 1.5 million unique visitors, more than 25 million page impressions and two million video views as well as tracking matches and featuring up-to-the-minute information on the competition.

HP Storage Empowers The Datacenter

Converged storage systems don’t sit under the desktop, they support large data centers, storage arrays, and dedicated data stores. Providing timely access to information has become paramount for companies and storage service providers alike. One of the latest trends is to use solid-state devices to supplement datacenter storage to speed up all applications by replacing or augmenting memory caches. HP is no stranger here, having championed solid-state drives since 1993 with the advent of the HP Omnibook 300.

More recent implementations of SSD drives have helped companies maximize the effectiveness of their existing datacenters. For example, Priceline.com - one of the world’s largest online hotel reservation and travel services - doubled its datacenter capacity and service speeds with HP’s converged storage products. More compact HP hardware with solid-state storage and HP software to manage the systems was installed, resulting in a 65% decrease in the space needed to run the storage array.

As trends go, storage has never been sexy, even by enterprise IT standards. But it’s importance in handling new technology trends such as Big Data, cloud computing and virtualization - as well as its ability to make datacenters more efficient - cannot be ignored. Hewlett-Packard’s Converged Storage Systems and Services has helped thousands of enterprise IT leaders make their companies leaner, more agile and more efficient. Enterprise storage needs are unlikely to become smaller, or less important, so it's important to know that companies like HP are already working to meet those requirements.

Image courtesy of Shutterstock.

Solid-state storage has become ubiquitous in mobile devices and increasingly in laptops as well. But it's also revolutionizing storage in corporate datacenters, cloud computing services and beyond.

Forget the portability factor, solid-state drives, or SSDs, are finding massive growth potential as a way for companies to speed up large-scale storage systems.

Historically, storage devices used in business have been either magnetic tape or hard disk drives, both of which spin, use a lot of energy and throw off a lot of heat (which usually requires expensive cooling).

SSDs, on the other hand, are much faster and more efficient, and generate much less heat - making them ideal for use in datacenters. The only reasons they haven't already become a direct replacement for rotating disks is due to their relatively high cost per gigabyte and the high volumes of data writing required in many datacenter applications.

Although SSD vendors try to engineer around these issues with advanced compression and deduplication techniques - as well as redirecting how and where data is stored - SSDs today essentially provide answers for limited use cases (e.g. small databases), but fall short when it comes to working with large loads of data. This includes the massive amounts of unstructured data that represents the bulk of storage growth today.

That said, the adoption of solid state storage as a complementary solution to standard hard disk (HDD) storage for enterprise applications continues to grow quickly. Worldwide solid state storage industry revenue reached $5 billion in 2011, a 105% increase from the $2.4 billion in revenue achieved in 2010, according to analysts at research firm IDC. SSD vendors are also giving enterprises and storage companies improved technical options. Some 18 million higher-capacity SSDs (ranging from 80GB to 512GB) will ship in 2013, and that number is expected to grow to 69 million units by 2016, estimate analysts with IHS iSuppli.

But if solid state drives aren’t going to kill off spinning hard drives or magnetic tape, where do all those SDD units fit into the storage ecosystem in efficient and cost-effective ways?

Today's Best Uses For SSD In Storage

Solid state drives are most useful for access to data that is needed fast but not in high volumes. SSD technology is also justifiable in applications where lag time or latency could mean lost dollars - such as in trading platforms and financial systems. These attributes also make SSDs valuable as the first line of access for cloud-based storage offerings. Data that needs to be accessed fast goes on the SSD, while less-critical data can be stored on more traditional slower, less expensive spinning disks. This arrangement lets cloud storage providers leverage SSDs along with other storage technologies to provide a broad range of services at the lowest cost.

“The challenge they are solving is that the IOPS (input/output - or I/O - operations per second) are superior in SSDs but the cost per gigabyte storage is orders of magnitude higher than rotational hard drives,” explains Sanjay Parekh, an early participant in solid-state-hard-drive-cloud-storage startup SolidFire. “They have developed a layer on top of SSDs that drive the effective storage cost down while maintaining high IOPS.”

As prices eventually come down, Parekh notes, SSDs should replace most other storage technologies. But there will always be a role for backup systems like tape and magnetic hard drives. A distributed system using multiple storage technologies is best for long-term data security, he adds, as it isn't wise to put all of your data on one storage medium.

Room To Grow With SSD

For now, using SSD as an acceleration tier on top of traditional storage can provide the best of all storage worlds. Relatively small, high-performance SSD units can be used to speed up all applications by replacing or augmenting memory caches.

“It instantly delivers performance on demand, while enabling hot data to stay on higher-cost SSDs and majority of data at rest to sit on well-protected traditional disk drives,” says Kirill Malkin, CTO of Starboard Storage. “That way, the SSD subsystem delivers performance efficiency and the disk subsystem optimizes capacity efficiency. Furthermore, using multiple SSD layers with varied performance and capacity characteristics enables highly effective, just-in-time I/O optimization for consolidating multiple applications on a single storage platform with right sized resources for every workload and use case. Notably, this approach requires a complete rethinking of the storage stack involving dynamic pooling of all resources and implementing performance controls.”

When used as a performance tier in a more traditional storage environment, SSD can extend the life of that architecture. But this kind of architecture is inherently inefficient because it involves heavy "shadow" transfers between the solid state and rotating storage devices, he adds. Successful "tuning" of such storage systems requires an intimate knowledge of the application behavior, making it more difficult to set up and use.

Given these issues, properly sizing the solid state storage tier can be very difficult, often ending up with significant over-provisioning to achieve the required performance target, Malkin adds. The SSD tiers in traditional storage systems architectures are often limited in the way they fit in with current complex storage systems or legacy RAID [redundant array of independent disks] controllers and RAID group management.”

Beyond SSD - What Does The Future Hold?

But for SSDs to take over the storage ecosystem, manufacturers will have to shrink the technology's price premium compared to traditional storage solutions. Today, an HDD might cost $0.24 cents per gigabyte, while an SSD could cost $2 or more per gigabyte. That's a hefty difference when you start talking about equipping large data centers with SSDs.

One way vendors have tried to balance price, capacity and performance is with multi-level cell (MLC) SSDs. However, due to their inherently low endurance, MLC-based SSDs quickly wear out in write-intensive uses. Another approach is TLC flash or triple-bit-per-cell flash SSDs - but the problem then becomes one of physics. Adding cells means shrinking the components smaller than 14 nanometers, making it difficult to manage and more of a work in progress than an actual solution to today's storage problems.

"Since SSDs became a viable option for the enterprise performance, cost and reliability have been critical points of evaluation. However, as IT purchasers begin to better understand the technology's strengths and weaknesses, we are seeing more importance placed on the balance between endurance and cost," according to John Scaramuzzo, president of SMART Storage Systems.

As an alternative, many companies are now considering hybrid storage systems. Hybrid systems combine spinning disk and SSD storage media to balance capacity and performance. Large-capacity hybrid storage devices can cost $500 a terabyte, making them appropriate for a single super-system but not necessarily as a way to cut costs in a large datacenter storage array.

For the foreseeable future, all types of storage formats will likely coexist, with SSDs added to traditional storage systems as the predominant model. (That's largely because of the large installed base and well-understood architecture of traditional storage systems.) Ultimately though, as SSD prices fall due to Moore's Law, experts expect the industry to gradually migrate to a world where all-SSD systems are used for a continually growing subset of applications. For workloads where SSD's performance improvements can't be financially justified, SSD-accelerated systems will continue to be used until the SSD price premium becomes insignificant.

With all the recent teeth gnashing about startup investment shifting from consumer to enterprise technology, it's worth noting that successfully cracking the enterprise market is no easy task:

• 70% of the U.S. economy hinges on consumer spending. Even with the pending fiscal cliff, it's kind of hard to ignore the numbers. 
• Enterprise technology is not a short game.

Unlike most consumer technologies, enterprise infrastructure and applications run on a much longer upgrade cycle: 5-7 years. While you might ditch your smartphone every year or two for a newer model, few companies are willing to swap out their CRM systems, storage or security technologies that quickly.

Switching behavior is both the most complicated and important subject in the enterprise technology market. Even if enterprise customers have good reasons to be unhappy with their technology vendors (e.g., lack of innovation, price gouging, poor support), their business runs on that technology. This makes them highly incentivized to see existing vendors address any issues and continue the relationship. As we all know, moving's a bitch.

Of course, enterprise tech is a rich, rewarding game, so it's worth exploring the strategies startups can use to overcome the barriers to switching in the enterprise market:

1. Transformational Technologies. The ultimate startup is the one that changes the game on an incumbent in such a way that the latter neither can block nor retaliate. Classic examples include Virtualization and Software-as-a-Service (SaaS). Because virtualization decouples compute functions from hardware (while running on top of the hardware), it is the ultimate disruptor because it's non-invasive. SaaS eliminates the stickiness of packaged software - and the lucrative support contracts that go along with it. Interestingly, while there tend to be many attackers in Virtualization and SaaS, only a few players tend to win big. Very big: witness VMware and Salesforce.

2. Changing Product Cycles. Catching technology giants in product transition cycles is one of the most effective ways to insert new technologies. However, this usually requires an outside force to speed insertion. Earlier in my career, Intel Centrino drove the need for enterprise Wi-Fi and forced an architectural change. In 2013 you can see many great examples of this idea, including Palo Alto Networks, Splunk, ServiceNow and Workday. These transition cycles don't last forever, though. Over time the incumbents typically build or buy their way into the new product segment and the situation stabilizes until a new cycle begins.

3. Trojan Horses. Sometimes a new enterprise IT category emerges in an indirect way. Cloud infrastructure eliminates the need to buy IT hardware and software; the rental model emerged as form of shadow IT for specific projects that could not wait for corporate IT to respond. It also became the preferred approach for brand new businesses (Netflix streaming). Amazon Web Services and Rackspace, two big early winners in cloud computing, sell computing cycles by the month, payable with with a credit card - often bypassing traditional IT purchasing processes. Once established, Cloud and SaaS vendors can then turn their attention to selling to mainstream IT.

4. New Buying Centers. The multi-hundred billion-dollar enterprise IT game now pivots on competition for the IT "stack," as we shift from the Client-Server/Web mobel to cloud computing. This change has created a new class of IT decision makers such as the "cloud architect." As companies move more to the cloud, this new IT leadership category drives key decisions for enabling new applications, also driving the buying all of the underlying IT components. And these new buyers may not be as wedded to the incumbent suppliers as were the decision makers they supplant.

5. The Consumerization of IT. The iPhone led to a watershed change both in enterprise mobility and computing. Not only did it challenge corporate purchasing patterns ("I buy, you enable," also known as BYOD, or Bring Your Own Device), it eliminated a final barrier to what constituted a business device. This is less about "consumerizing" enterprise IT, but rather, adapting enterprise IT to leverage consumer technologies. In addition to mobile devices, apps are challenging the application market for business software.

6. Coalitions of the Willing. For most small companies, hiring a large enterprise sales force and entering a year-long acquisition cycle is likely to be an expensive exercise in futility. Sure, you might be able to make a living selling to universities, hospitals and niche verticals, but attacking the Fortune 500 requires friends who need another reason to re-engage in a selling conversation. Manufacturing and strategic partnerships with hardware makers made a lot security companies rich during the client-server era (e.g., McAfee, Symantec). Today, companies like Box are changing the game through new kinds of partnership integrations.

Frontal assaults are the hardest attack strategy for an enterprise startup. Attacking a powerful technology company's profit sanctuary tends to piss them off. If you can pull it off, it might just get your company acquired, but run a big risk of perishing in the attempt.

That's why this tends to be the strategy of large companies (e.g., HP's acquisition of 3Com to attack Cisco) and does not have a great track record. The assault on the business PC by iOS and Android tablets and smartphones may turn out be a more successful example, but, Apple and Google and Samsung are hardly startups.

It can be done, of course. Many decades ago, Microsoft's PC operating system was such a technology and for a generation, a small company in Redmond changed the world. (With a big initial boost from IBM, of course.)

Current technologies that might have the power to force enterprises to switch and create hugely successful startups include Apache Hadoop, Network Virtualization, Flash Storage, and Cloud Storage and Collaboration. That's where I'd look for the next big thing.

Image courtesy of Shutterstock.

We tend not to think about storage - until we don't have enough. We carelessly store documents, emails, images, video, and massive amounts of all kinds data only to wonder why there never seems to be enough places to put our company's stuff. But as new technologies combine to provide storage over the Internet, easing fears of limited capacity and the promise of virtualized architectures are helping shape the next phase of the Internet.

Storage Isn't Sexy, But...

Virtual storage is neither as flashy nor as sexy as virtualized servers. Historically, enterprises set up a storage device, backed up data and content in regular intervals and forgot about it. But because hard drives offer limited capacity, it has become necessary to manage multiple storage strategies. Additionally, archiving digital content traditionally meant burning to a disc or transferring data to magnetic tape. The archived data and content was not readily accessible.

As cloud computing has emerged as a basic networking practice, more and more content is stored in virtualized, interconnected storage devices. Not only does this make it possible to access massive files online in an instant, it also makes storage more affordable, efficient and easier to manage.

By abstracting how storage functions from a set of individual physical hard drives to logical storage (or partitions) spread across any number of physical drives, storage can be made less expensive and much more flexible. With virtualized storage accessible in a cloud computing environment, companies and even individuals can now add as much storage space as they need, pretty much on demand. 

Hardware vs. Management

For consumers, this means devices like smartphones and tablets do not require massive storage drives. For enterprises, virtualized storage means spending less on hardware and more on efficiently managing data and content. The trend meant companies can protect their remote office data and remove the need for multiple storage networks. Virtualizing storage also helps with disaster recovery by spreading the information to remote locations and providing multiple copies of data. The trend is toward continued efforts to make cloud-based virtual storage even more efficient and less expensive. Some enterprising companies have already managed their cloud architectures with multiple storage technologies so well that they've adapted their own capabilities to deliver Storage as a Service to other companies.

There's still potential for further migration toward virtualized storage. Forecasts for the global cloud virtualization software market (currently estimated at $6.7 billion) between 2011 and 2015 show a year-over-year growth rate of 14.98%. Virtual machine and cloud system software represents the fastest growing segment, with research firm IDC pegging growth at 17.8% in the first half of 2012.

Investments in cloud-based storage also suggest future growth. Venture funding for storage companies totaled $458 million through the first three quarters of 2011, according to analysis from Strategic Advisory Services International. That is 42.4% more than the $321.5 million storage startups received in the same time a year before. Storage mergers and acquisitions are also on the rise with 23 deals adding up to $8.7 billion through the first three quarters of 2011.

Benefits Of Virtualized Storage

Outside of the obvious benefits of being able to access content from multiple locations on multiple devices, virtualized storage also allows for information sharing between large numbers of people. While it's still a relatively new technology trend, storage virtualization isn't hype. "But it's all about the use cases," says John McArthur, president of Walden Technology Partners and a board advisor at Starboard Storage Systems. "The use cases will evolve and mature over time, just as they are with server virtualization."

McArthur points to making storage asset management less of a problem, where the goal is migrating data from one device to another without having to physically link them together. Other benefits include replicating data between locations, making point-in-time copies of data, expanding storage capacity, and shrinking storage costs. Additionally, virtualized storage allows for a "pay as you go" subscription model that can increase storage capacities as needed, without having to grow data center footprints.

"Some companies will embed storage virtualization in an appliance to make their appliance simpler to manage and control," McArthur said. For example, hedge fund Thames River Capital virtualized its storage area network and saw a 40% improvement in the performance of its virtual machines as a result.

Virtualization Meets The Cloud

As the technology improves and devices continue to be connected to each other, cloud computing will increasingly merge with virtualized storage. One consideration is using cloud versus physical storage for high-performance computing at scientific research centers, according to John Bates, co-founder and CTO at TwinStrata.

"Cloud storage can solve some of the problems associated with big data, particularly in the areas of resource planning and infrastructure growth costs," Bates told industry reporters. "Cloud storage offers massive and automatic scalability, without requiring heavy capital expenditures on fixed storage systems that may reach capacity too fast."

The Internet Of Things, And More

Another use case for cloud-based virtualized storage is enabling a wide variety of non-computing devices connected to the network, also known as the "Internet of Things." In its estimates for 2020, IDC believes approximately 30 billion devices will be connected, each requiring cost-effective use of software and storage for the information gathered.

Considering the capabilities being developed for the next phase of the Internet, it's not much of a stretch to think that virtualized storage could be used to recreate virtual versions of specific events at specific points in time. A wide array of networked storage devices would hold the information from computers, sensors, cameras and other information sources to quickly recreate almost any event or scenario.

Perhaps the holographic event simulator (the "holodeck") from Star Trek might someday be a reality.

Holding on to old data slows applications, increases storage costs and backup times, and dramatically increases the danger of attacks. A good data disposal policy can reclaim some of your budget and help you sleep better at night.

For the sake of argument, let's assume your company already has a data retention policy. If it doesn't, stop reading right now and make one. No one wants to be left in the lurch when auditors come calling or a client claims you didn't pay that invoice back in 2011.

But what about the other side? Is there such a thing as too much data?

Absolutely.

Why You Need To Do It

According to the Compliance, Governance and Oversight Council, nearly three quarters of all data stored in an organization has no current business use. If that seems like a lot, consider the forms that data might take. The biggest and scariest culprit is email, which often contains sensitive personal and client information, as well as multiple versions of files forwarded as attachments. Email is a horrible storage and versioning system, but it's one of the most popular.

Then there's the problem of department-specific data silos, which often hold redundant records that can be orphaned. Imagine your HR, Marketing and legal departments each keep separate copies of employee records. For compliances's sake (or, more likely, because you never got around to integrating your systems), those records are all stored in separate systems. If HR terminates an employee but the information doesn't sync, you've just created orphans in the other system that may last forever.

On the other hand, maybe you've done it right. Your records share a common repository and each department has properly permissioned views.

You still might be in trouble.

HR might need to retain certain data after a termination, but retaining other sensitive information might actually be illegal. If you're in a highly regulated industry, you're probably aware of these restrictions. If you're not, you may not know about them until there's a lawsuit after a breach.

Don't forget about the storage issue. Slashing your storage by 50% to 75% would save a lot of cash. The CGOC estimates a savings of up to $50 million in some enterprises. In some highly virtualized enterprises, storage costs can account for as much as 40% of the total IT budget. Plus, everything – from record searches to backups – will run faster.

If you're still not sold, Ben Rothke's 2009 article, Why information Must Be Destroyed, remains valid and convincing.

You're on board. Less data equals the less risk carried, faster systems, and more money.

How Do You Get Started?

Create A Policy

This might sound obvious, but the first step toward disposing your data is to create a data disposal policy. It should mirror and integrate with your data retention policy, as well as any other physical destruction (e.g., shredding) policies you follow. You don't want anything falling through the cracks.

Don't try to make decisions on your own. Each department should have input, and the final policy should pass through legal and compliance reviews before landing on the CEO's desk. Everyone needs to be on board.

Assume The Worst

Try to minimize the amount of effort required by employees. For example, autoarchiving emails past an age threshold will point out inappropriate use pretty quickly. One CTO of a mid-sized firm remarked that when his company moved from POP to IMAP and began archiving older emails, his sales department panicked. "They'd been storing customer data in emails and spreadsheets instead of using our CRM system. We were storing sensitive data without gaining any value, and our sales reps weren't doing their jobs." There will always be room for human error, but prevention will ease the cleanup burden after the fact.

Consider The Hardware

Different types of data require different disposal methods. Medical records or confidential design documents may require physical destruction of a disk or a magnetic degaussing. Old tweets and press releases probably need only a simple overwrite. If you're still storing a mix of data on the same physical disks, this might be a good time to change that.

The disposal methods you choose will be based on your industry, so your Legal department is the ultimate authority, but you can start your research with the NIST's Guidelines for Media Sanitization.

Get Service Guarantees

This is a problem even the largest enterprises sometimes face. Much of your data is in the hands of third parties, and more will be shifting that way soon. It may be their cloud, but it's your data.

Send your disposal plan to your service providers and get a guarantee that they'll abide by it. This may add costs to your contract, but failing to do so makes the policy pointless. If your provider already specializes in government or industry compliance, this should be an easy talk to have. If its not, consider shopping around for new services.

Remember: It's A Process

You won't be able to do everything at once. Some parts of the policy may require more review than others. Some systems may require redesign. Get the low-lying fruit first.

If you're starting from scratch, even the first steps are steps in the right direction.

All images except chart courtesy of Shutterstock. Chart courtesy of the Compliance, Governance and Oversight Council.

While people who use Dropbox and Box worry about security, users of services like Google Drive and Apple's iCloud seem  more concerned with implementation issues. Is that a sign of dangerous complacency, or is it only natural that the level of of security concerns vary between different online storage services?

Natural or not, that variation is clearly in place, according to a recent report from a tech-support community site FixYa. Note that while FixYa lets post questions to the community at large in the hope of getting a helpful answer, it differs from typical tech sites by covering a much wider set of consumer products and services - from washing machines to strollers.

Taking a look at its own question and answer database, the folks at FixYa put together a synopsis of the issues that are the most relevant to public cloud services. The results were intriguing, particularly because of where the most expected cloud-storage bugaboo - security - was ranked (or not ranked) among the services outlined.

Security Yes? Security No?

Security topped the list of most commonly asked questions about Dropbox, coming in at 40% of issues raised. Storage limits came in at number two, with file syncing was the number three concern on the list.

Security was not quite as big an issue for Box users, tied for first place with upload issues, each accounting for 25% of issues reported. Backup file questions ranked third.

Perhaps most striking, though, was the fast that for iCloud and Google Drive users, security wasn't a ranked issue at all. The problems listed all deal with application-specific issues.

For Google Drive help-seekers, missing folders was the top-reported problem, at 30% of questions. Syncing files and Google Doc auto-conversion were the second- and third-ranked questions, respectively.

Syncing information figured more prominently for users of Apple's iCloud, with the number two and three ranked questions centering around syncing with non-Apple devices and syncing between devices in general. The biggest reported question was very application-specific: 35% of users had questions about Apple's Mountain Lion OS X operating system update and its affect on iCloud.

Who Cares About Security In The Cloud?

This disparity is notable because it seems to run counter to the popular perception that all such public online cloud services carry fears about security. It may very well be that iCloud and Google Drive consumers are just as worried about security, but they're just not expressing it on community support sites. But it seems odd that security isn't part of the conversation about these services.

Actually, it may not be that odd at all - a very public hack of Dropbox this summer has led corporations like IBM to block Dropbox, iCloud and similar online storage services. Consumer identification of security problems may be associated with the Dropbox brand and the similarly named Box service.

There seems to be a public perception that the Google and Apple services are secure and don't required much attention about security. (Though companies like IBM don't share that opinion.)

Those services may very well be relatively secure, but almost any individual account can be broken into unless strong security guidelines are followed. An online storage service can be as secure as Fort Knox, but if someone has the right key, the gold in your account is up for grabs.

Image courtesy of Shutterstock.

Here’s a question you may never have thought to ask:  When you’re using an application that’s in the cloud to begin with, why is it up to you to save your documents - especially if you’re saving them to the cloud anyway?  With Box’s latest update, your cloud app saves directly to your cloud storage.

If the programs you run on your device aren’t installed there locally, and the system on which you store the documents for those programs isn’t local either, then why is saving documents a manual act?  Put another way, why do you have to “click Save?”  Shouldn’t your program know how to save? Maybe you’re thinking the answer is, “Because I have to name my file at some point.”  Well, consider the pointlessness of that act, too, if the app knew the context of your documents by their contents anyway. (If, for instance, it could tell the title of your paper by looking at the first line.)

Saving To The Cloud

Saving to the cloud has quickly become a market in itself. So there’s been a land rush among Dropbox, Box (formerly Box.net), Microsoft SkyDrive, Apple iCloud, Google Drive and some others to stake their claims to automating the Save command. Granted, Google and Microsoft may have a built-in advantage in this regard, but Box is playing the role of “sooner” in this land rush. Box has begun working with Software-as-a-Service (SaaS) providers to embed storage and document saving functions directly into their apps, bypassing the manual click altogether.

The fruit of their labor is something called Box Embed. Essentially, it’s an interface for allowing applications to access customers Box storage and also their Box functions, such as file preview and task management, inside of SaaS applications so that they appear on the client side.

Last March, Box was working to integrate aspects of operating systems into its ecosystem, including a desktop for launching apps from multiple devices. Box Embed is essentially along those same lines: Just as Windows and Mac desktop applications rely on the Load and Save functions of their respective operating system, a SaaS service that uses Box Embed will rely on the Load and Save functions of Box.

Can Box Pry Away SaaS Storage Customers?

There’s one extra benefit in this for Box, at least potentially: It could pry some customers loose from the built-in, exclusive storage that SaaS providers employ.

“In many cases, you see cloud products requiring that you load content into their cloud,” notes Chris Yeh, Box’s platform vice president. He cites an example of a Salesforce user who’s working with a prospective customer, and needs to load several items of sales collateral. With Salesforce, he says, you have to load that material from the service’s own content store.

“You’ve got files in Salesforce’s cloud, then you’ve got files in NetSuite, and some more content sitting in Workday.  The idea for us was to find a way to use Box as the cloud file system, if you will, and embed that – where possible – in these third-party applications.”

If a desktop application can make its operating system create convenient, personal folders for its user, then the cloud-based counterpart should be able to do the same. NetSuite (above) is one of two SaaS applications that is already enabling Box Embed (the other is SugarCRM, below). As Yeh explains it, the automated configuration process for Box on NetSuite actually peruses customer records and generates a folder structure in Box storage for customers by name. Then the existing tagging system is used to flag these customer folders for NetSuite, so that those tagged folders may be shown in NetSuite as though they were NetSuite’s own attached storage.

What Box Embed is not - at least not yet - is an interoperability or cloud integration system. It appears Box could be reserving a niche for itself in this space, perhaps someday competing with services like Informatica that encourage customers to share their own scripts for converting formats and schemas from one service’s database to another.

Box Integration 

“In the past, if you were to take two systems of record and get them integrated,” says Yeh, “there were a certain number of paths.  You could build a custom integration where you’d take data out of one database and move it into another, in some form. You’d do that with a big systems integration: It was a lot of work, and super-costly. Another would be, you’d go through an integration tool like TIBCO, where when you moved data from one system to another, you’d put it on a third-party integration product, and have it be placed onto another system of record, so data was oftentimes duplicated.”

These types of integration, Yeh continues, pre-date the advent of apps run through the browser, where user interface components are not only easily generated, but readily shared. It becomes an almost academic matter, then, to make the user interface components of one app appear in the context of another.

“So in this particular case, NetSuite is creating a place for us to run - it’s an IFRAME - and what we’re doing is putting a version of Box into that IFRAME which is contextually set up correctly to match the customer records.”

It’s integration in one sense – enabling a unified cloud storage structure that’s automatically configured to fit the context of SaaS applications. It’s not integration in the TIBCO or Informatica sense, where the data becomes interoperable from that central location.  But Chris Yeh leaves that matter open for future exploration - and the last time Box left something open for exploration, we didn’t have to wait all that long.

If Box makes itself the preferred storage system for work, since its storage agents would already be on your devices, the likelihood you’ll use Box for your personal applications also instead of the others also rises.

Lead image courtesy Shutterstock.