What We Need is a Storage Jedi

“Pain, suffering, death I feel. Something terrible has happened. Young Skywalker is in pain. Terrible pain” – Yoda, George Lucas Star Wars

Oh come on…give us a break!

Yottabytes, Yoda?

You thought the same thing when you heard that we’re creating, copying, sending, copying, receiving, copying, archiving stuff. Someone had to invent a new word for all the stored stuff … Yottabyte.

EMC and IDC recently validated what we all already knew.

We’ve streaked right past Moore’s chip law and entered the brave new world of always on…always in touch…always creating…always accessing…always sharing.

Seems like only yesterday we were worrying about hanging onto Terabytes, Petabytes, Exabytes.

But by 2020, we’ll have an estimated 35 Zettabytes of stuff to contend with; and then we’ll move to Yottabytes to contend with!

Introducing BBZ – That’s right, in the not to distant future – and in our own galaxy – we’ll be routinely talking about Zettabytes of data.  To explain Zettabytes, we’ll simply say it is BBZ … big bunch of zeros.  Source – IDC

Our kids don’t worry about it.

They just downloaded a few cool videos…just stored a few dozen photos they snapped this past weekend…just downloaded a few research reports they need to use for school…grabbed 20 or 30 great songs they wanted.

More specifically, according to Parks Assoc individuals’ file ownership is:

• digital photos – 85%
• music – 67%
• home office docs – 59%
• PDF files – 55%
• Scanned docs – 43%
• Game files – 40%
• Movies (not family movies) – 37%
• Home movies – 34%
• TV shows 29%
• Graphic images 27%
• Other video files – 11%
• Books – 10%
• Other – 10%

Then, they backed them up onto their portable hard drive.  Just to be safe they copied them onto their private (locked) areas of the family network storage device.

Since they were out of storage space, we visited the big box store and bought a couple more really cheap 500GB USB HDs, because they couldn’t take the chance of throwing away the old stuff.

After all, a day after the stuff was deleted…they would need it.

The heck of it is…we understand that!

It’s no wonder analysts estimate that the average family will have 12TB of content and roughly half of that will be backup and user generated content.

Family Stuff – Photos, videos, family documents, friends’ information, materials and everything else will increasingly be captured and stored by you just because it’s so easy and you can, according to most industry experts.  The result will be the average family will have a couple of Terabytes of data sitting at home waiting to be accessed, used, enjoyed. —Source – Coughlin Assoc

That might not be so bad, but people seem to want to share things.

You know, upload some of the stuff to their Facebook accounts, to YouTube, to some photo sites, to some cloud storage places.

No Scream in Space

No wonder Yoda can feel the pains of the universe…the vast reaches of space are getting filled up!

Most of the stuff out there though is what we call inactive data.

Retired, Semi-retired – Studies have found that if data – photo, video, document – hasn’t been touched for 90 days, it will seldom, if ever, be accessed and could just as easily be archived for posterity, freeing up realtime storage for your use.  Source — Panasonic

You know, stuff you’ll never look at, never use ever again.

Delete it…erase its very existence?

Never !

So what ya’ gonna’ do?

What the ****, take it to the office and hide it in the boss’ huge storage farm.

Yours, Mine, Ours – Companies and government agencies are required to save and protect just about everything.  It’s a big enough challenge for IT personnel to save (and find, if necessary) corporate files; but they’re also required to provide the same level of preservation/availability for personal content on their devices.  Source — IDC

He and the IT folks will never notice a lousy Terabyte or two of your stuff.

Brilliant!

The problem is – theirs, not yours – all that stuff (yours, his, theirs) now has to be managed, tracked, protected, saved, whatever.

It’s the law!

Sure, you can blame it all on big business and big government.  After all, a business does more stuff, more bad stuff than you do, right?

You may not believe it, but the government really doesn’t care what you have at home, but business?

Big bad folks, big targets.

Of course, based on the track record of the financial industry, you can’t trust ‘em so you have to force them to save their stuff…all their stuff.

And governments are great at laying down the law and they want it all saved…just in case.

So businesses practice a lot of information management or data retention.  Last year, that cost firms $4 trillion, and the cost is going to continue to rise.

Managing Info – Analysts estimate that organizations spent more than four trillion dollars last year to manage digital stuff.  While the cost per GB is being reduced, the volume is increasing…significantly.  Source — IDC

Since disk space is cheap and no one – not even the government – can figure out their information compliance laws, IT folks decide it is best to error on the side of caution (it’s called CYA, and you can look it up) and save everything.

Buying more drives or stuffing the stuff in someone’s cloud is a helluva’ lot less expensive than going to jail.

The problem is, most of the stuff they save is what real techies call unstructured data or dark matter – it’s the useful information stuffed in the tons of raw data that’s just too difficult to find, grab, save.

Dark Stuff Everywhere

Yes, most of the content sitting on your own or your home HDs is also dark matter.

Most of that stuff is security, network, and system event logs.

You see, everything you do on your system is recorded (somewhere).  That makes it easier for the system (and you) to search, analyze data so your company and you can manage how the systems are used.

It makes it easier to be in compliance with government regulations — PCI, SOX, FISMA, HIPAA (check the acronyms in Wikipedia).

All that stuff is also what folks like Google and Facebook use to sell services/information to other companies about you as well as support customers.

Bet you thought all those free things they offered you were really free, didn’t you?

…silly boy/girl!

The big boys do something we wish we could do – without a lot of sweat, time – they deduplicate their data.

That means they have one copy of something/anything stored and anything that is referenced to it is linked.

Say you have a 1MB email attachment in your organization that 100 different people “touched.”

Left to our own devices, that would be 100MB of storage space.  But deduplication would only have one copy and all 100 links would go to the one 1MB file.

How cool is that?

For business, that means it would:

• increase their overall data integrity and reduce overall data protection costs
• reduce their carbon footprint because they’d require less power, space, cooling requirements
• improve data restoration speeds (when you hit that key and go *****!)
• reduce data being sent over the WAN for remote backup, replication, recovery
• lower overall hardware, data management, backup/retrieval costs

If that stuff (data deduplication software) is out there for the individual/family systems, we haven’t heard much about it.

Imagine if you could buy the software for oh $50, just think how many 1TB drives you wouldn’t have to buy for the family!

It probably wouldn’t make a ripple in the insatiable demand for storage, but we just haven’t seen many people review that stuff for SMBs, families, individuals.

Think it’s another conspiracy?

Naw…just not as sexy as reviewing another HD, another flash drive, another optical drive.

Lack of Jedis

That shouldn’t make you toss n turn as much at night as the fact that — surprise – most of your stuff isn’t protected.

Ripe for Picking – Everyone agrees that protecting content and making your information secure is not just a nice thing, it’s vital.   Despite the critical nature, about half of the documents, data, images, video sitting on storage devices – yours, mine, theirs – can be easily “borrowed” or destroyed because it isn’t protected.  Source — IDC

That’s right…on your notebook, netbook, tablet, home server, office system or “their” system; a lot of stuff is just waiting to be ripped off.

The majority of your privacy data, compliance information, confidential data, super sensitive stuff is just sitting there waiting for really smart, really sneaky/rotten people to rip it off and use it.

Just ask all those marvelous free social network and web services how often they’ve “lost” someone’s stuff.

But we (all of us—businesses and individuals) are still going to use the cloud storage because it’s sorta’, kinda’ necessary today and tomorrow.

In fact, IDC estimates that 12ZB of the world’s content will be touched by the cloud by 2020 and 5ZB will reside in the cloud services.

It has to be because: a) media life is decreasing over time and b) the volume of stuff being capture is increasing.

More, Less –There seems to be lots more – more content available, more content you want.   At the same time, you continually find you have less storage capacity, less time to enjoy the content, even less time to find the content you’re looking for.   Source — IDC

That’s a good solution because after all, you can continue to grab stuff, keep your carbon footprint down, lower your costs.

The thing that bothers us about dumping the stuff in the cloud is that it just seems like we (all of us) are simply shifting the garbage somewhere else, for someone else to manage, for someone else to do – something – with.

Garbage Scow – Somehow, when we hear all of the benefits of anyone/everyone storing anything/everything in the cloud, we envision a group of New York City garbage scows heading out to open sea to wander forever with refuse.  All we’ve done is dump our stuff on their devices to hang onto.  Of course, they are experts at saving your stuff.

Hey, that’s their problem!

2020’s 1.2ZB seems like a long way off but the way 2010 is going, it’ll be here before we know it.

Then BAM!! we’ll have to go out and buy a couple of PB HDs for the kids.

Don’t know, Yoda…we’ll just sit here and hope for an inspirational.

President Marken Communications, Inc.

Andy@markencom.com

________________

Please join us in our Forums

Become a Fan on Facebook

Follow us on Twitter

For the latest updates from ABT, please join our RSS News Feed

Join our Distributed Computing teams

• Folding@Home – Team AlienBabelTech – 164304
• SETI@Home – Team AlienBabelTech – 138705
• World Community Grid – Team AlienBabelTech