{"id":26836,"date":"2011-08-20T17:55:34","date_gmt":"2011-08-20T16:55:34","guid":{"rendered":"http:\/\/alienbabeltech.com\/main\/?p=26836"},"modified":"2011-08-20T18:02:19","modified_gmt":"2011-08-20T17:02:19","slug":"the-challenge-is-staying-afloat-in-a-rising-sea-of-data","status":"publish","type":"post","link":"http:\/\/alienbabeltech.com\/main\/the-challenge-is-staying-afloat-in-a-rising-sea-of-data\/","title":{"rendered":"The Challenge is Staying Afloat in a Rising Sea of Data"},"content":{"rendered":"
\n
The following is by G. A . “Andy” Marken, President of Marken Communications<\/span><\/a> Inc. He has kindly given us permission to republish his thought-provoking article. As with everything that we publish at AlienBabelTech, the opinions expressed are solely those of the individual writer and do not necessarily reflect the views and the opinions of the rest of the ABT staff.<\/strong><\/span><\/span> \u2013Mark Poppin,<\/span> ABT Senior Editor<\/span><\/span><\/strong><\/h6>\n<\/blockquote>\n

Swimming in Data<\/strong><\/h2>\n

\"\"<\/a>\u201c<\/em><\/strong>A $40 million computer tells you you’re chasing an earthquake, but you don’t believe.\u201d<\/em> \u2013 Capt Mancuso, \u201cThe Hunt for Red October,\u201d<\/em> Paramount Pictures (1990)<\/strong><\/p><\/blockquote>\n

\u00a0<\/strong>One of the toughest jobs in any country, any company is gathering, hoarding, protecting information that may give them an edge against \u201cthem.\u201d<\/p>\n

And we must be doing a helluva\u2019 job because organizations and individuals are accumulating data and meta data at an ever-increasing rate.<\/p>\n

The sheer volume that\u2019s coming at us is causing us to experience information overload\u2026emails to answer, virtual friends to contact, YouTube videos to watch, games to play, movies to sneak, reports to read\/digest, stuff you have to find\/do in the virtual world\u2026just in case.<\/p>\n

Computers, smartphones, tablets are helping to accelerate our hunger for more data and our ability to create new\/better information.<\/p>\n

Does it seem as though there\u2019s so much information, you can\u2019t keep up with it?\u00a0\u00a0\u00a0 Can\u2019t make sense of it?<\/p>\n

You\u2019re right!<\/p>\n

As Jack Ryan said, \u201c<\/em>Welcome to the New World, Captain.\u201d<\/em><\/p>\n

The Next Big Number<\/strong><\/p>\n

EMC (a world leader in big storage stuff) recently sponsored a report by IDC (international research firm) that says we have finally cracked the Zettabyte barrier.<\/p>\n

\"\"<\/a><\/p>\n

Piling Up \u2013 <\/strong>When it comes to storage capacity a MB isn\u2019t much, a GB is barely tolerable and a TB is almost a necessity for families today.\u00a0 It wasn\u2019t that long ago that we considered a PB as an amazingly big number; but today, we talk about a Zettabyte without blinking an eye.\u00a0\u00a0 Storage requirements are more than doubling every year and there\u2019s no end in sight — as long as we can properly manage it and find the specific information we want and need.\u00a0 Chart – IDC<\/strong><\/p><\/blockquote>\n

This year, we will have created, replicated more than 1.8 Zettabytes of data (images, video, music, graphs, stuff) which represents a 9x growth in just five years.<\/p>\n

And we\u2019re getting better at it.<\/p>\n

Over the next 10 years, data centers\/repositories \u2013 enterprise, cloud, personal \u2013 will grow by a factor of 50 while the number of files these locations will have to deal with will increase by a factor of 75.<\/p>\n

As Admiral Painter commented, \u201cThis business will get out of control.\u2019<\/em><\/p>\n

Why are we capturing, storing, hoarding all of this data?<\/p>\n

Industries of Knowledge<\/strong><\/p>\n

The global economy isn\u2019t based on \u201cthings\u201d anymore; it has become a knowledge economy and the countries\/organizations with the smartest, best-trained workforces, best ideas and the ecosystem to support innovation will be the leaders.<\/p>\n

The value of the data you and everyone captures was expressed by Stewart Brand, founder of Whole Earth Catalog, at the first Hackers\u2019 Conference in a conversation with Steve Wozniak (co-founder of Apple) in 1984.<\/p>\n

In his discussion with Woz, Brand said \u201cOn the one hand, information wants to be expensive, because it\u2019s so valuable.\u00a0 The right information in the right place just changes your life.\u00a0 On the other hand, information wants to be free because the cost of getting it out is getting lower and lower all the time.\u00a0 So you have these two fighting against each other.\u201d<\/em><\/p>\n

Viola!!\u00a0 Brand gave a whole band of Black Hatters (hackers) a rallying cry.<\/p>\n

No good deed goes unpunished.<\/p>\n

The innocent conversation also gave bands of super-bad, super-annoying Black Hatters like Wikileaks, Anonymous, Lulz Sec and others a rationalization for their activities.<\/p>\n

But simply outing private data because you can doesn\u2019t accomplish anything.\u00a0 In fact, most of the time, it does more harm than good.<\/p>\n

As Capt. Mancuso observed, \u201c<\/em>The hard part about playing chicken is knowin’ when to flinch.\u201d<\/em><\/p>\n

Opening Data Vaults<\/strong><\/p>\n

Whether they are being pulled or pushed, many organizations are opening their data vaults to countries, companies and individuals to develop even better information, better solutions.<\/p>\n

\"\"<\/a>The most valuable commodity companies and individuals have is the data they want to save as securely as possible.\u00a0 With knowledge becoming such a valuable commodity, some organizations have found that the more open they are with their information, the more solutions they find themselves.\u00a0 Art – NYTimes<\/strong><\/p><\/blockquote>\n

The World Bank, long a treasure trove of information, is making that information available to democratize development economics.<\/p>\n

Historically, the bank hoarded information and released bits and pieces in a choreographed fashion to a specific audience.<\/p>\n

With the change, organizations and people everywhere have been tapping into and using the free data.<\/p>\n

The information has been used to reshape government policies around the globe; and in some instances, to reshape governments when people use it with their smartphones, Twitter statements, Facebook\/Google+ postings.<\/p>\n

The new computing\/communications tools create \u2013 you guessed it \u2013 even more data.<\/p>\n

\"\"<\/a>Graphic Illustration \u2013 <\/strong>There are almost limitless ways to describe how our data is increasing.\u00a0 By 2020, the digital universe will be 44 times larger than last year and will include the Petabytes of storage individuals will have in their homes.\u00a0 Source – IDC<\/strong><\/p><\/blockquote>\n

With the right tools, IDC points out that the overload of information is becoming increasingly valuable and available across all sectors of the digital universe.<\/p>\n

Improved global network connectivity, lower cost computing power and streamlined\/focused apps make it easy for companies \u2013 large and small \u2013 to tap into and use transactional data, warehoused data, metadata and data located virtually anywhere to respond to, assist and serve customers.<\/p>\n

\u00a0Data on Everything, Everyone<\/h2>\n

\"\"<\/a>Created, Consumed \u2013 <\/strong>While producing data from data, about data, may seem like a whirlpool or black hole; it is our ability to analyze all of the data surrounding other data that delivers answers to unasked questions.\u00a0 From tomorrow\u2019s infrastructure, apps, governance and consumption, big data is also changing the datacenter and its team.\u00a0 New opportunities appear with each new content delivery source that is launched.\u00a0 Source \u2013 IDC sponsored by EMC<\/strong><\/p><\/blockquote>\n

The big data that is getting bigger all the time isn\u2019t just original content waiting to be used.<\/p>\n

Improving with Age<\/strong><\/p>\n

It\u2019s more valuable than ever because data is captured about the content consumers use, which is arguably even more valuable than the original content.<\/p>\n

In other words, the data around the data.<\/p>\n

As Jack Ryan said, \u201c<\/em>It is wise to study the ways of one\u2019s adversary.\u201d<\/em><\/p>\n

\"\"<\/a>Better Information, Better Assistance \u2013 <\/strong>Companies, both large and small, are increasingly turning to the broad range of internal\/external information to manage their organizations, plan products\/services for tomorrow, and speed\/improve product and customer support.\u00a0 Source \u2013 SAP<\/strong><\/p><\/blockquote>\n

As IDC explains, big data calls for new technologies to economically extract value from huge volumes of a wide variety of data so that it can be captured, analyzed and used.<\/p>\n

Fortunately, while the mass of information is growing exponentially, the cost of storing it is steadily dropping.<\/p>\n

Decreasing Data Storage Costs<\/h2>\n

\"\"<\/a>Demand Increases, Costs Decrease \u2013 <\/strong>As companies and their IT activities change, there will be changes as the digital universe expands.\u00a0 It will be more complex, require more processing, more storage and increased management which will raise the levels of investment.\u00a0 Fortunately, storage costs will decrease to meet the exploding storage requirements.\u00a0 Source \u2013 IDC sponsored by EMC<\/strong><\/p><\/blockquote>\n

That\u2019s a good thing, at least in our household where we\u2019re constantly adding TB drives.<\/p>\n

Why?<\/p>\n

Because even in a free information society, there are some data that must be safely\/securely protected.<\/p>\n

Privacy Still Vital<\/strong><\/p>\n

In the digital universe, there are key areas of security that must be maintained:<\/p>\n

–\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 Privacy \u2013 say your email address on a Sony account or YouTube upload<\/p>\n

–\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 Compliance \u2013 email retention in the event of litigation or legal compliance<\/p>\n

–\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 Account information \u2013 such as recent customer\/user identity thefts<\/p>\n

–\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 Confidential \u2013 customer lists, trade secrets<\/p>\n

–\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 Lockdown \u2013 the highest security for medical records, financial transactions, military actions,<\/p>\n

As Jeffrey Pelt pointed out, \u201c<\/em>I’m a politician which means I’m a cheat and a liar, and when I’m not kissing babies, I’m stealing their lollipops. But it also means I keep my options open.\u201d<\/em><\/p>\n

The bad Black Hats argue that all data, all information should be freely available; but somehow we don\u2019t believe that will hold up in any court\u2026including the court of public opinion.<\/p>\n

Our desire to accumulate more and more information is insatiable.\u00a0 The key to your success though is how you filter, discard and use that information \u2026 and how often you\u2019re right or wrong.<\/p>\n

There\u2019s still hope, as long as we review the data and rely on gut feelings.<\/p>\n

As Jack Ryan said, \u201c<\/em>I had a 50\/50 chance. I needed a break.\u201d\u00a0 <\/em><\/p>\n

\"\"<\/a>Finding the data in an ocean of files is becoming a business opportunity for app developers and submarine drivers.<\/p>\n

As Rich Wallace, publisher\/editor of The Next Silicon Valley<\/em><\/strong> pointed out recently, there are increasing opportunities for entrepreneurs, start-ups and established firms everywhere to take advantage of today\u2019s knowledge-based economy.<\/p>\n

# # #<\/p>\n

by: G.A. “Andy” Marken<\/strong><\/p>\n

President<\/strong><\/p>\n

Marken Communications Inc.<\/strong><\/p>\n

Andy@markencom.com<\/a><\/strong><\/p>\n

Please join us in our Forums<\/a><\/p>\n

Become a Fan on Facebook<\/a><\/p>\n

Follow us on Twitter<\/a><\/p>\n

For the latest updates from ABT, please join our RSS News Feed<\/a><\/p>\n

Join our Distributed Computing teams<\/p>\n