Computerworld recently posted an article describing the challenges of “big data”, or what they see as “really big data”. They’ve posted on these challenges before, and it’s interesting to see where they think the big data challenges lie:
“We’ve all heard the predictions: By 2020, the quantity of electronically stored data will reach 35 trillion gigabytes, a forty-four-fold increase from 2009. We had already reached 1.2 million petabytes, or 1.2 zettabytes, by the end of 2010, according to IDC. That’s enough data to fill a stack of DVDs reaching from the Earth to the moon and back — about 240,000 miles each way.
For alarmists, this is an ominous data storage doomsday forecast. For opportunists, it’s an information gold mine whose riches will be increasingly easy to excavate as technology advances.”
Right now, most companies in the “big data” space seem to be focusing on two things: (1) handling, storing, and moving data, and (2) building tools to visualize and analyze data. With the explosion of data that is generated, having the technology to store it and move it around is obvious. Also, since there is a lot of data, tools can be easily developed to grab the low hanging fruit that “big data” can provide.
These technology pushes are absolutely important, and necessary to even get at the easy problems that can be solved with mountains of data. However, as the problems get harder, there will come a need for engineering the right analytics. Data science and a rigourous engineering discipline can really push the envelope on what can be done in this “big data” space…
More on Computerworld’s take on the data mountain challenge can be found here…
Get My Newest Articles In Your Inbox
Subscribe to get my latest articles from Decisions & Discovery by email.