- February 16, 2012
- 89 views
Big Data is more than Hadoop and Analytics
How do you make a big issue small? Redefine the term!
The definition of Big Data is rapidly being morphed into a subset of what it really ought to be. Over the last 3-6 months, the prevailing sentiment, if you focus only on the buzz meter, is that the Big Data discipline is really a simple matter of adopting Hadoop and using it to create some business analytics app—most likely in sales and marketing— to do sentiment analysis. This all seems a bit BI-centric to us.
Hadoop Analytics apps are nice to be sure, but only about 1/100th of what Big Data really is!
Big Data is not a quick and dirty movement of data to analyze for a point in time application. It’s a pervasive, systemic problem domain that creates challenges the magnitude of which we’ve never before encountered in IT. That’s because data is really now a strategic asset of every business, just as valuable as your products, customers and even the cash in the bank. Data defines your value as a business and a service to your market. To not look at it as the fundamental issue to focus on and manage is as short-sighted as you can get.
This perspective is something that will no doubt change over time. Like many new areas, the initial burst of jumping to the end game will ultimately be replaced by a management process that addresses the problem at its core. In this case, this is the idea that Big Data is really about all of the massive amounts of data within our corporate domains.
We’ve been here before … take eDiscovery as a very recent example.
We really don’t need to look too far back in to history to see how this will play out. In 2006, the Federal Rules of Civil Procedure were published saying that all electronic data was now included as a source in discovery, thus the eDiscovery market began to take shape. At that time, law firms and corporate counsel were all well versed in the discipline of reviewing documents using automated review tools. They just never had to worry about searching for all of the data that was relevant.
The first generation of eDiscovery tools took off, offering review capabilities coupled with bulk load file ingestions that allowed packets of data to be moved in to these systems. They largely got there manually and a good 95% of it was irrelevant but the market was ‘white hot’.
Or at least until the bills started to come in on bigger cases, and then after that the fines—often in the tens of millions of dollars!