BigDataBytes

One of the major issues IT leaders face in directing their organizations on how best to embrace Big Data as both a problem and an opportunity is where to begin?

This is not an unusual scenario for CIOs.  It seems about once every five years, a major transformative disruption occurs in the enterprise that both business and IT struggle to consume.  It happened with the PC, client/server applications, CRM, the Internet, cloud, social and mobile.  Now it’s happening with Big Data.  Part of the success factors that we’ve noticed is an organization’s ability to embrace both the reality of their situation today and the strategic roadmap of where they want to be five years from now.  The perspective needs to be on the journey, not the event and the reality is this:

Big Data has a Maturity Curve.

The maturity curve has two major factors associated with it and two major pitfalls that organizations can fall into.  The axis are the growth in both management controls and business value.  As organizations seek to consume Big Data, they have to deal with both in unison.

On the business value side of the equation, the principles relate directly to a philosophy that there is actually a ‘Return on Data’ (ROD) value that can be calculated.  This can best be described as how well an organization can:

• Understand the assets that it has, find them and quantify their impact.
• Reduce the cost of and time it takes to retrieve information of value.
• Feed business users with relevant information to solve problems dynamically.
• Proactively identify patterns and trends that help make better decisions.

Underlying all of this is the philosophy of how many management controls to put in place.  As we look at 100’s of companies today grappling with the challenges of managing big data, we see clearly four camps of attitudes, cultures and maturity levels in their processes.  They are either:

1. Inactive – where companies deal with Big Data issues as a storage problem and essentially ‘deny that there is a problem’.  When issues such as eDiscovery and regulatory requests come up, they just push the work to third-party consultants.  This approach has several failings:  (1) it’s expensive, (2) it’s unpredictable, (3) it’s disruptive to employees, and (4) there are often information collection gaps even in the best case.  Note that an average eDiscovery request in a major legal proceeding costs $1.5 million to satisfy.

2. Reactive – where companies buy tools to solve problems one at a time, the ‘we’ll deal with that when we have to’.  They employ software tools in a reactive way to solve information management problems in specific departments or functional areas.  This creates some level of execution and cost predictability for individual groups, like legal or finance, but doesn’t create corporate information management system that would operate at maximum efficiency by lowering costs and risk across the board.

3. Proactive – where companies implement a system with an integrated set of tools for ‘proactively cleaning up the Big Data mess before it hurts the company’.  The impact can be compelling:  (1) much lower eDiscovery and regulatory costs, (2) streamlined IT infrastructure, (3) much visibility into and control over data, and (4) proper data management policies.

4. Active – where companies view Big Data as an asset and have people, platforms, processes, and technologies in place to gain insight into their data and make much more intelligent decisions.  Their view is that this is a core infrastructure component of the IT frameworks that needs to be ‘active and always on’.   These companies use Big Data as a competitive weapon, not just something to be governed and managed.

In the middle are the technologies, products and solutions that enable organizations to embrace and adapt to various trigger points on the curve to maturity.

1. Visualization tools that expose context of all data assets.
2. Storage solutions that optimize and migrate to the cloud.
3. eDiscovery solutions that dynamically respond to litigation or compliance needs.
4. Information Governance applications that enforce mandates.
5. Decision-making applications that are infused with relevant data.

Putting it all together – Matching your Maturity and Value Requirements with Management Discipline

No one place on the curve is better than the other to start from but the perspective of the organization needs to be very clear about that point and how the company will move along the continuum.  An in-depth assessment should always be coupled with a management strategy for growth.  The table below captures many of the characteristics organizations will seek during the process:

Where do you fit? 

Organizations need to move past talking about Big Data and getting ‘active’ about managing it.  We believe strongly that there is a strategy to turn Big Data into Big Business quickly and cost-effectively.  We’ll write more about this as this blog evolves but it includes implementing an infrastructure that views data as an asset, manages it in its native state in real-time and leverages it to make key processes of the business more effective and efficient.  The industry leaders in Big Data have moved beyond cost, risk and storage limitations as problems and moved into creating ‘Corporate Intelligence Centers’ that keep their companies aware of key trends and acting confidently to make data-driven decisions.

They are more productive, profitable and market leading vs. market following.