We’re in the era of big data, but what do we mean by that? In our view, big data is a relative, not absolute, term. It means that the organization’s need to handle, store and analyze data (its volume, variety, velocity, variability and complexity) exceeds its current capacity and has moved beyond the IT comfort zone [1]. Big data is the classic dual-edged sword – both potential asset and possible curse. Most agree that there is significant, meaningful, proprietary value in that data. But few organizations relish the costs and challenges of simply collecting, storing and transferring that massive amount of data. And even fewer know how to tap into that value, to turn the data into information.
Is the enterprise IT department merely an episode of TV’s “Hoarders” waiting to happen – or will we actually find ways to locate the information of strategic value that is getting buried deeper and deeper in our mountains of data? Quite simply: What are we going to do with all of this data?
At its essence, high-performance analytics (HPA) offers a simple, but powerful, promise: Regardless of how you store data or how much of it there is, complex analytical procedures can still access that data, build powerful analytical models using that data, and provide answers quickly and accurately by using the full potential of the resources in your computing environment.
With high-performance analytics, we are no longer primarily concerned with where the data resides. Today, our ability to compute has far outstripped our ability to move massive amounts of data from disk to disk. Instead, we use a divide-and-conquer approach to cleverly send the processing out to where the data lives.
Ultimately, HPA is about the value of speed and its effect on business behavior. If the analytic infrastructure requires a day to deliver a single computational result, you’re likely to simply accept the answer it provides. But if you can use HPA to get an answer in one minute, your behavior changes. You ask more questions. You explore more alternatives. You run more scenarios. And you pursue better outcomes.
