One of the hottest topics in business analytics today is “big data,” defined by Wikipedia as “a term applied to data sets whose size is beyond the ability of commonly used software tools to capture, manage and process the data within a tolerable elapsed time.”
How big is “big data”?
Last year, it was estimated that consumers and businesses around the world stored more than 13 exabytes of information on PCs, laptops and other devices — the equivalent of more than 52,000 times the information housed in the Library of Congress. An exabyte is 1 followed by 18 zeros, or a billion gigabytes. And the amount of data stored in such “technological memories” is growing 25 percent a year, according to Martin Hilbert, a researcher at the University of Southern California. These were some of the estimates shared at the Economist Big Data Conference last June in Santa Clara, CA. (complete story).
The experts at the conference had strong opinions on the value of learning to exploit big data. “Businesses that analyze data about customers increasingly have an edge over their competitors. Retail companies that are savvy about sifting through their data can improve operating margins by 60 percent,” said James Manyika, director of the McKinsey Global Institute.
Sounds great, let’s all build our big data systems and get back to making money.
Or maybe it’s not that simple. Jeff Jonas is the intelligent systems engineer who developed the casino fraud detection system that caught the infamous “MIT Team” portrayed in the movie “Bringing Down the House.” He observed that as the information overload has increased, the ability of organizations to keep up with it has actually flattened. Corporations are becoming “dumber” as the percentage of data that is used for decision making gets smaller (awesome video).
Still, increasing operating margins by 60% is a pretty attractive incentive. It’s a safe bet that eventually we will find a way to develop computer systems that will guide us to make fast decisions with reliable outcomes. What can chain store real estate planners do to prepare for this new paradigm?
The first step is to get used to working with computer-based information systems. One of the hottest companies in the big data space is Opera Solutions, which just received an $84 million round of financing to fund its astounding growth. The company is grounded in science, with 160 of its 600 employees holding advanced degrees in quantitative analysis (half are Ph.Ds.). However, its philosophy is that it’s the combination of big data, intelligent computer systems and people that produces the best decisions. As Opera’s CEO, Arnab Gupta, explains, “We often create false dichotomies for ourselves, but the truth is it’s never been man vs. machine; it has always been man plus machine. Increasingly, I think it’s an article of faith that the machine beats the human in most large-scale problems, even chess. But though the predictive power of machines may be better on a large-scale basis, if the human mind is trained to use it powerfully, the possibilities are limitless” (O'Reilly Radar article).
Another important step is to make sure that all of our important data sources are accurate and in one place. I am continually amazed at the way real estate and customer data are managed in chain store companies, even the ones with large IT departments. The Excel spreadsheet is still the leading “database.” In fact, it’s not ONE spreadsheet, but MANY spreadsheets with data of different formats and vintages. Sometimes there’s a great enterprise database but the real estate department can’t get access to the data on a timely basis in a format that can be used for mapping or analytics. Third-party data such as competitor locations or important retail traffic generators are often unreliable and there may not be a process in place for people in the field to correct the errors.
Finally, we need to begin thinking about how to use the data we have in CONTEXT. For now, we need to develop associations between the data using categories and “tags” such as “urban” or “freestanding.” In the future we will have smart databases that can create context “on the fly” based on the data, such as “urban store where competitor dominates and brand loyalty is growing.” Until the “Big Data Machine” arrives we can at least make sure that we have all the data we need, stored and maintained in a single database, and accessible for all decision support tools such as maps, reports and analytics.