Some years ago, we met a client who was wildly excited about large customer data sets. "It's the granularity that's so amazing," he enthused. "For instance, people who shop in petrol stations on a Thursday…" and so it went on. Eventually, we asked a simple question: what was happening to market share? He seemed slightly annoyed. Market share wasn't relevant for a complex business like his, he said. He wasn't selling baked beans!
So we analysed his data in a different way, not drilling down into the detail but aggregating up to find the trends. And we quickly found patterns that his data-mining techniques had missed. We identified six key measures of market share, and all were in long-term decline.
It is commonly assumed that the more data you have, the better. But in our experience, the more granular the data, the harder it is to see the wood for the trees. Digital data is often daily or hourly, which makes it easier to measure short-term marketing effects. But it makes it harder to measure long-term effects, which get lost in the noise. Similarly, if you analyse sales by store and SKU, the effects of promotions seem huge. But brand-level data shows they are much smaller once cannibalisation and store-switching are taken into account.
Those may sound like arcane points, but they can have big consequences. When EPOS first gave us high-frequency, granular sales data in the late 1980s, the result was a huge increase in the use of price promotions, because the new data exaggerated their effects. We did learn that excessive promotion can erode long-term profits and commoditise whole categories, but not before the damage was done.
Retailers did rather better than brand owners out of that, as the winner was often own label. But granular data doesn't always help them either. Some years ago, we tried to talk to a furniture retailer about his year-on-year sales performance. "I haven't got time for that!" he exclaimed. "I want to know why I haven't sold any beds in Chester this morning!" Not surprisingly, his company went bust some time ago.
Granular customer data can be dangerously seductive too. Too many firms obsess over fine segmentation and tight targeting, despite the evidence that the real money comes from broad reach and high market share. And now we have digital data at the customer level. That makes it possible to customise selling messages with uncanny accuracy, as any Amazon shopper knows. And as online data evolves, our understanding of how marketing affects behaviour will deepen. But analysing data at the individual level can't measure those all-important 'herd' effects, which only emerge at the group level. And social media metrics won't solve that problem, because herd effects are mostly non-verbal and offline.
Finally, there is the problem of false positives. The more things you measure, the more fluke results you get. So granular data often shows us patterns that aren't really there. Granular data does have its uses, of course, particularly in the realm of sales activation. But it can be misleading in the realm of brand building, where the effects are broad-brush and long term. And that's where the big profits lie.
So just as we need to balance short- and long-term marketing strategies, so we need to balance granular, bottom-up analysis with broader, top-down perspectives. That means designing data systems that allow us to shift easily from zoom to wide angle. It also means employing people who can see the big picture, not just the fine detail.
Get the balance wrong, and Big Data will lead you dangerously astray. Get it right, and you can move from Big Data to Big Insights, Big Brands and Big Profits.