Les Binet and Sarah Carter get a little bit angry about some of the nonsense they hear around them… like the vogue for real-time evaluation.

As we write, it's IPA Effectiveness Awards season once again. For the past few months, agencies around the world have been burning the midnight oil writing papers proving the effectiveness of their campaigns in exquisite detail. And now the judges will have their summers ruined by having to wade through 50 or 60 of these 4,000-word tomes.

We have always taken the awards very seriously here at adam&eveDDB, but not everyone is so convinced. "The IPA Awards?" said a planner from a rival agency, recently. "Aren't they a bit out of date? Real-time evaluation is where it's at now."

There does seem to be a growing dissatisfaction with 'traditional' post-campaign evaluation, which critics say is backward-looking and overly intellectual. What we need is evaluation we can act on here and now, they say. Hence the brave new world of real-time planning – all about using 'Big Data' to create 'dashboards' that allow clients and agencies to monitor performance minute by minute, and react accordingly.

"With no longer-term perspective, it becomes hard to monitor and build the long-term, consistent memory structures that are brands"

Leaving aside the fact that most of the planners and journalists who are bandying these sexy new buzzwords about have no idea what 'Big Data' really means, there is some truth in these criticisms of 'old style' evaluation. Too often, it is merely done to win awards or justify budgets; too rarely is it used to inform strategy or improve effectiveness.

But there are dangers associated with the real-time approach too. Recent analysis of the IPA Databank has shown that short-term success metrics are NOT a reliable guide to long-term growth. In fact, focusing on short-term goals can actually reduce clients' long-term profits. The metrics that matter most when it comes to profit simply don't move much in the short term, and so are likely to be ignored when constructing real-time dashboards.

With no longer-term perspective, it becomes hard to monitor and build the long-term, consistent memory structures that are brands. Instead, everything becomes short term, tactical and reactive. Promotions, price cuts and direct-response activity come to the fore, because these yield the big, measurable short-term effects. Brand-building gets neglected. If we immerse ourselves in the here and now, it becomes harder to learn from the past, and it becomes harder to plan for the future. New products and new campaigns often take time to bed in, and can't easily be evaluated against short-term results.

Building a brand is like flying a plane: yes, you need to monitor your instruments (real-time evaluation), but you also need a map to tell you where you're going and where you've been (long-term evaluation). You need both Big Data and Long Data.

Sadly, it seems that Big Data is squeezing Long Data out of the picture. As market research data becomes richer, more detailed and more rapidly updated, databases get bigger and bigger. And keeping long runs of back data is becoming increasingly costly. With clients and agencies focused on today's numbers, and sceptical about what they can learn from the past, research companies are keeping shorter and shorter runs of data. If this goes on, we soon won't be able to do much long-run analysis at all.

One criticism of the reckless young traders during the recent banking crisis was that 'they knew too much maths and not enough history'. Let's hope our industry doesn't go the same way. Perhaps we should take a leaf out of Google Books. Its database allows users to search within texts going back to 1500. Now that's what we call data.


This article originally appeared in the June 2014 issue of Admap. Click here for subscription information.