Les Binet and Sarah Carter get a little bit angry about some of the nonsense they hear around them… like the Rosser Reeves Fallacy.

Recently, a young planner came to us with some exciting research findings. She was trying to evaluate the effect of her client's new ad, and had found a fantastic piece of evidence. A research company had shown that people who recognised the ad had a much better image of the brand, and were significantly more likely to buy it. Perfect proof that it worked. Surely?

Grrr… This particular nonsense has cropped up with monotonous regularity for over 50 years. It's called The Rosser Reeves Fallacy. And it's still alive and kicking in 2014.

Rosser Reeves was one of the most famous ad men of the 1950s. In 1961, his book Reality in Advertising outlined a simple method for measuring ad effectiveness. Take a sample of your target audience. Ask them about your brand. Then show them your ad and see how many of them recognise it. Compare brand scores for ad recognisers with brand scores for non-recognisers. Hey presto, there's your ad effect!

Reeves' method is quick, simple and nearly always shows big ad effects. It is also completely flawed. The problem is that people are more likely to notice and remember ads for the brands they know well and like. So usually ad recognition is the effect. Not the cause.

You must have noticed this yourself. You buy a new car. And suddenly you see ads for it everywhere. You start working on a brand. And suddenly you see it everywhere you look. Familiarity with a brand primes you to notice and remember its advertising. So even if you exclude brand users, you will always see a spurious correlation between brand and ad responses.

Reeves' methodology was discredited decades ago. But as our young planner found, research companies continue to peddle it, and clients and agencies continue to be seduced by it. Worse still, it now seems to be mutating into a new, more dangerous strain.

Two years ago, a huge research project looking into the effects of online marketing for FMCG brands was published. Using a massive panel of a million internet users, it proved conclusively that people who visit a brand's website are more likely to buy that brand, and spend more on it than other buyers.

But this is just another new incarnation of the Rosser Reeves problem: correlation is not proof of causation. In fact, closer examination of the data reveals that people who visit a brand's website tend to spend a lot on the category. They are actually less loyal to the brand, and tend to pay less for it per purchase. This suggests they are deal hunters – probably what brings them to the website.

Similarly, we often come across research purporting to 'prove' the effectiveness of social media by showing that a brand's Facebook fans tend to be heavier buyers. Yes, they are. That's why they become fans.

These new digital incarnations of the Rosser Reeves Fallacy are particularly dangerous. Because they focus on heavy buyers, they lead to a flawed emphasis on loyalty over penetration, targeting over reach, and price promotion over brand building – all strategies proven to be less profitable.

The right way to deal with all these spurious correlations is by exposure. Take two identical groups of people. Expose one lot to your advertising and leave the other group unexposed. Then compare attitudes and behaviour between the two groups over the course of the test. These controlled exposure tests take more time, effort and money. But they produce more accurate and sensible results – showing that advertising does work, but not always. And revealing that short-term ad effects are modest, but still worthwhile. When applied to digital activity, they seem to produce similarly realistic findings.

This is the true 'Reality in Advertising'. Perhaps someone should write another book?


This article originally appeared in the April 2014 issue of Admap. Click here for subscription information.