Farm Road, Henley-on-Thames, Oxon RG9 1EJ, UK
Tel: +44 (0)1491 411000 Fax: +44 (0)1491 571188
|
Published by NTC Publications Ltd Farm Road, Henley-on-Thames, Oxon RG9 1EJ, UK Tel: +44 (0)1491 411000 Fax: +44 (0)1491 571188 |
Vol 18 No 2 (1999) |
This paper argues that much current campaign tracking and econometric modelling is systematically undervaluing good agency planning practice and the potential of media advertising. This is not a reflection on the detailed execution of such projects, but rather on basic design weaknesses and a concentration on average achievements. Unlike other marketing inputs, such as price and distribution, advertising is complex and multi-faceted. For this reason it is unreasonable to expect that it is possible to represent the advertising input adequately with a single expenditure or gross rating points (GRPs) figure. The paper identifies seven component factors within any campaign that could and should be incorporated as analysis factors in any assessment project. The problems of achieving this desirable objective are considered, and the likely benefits discussed. The paper also considers possible frameworks whereby the necessary co-operation between advertisers, agencies and media owners could be achieved.
The increasing demands from advertisers for research that will enable them to monitor the effectiveness of their advertising expenditure, partly to justify this expenditure and partly to learn how to improve its performance, has led to increased interest in two specific forms of research - campaign tracking studies and econometric modelling.
Although in recent years considerable progress has been made in the design, analysis and interpretation of such work, especially with respect to television advertising, this paper argues that much of this work is systematically undervaluing media advertising and the contribution of good agency work. The argument is not structured in terms of detailed design and execution, but rather on the basis of a broader look at what is being done and how it might be improved. Inadequate research will lead to wrong decision-making and cause harm to all involved in the advertising industry, including the advertisers.
This paper is concerned with both individual studies and the increasing extent to which attempts are being made to generalise across studies. Such attempts are vital, because they are the only way in which it will be possible to provide a better starter-kit of knowledge which agencies and brand managers can use when making campaign-planning decisions, but only if the work is well based. For example, it is not a question of the extent to which a model fits the measures taken from the market, but rather the risk that too blinkered a view will fail to reveal the significance of a number of factors which should be taken into account. This paper debates some of the most important of these factors, and suggests ways in which the position could be improved.
In essence, most current attempts to model across campaigns will produce planning guidelines based on average performance levels. However, no agency should be satisfied with planning an average campaign, no advertiser should dream of accepting that they are going to run such a campaign, and no media owner should allow their medium to be judged by the performance of average quality campaigns. The ideal level of campaign being discussed is not the one-off outstanding success with a flash of genius, but rather sound, professionally planned and executed campaigns with results in the top 20 per cent of results. The reality of what happens in practice is a quite different issue, but the objective must at least be to aim for such a target.
One example of generalising across campaigns is given in a paper by Adrian Weser (1997). He describes a modelling study based on five years of Nielsen data on market shares and distribution levels for 147 packaged-goods brands, and advertising expenditure levels and the media used. A regression formula was calculated which accounted for 98.6 per cent of the change in market share of expenditure between one period and the next. It showed that the market share in one period was heavily dependent on the market share in the previous period, with lesser contributions made by variations in the levels of distribution and expenditure in four media - television, print, radio and posters. The outcome of this work is shown in Figure 1.
I make no challenge to the mathematical soundness of this work. But I do argue that any work along such lines will undervalue all aspects of the media advertising input because it ignores the extent to which advertising expenditure is far more variable in its effect on changes in market share than price (taken into account because the market share variable is based on expenditure) and distribution. It is inevitable that such an approach will result in the potential benefit from media advertising, and the contribution made by good agency planning and execution, being undervalued. It encourages people to think of media advertising as a commodity, with budget size or GRPs being the only important dimensions.
The main focus of this paper is on econometric modelling, because most of the missing factors which are described apply to all media. However, before concentrating on econometrics, it is worth recalling the way in which Millward Brown, one of the leading international research companies in these fields of activity, acknowledged errors in the way it had been tracking campaigns in magazines, the way in which it put matters right, and what happened when it did so.
In January 1994 Gordon Brown, one of the founders of Millward Brown International, wrote that most previous campaign tracking and related research, including its own work, had systematically undervalued the extent to which magazine advertisements actually gained attention (Brown, 1994). He attributed this research failure to two weaknesses: one was concerned with the way in which the advertising effort is deployed over time, as far as individuals in the marketplace are concerned, and the other with the wear-out of individual creative treatments for print ads. These two considerations are debated later since they are also relevant to econometric modelling, but at this stage three comments can be made:
The implication of these comments is that there is a high risk that the undervaluation which Gordon Brown wrote about in 1994, and which was first described in 1992, is to a large extent still present in magazine campaign tracking research.
The usual aim of any econometric modelling process concerned with marketing is to identify those factors which appear to impinge on sales, to establish their significance and to study the relationships between them.
In most published models the main factors involved are price, distribution, promotions and advertising, either in total or split between the media involved, with each one being expressed in some appropriate fashion so that it can be accommodated within the mathematical modelling process. In the case of price and distribution this does not present any major problems, because these two factors are simple ones - they can be measured either relatively or absolutely and they go up or down. As such there is no difficulty in including them in the modelling process.
In contrast, advertising and, to a lesser extent, promotions are complex activities which are quite inadequately represented by a single figure, whether it be expressed in terms of expenditure or rating points. While it is certainly true that the size of the advertiser's budget is an important consideration in determining the results achieved, there are a large number of other planning and executional factors which need to be taken into account if there is to be any hope of accurately modelling the way in which a market responds to advertising pressure. The failure to make adequate allowance for these considerations is the simple explanation of the following major conclusion, given in a recent report by Bucklin and Gupta (1998) on the usefulness of scanner data:
Most practitioners expressed high levels of frustration with the inability to get clean, consistent answers about advertising effectiveness from scanner data.
What did these practitioners expect if, having spent hours discussing campaign strategy and execution, they end up by attempting to monitor the effectiveness of all their decisions by using a single input figure?
In a recent paper, Paul Dyson of Millward Brown (1998) summarises the conclusions drawn from around 60 different sales-modelling projects over a 15-year period and relating to a wide variety of product fields. Within each project Millward Brown has analyses of the sales effectiveness of each advertisement to give a database of top-line information for almost 300 ads. This material has been related to the results from its tracking studies, primarily the Awareness Index (AI), and the first crucial conclusion drawn from this database is that there is no simple relationship between the AI and sales. The correlation is virtually zero, a finding which is totally in line with the above comment from the Bucklin and Gupta report.
However, if allowance is made for six factors that Dyson identifies as being relevant there is a correlation of 0.87 between the AI and sales movements for established brands. Three of these factors are of a general nature and need not concern us further; they relate to the time period covered by the research, the size of the market, and various aspects of the nature of the brand and product field which are largely ingrained.
However, the other three relate to advertising and are directly under the control of
the brand manager and agency planners. It is argued that if the component elements of
these three factors, as described below, were routinely taken into account in tracking and
modelling activity there would be two major benefits:
It is already well known that some campaigns produced much higher levels of marketplace
response than others, but the work proposed would make it possible to generalise in a much
more comprehensive way about the characteristics of the top performers. This would enable
planners and brand managers to make more informed decisions about future campaigns. Better
results would attract larger media budgets. It is reported (Naples, 1998) that from
1992-94 Kraft Foods redeployed US$125 million of marketing effort from trade promotions
and free-standing inserts towards media advertising as a direct result of advanced
analytics.
By concentrating attention on the results achieved by the top-performing campaigns, better
indicators would be provided of the potential ROI on media advertising. It would provide
media owners with an improved platform for their sales activities.
It should be noted that all the listed component elements can be deduced from the external evidence of the campaign. They do not require those involved with the assessment to make use of any confidential information from the client and agency, although the co-operation of these parties would make it far easier to achieve the categorisations being proposed. There will, of course, be other campaign elements which will be equally or more important, particularly the nature of the responses to the advertiser's message, but it would not be feasible to think of including such elements at this stage.
Although the sample of different types of advertisements available for Dyson to study was limited he reports clear evidence that launch ads, those that have something new to say about the brand, and those linked to promotional activity tend to do better than might otherwise be expected. In which case, such advertisements need to be treated differently from mainstream campaigns.
There is a second reason why it is necessary to give promotional activity special consideration. It is reported by one major company operating a home-scanner panel (Taylor Nelson AGB UK, 1998) that the reported level of promotion-affected purchases by their panellists is lower than it should be, because of reporting failures. Hence, and after a great deal of development research, they have devised an algorithm which they use to determine what they report as promotion purchases. In essence this algorithm is based on the size of the blips experienced in the week-by-week purchase levels.
While such statistical manipulation could well be the best way to account for the
short-term variations in purchase, it has an unfortunate side-effect when it comes to
comparing results from different marketing inputs. This is because it only identifies the
reactions to successful promotions; i.e. those that cause the market to move. This means
that if some promotional activity is unsuccessful the Taylor Nelson AGB algorithm will
overestimate the effect of promotions, relative to other media.
The equivalent strategy for advertising would be to ignore all expenditure on campaigns
that did not produce measurable results. While some might argue that this would be a good
thing, it is not a sensible approach for studying the total picture.
The Awareness Index is one of the key measures employed by Millward Brown in their tracking projects. It measures the efficiency with which a campaign generates prompted awareness that the brand has been advertised in a particular medium. It tells us nothing about what message has been communicated, nor about the reaction generated, but nevertheless it is abundantly clear that awareness that the brand has been advertised is a vital prerequisite for the ad to work. For established brands, for example, Millward Brown have found that 77 per cent of the variation in the short-term return per exposure is accounted for by how well the ad communicates the name of the brand being advertised (Dyson, 1998). In the face of this evidence it is clear that as an explanatory factor an indicator of post-exposure brand-name communication should be incorporated into all tracking and modelling projects. This will add costs, of course, but without such an input factor a key analysis ingredient will be missing.
This same point is made in a paper on tracking studies by the German researcher Vöhl-Hitscher (1998). He concludes that about half of the 500 campaigns that IVE Research International is currently tracking in Germany are failing to achieve adequate returns from their ad expenditures, because the Awareness Index for the brand is too low.
One of the reasons for the low average level of brand-name communication is that most forms of ad pre-testing involve some form of forced exposure. This does not invalidate the test as far as attitudes and message communication are concerned, but forced exposure is likely to result in higher levels of attention to the ad than would occur under natural exposure. Hence much pre-testing is not good at checking the level of brand-name communication.
The likelihood that individual creative treatments for print ads experience a fairly rapid decline in impact with repeat exposures is a more subtle factor often leading to the undervaluation of print advertising. The original work in this area was again conducted by Millward Brown. It found that even after making due allowance for readership time lag (explained below), it was not able to achieve a good fit between the market responses it was getting for print campaigns and the advertising input until it made allowance for the possibility that print creative treatments 'wore out'.
This was quite contrary to its experiences with television. In the case of this medium, Millward Brown spent the early years of its tracking research looking for evidence of the wear-out of television commercials, but after three to four years' searching the company eventually concluded, to its surprise, that each time a TV ad appeared it would produce a very similar response. Initially this assumption was transferred to its work on print, but eventually its computer modelling led to the conclusion that print ads did wear out.
If the first and second exposures were taken as generating `full value', the third exposure was worth only 75 per cent, the fourth 25 per cent and the fifth plus only ten per cent.
The main explanation for this difference between the media would seem to stem from the extent to which readers are in total control of the time they spend with any item in a publication, whether it be editorial or an advertisement. One of the strengths of print advertising is that a reader can stay with an advertisement for as long as it holds their interest, and can return to it if need be, whereas if they are not interested in the brand, or have seen the advertisement before, they can move on in a fraction of a second. As a consequence, the number of individual print creative treatments used in a schedule plays a key role in the ad awareness levels which it generates. This needs to be taken into account.
The importance of this factor can be illustrated with a finding from the Kenco regional advertising test conducted by Millward Brown (Smith, 1997). Using their wear-out findings for individual creative treatments of print ads, Millward Brown calculated that the difference in performance between the campaign as it was actually run with six treatments, and what would have been achieved if it had been run as first planned with only four treatments was a gain of 35 per cent - up from 63 per cent efficiency to 85 per cent (see Figures 2 and 3).
One of the findings of the eight research studies into the potential communication benefits of combining television and print advertising (conducted in the early 1990s under the auspices of the International Federation of the Periodical Press - FIPP) was that the level of benefit was largely dependent on the extent to which the creative treatments were judged to be in harmony (Smith, 1991). This finding has not yet been adequately explored to establish how best to use these media in combination, but one way to isolate its significance would be to take the level of creative integration as a factor in advertising assessment research. This would have to be done on a subjective judgment basis into, say, three categories of high, medium and low harmony. This is somewhat fuzzy at the margins, but better than ignoring the factor.
On a broader front it might also be commented that the interest in integrated marketing communications is likely to highlight the issue of how best to integrate, in creative terms, the Internet with the traditional media.
This is not an issue for television, because to all intents and purposes the moment of transmission is the moment of impact, but this is not the case for print, especially monthly magazines. It is self-evident that the total number of readers attracted to many monthlies will be accumulated over a much longer period of time than the week or month of publication. For example, based on its own special study (Pincott, 1992) Millward Brown estimated that half of the total readerships of a group of titles they called 'fat quality monthlies' will not see an issue until nine or more weeks after the publication date.
Figures 4 and 5 illustrate the way in which this time-lag in the build up of readership could alter perceptions of how the effort from a magazine schedule of 12 insertions in six fat quality monthlies and four in two programme papers would be changed. It is clear that the market movements to be expected from the revised input profile would be quite different from those generated by the conventional approach.
Is it not surprising that in the second edition of its How to Plan Advertising handbook (Cooper (Ed), 1997) the Account Planning Group (APG) should find it necessary to write:
The tools for allocating budgets between the different communication channels are rudimentary and thin on the ground. A good deal of first-principle thinking still needs to be done.
The APG describes itself as the world's foremost and longest-established organisation representing the interests of account planners worldwide. Its handbook is written by a group of the most active and respected account planners in London who should know what they are writing about.
It is suggested that the only way to achieve such first-principle thinking on an informed basis would be by routinely taking the budget splits into account in all appropriate assessment projects.
In simplified terms all campaign-planning decisions can be grouped under two headings - those concerned with communication (what to say and how) and targeting (who to speak to and how often). These are the twin levers which agencies operate, but the truth of the matter is that there is little or no knowledge as to their relative power. Currently much more money is spent on targeting research, primarily audience and readership studies, than on communication projects, but is this the best allocation of the budgets available for media research?
Putting on one side the level of brand-name communication, different planning strategies would be employed for any mixed-media activity, depending on whether the mix is being used primarily for targeting or communication benefits. For example, if magazines were being used alongside television to boost the campaign's impact among light television viewers, the appropriate strategy would be to concentrate on titles read by lots of light viewers. However, if the aim was added communication power the best titles to use would be those that were good at reaching television viewers. If the selection strategy employed in such cases was treated as an analysis factor, our understanding would again be improved.
Probably not many readers will remember that around 150 years ago the Rothamsted Research Station, now called the Institute of Arable Crop Research (IACR), was responsible for the development of the basic theories of sampling widely employed in market research today. What is more, we can still learn from the same establishment.
In studying crop returns the IACR would customarily take into account a much wider range of variables than are currently used for advertising research. True, most of their variables are easier to measure physical ones, while it is also much easier to monitor the yield of farm crops, but the resources available today for monitoring advertising are much better than only a few years ago.
Thanks to the flow of scanner data and vastly increased computer power we are just beginning to do what has been commonplace in agriculture for many years.
The basic theme of this paper is that, because advertising is such a complex and multi-faceted activity, little progress can be expected if the advertising input into campaign tracking or econometric projects is restricted to a single expenditure or exposure figure for each medium. What is required is a much broader approach to the problem, with the object of making it possible for the researchers and modellers involved to take into account the seven variables listed above, at least. There may be others which are equally important. This will not be easy to achieve, because it will cost money and will require a level of co-operation between advertisers, agencies and media owners which does not exist at present.
Normally in an industry the responsibility for research and development into the basic product performance lies with the producer of the product, but in media the media owners currently feel no such responsibility. Although they spend much money on work to specify the size and nature of their audiences, such work is not research and development. It needs to be carried out but is not sufficient. It should be seen as the equivalent of glossy car brochures, but is far removed from the research and development work undertaken by the car manufacturers which has so much improved the performance of cars in the last 20 years. To put it bluntly, the media owners should be taking the initiatives because their interests will be most damaged by the extent to which current advertising evaluation work is systematically undervaluing the potential of their product, be it advertising sold in terms of space or time.
Although responsibility for basic research and development into the performance of media advertising clearly lies with the media owners, they cannot undertake the work on their own. It must be done in the marketplace with real-life advertising campaigns and this means with the co-operation of advertisers and agencies. And it also needs to be carried out across media, because increasingly media will be planned on an integrated basis, and across countries. This is due to the fact that more and more advertisers are planning marketing strategies on an international basis, and because shared research will generate larger databases.
Such an ambitious programme will not be easy to achieve, but unless there is a clear idea of the direction in which the industry needs to move, progress cannot be maximised.
There will also be problems of confidentiality, but these can be overcome if the right frameworks are established in the first place. At least two such frameworks already exist. In America the Managing Advertising Expenditure (MAX) project has been initiated by the Marketing Science Institute and the American Association of Advertising Agencies to explore how research could be developed to improve the practices and processes of advertising budgeting (Root & Naples, 1998), while in the UK the research and development initiative under Professor Andrew Ehrenberg from South Bank University, London could also provide a suitable umbrella organisation to co-ordinate what needs to be done. MAX has been described as a journey rather than a destination. That is good thinking.
The biggest problems will be to stimulate management enthusiasm for the potential benefits which will flow from a much better understanding of the inner workings of media advertising, and to convince the finance directors, especially within media owners, that the return on well-designed research and development into advertising activity will be high.
Account Planning Group (1997) How to Plan Advertising, Cooper, A. (ed.). London: Cassell.
Admap (1998) 'Advertising tracking', September, 49-62.
Ad Track (1994). IPC Magazines Ltd, London.
Brown, G. (1994) 'The awareness problem', Admap, January.
Bucklin, R. E. & Gupta, S. (1998) 'Commercial adoption of advances in the analysis of scanner data', Marketing Science Institute Working Paper, 98-103.
Dyson, P. (1998) 'Monitoring advertising performance', Admap Conference, January.
Naples, M. (1998) 'Best practices in the measurement of advertising expenditures', American Association of Advertising Agencies.
Pincott, G. (1992) 'An examination of wearout and exposure over time', Millward Brown International, June.
Root, P. & Naples, M. (1998) 'How should advertisers budget? First steps on the MAX journey', Admap, September.
Smith, A. (1991) 'More effective advertising by integrating magazines with television', ESOMAR Journal, Marketing and Research Today, June.
Smith, A. (1997) 'More food for thought', Admap, February.
Taylor Nelson AGB UK (1998) Unpublished correspondence with the author.
'Users and buyers guide to advertising tracking' (1998) Admap, September.
Vöhl-Hitscher, F. (1998) The Real Value of Advertising Tracking Studies. IVE Research International, Hamburg.
Weser, A. (1997) 'Advertising increases market shares and media mix achieves most'. ESOMAR Publishing Seminar, Lisbon, November.
 http://www.warc.com | © Advertising Association |