Evaluating The ROI Of Radio And Billboards

Introduction

All too often, analytic suppliers present marketing mix results that focus on volume decomposition and neglect actionable recommendations. As ROI (return on investment) becomes the key business decision criterion, the supplier-user relationship is more focused on actionable recommendations that the business team can more readily leverage.

This paper is a case study which reviews a billboard and radio advertising analysis that produces ROI based recommendations. The advertising campaign was for Kraft Foods' Bull's-Eye Barbeque Sauce, which is a meat enhancer product and part of the United States FMCG (fast moving consumer goods) industry. Of important note in this case is the fact that recent industry consolidation has resulted in manufacturers holding more brands in their portfolios. Currently, multiple brands are fighting to justify their marketing budgets within the company's and the category's portfolio. An important implication of this trend is the need for smaller brands to find creative low cost options for equity-building marketing. Finding themselves in this situation, the brand team turned to radio and billboard advertising as a low cost equity driver. Determining the payback of the advertising campaign is the primary goal of this analysis.

 

Background

Though a relatively small brand, this product is owned and operated by a large FMCG company. Although there was past TV advertising, there was limited ad support in the years immediately preceding this new radio and billboard campaign. Immediately preceding the new advertising, the brand had seen a prolonged decline in base volume but was still relatively profitable, which suggested the need to strengthen brand equity. Despite the lack of recent equity building activity, the business team still viewed the brand as having a strong brand image and growth potential. This view is supported by a marketing mix analysis completed during the last year in which television support was provided. The mix analysis suggested that TV was very effective in driving volume; however, due to the small size of the brand it was not efficient (there was a low increase in volume relative to dollars spent on TV). The primary aim of the new radio and billboard advertising was to communicate the brand equity and drive incremental sales. Due to past learnings and limited funding, TV advertising was not considered a feasible option. Therefore, the business team looked to lower-cost advertising options: radio and out-of-home.

The decision makers consisted of representatives from multiple functions including marketing, sales, finance, consumer insights, media, and the advertising agency. Before beginning the process there was a well-developed understanding of the high value consumer and what the advertisements needed to do in order to reach these people. These insights were applied to the copy and were the driving factors in shaping the campaign message.

 

Advertising Campaign Expectations

Expectations for this new advertising campaign were less well defined than for brands that advertise on a more regular basis. The marketing team's task was simply to find equity building measures that were financially sustainable, even when considering only short-term payback. This meant that the short-term return needed only to be as good as other marketing options for the brand and or portfolio. Because these forms of media were relatively new to the brand, expectations were based primarily on general views not on brand specific learnings. Therefore, the brand team hoped that this campaign would provide a greater total return on investment than TV advertising and assumed that radio would be the more impactful medium format. Quantitative copy tests were not performed on the radio or billboards prior to the campaign, only qualitative feedback was used to shape the creative and make final copy decisions.

 

Methodology

 

Business Process (Supplier Role vs. User Role)

There are essentially two focal points to the research approach. One is the explanation of the statistical analysis and the other is the method in which the client, Kraft Foods, and ACNielsen, the analytic provider (AP), worked together. The work process was founded in a flexible yet sustained working relationship, rather than a temporary project timeframe mentality. Specifically, the ongoing relationship between client and AP means that the AP is not contracted on a per project basis, rather the AP is permanently aligned with the client's business team.

Clearly both of these analysis elements contributed to the successful measurement of ROI. Kraft places a high degree of importance in the capability to quantify the ROI of marketing activity. This tight alliance between client and AP ensures that this capability is present in every business group and is regularly employed in marketing planning.

As noted above, analysis planning was a collaboration between client and AP. The primary specifications were all agreed upon by both parties prior to the analysis. The analytical work, in this case the statistical estimation of volume response, was completed by the AP and delivered to the client. Any issues or concerns were then discussed and resolved. Once volume estimates were agreed upon, the client and AP worked together to translate volumetrics into ROI and recommendations and to put the findings in proper perspective. Communicating the results was also a joint effort. The presentation was led by the AP, and the implementation of recommendations and follow-up questions were championed by the client.

 

Statistical Analysis

 

Data

The data came from two primary sources: the syndicated data supplier (who is also the AP) and the media company. In this case, the media company is independent from the advertising agency that developed the copy.

The media data consisted of GRPs (gross rating points) for both radio and billboards. The radio advertising level was measured by Arbitron, and the data was collected with a diary method. In this case, the GRP is a function of the number of people that actually heard the radio advertisement. In contrast, billboard or out-of-home advertising level is a function of traffic counts which measure the number of people that have the potential to see the advertising, but may or may not have. Advertising level for billboards was computed by utilizing Daily Effective Circulation (DEC), which is the fundamental unit of measure for out-of-home advertising. The DEC is the average number of individuals 18 years or older who were exposed to the advertising on a daily basis (DECs are measured by the Traffic Audit Bureau, or TAB). Total DEC over Total Population produces the percentage of population exposed to a given advertisement on a daily basis, commonly referred to as the showing level. Finally, the showing level multiplied by the days in the advertising period (in this case the data was collected weekly) produces GRPs, where GRPs are the total number of impressions delivered by an outdoor showing as expressed by a percentage of a population. Because the GRP measures were different between the two mediums, we initially treated each medium as a separate variable in the model. Importantly, none of the advertising measures included the quality of the advertising copy itself. The quality of the copy is expected to be reflected in the volumetric response. Better copy should yield a better volume lift than advertising with poor copy quality.

As barbeque sauce is a highly seasonal category, most promotional activity for the category occurs during the summer when volume is at its peak. This advertising program was generally aligned with the seasonality of the brand. The advertising started at the beginning of the season and continued into the second half of the season.

All other data came from one syndicated data supplier. This includes volume, pricing, trade support activity (promotion execution measures), and consumer promotions (Free Standing Inserts, coupons, in-store programming). (FSI data verification came from the manufacturer or promotion agency.)

The analysis used pooled market-level data. This is often referred to in statistical literature as a cross-sectional time series, or panel data (not to be confused with the use of the term “panel” in home panels). Much consideration also was given to a model using store level data. Store-level data are certainly superior for estimating price elasticity and all programs that occur within the store, but they do not do as well at measuring market-level effects such as advertising. One could certainly argue that market-level effects such as radio and billboard advertising need to be measured with models that allow for both market and store level effects; however, in this case the advantages of more complex data structures were not deemed large enough to warrant its use.

It was still necessary to model all effects for control purposes and all results were compared to existing store-level findings for verification. Because the question at hand relates only to market-level facts, the market-level data were sufficient and provided a more timely result. Additionally, past measurement of advertising for this brand was based on market-level data, which means that comparison of this campaign is reasonable with market level modeling.

 

The Model

The estimate of incremental volume due to advertising came from a loglinear multivariate regression model (see equation 1 and 2). Standard diagnostics were employed to evaluate the model, improve model fit, and minimize collinearity and serial correlation.

 

Model equation

 

See model equation 1.

 

X is the vector of independent control variables including price, promotion, etc., and e is a serially uncorrelated residual.

 

Results

The initial results are the volumetric impact of advertising that was estimated through the statistical modeling (equations I). From this estimation comes the final deliverable: the business metric used to make decisions (ROI). The statistical findings were decomposed in order to allocate volume to each marketing mix element. This process was completed twice. The first scenario treated radio and billboards as independent variables by reading them separately. The second scenario combined radio and billboards into one advertising variable and measured the impact of the entire program. The second formulation was intended only to verify results. Although the radio and billboards did not run at exactly the same time and level, they were very similar, resulting in collinearity in the model. As previously discussed, exposure to radio and billboards were not measured in exactly the same way, therefore, combining these data streams is not ideal. The second model specification (equation II) was only used to verify that the collinearity between radio and billboards did not substantially impact the results, which it did by producing the same volume estimate for total advertising as was found with equation (I).

The first surprising finding is that the model was able to distinguish the effect of radio and billboards when treating them independently. Often when one advertising campaign consists of multiple media vehicles the channels run concurrently, which prevents separate reads of the individual effects. Fortunately, this was not the case here.

The second surprising finding is that radio drove only slightly more volume than billboards. This is surprising because radio was generally perceived to be as impactful as billboards but three-fourths of the money was allocated to radio. Percent lift, calculated as incremental pounds over base pounds, serves to adjust for the fact that radio and billboards did not run in exactly the same weeks. Thus, radio produced only slightly more absolute and relative volume than billboards. In order to substantiate this finding, the second model was run. Now radio and billboards were treated as one combined advertising variable. This model confirmed the initial findings by producing the same volumetric impact as the sum of the radio and billboard effects from the independent model.

The next step was to take this raw result, incremental volume, put it in perspective relative to past findings, and communicate to the business team what it means. The primary metric here is ROI. Whereas radio did drive more raw volume, billboards provided a significantly higher return on investment due to their lower costs. When comparing these returns with past programs for this brand, the ROI for radio was approximately equal to a less successful TV campaign that executed four years earlier. This previous TV campaign was one of the lowest returns on record for the brand in question and much lower than current return on non-advertising activities such as consumer or trade promotions. In contrary, the return on billboards significantly outperformed a different, relatively successful, TV campaign that ran approximately two years prior to the new radio and billboards. In addition to comparing against past advertising, billboards provided a return that was approximately equal to average consumer and trade promotion ROI for this brand. However, the short term ROI on billboards was still lower than the best non-advertising alternative. For the overall campaign, radio and billboard generated approximately the same return as the last TV campaign. (See figure 1.)

 

There are several important caveats to keep in mind when reviewing the results. First, there were three test markets with this advertising. Two received both radio and billboards, the third received only radio. To obtain a reasonable number of observations it was necessary to use a pooled model rather than individual market models. This means that the estimated response rates are averages of the three test markets. This does not mean that we are assuming all the markets are identical. The data do allow for market differences, and the model controls for market differences via individual market intercepts. Although comparing the market intercepts does provide some insight into which markets had a larger impact on the results, this should not be interpreted as an alternative way to measure individual market responses. The second issue to consider is that the model specification that combines the advertising data, equation (II), would generally be thought of as much more robust than the model that isolates radio and billboards. Fortunately, the volume estimates from the two model specifications differ by only one percent, so model specification choice is not an issue in this situation.

 

Business Implications

As discussed earlier, the business goal addressed by this analysis was to find an equity building activity that provides an acceptable short-term payback. This model confirmed that the advertising campaign provided a payback that was equal to other marketing activities in terms of short-term payback. While not estimated by the model, it can also be assumed that this advertising also contributed towards building equity and hence long-term franchise strength. This long-term effect of advertising makes it difficult to directly compare the ROI from advertising to the ROI from other marketing mix elements that do not have a long-term effect (i.e. consumer promotions and trade promotions). To bridge this gap, other considerations were included in the decision of how much to invest in each element.

The analysis also provided strong evidence that these billboards outperform most examined alternatives. While this model does not explain why billboards outperformed radio, it does provide the business team with guidance as to what qualitative questions need to be answered. Specifically, the marketing managers now know that these billboards provided a higher return than expected and they can search for qualitative intuition behind this. Possible explanations are likely to come from differences in copy or the type of consumer exposed to the advertisements. If the billboards were reaching more receptive consumers this may explain the gap. It also may be the case that radio and billboards are not typically executed equally well, but when they are, the cost savings from billboards drives the higher ROI.

The more general implications for the business team are based around the following question: why did this advertising work so well despite a lackluster history for this company's billboards across all brands? Answers here also do not come directly from the model. One possible explanation is that this brand was in an ideal situation to use billboards. Specifically, past advertising was effective, meaning it drove volume, but it was not working against a large enough base to produce a justifiable short-run ROI. This suggests that consumers of this brand are receptive to quality advertising. It also means that costs must be managed to allow for an acceptable short-run return. This is a truly powerful learning because it enables equity building activities in times when marketing budgets must be justified in the short-run. It helps the team focus advertising dollars not only where they will drive volume, but where they will drive profits.

 

The Importance of the Business Relationship

In addition to the results and methodology, a key take away from this case study should be the importance of the AP/client relationship. As mentioned earlier, the AP and client have an ongoing relationship that is not contractually tied to individual projects.

 

Benefits

The long term nature of the working relationship allows the AP to be up to date on relevant business issues before the project begins. This knowledge provides many benefits. For example, there is increased dialogue between the AP and client throughout the analytical phase. This saves a significant amount of time in the project planning phase. It also helps ensure that all important factors are controlled for in the analysis. The time savings from a permanent relationship and a dedicated analytic partner allows for project delivery in a fraction of the time typically needed for external projects. This means results can be delivered as planning deadlines approach, rather than once a year. Additionally, the fact that the client incurs no incremental dollar cost per project means that greater analytical risk can be assumed. In this case, past attempts suggested that outdoor advertising may not be quantifiable at all which means that spending precious research dollars would be hard to justify. The ongoing relationship between AP and client allowed the client to take a chance and attempt to analyze the program.

The close relationship also allows for the AP to establish a rapport with the greater business team. This trust and understanding makes the communication process more efficient and the AP more valuable. It also allows the AP and the client to learn from each other and share knowledge.

 

Drawbacks

The potential drawbacks are certainly outweighed by the advantages but are still important watch outs. The closeness of the relationship has the potential to skew the normal objective viewpoint of the AP. For example, by working closely with a business team over time, the AP can also feel vested in the performance of the marketing activity and may be susceptible to pressure to find favorable results. While this is a possibility, the division of responsibility between the AP and the client helps to minimize this risk. Whereas the presentation is very much a joint effort, the primary volume responses are owned by the AP. The client is able to shield the analyst from the team's expectations until the statistical estimations are complete. This means that evidence of weak performance can be presented as confidently as evidence for strong performance.

 

Conclusion

 

Keys to Success

There are several important factors that aided in the success of the project. The advertising data used to quantify the exposure to billboards and radio provided specific enough information to distinguish the respective effects. The statistical methodology and the relationship between AP and client allowed for sufficient control of all other factors. The majority of past attempts to quantify billboard ROI were based on a test market vs. control market style advertising test. This campaign was long enough and in enough different markets to allow for pooled market regression analysis. This pooled market structure was likely a key factor in our ability to read each marketing element independently. An ongoing AP/client relationship that is not based on incremental project costs allowed for the freedom to take a risk and complete the project. Additionally, the analysis was likely able to provide more detailed findings because the brand in question is generally receptive to advertising, but has not had any in the several years preceding the campaign. In other words, it may be the case that billboards and radio are often found to be difficult to quantify because they are used in support of brands that are less likely to succeed. This does not mean that they are inferior brands but simply that the brands with high volume and known advertising responsiveness usually receive TV and Consumer Promotions, whereas the brands that are smaller and therefore less likely to provide high ROIs receive programs such as radio and billboards.

 

Outstanding Issues

Despite the wealth of information gained here, many important questions remain unanswered. One of the most obvious is that we have only examined the short-run return. Due to the media planning process, it was not possible to wait long enough to evaluate the long-term impact on brand equity. It was also not possible to specifically address the potential presence of a synergy effect between radio and billboard. Therefore, while we have significant evidence that billboards provided a higher return than radio, we cannot say with certainty that billboards alone would have provided a return equal to that of this campaign. Finally, although this information is useful for future planning and for decisions relating to category management, it cannot be generalized across brands and circumstances.