As part of the WARC x Spotify “Getting your brand heard” report, Brad MacDonald, Head of Digital Performance, Mobile at Virgin Media O2 explained the challenges and opportunities of measuring media investment in the digital audio space.

Headshot of Brad MacDonald

Brad MacDonald, Head of Digital Performance, Mobile, Virgin Media O2

What are the primary goals, KPIs and metrics that you’re targeted on?

My role is primarily focused on the performance aspect of digital media and achieving any trading objectives we are targeted with, so ultimately the KPIs will ladder back up to the commercial goals of the business, whether that’s net growth of our connected base or maximising customer value. When we think about ROI, we consider what level of profitability our activity might be driving, which then filters down to the specific levers we can pull across our paid and owned channels. It's not so much about tracking micro or softer metrics, rather we try to tie all metrics back to commercial outcomes, tracking things like cost per order using digital attribution, alongside a number of other metrics like measuring the quality of traffic that's coming through to the site to be converted.

How do considerations like scale, targeting and automation factor into what you're trying to do?

The challenge we face with measuring effectiveness is that we are still using a last touch digital attribution model. Before the joint venture we did have multi touch attribution at O2 and we were in the process of upgrading to a new multi touch attribution tool which would have given us greater understanding of the contribution of different media touchpoints, as opposed to just relying on a last touch view of effectiveness. Given we're talking about programmatic audio, unfortunately it doesn't typically look favourable using last touch attribution, undermining its true value and perceived contribution for driving desired outcomes.

In previous roles where I’ve had multi touch attribution up and running, we would look at whether we are getting the optimal reach and frequency amongst the target audiences that we were after – whether prospects or existing customers - and measure the impact of flexing these 2 metrics on our core KPIs of sales. For a business like Virgin Media O2, with two substantial and valuable customer bases, one benefit of bringing the two brands together is the ability to cross sell to our bases. To do this effectively, you're very dependent on your first party data and you need a lot of scale. When you’re matching that data through to any DSP or potential other partner, you need to have enough data to make a viable and worthwhile exercise.

But the starting point is always your measurement methodology and whether or not things stack up, which unfortunately is part of the reason why our team currently isn't looking at digital audio from a purely performance point of view. Although, it hopefully will in the future once we've got an aligned approach for digital attribution.

How does your goal setting ladder up into your media selection processes?

There’s a common theme here around measurement and attribution. As a performance team we are tasked with understanding that if we need to achieve X number of orders within a quarter, then we then need to go through the process of breaking down the budget required to achieve this target, based on what the expected cost per order will be. We then translate this investment into our activation plans and channel selection. That definitely does skew some of the thinking towards the areas that can track a little more reliably. Social is a good example. There are certain social platforms that are further advanced in terms of how they work with us and the data we can get back. This can skew our investment towards those platforms where we get greater visibility, as we can invest more confidently in them based on the results we track.

That said, the nice thing on the O2 side, and more recently on the Virgin Media side, are multiple measurement capabilities to better inform where we should be spending so it’s not purely dependent on digital last touch CPOs, but also media mix modelling and testing. That's helped us to understand that although we can't always see our inflight cost efficiency across certain areas, we can still invest into them knowing that if we do look at econometrics that it would suggest that this is a sensible area to go into. This gives us more flexibility and as soon as we can improve our digital attribution capability stack it should hopefully make it even easier.

How does experimentation with different media channels tend to work within your broader media selection and budgeting processes?

We typically use a multi-faceted measurement framework, one not reliant on any single methodology but rather complementary methodologies to provide us with a steer on the best direction. For example, our Brand and Marketing team use a combination of econometrics along with test and control studies, while my team will primarily look at digital attribution. While the absolute results will differ between all three methodologies, if they’re all trending in the same direction then this suggests whether something is actually working as intended and if it warrants an increase in investment.

A multi-faceted measurement framework also enables us to test for and identify areas of true incremental benefit, something which might be easily overlooked or misunderstood if reliant on purely a single measurement approach. For example the team who runs our econometrics also runs comprehensive geo testing, meaning we have the opportunity to apply their approach to testing different digital optimisation tactics typically used on our plans, to understand which one delivers the greatest ROI.

A great example of this is testing a conversion optimisation KPI, which you typically use to drive sales, vs. a reach optimisation KPI. While you may naturally assume the conversion KPI would be the more effective metric, and looking purely at last touch digital sales might suggest this to be true, the reality is that not all of these sales will be incremental as the algorithm will look for anyone it thinks will buy, regardless of whether your advertising will have an influence or not.

Therefore, although the conversion KPI might look best using last touch digital attribution, geo testing may suggest – and frequently does - that it's not the best thing to do. Instead you might find that optimising your media towards reaching a very relevant audience is the better way to go, as you have more opportunity to influence them. I mean big businesses spend a hell of a lot of money on understanding customers and their motivations better, so we shouldn't just throw all that insight out the door.

That’s why a multi-faceted measurement approach is key to investing with confidence, and has had a pretty substantial impact for us in the past, particularly in areas like digital audio but also the digital video space as well, which notoriously doesn't look very good from a last touch / last click basis but frequently looks much more positive in econometrics and geo-testing.

How does all the change around the products and services within the telco market factor into your media selection and planning choices?

What is important to note is the digital media transformation journey we have been on over the last six or seven months. We have in-housed all of the activation of our digital performance activity.  We've moved away from buying and activating through an agency to doing this entirely in-house across all channels - affiliates, paid search, natural search, programmatic and social.

What that’s meant for us initially is we've had to reduce the number of partners we actually work with. From an operational point of view the more partners there are the harder it is to get everybody on-boarded and to manage. We’re now reviewing where we have gaps to plug, and where we can take learnings from all the work we’ve done through agencies. One of the benefits of agencies is they can scale and go into different areas quite quickly. As an in-house team we are a little bit leaner, meaning we need to prioritise when making decisions on partners to use.

At the moment we've gone with the largest DSP in the market, so no big surprise of who that will be. We can then access plenty of inventory from multiple suppliers through our programmatic deals without having to have direct relationships per se. How we then go about deciding who we should work with really comes down to what are our KPIs and objectives. Do we think they are going to be able to deliver against those? Do we run a test initially?

Now we’ve completed the activation in-housing, we’re in the process of in-housing the ad tech and operations that enable everything. Part of this is setting ourselves principles around what we want from our digital advertising and defining a set of guidelines to operate within. Many of these guidelines will be familiar to people, things like is it brand safe? Is it viewable? Is every ad we serve delivered on time to the right target audience? Is it a quality exposure? Is it suitable for our brands? All of that starts to determine the inventory sources that you plug yourself into, to ensure as it should correlate with more relevant placements, with higher attention and higher dwell times and ultimately – hopefully – better performance.

How are you managing to balance the different pros and cons of targeting and reach?

It is tricky, but I think it all comes down to how we can demonstrate whether or not what we're doing actually has a real impact on the outcomes that we're trying to see. You can buy reach in all shapes and forms but not all reach is equal.  When it comes to Digital Audio, things like podcasts have exploded in popularity in the last few years  because the type of reach that you're getting is also incredibly personal and one to one and is far more impactful than, let's say, the reach we might achieve through standard ad banners plastered everywhere. To be honest, maybe standard ad banners look great from a viewability perspective, but when we’ve looked at attention measurement, nobody's looking at them for more than a half a second.

The reality is not all reach is equal. The really difficult part is in linking back to your objectives; just because something gets high attention, is it actually driving the outcome that you want? And I think that’s the part that we're going to be really focused on over the next while, having those guidelines of what we think to be true and tying it all together. Because when you've got good quality environments, where people are paying attention and if it's the right message, then we will see an increase in whatever it is we're trying to drive.

In terms of the impending death of a third party cookie, how is that affecting what you do next?

It's probably more of a gradual death rather than a sudden death, and we’re feeling a bit more reassured because of things we have been putting in place over time to try and address some of the expected challenges. For example one benefit we have is that our two businesses have a lot of data, and leveraging this will be a critical part of any future digital marketing strategy. There’s obviously a dependency on the capabilities to support the activation of this, which things like customer data platforms (CDPs) that don’t rely purely on third party cookies will help with, and we can leverage other signals through data we share via server-to-server integrations with our various media partners.

I've also always been a big fan of contextual targeting, and its renaissance recently makes me smile. When it comes to digital the beauty of it is that everything that we look at on our phones or on our computers, whatever it might be, we've made some form of conscious decision to have a look at it, and as a brand if we can be relevant to that specific moment in time then we will have a greater chance of connecting with that person and capturing their attention.

What do you see are the main strengths and potential pitfalls of audio?

I've been quite lucky in my career, having worked within both brand and performance marketing roles, covering traditional media and digital, so I think I’ve had a good level of exposure to a lot of marketing channels.

Audio for me, particularly digital audio and the likes of podcasts that we mentioned earlier, can be really powerful for advertisers. You are buying into a specific environment that someone may have had a personal recommendation for, or it might be about a subject that they’re particularly passionate about, with engaging presenters that are easy to listen to. A single ad, at that particular point in time, can be really impactful, particularly where it's a sponsored read from the presenter as this gives it all an element of credibility. The trade off is scale, and how to maximise this without losing that relevance.

Another challenge for advertisers is how can we place our media into these environments programmatically. I personally want us to be doing that as part of our ‘always on’ digital performance campaigns. Programmatic buys simply mean placing an ad anywhere we need to, in a digital environment, with control over the individual moment and experience, and I'd like to see us do that across all channels, not just display and video.

But the biggest challenge from a performance point of view goes back to attribution, because virtually no one is listening to their favourite playlist, hearing an ad and then immediately clicking through and making a purchase then and there. A very small percentage of people may do that if you’re lucky, which is why audio typically looks terrible in last touch digital attribution.

In reality the ad exposure will hopefully have contributed to a wider journey of media touchpoint exposures that ultimately result in a desired outcome somewhere down the line. If you can understand the true value of that exposure, alongside the other touchpoints they were exposed to, synced to some form of live attribution model, then you will have greater confidence in real time of the ads impact, enabling you to invest more where appropriate.

You could look at econometrics and it may give you guidance that yes, broadly you should be spending in those areas, but because it doesn't provide a view on the granularity of targeting from buying point of view, the effect will be somewhat dampened and like the old adage, you will be left not knowing which half of your advertising is wasted and which half is doing the job.

Ultimately this brings us back to the need for a multifaceted measurement approach, to get the best from your investments, balancing the in-flight digital attribution results with testing and media mix modelling, to build out a clearer picture of contribution.