Open letter warns of dangers of platform-based AI market mix modelling

More providers are offering automated market mix modelling (MMM) systems using AI/machine learning as cookie-based attribution declines, but 17 experts have published an open letter warning that many such systems are not nuanced enough to provide effective measurement.

Why it matters

Brands need to measure their marketing activity, and MMM is rightly becoming a more common tool in the marketer’s toolbox. Experts – including Les Binet, Grace Kite, and Louise Cook – caution that the algorithms behind many of these new solutions are too simplistic and not properly adapted to the specific businesses being analysed.

“Artificial intelligence isn’t enough for this task,” says Dr. Kite, in a press release shared with WARC. “[R]eal people are needed. Machines can’t yet understand the nuances of the situation, identify things that are missing in data, or help people get comfortable that results are reliable for important decisions.”

What to know

• Fast and cheap doesn’t mean good: “The push for faster and cheaper solutions is leading to approaches which are simply not able to embrace the complexities of brand response,” says Louise Cook, director at Holmes and Cook. “These approaches risk misleading rather than enlightening.”
• Heavy costs are at stake: “Pitch decks can be very convincing, but we’ve heard stories where 10 months of set up were followed by garbage results that were completely unusable,” adds Mike Cross, co-founder at Measure Monks. “Making the wrong decisions off a bad model costs up to -40% of your media driven revenue, versus MMM applied properly which can deliver +30%.”
• Apply a high burden of proof: "There is no plug and play solution to deliver results,” notes Nancy Smith, president and CEO at Analytic Partners. “Without deep experience leveraging data and analytics for commercial intelligence, brands risk significant losses due to poorly informed decisions. Greater yet is the risk to organizational buy-in to making data-driven decisions.”

Key quote

“We’re not that vocal a bunch usually, and we don’t often offer advice unless we’re asked for it. But if you want to know what we suggest you do, it’s shop for MMM like you’re buying a new kitchen. Ask around for recommendations, get at least three quotes, and don’t be too trusting,” the signatories advise in the letter, which you can read in full here.