As I whiled away the hours at Brussels Midi station waiting in hope and in confusion for a Eurostar train to get me back to London, it became clear that the law of unintended consequences was at work.
It has, of course, derailed many a well-intentioned policy.
In the UK, for example, NHS targets have resulted in distortions to patient care; educational targets have led to the abandonment of non-core subjects and so on. Back in Brussels unbeknownst to all, the train I was booked on had been cancelled and all passengers transferred to the next train, thereby displacing all its passengers onto the subsequent train.
No one had any idea which train they were entitled to board: the resulting entirely unnecessary confusion and angst amongst passengers was pitiful to see. Why would any business risk such alienation of its customers if it were not for the fact that by cancelling a train rather than allowing it to leave 3 hours late, the company’s targets and penalties would be less jeopardised?
Charles Goodhart identified the dangers of using performance metrics as targets in his eponymous law. An advisor to the Bank of England, Goodhart noticed that as soon as a metric was turned into a target, it lost its value as a measure of success.
The most obvious reason for this is that other facets of performance that had once correlated with the chosen metric, cease to do so because they are deliberately de-prioritised in order to pursue the target: the metric no longer remains a useful broader indicator of success and often becomes counterproductive.
An elegant example of this is the apparent contradiction between the robust PIMS finding that profitability correlates closely with market share and the similarly robust finding of Wharton and Monash professors Armstrong and Green, that companies that pursue market share as a primary target actually lose profitability.
The unintended consequence here is the buying of market share through non-profitable means such as discounting and price-promotion: target achieved, but at a cost.
Why my sudden interest in Goodhart’s law? Ever since I wrote a report for the UK-based Institute of Practitioners in Advertising (IPA) entitled ‘The link between creativity and effectiveness’, I have become increasingly uncomfortable about the level and nature of interest shown in the study by creative agencies.
Naturally it is nice to have interest shown in one’s work, but just in case the report is used to suggest that creativity should become a target for agencies or their management teams, let me explain why Goodhart’s law is especially relevant here.
The analysis I conducted showed that campaigns in the IPA databank that had won at least one major creative award (as measured by the Gunn Report) were on average 11 times more efficient than those that had not.
Whilst this suggests that creatively awarded campaigns are more likely to be commercially successful, it does not mean that they all are – indeed in Donald Gunn’s landmark 1996 study, he found that around 14% of creatively awarded campaigns failed to show any commercial success: usually because the strategy was wrong.
What the IPA analysis actually demonstrates is that creatively awarded strategically sound campaigns are 11 times more efficient than non-creative but strategically sound campaigns. No amount of pure creative genius will turn a misguided strategy into a commercial success.
The IPA analysis demonstrates that you need to focus on both effectiveness and creativity to hit the sweet spot. So an agency that targets creative awards alone as its key output success metric runs a very great risk of undermining the value of that creativity as a result of Goodhart’s law: because the drive for creative awards will mean deprioritising effectiveness.
Perhaps this already happens - there are many very conspicuously creatively awarded campaigns that never seem to submit effectiveness case studies. If I were a client of one of these campaigns I would want to know why.
So the warning to clients and other creative agency stakeholders is this: if your agency pursues creative award targets, ensure that it also has effectiveness award targets. If not, Goodhart’s law may see your growth plans derailed.
It has, of course, derailed many a well-intentioned policy.
In the UK, for example, NHS targets have resulted in distortions to patient care; educational targets have led to the abandonment of non-core subjects and so on. Back in Brussels unbeknownst to all, the train I was booked on had been cancelled and all passengers transferred to the next train, thereby displacing all its passengers onto the subsequent train.
No one had any idea which train they were entitled to board: the resulting entirely unnecessary confusion and angst amongst passengers was pitiful to see. Why would any business risk such alienation of its customers if it were not for the fact that by cancelling a train rather than allowing it to leave 3 hours late, the company’s targets and penalties would be less jeopardised?
Charles Goodhart identified the dangers of using performance metrics as targets in his eponymous law. An advisor to the Bank of England, Goodhart noticed that as soon as a metric was turned into a target, it lost its value as a measure of success.
The most obvious reason for this is that other facets of performance that had once correlated with the chosen metric, cease to do so because they are deliberately de-prioritised in order to pursue the target: the metric no longer remains a useful broader indicator of success and often becomes counterproductive.
An elegant example of this is the apparent contradiction between the robust PIMS finding that profitability correlates closely with market share and the similarly robust finding of Wharton and Monash professors Armstrong and Green, that companies that pursue market share as a primary target actually lose profitability.
The unintended consequence here is the buying of market share through non-profitable means such as discounting and price-promotion: target achieved, but at a cost.
Why my sudden interest in Goodhart’s law? Ever since I wrote a report for the UK-based Institute of Practitioners in Advertising (IPA) entitled ‘The link between creativity and effectiveness’, I have become increasingly uncomfortable about the level and nature of interest shown in the study by creative agencies.
Naturally it is nice to have interest shown in one’s work, but just in case the report is used to suggest that creativity should become a target for agencies or their management teams, let me explain why Goodhart’s law is especially relevant here.
The analysis I conducted showed that campaigns in the IPA databank that had won at least one major creative award (as measured by the Gunn Report) were on average 11 times more efficient than those that had not.
Whilst this suggests that creatively awarded campaigns are more likely to be commercially successful, it does not mean that they all are – indeed in Donald Gunn’s landmark 1996 study, he found that around 14% of creatively awarded campaigns failed to show any commercial success: usually because the strategy was wrong.
What the IPA analysis actually demonstrates is that creatively awarded strategically sound campaigns are 11 times more efficient than non-creative but strategically sound campaigns. No amount of pure creative genius will turn a misguided strategy into a commercial success.
The IPA analysis demonstrates that you need to focus on both effectiveness and creativity to hit the sweet spot. So an agency that targets creative awards alone as its key output success metric runs a very great risk of undermining the value of that creativity as a result of Goodhart’s law: because the drive for creative awards will mean deprioritising effectiveness.
Perhaps this already happens - there are many very conspicuously creatively awarded campaigns that never seem to submit effectiveness case studies. If I were a client of one of these campaigns I would want to know why.
So the warning to clients and other creative agency stakeholders is this: if your agency pursues creative award targets, ensure that it also has effectiveness award targets. If not, Goodhart’s law may see your growth plans derailed.
