LONDON: A new approach to content validation and development by BBC Worldwide, based on analysing and quantifying the emotions behind the usual survey opinions, has won the broadcaster the Grand Prix at the recent MRS Awards.
With the long-term business objective of the organisation being to move to an evidence-based approach to content development, it needed to come up with a methodology that was not only cost-effective and capable of working on a global scale but also easily quantified and understood by "number-focused stakeholders".
Further challenges included integrating any new methods with existing development processes and supporting creative development without constraining creativity.
Starting with the knowledge that what people say in focus groups frequently fails to tally with what they really think, BBC Worldwide and its partner Lightspeed GMI integrated facial coding into their research, in a pilot involving 4,657 Australian respondents.
Screen-mounted cameras captured second-by-second variations in respondents' micro-facial expressions at the same time as their conscious appreciation scores for the programming being shown were collected using an online survey.
The findings were then "dashboarded" in a way that made "the right decision interpretation easy and the wrong decision hard", according to David Boyle, executive vice president of Insight at BBC Worldwide.
Early results highlighted a major gap in conscious and unconscious responses for several programmes. Further investigation revealed that the most hated clip for one show also made viewers happy – happier, in fact, than 82% of programmes tested.
"Combining emotion data with survey data is considerably more reliable for predicting viewer interest," the partners reported.
BBC Worldwide is convinced enough to be rolling out the new development methodology across all English-speaking territories initially with an international roll-out to follow.
And over time it expects to accumulate data that will inform the building of norms to benchmark new development – norms for brands, audiences and regions.
"We've found a way to quantify emotion so that everyone can get behind it," said Boyle. "It's really radical for us but also realistic: now we can start embedding it at scale to raise programming quality all over the world."
With the long-term business objective of the organisation being to move to an evidence-based approach to content development, it needed to come up with a methodology that was not only cost-effective and capable of working on a global scale but also easily quantified and understood by "number-focused stakeholders".
Further challenges included integrating any new methods with existing development processes and supporting creative development without constraining creativity.
Starting with the knowledge that what people say in focus groups frequently fails to tally with what they really think, BBC Worldwide and its partner Lightspeed GMI integrated facial coding into their research, in a pilot involving 4,657 Australian respondents.
Screen-mounted cameras captured second-by-second variations in respondents' micro-facial expressions at the same time as their conscious appreciation scores for the programming being shown were collected using an online survey.
The findings were then "dashboarded" in a way that made "the right decision interpretation easy and the wrong decision hard", according to David Boyle, executive vice president of Insight at BBC Worldwide.
Early results highlighted a major gap in conscious and unconscious responses for several programmes. Further investigation revealed that the most hated clip for one show also made viewers happy – happier, in fact, than 82% of programmes tested.
"Combining emotion data with survey data is considerably more reliable for predicting viewer interest," the partners reported.
BBC Worldwide is convinced enough to be rolling out the new development methodology across all English-speaking territories initially with an international roll-out to follow.
And over time it expects to accumulate data that will inform the building of norms to benchmark new development – norms for brands, audiences and regions.
"We've found a way to quantify emotion so that everyone can get behind it," said Boyle. "It's really radical for us but also realistic: now we can start embedding it at scale to raise programming quality all over the world."
Data sourced from MRS; additional content by Warc staff
