LONDON: Traditional survey rating metrics often fail to give the full picture, but there are now a range of online panel research measures to help advertisers test their creative, according to a leading market researcher.
Writing in the current issue of Admap, Alex Wheatley, the research innovator at market research firm Lightspeed GMI, said better data can be gained from surveys that are shorter, more engaging and enjoyable.
With an engaged panel, Lightspeed GMI found that one simple open-ended question could yield the amount of data it would take 20 closed-ended questions to achieve.
On top of that, open-ended questions can draw out a wealth of extra unprompted and unbiased data that can be missed by the predetermined nature of a closed question.
For example, instead of asking respondents to rate how much they like a piece of creative, questions based on binary instincts ("thumbs up" or "thumbs down") or trading games, such as asking respondents to predict the success of ads and giving them immediate feedback from others on the panel, create better and more differentiated data sets.
As a consequence, researchers are able to ask a much smaller group of respondents to reach the same level of certainty about a piece of creative.
Wheatley said that as few as 40 respondents would be needed when a trading game methodology is employed. By comparison, four times as many would be required when using a traditional Likert scale.
Likert-scaled ratings yield lower standard differentiation in the data than other metrics, Wheatley asserted. "In a nutshell, they tend to be dull, confusing and take too much time to complete," he said.
Another advantage in using a shorter and more engaging survey is in maintaining the integrity of the panel and mitigating the risk of participants dropping out.
"If we want them to not only stay in one study but expect the next one and be immediately ready to take it, we need to harness this engagement and create a draw across multiple surveys," Wheatley said. "Ultimately, we need to give them something to look forward to."
Writing in the current issue of Admap, Alex Wheatley, the research innovator at market research firm Lightspeed GMI, said better data can be gained from surveys that are shorter, more engaging and enjoyable.
With an engaged panel, Lightspeed GMI found that one simple open-ended question could yield the amount of data it would take 20 closed-ended questions to achieve.
On top of that, open-ended questions can draw out a wealth of extra unprompted and unbiased data that can be missed by the predetermined nature of a closed question.
For example, instead of asking respondents to rate how much they like a piece of creative, questions based on binary instincts ("thumbs up" or "thumbs down") or trading games, such as asking respondents to predict the success of ads and giving them immediate feedback from others on the panel, create better and more differentiated data sets.
As a consequence, researchers are able to ask a much smaller group of respondents to reach the same level of certainty about a piece of creative.
Wheatley said that as few as 40 respondents would be needed when a trading game methodology is employed. By comparison, four times as many would be required when using a traditional Likert scale.
Likert-scaled ratings yield lower standard differentiation in the data than other metrics, Wheatley asserted. "In a nutshell, they tend to be dull, confusing and take too much time to complete," he said.
Another advantage in using a shorter and more engaging survey is in maintaining the integrity of the panel and mitigating the risk of participants dropping out.
"If we want them to not only stay in one study but expect the next one and be immediately ready to take it, we need to harness this engagement and create a draw across multiple surveys," Wheatley said. "Ultimately, we need to give them something to look forward to."
Data sourced from Admap
