AUSTIN, TX: Researchers must be aware of "confirmation bias" impacting the outcomes of qualitative exercises such as empathy interviews and longitudinal studies, a leading executive from Capital One has argued.
Alison Abbott, a design researcher/Capital One Labs, told delegates at South by Southwest (SXSW) 2016 that natural responses of the brain can play "tricks" on researchers as they undertake complex tasks like developing new products.
"In this world of innovation, we have to have strongly-held beliefs," she said. (For more, including further tips for brands, read Warc's exclusive report: Avoiding confirmation bias in research: Insights from Capital One.)
"However, having those strongly-held beliefs sometimes can hurt us as much as it helps us. We'll start to seek information and interpret information in a way that confirms our own beliefs, as opposed to seeking out information that might lead us to even better ideas."
One example of this trend in action is the "primacy effect", where feedback heard early in the research process becomes the foundational truth, and is afforded greater weight than equally valid insights arising at a later stage.
"Are you looking at your hundredth participant with the same judgement that you are looking at your first participant's feedback?" Abbott said.
Similar problems relate to the "frequency illusion", where a new learning yielded by qualitative analysis seemingly appears everywhere a researcher looks.
"The area where this comes up a lot is when you are trying to identify pain points," said Emma Sagan, a product manager at Capital One Labs.
"And that could be true; you may have stumbled on something really profound. Or you might suddenly be open to hearing it, and it might not be as big as you think it is."
Hard numbers are useful in gaining corroboration either way. "Try and find as much quant data as possible to back up what you're finding," Sagan recommended.
"Really push your team … to try and find sources, try and conduct your own surveys to find that data to couple with it."
Alison Abbott, a design researcher/Capital One Labs, told delegates at South by Southwest (SXSW) 2016 that natural responses of the brain can play "tricks" on researchers as they undertake complex tasks like developing new products.
"In this world of innovation, we have to have strongly-held beliefs," she said. (For more, including further tips for brands, read Warc's exclusive report: Avoiding confirmation bias in research: Insights from Capital One.)
"However, having those strongly-held beliefs sometimes can hurt us as much as it helps us. We'll start to seek information and interpret information in a way that confirms our own beliefs, as opposed to seeking out information that might lead us to even better ideas."
One example of this trend in action is the "primacy effect", where feedback heard early in the research process becomes the foundational truth, and is afforded greater weight than equally valid insights arising at a later stage.
"Are you looking at your hundredth participant with the same judgement that you are looking at your first participant's feedback?" Abbott said.
Similar problems relate to the "frequency illusion", where a new learning yielded by qualitative analysis seemingly appears everywhere a researcher looks.
"The area where this comes up a lot is when you are trying to identify pain points," said Emma Sagan, a product manager at Capital One Labs.
"And that could be true; you may have stumbled on something really profound. Or you might suddenly be open to hearing it, and it might not be as big as you think it is."
Hard numbers are useful in gaining corroboration either way. "Try and find as much quant data as possible to back up what you're finding," Sagan recommended.
"Really push your team … to try and find sources, try and conduct your own surveys to find that data to couple with it."
Data sourced from Warc
