Identifying the real differences of opinion in social media sentiment
Scales are a cornerstone of market research. Every survey labels and categorises opinions based on scales from 1 to 5, 1 to 7, or even 1 to 100, helping us determine the strength and distribution of opinions on any topic imaginable. They’re how we determine that 49% of people like Coca-Cola and 42% of people like Pepsi, that men like watching sports more than women do, that younger people like electronics more than older people do, and that Canadians like Shania Twain more than Brits or Americans do. Or do they?
One of the problems with scales is that people interpret them differently. I have a rule that I never choose ‘strongly agree’ or ‘strongly disagree’ except in the rarest 1% of circumstances. But other people are far more liberal, opting to use each scale option an equal percentage of the time. Similarly, when men describe once per week as being ‘often’, have we considered that women may instead describe once per month as ‘often’? Do these types of differences manifest themselves among older and younger people? People from Canada vs people from the United States? If one group of people has a naturally different way of responding to scales than another group of people, how are we to know whether any differences in scores are due to differences of opinion as opposed to differences in how they use scales?