Measuring emotions through a mobile device across borders, ages, genders and more

Rolfe Swinton and Rana El Kaliouby
RealityMine and Affectiva, USA


Measuring emotion is no longer solely the domain of desktops, or laptops for that matter. Mobile devices have increasingly become a robust platform in which to capture consumer insight into ads and other forms of media. Mobile devices are ubiquitous and are ideal for delivering rich media while capturing people's response to the media. This enables advanced computer vision technology to identify the facial expressions captured in the face videos and to map these expressions to emotional states such as enjoyment, confusion, surprise and attention. This paper documents an iterative process to develop, test, and validate the use of mobile as a powerful way to test ad recall and to assess the emotional impact a given ad has on a group of people through facial coding. This paper shares findings from initial iterations, namely testing ad recall of a battery of TV ads in Brazil and other emerging markets, and then proceeds to test six ads in the beverage category in two cities in India (Hyderabad and Delhi), highlighting differences in ad responses by demographics and location. This innovative use of mobile devices creates many opportunities for innovation in market research.