How far can you rely on a concept test: the generalizability of testing over occasions

Ling Peng

Lingnan University

Adam Finn

University of Alberta


Concept testing, whether traditional (Moore 1982) or conjoint (Green & Wind 1975), is an important tool used to assess the likely market demand and the best customers to target with potential new products early on in their development (Page & Rosenbaum 1992; Ozer 1999; Crawford & Di Benedetto 2006). It uses customer response data to pick likely winners from among the many likely losers, and so is used to help allocate product development resources. Improving the design and interpretation of the results of concept testing can make an important contribution to improving the efficiency of new product development.

A time delay between concept testing and potential market introduction is an inherent feature of new product development. Changes in the environment can create unexpected threats or opportunities at the time of market introduction. In practice, product managers have to assume consumer evaluations of concepts generalise from the time (and research environment) of concept testing to the time (and market environment) of market introduction. However, little is known about the temporal stability or generalisability of the results of concept testing over occasions. Rarely have concept-testing studies incorporated testing of the same concepts on the same respondents on more than one occasion. There are at least two reasons for this shortcoming. First, the logistics have been difficult when using traditional data collection methods, but this is changing with the development of online methods. Second, collecting costly data from respondents on more than one occasion is not considered a necessary part of the testing process.