Research Quality: The Interaction of Sampling and Weighting in Producing a Representative Sample Online: An Excerpt from the ARF’s “Foundations of Quality 2” Initiative

John Bremer

Toluna

INTRODUCTION

The latest phase of the Advertising Research Foundation’s (ARF) “Foundations of Quality” research initiative (FoQ 2) began in 2010 designed, in part, to study new questions about the quality of the online samples developed and how new sampling methodologies and technologies have developed in a new marketing ecosystem.

Weighting is a controversial practice as it gives certain respondents more influence over the results than other respondents and affects the variability within the data. Some practitioners are opposed to the use of weighting for these reasons.

Within probability samples, there is a theoretical basis for weighting, equalizing the probability of selection, so whereas some practitioners are opposed to the practice, there can be good rationale to weight the data, even minimally. As the vast majority of online-sourced samples are not selected via a probability basis, the theoretical reason behind weighting goes away, and those opposed to the practice can induce control through the samp­ling process.