As online research has transitioned over the past 15 years from random digit dialing or door-to-door methods to online, a shift from probability to non-probability sampling has occurred. To assuage the concerns of researchers, rigorous sampling methods were developed to ensure representative samples. Despite this, however, there has been a rapid change in sampling practices as respondent efficiency became the focus of fieldwork companies and respondent routing replaced traditional sampling for much of the market research industry.
To create a ‘representative sample’ using non-probability techniques, research practitioners use a variety of demographic variables that are balanced to a benchmark such as census data. This process works well when a panel for which the demo variables are known is used. When a router is employed, it is most commonly used in conjunction with river or dynamic traffic where minimal information about the demographic mix of respondents is known in advance. In this environment, the survey system essentially controls the sample, allowing in those respondents for which an open quota exists. This is an efficient, cost effective process; however, the completed sample from a router tends to be less well balanced on the non-quota variables.
At Lightspeed GMI, we have been struggling with this issue for some time. The vast majority of our business is still traditional sample selected from our online panel, but we are finding it increasingly difficult to get certain demographic groups to join our panels. While cost effective and abundant, river sources are much more difficult to balance on non-quota variables since those variables are typically unknown. We feel that some types of research require highly balanced sample while other types of research are less sensitive to sample composition.
We have done extensive testing to find the right way to blend in river sample and to control routers. Our research is ongoing -- have a look at this white paper that was written by Frank Kelly from Lightspeed GMI in conjunction with Deb Santus and Peter Kwok from TNS.
