--- title: "Sampling Best Practice:  Probability versus Nonprobability, Redux" description: The same techniques that nonprobability samples use to weight and model data now often need to be done on probability samples to account for nonresponse bias. image: https://info-profiles.kantar.com/hubfs/handdrawnlightbulb_jpg.jpg --- - - - - [mailto:info@gmi-mr.com](mailto:info@gmi-mr.com) [![Kantar-Logo-Small-Use-Black](https://info-profiles.kantar.com/hs-fs/hubfs/logos/Kantar-Logo-Small-Use-Black.png?width=119&name=Kantar-Logo-Small-Use-Black.png)](http://www.lightspeedresearch.com) - [Home](http://www.lightspeedresearch.com/) - [Sample Audiences](http://www.lightspeedresearch.com/sample-audiences/) - [Quality Suite](http://www.lightspeedresearch.com/sample-audiences/quality-suite/) - [Panel Profiling](http://www.lightspeedresearch.com/sample-audiences/panel-profiling/) - [B2B Panels](http://www.lightspeedresearch.com/sample-audiences/business-to-business-panels/) - [Custom Panels](http://www.lightspeedresearch.com/sample-audiences/custom-panels/) - [Millennial Panel](http://www.lightspeedresearch.com/sample-audiences/vice-voices/) - [Research Solutions](http://www.lightspeedresearch.com/research-solutions/) - [Quick Turn Surveys](http://www.lightspeedresearch.com/research-solutions/lightspeed-quick-turn-surveys/) - [Survey Design](http://www.lightspeedresearch.com/research-solutions/survey-design/) - [Tracking Research Design](http://www.lightspeedresearch.com/research-solutions/tracking-research-design/) - [Integrated Technology Solutions](http://www.lightspeedresearch.com/research-solutions/integrated-technology-solutions/) - [Data Processing](http://www.lightspeedresearch.com/research-solutions/data-processing/) - [Ad Tracker](http://www.lightspeedresearch.com/research-solutions/ad-tracker/) - [Survey Programming](http://www.lightspeedresearch.com/research-solutions/ad-tracker/) - [Sample Only Services](http://www.lightspeedresearch.com/research-solutions/sample-only-services/) - [Data Services](http://www.lightspeedresearch.com/data-services) - [Data Appends](http://www.lightspeedresearch.com/data-services/data-appends/) - [Mobile](http://www.lightspeedresearch.com/mobile/) - [Mobile Solutions](http://http://www.lightspeedresearch.com/mobile/) - [Resource Center](http://www.lightspeedresearch.com/resource-center/) - [Events](http://www.lightspeedresearch.com/resource-center/events/) - [Demos](http://www.lightspeedresearch.com/resource-center/demos/) - [Case Studies](http://www.lightspeedresearch.com/resource-center/case-studies/) - [Blog](https://blog.lightspeedresearch.com/) - [General Inquiries](http://www.lightspeedresearch.com/contact-us/general-inquiries/) - [About Us](http://www.lightspeedresearch.com/about-us/) - [Leadership Team](http://www.lightspeedresearch.com/about-us/leadership-team/) - [Press Releases](http://www.lightspeedresearch.com/about-us/news-press-releases/) - [Why Lightspeed](http://www.lightspeedresearch.com/about-us/why-lightspeed/) - [Careers](http://www.lightspeedresearch.com/about-us/careers/) - [Regions](http://www.lightspeedresearch.com/about-us/regions/) - [Awards](http://www.lightspeedresearch.com/awards/) - [Contact Us](http://www.lightspeedresearch.com/contact-us/) # Kantar's Profiles Blog # Sampling Best Practice:  Probability versus Nonprobability, Redux Posted by [Susan Frede](https://info-profiles.kantar.com/author/susan-frede) on May 31, 2016 ![](https://info-profiles.kantar.com/hubfs/Blog_Authors/Susan_Frede.jpg) - [Tweet](https://twitter.com/share) A dozen years ago a debate raged in the marketing research community over the switch from probability sampling methods such as telephone RDD to nonprobability sampling methods as are typical with online access panels. In the interim years, most clients moved to online samples but there are still some that cling to probability methods. However, we now see the quality of probability samples being questioned because of low response rates for RDD. In an interesting twist, the very same techniques that nonprobability samples use to weight and model data now often need to be done on probability samples to account for nonresponse bias.  In early May [Pew Research Center](http://www.pewresearch.org/) published a report entitled “Evaluating Online Nonprobability Survey.”  Pew fielded a survey to nine online nonprobability samples from eight different suppliers. The survey included 56 measures including 20 benchmarks with comparison data obtained from high quality government research. Results were also compared to Pew’s American Trends Panel (ATP) which was recruited via random digit dialing (RDD) in early 2014.  Many of Pew’s findings coincide with what [Lightspeed GMI](http://www.lightspeedgmi.com/) recommends as best in class sampling. Our basic premise is to pull an appropriate sample using the right variables for a particular client. Sampling should be customized for each client based on their expressed objectives and needs. We were one of the suppliers in the Pew work and generally performed in the middle or middle-upper part of the pack. In this case, we were given sampling instructions to control on only age and gender. In hindsight, we should have insisted on additional controls to improve sample quality and performance. Applying our best practices would have moved us to the top of the pack.  The Lightspeed GMI Sampling Best Practices include: 1. Understand the **objectives and analytic plan** for the research and who the end client is so an appropriate sample is built. 2. Regardless of objectives/analytic plans, at a minimum balance the outgoing sample on **age, gender, region, and income/social class**.  Additional variables may be needed based on the objectives and analytic plan.  3. Be cautious with too much interlocking of quotas.  The basic recommendation is to **only interlock age and gender**.  When more is interlocked it creates a level of complexity that is very hard to deliver against.  4. Instead of interlocking, if a subgroup is particularly important (e.g., Hispanics, African Americans, young adults, males, etc.) consider **splitting the sampling** by those subgroups to make sure the subgroup is truly representative.  For example, instead of building a single sample two samples might be built—one for males and another for females. 5. **Set survey quotas** for key variables.  This should generally be age and gender plus one other key variable. 6. Be cautious with short field periods.  At a minimum we would recommend a **two to three day field period** and for more academic research might go to at least seven days.  7. Don’t be afraid to weight data.  **RIM weighting** is the recommended approach.  Before weighting on non-demographic characteristics carefully think about the implications.  8. Remember sample sources aren’t the same so devise a sampling plan that allows the **sample blend** to remain fairly consistent for the life of the project.  Topics: [Online Sampling](https://info-profiles.kantar.com/topic/online-sampling), [marketing research best practices](https://info-profiles.kantar.com/topic/marketing-research-best-practices), [nonprobabliity samples](https://info-profiles.kantar.com/topic/nonprobabliity-samples), [probability samples](https://info-profiles.kantar.com/topic/probability-samples) ### Subscribe to Email Updates ### Recent Posts ### Posts by Topic - [Marketing Research (46)](https://info-profiles.kantar.com/topic/marketing-research) - [Market Research (28)](https://info-profiles.kantar.com/topic/market-research) - [Marketing Research Data (28)](https://info-profiles.kantar.com/topic/marketing-research-data) - [modern surveys (27)](https://info-profiles.kantar.com/topic/modern-surveys) - [Mobile (26)](https://info-profiles.kantar.com/topic/mobile) - [Data Driven Marketing (21)](https://info-profiles.kantar.com/topic/data-driven-marketing) - [Market Research Trends (18)](https://info-profiles.kantar.com/topic/market-research-trends) - [Consumer Insights (17)](https://info-profiles.kantar.com/topic/consumer-insights) - [Online Sampling (15)](https://info-profiles.kantar.com/topic/online-sampling) - [Survey Design (15)](https://info-profiles.kantar.com/topic/survey-design) - [Blog Post (13)](https://info-profiles.kantar.com/topic/blog-post) - [Data Quality (13)](https://info-profiles.kantar.com/topic/data-quality) - [Digital Data Collection (12)](https://info-profiles.kantar.com/topic/digital-data-collection) - [Financial Services (11)](https://info-profiles.kantar.com/topic/financial-services) - [APAC (10)](https://info-profiles.kantar.com/topic/apac) - [Healthcare Research (10)](https://info-profiles.kantar.com/topic/healthcare-research) - [data series (10)](https://info-profiles.kantar.com/topic/data-series) - [Video (9)](https://info-profiles.kantar.com/topic/video) - [Digital Consumer (8)](https://info-profiles.kantar.com/topic/digital-consumer) - [data append (8)](https://info-profiles.kantar.com/topic/data-append) - [marketing research best practices (8)](https://info-profiles.kantar.com/topic/marketing-research-best-practices) - [mobile first (8)](https://info-profiles.kantar.com/topic/mobile-first) - [mobile marketing research (8)](https://info-profiles.kantar.com/topic/mobile-marketing-research) - [Research Quality (7)](https://info-profiles.kantar.com/topic/research-quality) - [marketing research technologies (7)](https://info-profiles.kantar.com/topic/marketing-research-technologies) - [marketing research trends (7)](https://info-profiles.kantar.com/topic/marketing-research-trends) - [thought leadership (7)](https://info-profiles.kantar.com/topic/thought-leadership) - [Ask the Expert (6)](https://info-profiles.kantar.com/topic/ask-the-expert) - [Context (6)](https://info-profiles.kantar.com/topic/context) - [Panels (6)](https://info-profiles.kantar.com/topic/panels) - [Social Media (6)](https://info-profiles.kantar.com/topic/social-media) - [data enrichment (6)](https://info-profiles.kantar.com/topic/data-enrichment) - [Innovation (5)](https://info-profiles.kantar.com/topic/innovation) - [MRX Trends (5)](https://info-profiles.kantar.com/topic/mrx-trends) - [Sourcing (5)](https://info-profiles.kantar.com/topic/sourcing) - [behavioral data (5)](https://info-profiles.kantar.com/topic/behavioral-data) - [Big Data (4)](https://info-profiles.kantar.com/topic/big-data) - [Honesty Detector (4)](https://info-profiles.kantar.com/topic/honesty-detector) - [Lightspeed FSG (4)](https://info-profiles.kantar.com/topic/lightspeed-fsg) - [Millennials (4)](https://info-profiles.kantar.com/topic/millennials) - [Online access panels (4)](https://info-profiles.kantar.com/topic/online-access-panels) - [Privacy (4)](https://info-profiles.kantar.com/topic/privacy) - [Respondent experience (4)](https://info-profiles.kantar.com/topic/respondent-experience) - [media buying (4)](https://info-profiles.kantar.com/topic/media-buying) - [mobile surveys (4)](https://info-profiles.kantar.com/topic/mobile-surveys) - [questionnaire design (4)](https://info-profiles.kantar.com/topic/questionnaire-design) - [Brand Advertisting (3)](https://info-profiles.kantar.com/topic/brand-advertisting) - [Infographics (3)](https://info-profiles.kantar.com/topic/infographics) - [Survey Engagement (3)](https://info-profiles.kantar.com/topic/survey-engagement) - [data search (3)](https://info-profiles.kantar.com/topic/data-search) - [disABILITYincites Partnership (3)](https://info-profiles.kantar.com/topic/disabilityincites-partnership) - [gdpr (3)](https://info-profiles.kantar.com/topic/gdpr) - [qualitative research (3)](https://info-profiles.kantar.com/topic/qualitative-research) - [quant vs. qual (3)](https://info-profiles.kantar.com/topic/quant-vs-qual) - [validation (3)](https://info-profiles.kantar.com/topic/validation) - [Amazon (2)](https://info-profiles.kantar.com/topic/amazon) - [Communities (2)](https://info-profiles.kantar.com/topic/communities) - [Customized Advertising (2)](https://info-profiles.kantar.com/topic/customized-advertising) - [Dynamic Sourcing (2)](https://info-profiles.kantar.com/topic/dynamic-sourcing) - [Emerging Markets (2)](https://info-profiles.kantar.com/topic/emerging-markets) - [Emotional Insights (2)](https://info-profiles.kantar.com/topic/emotional-insights) - [Holidays (2)](https://info-profiles.kantar.com/topic/holidays) - [Marketing & Insights Event (2)](https://info-profiles.kantar.com/topic/marketing-insights-event) - [Mobile Advertising (2)](https://info-profiles.kantar.com/topic/mobile-advertising) - [Mobile Banking (2)](https://info-profiles.kantar.com/topic/mobile-banking) - [Mobile Purchasing (2)](https://info-profiles.kantar.com/topic/mobile-purchasing) - [Prediction (2)](https://info-profiles.kantar.com/topic/prediction) - [Survey Health Score (2)](https://info-profiles.kantar.com/topic/survey-health-score) - [Survey Length (2)](https://info-profiles.kantar.com/topic/survey-length) - [Trackers (2)](https://info-profiles.kantar.com/topic/trackers) - [Voice Technology (2)](https://info-profiles.kantar.com/topic/voice-technology) - [facial coding (2)](https://info-profiles.kantar.com/topic/facial-coding) - [minipoll (2)](https://info-profiles.kantar.com/topic/minipoll) - [online surveys (2)](https://info-profiles.kantar.com/topic/online-surveys) - [panelist engagement (2)](https://info-profiles.kantar.com/topic/panelist-engagement) - [research on research (2)](https://info-profiles.kantar.com/topic/research-on-research) - [smart speaker (2)](https://info-profiles.kantar.com/topic/smart-speaker) - [visualization (2)](https://info-profiles.kantar.com/topic/visualization) - [weighting data (2)](https://info-profiles.kantar.com/topic/weighting-data) - [Clients (1)](https://info-profiles.kantar.com/topic/clients) - [Data Breach (1)](https://info-profiles.kantar.com/topic/data-breach) - [Device Agnostic (1)](https://info-profiles.kantar.com/topic/device-agnostic) - [Ecommerce (1)](https://info-profiles.kantar.com/topic/ecommerce) - [Empathetic Questioning (1)](https://info-profiles.kantar.com/topic/empathetic-questioning) - [Multicultural Research (1)](https://info-profiles.kantar.com/topic/multicultural-research) - [PCs (1)](https://info-profiles.kantar.com/topic/pcs) - [Political Profiler (1)](https://info-profiles.kantar.com/topic/political-profiler) - [Predictive Analytics (1)](https://info-profiles.kantar.com/topic/predictive-analytics) - [Research Skills (1)](https://info-profiles.kantar.com/topic/research-skills) - [Science of Prediction (1)](https://info-profiles.kantar.com/topic/science-of-prediction) - [Talent Management (1)](https://info-profiles.kantar.com/topic/talent-management) - [US election (1)](https://info-profiles.kantar.com/topic/us-election) - [audience targeting (1)](https://info-profiles.kantar.com/topic/audience-targeting) - [eWallets (1)](https://info-profiles.kantar.com/topic/ewallets) - [emerging talent (1)](https://info-profiles.kantar.com/topic/emerging-talent) - [esomar (1)](https://info-profiles.kantar.com/topic/esomar) - [health tech (1)](https://info-profiles.kantar.com/topic/health-tech) - [location services (1)](https://info-profiles.kantar.com/topic/location-services) - [nonprobabliity samples (1)](https://info-profiles.kantar.com/topic/nonprobabliity-samples) - [probability samples (1)](https://info-profiles.kantar.com/topic/probability-samples) - [social listening (1)](https://info-profiles.kantar.com/topic/social-listening) see all © 2018, Lightspeed LLC - [Site Map](http://www.lightspeedresearch.com/site-map/) - [Privacy Policy](http://www.lightspeedresearch.com/privacy-policy/) - [Terms of Use](http://www.lightspeedresearch.com/terms-of-use/) [![Lightspeed](https://info-profiles.kantar.com/hs-fs/hubfs/Lightspeed-Sep2016/images/logo_orange_footer.png?width=124&name=logo_orange_footer.png "Lightspeed")](http://www.lightspeedresearch.com)[![](https://info-profiles.kantar.com/hs-fs/hubfs/logos/LightspeedHealth_Footer_Logo_orange_rollover.png?width=200&name=LightspeedHealth_Footer_Logo_orange_rollover.png)](http://www.lightspeed-health.com)[ ![](https://info-profiles.kantar.com/hs-fs/hubfs/logos/FSG_Footer_1.png?width=201&name=FSG_Footer_1.png)](http://www.lightspeedfsg.com)