--- title: Reducing false positives in survey data quality checks description: Marketing research clients include quality checks in surveys to make sure respondents are engaged; however, many of these checks identify false positives. image: https://info-profiles.kantar.com/hubfs/iStock_000012995878Small_jpg.jpg --- - - - - [mailto:info@gmi-mr.com](mailto:info@gmi-mr.com) [![Kantar-Logo-Small-Use-Black](https://info-profiles.kantar.com/hs-fs/hubfs/logos/Kantar-Logo-Small-Use-Black.png?width=119&name=Kantar-Logo-Small-Use-Black.png)](http://www.lightspeedresearch.com) - [Home](http://www.lightspeedresearch.com/) - [Sample Audiences](http://www.lightspeedresearch.com/sample-audiences/) - [Quality Suite](http://www.lightspeedresearch.com/sample-audiences/quality-suite/) - [Panel Profiling](http://www.lightspeedresearch.com/sample-audiences/panel-profiling/) - [B2B Panels](http://www.lightspeedresearch.com/sample-audiences/business-to-business-panels/) - [Custom Panels](http://www.lightspeedresearch.com/sample-audiences/custom-panels/) - [Millennial Panel](http://www.lightspeedresearch.com/sample-audiences/vice-voices/) - [Research Solutions](http://www.lightspeedresearch.com/research-solutions/) - [Quick Turn Surveys](http://www.lightspeedresearch.com/research-solutions/lightspeed-quick-turn-surveys/) - [Survey Design](http://www.lightspeedresearch.com/research-solutions/survey-design/) - [Tracking Research Design](http://www.lightspeedresearch.com/research-solutions/tracking-research-design/) - [Integrated Technology Solutions](http://www.lightspeedresearch.com/research-solutions/integrated-technology-solutions/) - [Data Processing](http://www.lightspeedresearch.com/research-solutions/data-processing/) - [Ad Tracker](http://www.lightspeedresearch.com/research-solutions/ad-tracker/) - [Survey Programming](http://www.lightspeedresearch.com/research-solutions/ad-tracker/) - [Sample Only Services](http://www.lightspeedresearch.com/research-solutions/sample-only-services/) - [Data Services](http://www.lightspeedresearch.com/data-services) - [Data Appends](http://www.lightspeedresearch.com/data-services/data-appends/) - [Mobile](http://www.lightspeedresearch.com/mobile/) - [Mobile Solutions](http://http://www.lightspeedresearch.com/mobile/) - [Resource Center](http://www.lightspeedresearch.com/resource-center/) - [Events](http://www.lightspeedresearch.com/resource-center/events/) - [Demos](http://www.lightspeedresearch.com/resource-center/demos/) - [Case Studies](http://www.lightspeedresearch.com/resource-center/case-studies/) - [Blog](https://blog.lightspeedresearch.com/) - [General Inquiries](http://www.lightspeedresearch.com/contact-us/general-inquiries/) - [About Us](http://www.lightspeedresearch.com/about-us/) - [Leadership Team](http://www.lightspeedresearch.com/about-us/leadership-team/) - [Press Releases](http://www.lightspeedresearch.com/about-us/news-press-releases/) - [Why Lightspeed](http://www.lightspeedresearch.com/about-us/why-lightspeed/) - [Careers](http://www.lightspeedresearch.com/about-us/careers/) - [Regions](http://www.lightspeedresearch.com/about-us/regions/) - [Awards](http://www.lightspeedresearch.com/awards/) - [Contact Us](http://www.lightspeedresearch.com/contact-us/) # Kantar's Profiles Blog # Reducing false positives in survey data quality checks Posted by [Kantar](https://info-profiles.kantar.com/author/kantar) on May 9, 2016 ![](https://app.hubspot.com/settings/avatar/d41d8cd98f00b204e9800998ecf8427e) - [Tweet](https://twitter.com/share) Many clients include quality checks in surveys to make sure respondents are engaged and are answering honestly. However, many of these checks identify false positives, which often mean valid, engaged respondents are thrown out of the sample. How can we reduce false positives?  First and foremost, don’t use a one strike and you’re out rule. Respondents need to be given the benefit of the doubt and not removed because they fail just one quality check. A respondent can easily be [momentarily distracted](https://blog.lightspeedgmi.com/still-with-me-ten-ways-to-drive-survey-engagement) and fail a quality check. This doesn’t make them a poor respondent…it makes them human! The key is to look for a pattern of questionable behavior with multiple failures.  Even on individual checks there is a need to avoid the one strike and you’re out rule. One popular check is to eliminate people who claim to participate in low incidence activities or purchase low incidence categories. There is a good chance some respondents could choose one activity or category, so it is better to use two plus activities/categories as the quality check. Also, the list of activities and categories needs to be carefully devised so items aren’t related to one another. When activities/categories are related it increases the odds that people saying two plus activities/categories can be valid answers. For example, someone attending an opera in the past year may be more likely to have also attended the ballet in the past year.  Red herrings are another popular check. This is where fake items are included in the answer list. In some instances all answer choices might be red herrings. It is difficult to devise fake items that aren’t confused with real items. For example, a fake brand may be very similar to a real brand or could even be a valid regional brand. On some recent research respondents were asked to pick which of six fake items they had purchased in the past six months. We found that respondents choosing 1-3 items were spending more time answering the question than those choosing 4+ items. Based on the time to answer, the recommendation was to set the failure criteria at 4+ items.  ![Quality Checks in Online Surveys ](https://info-profiles.kantar.com/hs-fs/hubfs/FalsePositives.jpg?width=320&name=FalsePositives.jpg "Quality Checks in Online Surveys ") Clients also often use open ended responses as a quality check. In this check nonsense answers, such as a random set of letters, are identified. This check can be tricky though. [Some respondents simply don’t like open ended questions](https://blog.lightspeedgmi.com/debunking-marketing-research-myths-mobile-open-ends-dont-mix/) but give perfectly valid answers on all other questions. Also the number of failures can vary widely depending on the type of open ended question and the location of the question in the survey. Often once respondents who fail other checks are eliminated it reduces the number of nonsense answer on open ended questions to negligible levels. For example, in recent research we went from 6.8% nonsense answers to 1.7% after removing those who failed other quality checks.   Bottom line, carefully design quality checks and require multiple failure points to avoid getting rid of false positives.   TO CONDUCT EFFECTIVE ONLINE RESEARCH, YOU SHOULD EXPLORE A NUMBER OF RESEARCH TECHNIQUES.  [![DOWNLOAD FOUR TIPS TO MAXIMIZE YOUR MARKETING RESEARCH ROI](https://no-cache.hubspot.com/cta/default/641505/9e7c25ce-f34e-40a6-a4c4-5fb051e101bf.png)](https://cta-redirect.hubspot.com/cta/redirect/641505/9e7c25ce-f34e-40a6-a4c4-5fb051e101bf)   Topics: [Research Quality](https://info-profiles.kantar.com/topic/research-quality), [Marketing Research](https://info-profiles.kantar.com/topic/marketing-research), [Data Quality](https://info-profiles.kantar.com/topic/data-quality) ### Subscribe to Email Updates ### Recent Posts ### Posts by Topic - [Marketing Research (46)](https://info-profiles.kantar.com/topic/marketing-research) - [Market Research (28)](https://info-profiles.kantar.com/topic/market-research) - [Marketing Research Data (28)](https://info-profiles.kantar.com/topic/marketing-research-data) - [modern surveys (27)](https://info-profiles.kantar.com/topic/modern-surveys) - [Mobile (26)](https://info-profiles.kantar.com/topic/mobile) - [Data Driven Marketing (21)](https://info-profiles.kantar.com/topic/data-driven-marketing) - [Market Research Trends (18)](https://info-profiles.kantar.com/topic/market-research-trends) - [Consumer Insights (17)](https://info-profiles.kantar.com/topic/consumer-insights) - [Online Sampling (15)](https://info-profiles.kantar.com/topic/online-sampling) - [Survey Design (15)](https://info-profiles.kantar.com/topic/survey-design) - [Blog Post (13)](https://info-profiles.kantar.com/topic/blog-post) - [Data Quality (13)](https://info-profiles.kantar.com/topic/data-quality) - [Digital Data Collection (12)](https://info-profiles.kantar.com/topic/digital-data-collection) - [Financial Services (11)](https://info-profiles.kantar.com/topic/financial-services) - [APAC (10)](https://info-profiles.kantar.com/topic/apac) - [Healthcare Research (10)](https://info-profiles.kantar.com/topic/healthcare-research) - [data series (10)](https://info-profiles.kantar.com/topic/data-series) - [Video (9)](https://info-profiles.kantar.com/topic/video) - [Digital Consumer (8)](https://info-profiles.kantar.com/topic/digital-consumer) - [data append (8)](https://info-profiles.kantar.com/topic/data-append) - [marketing research best practices (8)](https://info-profiles.kantar.com/topic/marketing-research-best-practices) - [mobile first (8)](https://info-profiles.kantar.com/topic/mobile-first) - [mobile marketing research (8)](https://info-profiles.kantar.com/topic/mobile-marketing-research) - [Research Quality (7)](https://info-profiles.kantar.com/topic/research-quality) - [marketing research technologies (7)](https://info-profiles.kantar.com/topic/marketing-research-technologies) - [marketing research trends (7)](https://info-profiles.kantar.com/topic/marketing-research-trends) - [thought leadership (7)](https://info-profiles.kantar.com/topic/thought-leadership) - [Ask the Expert (6)](https://info-profiles.kantar.com/topic/ask-the-expert) - [Context (6)](https://info-profiles.kantar.com/topic/context) - [Panels (6)](https://info-profiles.kantar.com/topic/panels) - [Social Media (6)](https://info-profiles.kantar.com/topic/social-media) - [data enrichment (6)](https://info-profiles.kantar.com/topic/data-enrichment) - [Innovation (5)](https://info-profiles.kantar.com/topic/innovation) - [MRX Trends (5)](https://info-profiles.kantar.com/topic/mrx-trends) - [Sourcing (5)](https://info-profiles.kantar.com/topic/sourcing) - [behavioral data (5)](https://info-profiles.kantar.com/topic/behavioral-data) - [Big Data (4)](https://info-profiles.kantar.com/topic/big-data) - [Honesty Detector (4)](https://info-profiles.kantar.com/topic/honesty-detector) - [Lightspeed FSG (4)](https://info-profiles.kantar.com/topic/lightspeed-fsg) - [Millennials (4)](https://info-profiles.kantar.com/topic/millennials) - [Online access panels (4)](https://info-profiles.kantar.com/topic/online-access-panels) - [Privacy (4)](https://info-profiles.kantar.com/topic/privacy) - [Respondent experience (4)](https://info-profiles.kantar.com/topic/respondent-experience) - [media buying (4)](https://info-profiles.kantar.com/topic/media-buying) - [mobile surveys (4)](https://info-profiles.kantar.com/topic/mobile-surveys) - [questionnaire design (4)](https://info-profiles.kantar.com/topic/questionnaire-design) - [Brand Advertisting (3)](https://info-profiles.kantar.com/topic/brand-advertisting) - [Infographics (3)](https://info-profiles.kantar.com/topic/infographics) - [Survey Engagement (3)](https://info-profiles.kantar.com/topic/survey-engagement) - [data search (3)](https://info-profiles.kantar.com/topic/data-search) - [disABILITYincites Partnership (3)](https://info-profiles.kantar.com/topic/disabilityincites-partnership) - [gdpr (3)](https://info-profiles.kantar.com/topic/gdpr) - [qualitative research (3)](https://info-profiles.kantar.com/topic/qualitative-research) - [quant vs. qual (3)](https://info-profiles.kantar.com/topic/quant-vs-qual) - [validation (3)](https://info-profiles.kantar.com/topic/validation) - [Amazon (2)](https://info-profiles.kantar.com/topic/amazon) - [Communities (2)](https://info-profiles.kantar.com/topic/communities) - [Customized Advertising (2)](https://info-profiles.kantar.com/topic/customized-advertising) - [Dynamic Sourcing (2)](https://info-profiles.kantar.com/topic/dynamic-sourcing) - [Emerging Markets (2)](https://info-profiles.kantar.com/topic/emerging-markets) - [Emotional Insights (2)](https://info-profiles.kantar.com/topic/emotional-insights) - [Holidays (2)](https://info-profiles.kantar.com/topic/holidays) - [Marketing & Insights Event (2)](https://info-profiles.kantar.com/topic/marketing-insights-event) - [Mobile Advertising (2)](https://info-profiles.kantar.com/topic/mobile-advertising) - [Mobile Banking (2)](https://info-profiles.kantar.com/topic/mobile-banking) - [Mobile Purchasing (2)](https://info-profiles.kantar.com/topic/mobile-purchasing) - [Prediction (2)](https://info-profiles.kantar.com/topic/prediction) - [Survey Health Score (2)](https://info-profiles.kantar.com/topic/survey-health-score) - [Survey Length (2)](https://info-profiles.kantar.com/topic/survey-length) - [Trackers (2)](https://info-profiles.kantar.com/topic/trackers) - [Voice Technology (2)](https://info-profiles.kantar.com/topic/voice-technology) - [facial coding (2)](https://info-profiles.kantar.com/topic/facial-coding) - [minipoll (2)](https://info-profiles.kantar.com/topic/minipoll) - [online surveys (2)](https://info-profiles.kantar.com/topic/online-surveys) - [panelist engagement (2)](https://info-profiles.kantar.com/topic/panelist-engagement) - [research on research (2)](https://info-profiles.kantar.com/topic/research-on-research) - [smart speaker (2)](https://info-profiles.kantar.com/topic/smart-speaker) - [visualization (2)](https://info-profiles.kantar.com/topic/visualization) - [weighting data (2)](https://info-profiles.kantar.com/topic/weighting-data) - [Clients (1)](https://info-profiles.kantar.com/topic/clients) - [Data Breach (1)](https://info-profiles.kantar.com/topic/data-breach) - [Device Agnostic (1)](https://info-profiles.kantar.com/topic/device-agnostic) - [Ecommerce (1)](https://info-profiles.kantar.com/topic/ecommerce) - [Empathetic Questioning (1)](https://info-profiles.kantar.com/topic/empathetic-questioning) - [Multicultural Research (1)](https://info-profiles.kantar.com/topic/multicultural-research) - [PCs (1)](https://info-profiles.kantar.com/topic/pcs) - [Political Profiler (1)](https://info-profiles.kantar.com/topic/political-profiler) - [Predictive Analytics (1)](https://info-profiles.kantar.com/topic/predictive-analytics) - [Research Skills (1)](https://info-profiles.kantar.com/topic/research-skills) - [Science of Prediction (1)](https://info-profiles.kantar.com/topic/science-of-prediction) - [Talent Management (1)](https://info-profiles.kantar.com/topic/talent-management) - [US election (1)](https://info-profiles.kantar.com/topic/us-election) - [audience targeting (1)](https://info-profiles.kantar.com/topic/audience-targeting) - [eWallets (1)](https://info-profiles.kantar.com/topic/ewallets) - [emerging talent (1)](https://info-profiles.kantar.com/topic/emerging-talent) - [esomar (1)](https://info-profiles.kantar.com/topic/esomar) - [health tech (1)](https://info-profiles.kantar.com/topic/health-tech) - [location services (1)](https://info-profiles.kantar.com/topic/location-services) - [nonprobabliity samples (1)](https://info-profiles.kantar.com/topic/nonprobabliity-samples) - [probability samples (1)](https://info-profiles.kantar.com/topic/probability-samples) - [social listening (1)](https://info-profiles.kantar.com/topic/social-listening) see all © 2018, Lightspeed LLC - [Site Map](http://www.lightspeedresearch.com/site-map/) - [Privacy Policy](http://www.lightspeedresearch.com/privacy-policy/) - [Terms of Use](http://www.lightspeedresearch.com/terms-of-use/) [![Lightspeed](https://info-profiles.kantar.com/hs-fs/hubfs/Lightspeed-Sep2016/images/logo_orange_footer.png?width=124&name=logo_orange_footer.png "Lightspeed")](http://www.lightspeedresearch.com)[![](https://info-profiles.kantar.com/hs-fs/hubfs/logos/LightspeedHealth_Footer_Logo_orange_rollover.png?width=200&name=LightspeedHealth_Footer_Logo_orange_rollover.png)](http://www.lightspeed-health.com)[ ![](https://info-profiles.kantar.com/hs-fs/hubfs/logos/FSG_Footer_1.png?width=201&name=FSG_Footer_1.png)](http://www.lightspeedfsg.com)