Those who know GMI know that we have dedicated a significant amount of time to studying creative survey techniques. We have done a lot of research on research, and found, as other researchers have, that gamification techniques have been shown to increase data quality by reducing dropout rates and effectively engaging survey respondents.

But two questions had been unanswered in our work until recently: First, how well do these techniques work in non-Western cultures? And second, do they affect cultural response style biases? Lately, these questions have become increasingly important as the number of our market research clients requesting multi-country studies is on a steep incline. In addition, because there had been very little academic-based scrutiny of these techniques, we were eager to introduce that element into further study.

So with Duncan Rintoul of the Institute for Innovation in Business and Social Research at the University of Wollongong, Australia, GMI conducted research into these questions and presented the results in a paper at the ESOMAR Asia Conference earlier this year. We are pleased to report that it won Best Paper at the conference.

To answer the first question– do creative and gaming techniques cross borders — in a word, yes. After interviewing nearly 3,900 respondents in seven countries (U.S., Australia, China, India, Japan, Singapore and Korea), we concluded that the instinct to play seems to be a fundamental human characteristic that transcends cultural boundaries. The creative and gaming techniques that work in Western markets seem to work equally well, if not better, in the Asian markets we tested.

Our experiments pitted standard market research question formats against various creative techniques, such as employing role-play scenarios, gambling and competitive point scoring, graphics in questions and response options, and visually oriented response formats. In nearly every case in all seven countries, the creative techniques produced improvements in consideration time and respondent engagement, just as we had seen in previous single-market experiments.

Cultural Response Style Bias

The answer to the second question — do creative and gaming techniques affect cultural response style biases — is less clear cut.

To understand the answer, we must first understand the question. So, what is cross cultural response style bias? The term refers to the phenomenon of different groups of respondents using answer formats in different ways. In short, people from different countries answer surveys in different ways. An example often discussed in academic literature is the tendency to use only certain response options, like the “extreme” response style, which uses only the top and bottom response options, the “central” style, which uses only the neutral options, and the “positive” style, which uses just the top half of the scale. The existence of this bias is known to substantively impact data in multi-country studies, but as yet a satisfactory solution has not been found. We wanted to know whether creative questioning techniques could help.

We analyzed survey responses from 15 large scale multi-country studies, looking at answer patterns for over 1,000 questions. With this analysis as the background, we interviewed nearly 3,900 respondents in seven countries (U.S., Australia, China, India, Japan, Singapore and Korea). We concluded that creative techniques can help reduce the impact of cultural response style bias in three ways:

1. They reduce speeding and straight-lining. Where these behaviors are more prevalent in some countries than others, this helps to level out the results.
2. They allow respondents to make comparisons to choose among available responses, which reduces cultural bias.
3. They use visuals to help overcome language and translation issues.It is important to note that these techniques stop a long way short of eradicating cross-cultural style bias. But they do hold promise, as they allow us to significantly reduce the level of distortion which, in certain markets can have a major influence on data.

To learn more about GMI Interactive, GMI’s state-of-the-art survey design technology and data capture techniques, visit the GMI website. You can purchase GMI’s award-winning paper “Can Survey Gaming Techniques Cross Continents? Examining cross cultural reactions to creative questioning techniques” by visiting ESOMAR’s website.

- See more at: http://www.ls-gmi.com/data-quality/can-survey-gaming-techniques-cross-continents/#sthash.mKkGnyeT.dpuf