In any context there are many questions with factual answers which are difficult to answer: “have you considered an affair?,” “how many vegetables do you eat?,” “how often do you go to the gym?” and “have you lied to your boss?” to name just a few.
When it comes to a question like “how much do you drink” it can be hard enough to be honest with ourselves let alone a researcher! Fortunately the anonymity and context of online research puts it in a unique position to secure honest answers to sensitive questions; however, this is no easy feat. When we ask a question there are many hurdles we must overcome to reach an honest answer.
Some of these are unique to online research. If we ask someone ‘which of these do you own?’ for instance, there’s an array of biasing factors that can impact our responses:
- Acquiescence bias creates over-claim simply because we like to say yes.
- You may have owned an item in the past or plan to buy one, that’s basically a yes… right?
- You could be curious to see what questions come up if you click something you like.
- You’ve got something close, I’m sure that counts…
- There’s the dreaded miss-click.
- Sod it I’ll lie, why not! I’m sure my answers will still be interesting for them.
Moreover in any context there are a plethora of barriers to the truth that aren’t unique to survey research and go beyond simple questions of ownership. Some questions are embarrassing to answer, some are tricky to put a number on and others can be hard to remember.
In a survey these issues can be further exacerbated by privacy concerns and the desire to get through the questions quickly. Whatever the issue, the result however is the same - we cannot take an honest answer for granted.
To help the researcher in navigating this minefield, Lightspeed GMI embarked on a multination project to establish the best means of securing honest answers. By experimenting with different methodologies and observing in the data the amount of confessions we elicited, we were able to evaluate which produced the most honest responses. Providing a toolset to exploit the online survey’s unique ability to illicit an invaluable level of honesty.
Our findings divided the issue into three main categories: ‘Questions we’re scared to tell the truth about,’ ‘Questions we lie to ourselves about’ and ‘Questions we want to lie about.’ Each category presenting its own set of issues.
The investigation into these three categories utilised: a series of sensitive questions such as “Do you always wash your hands in the toilet?,” the self-observation question of "How much alcohol do you drink?" and the over-claim tempting question “Do you own a tablet?
The conclusion of our theory and research has been the development of the following three doctrines:
- Questions we’re scared to tell the truth about
When you ask someone a sensitive questions that they might be scared or embarrassed to answer, the determining factor in whether they feel comfortable giving you a truthful answer is your relationship with them. In a survey you only have one opportunity to build a rapport that puts your respondent at ease, and that’s the introduction.
Our research found that the key to getting people to answer sensitive questions is to engage them emotively and firmly with a powerful introduction. The most successful approach for us was one which broke down our key messages of trust framed within the challenge; “Can you answer our questions as if no one was listening?” and followed by the pledge “I promise to tell the truth!”
We’re not suggesting every survey should have a pledge; but ultimately our approach was much more effective than a traditional introduction because it didn’t make our respondents ever think ‘what is that to you?’ when asked a sensitive question!
- Questions we lie to ourselves about
The self-deception at play when you ask respondents to make an observation about their behaviour is not something that a strong rapport can solve. In fact our research found the barriers to truth here are not necessarily a matter of honesty or deception at all.
The biggest obstacle a respondent faces when trying to make an accurate self-observation is simply calculating the value. We get them most truthful data not when we attempt to combat dishonesty; but when we break down the task to make it simple for the respondent to evaluate, and by giving a time frame rather than burdening the respondent with ambiguous task of averaging, a task that should be the researcher’s anyway!
- Questions we want to lie about
When asking questions which might tempt respondents to actively lie we go beyond the truth invoking powers of a strong rapport. Instead we need to be craftier and ask the question without them realising we have asked it!
We got the most truthful responses to our tablet ownership question when we disguised the question by asking it less directly. Asking instead if they wanted a tablet first; then if they said ‘yes’ asking if this was to replace an old one, and if they said ‘no’ asking if this was because they already had one.
However our research showed that while we should ask our question in a less direct form, we must remember to still ask it directly; avoid the temptation to utilise a question format which does not force an answer as component based selection biases strongly influence data.
