Getting feedback from panelists gives you a perspective about survey design that you sometimes miss. They are honest and straight to the point. Whether on PC or mobile, respondents consider grid questions tedious and rate surveys with more grid questions as less enjoyable. Comments include:
- Too many questions on the pages. Too many columns. Difficult keeping track of what box to check.
- This survey was HORRIBLE. It was TOO wide & TOO long to fit the screen so I had to keep sliding back & forth.
- I hate having to fill in so many bubbles.
- I can’t stand surveys with buttons across and all those rows down. They give me a headache.
Grid questions are an even bigger issue on mobile devices because of the limited visual real estate. There are a number of specific issues with grid questions and mobile devices:
- In many cases a grid simply does not display well on mobile devices because it runs off two sides of the screen.
- If a grid does fit on a page, then the selection space can be so small that respondents are unable to choose the option(s) they want. The old ‘fat fingers’ problem.
- Grids will sometimes display better if respondents turn their phone so the display is landscape, but there is no guarantee that respondents will do this.
- Pinching the screen so the entire questions displays makes the text so small it isn’t readable plus not all respondents do this or even know to do this.
Given all these difficulties, mobile may truly lead to the death of grids or at least the death of large grids. There is no magic replacement or display for large grids that keeps respondents engaged. Researchers have tried different options but it is like putting a formula one engine in a VW, it still handles like a VW. No matter what you do, a grid is still a grid. Some questionably approaches include:
- Breaking a large grid into several smaller grids, but when several grid questions are asked in a row the dropout rate increases with each additional grid.
- Breaking the grid down into completely separate questions, but that become repetitious for the respondent.
- Using drop down boxes rather than the grid, but then that just requires an extra step of clicking on the box to activate it.
Even newer tools like dynamic grids have their limits. Dynamic grids present statements, logos, brand names, etc. on 3D cards above the answer choices. As each response is given, the top card flips to the left and the next card flips in from the right. Just like a traditional grid the more items in a dynamic grid the less enjoyable the survey and the higher the dropout rate. Research suggests the sweet spot to be 14-15 items in a dynamic grid. With mobile devices that number is likely lower.
Truly the best way to handle a grid is to reduce the size to no larger than 6 x 6. This requires identifying the need to know information while getting rid of the nice to know information. Removing redundant items and attributes from the list can also reduce the size. Ultimately, you need to get to something that will render adequately on the smallest screens with legible writing and no scrolling.
Grids may never completely die because they allow researchers to capture a lot of information in the fewest pages. However, we clearly need to be aware of how respondents are evolving and how they complete surveys. With more and more respondents answering surveys via their mobile devices we truly need to adapt grid question or potentially see fewer and fewer willing respondents.