Had a discussion with a client the other day over “vertical” vs. “horizontal” layout of online questions. We were discussing potential issues around the “never” option always appearing off the screen, requiring scrolling to see that answer choice.

During the programming of a recent ad test, the “ad preference” question was programmed vertically – ie. 3 print ads were shown, but only 1 visible at a time, and 2 could only be seen if the respondent scrolled down. I’ll save the Usability lesson on ‘Task Workload” for another post, but wanted to ask this:

If what we are asking a respondent to rate, or comment on is not visible without scrolling up, down and all around – is that a source of potential bias?

To mitigate this, make sure that what you are showing/asking fits on one screen. In the case of this print ad test, I asked the programmer to resize the images so they could appear horizontally (and rotated) onscreen, so no scrolling was required for respondents to evaluate their options.