If I had a dollar for every time a client has asked me this question, I would honestly be able to take my entire family (we have 4 kids) to Disney for a full week. My response is always the same: “Make it as short and simple as you possibly can”. As a matter of fact, I would prefer to change the mindset of clients and have them ask “How short should we make our survey?”
Yes, I know this is an overly simplistic way to address a difficult question however the relationship between survey length and response quality has been explored ad nauseum for years. Research findings clearly show:
- As survey length increases, so does respondent fatigue
- As fatigue sets in, response quality begins to deteriorate
- As response quality deteriorates, data quality deteriorates
Before anyone criticizes these statements, I fully understand there are several variables other than survey length impacting respondent behavior and fatigue. The type of audience, survey topic or content, incentives, programming engagement techniques, etc., all contribute positively or negatively to the degree in which a respondent remains engaged and thoughtful. These factors cannot be overlooked since they play a critical role in every online survey we program. That said, if I had to choose one variable to control over all others, it would be survey length. I choose survey length over incentives because you honestly could not offer me any incentive that would entice me to complete surveys over 40-minutes in length and practical experience indicates I am not in the minority on this issue.
As surveys get longer, we see higher rates of streamliners and mid-interview drop-outs. We also see an increase in “don’t know/refused” responses and a decrease in the number of typed characters in open-end and other specify fields. All of this suggests that the respondents are trying to get to the end of the survey — they are rushing to complete the study and are less willing to commit thoughtful attention to each survey question. At Brookmark, we also monitor respondent progress in every survey we program to see if, where and when respondents attempt to simply skip survey questions. It should come as no surprise that “skipping” behavior increases as survey length increases.
So what should we all be aiming for in terms of survey length? Research conducted by Survey Sampling International, suggests the critical point in response fatigue is the 20 minute mark. Interviews extending beyond this mark result in a more fatigued respondent. Our response will always be the same:“Make it as short and simple as you possibly can”.