A Better Survey Experience: What People Dislike About Surveys
The role of Insights is to understand. Our job is to listen to the voice of the people, and solve business problems based on what we have heard. But to have a good conversation with people, we need to create a good survey experience. To better understand how to do that, we asked what people dislike about surveys, so that we knew what not to do.
Long is too long
Surveys that are too long, are a very common dislike — especially surveys that are longer than advertised. It leads to poor quality information. As one person told us: “When surveys drag on too long, then you lose interest and don’t give the answers that really mean something.”
The long survey reflects both the heritage of survey research, a greed for information and a disregard for the time pressures of modern life. When survey research began, interviews were face to face and long surveys were common — because it was expensive and slow to send someone out to do an in-person interview. Companies took advantage of the opportunity and crammed in every question they could. Besides, they didn’t know anything about the people they were talking to, so they started from scratch with each interview. Also, having a face to face interviewer present meant there was someone whose job it was to keep people engaged. The times have changed.
Now, the cost of doing a survey is infinitesimally less expensive, and we know a great deal about the well-profiled people who belong to our communities. There is no excuse for the long survey today.
Being disqualified
Another common complaint was being asked a series of screening questions only to be told you don’t qualify for the study. My colleague, Rob Berger of sister company Maru/Blue, covered this in an earlier article. The good news is that, with careful profiling, most disqualifications can be avoided — because you already know the person you are inviting to the study qualifies.
Difficult to answer questions
Another common complaint is long questions, often grids, which are time-consuming, repetitive, and difficult to answer. We did another study, in which we invited people to complete a survey with a tedious set of grid questions about media habits and their perceptions of different media companies. It was difficult but, sadly, not unusual. After about 12 minutes of this, some people had thrown their hands up and dropped out. But others soldiered on, used to the experience. We asked the persistent to turn on their video cameras so that we could use facial coding to determine how they felt.
We saw some contempt.
And disgust.
But mostly what we saw was boredom.
Here is what people had to say about that experience.
A better experience
To maintain the quality of our research, we need to step away from practices that are negative for the people who do our surveys.
Long grids are obviously a problem. Do we really need all those brands rated on all those attributes? Our research has shown that in any big attribute bank there are many attributes that are measuring the same underlying concept. Look at the correlations, or run a factor analysis of your last study that used those attributes, and you’ll see that you can probably drop two-thirds of the items because they are duplicative. The same with attitudes statements.
And skip the scaled ratings. Our research has shown that binary brand ratings are three times faster to answer, and provide the same insights. Doing these things also makes studies more mobile friendly, which is another pet peeve — especially for younger people.
If long surveys are tiresome, we must embrace shorter surveys. A more agile way of conducting research with shorter surveys is the future. It fits better with the speed of business today. The era of “I’ll get back to you with your answer in 8 weeks” is drawing to a close. According to client-side researchers I have been interviewing, time is of the essence.
We know that being disqualified is a big problem, but thankfully there is an easy answer: careful and well thought out profiling. It may take a little planning, but it makes a huge difference to the people who do our surveys.
When a person doing our surveys complains that with long surveys “you lose interest and don’t give the answers that really mean something,” I take that very seriously. The whole reason to do a survey is to get answers that mean something.
Let’s heed the voice of people, and avoid long, repetitive surveys.
Also Read
The Plea to Think of Respondents as People Not Sample
The People’s Voice: Feedback From Those Who Fuel Our Industry (Power Point Presentation)
To make sure you stay up to date with our latest content, sign up for our articles.
