This blog post is part of our surveying best practices series that we’re featuring during the summer of 2021. The series will highlight important lessons we’ve learned over the past 20 years.
This week, I’ll focus on the part of the survey that often becomes an after-thought: the response scale. When we’re designing a new survey, though, I often tell my team to begin here (the before-thought?). I say this because of the four reasons below.
1. Use answers that will answer your questions.
This sounds obvious, but that isn’t always so. Like I mentioned last week, you should begin the survey process by understanding why your survey is important, why it’s being conducted, how the results will be used, and what you expect to learn.
For example, consider one of our community surveys. If your school board wants to better understand what’s most important to the public, a priority scale (high, medium, low) makes a great deal of sense. This will allow the board to “back plan” and say to themselves, “We have a series of needs, but we only have a limited pool of dollars.
While not intuitive, this approach may be more productive then asking a simple yes-no scale in response to, “Would you support the project?” These results are very easy to interpret, but they can hinder the decision-making process, leaving it difficult to fund any project that did not reach the perceived 50 percent “yes” threshold.
2. Vary but don’t overwhelm.
This is, admittedly, a very fine balance. A survey, especially if it’s a little longer, can get monotonous with one scale. Survey-takers tend to zone out and put less thought in each response. On the other hand, a different scale on each page will only serve to frustrate survey-takers. The survey gets confusing, the mental load gets to be too much, questions must be re-read and re-re-read, and many people simply quit. Obviously, this is not what we want to happen.
Just like life, change is good … but not too much.
3. To be [neutral]? Or not to be [neutral]?
We’ve had this conversation many times throughout School Perceptions 20-year history. In most cases, we do not include a neutral answer response.
When a neutral option is present, people “satisfice.” They stick to the middle of the road because the middle tends to be safe, and safety is appealing. And ambivalence keeps you safe. Survey-takers feel like they won’t hurt anyone’s feelings with this response.
However, that’s not necessarily good for a survey (again, see last week!). Continually selecting neutral decreases the cognitive load of the survey-taker, i.e., people don’t think very critically. Put simply, people gravitate toward neutral because it’s there, and it allows for easy answers without much thought. When neutrality is removed, answers are thought about more deeply and become more reliable. (By the way, we’ve used this resource for some of our support here. Credit’s due.)
4. Who’s taking the survey?
Our survey system is designed to gather feedback from staff, parents, community members, and students. But, obviously, students are a special group here as non-adults. We want to glean useful information for the adults in the room, but useful information isn’t gleaned unless students a) take and finish the survey and b) answer the questions truthfully. If a response scale is too confusing for students for their specific developmental level or if the response scale is, well, just boring, students are not going to be engaged. That’s not good for you, and it’s not good for us.
This past year, we piloted a new set of student surveys with students at the Slinger School District in southeastern Wisconsin. These surveys included an emoji-based agreement scale.
Before we shared this survey far and wide, we (virtually) sat down with students to understand if the emoji scale was a little childish and hokey or if it was actually useful. The feedback we got was overwhelmingly in support! The kids like it, thought it made the survey more interactive, fun, and interesting, and spoke to them in a way they speak to each other.
Do not underestimate the importance of your survey’s response scale. It gets read for every question!
TBD on our use of GIFs and memes yet though…
The School Perceptions Blog and Resource Center features the voices of our team members. This post was written by Bill Foster, President & Founder.