Reading and analyzing respondents' comments helps provide valuable context and gives you a better understanding of your survey results. Researchers use comment analysis (qualitative data) and survey stats (quantitative data) together to provide more accurate and useful plans. This post explains the process School Perceptions uses to create our comment analysis reports and covers a few ways to use this information.
Our analysis process starts with reading through each and every comment and recording the main idea of each. These themes are a phrase or word that summarizes the message of a comment. After this initial read-through, we reread the comments to ensure our first-round analyses were on target. In the research parlance, this is cyclical or iterative—i.e., it takes a few times through. We identify four to five of respondents’ most prevalent themes, discarding the rest or making them sub-themes. This process is called “theming.”
The next step is coding, by which each comment is grouped under the theme that best represents it. A comment can be coded under one or more themes or placed into an “Other” category if none of the main themes represent it. During this process, it’s essential to keep in mind that opposing views are a good thing since they show the variety of ideas surrounding a theme. If everyone agrees on a certain point, it might be worth changing the theme’s name to reflect that. Once the comments are themed and coded, we have another analyst review them to make sure they are sensible and accurately reflect the ideas expressed in the comments.
Once you, on your end, analyze this report, it’s time to decide what to do with them. The easiest way to use your results is to let respondents know you listen to what they have to say. Communicating the main takeaways or sharing the district’s action plan with your respondents helps them feel like their voices are being heard. Additionally, seeing that the district takes their feedback into account can encourage participation in future surveys!
Comment analysis can also provide context for your survey data. If your district has low scores in a given area, looking through related comments can give you more detail/specific problems that need to be addressed. We refer to this as “nuancing the data.” Positive comments can be helpful too. Learning what changes or policies people are happy with can let you know if you’re headed in the right direction.
One note of caution: comment theme analyses provide a wealth of data. However, it can be difficult to know just how representative the comments are. Very few people who shrug their shoulders and say, “Meh…” choose to leave a comment. Commenters are typically your most passionate people. Thus, the comments might be representative, but our approach is primarily quantitative. Representative qualitative data analyses can absolutely be done through long-term focus groups, interviews, and other written communications.
Combining quantitative and qualitative results is one of the most powerful ways to draw accurate conclusions from your survey. School Perceptions helps clients by offering comment theming and analysis services alongside our suite of surveys.
Utilizing this data makes respondents feel heard and encourages them to engage with the district.
If you have any questions about comment analyses or want to share a technique that has worked for you, leave a comment below or contact me at twade@schoolperceptions.com.
Thanks for reading and have a relaxing spring break!
The School Perceptions Blog and Resource Center features the voices of our team members. This post was written by Tom Wade, Project Manager.