We're in the holiday spirit at School Perceptions. In the run-up to the holidays, we're very busy presenting fall survey reports to administrators and school boards across the country. When we present survey reports, we always open with our mission. And we always mention that it’s never changed: we help you gather, organize, and use data to make strategic decisions. The first part of this is the focus of today. In order to organize and use data, we must ensure that we gather credible (i.e., valid) data.
Validity has a particular meaning in the survey world. It can be tricky, but it is vital to the work we do at School Perceptions. I cover three types of validity in this post. I will go over each in detail, then discuss why validation matters and how we use these validation techniques.
Content validity asks if a measure captures the full meaning of the construct it is designed to measure. Imagine a survey designed to measure love of the Green Bay Packers that only asked respondents how many articles of gold clothing they owned. Any Wisconsinite knows that there are many aspects to Packer fandom beyond just wearing the green and gold. This tells us that our survey has low content validity; it doesn’t measure all aspects of our construct (Packer fandom) and could be inaccurate.
Concurrent validity is a bit simpler. It asks if the measure agrees with another, more well-established one. Predictive validity refers to how accurately a measure predicts future success, like judging the validity of a college entrance exam based on first semester grade point averages.
Using validated surveys ensures that we at School Perceptions provide accurate data and recommendations to our clients. If we want to honestly measure our respondents’ perceptions about schools, then we need to continually assess the content and predictive validity of our surveys.
The two main tools we use for this are item and factor analysis. Item analysis is a process by which we test each survey question against overall scores to make sure they actually measure our construct of interest. This lets us remove or change ineffective items and shorten our surveys. Factor analysis groups questions based on how respondents answer them to let us confirm the organization of our surveys and to discover any unknown factors influencing responses.
School Perceptions uses these tools and concepts to improve our product and the recommendations we offer to clients. If you have any questions about survey validation or how we use it in our work, please contact Thomas Wade at firstname.lastname@example.org.
The School Perceptions Blog and Resource Center features the voices of our team members. This post was written by Thomas Wade, Operations Manager.