Making the objective subjective: A sociopsychometric exploration of fairness and standardized testing
This study examined potential differences among legal professionals, measurement professionals and members of the general public in their evaluations of fairness in response to vignette-based presentation of hypothetical standardized testing scenarios and explored possible differences in generalized attitudes and beliefs toward standardized testing held by the three groups. Vignettes presented to study participants varied in the type of testing application presented (diploma vs. merit), whether opportunities were provided for stakeholder input into the testing process (or "voice"), whether evidence of content validity was demonstrated, and whether disparate outcomes were experienced by minority and non-minority students as a result of testing. A total of 857 participants completed the study.
Significant differences were found among fairness ratings provided by measurement professionals, legal professionals, and members of the public. The type of testing application, opportunity for voice, presentation of validity evidence, and similarity or difference of outcomes for minority versus non-minority students were also found to have a significant effect on participant fairness ratings.
Significant differences were also found among the three groups with regard to non-context-specific attitudes and beliefs about standardized testing. Measurement professionals tend to differ significantly from both legal professionals and members of the general public; however, few significant differences in attitudes were observed between legal professionals and members of the general public. Overall, measurement professionals tended to more strongly endorse items indicating a positive view of testing, while legal professionals and members of the public provided stronger endorsements of items suggesting a more negative attitude toward standardized testing.