ETS on the Issues

The world of politics is often one of sound bites and snapshots, which may at times produce sweeping summations and snap judgments. When that occurs, the result is often a series of images bereft of context that linger like holograms. Unfortunately, that’s what too often happens in the course of political campaigns and debates when student test scores are used to generalize about the relative health and well being of education in the United States.

Not that candidates for office are alone in proffering isolated images of education as an assessment of the entire landscape. All too often everyone from journalists to parents and from principals to business leaders will focus on single, individualized test results to judge which schools are succeeding or failing; which children are learning or struggling; which teachers are effective or inadequate; which programs are cost-effective or fiscally draining. The result is truly unfortunate, because doing so creates a false sense of how to evaluate educational programs and obscures the power that testing can have in driving meaningful education reform.

Psychometrically sound tests have analytical value. They provide educators with insights into learning and instruction. They measure knowledge and performance. They provide gauges that can lead to the identification of significant strengths to build on, immediate needs to address, and eventual changes to implement.

But a test score, like a snapshot, is static, capturing a moment in time within a fixed context. Neither a test result nor a snapshot becomes dynamic on its own, though dramatic differences result by changing the focus of the subject, the nature of the content, and the quality of the scores form valid tests provide distinct looks at isolated images. Analyzing fine points can provoke thinking about how the picture might have turned out differently with, say, a slight change in lighting. And analyzing the results of tests in the context of other factors can lead educators to important conclusions about new directions to take to improve student learning, classroom instruction, and school environment.

Too often, however, hasty judgments about test scores result in superficial responses to real educational issues, which, in turn, too often leave little opportunity to use the scores in meaningful ways. It is difficult to respond thoughtfully to test data when they consistently evoke superficial reactions. Ideally, test scores would not lead to extensive praise when they appear high, nor unreasonable criticism when they seem low. That ideal, however, is rarely achieved.

Consequently, a major challenge facing educational leaders in school systems around the country is how to keep themselves, the school community and public officials on an even keel when test scores are announced. Educators need time to interpret scores, determine how to use them to stimulate analytical thinking, and consider them in assessing changes that are needed. As one superintendent of a large, diverse, school system said recently when college admission test scores at every high school in his school system rose between 15 and 40 points. "Everyone says this is great parents love it; politicians are praising us; real estate agents are racing to get the word out. But I don't know what the rise in scores means. I don't know what we did differently, if anything, to help bring it about and I don't know what it says about instruction. I figure I've got a year to figure it out, because if the scores go down…."

This particular superintendent has the right idea - use the scores to analyze a whole range of factors that contribute to quality education. He knows he needs time to study what factors might have contributed to the rising test scores, but when his voice trails off, he is poignantly expressing the reality he faces - time is short, because if the scores go down, another, less-favorable judgment is but a heartbeat away.

Inherently, candidates and elected officials must know that presenting test scores as isolated measures of success or failure is a far too narrow and easy way of assessing education today.

Anyone interested in going beyond making snap judgments about the academic health of students and service by focusing instead on using scores to analyze a range of factors that contribute to student performance in school. Quality tests provide thoughtful and creative educators with important insights into prior learning contexts; the appropriate depth and breadth of curriculum; the effectiveness of professional development; the impact of parental influence; and the academic and social fitness of the school environment.

It is correct and necessary for educators to champion the usefulness of valid, reliable, high quality testing. But all of us should help extend the conversation beyond test scores by discouraging the snap judgments and easy answers that inevitably follow the snapshots those scores present. Instead, they should encourage a deeper conversation, one that uses testing and test scores to drive a meaningfully analytical and constantly evolving education system.

         We invite you to continue this dialog by contacting us via e-mail at: issues@ets.org