Whose Standards for Judging Reform Success Are in Play? Part 3

Image via Wikipedia

Although policymakers, researchers, practitioners have vied for attention in judging the success  of school reforms, policy elites, including civic  and business leaders and their accompanying foundation- and corporate-supported analysts and evaluators, dominate the game of judging reform success.

Sometimes  called a “growth coalition,” they see districts and schools as goal-driven organizations with  leaders exerting top-down authority through structures. They juggle highly prized values of equity, efficiency, excellence, and getting reelected. They are also especially sensitive to public expectations for school accountability and test scores. Hence, these policy making elites favor standards of effectiveness, fidelity, and popularity—even when they conflict with one another. Because the world they inhabit is one of running organizations, their authority and access to the media give them the leverage to spread their views about what constitutes success.

Since the SAT test score decline and Effective Schools research in the late 1970s relied on standardized achievement test scores to identify largely low-income schools that exceeded academic expectations, using test results seemed appropriate then and especially now after passage of the No Child Left Behind law. Moreover, for policy elites, these numbers display to an increasingly skeptical public that schools can succeed and be held accountable for their performance. With newspapers publishing school-by-school standardized test scores since the late 1960s, the popular notion that schools were engaged in a contest with clear winners—those at the top of the list—and losers—those at the bottom—became an easy segue for a public that eagerly followed the World Series and Bowl games.

Yet the widespread notion of improved test scores as a valid measure of school improvement remained slippery. Researchers and test experts have often told policymakers about technical flaws in using standardized achievement test scores to judge a school’s performance. They have pointed out how even the lowest-performing schools can improve simply by shifting the metric used, changing definitions of what constitutes improvement, and preparing children for tests. They have pointed out how reliance on test scores can steer a district’s programs in undesired directions.

The world that policy elites inhabit, however, is one driven by values and incentives that differ from the worlds that researchers and practitioners inhabit. Policymakers respond to signals and events that anticipate reelection and media coverage. They consider the standards of effectiveness, fidelity, and popularity rock-hard fixtures of their policy world.

Most practitioners look to different standards. Although many teachers and principals have expressed initial support for Effective Schools (or now KIPP, Green Dot, or similar models) because of the attention, extra resources, and commitment to social justice, most teachers and principals have expressed strong skepticism about test scores as a measure of either their effects on children or the importance of their work.

Such practitioners are just as interested in student outcomes as are policymakers, but the outcomes differ. They ask: What skills, content, and attitudes have students learned beyond what is tested? To what extent is the life lived in our classrooms and schools healthy, democratic, and caring? Can reform-driven programs, curricula, technologies be bent to our purposes? Such questions are seldom heard.

A third set of standards comes from researchers. Researchers judge success by the quality of the theory, research design, methodologies, and usefulness of their findings. In most cases, by the mid-1980s, researchers had pronounced the Effective Schools research seriously flawed in theory, design, and methodology, although a few did point out redeeming qualities in that body of literature. These researchers’ standards have been selectively used by both policy elites and practitioners in making judgments about Effective Schools programs  or any of the subsequent incarnations since the late 1980s such as the Coalition of Essential Schools, KIPP,  charter schools, etc.

So at least three sets of standards for judging success are available. Practitioner-and researcher- derived standards have occasionally surfaced and received erratic attention from policy elites. But it is this strong alliance of policymakers, civic and business elites, and associates in the corporate, foundation, and media worlds with their standards of effectiveness, fidelity, and popularity that continues to dominate both public debate and school reform agendas.

Now, here is the spot where I should say which of these three sets of standards for judging success of reforms is best and should be used. Won’t do that. Why? Because while I prefer standards driven by practitioner values, the current dominance of policymakers’ test-driven criteria in judging reform reflects the abiding fact that public schools have been shaped by political decisions since their founding nearly two centuries ago. In a democracy, whose standards dominate decisions on reform success are politically determined. Without a collective political effort to change policy elites’ standards of effectiveness, fidelity, and popularity, they will remain and alternatives to those standards will remain marginal.

Related Articles

• What Works in the Classroom? Ask the Students (nytimes.com)