Picking and Choosing among “Best Practices”: The Case of Pay-for-Performance and Student Test Scores

When I was a high school teacher, I had always thought that “best practices” referred to the shared experiences of seasoned practitioners who believed that what they do in their classrooms works. I borrowed many “best practices” from veterans whose work I respected when I was teaching in different high schools. Less obvious in the research literature and circulating by word-of-mouth then as well as now, this informal information network (sometimes called “folk wisdom” by academics) is evident on the Internet with websites displaying available lessons across subject areas that “work.”

After I moved into administration I came to see that what experts said was a “best practice” in, say, teaching reading or integrating a new technology into math lessons was often transmitted to teachers and principals in practitioner publications. These experts—often denying or pooh-poohing the wisdom of practitioners–knew the research on teaching and learning that was effective across classrooms.

When I became acquainted with evidence-based medicine I realized that “best practices” among researchers and physicians referred to particular therapies that emerged from clinical trials of experimental treatments—the gold standard in scientific research—as reducing mortality or otherwise benefitting patients. In education, however, such experimental trials occur infrequently although quasi-experimental studies or naturally occurring experiments do occur such as studies of twins’ performance in early reading.

So “best practices” in teaching and learning, leaning on studies that seldom show strong connections between causes and effects, fall well behind the “evidence-based practice” that medicine has accumulated over time through clinical trials of experimental treatments. A simple peek at the U.S. Department of Education’s “What Works Clearinghouse” website confirms the few studies that rise to the level of evidence-based practice

Finally, there are leap-frog “best practices,” that is, where experts in other institutions decide that certain practices can leap over institutional boundaries to be applied in a totally different setting. If you flashed forward and said, Larry is going to point out that some “best practices” in business are transferred to schools—you guessed correctly.

Which brings me to pay-for-performance and the “metric mania” that surrounds evaluating and paying teachers on the basis of student test scores. Where are the “best practices” to inform such policies?

There aren’t any. Experts in testing have said again and again, based on the best historical and statistical evidence available that student scores on group norm-referenced and criterion-referenced tests should not be used to make high-stake judgments on individual students (e.g., be retained in a grade, withholding  diplomas) because of measurement error, instability in scores over time, and other reasons (see American Educational Research Association Position Statement on High-Stakes Testing adopted in 2000)

Federal, state, and local policymakers have not listened to the experts. Neither have foundation executives (the “Billionaires Boys Club” in Diane Ravitch’s withering phrase) determined to steer school reform in the U.S.

So who have policymakers, foundation executives, and others listened to? The answer is self-evident: corporate leaders  who believe in their heart-of-hearts that the “best practice” in the business world is to pay employees who produce results. Corporate profits and shareholder satisfaction have proved the efficacy of such a “best practice” except for occasional “oops” when CEOs make terrible decisions and still take away huge bonuses or the Great Meltdown of 2008.

So “best practices” in teaching reading, math, cooperative learning, or using technology in classrooms may be strenuously pushed by policymakers upon teachers (while largely ignoring the experienced-based wisdom of practitioners) but when it comes to pay-for-performance schemes, it still is pick-and-choose what best fits the situation, not any practitioner-derived expertise or current research studies—especially if it contradicts the desired policy. Thus, business-driven experience with pay-for-performance, corporate animus toward teacher unions, and MBA-produced entrepreneurs carrying eureka-like stories from the private sector–not teachers’ experienced-produced “best practices” account for the fierce battles over pay-for-performance.

Seldom, however, do these business-driven cheerleaders for pay-for-performance ask straightforward and simple questions in applying business “best practice” to teaching and learning:
* How much (and exactly what) do high-stakes test scores capture of what students learn in school?
*How reliable and how accurate are those test scores in measuring elementary school teacher performance in teaching reading, math, science, social studies, and the humanities? Or secondary school teachers who see five classes of 150 students for an hour a day?
*Does pay-for-performance harm or help teacher collaboration to improve student learning?

The answers to these questions do not advance pay-for-performance. That is why business leaders and policymakers continue  to pick-and-choose among “best practices” and borrow heavily from business instead of pausing to reconsider pay-for-performance. The fact is that schools are complex, ever-changing systems where blueprints, organizational procedures, and engineer designs may work in the private sector but  not in  schools.  More on complex  adaptive systems in the next post.