Disturbing (But, Alas, Predictable) Policy Outcomes for Teachers and Doctors’ Pay-4-Performance

Counting up the many differences between doctors and teachers make comparisons about pay-for-performance policies unlikely. Here’s a few:

*Doctors go to school four years to secure an M.D. and  then another two to six years (of which most are clinical  practice) to garner a specialty; most teachers spend two years, maybe three (of which six months to a year are in supervised classroom practice) to earn a bachelors and then a master’s degree in education; those who receive alternative certification have little to no time in supervised teaching prior to becoming the teacher of record.

*Doctors see patients one-on-one; teachers teach groups of 20 to 35 students all day.

*Most U.S. doctors get paid on a fee-for-service basis, nearly all public school teachers are salaried.

*Evidenced-based practice of medicine in diagnosing and caring for patients is more fully developed and used by doctors than the science of teaching and learning available to and accessed by teachers.

*Medicine is largely male-dominated while K-12 schooling is female-dominated.

*In terms of both social status and respect, polls rank doctors higher than teachers.

Enough?

With so many differences between doctors and teachers in training,  work, social, political, and institutional behaviors, why even compare the two?

The answer is that business-inspired medical and educational policymakers eager to reduce costs while improving health care and public schools have seized on pay-for-performance as a high-profile tool to accomplish both increased efficiencies and improved performance. Because of this fundamental similarity, untoward outcomes not only can be predicted from past incarnations of such policies but also have already emerged across both institutions. Here are just two.

Incentives corrupt measures.

Since the mid-1970s, social scientists have criticized the use of specific quantitative measures to monitor or steer policies because those implementing such policies alter their practices to insure better numbers. In the previous post, I gave instances of how Medicare and private insurers use of “quality measures” have distorted doctors’ practices. The work of social scientist Donald T. Campbell and economists in the mid-1970s about the perverse outcomes of incentives was available but went ignored. Campbell wrote in 1976.

“The more any quantitative social indicator is used for social decision-making, the more subject it will be to corruption pressures and the more apt it will be to distort and corrupt the social processes it is intended to monitor” (Campbell 1976, p.54)

Campbell used examples drawn from statistics on police solving crimes (p. 55), the Soviets setting numerical goals in industry (p. 57), and the U.S.’s use of “body counts” in Vietnam as evidence of winning (p.58). For public schools, Campbell said that “achievement tests are … highly corruptible indicators (p.57).”

That was nearly forty years ago. In the past decade, researchers have documented  (also see here) the link between standardized test scores and narrowed instruction to prepare students for test items, instances of  state policymakers fiddling with cut-off scores on tests, increased dropouts, and straight out cheating. Although how the distortions occur are unclear, the evidence confirms Campbell’s insight.

Easy To Measure Indicators Trump Hard To Measure Ones

Few in medicine or education question that some indicators are easier to quantify than others. In medicine, for example, hospital mortality and surgical procedures are fairly easy to measure but the results even when compared to other hospitals and surgeons hide as much as they reveal about effective health care. So it is with standardized tests.

Easily accessible test scores have led to evaluating teachers on the basis of student performance. In Los Angeles, Washington, D.C., and  states–responding to federal incentives in Race to the Top–value-added measures have become a staple for determining effective teaching. Moreover, because test scores are easy (and inexpensive) to gather, more tests have been added across the nation giving a misleading, even inaccurate, gloss to the numbers. (For an extreme example, see here; a critique of piece is here and response here).

Because test scores are inexpensive and efficient to collect, they draw attention away from important but hard-to-measure aspects of teaching and learning such as student engagement, rapport between teachers and students, academic climate in classrooms, and principal leadership. Cumulative practitioner experience and stories about teaching over centuries have established these as crucial factors in working with gifted and vulnerable students. For physicians, quantifying the relationship with patients is just as hard to grasp leaving insurers with the leeway to evaluate physician quality by using readily available statistics on, for example, whether or not doctors give antibiotics within six hours to patients who might have pneumonia (see previous post).

Thus, over time, teachers and doctors come to see the “quality” measures used to evaluate and pay them as perverse eroding loyalty they have to their students, the institution, and the very process itself. Perhaps, the oath that medical school graduates take upon receiving the M.D. degree should also be taken by policymakers eager for pay-for-performance: Do no harm.