Cheating Scandals Reaffirm, Not Diminish, Testing

Not until the trials (or plea bargains) are over, will a verdict be rendered on former Superintendent Beverly Hall’s guilt or innocence in what is called the Atlanta cheating scandal. Hall’s indictment follows on the heels of finding El Paso Superintendent Lorenzo Garcia guilty last Fall. He is now serving three and a half years in jail (see here and here).

Even before a judge or jury decides on her guilt or innocence, anti-testing groups, feeding on Atlanta, El Paso, and the investigation of tampering with test scores under Washington, D.C. school chief, Michelle Rhee, have grabbed the case to further their cause. Moreover, over the years, journalists have uncovered oddities in test scores jumping sky-high in one year in other districts across the nation.

Foes of standardized tests feel the rush of adrenalin in saying that these examples of dishonest adults raising student test scores to receive applause and cash awards are pervasive. Defenders of standardized testing and accountability, however, see the  cheating as exceptions, as a few rotten apples in a barrel full of worm-free ones. Most educators, advocates of test-driven accountability say, are decent, hard working professionals who play by the rules and can be trusted to do the right thing.

In this volleying back-and-forth between advocates and foes of standardized testing,  school scandals have been compared to cheating in baseball, bicycle racing, and other sports.

From Mark McGuire‘s stained home run record to Tour de France winner Lance Armstrong‘s admission that he doped while racing, these and other sports have come under a dark cloud of suspicion–an outcome damaging to top athletes, companies dependent upon income derived from professional sports, fans turning into cynics, and disappointed youth who only want to play the game by the rules.

Cheating in both sports and schools can be traced to the unleashed and fierce competition in performing better and better to gain ever-larger rewards. Professional sports are money machines and being a top performer is rewarded handsomely; scores on international tests, ranking schools within a state and district based on performance, a broader array of school choices, and federal regulations in No Child Left Behind and Race to the Top  have ratcheted upward intense pressure to beat  state tests.

Also common to school cheating and drug-drenched sports is betraying the public trust to gain personal advantage.  When adults erase student answers and professional athletes take illegal drugs to enhance performance, such acts erode the faith that adults and youth have in social institutions being fair.

Another common feature is the unshaken confidence that current authorities have in written and computerized tests assessing student learning and drug tests determining whether athletes are cheating. When cheating is uncovered, few decision-makers question the tests. Tighter security and better tests are the solutions.

*Few decision-makers question whether there might be something wrong in professional athletics (i.e., expansion of baseball, football, hockey, and basketball leagues and over-the-top competition for more money).

*Few decision-makers question whether most toddlers and young children from low-income families should be tested especially since they bring to school very different strengths and weaknesses than children from middle and upper-income homes. Or that such early testing of young children squeezes inequities into judgments of what they can and cannot do in preschool and elementary school classrooms.

*Few decision-makers question the national obsession with student test scores as the correct metric to judge schools, teachers, and students.

This deep reluctance to question powerful interests invested in socioeconomic structures and cultures in which cheating occurs is why I believe that standardized tests in schools, like drug testing in sports, will be reaffirmed rather than overturned. There will be continuing challenges–as there should be–but standardized testing will remain rock-solid. Why?

First, note that most of the cheating incidents have been largely in districts where high percentages of poor and minority students attend school. Sure, there are exceptions but when you look closely at where dishonesty is found, those charters and regular public schools enroll large numbers of children from low-income families. I have yet to find any district school boards, investigators, charter school leaders or policymakers recommend examining the tests to see if they do what they are supposed to do or, after conducting such an examination, finding unworthy tests and getting rid of them. Yes, there have been protests by educators, students, and middle- and upper-middle class families against too much standardized testing (see here and here). These protests have led to occasional boycotts but none have occurred, to my knowledge, in poor neighborhoods. If anything, there is a reaffirmation of tests, calls for greater security, and plaudits for any whistle-blowers.

The point is that these tests sort students and schools by scores that  reinforce rather than erase existing gaps in achievement. And sorting is necessary to determine who, beginning at the age of four, shall climb each rung of that ladder reaching college. The system of private and public schooling requires such tests to distinguish high achievers from others. If the tests were really that accurate in making such distinctions across children and youth of being smart on paper, with people, and in life now and later, then, perhaps we need such tests . But that is not the case now… by a long shot.

Second, to underscore the above point, consider the experience of cheating on the SAT. After a scandal revealed that high-scoring individuals with fake IDs were paid to take the SAT test, Educational Testing Service tightened security at test sites. No challenges of the test itself occurred. SAT scores remain crucial for college admission and no school boards, teachers, or parent groups called for the end of the test.

Count on cheaters getting more clever and investigators still hunting them down. Amid increasing numbers of cheating incidents, standardized tests will be challenged, maybe the numbers even reduced, but nonetheless, they will reign for the immediate future.

Districts as the Engine of School Reform: Past and Present (Part 2)

Districts have again become the darlings of school reformers. Where once reformers, past and present, skipped back and forth lining up their cross-hairs on the best targets  for improving schooling such as individual teachers and principals, whole schools, and districts, today’s school reformers generally target districts. Many reasons explain the shift to districts but one, in my opinion, that accounts for the current passion among self-proclaimed reformers to turnaround failing schools and a mediocre national system of K-12 education is the increased authority that state and federal officials have accumulated over time to make local decisions.

Historically, states have the constitutional duty to provide education. States created districts and delegated authority to run schools. U.S. education, then, has been a decentralized operation for two centuries. In the early 1930s, there were nearly 130,000 districts in 50 states. Since then, the trend has been to merge districts into larger ones (there are now 14,000 districts). Mergers continued and since the 1960s with the federal Elementary and Secondary Education Act, federal and state authority over district schools have become more and more centralized.  State and the federal authorities now mandate what curriculum standards have to be taught in districts, what texts have to be used in classrooms, which tests must be given, what happens when students fail to perform satisfactorily on tests, and–increasingly–how content and skills should be taught. Oops! Did I forget that states (44 percent) and feds (nearly 10 percent) supply most funding for districts?

To state and federal officials, mandates, money, penalties for non-performance, and the stigma of shame are the primary levers to institute desired changes in districts from offering parents choices in charter schools to adopting Common Core standards to evaluating teachers on the basis of student test scores. Yet to these reform-driven officials, too many districts lack the political will and resolve to turn the corner on poor performance. Mandates, money, penalties, and shame seem to have little effect on persistently low-performing schools and districts.

What’s an eager state and federal official, armed with the authority to make rules and dispense funds to do when district inaction or minimal compliance occurs? One answer may be to look at districts, past and present, that have succeeded in turning themselves around, in adopting reforms that they worked at for years, and ask: how did they do it? What factors were common to them?

A recent article on Union City (NJ) does exactly that. David Kirp details what district officials in this largely immigrant and poor school system (10,300 students in 2013) did over a quarter-century–yes, 25 years–to make incremental changes from adding preschools to curriculum overhaul to a culture of learning and respect for community to, even new technologies. All of these changes were coordinated and eventually funded under the state Supreme Court’s Abbott decision. Stable leadership from school boards and superintendents  over decades converted these changes into standard operating procedures. Current school chief is Stanley Sanger who has spent a decade as ssuperintendent after a career as social studies teacher, principal, and assistant superintendent–all in Union City. These incremental and steady changes accumulated into a success story, including the district’s one high school.

Push the rewind button  and go back in time to 1907 in Gary (IN). A company town literally owned by U.S. Steel, the Gary school board appointed William Wirt  superintendent; he served over 30 years. Influenced by the ideas of John Dewey and the emerging efficiency movement, Wirt introduced an innovative way of organizing schools, teaching, and learning for mostly immigrant students to work-study-and play called the Gary Plan or Platoon school. At a time when urban schools across the nation were looking  for ways to solve the problems of slums, overcrowded schools, and how to teach immigrant children the Gary Plan offered solutions.

The innovation was introduced into reorganized schools holding children from kindergarten through the twelfth grade. Administrators divided each school’s students  into two groups or “platoons.” One platoon would be in the classrooms or auditorium while the other would be in the basement where there were woodworking, printing, and other shops; upstairs in music, art, and play rooms; or outside on the playground. During the day, each platoon would change places, giving each child academic, practical, recreational, and aesthetic experiences while using the entire facility. While most urban elementary school children before World War I stayed the entire 6-8 hour school day in a self-contained classroom with one teacher, Gary pupils worked-studied-and played during an eight-hour day, even receiving released time for religious instruction. Adults used the school at night to take English courses and pick up other job skills.

Such a work-study-play-community school arrangement—a revolutionary shift in school organization and curriculum—made it possible to have many more students attend school–over 20,000 in the 1920s–since the schedule permitted all available space to be used by students during the day with adults taking courses at night. The Gary innovation spread swiftly across the nation but by the 1930s and the Great Depression had largely disappeared from the agendas of reform-minded policymakers.

In Part 3, I offer one more example of a district reform and then offer answers to the questions asked above: how did districts do it? What factors were common to them?

Legacies of the Civil Rights Era: Accountability and Attention to Poverty (John Spencer)

John P. Spencer is a former high school social studies teacher and an associate professor in the Education department at Ursinus College in Pennsylvania. He is the author of In the Crossfire: Marcus Foster and the Troubled History of American School Reform.

 The past decade has brought a steady stream of commentary on how education is the “civil rights issue of our time,” most recently from Republican presidential candidate Mitt Romney. But education has been a civil rights issue for decades—and not just in Brown v. Board of Education or Little Rock, but in urban communities with low-performing schools.

Revisiting the 1960s shows us that the civil rightsera left a dual legacy in school reform, half of which echoes loudly today and half of which is too often ignored. The part that still echoes is an ethos of accountability: sixties-era activists and educators helped to pioneer the idea that urban schools should be held accountable for student achievement. The part that is being ignored is a recognition that achievement is also powerfully shaped by what goes on outside of schools—especially the effects of poverty. Unfortunately, neglect of the latter lesson is seriously undermining the potentially useful impact of the former one.

The movement for “community control” of urban schools in the late 1960s is a striking example of how the activism of the civil rights era prefigured the current accountability agenda, in spirit if not in terms of specific policies and approaches. The battle lines of community control will sound familiar to anyone following recent controversies over charter schools, “parent trigger” laws, and the like: on one side, black parents and community activists fed up with low-performing schools and eager to take charge of them; on the other, teachers unions and school bureaucracies. In the most famous case, in Ocean Hill-Brownsville, Brooklyn, the activists managed to fire unionized teachers, sparking a bitter and prolonged strike. (The teachers were reinstated.)

The community control movement shifted the spotlight from problems in communities to problems with schools. Since the early 1960s, explanations for low achievement in urban schools had focused on the idea that black students, many of whom had migrated from the rural South, were “culturally deprived” and caught in a self-defeating “culture of poverty.” It was a liberal idea at the time—a way of saying urban students were struggling not because they were black (as racists had insisted) but because they were poor. In the late 1960s, though, civil rights activists vehemently rejected the cultural deprivation argument as a form of racism. They believed the problem was low expectations in schools. Dwelling on the impoverished background of the students was, as one critic said in a newly coined phrase, “blaming the victim.”

We hear a similar argument today: to emphasize the effects of poverty is to make excuses. Reformers may make this argument in various ways and for various reasons (with some standing to benefit from emphasizing the deficiencies of schools that in turn become candidates for privatization); but with the language of “no excuses,” they all tap into the unrealized expectations of the civil rights era. Unfortunately, the advocates of holding schools accountable tend to neglect or dismiss an equally important legacy of the 1960s, to the detriment of their professed goal of eliminating achievement gaps: the Coleman Report (1966) and nearly five decades of subsequent research showing that socioeconomic status, cultural capital and other non-school factors have even more impact on academic achievement than do teachers and schools.

How to focus on those external factors while maintaining high expectations of schools? One example from the 1960s was the leadership of African American educator Marcus Foster. Foster earned acclaim as a principal in Philadelphia and superintendent in Oakland, before being assassinated by the Symbionese Liberation Army in 1973 in a bizarre protest against an allegedly racist school system.

Foster exemplified the accountability ethos of the civil rights era: “Inner city folks . . . want people in there who get the job done, who get youngsters learning no matter what it takes,” he once wrote. “They won’t be interested in beautiful theories that ex­plain why the task is impossible.” But Foster did not win awards for improving achievement in struggling schools by pitting communities against educators; he got communities and schools to work together—and to insist upon accountability from taxpayers and political and economic institutions, too. On one occasion, for example, he closed the Oakland schools and transported thirty busloads of Oaklanders to the state capitol to seek more support for needy urban students—resulting not only in more money but in “three-thousand folks of all persuasions saying, ‘We stand together for schools.’”

Foster’s accomplishments in the 1960s, including his call for broader societal support, are echoed today—though not often enough—in such efforts as the Broader, Bolder Approach to Education, which calls for school reform to be combined with policies aimed at improving the health, the early childhood learning, and the out-of-school experiences of underachieving children. Not using poverty as an excuse to blame the victim is an important lesson from the 1960s. But it’s only half the story for those who truly hope to make equal education a civil right.

Reframing Shame: How and When Blame for Student Low Achievement Shifted

The shame that many teachers and principals feel at being made responsible for a school’s low academic performance is a recent phenomenon. Historically, policy elites and educators explained poor academic performance of groups and individual students by pointing to ethnic and racial discrimination, poverty, immigrants’ cultures, family deficits, and students’ lack of effort. School leaders would say that they could hardly be blamed for reversing conditions over which they had little control. Until the past quarter-century, demography as destiny was the dominant explanation for unequal school outcomes.

Things began to change by the mid-1970s. Other explanations for low academic performance among different groups of students gained traction: The school—not racism, poverty, family, culture, or even language differences–caused disadvantages in students. This explanation grew from research studies of urban elementary schools with high percentages of poor and minority students that did far better on national tests than researchers would have been predicted from their racial and socioeconomic status.

These high-flying ghetto and barrio schools had common features: staff’s belief that all urban children could learn; the principal of the school was an instructional leader; staff established high academic standards with demanding classroom lessons, frequent testing, and an orderly school (PDF el_197910_edmonds-2).  These “effective schools” proved to many skeptics that high poverty urban schools could be successful, as measured by tests. Students’ race, ethnicity, and social class did not doom a school to failure. And most important, that committed and experienced staff working closely together could make a decided academic difference in the lives of impoverished children of color. No longer could teachers and administrators blame students and their families for failing. Now, it was the responsibility of school staff to insure student success.

This fundamental swing in blame for the causes of inequities in outcomes are captured in the words of national leaders who often  admonish teachers and administrators to avoid the “soft bigotry of low expectations.” This reversal of responsibility for inequitable outcomes has shifted the burden for academic success completely from students’ shoulders to those of their teachers, principals, and superintendents.

While most of us cherish the egalitarian thought—enshrined in NCLB that all students will test proficient by 2014–research studies and the facts of daily experience should give us pause before nodding in agreement. Perhaps this total equality in results may occur in heaven but not on earth where variability in families’ behaviors and students’ talents, motivation, interests, and skills remain stubborn facts.

Thus, within a few decades, a 180-degree shift in responsibility for chronic academic failure has occurred. Neither extreme, however, squares with the facts. Responsibility rests with both community and district, both school and family, both teachers and students.

Blaming others may be momentarily satisfying but unhelpful in either improving schools or motivating students to do their best. On the one hand, expecting a school staff to have the full responsibility for students’ academic results neglects the long history of research and daily experience of students who come to school unready to learn. Family income, parental education and interest, health, neighborhood, and other factors influence what happens to growing children even before they enter kindergarten. If there is one fact researchers have established over and over it is that family income and education play a large role in children’s behavioral and academic performance in schools.

Striking a balance between documented facts of inequities among students when they appear at the schoolhouse door and documented facts of some educators’ shabby inaction while other educators turn basket-case schools into high-fliers is essential. But it is hard to strike this balance in the current unforgiving climate of state and federal accountability rules that name, blame, and shame districts and schools for gaps in achievement, high drop out rates, and low graduation numbers (SAN11-01).

In the current frenzied climate of state and federal penalties for low performance, what students bring to school, both their strengths and weaknesses, are seldom mentioned publicly because of policymakers’ and educators’ fear of being called racist, making excuses, or having low expectations. The dominant one-liner repeated again and again is that efficient, well-managed schools and districts are accountable for students’ academic success.

This situation pains those federal, state, and local policymakers and reformers who want to address those socioeconomic structures in the larger society that contribute to economic inequalities and students’ disadvantages such as tax policies favoring the wealthy, residential segregation, lack of health insurance, immigration policies, and discriminatory employment practices but find it hard to do in a political climate where top-down, business-driven reforms that blame teachers and their unions and use test scores to determine futures of teachers and schools blow like gale-force storms.

In such a climate, entrepreneurial reformers, federal policymakers, and wealthy donors direct attention to only fixing schools, a strategy that is both politically attractive and economically inexpensive compared to the uproar that would occur from attacking those who enjoy privileges from leaving those policies and structures untouched.

---

Can Superintendents Raise Test Scores?

John Deasy, who was appointed superintendent of the 672,000-student Los Angeles district in January, will be evaluated … on improvement in the graduation rate, student proficiency and attendance. But he also has the opportunity to earn up to $30,000 in bonuses if the district sees an 8 or more percentage point increase in 3rd-grade scores on the state’s reading test, the percentage of students scoring proficient or advanced in 9th-grade algebra, and the four-year-graduation rate….

Jean-Claude Brizard, the chief executive officer of the 409,000-student Chicago schools, will be evaluated on such performance measures as: increasing the percentage of high schoolers who graduate within five years from 55.8 to 60 percent; improving from 27 to 35 percent the share of high schoolers who earn at least a 20 out of 36 on the ACT college entrance exam, and increasing the percentage of students passing the state standardized test for 3rd-grade reading from 57.8 to 70 percent….

Because school boards and mayors assume that measures of good schools can be found  in rising test scores, high school graduation rates, and college admissions, they hire superintendents to be instructional leaders, astute managers, and wily politicians to carry out board mandates and ensure that desired improvements occur. They also push out superintendents–just ask Chicago’s Jean-Claude Brizard who just left days ago after 17 months in office.

So superintendent contracts include clauses on raising test  scores. But can they do so? The literature on the superintendency, with few exceptions, answers  “yes” to the question.  When writers, policy makers, and administrators mention successful school chiefs they point to increasing scores on standardized achievement tests, high percentages of graduates entering college, and National Merit Scholarship finalists (SuperintendentLeadership)

Yet when superintendents are asked how they get scores or graduation rates to go up, the question is often answered with a wink or a shrug of the shoulders. Even among most researchers and administrators who write and grapple with this question of whether superintendents can improve test scores, there is no explicit model of effectiveness.

How exactly does a school chief who is completely dependent on an elected school board, district office staff that prior superintendents appointed, a cadre of principals in schools whom he or she may see monthly, and teachers who shut their doors once class begins–raise test scores, decrease dropouts, and increase college attendance? Without some model by which a superintendent can be shown to have causal effects, test scores going up or down remain a mystery, a matter of luck that the results occurred during that school chief’s tenure.

Many school chiefs, of course, believe they can improve student achievement. They have in their heads what I call the Rambo or Michelle Rhee model of superintending. Strong leader + clear reform plan + swift reorganization + urgent mandates + crisp incentives and penalties =  desired student outcomes. Think former New York City Chancellor Joel Klein, ex-Miami-Dade Superintendent Rudy Crew, and Alan Bersin in San Diego.

There are, of course, other models that are less heroic and mirror more accurately the complex, entangled world of moving policy to classroom practice through a school board, superintendent, principals, teachers, students, and parents. One model depicts indirect influence where superintendents shape a district culture of improvement, spend time on instructional issues, train principals to run schools, and work closely with teachers in supporting and prodding them to take on new challenges in their classrooms. Think Carl Cohn in Long Beach (CA), Tom Payzant in Boston (MA) and Laura Schwalm in Garden Grove (CA). Such an indirect approach is less mythical, takes a decade or more, and is less dependent upon the superintendent being Superman or Wonder Woman.

Whether school chiefs or their boards have a Rambo model, one of indirect influences, or other models in their minds, some theory exists to explain how they have an impact on student academic performance. Without some explanation for how they influence district office administrators, principals, teachers, and students to perform better than they have, most school chiefs have to figure out their own personal cause-effect model or rely upon chance.

Some superintendents, for example, figure that working 60-70-hour weeks insures that there will be payoff in student improvements. Other superintendents figure that showering the district with reforms will eventually produce some results that might improve student performance. And even other superintendents size up the situation as mysterious; they hope that they will get lucky and the students tested next year will make higher scores than this year’s group. The lack of attention to linkages between superintendent actions and student outcomes prompts those in office to keep their fingers crossed behind their backs.

What is needed are GPS navigation systems imprinted in school board members’ and superintendents’ heads that contains the following:

*A map of the political, managerial, and instructional roles superintendents perform, public schools’ competing purposes, and the constant political responsiveness of school boards to constituencies that inevitably create persistent conflicts.

*a clear cause-effect model of how superintendents influence others to do what has to be done,

*a practical and public definition of what will constitute success for school boards and superintendents.

Such a navigation system and map are steps in the direction of accurately answering the question of whether superintendents can raise test scores.

School Reformers Who Disagree Find Common Ground (John Thompson and Neerav Kingsland)

The following post was written by teacher John Thompson of Oklahoma City  and Neerav Kingsland, CEO of New Schools for New Orleans. They introduce themselves below. The post was published by Rick Hess, a blogger for Education Week on September 21, 2012.

 There is much we disagree on – don’t worry we’ll get to that. But in writing this joint post we hope to flesh out some common beliefs that unite two very different people – and, perhaps, two different wings of current education debates. The recent events in Chicago make very clear that there is a great divide between different factions of reform and that this divide continues to greatly impact children. We hope that this divide need not be permanent – and that a common agenda may be found between different reform camps. At the very least, we have found common ground where few would have expected.

 

Some Background on Us

I (John) taught and participated in whole school and district-wide reforms in the Oklahoma City Public School System (which is 90% low income). On the eve of No Child Left Behind (NCLB), I served on the steering committee of a bipartisan reform effort that was a down home version of the Broader, Bolder Approach – and was a team member of a school that was improving faster than any other high school in the district. I blame NCLB for wrecking our promising community-wide school improvement effort, driving hundreds of students out of my school, and turning it into the lowest performing school in the state.

I (Neerav) am the CEO of New Schools for New Orleans (NSNO), which launches and scales charter operators and human capital organizations. NSNO has been an accelerator of the New Orleans reform efforts – which has led to over 80% of New Orleans students attending charter schools. I (Neerav) blame local government monopolies for taking power away from educators and parents – as well as for operating stagnant school systems that fail to harness entrepreneurship, innovation, and competition.

Where We Agree

We agree that top-down, command and control governance, has failed. And that it will continue to fail regardless of the talents of the elites who run these systems. Technocratic reformers will never be able to design enough “transformational policies” to solve the complex problems facing families, educators, and communities. We are against: district-wide curriculum mandates, legislatively enforced teacher evaluation systems, and personnel decisions driven by central office bureaucrats. We’ve each seen some of our generation’s greatest minds seduced by the idea that “if I’m in power, I’ll be able to fix this.”

It’s hard to overstate the importance of this agreement: a “labor activist progressive” and a “Relinquisher” both feel that the much of the current reform movement is extremely misguided. And we both point to overreaching bureaucratic elites as the source of the problem – even if we sympathize with their intentions and are hopeful that they succeed in increasing student learning.

Where We Somewhat Disagree: How to Empower Educators

We agree on the problem, but our solutions take somewhat different paths. In short, where we agree and, yet, start to disagree is with school autonomy. Both of us are for it – but in different ways.

I (John) steadfastly oppose vouchers and worry about charter chains (CMOs). I never criticize charters. I’ve always celebrated when my students get into charters, magnets, or (below the radar) get into suburban schools. I believe that the safest way to gain the benefits of autonomy can be through “enterprise schools,” or neighborhood schools that are granted autonomy. These schools should be governed by “thin contracts” that allow for collective bargaining agreements but do not restrict the operational autonomy of school site decision making for educators.

I (Neerav) believe that true autonomy can only be achieved by government relinquishing its power of school operation. I believe that well regulated charter and voucher markets – that provide educators with public funds to operate their own schools – will outperform all other vehicles of autonomy in the long-run. In short, autonomy must be real autonomy: government operated schools that allow “site level decision making” feels more Orwellian than empowering – if we believe educators should run schools, let’s let them run schools.

Where We Really Disagree: Standardized Testing

Our biggest disagreement is all about standardized testing. The gulf here is significant but not as wide as one might think…

I (John) have mixed feeling about graduation examinations, but they are state mandates ratified by the voters. Educators should not impose high stakes standardized tests without the consent of educators and students. Choice schools, whether they are charters or enterprise schools, should be free to use high stakes tests if they choose. Educators in those schools, however, should stand with their colleagues and oppose such testing in neighborhood schools. We should unite in condemning value-added evaluations that are likely to increase primitive test prep and drive teaching talent out of schools where it is harder to raise test scores.

I (Neerav) am very conflicted about standardized testing. The libertarian in me just wants to give parents choice, provide them with a lot of information, and let the market work itself out. The pragmatist in me is familiar with the research on parents being unaware of the poor performance of schools to which they claim deep allegiance. I also have mixed feelings about annual high stakes testing (compared to once every couple of years) – which I worry (a) forces schools to shallowly cover grade level material and (b) is at odds with personalized learning. That being said, I feel that the near term costs to student achievement would be high if we eliminated testing – but am very open to the idea that long-term testing mania may have deleterious educational effects. So, for now, I’m on board.

A Not Quite Manifesto

To sum it all up: we agree that the progressive labor movement and Relinquishers should unite in support of the areas where we find common ground. We enthusiastically welcome all allies in liberating educators and schools from top-down management. And we feel that ideological blinders continue to prevent educators from supporting all forms of autonomy. Yes, we disagree on the structure of autonomy and standardized testing – but both of us are aware of the risks of our preferred approaches – and neither of us vilifies the other for his beliefs.

The outcome of current educational debates will affect the happiness and prosperity of the future adults of our nation. The stakes are high.

So let us end with this:

We believe that educator empowerment – in some form or another – must be the North Star of reform efforts.

We believe that the coalition in support of educator empowerment can cross political, ideological, and geographical lines.

And we believe that the coalition around education empowerment should air its disagreements on crucial issues – but that these issues should (for now) take a back seat to educator empowerment.

And we encourage you both to visit Oklahoma City and New Orleans. They are truly wonderful places.

Chicago Teachers’ Strike, Performance Evaluation, and School Reform (Jack Schneider and Ethan Hutt)

For the fifth consecutive day in Chicago, nearly 30,000 teachers are out on strike. At issue are many of the contractual details that typically complicate collective bargaining—pay, benefits, and the length of the work day. But the heart of the dispute roiling the Windy City is something relatively new in contract talks: a complicated statistical algorithm.

District leaders in Chicago, following the lead of reformers in cities nationwide, are pushing for a “value-added” evaluation system. Unlike traditional forms of evaluation, which rely primarily on classroom observations, policymakers in Chicago propose to quantify teacher quality through the analysis of student achievement data. Using cutting-edge statistical methodologies to analyze standardized test scores, the district would determine the value “added” by each teacher and use that information as a basis for making personnel decisions.

Teachers are opposed to this approach for a number of reasons. But educational researchers are generally opposed to it, too, and their reasoning is far less varied: value-added evaluation is unreliable.

As researchers have shown, value-added methodologies are still very much works-in-progress. Scholars like Heather Hill have found that value-added scores correlate not only with quality of instruction, but also with the population of students they teach. Researchers examining schools in Palm Beach, Florida discovered that more than 40 percent of teachers scoring in the bottom decile one year, according to value-added measurements, somehow scored in the top two deciles the following year. And according to a recent Mathematica study, the error rate for comparing teacher performance was 35 percent. Such figures could only inspire confidence among those working to suspend disbelief.

And yet suspending disbelief is exactly what reformers are doing. Instead of slowing down the push for value-added, they’re plowing full steam ahead. Why?

The promise of a mechanized quality-control process, it turns out, has long captivated education reformers. And while the statistical algorithm in question right now in Chicago happens to be quite new, reformer obsession with ostensibly standardized, objective, and efficient means of gauging value is, in fact, quite old. Unfortunately, as the past reveals, plunging headlong into a cutting-edge measurement technology is also quite problematic.

Example 1:
Nearly a century ago, school leaders saw a breakthrough in measurement technology as a way of measuring teacher quality. By using newly-designed IQ tests to assess “native ability,” school administrators could translate student scores on standardized tests into measures of teacher effectiveness. Of course, not everyone was on board with this effort. As one school superintendent noted, some educators were concerned “that the means for making quantitative and qualitative measures of the school product” were “too limited to provide an adequate basis for judgment.” But the promise of the future was too tempting and, as he argued, though it was “impossible” to measure teacher quality rigorously, “a good beginning” had been made. Reformers plowed ahead.

The IQ movement was deeply flawed. The instruments were faulty and culturally-biased. The methodology was inconsistent and poorly applied. And the interpretations were horrifying. “If both parents are feeble-minded all the children will be feeble-minded,” wrote H.H. Goddard in 1914. “Such matings,” he reasoned, “should not be allowed.” Others drew equally shocking conclusions. E.G. Boring, a distinguished psychologist of the period, wrote in 1920 that “the average man of many nations is a moron.” The average Italian-American adult, he calculated, had a mental age of 11.01 years. African-Americans were at the bottom of his list, with an average mental age of 10.41.

Value-added proponents like to make the argument that “some data is better than no data.” Yet in the case of the mental testing movement, that was patently false. For the hundreds of thousands of students
tracked into dead-end curricula, to say nothing of the forced sterilization campaigns that took place outside of schools, reform was imprudent and irresponsible.

But one need not go back so far into the educational past for examples of half-baked quality-control reforms peddled by zealous policymakers.

Example 2:
In the 1970s, 37 states hurriedly adopted “minimum competency testing” legislation and implemented “exit examinations,” ignoring the concerns of experts in the field. As one panel of scholars observed, the plan was “basically unworkable” and exceeded “the present measurement arts of the teaching profession.” Reformers, however, were not easily dissuaded.

The result of the minimum competency movement was the development of a high stakes accountability regime and years of litigation. Reformers claimed that the information revealed by such tests would provide the sunlight and shame that schools needed to improve. Yet while they awaited that outcome, thousands of students suffered the indignity of being labeled “functionally illiterate,” were forced into remedial classes, and had their diplomas withheld despite having enough units to graduate—all on the basis of a test that leading scholars described as “an indefensible technology.”

Contrary to what reformers claimed, the information provided by such deeply flawed tests did little to improve students’ learning or the quality of schools.

Today’s policymakers, like those of the past, want to adopt new tools as swiftly as possible. Even flawed value-added measures, they argue, are better than nothing. Yet the risks of early adoption, as the past reveals, can far outweigh the rewards. Simply put, acting rashly on incomplete information makes mistakes more costly than necessary.

Today’s value-added boosters believe themselves to be somehow different—acting on better incomplete information. Yet the idea that incomplete information can be good strains credulity.

Good technologies do tend to improve over time. And if advocates of value-added models are confident that they can work out the kinks, they should continue to judiciously experiment with them. In the meantime, however, such models should be kept out of all high-stakes personnel decisions. Until we can make them sufficiently work, they shouldn’t count.

________________________________

Jack Schneider is an Assistant Professor of Education at the College of the Holy Cross and author of Excellence For All: How a New Breed of Reformers Is Transforming America’s Public Schools. Ethan Hutt is a doctoral candidate at the Stanford University School of Education and has been named “one of the world’s best emerging social entrepreneurs” by the Echoing Green Foundation.

The Paradoxical Logic of School Turnarounds: A Catch-22 (Tina Trujillo)

TINA TRUJILLO is an assistant professor in the Graduate School of Education at the University of California, Berkeley. She earned a Ph.D. in education from the University of California at Los Angeles and an M.A. in education from the University of Colorado Boulder. This commentary appeared in Teachers College Record on June 14, 2012. Sources cited here are listed in the online article.

In the 1955 classic novel Catch-22, Joseph Heller chronicles the absurdity of the bureaucratic rules and constraints to which a conflicted Air Force bombardier and countless others are subjected. Each character lives under the absolute, yet illogical, power of these policies. One by one, they come to see the paradoxical nature of the policies, and eventually the bombardier realizes that the main policy—Catch-22—is an empty one, based on illogical reasoning. Its real purpose is to justify the behavior of those in power and maintain the status quo.

The Obama administration’s 2009 school turnaround policy is in many ways a catch-22.  After announcing its intention to turn around 5,000 of the nation’s lowest performing schools over the next 5 years, the federal government poured an unprecedented $3.5 billion into the federal School Improvement Grant (SIG) program. Such a worthy goal, in the administration’s view, required quick, drastic action at the school level. Out of four policy options prescribed by the administration, districts that receive SIG monies choose the turnaround option the second most frequently for schools (Hurlburt, LeFloch, Therriault, & Cole, 2011). This policy mandates that schools fire the principals and teachers and change schools’ overall management. If the schools’ performance does not adequately improve (each state sets the definition of improvement), states can pull the funding.

Yet the assumption behind school turnarounds—that rapid, dramatic changes in staffing and management can fundamentally improve persistently low-performing schools—is inherently paradoxical because the turnaround option rests on faulty, unwarranted claims. In fact, the policy exists despite evidence to the contrary. Such reforms engender the exact conditions that long lines of research have linked with persistent low performance—high turnover, instability, poor climate, inexperienced teachers, and racial and socioeconomic segregation.

In this way, the policy presents schools targeted for turnaround with certain impossible dilemmas, or catch-22s, because implementing the policy is likely to lead schools back to the original problems that the turnaround was supposed to solve.

For example, architects of the turnaround concept ignored earlier research on analogous reforms that have been implemented to dramatically change school staffing, organization, and management in an effort to achieve similarly dramatic changes in student performance. This research documents a range of negative unintended consequences of these reforms, such as heightened racial or socioeconomic isolation and organizational instability (Mathis, 2009).

Consider reconstitution, the practice whereby employees are forcibly transferred out of chronically low-performing schools. This research shows that firing and replacing school staffs not only fails to yield the intended gains in test scores but also reproduces the same challenges that the reconstitution aims to ameliorate. In Chicago, reconstitution resulted in deteriorated teacher morale and staff replacements of no higher quality than their predecessors (Hess, 2003). Under these same reforms, low-income African American and Latino communities’ democratic participation in schools was restricted when the central office made reconstitution decisions in lieu of the discretion of local school councils—the district’s earlier school governance structures that were composed largely of parents and community residents (Lipman, 2003). In San Francisco, reconstituted schools repeatedly showed up on later lists of low-performing schools (Mintrop & Trujillo, 2005). In Maryland, in addition to not being associated with higher student performance, reconstitution inadvertently reduced schools’ social stability and climate (Malen, Croninger, Muncey, & Jones, 2002). Another study showed how hiring difficulties in five states forced many reconstituted schools to begin the school year with high numbers of substitutes (Center on Education Policy, 2008).

In the Philadelphia School District, where district and state leaders pinned their hopes on turnaround-style reforms and outsourcing of school management for over a decade, administrators recently announced the dissolution of the entire district (Mezzacappa, 2012). The turnarounds were too costly and just did not work (though former district administrators still plan to farm out the remaining schools to external management providers).

Recent examinations of the initial SIG turnaround efforts have documented other ways in which logistical challenges associated with the dramatic staffing overhauls may outweigh potential benefits. In New York, districts that selected the turnaround option struggled to find enough qualified personnel to fill vacant slots; they ended up swapping principals from one SIG school to another. In Louisville, over 40% of the teachers hired to work in turnaround schools were completely new to teaching (Klein, 2012).

From an empirical perspective, reconstituting school staff appears to precipitate the same conditions that led to the low performance in the first place. It destabilizes schools organizationally and socially. It undermines the climate for students and teachers. It increases racial and socioeconomic segregation. It does not improve the quality of new hires. And it actually breeds more problems with turnover and permanent staffing.

Like the characters in Heller’s novel, educators in potential turnaround schools can earnestly try to improve the situation quickly, yet when the conditions triggered by the policy mirror the conditions that led to their dismal situation to begin with, research shows that they find themselves, and their students, stuck. They’re stuck in a catch-22.

Despite this paradox, advocates of turnarounds maintain that education should take up such dramatic reforms because presumably they have produced the desired results in the corporate sector from which the interventions have been borrowed (e.g., Murphy & Meyers, 2007).¹  However, research also shows that corporate turnarounds rarely yield the positive results that reformers expect (Altman, 1968; Nystrom & Starbuck, 1984). Studies show how turnarounds in the corporate sector have fared little to no better than less dramatically reformed organizations (Hassel & Hassel, 2009). Some analysts have found that such popular management techniques are associated not with greater economic performance but with greater perceptions of innovation (Staw & Epstein, 2000). Others have found that the most popular turnaround approaches successfully improve organizational performance in only a quarter of the cases (Hess & Gift, 2008).

By ignoring this evidence, architects of the federal turnaround policy don’t just overestimate the promise of turnarounds to bring about radical school improvement; they misrepresent the record of corporate turnaround practices and their applicability to public schools.

If we know that turning around a chronically low-performing school is unlikely to happen under such a policy—in part because of the policy’s inherently illogical rules and the conditions it facilitates—why does the federal government prop up the policy with such conviction?

Heller offers us some ideas here, too. In the novel, a character decides to sell chocolate-covered cotton to the government. Why? Because he knows it looks good on the surface. People will want it, at least for a while, because the lack of substance on the inside is hidden from view. In his bureaucratic environment, the visual appeal of his product is enough to hide the lack of deep, authentic quality.

School turnarounds are a lot like chocolate-covered cotton. Turnaround policies look good on the surface. The government pours extraordinary funds into the school. A flurry of staffing changes occurs. It’s a spectacle. But deeper changes are not required. The policy does not address the racial and economic isolation of schools targeted for turnaround. It sidesteps the challenges of building a pipeline of qualified teachers and principals into the hardest to staff schools and districts—and retaining them. It says nothing about how to deepen educators’ professional learning opportunities.

The federal turnaround policy creates the outward appearance of well-intentioned policy makers without addressing broader systemic inequities. And in a political environment that puts a premium on bureaucratic accountability and managerial efficiency in schools, appearances go a long way (Trujillo, 2012; Trujillo, in press).

Unfortunately, when all is said and done, the current policy is grounded in the familiar arguments in support of corporate-style reforms—arguments contradicted by empirical evidence. The policy is paradoxical. It uses our nation’s neediest schools—those serving primarily poor children and children of color—as laboratories for educational experiments, notwithstanding existing proof that the experiments will not succeed because they recreate the conditions that produced the needs in the first place.

The upshot of this paradox is an illogical reliance on turnarounds as a tactic for lifting the performance of chronically low-scoring schools, and an exaggerated promise of a strategy for dramatically altering the staffing, organization, and management of our country’s neediest schools. In doing so, the current turnaround policy distracts the public from more fundamental questions about the types of policies that can genuinely challenge the status quo by securing the necessary conditions for all students to succeed. Because at the end of the day, the schools targeted for turnaround are still caught, inescapably, in a catch-22.

Evaluating Teachers Using Student Test Scores: IMPACT (Part 2)

Since 2009, a new system of teacher evaluation has been put into practice in Washington, D.C. called IMPACT. The “Teaching and Learning Framework” for IMPACT lays out a crisp definition of “good” teaching in what D.C. teachers call the “nine commandments”:

1. Lead well-organized, objective-driven lessons.

2. Explain content clearly.

3. Engage students at all learning levels in rigorous work.

4. Provide students with multiple ways to engage with content.

5. Check for student understanding.

6. Respond to student misunderstandings.

7. Develop higher-level understanding through effective questioning.

8. Maximize instructional time.

9. Build a supportive, learning-focused classroom community.

IMPACT uses multiple measures to judge the quality of teaching. Fifty percent of an annual evaluation is based upon student test scores; 35 percent based on judgments of instructional expertise drawn from five classroom observations by the principal and “master educators,” and 15 percent based on other measures. Using these multiple measures, IMPACT has awarded 600 teachers (out of 4,000) bonuses ranging from $3000 to $25,000 and fired nearly 300 teachers judged as “ineffective” in its initial years of full operation. For those teachers with insufficient student test data, different performance measures were used.

Here is a description of one classroom being observed by a “master educator.”

“A case in point is the lively classroom of Andrea Stephens (not her real name), a first-grade teacher at a racially mixed elementary school in Northeast D.C.

Master educator [Cynthia] Robinson-Rivers is conducting an informal observation as Stephens teaches a lesson about capital letters, punctuation marks, and the short “a.” Stephens is kind, firm, and engaging, and she wins points for gestures like asking a reluctant pupil if she could “get one of his smiles,” making him feel valued. But she is apparently not engaging enough. Several students are not paying attention; one is a mugger and a performer, and he can’t sit still. After several attempts to quiet him, Stephens gently pulls him up next to her, holding his hand while she addresses the rest of the class.

The general atmosphere suggests to Robinson-Rivers a need for better management. “The children weren’t completely out of control,” Robinson-Rivers says. “But if they  aren’t facing you it can suggest a lack of interest.”

The session reveals other perceived shortcomings, despite Robinson-Rivers’ respect for Stephens as “a warm, thoughtful practitioner.” It was too teacher directed, Robinson-Rivers says; it failed to make the objectives fully clear, and it didn’t make the most of limited instructional time. “If the pacing is too slow, you can lose valuable time from the lesson,” Robinson-Rivers says. “If in a 20-minute morning meeting the kids participate in a variety of engaging activities, it’s much easier to maintain their interest and enthusiasm.” Stephens also falls short on Teach 5 [one of the “commandments”]—checking to see whether students actually understood her. “There was no way to know whether the shy girl or the boy who spoke little English understood or not,” Robinson-Rivers says. Instead of having all the pupils answer in unison, she suggests that Stephens cold-call on individual students, or have all the boys or all the girls answer in some non-verbal way. “It’s hard because teachers do think they are checking for understanding. But it’s actually an easy one for professional development; you could just say there are three easy things you can do.” Stephens, whose overall score for the year was in the “effective” range, is open to evaluation and receptive to feedback—she even asked for an extra observation—and in this regard, master educators say she is fairly typical.”

Just as any system of evaluation with observers using subjective criteria can distinguish between acceptable and unacceptable performances, IMPACT does sort out on a four-point scale “effective” from “ineffective” teachers according to their nine commandments of “good” teaching. That the use of VAM and other indicators reward and punish teachers is also clear. Questions, however, about teacher turnover—has IMPACT reduced attrition among “effective” teachers?—and student performance—has IMPACT improved test scores? have gone unanswered thus far. Allegations of teachers and administrators erasing student answers and fiddling with tests further cloud the picture of IMPACT’s influence on teaching practice and student achievement.

Among teachers and principals, the degree to which IMPACT has influenced daily teaching is disputed. According to some teachers, there are colleagues who pull out special lessons when principals and “master educators” appear for 30-minute unannounced visits. Other teachers tremble and panic when an evaluator walks into their classroom and the lesson becomes a shambles.  And there are many teachers who relish the feedback they get from conferences after observations and assert that they have made changes and their lessons have improved. While the principal’s workload demanded by teacher observations has become unmanageable for some, others have welcomed the role of instructional leader. Some principals, however, find IMPACT destroying their ways of supervising  teaching in their schools.

What has occurred in Washington, D.C. with new curriculum standards, new tests, and IMPACT mirrors what has occurred in urban districts across the country that have put into place testing and accountability structures. Since the early 1990s and especially after the passage of No Child Left Behind most urban schools across the nation have narrowed their curriculum leaving less time for non-tested subjects, intensified teacher-centered instruction with more lessons devoted to preparing students for tests, and, in general, reduced instructional time for reaching all of the curricular standards they are expected to meet over the school year. New ways of evaluating teachers that anchor judgments of effectiveness, in part, on student test scores certainly reinforces accountability but to what degree teaching practices across elementary and secondary school classrooms have improved as a consequence of IMPACT or similar programs is yet undetermined. And whether those improved teaching practices have led to gains in student academic achievement, well, no one knows.

---

Evaluating Teachers Using Student Test Scores: Value-added Measures (Part 1)

In most organizations, supervisors measure and evaluate employees’ performance. Consequences, both positive and negative, flow from the judgments they make. Not in public schools where supervisors have commonly judged over 95 percent of all teachers “satisfactory.” Such percentages clearly do not distinguish between effective and ineffective teaching. The reform-driven agenda for the past decade that included testing, accountability, expanding parental choice through charter schools, and establishing a Common Core curriculum across the nation now includes in its to-do list distinguishing between good and poor teaching.[i]

The current generation of reform-driven policymakers, donors, and educational entrepreneurs are determined to sort out “good” from mediocre and poor teaching if for no other reason than identifying those high-performers who have had sustained effects on student learning and reward them with recognition, bonuses, and high salaries. They are equally determined to rid the teacher corps of persistently ineffective teachers.[ii]

How to identify the best and worst in the profession in ways that teachers perceive as fair, improves the craft of teaching, and retain their support for the process has, in most places, thwarted reformers. But not enough to stop policymakers and donors from launching a flurry of programs that seek to recognize high performers while firing time-servers.

Reform-minded policymakers, donors, and media have concentrated on annual test scores. Some big city districts like Washington, D.C., Los Angeles and New York have not only used student scores to determine individual teacher effectiveness but also permitted publication of each teacher’s “effectiveness” ranking  (e.g., Los Angeles Unified school district). Because teachers see serious flaws in using test scores to reward and punish teaching, they are far less enthusiastic about new systems to evaluate teaching and award bonuses.

Behind these new policies in judging teaching performance are models of teaching effectiveness containing complex algorithms drawn from research studies done a quarter-century ago by William Sanders and others called “value-added measures” (VAM).

How do value-added measures work? Using an end-of-year standardized achievement test in math and English, VAM predicts how well a student would do based on student’s attendance, past performance on tests, and other characteristics. Student growth in learning (as measured by a standardized test) is calculated. Thus, how much value—the test score–does the teacher add to each student’s learning in a year. Teachers of those students who take these end-of-year tests are then held responsible for getting their students to reach the predicted level. If a teacher’s students do reach or exceed their predicted test scores, the teacher is rated effective or highly effective. For those teachers whose students miss the mark, they are rated ineffective.

Most teachers perceive VAM as unfair. Less than half of all teachers (mostly in elementary schools and even a smaller percentage in secondary schools) have usable data (e.g., multiple years of students’ math and reading scores) to be evaluated. For those teachers lacking student test scores, new tests will have to be developed and other metrics will be used. Also teachers know that other factors such as student effort and family background play a part in their academic performance. They also know that other data drawn from peer and supervisor observations of lessons, the quality of instructional materials used by teachers, student and parent satisfaction with the teacher are weighed much less or even ignored in judging teaching performance.

Moreover, student scores are unstable year to year, that is, different students are being tested not the same ones as they move through the grades yet teacher “effectiveness” ratings are based on different cohorts of students. What this means is that a substantial percentage of teachers in one year might be ranked “highly effective” and in the next year may be ranked “ineffective.” False positives (e.g., tests that say you have cancer when you do not) are common in such situations. Furthermore, many teachers know that both measurement error and teaching experience (i.e., over time, teachers improve as they have bad years and good years) accounts for instability in ratings of teacher effectiveness. Finally, many teachers see the process of using student scores to judge effectiveness as pitting teacher against teacher, increasing competition among teachers rather than collaboration across grades and specialties within a school; such systems, they believe, are not aimed at helping teachers improve daily lessons but to name, blame, and defame teachers. [iii]

Yet with all of these negatives, there are also many teachers, principals, policymakers, and parents who are convinced that something has to be done to improve evaluation and distinguish between effective and ineffective teaching. In Washington, D.C. a new system of evaluation and pay-for-performance, inaugurated by former Chancellor Michelle Rhee, reveals both the strengths and the flaws in VAM.

 

[i] Daniel Weisberg, et. al., “The Widget Effect,” (Washington, D.C., The New Teacher Project, 2009).

[ii] There is a crucial distinction between “good” and “successful” teaching and an equally important one between “successful” teaching and “successful” learning that avid reformers ignore. See Gary Fenstermacher and Virginia Richardson, “On Making Determinations of Quality in Teaching,” Teachers  College Record, 2005, 107, pp. 186-213.

[iii] Linda Darling-Hammond and colleagues summarize the negatives of VAM in “Evaluating Teacher Evaluation,” Education Week, February 12, 2012 at:
http://www.edweek.org/ew/articles/2012/03/01/kappan_hammond.html
.   For another view that argues, on balance, that VAM is worthwhile in evaluating teachers, see Steven Glazerman, et. al., “Evaluating  Teachers: The Important Role of Value-Added,” Brookings Institution, November 17, 2010, at:
http://www.brookings.edu/reports/2010/1117_evaluating_teachers.aspx

 For stability in teacher ratings over time see, Dan Goldhaber and Michael Hansen, “Is It Just a Bad Class? Assessing the Stability of Measured Teacher Performance,” Center for Education Data & Research, University of Washington, 2010, CEDR Working Paper #2010-3. On issues of reliability and validity in value-added measures, see Matthew Di Carlo posts April 12 and 20, 2012 at:
http://shankerblog.org/?p=5621
;
http://nepc.colorado.edu/blog/value-added-versus-observations-part-two-validity
.