Data-Driven To Distraction (Dennis Shirley and Andy Hargreaves)

This Education Week “Commentary” by Boston College professors Shirley and Hargreaves was published October 4, 2006. In England, RATL (see below) continues its collaborative, educator-driven process of reform “by schools for schools” in 2011.

Every few years in American education, a new slogan is coined as the Next Big Thing. Total quality management, shared decision-making, and outcomes-based education all marched across the educational landscape once, grabbing headlines, filling copy—and leaving little improvement in learning in their wake.

Right now, data-driven instruction, results-oriented improvement, and evidence-based education are the watchwords. They show up everywhere—from state education department Web sites to principals’ and superintendents’ job descriptions—insisting that instructional practices  should be driven by the analysis of student-achievement data as measured by prescribed standardized tests….

Teachers are no longer the drivers of reform, but the driven. Many teachers and schools, in fact, are being driven to distraction. Under the pressures of the federal No Child Left Behind Act and its mandate for “adequate yearly progress,” teachers in struggling schools are being told that only results matter—and even these rarely extend beyond tested achievement in literacy and math….

With AYP deadlines looming and time running out, teachers have little chance to consider how best to respond to the figures in front of them. They find themselves instead scrambling to apply instant solutions to all the students in the problematic cells—extra test-prep, a new prescribed program, or after-school and Saturday school sessions. There are few considered, professional judgments here, just simplistic solutions driven by the scores and the political pressures behind them.

Data-driven instruction obliterates the crucial fact that to be effective, educators have to use many different kinds of information to think about what they are doing in classrooms. While statistics can be immensely useful, they do not automatically point to which instructional approaches will work best with the diverse learners that make up a school’s classes, or a nation’s schools. One child may struggle with under-performance because she has difficulties with reading, a second because he has a turbulent home life, and and a third because she is a recent immigrant learning English as a second language. Faced with such diversity, teachers and educational leaders have to be intelligently informed by evidence, not blindly driven by it to teach a certain way….

Our instructional choices should be based on all kinds of evidence and experience, processed together in professional learning communities that help us identify common problems, swap ideas and strategies, and develop and deploy our own school-based assessment instruments. Mindful teaching needs to be evidence-informed, not data-driven.

Better alternatives already exist. This has become clear to us through a series of visits to schools in England. There, we have led a research team evaluating the Raising Achievement Transforming Learning, or RATL, project of the Specialist Schools and Academies Trust, an organization that coordinates many of the state secondary schools across the country. RATL comprises more than 330 English secondary schools. To be eligible to join the project, the schools first had to be identified as underachieving, as defined by a composite of diverse pupil-test-score data. In its first two years, student achievement at project schools has risen steadily, and, in and, in almost three-quarters of them, strikingly so.

How has this happened?

First, RATL is the antithesis of top-down improvement. Its principals receive the equivalent of about $16,000 each year to spend on whatever they deem best to raise pupil achievement. School leaders provide project members with a menu of short-, medium-, and long-term strategies that have had proven success when applied elsewhere. Schools are then networked with each other, attached to coaching principals from high-achieving mentor schools, and offered regional conferences to help principals understand what their pupil-achievement data mean and how best to capitalize on the information with their teachers. By following a teacher-friendly principle of schools helping schools, and providing principals and teachers with considerable latitude in defining and addressing problems, the network has achieved rapid and impressive success. Data-driven instruction obliterates the crucial fact that to be effective, educators have to use many different kinds of information to think about what they are doing in classrooms.

Second, the project has a key cultural component that is based on the insight that test results rarely present self-evident instructional strategies to address the needs of struggling pupils. Rather, the data are, in and of themselves, often ambiguous, reflecting the nonlinear, and sometimes ingenious, ways that diverse learners acquire knowledge. As a consequence, project leaders do not rush from diagnosis to action, but emphasize the intermediary step of professional reflection and analysis. This step requires deep cultural change in many schools, as teachers work to shift their school culture from one of isolated instructors responsible only for their own pupils, to one of lifelong learners with the mission of improving the education of all learners in a school.

As part of this cultural transformation, RATL leaders try to ensure that pupil-achievement data are embedded in school-based cultures that appreciate the value of tests, but are not limited to them. In a country that has just abolished national achievement targets (with Wales going even further and abolishing standardized tests), RATL schools now have the freedom to set their own ambitious goals and targets, instead of frantically trying to comply with targets imposed by others. They emphasize that much school improvement involves cultural change, and that it takes time and professional sophistication to understand what test scores can and cannot tell us. In this model, careful scrutiny and discernment, not “drivenness,” are valued, and indeed are viewed as the heart and soul of successful educational change.

Third, RATL’s consistent focus on pupil achievement has not distorted or diminished the curriculum in ways that are becoming increasingly evident in many American schools. Here, standardized tests often have become the curriculum. In England, many principals have used the RATL funds to support art projects, physical education, or foreign-language courses. Principals of RATL schools in poor and working-class communities try to both broaden and deepen the curriculum to give all children multiple opportunities to flourish academically. In the United States, on the other hand, the achievement gap in tested performance coexists with a widening learning gap between functional basics for the poor and working class and an enriched and enlarged set of learning experiences for the privileged in the suburbs—where schools are free of many testing constraints and can (and do) fly far beyond the standards.

Evidence and experience, teachers working with teachers, schools helping schools, and continual reflective inquiry by educators—this path to improvement offers students so much in terms of  teacher professionalism and creative responses to the on-the-ground realities of classrooms and schools. Do we American educators dare to learn from our British counterparts? Can we apply the inventiveness of professionals collaborating at their best, rather than  adhere blindly to the fad of the moment marketed as “data-driven instruction”?


Filed under Reforming schools

4 responses to “Data-Driven To Distraction (Dennis Shirley and Andy Hargreaves)

  1. Pingback: The Best Resources Showing Why We Need To Be “Data-Informed” & Not “Data-Driven” | Larry Ferlazzo's Websites of the Day...

  2. The school where I am a Governor(trustee) is part of the SSAT network and has found the RATL very helpful in moving out of “Special measures” and indeed our Head teacher is now part of the National Leaders network supporting other schools,in similar contexts,along their transformation journey.

    Sadly the incoming government,influenced by the “free” school movement in Sweden and your own Charter school movement,(predicated on the research of Caroline Hoxby)now believes supply side reform and “competition” will raise standards faster.
    Of course the government contend that collaboration and competition are not mutually exclusive.However having spent over 30 yrs teaching and leading schools and colleges I tend to disagree.

  3. Anyone who wishes to know more about RATL and the SSAT’s work with schools in the RATL programme can contact me David Douglass

  4. Pingback: Today’s “Round-Up” Of Recent School Reform Posts | Larry Ferlazzo's Websites of the Day...

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s