From time to time, I dip back into my blog archives to resurrect and update an earlier post because the issue raised in the post remains current. This updated post was originally published January 15, 2010,
As a teacher and later as superintendent, I used to snicker when I read research studies that stated findings confidently, brandishing strong numbers showing statistical significance about school phenomena that I and colleagues had known were true based on our experience. Subsequently, I became a researcher and I published studies on how teachers taught, technologies in schools, and how superintendents ran districts. In those studies, truth be told, I offered conclusions drawn from my data that must have sent other teachers and superintendents into hilarious guffaws.
These memories preface a few comments–and, I confess, a few chuckles–on a study completed in 2009 by top-of-the-line researcher Brian Rowan and colleagues called the Study of Instructional Improvement ( sii final report_web file). Education Week and The Harvard Education Newsletter featured this study.
The five year quasi-experimental study of three Comprehensive School Reform models (Accelerated Schools Program, America’s Choice, and Success for All) included analysis of 75,000 teacher logs from nearly 2000 teachers teaching literacy (reading and language arts) in grades one to five in schools across the nation . What did they find?
They found that teachers in the three models varied in how often they taught reading and language arts in the same grade and school.
With many more examples, the researchers pointed out that these variations in instructional practices was NOT due (yes, that is a NOT) to the students’ achievement levels, “their previous instructional histories … or to variations in ethnic and socioeconomic composition.”
They also found that the startling variation in teaching practices within the same grade and school came, again, NOT from “teachers’ professional preparation, … years of experience, or pedagogical knowledge.” If anything these factors “have only tiny effects on teaching practices.”
These findings are not intuitive or common sense; neither are they laughing matters. They confound and confuse all those fervent reformers who believed in their heart of hearts that these factors—student achievement and professional preparation–had large, not “tiny effects on teaching practices.”
Such conclusions would not surprise principals, instructional coaches, and supervisors who regularly visit classrooms and observe teachers. Nor did these conclusions surprise me–this is where the chuckles enter the picture–since I have spent many years visiting classrooms as a teacher, administrator, and researcher. The conclusions might, however, surprise teachers, school board members, state and federal policymakers, donors with deep pockets, and parents since few ever have the chance or taken the opportunity to step into classrooms and stay for awhile.
The researchers concluded that “schools remain ‘loosely coupled’ organizations where teachers have considerable autonomy and function largely as curriculum brokers.” They call this conclusion a “dismal observation.”
The word “dismal” (how about the word “realistic?”) signals readers that these disappointed reformers/researchers would have welcomed data showing that teachers implementing these whole school reform models narrowed the band of variation in teaching reading and language arts so that students’ learning opportunities were not subject to the luck of the draw in getting one teacher or another. But they did find an exception buried in their data.
Two of the three reform models–America’s Choice* and Success for All**–well-defined, specified programs with on-site coaches, principals pressing and supporting teachers to be faithful to the program design, led to classrooms where teachers hewed more closely to prescribed instructional practices. Whether or not, such faithful implementation of teaching practices–what researchers call “fidelity to design”–translated into higher test scores, they cannot say, however (See PDF AERJ on instruction). What they can say is that variation in teaching practices shrunk considerably in these two models and students experienced more consistent teaching practices as they went through the grades. And that is news that even an ex-high school teacher and superintendent with plenty of miles on the odometer found encouraging.
• The National Center for Education and the Economy, home to America’s Choice for years, sold program to Pearson and it is part of their offerings in professional development. I would pronounce it dead.
**Success for All is alive and well. See here.