As a teacher and later as superintendent, I used to snicker when I read research studies that stated findings confidently, brandishing strong numbers showing statistical significance about school phenomena that I and colleagues had known were true based on our experience. Subsequently, I became a researcher and I published studies on how teachers taught, technologies in schools, and how superintendents ran districts. In those studies, truth be told, I offered conclusions drawn from my data that must have sent other teachers and superintendents into hilarious guffaws.
These memories preface a few comments–and, I confess, a few chuckles–on a recent study completed by top-of-the-line researcher Brian Rowan and colleagues called the Study of Instructional Improvement ( sii final report_web file). Education Week and The Harvard Education Newsletter featured this study–thus, the title–“Breaking News” (Yes, there is a mildly sarcastic edge to the phrase).
The five year quasi-experimental study of three Comprehensive School Reform models (Accelerated Schools Program, America’s Choice, and Success for All) included analysis of 75,000 teacher logs from nearly 2000 teachers teaching literacy (reading and language arts) in grades one to five in schools across the nation in. What did they find?
They found that teachers in the three models varied in how much time they spent teaching reading and language arts in the same grade and school.
“…it would be very to find two first-grade teachers in the same school, one of whom focused on word analysis skills about one day a week and another who focused on this topic four days a week.”
With many more examples, the researchers pointed out that these variations in instructional practices was NOT due (yes, that is a NOT) to the students’ achievement levels, “their previous instructional histories … or to variations in ethnic and socioeconomic composition.”
They also found that the startling variation in teaching practices within the same grade and school came, again, NOT from “teachers’ professional preparation, … years of experience, or pedagogical knowledge.” If anything these factors “have only tiny effects on teaching practices.”
These findings are not intuitive or common sense; neither are they laughing matters. They confound and confuse all those fervent reformers who believed in their heart of hearts that these factors had large, not “tiny effects on teaching practices.”
Such conclusions would not surprise principals, instructional coaches, and supervisors who regularly visit classrooms and observe teachers. Nor did these conclusions surprise me–this is where the chuckles enter the picture–since I have spent many years visiting classrooms as a teacher, administrator, and researcher. The conclusions might, however, surprise teachers, school board members, state and federal policymakers, and parents since few ever have the chance or taken the opportunity to step into classrooms and stay for awhile.
The researchers concluded that “schools remain ‘loosely coupled’ organizations where teachers have considerable autonomy and function largely as curriculum brokers.” They call this conclusion a “dismal observation.”
The word “dismal” (how about the word “realistic?”) signals readers that these disappointed reformers/researchers would have welcomed data showing that teachers implementing these whole school reform models narrowed the band of variation in teaching reading and language arts so that students’ learning opportunities were not subject to the luck of the draw in getting one teacher or another. And they did find an exception buried in their data.
If there is any “Breaking News,” it is that in one of the three reform models–America’s Choice and Success for All–well-defined, specified programs with on-site coaches, principals pressing and supporting teachers to be faithful to the program design, led to classrooms where teachers hewed more closely to prescribed instructional practices. Whether or not, such faithful implementation of teaching practices translated into higher test scores, they cannot say. See PDF AERJ on instruction. What they can say is that variation in teaching practices shrunk considerably in these two models and students experienced more consistent teaching practices as they went through the grades. And that is news that even an ex-high school teacher and superintendent with plenty of miles on the odometer found both surprising and encouraging.