Principals And Test Scores

I read a recent blog from two researchers who assert that principals can improve students’ test scores. The researchers cite studies that support their claim (see below). These researchers received a large grant from the Wallace Foundation to alter their principal preparation program to turn out principals who can, indeed, raise students’ academic achievement.

I was intrigued by this post because as a district superintendent I believed the same thing and urged the 35 elementary and secondary principals I supervised—we met face-to-face twice a year to go over their annual goals and outcomes and I spent a morning or afternoon at the school at least once a year—to be instructional leaders and thereby raise test scores. Over the course of seven years, however, I saw how complex the process of leading a school is, the variation in principals’ performance, and the multiple roles that principals play in his or her school to engineer gains on state tests (see here and here). And I began to see clearly what a principal can and cannot do. Those memories came back to me as I read this post.

First the key parts of the post:

A commonly cited statistic in education leadership circles is that 25 percent of a school’s impact on student achievement can be explained by the principal, which is encouraging for those of us who work in principal preparation, and intuitive to the many educators who’ve experienced the power of an effective leader. It lacks nuance, however, and has gotten us thinking about the state of education-leadership research—what do we know with confidence, what do we have good intuitions (but insufficient evidence) about, and what are we completely in the dark on? ….

Quantifying a school leader’s impact is analytically challenging. How should principal effects be separated from teacher effects, for instance? Some teachers are high-performing, regardless of who leads their school, but effective principals hire the right people into the right grade levels and offer them the right supports to propel them to success.

Another issue relates to timing: Is the impact of great principals observed right away, or does it take several years for principals to grapple with the legacy they’ve inherited—the teaching faculty, the school facilities, the curriculum and textbooks, historical budget priorities, and so on. Furthermore, what’s the right comparison group to determine a principal’s unique impact? It seems crucial to account for differences in school and neighborhood environments—such as by comparing different principals who led the same school at different time points—but if there hasn’t been principal turnover in a long time, and there aren’t similar schools against which to make a comparison, this approach hits a wall.

Grissom, Kalogrides, and Loeb carefully document the trade-offs inherent in the many approaches to calculating a principal’s impact, concluding that the window of potential effect sizes ranges from .03 to .18 standard deviations. That work mirrors the conclusions of Branch, Hanushek, and Rivkin, who estimate that principal impacts range from .05 to .21 standard deviations (in other words, four to 16 percentile points in student achievement).

Our best estimates of principal impacts, therefore, are either really small or really large, depending on the model chosen. The takeaway? Yes, principals matter—but we still have a long way to go to before we can confidently quantify just how much.

I thoroughly agree with the researchers’ last sentence. But I did have problems with these assertions supported by two studies they listed.

*That principals are responsible for 25 percent of student gains on test scores (teachers, the report account for an additional 33 percent of those higher test scores). I traced back the source they cited and found these statements:

A 2009 study by New Leaders for New Schools found that more than half of a school’s impact on student gains can be attributed to both principal and teacher effectiveness – with principals accounting for 25 percent and teachers 33 percent of the effect.

The report noted that schools making significant progress are often led by a principal whose role has been radically re-imagined. Not only is the principal attuned to classroom learning, but he or she is also able to create a climate of hard work and success while managing the vital human-capital pipeline.

These researchers do cite studies that support their points about principals and student achievement but cannot find the exact study that found the 25 percent that principals account for in student test scores. Moreover, they omit  studies that show  higher education programs preparing principals who have made a difference in their graduates raising student test scores (see here).

I applaud these researchers on their efforts to improve the university training that principals receive but there is a huge “black box” of unknowns that explain how principals can account for improved student achievement. Opening that “black box” has been attempted in various studies that Jane David and I looked at a few years ago in Cutting through the Hype

The research we reviewed on stable gains in test scores across many different approaches to school improvement all clearly points to the principal as the catalyst for instructional improvement. But being a catalyst does not identify which specific actions influence what teachers do or translate into improvements in teaching and student achievement.

Researchers find that what matters most is the context or climate in which the actions occurs. For example, classroom visits, often called “walk-throughs,” are a popular vehicle for principals to observe what teachers are doing. Principals might walk into classrooms with a required checklist designed by the district and check off items, an approach likely to misfire. Or the principal might have a short list of expected classroom practices created or adopted in collaboration with teachers in the context of specific school goals for achievement. The latter signals a context characterized by collaboration and trust within which an action by the principal is more likely to be influential than in a context of mistrust and fear.

So research does not point to specific sure-fire actions that instructional leaders can take to change teacher behavior and student learning. Instead, what’s clear from studies of schools that do improve is that a cluster of factors account for the change.

Over the past forty years, factors associated with raising a school’s academic profile include: teachers’ consistent focus on academic standards and frequent assessment of student learning, a serious school-wide climate toward learning, district support, and parental participation. Recent research also points to the importance of mobilizing teachers and the community to move in the same direction, building trust among all the players, and especially creating working conditions that support teacher collaboration and professional development.

In short, a principal’s instructional leadership combines both direct actions such as observing and evaluating teachers, and indirect actions, such as creating school conditions that foster improvements in teaching and learning. How principals do this varies from school to school–particularly between elementary and secondary schools, given their considerable differences in size, teacher peparation, daily schedule, and in students’ plans for their future. Yes, keeping their eyes on instruction can contribute to stronger instruction; and, yes, even higher test scores. But close monitoring of instruction can only contribute to, not ensure, such improvement.

Moreover, learning to carry out this role as well as all the other duties of the job takes time and experience. Both of these are in short supply, especially in urban districts where principal turnover rates are high.

I am sure these university researchers are familiar with this literature. I wish them well in their efforts to pin down what principals do that account for test score improvement and incorporate that in a program that has effects on what their graduates do as principals in the schools they lead.

 

 

Advertisements

9 Comments

Filed under school leaders, testing

9 responses to “Principals And Test Scores

  1. jeffreybowen

    I am surprised no mention is made of Robert Marzano’s superlative meta-analysis. He identified more than 20 factors that link to principals’ behaviors and influence student achievement. His framework for leadership evaluation has been very useful for my work with both traditional public schools and charter schools.

  2. jeffreybowen

    You really did miss the research boat on this one. Take a good look at the work of Robert Marzano and colleagues.

    • larrycuban

      Thanks for reminder of work by Robert Marzano. I should have mentioned it. My focus is on those studies that specifically try to show correlation between principal behavior and test scores. Marzano’s work initially was on principal and “effective” schools with academic achievement writ large–not just test scores. And his model included a raft of research on the importance of what Heck and Hallinger (1998) called “mediating variables” or factors such as climate of school, relationships with teachers, etc. His most recent work on school leadership (2018) again points out the power of the mediating variables in evaluating principal leadership and effect on student achievement, again, writ large. See: http://www.marzanocenter.com/wp-content/uploads/2018/03/Marzano-Focused-School-Leader-Evaluation-Model-031218.pdf
      Thanks for comment, Jeffrey.

  3. Laura H. Chapman

    http://hanushek.stanford.edu/sites/default/files/publications/Branch%2BHanushek%2BRivkin%202013%20EdNext%2013%281%29_0.pdf

    Achievement equals value-added in math scores. Is this study a compelling argument for anything other than improving math test scores as the basis for thinking about a principal’s influence, and in this case only in Texas? I think not. I remain skeptical about the narrow focus on test scores as if these function as some gold standard for excellence in education. I am also skeptical about the tendency of economists to think that a standard deviation of a certain size has any real significance for educational theory and practice.

    • larrycuban

      Thanks for comment, Laura.

    • I am skeptical as well. Demographic changes are also huge. Across my district we have three newcomer English language learner programs. However, the program at my school serves primarily SLIFE (Students with Limited or Interrupted Formal Educatio) from Central America. The other two programs serve a much more varied and previously educated population. At the district level we have had to advocate for the particular needs of our students, such as a basic pre-algebra math course. It has been a challenge for our district to give our students such supports. Meanwhile, when we look at math data, it has changed significantly from five years ago when our recent waves of unaccompanied minors started arriving. Couple this with the fact that the local charter school does not take English language learners, and other schools in our district have principals that shed their more vulnerable students into our school, we may want to look more closely at how a principal can game the system and choose which students sit for the tests at their school. Such issues are well-known. Is this part of the study?

      • larrycuban

        Thanks for the comment, Olga. Particularly the point about principals where student demography changes. I cannot say for sure whether the studies cited by the university researchers contain a cross-section of schools that include ones you describe but I would guess it would be unlikely.

  4. Jennie Schmidt

    The problem with this angle is that the test score is the holy grail. Indeed it could be that principals who enforce test prep see test scores rise. Does that mean they are good? Good for what? Good for whom?

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s