Holding Donors Accountable

The sad story that Washington Post journalist Valerie Strauss tells about the recent RAND report on improving teacher effectiveness, an effort partially funded by the Bill and Melinda Gates Foundation–$215 million of a half-billion dollar project–is neither the first (nor the last) failure in donor funding. After all, philanthropists take moderate to great risks in funding projects that promise high returns (as do  venture capitalists) and such ventures do fail. Here is the background story for the Intensive Partnerships for Effective Teaching.

Based upon the extensive research of Tom Kane, professor of Economics and Education at Harvard University, the premise of the project was that a multi-measure evaluation system of teachers anchored in student test scores (and peer evaluators) would sort out “effective” from ineffective teachers of low-income students in five districts and charter school networks. Districts and charter schools would then staff classrooms with “effective” teachers–the “good” teachers–and their students, following the premise of the initiative,  would outscore similar students in classrooms with regular teachers during the six year project (2009-2015). In addition, participating districts and charter networks gave cash bonuses to teachers designated as “effective” and substantially increased professional development.

Result? Outcomes for economically poor students in those schools staffed by these designated “effective” teachers, according to the RAND report, “were not dramatically better than outcomes in similar sites that did not participate in the … initiative.”

I could only guess that Professor Tom Kane was disappointed in the findings of the report. As were, again only a guess, program officers at the Gates Foundation. For donor Bill Gates, it must have been dismaying since this was another top-down, research-informed loss that the Foundation had launched to improve schools with results falling far short of what was promised (e.g., spreading technology into libraries and schools, the “Small Schools” initiative, backing Common Core standards). These top-down funding strategies have now given way to yet another Foundation effort: a $1.7 billion project aimed at K-12 school improvement announced a few months ago. More than 500 applications have poured in to get a chunk of that money that will go to local stakeholders.

Journalist Valerie Strauss classifies the RAND report documenting the Foundation’s effort between 2009-2015 to raise test scores of low-income students as an instance of “they-were-warned-but-didn’t-listen….” Donors, she says should have listened to individual researchers and organizations of researchers who repeatedly criticized efforts  to evaluate teachers by  relying on student test scores (see here, here, and here).

Perhaps. Let me imagine a conference room at the Gates Foundation in 2008 before the Intensive Partnerships for Effective Teaching was launched. In this conference room are  program officers and researchers who had provided the underpinning the experiment. Here are the exchanges that I imagine:

Senior program officer:

“The American Statistical Association and the Board on Testing and Assessment of the National Research Council have argued against our doing this project. Many researchers have written us and spoken out against using student test scores to determine how good the teachers are. How should we handle these criticisms?”

Senior researcher funded by Foundation to do studies upon which initiative is based:

“C’mon, guys, when was their ever unanimity or even a preponderance of evidence supplied by researchers on any innovative experiment? OK, that is a rhetorical question because we all know the answer: the quality of educational research ranges from poor to mediocre and even when a study is rigorous, researchers split over how the study was done and the statistical significance of the findings. So I say, plow ahead because the research we rely upon here is solid. The research we have done is rigorous and the findings we can depend upon.”

Junior program officer:

“May I say that I, too, am impressed with what studies I have examined. They do point in this direction for identifying effective teachers of low-income kids. What I worry about is what happens if the results of working with these districts and schools that have volunteered show that the students of these teachers we identified as effective did pretty much the same as poor kids in classes of regular teachers outside of our project?

We would have intervened in the district, claimed that we have identified “good” teachers, assigned them to classes of at-risk kids and nothing much happened when these kids took the state tests, compared to a group of teachers and students we did nothing with. Who is going to be held accountable for disappointing outcomes of this initiative?”

*******************************************

Yes, this is an imaginary conversation that I constructed among Foundation staff and researchers in the year prior to implementation of the project.  And, yes, again, I end it with the question of who is held accountable when donors fund moderate-to-high-risk projects and the results range from disappointing to harmful.

Insiders to the donor strategy of taking funding risks have noted the issue of responsibility for outcomes.

If you agree that philanthropy should be taking big risks, you shouldn’t be too surprised by big failures. Nor should you be reflexively critical of the funders behind them, since they’re doing what we want them to do. At Inside Philanthropy, we make a point of not piling on when funders fail. If donors and foundations are kicked around too harshly for their mistakes, they’ll take fewer risks and we’ll all be worse off. 

All that said, risk taking in philanthropy needs to be approached with great care, especially in an area like public education. When risky initiatives go wrong, they can impose major costs on all involved. In the case of K-12, that means students, parents, teachers, administrators and the taxpayers who pick up the bill for failure. 

The Los Angeles Times editorial board went one step further in slamming an earlier Bill and Melinda Gates Foundation initiative in California:

Philanthropists are not generally education experts, and even if they hire scholars and experts, public officials shouldn’t be allowing them to set the policy agenda for the nation’s public schools. The Gates experience teaches once again that educational silver bullets are in short supply and that some educational trends live only a little longer than mayflies

I end this post on this issue of accountability and setting the educational agenda through funding projects that philanthropists believe will make a difference in schooling U.S. children.

Lack of donor accountability

When foundation grants fail to achieve the objectives officials sought, philanthropists turn their backs, shrug, and walk away. Or try again. They have no responsibility to districts, individual schools, teachers, students and parents for hopes raised and dashed. Donors are beyond the reach of being fired or voted out of office. Yet as anyone knows from personal experience, admitting error is crucial to insights into a problem and, ultimately invention of better ways to solve it.

For those who support philanthropic giving, this unaccountability is an exercise of personal liberty and risk-taking in acting for the public good and is in the best tradition of a democracy. Moreover, some have argued: “[S]uch virtual immunity represents foundations’ greatest strength: the freedom to take chances, to think big, to innovate, to be, in the words of the late Paul Ylvisaker of the Ford Foundation, ‘society’s passing gear.’”

Being society’s “passing gear,” however, assumes that funders and their retinue of experts know best how to identify educational problems, sort out symptoms from fundamental causes, and adopt solutions that solve the problem. When donors bet foolishly or are simply wrong and projects and programs fail who are these funders answerable to for their errors in judgment? No one, as far as I can see.

Nothing “nasty” will happen to the Gates Foundation for their slip-ups in top-down policy making to improve schooling. At least so far. I do have a proposal, however, were foundations to awake to their responsibility for errors in policy judgment rather than shrug and walk away.

Proposal

When foundations give money to districts and schools to alter school organization, curriculum, or instruction (or all three) stipulating what districts have to do to spend donor money and what results are expected, then districts and donors should have a separate written agreement for what happens if the project falls short of achieving desired goals.  If the district has met its part of the agreement–as confirmed by an independent outside source chosen by both donor and grantee–and still the project fails according to the agreed-upon metrics to be used, the foundation would provide additional funding–no strings attached–to districts equal to the original grant.

Would this reduce risk-taking on the part of foundations? Possibly. Would it be an incentive for districts and schools to insure full implementation of the strategy and program? Yes, it would.

Were such an agreement to be struck between donor and recipient it would be a small step in the direction of foundations being held accountable for their initiatives to improve schools in the U.S.

 

 

 

Advertisements

7 Comments

Filed under Uncategorized

7 responses to “Holding Donors Accountable

  1. David F

    Hi Larry—thanks for this. One other aspect of the RAND study needs to be highlighted: the underlying evaluation system in all but one of the schools was the Danielson Framework. On page 73, the report’s authors note: “Although sites designed and implemented their observation systems differently, most used Danielson’s FFT as a starting point. All but one of the sites developed rubrics based on the Danielson framework, which meant that these sites emphasized a constructivist approach to pedagogy that involves high levels of student engagement and communication (Danielson Group, undated).”

    So, if that’s how these schools were determining “effective teaching”, no wonder the thing failed. This isn’t the first time that Danielson has been shown to do nothing with regards to student learning: see Geoffrey Borman and Steven Kimball, “Teacher Quality and Educational Equality: Do Teachers with Higher Standards-Based Evaluation Ratings Close Student Achievement Gaps?” The Elementary School Journal, vol. 10 (Sept, 2005) pp. 3-20 (http://dx.doi.org/10.1086/496904

    But I suppose the Gates people liked it, as it emphasizes just the sort of things that can make use of various ed tech programs and plays into project based learning, personalized learning, etc.

  2. Well said, Larry. I’d like to add another lesson that ought to be learned from such failures, about the relationship of the schools to educational research. What are educational leaders (and policy makers and philanthropists) to make of this effort? It was brought to them in good faith as a promising, research-based intervention. It didn’t work. Should the educational leaders conclude that educational research can’t be used as a guide for practice? If so, should they conclude that participating in such research is a waste of effort — or worse, a source of potential damage, as you warn?

    To be more precise, for the sake of argument let’s assume this was good effectiveness research on a sound implementation of the intended intervention, but it was the intervention research that was flawed in some way. Then the counter-argument would be that only good research-based interventions should guide practice, and this was a bad intervention that ignored important disconfiming research evidence (as the ASA and others warned). But we have no consensus on what constitutes a good research-based intervention! How are we to guide practitioners and policy makers on how to distinguish between good intervention research (worth the risk of a large-scale trial like this) and bad intervention research, whether the school is doing it, or someone else is doing it within their educational system?

    I would direct readers to the work of Robert Slavin. I think he’s the most articulate critic, with the most constructive solutions, to the issue of how to make sense of efficacy and effectiveness research evidence for educational practitioners and policy makers. The philanthropists would do well to follow his lead as they plan their interventions. I would agree that public education is generally risk-averse, and we need the willingness of the philanthropists to take risks and to learn from failures. But we need those risks to be well informed by the best evidence available.

  3. Pingback: Holdin Bill Gates Accountable? | GFBrandenburg's Blog

  4. mike g

    I think that’s a very clever idea! Akin perhaps to a Letter Of Credit.

    A challenge, perhaps, is that there are lots of details impossible to figure out “in advance.” What typically happens is both Donor and District disagree on these, wrestle as best they can. Certainly happened in this instance.

    With the “Cuban Guarantee” in place, what is normal District-Donor “carping” might have an additional component, where Donor wonders if District is purposefully sabotaging the project (by fighting on key details that aren’t discussed in the original agreement) in order to get the “free money.”

    • larrycuban

      Yes, Mike,the scenario you sketch out might occur. And there may well be more hitches in the proposal that I have not considered. The principle of the proposal nonetheless is clear and people of goodwill can work on potential glitches. The point as you know is that the process of coming together and making clear each side’s responsibilities would probably produce a more mindful and do-able project. Thanks for comment.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

w

Connecting to %s