Here is a story about a program I taught in and directed over 50 years ago. What I experienced raised puzzling questions about what constitutes program success and failure that I thought I could answer then but cannot do so now.
In the mid-1960s, I taught in and later directed a federally-funded teacher training program located in Grimke elementary school, Banneker and Garnet-Patterson Junior High Schools, and Cardozo High School in Washington, D. C. The Cardozo Project in Urban Teaching, as it was then called, prepared returned Peace Corps Volunteers to teach in urban schools. The “interns,” as they were called, taught for half-days under the supervision of master teachers, took university seminars on-site after-school, and in evenings and late-afternoons developed curriculum materials and worked in the community. At the end of the year the “interns” were certified to teach in the District of Columbia and were on their way to earning a master’s degree in their field through two local universities (see here and here).
An independent evaluation confirmed that 61 interns had completed the training between 1963-1967. Of the 56 who had finished (two had died and three left the program to raise families), 42 (or 75 percent) were teaching in urban schools, other federally-funded programs, or overseas–one goal of the program. The on-site training, the supervision by D.C. teachers, and after-school seminars seemed to be a fruitful mix for channeling trained rookies into the system. The evaluation and praise for the program led the D.C. School Board to fund and rechristen the program as the Urban Teacher Corps–another goal of the pilot program.
Getting a school board to use its limited monies to continue a federally-funded pilot program meant that school officials saw its worth in meeting District of Columbia goals. That is a mark of success in any playbook of school reform then and now.
Consider further that the program model of training new teachers became the poster-child for a federal initiative to train teachers nationally for high-poverty urban and rural schools. The National Teacher Corps legislation (1966) adopted the model used at Cardozo, Banneker, and Grimke for training teachers on-site but rather than fund districts, federal officials funneled monies to universities that took responsibility for awarding degrees (see here and here).
Surely, the pilot program achieved two of its goals: three of four interns became full-time teachers after completing the program. And the program was adopted by an urban district using locally budgeted funds. Accomplishing both goals suggests program effectiveness, a sign of clear success. That the pilot program became the model for a National Teacher Corps further cements the sweet smell of success.
There is a “however” to this seeming success story that needs to be mentioned. After the Urban Teacher Corps became part of the D.C. schools, the Board of Education fired its superintendent and in 1970 appointed Hugh Scott its first black superintendent. During Scott’s brief administration (he resigned in 1973), he dismantled the Urban Teacher Corps. Together the federally-funded pilot program and locally-funded UTC existed for just under a decade.
Similarly, with the election of Ronald Reagan in 1980, the National Teacher Corps disappeared as federal monies for education went to the states in bloc grants. The NTC had lasted just over a decade.
The disappearance of both teacher-training programs within ten years suggests failure even though they could be fairly characterized as successes in achieving their primary goals. Success or failure? If it is either/or then determining whether the program should go down in the books as a successful reform in districts recruiting and training new teachers and then have it disappear raises questions about how does one define program success—achieving goals? Longevity? Or are there gray areas in defining success that seldom get attention?
I want to add one other piece to the puzzle and finish the story.
On the cusp of leaving the Cardozo Project in Urban Teaching, I asked a professor at the University of Maryland to find out if the students in classes of elementary and secondary school interns achieved less, about the same, or more than students who had non-intern teachers. While raising student achievement was not an explicit goal for the Cardozo Project in Urban Teaching or the Urban Teacher Corps there was an assumption (which I shared), that well trained teachers would eventually lead to better teaching and better teaching would lead to higher student achievement.
The professor designed a study where students in classes taught by interns were matched with students taught by regular D.C. teachers. With no district-wide standardized test available, the professor used the Iowa Test of Basic Skills as the outcome measure of teaching in reading, math, and other skills. About a year later, the professor called me up and said he had the results. We met for coffee and he showed me what he had found.
In both elementary and secondary classrooms of interns and regular teachers, students in regular classrooms did marginally better than those in intern classrooms. While the percentile scores in both sets of classes were fairly low compared to the national average, I was still shocked. I had believed that the teacher-training program I taught in and eventually directed was so strong that even in one year with “interns” D.C. students would do well academically. I was wrong.
Although standardized testing was becoming common–the year is 1968–as a consequence of the Coleman Report (1966) and the federal Elementary and Secondary Education Act (1965) which required outcome measures to hold districts accountable for the federal dollars they received, I knew little then about the design and methodology the professor had used to evaluate student achievement.
Of course, now I realize that there were flaws in the evaluation design—it was not a random sample of students or interns; the test questions covered content and skills that students had not yet reached in the D.C. curriculum; only one year was covered–still I was shaken by the results.
So I come to the end of my story and the puzzle of defining success that still has its hooks in me. Here is an example of a pilot program that initially appeared as a success in achieving its primary goals. The Cardozo Project in Urban Teaching baptized later by the D.C. administration as the Urban Teacher Corps had the distinct smell of success. Adding to the fragrance was the founding of the National Teacher Corps. Yet within a decade these teacher training programs disappeared.
And, finally, as an after-thought, I discovered that students achieved less well in classes taught by interns than did students in regular classrooms. Even though raising student achievement was not one of the goals of these programs, the results turned my assumptions inside out. Reform outcomes are seldom tidy.
Why this story? It is part of a puzzle that policymakers, administrators, practitioners, and political leaders still cannot resolve when it comes to determining success or failure of a particular reform. And it is one that I continue to work on.