Category Archives: school reform policies

Too Personalized (Peter Greene)

Peter Greene has been a high school English teacher in Northwest Pennsylvania for over 30 years. He blogs at Curmudgucation. This post appeared July 12, 2018

Personalized learning is the hot new idea in education reform, but some versions could get a little too personal.

While personalized learning is a broad and ill-defined field these days, many folks want to harness computer power to match students up with perfectly suited educational materials. This involves some sort of algorithm that collects and crunches data, then spits out a result, not unlike the way Facebook or Netflix collect data with users in order to match them up with the right products, or at least the best marketing for those products. As we’ve seen with the Cambridge Analytica scandal, there are some real privacy issues with data mining on this scale, but that has not stopped developers from digging deeper and deeper.

Personalized learning can be as simple as an exercise management system. Pat completes Widget Studies Worksheet 457A/rq, and because Pat missed questions 6, 9, and 11, the algorithm says Pat should next complete Worksheet 457B/sg, and so on until Pat completes Unit Test 1123-VZ and is declared a master of widgetry. This may sound like a boring mass work worksheet, but instead of paper worksheets, the modern system puts all the worksheets on a computer and students complete them on a computer screen, so it’s like super-exciting.

Data mining academics is central to many personalized systems. AltSchool, the Silicon Valley Wunderschool (now a business marketing wunderschool-in-a-box) touted its massive data mining, with teachers recording every significant learning moment and turning it over to a data team in order to create a program of perfectly personalized instruction for each student.

But many personalized learning developers are certain that data mining the academics is not enough. Social and emotional learning is another growth sector in education programming, and also, many folks have suggested that the young people are not automatically entranced by dull work just because it’s on a computer screen.

So we’re seeing attempts to mine other sorts of data. NWEA, the company that brought us the MAP test, now offers a feature that tells you whether or not the student taking the computer test is engaged or not. They believe that by analyzing the speed with which a student is answering questions, they can determine whether or not said student is trying. During test time, the teacher dashboard will toss up a little warning icon beside the name of any not-trying-hard-enough student so that the teacher can “redirect” the student.

That is more redundant than creepy; many teachers perform a similar analysis and intervention with a technique called “looking with their eyes.” But the personalization can get creepier.

There are several companies like LCA and its Nestor program. The program uses the students’ computer webcam to track and analyze facial expressions in order to determine if the instructional program is working. Monitoring programs like Nestor (there are several out there) claim they can read the student’s face for different emotional reactions the better to personalize the educational program being delivered. The beauty of these systems, of course, is that if we have students taking computerized courses that read their every response, we don’t really need teachers or school. Anywhere there is a computer and a webcam, school is in session and the program is collecting data about the students.

Does that seem excessive? Check out Cognition Builders, a company that offers to help you deal with your problem child by monitoring that child 24/7.

There are huge issues with all of these. From the educational standpoint, we have to question if anyone can really develop an algorithm or a necessarily massive library of materials that will actually work better than a trained human. From a privacy standpoint, the data collection is troubling. It’s concerning enough to create a system that allows employers to “search” for someone who is strong in math and moderately strong in written language based simply on algorithm-driven worksheet programs. It’s even more concerning when the program promises that it can also screen out future workers who are flagged as “Uncooperative” because of behavior patterns marked by a computer program in third grade.

And we still haven’t found the final frontier of creepitude.

Meet the field of educational genomics. The dream here is to use genetic information  to create “precision education,” which much like “precision medicine,” “precision agriculture” and “precision electioneering” would use huge levels of data down to the genetic level to design a perfect program.  The MIT Technology Review this spring profiled $50 DNA tests for IQ.

Imagine a future in which doctors perform a DNA test on an embryo and by the time that child is born, an entire personalized education program is laid out for her. The constant computer monitoring collects her performance and behavior data, so that by the time she’s ten years old, her digital record already makes a complete profile of her available, with an algorithm judging her on academic abilities as well as judging whether she’s a good person.

There are a thousand reasons to question whether or not we could do any of this well or accurately. But before we try to see if we can enter this impersonally personalized brave new world, we really need to talk about whether or not we should.

 

Advertisements

1 Comment

Filed under school reform policies, technology use

Personalized Learning and Personalized Medicine (Part 2)

No more bumbling Inspector Clouseau who I introduced in the previous post (for snippets from his films, see here, here, and here). For this post, I turn to another film character for inspiration: scientist Mr. Spock on the starship Enterprise. Logical and imperturbable–see here and here— I (but without the pointed ears) copy him by comparing and contrasting Personalized Learning (PL) and Personalized (or Precision) medicine (PM).

Similarities:

*History of individualizing treatment.

In medicine, currently, the mantra repeated in medical journals, conferences, and in hospital corridors is “patient-centered” care. Within the past half-century, the explosion of technology-driven diagnosis and treatment, rising costs, and growing dismay with patients being sent from one specialist to another has led to calls for clinicians to individualize their diagnosis and therapy to the varied needs of their patients.

…. In the quest to conquer disease, the fact that the patient is a person can often get overlooked. In the predominant U.S. healthcare model, people are often treated as a collection of diseases that episodically rear their ugly head and require drastic, increasingly expensive medical interventions. Practitioners of patient-centered medicine hope to change this, focusing on the overall well-being of the patient from day one with a combination of prevention, early detection and treatment that respects the patient’s goals, values and unique characteristics.

Counter to “doctor-centered,” the individualizing of diagnosis and treatment can be traced back to Hippocrates.  But it is only in the past half-century that calls for “patient-centered” practice have become front-and-center in the debate over how to deal with chronic diseases which afflict nearly half of all adult Americans.

As for schools, historical efforts to “personalize” teaching and learning have periodically occurred ranging from getting rid of the age-graded school to varied groupings of children during a lesson to teaching machines used in the 1920s and 1950s to the technology-driven “personalized learning” in the early 21st century (see here, here, and here)

*Reliance on technology to diagnose and treat differences among patients and students.

Hospital nurses have COWs–Computers on Wheels–that they bring to a patient’s room; doctors have scribes who take down what they say to patients. And teachers carry tablets with them as they traverse a classroom while students click away on their devices.

Technologies for diagnosing and treating patients’ new and chronic ailments and technologies that assess students’ learning strengths and limitations have become ubiquitous in doctors’ suites and classrooms.

*Over-promising and hype.

From miracle drugs to miracle software, both medical and school practitioners have experienced the surge of hope surrounding, say, a new treatment for Alzheimer’s disease or a quicker way to learn math.

Doctors will diagnose and treat diseases through mapping a person’s genome or by analyzing one drop of blood from a prick of the finger; childhood cancers will disappear (see here and here).

Claims that children using computers will have higher test scores and get high-paying jobs came with the earliest desktops in the 1980s. Promises that teachers will teach faster and better (see here and here) accompanied those devices then and since.

In a society where both business and government compete to provide private and public goods, where Americans are both consumers and citizens, the tension between making money and providing the best medicare care and education inexorably lead to over-promising and hyperbole.

Differences:

*In PM, analysis of patient’s DNA to find genetic disease markers (found in human genome) and then matching a specific, already tested drug matched to specific gene in patient’s genome that is connected to patient’s disease is common practice now.

In PL, no such intense and specified diagnosis of each student’s strengths and limitations currently exists. Nor are treatments for students–new curricula, new devices– tested clinically prior to use on individuals. Finally, the essential, overall knowledge and skills of a subject such as math, biology, U.S. history, or reading–analogous to the human genome–that can be targeted to the strengths and weaknesses of an individual child or youth is, in a word, absent (see here)

*In PM, individual patients do decide whether a new treatment for diabetes or atrial fibrillation should be administered.

In PL, however, adults decide on overall goals for students to reach. Both content and skills necessary to master come from state and district standards upon which students are tested to see if they have acquired both. In some settings such as problem-based instruction, students may decide what goals they want to achieve on a particular day in a particular lesson but not what they should learn overall–that is what district and state curriculum standards and tests determine.

*While in PM there is some research and clinical trials on specific therapies for particular diseases (e.g., breast and ovarian cancers), very little research (or clinical trials) for brand-name software content and skills exists currently. If anything, use of new math, reading, science, and social studies software in classrooms becomes a de facto clinical trial but without control groups.

***************************************

These are similarities and differences between PL and PM that I see. I am certain there are more than what I have listed. Readers can suggest others.

Like Inspector Clouseau I stumbled over the connection between PL and PM and, unlike the French detective, I now, inspired by Mr. Spock, have analyzed both similarities and differences in being applied to both students and patients. Thank you Peter Sellers and Leonard Nimoy!

 

 

Leave a comment

Filed under compare education and medicine, technology use

Donor Fails Policy 101 (Valerie Strauss)

Education reporter and editor for the “Answer Sheet” at the Washington Post, Valerie Strauss published this piece June 29, 2018. For other views on donors’ initiatives and mishaps, see here and here.

A major new report concludes that a $575 million project partly underwritten by the Gates Foundation that used student test scores to evaluate teachers failed to achieve its goals of improving student achievement — as in, it didn’t work.

Put this in the “they-were-warned-but-didn’t-listen” category.

The six-year project began in 2009 when the foundation gave millions of dollars to three public school districts — Hillsborough County in Florida (the first to start the work), Memphis and Pittsburgh. The districts supplied matching funds. Four charter management organizations also were involved: Alliance College-Ready Public Schools; Aspire Public Schools; Green Dot Public Schools; and Partnerships to Uplift Communities Schools.

The Bill & Melinda Gates Foundation pumped nearly $215 million into the project while the partnering school organizations supplied their own money, for a total cost of $575 million. The aim was to create teacher evaluation systems that depended on student standardized test scores and observations by “peer evaluators.” These systems, it was conjectured, could identify the teachers who were most effective in improving student academic performance.

This, in turn, would help school leaders staff classrooms with the most effective teachers and would lead more low-income minority students to have the best teachers — or so the thinking went. Schools also agreed to boost professional development for teachers, give bonuses to educators evaluated as effective and change their recruitment process.

The 526-page report titled “Improving Teacher Effectiveness: Final Report,” conducted by the Rand Corp. says:

Overall, the initiative did not achieve its stated goals for students, particularly LIM [low-income minority] students. By the end of 2014-2015, student outcomes were not dramatically better than outcomes in similar sites that did not participate in the IP [Intensive Partnerships] initiative. Furthermore, in the sites where these analyses could be conducted, we did not find improvement in the effectiveness of newly hired teachers relative to experienced teachers; we found very few instances of improvement in the effectiveness of the teaching force overall; we found no evidence that LIM students had greater access than non-LIM students to effective teaching; and we found no increase in the retention of effective teachers, although we did find declines in the retention of ineffective teachers in most sites.

Why didn’t it work? The report’s authors couldn’t say:

Unfortunately, the evaluation cannot identify the reasons the IP initiative did not achieve its student outcome goals by 2014-2015. It is possible that the reforms are working but we failed to detect their effects because insufficient time has passed for effects to appear. It is also possible that the other schools in the same states we use for comparison purposes adopted similar reforms, limiting our ability to detect effects. However, if the findings of no effect are valid, the results might reflect a lack of successful models on which sites could draw in implementing the levers, problems in making use of teacher-evaluation measures to inform key HR decisions, the influence of state and local context, or insufficient attention to factors other than teacher quality.

The project began at a time when the newly elected Obama administration was supporting school reforms that used student test scores to evaluate teachers, despite warnings from assessment experts of big problems with doing so. Gates and Arne Duncan, who was education secretary at the time, were on the same page, believing that test scores were valid measures for high-stakes decisions.

The Obama administration, through its Race to the Top initiative, dangled federal funds in front of states that agreed to establish teacher evaluation systems using test scores to varying extents. And Gates funded his “Empowering Effective Teachers” project with the aim of finding proof that such systems could improve student achievement.

Some assessment experts were concerned from the start that the methods used to link student test scores to teacher evaluations were largely unfair and lacked statistical validity. Some educators noted that there were already effective evaluation systems for teachers that did not give weight to student test scores, including in Maryland’s Montgomery County and Virginia’s Fairfax County.

But the Gates project and Race to the Top continued, and most states adopted test-based teacher evaluation systems. In a desperate attempt to evaluate all teachers on tested subjects — reading and math — some of the systems wound up evaluating teachers on subjects they didn’t teach or on students they didn’t have. Some major organizations publicly questioned them, including the American Statistical Association, the largest organization in the United States representing statisticians and related professionals. And so did the Board on Testing and Assessment of the National Research Council.

But the Gates project continued. What happened in Hillsborough County is illustrative of problems that many warned about early on. Teachers who initially supported it came to realize its weaknesses. The project required district and union leaders to work together, which happened — but not for long. In 2015, Hillsborough County gave up on it, after more than $180 million was spent there. This is what I wrote in a 2015 post:

Under the system, 40 percent of a teacher’s evaluation would be based on student standardized test scores and the rest by observation from “peer evaluators.” It turned out that costs to maintain the program unexpectedly rose, forcing the district to spend millions of dollars more than it expected to spend. Furthermore, initial support among teachers waned, with teachers saying that they don’t think it accurately evaluated their effectiveness and that they could be too easily fired.

Now the new superintendent of schools in Hillsborough, Jeff Eakins, said in a missive sent to the evaluators and mentors that he is moving to a different evaluation system, according to this article in the Tampa Bay Times. It says:

Unlike the complex system of evaluations and teacher encouragement that cost more than $100 million to develop and would have cost an estimated $52 million a year to sustain, Hillsborough will likely move to a structure that has the strongest teachers helping others at their schools.

Eakins said he envisions a new program featuring less judgmental “non-evaluative feedback” from colleagues and more “job-embedded professional development,” which is training undertaken in the classroom during the teacher work day rather than in special sessions requiring time away from school. He said in his letter that these elements were supported by “the latest research.”

From the start, critics had warned about using a standardized test designed for one purpose to evaluate something else — a practice frowned upon in the assessment world. The Rand report affirmed those concerns and said problems with using test scores as a metric were significant:

Teacher evaluation was at the core of the initiative, and the sites were committed to using the measures to inform key HR decisions. But, as we described in Chapters Three through Eight, the sites encountered two problems related to these intended uses of the TE measures. First, it was difficult for the sites to navigate the underlying tension between using evaluation information for professional improvement and using it for high-stakes decisions. Second, some sites encountered unexpected resistance when they tried to use effectiveness scores for high-stakes personnel decisions; this occurred despite the fact that the main stakeholder groups had given their support to the initiative in general terms at the outset.

The findings revive questions about whether the country is well-served when America’s wealthiest citizens choose pet projects and fund them so generously that public institutions, policy and money follow — even if those projects are not grounded in sound research. Such concerns have been raised most often about Gates, because he is the largest education philanthropist by far, and because he was a key player in Obama administration education reforms.

Gates, though, was pushing his own ideas for school reform before Obama became president, and he has since acknowledged that none of them turned out as well as he had hoped. In 2014, he gave a nearly hour-long interview at Harvard University, saying, “It would be great if our education stuff worked, but that we won’t know for probably a decade.”

In 2000, his foundation began investing in education reform with an expensive effort to turn big dropout high schools into smaller schools, which he abandoned, writing in hisfoundation’s 2009 annual letter that the results had been unimpressive. Instead, he said he would focus on teacher effectiveness and the dissemination of best teaching practices. He spent hundreds of millions of dollars to help create and implement the Common Core State Standards, which became highly controversial.

Now, Rand has declared his massive teacher effectiveness project to have fallen short of his goals. The Rand report does say that “the initiative did produce benefits, and the findings suggest some valuable lessons for districts and policymakers.” What lessons? Well, the report’s authors say some teachers reported learning how to improve from the observations. They also said the project had succeeded in helping schools “measure effectiveness” but not how to “increase it.” Of course, that is a loaded finding, given that there are many definitions of “effectiveness.”

Some school reformers are reluctant to say the project was a waste of time and money. They say the project taught us what doesn’t work. That ignores the fact that some education experts warned from the start that some of the premises on which it rested were not sound.

The bottom line: School reformers, led by Gates and supported by Duncan, felt the need to spend $575 million to prove their critics right.

6 Comments

Filed under how teachers teach, school reform policies

Asking Different Questions about Personalized Learning (Leigh McGuigan)

Leigh McGuigan worked in district leadership roles in New York, Chicago, and Cleveland, He is now CEO and co-founder of Vertus High School. Addressed to Rick Hess, head of educational policy at the American Enterprise Institute, this letter appeared May, 30, 2018.

Dear Rick,

I appreciated the recent blog posts from Larry Berger, Joel Rose, and Jonathan Skolnick on getting real about personalized learning. I loved their straight talk about the challenges of “engineering,” the need to rethink classrooms, and how to get students to “eat their vegetables.” But I wanted to raise a different issue based on our experience at Vertus High School, a blended high school for at-risk boys in upstate New York. Our students arrive at our door very far behind. Most do not know basic math, cannot recognize an adverb, and have never met an engineer. But when they graduate, most will go to college, some to the military or technical training, and a few to living-wage jobs.

We have four years to prepare our students for the world they will encounter. For our boys—like for most people—success after high school will mostly require that they do things someone else’s way, on someone else’s schedule. Much of this will be boring, and very little will be “personalized.” In most colleges, they will be expected to learn what their professor teaches, in the way he or she teaches it. In their jobs, their boss will likely dictate what they should do, and how and by when they should do it. Maybe a lucky few will go to colleges that nurture their individual interests and cater to their learning preferences, and to first jobs with lots of agency to pursue interesting questions as they see fit. But not many. We have a moral obligation to prepare them to succeed in the world they’re going to actually encounter.

Of late, it seems that talk of personalization focuses on the question, “What kind of personalization will make school engaging for students?” My experience leads me to think that’s the wrong question. And I worry that much of the thinking that results when it comes to personalization approaches fantasy—or educational malpractice.

I think the more useful question about personalized learning is, “How do we personalize learning for students while preparing them for what life will actually be like after high school—which, in truth, will be largely impersonal?” Some might wave this off as a misguided concern, but I think that’s a profound mistake and a disservice to our charges. As Vertus has grown over our first few years, this tension has been central to our work.

An undue focus on “engagement” personalization risks students not building the broad body of secure, automatic knowledge and skills they’ll need to succeed in college, and that they may not develop the self-control and grit to independently weather challenges, setbacks, and annoyances. Our students need a great deal of practice in that stuff which we might call “the basics.” We’ve found that we can’t let them just rely on their strengths or follow their preferences if we’re going to help them master those.

At Vertus, we do personalize, of course. Our students spend about half their time in learning labs completing online courses. We meet each student at their starting point, and each moves through courses at his own pace. In a self-paced environment, we learned early on that we had to provide strong incentives for making progress, as students who have not had success in school don’t have a compelling vision of the future to motivate them. We have learned the importance of giving our students explicit instruction and patient practice in how to concentrate and motivate themselves.

We also make it a point to incorporate plenty of traditional instruction. Students spend the other half of their time in typical small classrooms. The so-called “tired old model” of teaching a group of students the same thing in the same way is easy to dismiss, but it is still mainly what students will encounter after high school. In classrooms, students can learn to be part of respectful discussions and how to wait patiently while someone else’s needs are attended to. Since many of our students come to us with bad classroom habits, we’ve doubled down on fostering strong classroom cultures and student engagement. Our students use their classroom time to deepen skills in reading, writing, and math and to learn and practice the specific knowledge and skills that the New York State Regents tests require. Learning to succeed in a classroom and learning material that may not feel relevant or seem interesting are core skills in college.

Personalization done right can help cultivate self-control and self-motivation, the characteristics that students will need in the real world. But personalization done wrong risks graduating students who are ill-equipped to succeed in the real world, lack important knowledge and skills—and of doing all this because it’s trying to answer the wrong question. I hope we’re not experimenting on our students to satisfy our theologies, as they won’t get many second chances.

 

6 Comments

Filed under how teachers teach, school reform policies, technology use

Creating New Schools: Regression To the Mean (Part 2)

In 2003, Microsoft Corporation went into a partnership with the Philadelphia public schools to build and staff a brand-new high school called “The School of the Future” in the middle of a West Philadelphia low-income, African American neighborhood. Microsoft would supply the technological expertise and the district would staff and operate the school. The mission: prepare youth to go to college and enter the high-tech information-saturated workplace prepared to get entry-level jobs and launch careers.

In 2006, this shining new eco-sensitive, high-tech school, adjacent to a large park and the city’s zoo,  costing $62 million opened for 750 students. Students were chosen by lottery. The founders and district leaders were committed to educating students–called “learners”–to use software-laden laptops using a Microsoft developed portal rather than printed textbooks. A shining new media center, science labs galore, and especially equipped classrooms supported interdisciplinary projects and team-driven projects driven by students’ interests. The facility sparkled. As did the hopes and dreams of the teachers, “learners”, and parents.

MicrosoftSchool06.jpg

hsf3-420x359.pngphilly_school_of_the_future_bridges_the_gap_to_graduation.jpg

In 2012, School of the Future graduated its first class of 117 seniors–three years after it opened and every single one was admitted to college. But it was a rocky ride for these largely poor and African American graduates and subsequent classes.

Frequent changes in principals, unstable funding from district–the state had taken over the Philadelphia schools–mediocre academic achievement, and troubles with technologies–devices became obsolete within a few years–made the initial years most difficult in reaching the goals so admirably laid out in the prospectus for the school.

In 2018, the School of Future remains in operation but even with its surfeit of technology devices and software SOF has slowly become similar to traditional schools elsewhere in its district in its goals, policies, and practices (see here, here, and here).

As Richard Sherin, principal or “Chief Learner” since 2014 said:

At one point this school functioned very much through technology….Where our innovation is now is to get back to the fundamentals of what an educational academic program is supposed to be like, and how you get technology to mirror or augment that.

Part of those “fundamentals” is having a regular school day of seven 56-minute periods like most high schools with an 11-minute hiatus for what used to be called “home room.” Textbooks have returned as have paper and pencil. While project-based learning occurs in different academic subjects, state standards, yearly testing, and accountability have pressed both administration and faculty to focus on getting better-than-average test scores and graduating most of their students–SOF exceeds other district high schools in the percentage they graduate.

This slippage from grand opening of a futuristic school to one resembling a traditional high school is common in public schooling as it is in other institutions.

Why is there this slow movement back from a school built for the future  to the traditional model of schooling as seen in New York’s Downtown School (Part 1) and here in Philadelphia’s School of Future?

I have one but surely not the only answer. Designers of future schools and innovations overestimate the potency of their vision and product and underestimate the power of the age-graded school’s structure and culture (fully supported by societal beliefs) that sustain traditional models of schooling. That see-saw of underestimation vs. overestimation neatly summarizes the frequent cycles of designers’ exhilaration with a reform slowly curdling into disappointment as years pass.

The overestimation of a design to alter the familiar traditional school has occurred time and again when reformers with full wallets, seeing how out of touch educators were as changes in society accelerated, created new schools chock-a-block across the country in the 1960s such as “free schools” and non-graded schools  (see here and here).

Within a decade, founders of these schools of the future had departed, either  burned out or because they had ignored politically the two constituencies of parents and teachers who had to be involved from the start but were not. These well intentioned reformers also ignored how the structures and culture of the age-graded school have been thoroughly accepted by most parents and teachers as “real schools.”

Designers of reform seldom think about the inherent stability of the institution that they want to transform. They seldom think about the strong social beliefs of taxpayers, voters, teachers, and parents who have sat in age-graded schools and who sustain generation after generation the “grammar of schooling.”

From daily schedules of 50-minute periods to the fact that teachers ask questions far more than students during lessons to the use of textbooks, homework, and frequent tests–these features of the “grammar of schooling” or what Seymour Sarason in The Culture of the School and the Problem of Change,  called the “regularities” of schooling–persist generation after generation. While they exaggerate the reform they champion, they neglect  the influences of organizational structures and cultures.

Some designers give up. They realize that their grand visions cannot be accommodated by public schools quickly so they create schools of the future in private venues such as “micro-schools” or  the Khan Lab School and the like.

The notion of mindful incremental change over a lengthy period of time in the direction of gradually building a “school of the future” is anathema to fired-up, amply funded designers who see their visions enacted in one fell swoop. Thus, disappointment arises when futuristic schools slip back into routines that designers scorned. Regression to the mean smells like failure to these reformers who underestimated the power of the “grammar of schooling.”

 

 

 

 

 

 

17 Comments

Filed under Reforming schools, technology use

Creating New Schools: Regression To the Mean (Part 1)

Historians have a unfortunate reputation for being wet blankets. Reformers propose a new idea or a program aimed at transforming what children or youth do in schools and some historian will say: “Hey, that ain’t new. It was tried in the 1890s and lasted less than a decade.”

The historian is nearly always accurate in the facts that similar or even identical innovations did occur before but those very same historians too often forget to add that the contexts then are not the same contexts now. Times change. Teaching machines in the 1950s, for example, are surely similar to widespread “personalized learning” in the present decade in seeking individualized learning but the 1950s and 2010s were politically, economically, technologically, and socially very different.

Historians, then, can see the similarities in innovations but must note the differences in how the innovation began and played out over time in two different contexts. In doing so, such historians can inform current school reformers on the policy strengths, defects, and outcomes–both anticipated and unanticipated–in previous efforts suggesting where there are potholes and bends in the road that have to be noted and avoided by contemporary policymakers. While there are no “lessons” or an easy “usable past” that historians can tell policymakers, historians can point out similarities and differences that can help decision-makers, practitioners, and parents in current policy debates and actions.

There is also another reason for historians to draw upon the past to inform decision-makers about consequential policies; those innovators who come up with an idea and put it into practice already have a view of the past and they act on it. They already have views and identities shaped by history. Those views and “facts” may be uniformed, naked of accurate information of what happened in earlier years but it is, nonetheless, a view of the past that entrepreneurs and policymakers–who are eager to create schools that will best prepare the young for an uncertain future–hold.

Essayist and novelist James Baldwin said it all in 1965.

History, as nearly no one seems to know, is not merely something to be read.  And it does not refer merely, or even principally, to the past.  On the contrary, the great force of history comes from the fact that we carry it within us, are unconsciously controlled by it in many ways, and history is literally present in all that we do.  It could scarcely be otherwise, since it is to history that we owe our frames of reference, our identities, and our aspirations.

As always, historian David Tyack put it crisply: Policymakers do not have a choice about whether to use history. They do it willy-nilly. The question is: How accurate is their historical map?

And the historical maps used by entrepreneurial innovators inspired to transform traditional schooling into futuristic venues–“learning spaces”–that better prepare students for an information-saturated world where yesterday’s careers are obsolete and today’s jobs disappear each year bear little resemblance to what happened before.  These techno-utopians believe that, while the task will be difficult and complicated, they can succeed where previous efforts failed because, well, they are smarter, know exactly what to do and how to do it, have more technological tools, and pocketfuls of cash. In short, they are arrogant–they know better than those who do the work daily in schools and ignorant of past similar efforts where just as smart, well-intentioned reformers put into practice innovations a generation ago.

All of this occurred to me as I finished reading Disruptive Fixation: School Reform and the Pitfalls of Techno-Idealism an ethnography about a New York City public middle school that opened in 2009. Amply funded by exceedingly idealistic and optimistic technology entrepreneurs, students would create gaming software, work on high-tech projects in teams, and learn in spaces similar to start-up companies. This would be a school where coding and digital media production practices  across the curriculum became routine, where pedagogy was redesigned to be game-like, and where the school would “cultivate student agency, creativity [and] improvisational problem-solving capacities” (p.98). In short, a media technology, student-centered school of the future.

Christo Sims who was there as a researcher when the public school opened with a sixth grade class spent three years at Downtown School–a pseudonym–and described the thinking that produced the school, its policies, and practices.

Things didn’t work out the way the designers intended, however.

Consider how school-made rules for controlling student movement and reducing noise appeared. Sims asks reader to consider how school leaders and teachers broke down  classroom lessons into step-by-step procedures and set activities. Sims notes that in some classrooms rows of desks facing the front of the room replaced tables and chairs arranged in circles with students facing one another. He documents that as lessons ended, teachers organized students into “quiet, forward-facing, single-file lines before they left the classroom” and then “teachers  marched students down the hallway to their next class” (p.97). Furthermore, teachers and students became time-minded, both having a sharp awareness of completing an activity in a given amount of time. This, according to Sims, this student-centered ideal school turned into practices eerily similar to a traditional school.

One part of the school year did come close to the aspirations of the school designers. Called “Level Up,” a week-long period, three times a year, when the school completely altered their daily schedules, classroom lessons, and interactions. School leaders issued a challenge to teams of students to work on. The first “Level Up” week students were challenged to build a Rube Goldberg machine out of common materials (popsicle sticks, paper clips, rubber bands, plastic bags,etc.) that parents and teachers had provided. Another week-long session had students writing and producing short plays based on fairy tales that they had “remixed” from music, videos, photos, and art.

These interludes during the school year were moments when the school designers’ rhetoric of student agency, participation, and involvement matched what occurred in the school. Students chose which kind of machine or “remixed” fairy tale to create, worked on it together and turned in a product that they exhibited to the rest of the school. But these interludes were three weeks out of a 36-week school year.

After close observation and participation in the school for three years, Sims concludes that: “While the reformers championed student agency and creativity, students had very little say about what they could do, and most of what they were supposed to do was quite similar to the very schooling practices that reformers criticized and aimed to replace” (p.94).

The Downtown School continues to operate in 2018. And so does the historical paradox of creating schools for the future that end up resembling present-day schools. A well-funded redesigned school where well-intentioned, optimistic reformers reject the traditional model of teaching and learning only to slide inexorably into the kind of schooling similar to what they found lacking is not rare but common in the history of public schooling.

Smart, well-funded idealists thought (and continue to think) that creating a brand-new school with a novel curriculum and state-of-the-art technologies would be free and clear of traditional space, schedule, parents’ social beliefs about what a “real” school is, and the inherent asymetrical power relationships between teachers and students that have marked tax-supported public schools for at least two centuries. The Downtown School that Christo Sims describes may well be an instance of “regression to the mean,” a statistical phrase all to common in the performance of organizations and individuals. That movement to the middle of a continuum is what Part 2 explores.

 

5 Comments

Filed under school reform policies

No, Educators and Policymakers Shouldn’t Just ‘Do What the Research Shows’ (Rick Hess)

…. I routinely advise policymakers and practitioners to be real nervous when an academic or expert encourages them to do “what the research shows.” As I observed in Letters to a Young Education Reformer, 20th-century researchers reported that head size was a good measure of intelligence, girls were incapable of doing advanced math, and retardation was rampant among certain ethnic groups. Now, I know what you’re thinking: “That wasn’t real research!” Well, it was conducted by university professors, published in scholarly journals, and discussed in textbooks. Other than the fact that the findings now seem wacky, that sure sounds like real research to me.

Medical researchers, for instance, change their minds on important findings with distressing regularity. Even with their deep pockets and fancy lab equipment, they’ve gone back and forth on things like the dangers of cholesterol, the virtues of flossing, whether babies should sleep on their backs, how much exercise we should get, and the effects of alcohol. Things would be messy if lawmakers or insurers were expected to change policies in response to every new medical study.

In truth, science is frequently a lot less absolute than we imagine. In 2015, an attempt to replicate 97 studies with statistically significant results found that more than one-third couldn’t be duplicated. More than 90 percent of psychology researchers admit to at least one behavior that might compromise their research, such as stopping data collection early because they liked the results as they were, or not disclosing all of a study’s conditions. And more than 40 percent admit to having sometimes decided whether to exclude data based on what it did to the results.

Rigorous research eventually influences policy and practice, but it’s typically after a long and gradual accumulation of evidence. Perhaps the most famous example is with the health effects of tobacco, where a cumulative body of research ultimately swayed the public and shaped policy on smoking—in spite of tobacco companies’ frenzied, richly funded efforts. The consensus that emerged involved dozens of studies by hundreds of researchers, with consistent findings piling up over decades.

When experts assert that something “works,” that kind of accumulated evidence is hardly ever what they have in mind. Rather, their claims are usually based on a handful of recent studies—or even a single analysis—conducted by a small coterie of researchers. (In education, those researchers are not infrequently also advocates for the programs or policies they’re evaluating.) When someone claims they can prove that extended learning time, school turnarounds, pre-K, or teacher residencies “work,” what they usually mean is that they can point to a couple studies that show some benefits from carefully executed pilot programs.

The upshot: When pilots suggest that policies or programs “work,” it can mean a lot less than reformers might like. Why might that be?

Think about it this way. The “gold standard” for research in medicine and social science is a randomized control trial (RCT). In an RCT, half the participants are randomly selected to receive the treatment—let’s say a drug for high blood pressure. Both the treatment and control groups follow the same diet and health-care plan. The one wrinkle is that the treatment group also receives the new drug. Because the drug is the only difference in care between the two groups, it can be safely credited with any significant difference in outcomes.

RCTs specify the precise treatment, who gets it, and how it is administered. This makes it relatively easy to replicate results. If patients in a successful RCT got a 100-milligram dosage of our blood pressure drug every twelve hours, that’s how doctors should administer it in order obtain the same results. If doctors gave out twice the recommended dosage, or if patients got it half as often as recommended, you wouldn’t expect the same results. When we say that the drug “works,” we mean that it has specific, predictable effects when used precisely.

At times, that kind of research can translate pretty cleanly to educational practice. If precise, step-by-step interventions are found to build phonemic awareness or accelerate second-language mastery, replication can be straightforward. For such interventions, research really can demonstrate “what works.” And we should pay close attention.

But this also helps illuminate the limits of research when it comes to policy, given all the complexities and moving parts involved in system change. New policies governing things like class size, pre-K, or teacher pay get adopted and implemented by states and systems in lots of different ways. New initiatives are rarely precise imitations of promising pilots, even on those occasions when it’s clear precisely what the initial intervention, dosage, design, and conditions were.

If imitators are imprecise and inconsistent, there’s no reason to expect that results will be consistent. Consider class-size reduction. For decades, advocates of smaller class sizes have pointed to findings from the Student Teacher Achievement Ratio (STAR) project, an experiment conducted in Tennessee in the late 1980s. Researchers found significant achievement gains for students in very small kindergarten and first-grade classes. Swayed by the results, California legislators adopted a massive class-size reduction program that cost billions in its first decade. But the evaluation ultimately found no impact on student achievement.

What happened? Well, what “worked” on a limited scale in Tennessee played out very differently when adopted statewide in California. The “replication” didn’t actually replicate much beyond the notion of “smaller classes.” Where STAR’s small classes were 13 to 17 students, California’s small classes were substantially larger. STAR was a pilot program in a few hundred classrooms, minimizing the need for new teachers, while California’s statewide adoption required a tidal wave of new hires. In California, districts were forced to hire thousands of teachers who previously wouldn’t have made the cut, while schools cannibalized art rooms and libraries in order to find enough classrooms to house them. Children who would have had better teachers in slightly larger classrooms were now in slightly smaller classrooms with worse teachers. It’s no great shock that the results disappointed.

Research should inform education policy and practice, but it shouldn’t dictate it. Common sense, practical experience, personal relationships, and old-fashioned wisdom have a crucial role to play in determining when and how research can be usefully applied. The researchers who play the most constructive roles are those who understand and embrace that messy truth.

2 Comments

Filed under research, school reform policies