Category Archives: research

No, Educators and Policymakers Shouldn’t Just ‘Do What the Research Shows’ (Rick Hess)

…. I routinely advise policymakers and practitioners to be real nervous when an academic or expert encourages them to do “what the research shows.” As I observed in Letters to a Young Education Reformer, 20th-century researchers reported that head size was a good measure of intelligence, girls were incapable of doing advanced math, and retardation was rampant among certain ethnic groups. Now, I know what you’re thinking: “That wasn’t real research!” Well, it was conducted by university professors, published in scholarly journals, and discussed in textbooks. Other than the fact that the findings now seem wacky, that sure sounds like real research to me.

Medical researchers, for instance, change their minds on important findings with distressing regularity. Even with their deep pockets and fancy lab equipment, they’ve gone back and forth on things like the dangers of cholesterol, the virtues of flossing, whether babies should sleep on their backs, how much exercise we should get, and the effects of alcohol. Things would be messy if lawmakers or insurers were expected to change policies in response to every new medical study.

In truth, science is frequently a lot less absolute than we imagine. In 2015, an attempt to replicate 97 studies with statistically significant results found that more than one-third couldn’t be duplicated. More than 90 percent of psychology researchers admit to at least one behavior that might compromise their research, such as stopping data collection early because they liked the results as they were, or not disclosing all of a study’s conditions. And more than 40 percent admit to having sometimes decided whether to exclude data based on what it did to the results.

Rigorous research eventually influences policy and practice, but it’s typically after a long and gradual accumulation of evidence. Perhaps the most famous example is with the health effects of tobacco, where a cumulative body of research ultimately swayed the public and shaped policy on smoking—in spite of tobacco companies’ frenzied, richly funded efforts. The consensus that emerged involved dozens of studies by hundreds of researchers, with consistent findings piling up over decades.

When experts assert that something “works,” that kind of accumulated evidence is hardly ever what they have in mind. Rather, their claims are usually based on a handful of recent studies—or even a single analysis—conducted by a small coterie of researchers. (In education, those researchers are not infrequently also advocates for the programs or policies they’re evaluating.) When someone claims they can prove that extended learning time, school turnarounds, pre-K, or teacher residencies “work,” what they usually mean is that they can point to a couple studies that show some benefits from carefully executed pilot programs.

The upshot: When pilots suggest that policies or programs “work,” it can mean a lot less than reformers might like. Why might that be?

Think about it this way. The “gold standard” for research in medicine and social science is a randomized control trial (RCT). In an RCT, half the participants are randomly selected to receive the treatment—let’s say a drug for high blood pressure. Both the treatment and control groups follow the same diet and health-care plan. The one wrinkle is that the treatment group also receives the new drug. Because the drug is the only difference in care between the two groups, it can be safely credited with any significant difference in outcomes.

RCTs specify the precise treatment, who gets it, and how it is administered. This makes it relatively easy to replicate results. If patients in a successful RCT got a 100-milligram dosage of our blood pressure drug every twelve hours, that’s how doctors should administer it in order obtain the same results. If doctors gave out twice the recommended dosage, or if patients got it half as often as recommended, you wouldn’t expect the same results. When we say that the drug “works,” we mean that it has specific, predictable effects when used precisely.

At times, that kind of research can translate pretty cleanly to educational practice. If precise, step-by-step interventions are found to build phonemic awareness or accelerate second-language mastery, replication can be straightforward. For such interventions, research really can demonstrate “what works.” And we should pay close attention.

But this also helps illuminate the limits of research when it comes to policy, given all the complexities and moving parts involved in system change. New policies governing things like class size, pre-K, or teacher pay get adopted and implemented by states and systems in lots of different ways. New initiatives are rarely precise imitations of promising pilots, even on those occasions when it’s clear precisely what the initial intervention, dosage, design, and conditions were.

If imitators are imprecise and inconsistent, there’s no reason to expect that results will be consistent. Consider class-size reduction. For decades, advocates of smaller class sizes have pointed to findings from the Student Teacher Achievement Ratio (STAR) project, an experiment conducted in Tennessee in the late 1980s. Researchers found significant achievement gains for students in very small kindergarten and first-grade classes. Swayed by the results, California legislators adopted a massive class-size reduction program that cost billions in its first decade. But the evaluation ultimately found no impact on student achievement.

What happened? Well, what “worked” on a limited scale in Tennessee played out very differently when adopted statewide in California. The “replication” didn’t actually replicate much beyond the notion of “smaller classes.” Where STAR’s small classes were 13 to 17 students, California’s small classes were substantially larger. STAR was a pilot program in a few hundred classrooms, minimizing the need for new teachers, while California’s statewide adoption required a tidal wave of new hires. In California, districts were forced to hire thousands of teachers who previously wouldn’t have made the cut, while schools cannibalized art rooms and libraries in order to find enough classrooms to house them. Children who would have had better teachers in slightly larger classrooms were now in slightly smaller classrooms with worse teachers. It’s no great shock that the results disappointed.

Research should inform education policy and practice, but it shouldn’t dictate it. Common sense, practical experience, personal relationships, and old-fashioned wisdom have a crucial role to play in determining when and how research can be usefully applied. The researchers who play the most constructive roles are those who understand and embrace that messy truth.

Advertisements

2 Comments

Filed under research, school reform policies

Assessing My Writing: A Look Backward

A month ago, a colleague wrote to me and asked me to write about my career as a practitioner/scholar over the past half-century. I accepted. Part of the request was to include what I have written about policy and practice as a historian of education that contributed to both research and practice.

Sure, there are metrics that suggest what a “contribution” may be. There are Google scholar and Edu-Scholar rankings. There are Web of Science citations. All well and good but influence or impact on practitioners and researchers? Maybe yes, a bit here and there. And maybe no, not a trace. Rankings and citations are, at best, no more than fragile, even shaky proxies of a “contribution.”

I thought about these metrics a lot and decided instead to describe those works that gave me the most satisfaction in writing. This is not false modesty. What I think may be a contribution, others may yawn at its banality. What I think is a mundane article,  I will receive notes from readers about how powerful the piece was in altering their thinking. Writing to me is a form of teaching: some lessons fly and others flop.

So what follows is my self-assessment of those writings that gave me the most satisfaction and feeling of pride in doing something worthwhile.  Others would have to judge whether what I have written over the past half-century has contributed to what practitioners, policymakers, researchers, and the general public—audiences I have written for—know and do. In all instances, what I offer are publications that were prompted by questions that grew out of my teaching and administrative experience and what I learned as a researcher. Both have played a huge part in what I chose to research, write, and teach.

How Teachers Taught (1984, 1993)

This study of three different generations of reformers trying to alter the dominant way of classroom teaching (1900s, 1960s, and 1980s) was my first historical analysis of teaching. The question that prompted the study came out of my visits to Arlington (VA) public school classrooms over the seven years I served as superintendent in the 1970s and early 1980s. I kept seeing classroom lessons that reminded me of how I was taught in elementary and secondary schools in Pittsburgh (PA) in the 1940s. And how I taught in Cleveland (OH) in the 1950s. How could that be, I asked myself? That question led to a three-year grant to study how teachers taught between 1880-1990.

I used district archives, photographs, and first-hand accounts to cover a century of policy efforts to shift teaching from teacher-centered to student-centered instruction. I documented the century-long growth of classroom hybrids of both kinds of classroom instruction. Few historians, sadly, have since pursued the question of how reform policies aimed at altering teachers’ classroom behavior actually get put into practice.

 The Managerial Imperative and the Practice of Leadership (1988)

Here again, a question that grew out of my being in classrooms as a teacher and a district administrator nudged me. What I saw and experienced in classrooms and administrative offices looked a great deal alike insofar as the core roles that both teachers and administrators had to perform. Was that accurate and if so, how did that come to be? So I investigated the history of teaching, principaling and superintending. I saw that three core roles dominated each position: instructional, managerial, and political. I compared and contrasted each with vivid examples and included chapters on my experiences as both a teacher and administrator.

Reform Again, Again, and Again (1990)

The article that appeared in Educational Researcher looked at various cycles of change that I had documented in How Teachers Taught and The Managerial Imperative. The central question that puzzled me was how come school reform in instruction, curriculum, governance and organization recurred time and again. I was now old enough and had experienced these reform cycles.

I presented a conceptual framework that explained the recurring reforms. My prior studies and direct school experiences gave me rich examples to illustrate the framework.

Tinkering toward Utopia (1995)

David Tyack and I collaborated in writing this volume. We drew heavily from the “History of School Reform” course we had been teaching to graduate students and each of our prior studies. In only 142 pages (endnotes and bibliography excluded), we summed up our thinking about the rhetoric and actuality of school reform policies in curriculum, school organization, governance, and instruction over the past two centuries in the U.S.

Oversold and Underused: Computers in the Classroom (2001)

In 1986, Teachers and Machines: The Classroom use of Technology since 1920 was published. In that study, I looked at teacher access and use of film and radio in classrooms during the 1920s and 1930s, educational television in the 1950s and 1960s and the first generation of desktop computers in the early 1980s. The central question driving that study was: what did teachers do in their lessons when they had access to film, radio, television, and later computers?

The question derives from the larger interest I have had in school reform policies and the journey they take as they wend their way into classroom practice. Like new curricula, governance changes, and shifts in how best to organize schools, grasping at new technologies that promise deep changes in how teachers teach is simply another instance of school reformers using policy mandates to alter classroom instruction. In short, adopting new technologies is simply another thread in the recurring pattern of school reformers seeking classroom changes during the 20th and 21st centuries.

Fifteen years after Teachers and Technology appeared, computers had become common in schools. So in Oversold and Underused, I asked: to what degree were teachers in Silicon Valley schools using computers in their classrooms, labs, and media centers for lessons they taught? Such questions about classroom use go beyond the rhetoric surrounding new devices and software. I wanted to see what actually occurred in classrooms when districts adopted policies pushing new technologies into pre-school, high school, and university classrooms.

Teaching History Then and Now (2016)

The question that prompted this study came out of writing for my blog on how I taught history in two urban high schools in the 1950s and 1960s. I wondered how history was taught in those very same high schools a half-century later. Those personal questions led to reconstructing my teaching a half-century ago from personal records and archives I found at each school and then traveling to those very same schools to do observations and interviews with current teachers of history.

***********************************************

These publications have given me great satisfaction in writing. Converting questions and ideas into words on a screen or jottings on a piece of paper is what I have done since I published my first article in 1960 in the Negro History Bulletin. Have I written things that have never left my home and remain in closets and bottom drawers? You bet. But writing, a different way of teaching, remains important to me and as I long as I can I will write about the past as it influences the present, especially policies that aim at at altering how teachers teach.

Yet the act of writing historically remains mysterious to me. Why do the words flow easily and excite me in their capturing illusive ideas and rendering them in a graceful way and other times what I see on paper or on the screen are clunky sentences, if not clumsy wording? I do not know. Immersed in writing about policy and practice historically (as it has been for me in teaching in high school and graduate seminars) has given me highs and lows over the years and much satisfaction. While I may not understand the mystery of writing, I remain most grateful to Clio, the muse of historians.

 

Leave a comment

Filed under Reforming schools, research

How To Get Your Mind To Read (Daniel Willingham)

 

“Daniel T. Willingham (@DTWillingham) is a professor of psychology at the University of Virginia and the author, most recently, of ‘The Reading Mind: A Cognitive Approach to Understanding How the Mind Reads.'”

This post appeared as an op-ed in the New York Times November 25, 2017.

 

Americans are not good readers. Many blame the ubiquity of digital media. We’re too busy on Snapchat to read, or perhaps internet skimming has made us incapable of reading serious prose. But Americans’ trouble with reading predates digital technologies. The problem is not bad reading habits engendered by smartphones, but bad education habits engendered by a misunderstanding of how the mind reads.

Just how bad is our reading problem? The last National Assessment of Adult Literacy from 2003 is a bit dated, but it offers a picture of Americans’ ability to read in everyday situations: using an almanac to find a particular fact, for example, or explaining the meaning of a metaphor used in a story. Of those who finished high school but did not continue their education, 13 percent could not perform simple tasks like these. When things got more complex — in comparing two newspaper editorials with different interpretations of scientific evidence or examining a table to evaluate credit card offers — 95 percent failed.

There’s no reason to think things have gotten better. Scores for high school seniors on the National Assessment of Education Progress reading test haven’t improved in 30 years.

Many of these poor readers can sound out words from print, so in that sense, they can read. Yet they are functionally illiterate — they comprehend very little of what they can sound out. So what does comprehension require? Broad vocabulary, obviously. Equally important, but more subtle, is the role played by factual knowledge.

All prose has factual gaps that must be filled by the reader. Consider “I promised not to play with it, but Mom still wouldn’t let me bring my Rubik’s Cube to the library.” The author has omitted three facts vital to comprehension: you must be quiet in a library; Rubik’s Cubes make noise; kids don’t resist tempting toys very well. If you don’t know these facts, you might understand the literal meaning of the sentence, but you’ll miss why Mom forbade the toy in the library.

Knowledge also provides context. For example, the literal meaning of last year’s celebrated fake-news headline, “Pope Francis Shocks World, Endorses Donald Trump for President,” is unambiguous — no gap-filling is needed. But the sentence carries a different implication if you know anything about the public (and private) positions of the men involved, or you’re aware that no pope has ever endorsed a presidential candidate.

You might think, then, that authors should include all the information needed to understand what they write. Just tell us that libraries are quiet. But those details would make prose long and tedious for readers who already know the information. “Write for your audience” means, in part, gambling on what they know.

These examples help us understand why readers might decode well but score poorly on a test; they lack the knowledge the writer assumed in the audience. But if a text concerned a familiar topic, habitually poor readers ought to read like good readers.

In one experiment, third graders — some identified by a reading test as good readers, some as poor — were asked to read a passage about soccer. The poor readers who knew a lot about soccer were three times as likely to make accurate inferences about the passage as the good readers who didn’t know much about the game.

That implies that students who score well on reading tests are those with broad knowledge; they usually know at least a little about the topics of the passages on the test. One experiment tested 11th graders’ general knowledge with questions from science (“pneumonia affects which part of the body?”), history (“which American president resigned because of the Watergate scandal?”), as well as the arts, civics, geography, athletics and literature. Scores on this general knowledge test were highly associated with reading test scores.

Current education practices show that reading comprehension is misunderstood. It’s treated like a general skill that can be applied with equal success to all texts. Rather, comprehension is intimately intertwined with knowledge. That suggests three significant changes in schooling.

First, it points to decreasing the time spent on literacy instruction in early grades. Third-graders spend 56 percent of their time on literacy activities but 6 percent each on science and social studies. This disproportionate emphasis on literacy backfires in later grades, when children’s lack of subject matter knowledge impedes comprehension. Another positive step would be to use high-information texts in early elementary grades. Historically, they have been light in content.

Second, understanding the importance of knowledge to reading ought to make us think differently about year-end standardized tests. If a child has studied New Zealand, she ought to be good at reading and thinking about passages on New Zealand. Why test her reading with a

passage about spiders, or the Titanic? If topics are random, the test weights knowledge learned outside the classroom — knowledge that wealthy children have greater opportunity to pick up.

Third, the systematic building of knowledge must be a priority in curriculum design. The Common Core Standards for reading specify nearly nothing by way of content that children are supposed to know — the document valorizes reading skills. State officials should go beyond the Common Core Standards by writing content-rich grade-level standards and supporting district personnel in writing curriculums to help students meet the standards. That’s what Massachusetts did in the 1990s to become the nation’s education leader. Louisiana has recently taken this approach, and early results are encouraging.

Don’t blame the internet, or smartphones, or fake news for Americans’ poor reading. Blame ignorance. Turning the tide will require profound changes in how reading is taught, in standardized testing and in school curriculums. Underlying all these changes must be a better understanding of how the mind comprehends what it reads.

 

12 Comments

Filed under research, school reform policies

Research Counts for Little When It Comes to Adopting “Personalized Learning”

The K-12 sector is investing heavily in technology as a means of providing students with a more customized educational experience. So far, though, the research evidence behind “personalized learning” remains thin.

Ben Herold, Education Week, October 18, 2016

The pushers of computer-based instruction want districts to buy products and then see if the product works. Students and teachers are being used for marketing research, unreimbursed research. Districts are spending money based on hype and tests of the educational efficacy of an extremely narrow range of products as if this is a reasonable way to proceed in this era of extreme cuts in budgets.

Laura Chapman, comment on above guest post, May 21, 2017

Both Ben Herold and Laura Chapman are correct in their statements about the thinness of research on “personalized learning” and that districts spend “money based on hype and tests of the educational efficacy of an extremely narrow range of products….”

In short, independent studies of “personalized learning,” however, defined, are rare birds but of even greater importance, are subordinate to decisions on buying and deploying software and programs promising to tailor learning to each and every student from kindergarten through high school. To provide a fig leaf of cover for spending on new technologies, policymakers often use vendor-endorsed studies and quick-and-dirty product evaluations. They are the stand-ins for “what the research says” when it comes to purchasing new products advertising a platform for “personalized learning.”

Why is research nearly irrelevant to such decisions? Because other major criteria come into play that push aside educational research either independent or vendor-sponsored, on technology. Policymakers lean far more heavily upon criteria of effectiveness, popularity, and longevity in spending scarce dollars on new technologies championing “personalized learning.”

Criteria policymakers use 

The dominant standard used by most policymakers, media editors, and administrators to judge success is effectiveness: What is the evidence that the policy of using new technologies for classroom instruction has produced desired outcomes? Have you done what you said you were going to do and can you prove it? In a society where “bottom lines,” Dow Jones averages, Super Bowl victories, and vote-counts matter, quantifiable results determine effectiveness.

Since the Elementary and Secondary Education Act (1965), federal and state policymakers have relied on the effectiveness standard to examine what students have learned by using proxy measures such as test scores, high school graduation rates, college attendance, and other indicators. For example, in the late-1970s policymakers concluded that public schools had declined because scholastic aptitudes test (SAT) scores had plunged downward. Even though test-makers and researchers repeatedly stated that such claims were false—falling SAT scores fueled public support for states raising academic requirements in the 1980s and adding standardized tests to determine success. With the No Child Left Behind Act (2001-2016) test scores brought rewards and penalties. [i]

Yet test results in some instances proved unhelpful in measuring a reform’s success. For example, studies on computer use in classroom instruction show no substantial gains in students’ test scores. Yet buying more and more tablets and laptops with software programs has leaped forward in the past decade.

Or Consider the mid-1960s’ evaluations of Title I of the Elementary and Secondary Education Act (ESEA). They revealed little improvement in low-income children’s academic performance thereby jeopardizing Congressional renewal of the program. Such evidence gave critics hostile to federal initiatives reasons to brand President Lyndon Johnson’s War on Poverty programs as failures. [ii]

Nonetheless, the program’s political attractiveness to constituents and legislators overcame weak test scores. Each successive U.S. president and Congress, Republican or Democrat, have used that popularity as a basis for allocating funds to needy students in schools across the nation including No Child Left Behind (2001) and its successor, Every Student Succeeds Act (2016). Thus, a reform’s political popularity often leads to its longevity (e.g., kindergarten, new technologies in classrooms).

Popularity, then, is a second standard that public officials use in evaluating success. The spread of an innovation and its hold on voters’ imagination and wallets has meant that attractiveness to parents, communities, and legislators easily translates into political support for reform. Without the political support of parents and teachers, few technological innovations such as “personalized learning” could fly long distances.

The rapid diffusion of kindergarten and preschool, special education, bilingual education, testing for accountability, charter schools, and electronic technologies in schools are instances of innovations that captured the attention of practitioners, parents, communities, and taxpayers. Few educators or public officials questioned large and sustained outlays of public funds for these popular reforms because they were perceived as resounding successes regardless of the research. And they have lasted for decades. Popularity-induced longevity becomes a proxy for effectiveness. [iii]

A third standard used to judge success is assessing how well innovations mirrored what designers of reforms intended. This fidelity standard assesses the fit between the initial design, the formal policy, the subsequent program, and its implementation.

Champions of the fidelity standard ask: How can anyone determine effectiveness if the reform departs from the design? If federal, state, or district policymakers, for example, adopt and fund a new reading program because it has proved to be effective elsewhere, teachers and principals must follow the blueprint as they put it into practice or else the desired outcomes will go unfulfilled (e.g., Success for All). When practitioners add, adapt, or even omit features of the original design, then those in favor of fidelity say that the policy and program cannot be determined effective because of these changes. Policy adaptability is the enemy of fidelity. [iv]

Seldom are these criteria debated publicly, much less questioned. Unexamined acceptance of effectiveness, fidelity, and popularity avoids asking the questions of whose standards will be used, how they are applied and alternative standards that can be used to judge reform success and failure.

Although policymakers, researchers, practitioners have vied for attention in judging the success of school reforms such as using new technologies in classroom instruction, policy elites, including civic and business leaders and their accompanying foundation- and corporate-supported donors have dominated the game of judging reform success.

Sometimes  called a “growth coalition,” these civic, business, and philanthropic leaders see districts and schools as goal-driven organizations with top officials exerting top-down authority through structures. They juggle highly prized values of equity, efficiency, excellence, and getting reelected or appointed. They are also especially sensitive to public expectations for school accountability and test scores; they also reflect societal optimism that technologies can solve individual and community problems. Hence, these policy making elites favor standards of effectiveness, fidelity, and popularity—even when they conflict with one another. Because the world they inhabit is one of running organizations, their authority and access to the media give them the leverage to spread their views about what constitutes “success.” [v]

The world that policy elites inhabit, however, is one driven by values and incentives that differ from the worlds that researchers and practitioners inhabit. Policymakers respond to signals and events that anticipate reelection and media coverage. They consider the standards of effectiveness, fidelity, and popularity rock-hard fixtures of their policy world. [vi]

Most practitioners, however, look to different standards. Although many teachers and principals have expressed initial support for high-performing public schools serving the poor and children of color, most practitioners have expressed strong skepticism about test scores as an accurate measure of either their effects on children or the importance of their work.

Such practitioners are just as interested in student outcomes as are policymakers, but the outcomes differ. They ask: What skills, content, and attitudes have students learned beyond what is tested? To what extent is the life lived in our classrooms and schools healthy, democratic, and caring? Can reform-driven programs, curricula, technologies be bent to our purposes? Such questions, however, are seldom heard. Broader student outcomes and being able to adapt policies to fit the geography of their classroom matter to practitioners.

Another set of standards comes from policy and practice-oriented researchers. Such researchers judge success by the quality of the theory, research design, methodologies, and usefulness of their findings to policy and student outcomes. These researchers’ standards have been selectively used by both policy elites and practitioners in making judgments about high- and low-performing schools. [vii]

So multiple standards for judging school “success” are available. Practitioner-and researcher- derived standards have occasionally surfaced and received erratic attention from policy elites. But it is this strong alliance of policymakers, civic and business elites, and friends in the corporate, foundation, and media worlds that relies on standards of effectiveness, fidelity, and popularity. This coalition and their standards continue to dominate public debate, school reform agendas, and determinations of “success” and “failure.”

And so for “personalized learning,” the effectiveness criterion lacking solid evidence of student success, gives way to the political popularity criterion that currently dominates policy debates over districts buying tablets and laptops to get teachers to put the new technological fad into classroom practice.

____________________________________________________

[i] Patrick McGuinn, No Child Left Behind and the Transformation of Federal Education Policy, 1965-2005 (Lawrence, KS: University Press of Kansas, 2006)

[ii]Harvey Kantor, “Education, Reform, and the State: ESEA and Federal Education Policy in the 1960s,” American Journal of Education, 1991, 100(1), pp. 47-83; Lorraine McDonnell, “No Child Left Behind and the Federal Role in Education: Evolution or Revolution?” Peabody Journal of Education, 2005 80(2), pp. 19-38.

[iii] Michael Kirst and Gail Meister, “Turbulence in American Secondary Schools: What Reforms Last,” Curriculum Inquiry, 1985, 15(2), pp. 169-186; Larry Cuban, “Reforming Again, Again, and Again,” Educational Researcher, 1991, 19(1), pp. 3-13.

[iv]Janet Quinn, et. al., Scaling Up the Success For All Model of School Reform, final report, (Santa Monica (CA): Rand Corportation, 2015).

[v] Sarah Reckhow, Follow the Money: How Foundation Dollars Change Public School Politics (New York: Oxford University Press, 2013); Frederick Hess and Jeff Henig (eds.) The New Education Philanthropy: Politics, Policy, and Reform (Cambridge, MA: Harvrd Education Press,, 2015).

[vi] Linda Darling Hammond,”Instructional Policy into Practice: The Power of the Bottom over the Top,” Educational Evaluation and Policy Analysis, 1990, 12(3), pp. 339-347. Charles Payne, So Much Reform, So Little Change (Cambridge, MA: Harvard Education Press, 2008). Joyce Epstein, “Perspectives and Previews on Research and Policy for School, Family, and Community Partnerships,” in(New York: Routledge, 1996), pp. 209-246.

[vii] Anita Zerigon-Hakes, “Translating Research Findings into Large-Scale Public Programs and Policy,” The Future of Children, Long-Term Outcomes of early Childhood Programs, 1995, 5(3), pp. 175-191; Richard Elmore and Milbrey McLaughlin, Steady Work (Santa Monica, CA: RAND Corporation, 1988);

11 Comments

Filed under research, school reform policies, technology use

More Comments on Personalization Continuum (Tom Hatch)

Tom Hatch is an Associate Professor at Teachers College, Columbia University. I met Tom at Stanford University when his wife, Karen Hammerness, was a graduate student and took one of my classes. Hatch had worked closely with Ted Sizer, Howard Gardner, and James Comer–leaders of whole-school reform movement in the 1980s and 1990s. He subsequently wrote thoughtfully about theories of action. I used an article he wrote in my seminars for many years (“The Difference in Theory That Matters in the Practice of School Improvement,” 1998 in American Educational Research Journal).

He posted this letter to me April 7, 2017. In it, he offers comments on the  Personalization Continuum that I had drafted, weaving together readers’ comments with his own research and experiences. 

Dear Larry

Your recent post describing the many versions of “personalization” you’ve seen in your visits to schools seems particularly relevant these days for a number of reasons:

Ironically, it’s probably worth noting that this surge in interest in personalization coincides with the closing of the national organization of the Coalition of Essential Schools – founded by Ted Sizer who put personalization on the map in the 1980’s and 90’s.

Your post prompted me to reflect on some of these developments and what I have been seeing in my own research on improvement efforts and “innovation” in several developing and developed countries.  In particular, I think your draft of what you call a “continuum” of personalization in many of the “lessons” you’ve observed nicely highlights the way that personalization often involves a mix of teacher-centered and student-centered activities. One way to expand the continuum and get at some of the complexities that you and your commentators have acknowledged might be to look at the extent to which several different educational decisions are made by teachers and/or students.  For example, many approaches to personalization talk about customizing the goals, the content, and the pace of educational activities. It seems like those approaches at your “teacher-directed” end of the spectrum adapt instruction to the needs/interests of students, but, for the most part teachers are still making the decisions about:

  • What the students should be learning (and why)
  • The materials they should use and the paths they should follow
  • The speed with which they move along those paths

At the other end, students may be making more of those decisions themselves.  In the middle, teachers and students may be sharing those decisions, teachers may make some decisions and students others, and teachers may make those decisions sometimes while students make them at others (e.g. higher performing students may be allowed/encouraged to make more of those decisions than their peers).

To make things more complicated, each decision about goals, content, and pace can be broken down into a whole series of related choices. Decisions related to content and materials, for instance, include who chooses reading materials, what to focus on in the reading, how to read it, and how material should be presented (as one of your commenters, Dylan Kane, noted).

It’s also possible to imagine a whole bunch of other decisions that we might (or might not!) want to take into account.  For example, I’m beginning to experiment with letting my graduate students choose not only when to take on particular assignments but also where (e.g. in the classroom or not; alone or with others).  I also remember passionate discussions at one meeting of educators working on designing a new school (the Celebration School, developed as part of the planned community connected to Disneyworld) about whether or not to enable students to determine the kind of lighting that best suited their “learning style.”

Adding to the complexities, as Laura Chapman pointed out in the comments, these decisions can also be made by those who develop the technologies used to support personalization.  It’s also possible, with the developments in distance and blended learning to imagine a variety of other people, including parents, taking a more direct role in these lesson-level decisions as well. (Extending the scope of personalization beyond “lessons” and courses, and making it a core concept in a reimagined system of education as in approaches like ReSchool Colorado can make it more complicated still.)

However we define the key instructional decisions, I think you’re right that the extent to which teachers or students make those decisions distinguishes many of the current approaches.  I’d be interested to know, though, how often you see personalized approaches that really give students wide latitude and extensive control over their own learning? Chris Ongaro, a graduate student here at Teachers College, is looking at student’s experiences in a variety of “personalized” courses (many of them online), and he tells me that even when students are given choices, those choices are usually extremely limited, rarely allowing students to imagine or pursue their own options.  As he said to me, students may play a role in shaping the means, but the ends are often predetermined.

While I raise these questions, following your descriptive lead, I’m trying not to place a particular value on one end of the continuum or the other.  But as we describe the role of the teacher and the student, I’m also reminded again of what Sizer often said (quoting James Comer, eminent psychologist and founder of the School Development Program):  The three most important things in schools (and school improvement) are “relationships, relationships, relationships.” For Sizer, personalization grew out of the belief that “we can’t teach students well if we do not know them well.”  That relationship both allows those in the role of teacher to recognize and respond to each student’s needs and interests, but it also opens up those in the student’s role to opportunities and challenges they may never have encountered on their own. While I often ask my students these days to explain to me why teachers are needed in schools (truth be told, I also ask them why we need “students”), it may be worth trying to capture something about the nature of the teacher-student relationships in these approaches to personalization as well.  But now your straightforward and clear continuum looks a lot more like one of those polygons and polyhedrons that you and W[ilfred] Rubens discussed…

At the end of the day, though, I see many of the same things you do: approaches to personalizing activities, classes, and courses that are often carried out in the regular school day or within typical course structures and with the expectation that “success” will mean meeting conventional graduation standards, going to college and getting a “good” job.  Perhaps it should be no surprise then, that under these circumstances, as you… put it:

…wherever these classrooms, programs, schools, and districts  fall on the continuum of personalized learning with their playlists, self-assessment software, and tailored lessons all of them work within the traditional age-graded school structure. No public school in Silicon Valley that I visited departed from that century-old school organization.

5 Comments

Filed under how teachers teach, research, school reform policies, technology use

Teachers and Researchers: Searching for the Truth of Classroom Change

I am preparing to write a section in my forthcoming book on technology integration about  the different perspectives that teachers and researchers have on changes in classroom lessons. To do that, I have looked back at the handful of posts I have written since 2009 on this point so I can figure out what to say in this forthcoming book.

Here is one from November 2009 along with a reader’s thoughtful comment (and criticism) of the position I take in the post.

Over the years, I have interviewed many teachers across the country who have described their district’s buying computers, deploying them in classrooms while providing professional development. These teachers have told me that using computers, interactive white boards, and other high-tech devices with accompanying software have altered their teaching significantly. They listed changes they have made such as their Powerpoint presentations and students doing Internet searches in class. They told me about using email with students.Teachers using interactive white boards said they can check immediately if students understand a math or science problem through their voting on the correct answer.

I then watched many of these teachers teach. Most teachers used the high-tech devices as they described in their interviews. Yet I was puzzled by their claim that using these devices had substantially altered how they taught. Policymaker decisions to buy and deploy high-tech devices was supposed to shift dominant ways of traditional teaching to student-centered, or progressive approaches. All of this in years when No Child Left Behind, extensive testing, and coercive accountability reigned. What I encountered in classrooms, however, departed from teachers’ certainty that they have changed how they teach.

I am not the first researcher to have met teachers who claimed substantial changes in their teaching in response to district or state policies. Consider “A Revolution in One Classroom; The Case of Mrs. Oublier.”

In the mid-1980s, California policymakers adopted a new elementary math curriculum intended to have students acquire a deep understanding of math concepts rather than memorizing rules and seeking the “right” answer. The state provided staff development to help elementary teachers implement the new curriculum. Then, researchers started observing teachers using the new math curriculum.

One researcher observed third grade teacher Mrs. Oublier (a pseudonym but hereafter Mrs. O) to see to what degree Mrs. O had embraced the innovative math teaching the state sought. Widely respected in her school as a first-rate math teacher, Mrs. O told the researcher that she had “revolutionized” her teaching. She was delighted with the new math text, used manipulatives to teach concepts, organized students desks into clusters of four and five, and had student participate in discussions. Yet the researcher saw her use paper straws, beans, and paper clips for traditional classroom tasks. She used small groups, not for students to collaborate in solving math problems, but to call on individuals to give answers to text questions. She used hand clapping and choral chants—as the text and others suggested—in traditional ways to get correct answers. To the researcher, she had grafted innovative practices onto traditional ways of math teaching and, in doing so, had missed the heart and soul of the state curriculum.

How can Mrs. O and teachers I have interviewed tell researchers that they had changed their teaching yet classroom observations of these very same teachers revealed familiar patterns of teaching? The answer depends on what each person means by “change” and who judges the worth of the change.

Change clearly meant one thing to teachers and another to researchers. Teachers had, indeed, made a cascade of incremental changes in their daily lessons. Researchers, however, keeping in mind what policymakers intended, looked for fundamental changes in teaching. In the case of Mrs. O—from memorizing math rules and getting the correct answer to focusing on conceptual understanding. Or in my case, getting teachers to shift from traditional to non-traditional instruction in seating arrangements, lesson activities, teacher-talk, use of projects, etc. In one instance, teachers saw substantial incremental “changes,” while researchers saw little fundamental “change.”

Whether those teachers’ incremental changes or the fundamental changes state policymakers sought led to test score gains, given the available evidence, no one yet knows.

So whose judgment about change matters most? “ Should researchers “consider changes in teachers’ work from the perspective of new policies….[or intentions of policymakers]? Or should they be considered from the teachers’ vantage point? (p.312).

Researchers, however, publish their studies and teachers like Mrs. O seldom tell their side of the story. Yet teachers’ perceptions of change have to be respected and voiced because they have indeed altered their practices incrementally and as any practitioners (lawyers, doctors, accountants) will tell you, that is very hard to do. How to honor teachers’ incremental changes while pointing out few shifts in fundamental patterns of teaching is the dilemma with which I have wrestled in researching high-tech use in schools.

____________________________

I now include a long comment to the above post from Brian Rude, a community college teacher. It was written on November 18, 2009

Larry, you sound frustrated. You are frustrated because teachers don’t do things quite the way you believe they should? So who’s right, you or the teachers?

I am no fan of B. F. Skinner, but he did say one thing that I think is very important. (At least I think it was Skinner. It was decades ago when I read this.) He said “The mouse is always right.” The context here is a psychologist doing an experiment with a mouse, and being frustrated because the results don’t come out as the psychologist would like and expect. It is quite understandable that the psychologist would blame the mouse, but it doesn’t take much reflection to realize how wrong that is. Of course the mouse doesn’t get it! It’s a mouse!

You say, “Policymaker decisions to buy and deploy high-tech devices was supposed to shift dominant ways of traditional teaching to student-centered, or progressive approaches.” Why on earth should it? Who’s right, the policy makers or the teachers? High tech is going to change the essential nature of teaching? Why should it?

I think my view is evident by now. If we want to learn about teaching and learning, we’d better look closely at what teachers and learners actually do, not what we think they should do. We need to ask why they do what they do, not why they don’t do what someone else thinks they should.

I teach lower level math courses in a community college. Every day I struggle with how to make students understand. I use high tech, everyday. I’m using high tech right now to write this. I’m not writing with a pencil or a fountain pen as I did in my youth. But that is pretty much irrelevant to the essential task of stringing words together in a way that will effectively communicate thoughts. Similarly the essential tasks of teaching have never changed. You need some way to present information. Students must attend to that information. They must build structures of knowledge in their minds. A lot of feedback is necessary for this to happen. My job is to provide them with the raw materials to build those structures of knowledge, and to guide them, as best I can, in how to build those structures of knowledge. Thus everyday I go to class and very carefully explain mathematical ideas and how to put them together. Everyday I do my best to put together well chosen problems for well designed homework assignments. Everyday I complain, at least to myself, about the bad textbooks we are stuck with. Every day, both in class and helping students individually in my office, I get questions that reveal misconceptions and errors of one sort or another in the thinking of students. I struggle to understand those misconceptions and errors of thinking, and to set the students on the right path again.

Almost everyday I make out handouts (usually pull them up from previous semesters and revise as needed) because that’s often the best way to make an assignment that meets my idea of what a good, effective, productive assignment should be. Well designed homework assignments are crucial, in my opinion. That’s where the rubber meets the road, so to speak. Math is a subject of ideas, but almost all math is learned by almost all students by doing problems. And they have to be the right problems, problems that provide the framework for students to put together mathematical ideas in ways that construct real knowledge. I am quite aware that many would dismiss me as a “worksheet teacher”. Who cares? I use high tech to make these handouts. But that is irrelevant to the essential nature of what I am doing. I am essentially doing the very same thing I did as a young teacher in the 60’s when I would make handouts on a spirit duplicating machine (and the kids would sniff them when they got them). All that high tech is no more central to the essence of teaching than is having a nice car to get me to work, rather than the 55 Chevy that took me to work in the early sixties.

I’m not claiming Mrs. O and the other teachers you describe are doing the best possible job in every situation. And I’m certainly not claiming that I do a perfect job in every situation. I am just saying that to be frustrated because they don’t do things the way you think they should, is to be like the psychologist who blames the mouse. The mouse just doesn’t get it. Of course. It’s a mouse. The teachers just don’t get it? Of course, they are teachers. They have reasons for what they do, though they may be no good at all in explaining those reasons, or even recognizing them. When there is a difference in what researchers and policy makers think is desirable, and what teachers actually do in the real world, I’ll go with the teachers every time.

2 Comments

Filed under how teachers teach, research

Toddlers and Touchscreens: What Does the Research Actually Say? (Marnie Kaplan)

“Prior to joining Bellwether, Marnie [Kaplan] worked as a policy analyst at Success Academy Charter Schools, where she analyzed local, state, and federal education policies. Previously she worked as a program manager at the District of Columbia Public Schools, where she tracked and analyzed special education compliance, and as a Stoneleigh Emerging Leaders Fellow at the Education Law Center, where she proposed solutions to reform Pennsylvania’s alternative education system and improve the accountability of cyber charter schools. Marnie began her career as a middle school English and social studies teacher in New York City. She went on to earn her M.P.P. and J.D. from Georgetown University. While in graduate school, Marnie interned at the Department of Education Office of Civil Rights, the Lawyers Committee for Civil Rights Under Law, and the DC Public Schools’ Urban Education Leaders Internship Program; taught street law to high school students; worked in a day care center; volunteered with 826DC; and served as a research assistant to the Georgetown Center on Poverty and Inequality. Marnie also holds a master’s in the science of teaching from Pace University and a bachelor’s degree from the University of Pennsylvania” (Bio taken from Bellwether staff descriptions)

This post appeared December 8, 2016 in Ahead of the Heard, A Bellwether blog.

 

 

 

toddler-with-iPad-300x150.jpg

You walk by an outdoor restaurant and see a toddler watching a movie on an iPad while his parents eat dinner. Your first thought is:

  • a) those parents deserve a break
  • b) screens don’t belong at meal time
  • c) is the video educational?
  • d) alert: bad parenting

Is there an app to help us decide how to respond? No. But a quorum of pediatricians might be able to help.

From 1999 till 2016, the American Academy of Pediatrics (AAP) discouraged the use of screen media by children under two (which might have led an informed passerby to loosely circle answer d while feeling slightly judgmental). But just last month, the AAP departed from its previous strict restriction on screen exposure for this age group.

There was a lot of media attention heralding the departure from the “no screens under two rule.” Some celebrated the beginning of the end of the “screen wars.” In reality, while the new guidelines offer a more nuanced view of screen exposure, the debate will likely rage on. Screens continue to pervade modern life so rapidly that research can’t keep up.

Let me fill in some background on why the AAP changed its recommendations. The “no screens before two” rule was first issued in 1999 as a response to interactive videos for infants such as Baby Einstein. Research showed these videos decreased children’s executive functioning and cognitive development. In October 2011, the AAP reaffirmed its original statement regarding infants and toddlers and media. The AAP’s statement cited three reasons: a lack of evidence on children learning from television or video before age two, studies showing a link between the amount of TV that toddlers watch and later attention problems, and studies pointing to how parents and playtime are affected by always-on TV. Since this statement was developed through  a lengthy internal review process, it was drafted before the iPad was first introduced to the market in April of 2010. So for the last five years, the strict restriction on screen time included touch screens even though the committee hadn’t evaluated the emerging research on this media.

In the intervening years, many doctors and scientists urged the AAP committee on children and media to revisit their recommendations and take a more balanced approach to media. In 2014, Dr. Michael Rich, the director of the Center on Media and Child Health at Boston Children’s Hospital, urged experts to base their recommendations on evidence-based decision making instead of values or opinions. He criticized pediatricians for focusing too much on negative effects and overlooking the positive effects of media on children. Later that year, Dr. Dimitri Christikas, director of the Center for Child Health, Behavior, and Development at University of Washington, suggested rethinking the guidelines to distinguish between TV and interactive screens. Dr. Christikas was one of the first researchers to determine that the time babies and toddlers spend in front of the TV was detrimental to their health and development. He posited that the time young children spend interacting with touch screens is more analogous to time playing with blocks than time passively watching a television. In 2015, a trio of pediatricians published an article offering further support for the idea that interactive media necessitated different guidelines than television. In the same article, they recognizing the need for further research and argued that doctors should emphasize the benefits of parents and children using interactive media together.

So what are a quorum of pediatricians saying in 2016?

The new AAP guidelines still set rather strict restrictions for children under eighteen months. The AAP recommends that infants and toddlers only be exposed to screens for the purpose of video chatting with family members. This squares with some emerging observational research but likely also displays pediatricians’ understanding of modern life. The new AAP guidelines say parents can introduce children between 18 and 24 months to education shows. For children between the ages of two and five, the AAP recommends a max of one hour per day of “high-quality programs,” which they define as PBS and Sesame Network.

But there remains a lot that pediatricians, neuroscientists, and developmental psychologists cannot say conclusively. How does a small child clamoring to watch videos of herself affect a child’s conception of self?  Does the sensory experience of interactive screens have negative effects on small children’s brains?

Scientists continue to approach the research regarding long-term effects of this exposure from different perspectives. In fact, earlier this month, at the annual meeting of the Society of Neuroscience, new research was presented which hinted at the possible detrimental effects of touch screens on young brains. Dr. Jan Marino Ramirez, from the Center for Integrative Brain Research at the Seattle Children’s Research Institute, presented new research which revealed that excessive exposure to sensory stimulation early in life had significant effects on the behavior and brain circuits of mice. The mice acted like they had attention deficit disorder (ADD), showed signs of learning problems, and engaged in risky behavior. Ramirez therefore recommends minimizing screen time for young children. In a recent interview, Dr. Leah Krubitzer, an evolutionary neurobiologist at University of California, Davis, was less concerned about the detrimental impacts of screen time. She believes the benefits may outweigh the negative effects. Krubitzer argues that fast-moving interactive touch screens may prepare children for our increasingly fast-paced world.

So, parents of young children can now feel less guilty encouraging their toddlers to video chat with family across the country. And possibly we have a more clear answer for the scenario above (e.g., If the child is at least two years old, the appropriate response is c, at least for now).

7 Comments

Filed under preschool, raising children, research, technology