Category Archives: school reform policies

More Cartoons on Common Core Standards

One of the most viewed monthly installments of cartoons that I have posted over the past three years has been on the Common Core. Here is another batch that I have gathered. Enjoy!

 

images-1

 

 

 

 

 

 

mike2aug1

 

 

 

 

 

 

 

 

 

images-2

 

 

 

 

 

 

 

images-4

 

 

 

 

 

 

 

images-3

 

 

 

 

 

 

 

images

 

 

 

 

 

 

 

 

images-5

 

 

 

 

 

 

 

 

 

images-6

 

 

 

 

 

 

images-1

 

 

 

 

 

 

NY Common Core

 

 

 

 

 

 

I end with a Calvin and Hobbes panel about a “typical” day in school.

 

0d488a28250a102d94d7001438c0f03b

 

 

 

3 Comments

Filed under school reform policies

A Poem about Teaching (Ann Staley)

I knew Ann Staley as a student in one of my classes soon after I came to Stanford to teach three decades ago. Over the time she was in graduate school, we ended up having many conversations about teaching, learning, graduate school, and life in all of its twists and turns. After leaving Stanford, she settled in Oregon and became a teacher and writer. She has taught in high school–winning an award in 1996–supervised student teachers, and taught writing at community college. In every one of her classes students wrote daily. A few years ago, she retired from teaching but continued writing both prose and poetry. Every so often we would contact one another. She sent me her first volume of poems, Primary Sources. And from her second published collection of poems, Instructions for the Wishing Light, I chose “Mrs. Kitchen” for this post.

Why a poem? Because in writing posts for this blog and for the articles and books I have written over the past half-century, I have used expository writing. I describe, analyze, and capture the nature of school reform, policy-making, and the practice of teaching using facts, evidence, and explanation.

Yet art, dance, drama, short stories, novels, and poetry can capture features of teaching and what students learn in ways that exposition cannot. Thus, “Mrs. Kitchen.”

 

 

MRS. KITCHEN

Teaching is about making 400 close-judgment calls a day.

Wise teacher comment

 

…traveled the world with her M.D. husband,

both working for the American Red Cross.

They returned to suburban Harrisburg

and began the next chapter of their lives.

Mrs. Kitchen became a 2nd grade teacher at Progress Elementary School.

Our classrooms had floor-to-ceiling windows,

which opened so you could hear recess voices,

and dark wooden floors polished to a sheen.

We were seated, not in usual rows,

but in a square “u” of desks.

We were allowed to sit with whomever

we wanted, as long as our work was uninterrupted

by giggling (the girls) or hitting (the boys).

Mrs. Kitchen was small in stature, big in heart.

 

She wore glasses and had curly brown hair.

She loved all of her students, but had,

I realized even then, a soft spot for me.

I didn’t understand why and still don’t.

Every afternoon, in the hour before school ended,

she read aloud to us–from books

on the New York Times Bestseller list.

Kon Tiki is one I remember most vividly.

 

Winifred Kitchen taught “up” to us,

believing that eight-year-olds could understand more

than the 1950s psychology books expected.

This was her great gift to her fortunate students.

We studied Cro-Magnon and Neanderthal men,

then made shadow boxes depicting their lives.

 

One day when I’d finished my work early,

she sent me to the library, alone, saying,

Get whatever book you want, Ann.

That day I chose a book titled The Pigtailed Pioneer,

about a girl whose covered wagon arrives in Portland, Oregon,

where she meets her first Indian in an encampment south of town.

I had braids, then, which my mother plaited each morning,

tying on plaid or satin ribbons that she ironed.

Girls still wore dresses to school in those days,

no pants were allowed until we got to Junior High School.

Jeans–never!

 

One afternoon I asked Mrs. K if I could go to the office

without being sent there. I wanted to meet the principal,

a woman, but wanted to go there on good terms.

She arranged an interview with this imposing woman.

After we finished speaking, the Principal told me to

sit behind her desk, answer the phone if it rang.

She was going out for her usual late afternoon of listening

to the classrooms with open doors. I was thrilled.

 

My 2nd grade year convinced me that I wanted to be a teacher.

I set up summer school for my dolls in the basement

and began, in earnest, my professional life.

 

 

6 Comments

Filed under how teachers teach

Pay-for-Performance for CEOs and Teachers

Oracle CEO Larry Ellison earns $37,692 an hour. No, that is not a typo or misplaced comma. Ellison’ annual salary ran $78.4 million, much of it in stock option awards. His salary was based on the annual performance of the company’s stock. Oracle’s Board of Directors set the pay scale (Ellison owns one-fourth of the company’s shares) to spur better management to increase profits and shareholders’ dividends.

They pay Ellison to perform well on the metric they have chosen (“company earnings before income taxes minus the costs of stock-based compensation, acquisitions, restructuring, and other items.” This CEO’s performance pay is not, however, a metric used by other major corporations for paying their top person. I return to the point of different measures used by companies to judge CEO performance later.

Switch now to the average U.S. public school teacher who earns an annual salary of over $55,000. That figure translates to around $27.00 for a 40-hour week. Like Ellison, hundreds of thousands of teachers are involved in pay-for-performance plans. In response to the federal Race To the Top competition, many states have mandated that teachers’ performance and salary be tied to students’ test scores to spur better teaching and student learning. Those test scores, as a factor in assessing effectiveness and determining salary (or bonuses), can range from as much as over half to one-quarter of the decision to set salary and retain or fire a teacher.

While I have written about this pay-for-performance reform over the past few years (see here, here, and here), for this post I want to inspect how the private sector–often a model for U.S. school reform–has its own problems, often undisclosed by business-oriented champions of school metrics, in determining CEO pay.

The lesson to learn from this post is: Paying for CEO performance in companies and schools is as flawed as the measures used to determine it.

A recent study of the metrics used in 195 large companies over the past five years showed that the most popular gauge measuring CEO performance was “total shareholder return.” Over half of the companies using that measure, however,  lost nearly two percent over the five-year period. Companies using less popular equations such as “earnings-per-share growth” gained almost three percent.

Now, here’s the clincher. Most companies judging CEO performance are relying on a metric that yielded loses for investors (“total shareholder return”)  yet,  at the same time, those very same companies continued to give their CEOs substantial raises year after year.

The authors of the study believe that the popularity of the performance measure, i.e., “total shareholder return,” stems from how easy it is for boards of directors and CEOs to manipulate the metric by “removing costs from the equation” such as “discontinuing product lines or closing factories.” Boards of directors then can reward CEOs with higher compensation packages. Earnings-per-share growth, a less popular metric and one of multiple measures that many firms use, sorts out under-performing from high performing firms, the authors found. This one as well as other measures, they concluded, are less easily manipulated by top corporate officials. CEO pay, then, can be better associated with company performance.

The main takeaways from this study is that boards of directors and CEOs do manipulate the numbers,  “one size does not fit all when measuring pay for performance,”and that multiple measures for determining effectiveness and salary have a better chance of capturing performance than single ones do.

Now, consider teacher pay-for-performance where one measure–student test scores–is often used to determine to what degree a teacher is effective. Like “total shareholder return” there are serious problems of using this metric alone or even in concert with other measures to judge teacher performance (see here and here).

Consider the following:

Incentives corrupt measures.

Since the mid-1970s, social scientists have criticized the use of specific quantitative measures to monitor or steer policies because those implementing such policies alter their practices to insure better numbers. The work of social scientist Donald T. Campbell and economists in the mid-1970s about the perverse outcomes of incentives was available but have largely been ignored. Campbell wrote in 1976.

“The more any quantitative social indicator is used for social decision-making, the more subject it will be to corruption pressures and the more apt it will be to distort and corrupt the social processes it is intended to monitor” (Campbell 1976, p.54)

Campbell used examples drawn from statistics on police solving crimes (p. 55), the Soviets setting numerical goals in industry (p. 57), and the U.S.’s use of “body counts” in Vietnam as evidence of winning (p.58). For public schools, Campbell said that “achievement tests are … highly corruptible indicators (p.57).”

That was nearly forty years ago. In the past decade, researchers have documented  (also see here) the link between standardized test scores and narrowed instruction to prepare students for test items, instances of  state policymakers fiddling with cut-off scores on tests, increased dropouts, and straight out cheating. Although how the distortions occur are unclear, the evidence confirms Campbell’s insight.

Easy To Measure Indicators Trump Hard To Measure Ones

Few in business, medicine or education question that some indicators are easier to quantify than others. In medicine, for example, hospital mortality and surgical procedures are fairly easy to measure but the results even when compared to other hospitals and surgeons hide as much as they reveal about effective health care. So it is with standardized tests.

Because test scores are inexpensive and efficient to collect, they draw attention away from important but hard-to-measure aspects of teaching and learning such as student engagement, rapport between teachers and students, academic climate in classroom and school, and principal leadership. Cumulative practitioner experience and stories about teaching over centuries have established these as crucial factors in working with gifted and vulnerable students.

*******************************************************************

These are known results of using single measures to judge individual or organizational performance. Consequences of their use can be anticipated. Historical examples abound. Some districts (e.g., Denver) wisely have moved to using multiple measures with student outcomes included that go beyond test scores but in most states where such mandates reign, test scores still remain a major part of the equation used to judge teacher performance (e.g., New York City, Washington, D.C., Houston, Texas) and allocate bonuses to teachers and principals.

This manipulation of data and one-size-fits-all measures show up in businesses as well as schools raising serious questions about the worth of this frenetic passion for pay-for-performance in both public and private sectors.

In the meantime, if Oracle’s Larry Ellison read this post in his office–say 10 minutes–he would have earned over $6,000. Ah, to be a CEO.

 

 

 

7 Comments

Filed under school reform policies

Evidence Based Education Policy and Practice: A Conversation (Francis Schrag)

 

This fictitious exchange between two passionate educators over making educational policy and influencing classroom practice through careful scrutiny of evidence–such as has occurred in medicine and the natural sciences–as opposed to relying on professional judgment anchored in expertise gathered in schools brings out a fundamental difference among educators and the public that has marked public debate over the past three decades. The center of gravity in making educational policy in the U.S. has shifted from counting resources that go into schooling and relying on professional judgment to counting outcomes students derive from their years in schools and what the numbers say.

That shift can be dated from the Elementary and Secondary Education Act of 1965 but gained sufficient traction after the Nation at Risk report (1983) to dominate debate over innovation, policy, and practice. Although this is one of the longest guest posts I have published, I found it useful (and hope that viewers will as well) in making sense of a central conflict that exist today within and among school reformers, researchers, teachers, policymakers and parents.

Francis Schrag is professor emeritus in the philosophy of education at the University of Wisconsin, Madison. This article appeared in Teachers College Record, March 14, 2014.

A dialogue between a proponent and opponent of Evidence Based Education Policy. Each position is stated forcefully and each reader must decide who has the best of the argument.

Danielle, a professor of educational psychology and Leo, a school board member and former elementary school teacher and principal, visit a middle-school classroom in Portland Maine where students are deeply engaged in building robots out of Lego materials, robots that will be pitted against other robots in contests of strength and agility.  The project requires them to make use of concepts they’ve learned in math and physics.  Everything suggests that the students are deeply absorbed in what is surely a challenging activity, barely glancing around to see who has entered their classroom.

Leo:  Now this is exciting education. This is what we should be moving towards.  I wish all teachers could see this classroom in action.

Danielle:  Not so fast.  I’ll withhold judgment till I have some data.  Let’s see how their math and science scores at the end of the year compare with those of the conventional classroom we visited this morning.  Granted that one didn’t look too out of the ordinary, but the teacher was really working to get the kids to master the material.

Leo:  I don’t see why you need to wait.  Can’t you see the difference in level of engagement in the two classrooms?  Don’t you think the students will remember this experience long after they’ve forgotten the formula for angular momentum? Your hesitation reminds me of a satirical article a friend showed me; I think it came from a British medical journal.  As I recall the headline went: “Parachute use to prevent death and major trauma related to gravitational challenge: systematic review of randomized controlled trials.”

Danielle:  Very cute, but let’s get serious.  Spontaneous reactions can be misleading; things aren’t always what they appear to be, as I’m sure you’ll agree.  I grant you that it looks as if the kids in this room are engaged, but we don’t know whether they’re engaged in the prescribed tasks and we don’t know what they’re actually learning, do we?  We’ll have a much better idea when we see the comparative scores on the test.  The problem with educators is that they get taken in with what looks like it works, they go with hunches, and what’s in fashion, but haven’t learned to consult data to see what actually does work.  If physicians hadn’t learned to consult data before prescribing, bloodletting would still be a popular treatment.

Suppose you and I agreed on the need for students to study math and physics.  And suppose that it turned out that the kids in the more conventional classroom learned a lot more math and physics, on average, as measured on tests, than the kids in the robotics classroom.  Would you feel a need to change your mind about what we’ve just seen?  And, if not, shouldn’t you?  Physicians are now on board with Evidence Based Medicine (EBM) in general, and randomized controlled trials (RCTs) in particular, as the best sources of evidence.  Why are teachers so allergic to the scientific method?  It’s the best approach we have to determine educational policy.

Leo:  Slow down Danielle.  You may recall that a sophisticated RCT convincingly showed the benefits of smaller class sizes in elementary schools in Tennessee, but these results were not replicated when California reduced its elementary school class size, because there was neither room in the schools for additional classrooms nor enough highly skilled teachers to staff them.  This example is used by Nancy Cartwright and Jeremy Hardie in their book on evidence-based policy to show that the effectiveness of a policy depends, not simply on the causal properties of the policy itself, but on what they call a “team” of support factors (2012, p. 25).  If any one of these factors were present in the setting where the trial was conducted but is lacking in the new setting, the beneficial results will not be produced.  This lack of generalizability, by the way, afflicts RCTs in medicine too.  For instance, the populations enrolled in teaching hospital RCTs are often different from those visiting their primary care physician.

Danielle:  I have to agree that educators often extrapolate from RCTs in a way that’s unwarranted, but aren’t you, in effect, calling for the collection of more and better evidence, rather than urging the abandonment of the scientific approach.  After all, the Cartwright and Hardie book wasn’t written to urge policy makers to throw out the scientific approach and go back to so-called expert or professional judgment, which may be no more than prejudice or illicit extrapolation based on anecdotal evidence.

Leo:  You seem to be willing to trust the data more than the judgment of seasoned professionals.  Don’t you think the many hours of observing and teaching in actual classrooms counts for anything?

Danielle: If your district has to decide which program to run, the robotics or the traditional, do you really want to base your decision on the judgment of individual teachers or principals, to say nothing of parents and interested citizens?  In medicine and other fields, meta-analyses have repeatedly shown that individual clinical judgment is more prone to error than decisions based on statistical evidence (Howick, 2011, Chap. 11). And, as I already mentioned, many of the accepted therapies of earlier periods, from bloodletting to hormone replacement therapy, turned out to be worse for the patients than doing nothing at all.

Now why should education be different?  How many teachers have “known” that the so-called whole-word method was the best approach to teaching reading, and years later found out from well-designed studies that this is simply untrue?  How many have “known” that children learn more in smaller classes?  No, even if RCTs aren’t always the way to go, I don’t think we can leave these things to individual educator judgment; it’s too fallible.

And you may not need to run a new study on the question at issue.  There may already be relevant, rigorous studies out there, testing more exploratory classrooms against more traditional ones in the science and math area for middle-schoolers.  I recommend you look at the federal government What Works website, which keeps track of trial results you can rely on.

Leo:  I’ve looked at many of these studies, and I have two problems with them.  They typically use test score gains as their indicator of durable educational value, but these can be very misleading.  Incidentally, there’s a parallel criticism of the use of “surrogate end points” like blood levels in medical trials.  Moreover, according to Goodhart’s Law—he was a British economist—once a measure becomes a target, it ceases to be a good indicator.  This is precisely what happens in education: the more intensely we focus on raising a test score by means of increasing test preparation to say nothing of cheating—everything from making sure the weakest, students don’t take the test to outright changing students’ answers—the less it tells us about what kids can do or will do outside the test situation.

Danielle:  Of course we need to be careful about an exclusive reliance on test scores.  But you can’t indict an entire approach because it has been misused on occasion.

Leo: I said there was a second problem, as well.  You recall that what impressed us about the robotics classroom was the level of involvement of the kids.  When you go into a traditional classroom, the kids will always look at the door to see who’s coming in.  That’s because they’re bored and looking for a bit of distraction.  Now ask yourself, what does that involvement betoken. It means that they’re learning that science is more than memorizing a bunch of facts, that math is more than solving problems that have no meaning or salience in the real world, that using knowledge and engaging in hard thinking in support of a goal you’ve invested in is one of life’s great satisfactions.  Most kids hate math and the American public is one of the most scientifically illiterate in the developed world.  Why is that?  Perhaps it’s because kids have rarely used the knowledge they are acquiring to do anything besides solve problems set by the teacher or textbook.

I’m sure you recall from your studies in philosophy of education the way John Dewey called our attention in Experience and Education to what he called, the greatest pedagogical fallacy, “the notion that a person learns only the particular thing he is studying at the time” (Dewey, 1938, p. 48).  Dewey went on to say that what he called “collateral learning,” the formation of “enduring attitudes” was often much more important than the particular lesson, and he cited the desire to go on learning as the most important attitude of all.  Now when I look at that robotics classroom, I can see that those students are not just learning a particular lesson, they’re experiencing the excitement that can lead to a lifetime of interest in science or engineering even if they don’t select a STEM field to specialize in.

Danielle:  I understand what Dewey is saying about “collateral learning.”  In medicine as you know, side effects are never ignored, and I don’t deny that we in education are well behind our medical colleagues in that respect.  Still, I’m not sure I agree with you and Dewey about what’s most important, but suppose I do.  Why are you so sure that the kids’ obvious involvement in the robotics activity will generate the continuing motivation to keep on learning?  Isn’t it possible that a stronger mastery of subject matter will have the very impact you seek?  How can we tell?  We’d need to first find a way to measure that “collateral learning,” then preferably conduct a randomized, controlled trial, to determine which of us is right.

Leo:  I just don’t see how you can measure something like the desire to go on learning, yet, and here I agree with Dewey, it may be the most important educational outcome of all.

Danielle:  This is a measurement challenge to be sure, but not an insurmountable one.  Here’s one idea: let’s track student choices subsequent to particular experiences.  For example, in a clinical trial comparing our robotics class with a conventional middle school math and science curriculum, we could track student choices of math and science courses in high school.  Examination of their high school transcripts could supply needed data.  Or we could ask whether students taking the robotics class in middle school were more likely (than peers not selected for the program) to take math courses in high school, to major in math or science in college, etc.  Randomized, longitudinal designs are the most valid, but I admit they are costly and take time.

Leo: I’d rather all that money went into the kids and classrooms.

Danielle:  I’d agree with you if we knew how to spend it to improve education.  But we don’t, and if you’re representative of people involved in making policy at the school district level, to say nothing of teachers brainwashed in the Deweyian approach by teacher educators, we never will.

Leo:  That’s a low blow, Danielle, but I haven’t even articulated my most fundamental disagreement with your whole approach, your obsession with measurement and quantification, at the expense of children and education.

Danielle:  I’m not sure I want to hear this, but I did promise to hear you out.  Go ahead.

Leo:  We’ve had about a dozen years since the passage of the No Child Left Behind Act to see what an obsessive focus on test scores looks like and it’s not pretty.  More and more time is taken up with test-prep, especially strategies for selecting right answers to multiple-choice questions.  Not a few teachers and principals succumb to the temptation to cheat, as I’m sure you’ve read.  Teachers are getting more demoralized each year, and the most creative novice teachers are finding jobs in private schools or simply not entering the profession.  Meanwhile administrators try to game the system and spin the results.  But even they have lost power to the statisticians and other quantitatively oriented scholars, who are the only ones who can understand and interpret the test results.  Have you seen the articles in measurement journals, the arcane vocabulary and esoteric formulas on nearly every page?

And do I have to add that greedy entrepreneurs with a constant eye on their bottom lines persuade the public schools to outsource more and more of their functions, including teaching itself.  This weakens our democracy and our sense of community.  And even after all those enormous social costs, the results on the National Assessment of Educational Progress are basically flat and the gap between black and white academic achievement—the impetus for passing NCLB in the first place—is as great as it ever was.

Danielle:  I agree that it’s a dismal spectacle.  You talk as if educators had been adhering to Evidence Based Policy for the last dozen years, but I’m here to tell you they haven’t and that’s the main reason, I’d contend, that we’re in the hole that we are.  If educators were less resistant to the scientific approach, we’d be in better shape today.  Physicians have learned to deal with quantitative data, why can’t teachers, or are you telling me they’re not smart enough?  Anyhow, I hope you feel better now that you’ve unloaded that tirade of criticisms.

Leo:  Actually, I’m not through, because I don’t think we’ve gotten to the heart of the matter yet.

Danielle:  I’m all ears.

Leo:  No need to be sarcastic, Danielle.  Does the name Michel Foucault mean anything to you?  He was a French historian and philosopher.

Danielle:  Sure, I’ve heard of him.  A few of my colleagues in the school of education, though not in my department, are very enthusiastic about his work.  I tried reading him, but I found it tough going.  Looked like a lot of speculation with little data to back it up.  How is his work relevant?

Leo:   In Discipline and Punish, Foucault described the way knowledge and power are intertwined, especially in the human sciences, and he used the history of the school examination as a way of illustrating his thesis (1975/1995, pp. 184-194).  Examinations provide a way of discovering “facts” about individual students, and a way of placing every student on the continuum of test-takers.  At the same time, the examination provides the examiners, scorers and those who make use of the scores ways to exercise power over kids’ futures.  Think of the Scholastic Assessment Tests (SATs) for example.  Every kid’s score can be represented by a number and kids can be ranked from those scoring a low of 600 to those with perfect scores of 2400.  Your score is a big determinant of what colleges will even consider you for admission.  But that’s not all: Foucault argued that these attempts to quantify human attributes create new categories of young people and thereby determine how they view themselves.  If you get a perfect SAT score, or earn “straight As” on your report card, that becomes a big part of the way others see you and how you see yourself.  And likewise for the mediocre scorers, the “C” students, or the low scorers who not only have many futures closed to them, but may see themselves as “losers,” “failures,” “screw-ups.”  A minority may, of course resist and rebel against their placement on the scale—consider themselves to be “cool”, unlike the “nerds” who study, but that won’t change their position on the continuum or their opportunities.  Indeed, it may limit them further as they come to be labeled “misfits” “ teens at-risk,” “gang-bangers” and the like. But, and here’s my main point, this entire system is only possible due to our willingness to represent the capabilities and limitations of children and young people by numerical quantities.  It’s nothing but scientism, the delusive attempt to force the qualitative, quirky, amazingly variegated human world into a sterile quantitative straight-jacket.  You recall the statement that has been attributed to Einstein, don’t you, “Not everything that can be counted counts, and not everything that counts can be counted.” I just don’t understand your refusal to grasp that basic point; it drives me mad.

Danielle:  Calm down, Leo.  I don’t disagree that reducing individuals to numbers can be a problem; every technology has a dark side, I’ll grant you that, but think it through.  Do you really want to go back to a time when college admissions folks used “qualitative” judgments to determine admissions?  When interviewers could tell from meeting a candidate or receiving a letter of recommendation if he were a member of “our crowd,” would know how to conduct himself at a football game, cocktail party, or chapel service, spoke without an accent, wasn’t a grubby Jew or worse, a “primitive” black man or foreign-born anarchist or communist.  You noticed I used the masculine pronoun:  Women, remember, were known to be incapable of serious intellectual work, no data were needed, the evidence was right there in plain sight.  Your Foucault is not much of a historian, I think.

Leo:  We have some pretty basic disagreements here.  I know we each believe we’re right.  Is there any way to settle the disagreement?

Danielle:  I can imagine a comprehensive, longitudinal experiment in a variety of communities, some of which would carry out EBEP and control communities that would eschew all use of quantification.  After a long enough time, maybe twenty years, we’d take a look at which communities were advancing, which were regressing.  Of course, this is just an idea; no one would pay to actually have it done.

Leo:  But even if we conducted such an experiment, how would we know which approach was successful?

Danielle:  We shouldn’t depend on a single measure, of course.  I suggest we use a variety of measures, high school graduation rate, college attendance, scores on the National Assessment of Educational Progress, SATs, state achievement tests, annual income in mid-career, and so on.  And, of course, we could analyze the scores by subgroups within communities to see just what was going on.

Leo:  Danielle, I can’t believe it.  You haven’t listened to a word I’ve said.

Danielle:  What do you mean?

Leo:   If my favored policy is to eschew quantitative evidence altogether, wouldn’t I be inconsistent if I permitted the experiment to be decided by quantitative evidence, such as NAEP scores or worse, annual incomes?  Don’t you recall that I reject your fundamental assumption—that durable, significant consequences of educational experiences can be represented as quantities?

Danielle:  Now I’m the one that’s about to scream.  Perhaps you could assess a single student’s progress by looking at her portfolio at the beginning and end of the school year.  How, in the absence of quantification, though, can you evaluate an educational policy that affects many thousands of students?  Even if you had a portfolio for each student, you’d still need some way to aggregate them in order to be in a position to make a judgment about the policy or program that generated those portfolios.  You gave me that Einstein quote to clinch your argument.  Well, let me rebut that with a quotation by another famous and original thinker, the Marquis de Condorcet, an eighteenth century French philosopher and social theorist.  Here’s what he said:  “if this evidence cannot be weighted and measured, and if these effects cannot be subjected to precise measurement, then we cannot know exactly how much good or evil they contain” (Condorcet, 2012, p.138).  The point remains true, whether in education or medicine.  If you can’t accept it, I regret to say, we’ve reached the end of the conversation.

References

Cartwright, N & Hardie, J. (2012). Evidence-based policy:  A practical guide to doing it better.  Oxford and New York: Oxford University Press.

Condorcet, M. (2012). The sketch. In S. Lukes, and N. Urbinati (Eds.), Political Writings (pp. 1-147). Cambridge: Cambridge University Press.

Dewey, J. (1938/1973). Experience and education.  New York: Collier Macmillan Publishers.

Foucault, M. (1995).  Discipline and punish: The birth of the prison. (A. Sheridan, Trans.) New York: Vintage Books. (Original work published in 1975)

Howick, J. (2011). The Philosophy of evidence-based medicine. Oxford: Blackwell Publishing.

21 Comments

Filed under comparing medicine and education, school reform policies

What’s The Evidence on School Devices and Software Improving Student Learning?

The historical record is rich in evidence that research findings have played a subordinate role in making educational policy. Often, policy choices were (and are) political decisions. There was no research, for example, that found establishing tax-supported public schools in the early 19th century was better than educating youth through private academies. No studies persuaded late-19th century educators to import the kindergarten into public schools. Ditto for bringing computers into schools a century later.

So it is hardly surprising, then, that many others, including myself, have been skeptical of the popular idea that evidence-based policymaking and evidence-based instruction can drive teaching practice. Those doubts have grown larger when one notes what has occurred in clinical medicine with its frequent U-turns in evidence-based “best practices.”

Consider, for example, how new studies have often reversed prior “evidence-based” medical procedures.

*Hormone therapy for post-menopausal women to reduce heart attacks was found to be more harmful than no intervention at all.

*Getting a PSA test to determine whether the prostate gland showed signs of cancer for men over the age of 50 was “best practice” until 2012 when advisory panels of doctors recommended that no one under 55 should be tested and those older  might be tested if they had family histories of prostate cancer.

And then there are new studies that recommend women to have annual mammograms, not at age  50 as recommended for decades, but at age 40. Or research syntheses (sometimes called “meta-analyses”) that showed anti-depressant pills worked no better than placebos.

These large studies done with randomized clinical trials–the current gold standard for producing evidence-based medical practice–have, over time, produced reversals in practice. Such turnarounds, when popularized in the press (although media attention does not mean that practitioners actually change what they do with patients) often diminished faith in medical research leaving most of us–and I include myself–stuck as to which healthy practices we should continue and which we should drop.

Should I, for example, eat butter or margarine to prevent a heart attack? In the 1980s, the answer was: Don’t eat butter, cheese, beef, and similar high-saturated fat products. Yet a recent meta-analysis of those and subsequent studies reached an opposite conclusion.

Figuring out what to do is hard because I, as a researcher, teacher, and person who wants to maintain good health has to sort out what studies say and  how those studies were done from what the media report, and then how all of that applies to me. Should I take a PSA test? Should I switch from margarine to butter?

If research into clinical medicine produces doubt about evidence-based practice, consider the difficulties of educational research–already playing a secondary role in making policy and practice decisions–when findings from long-term studies of innovation conflict with current practices. Look, for example, at computer use to transform teaching and improve student achievement.

Politically smart state and local policymakers believe that buying new tablets loaded with new software, deploying them to K-12 classrooms, and watching how the devices engage both teachers and students is a “best practice.” The theory is that student engagement through the device and software will dramatically alter classroom instruction and lead to improved  achievement. The problem, of course–sure, you already guessed where I was going with this example–is that evidence of this electronic innovation transforming teaching and achievement growth is not only sparse but also unpersuasive even when some studies show a small “effect size.”

Turn now to the work of John Hattie, a Professor at the University of Auckland (NZ), who has synthesized the research on different factors that influence student achievement and measured their impact on learning. For example, over the last two decades, Hattie has examined over 180,000 studies accumulating 200, 000 “effect sizes”  measuring the influence of teaching practices on student learning. All of these studies represent over 50 million students.

He established which factors influenced student learning–the “effect size–by ranking each from 0.1 (hardly any influence) to 1.0 or a full standard deviation–almost a year’s growth in student learning. He found that the “typical” effect size of an innovation was 0.4.

To compare different classroom approaches shaped student learning, Hattie used the “typical” effect size (0.4) to mean that a practice reached the threshold of influence on student learning (p. 5). From his meta-analyses, he then found that class size had a .20 effect (slide 15) while direct instruction had a .59 effect (slide 21). Again and again, he found that teacher feedback had an effect size of .72 (slide 32). Moreover, teacher-directed strategies of increasing student verbalization (.67) and teaching meta-cognition strategies (.67) had substantial effects (slide 32).

What about student use of computers (p. 7)? Hattie included many “effect sizes” of computer use from distance education (.09), multimedia methods (.15), programmed instruction (.24), and computer-assisted instruction (.37). Except for “hypermedia instruction” (.41), all fell below the “typical ” effect size (.40) of innovations improving student learning (slides 14-18). Across all studies of computers, then, Hattie found an overall effect size of .31 (p. 4).

According to Hattie’s meta-analyses, then, introducing computers to students will  fall well below other instructional strategies that teachers can and do use. Will Hattie’s findings convince educational policymakers to focus more on teaching? Not as long as political choices trump research findings.

Even if politics were removed from the decision-making equation, there would still remain the major limitation of  most educational and medical research. Few studies  answer the question: under what conditions and with which students and patients does a treatment work? That question seldom appears in randomized clinical trials. And that is regrettable.

 

 

28 Comments

Filed under comparing medicine and education, how teachers teach, technology use

Moving Forward without a Backward Glance: MOOCs and Technological Innovations

In a recent commentary on the rock star Sting’s dipping back into his childhood to revitalize his song writing, David Brooks said: “how important it is to ground future vision in historical consciousness.” I agree with Brooks when it comes to the half-life  of technological innovations. The experience of Massive  Open Online Courses (MOOCs) over the past few years is an unexpected example of what Brooks meant.

Much has been written about MOOCs  since they went viral in the past three years (see here, here, here, and here). This vision of creating platforms for college-level courses that would give anyone with an Internet connection access to college courses while reducing ever-escalating costs of higher education has turned some professors into academic entrepreneurs. Here is a two-for-one innovation (increased efficiency and equity) that has married new technologies with global access to higher education. MOOCs spread rapidly among elite institutions (e.g., Harvard, MIT, Stanford) and some second- and third-tier universities. For those familiar with the Gartner hype cycle–which many acolytes of MOOCs somehow either missed or ignored–the first two phases of the cycle were textbook examples:

“Technology Trigger: A potential technology breakthrough kicks things off. Early proof-of-concept stories and media interest trigger significant publicity. Often no usable products exist and commercial viability is unproven.

Peak of Inflated Expectations: Early publicity produces a number of success stories—often accompanied by scores of failures. Some companies take action; many do not.”

Recent articles (see here and here) express disappointment mixed with hope over how MOOCs have fared since the first blush of the academic love affair with the innovation. The evidence thus far is ample: high dropout rates, little knowledge of what students who completed a MOOC actually learned, lack of faculty enthusiasm, and the real sticking point for universities–how to make money from offering MOOCs? No surprise, then, that the birth rate of new MOOCs has plummeted. We are now in the “Trough of Disillusionment” phase of the cycle.

images

 

 

 

 

The high hopes and inspired rhetoric pushing MOOCs have collapsed. Looking back, the creators were pained–one of them, Sebastian Thrun, has departed from the MOOC scene–and I must add, terribly innocent about earlier technological innovations in education.

Of course, I do not know how (or whether) the next phase (“Slope of Enlightenment”) will unfold. No one does. It is a work in progress. But how does all of this current disappointment with MOOCs connect to the point I raised in the first paragraph: “how important it is to ground future vision in historical consciousness?”

Would knowing the checkered history of technological innovations in K-12 schools and higher education–including the Hype Cycle–help high-tech innovators “ground their future vision?” Yes, it would but I doubt if lessons drawn from earlier innovations would help them alter what they will do anyway. While innovators are creative and hopeful about the future they may be, in David Brooks’ words, “necessarily naive.”

And it is that phrase “necessarily naive” that creates the paradox previous high-tech innovators and school reformers have faced and do so now.

The paradox works like this: If I know well what has occurred with past technological innovations seeking to reshape K-12 and higher education, that is, most fail in the first few years, I would not even try. However, if I don’t care about those past efforts  but still forge ahead because I have faith that what I propose will work regardless of the odds, then I can succeed.

The paradox of forging ahead without a backward glance is 100 percent  American.  Consider often described characteristics of being American: highly individualistic, competitive, optimistic, believes in change, especially technological, as an unvarnished good and that anyone with grit who works hard can overcome any obstacle. There are other characteristics associated with being American including beliefs in equality, a strong work ethic, and fairness.

Running like a red thread in the white fabric of being American, however, is the pervasive belief that if you know the past well, it can be a drag–a disincentive, economists would say–for action, invention, and making progress. To avoid looking backward in order to innovate, one has to be “necessarily naive” in the face of past failures in new technologies. Hence, with “naive” entrepreneurs ignoring the past, there has been a swift rise in and decline of MOOCs.

A skeptic might say: Really, Larry, what would you have to know about past technological innovations that might have helped the founders of MOOCs avoid the “trough of disillusionment?”

My answer is:

1. Technological innovations aimed primarily at increasing productivity and efficiency in schooling have largely ignored teacher knowledge and expertise.

2. High-tech innovators seldom ask the questions teachers ask about a new classroom technology.

3. Innovators have cared little about whether their new technology can be integrated into teachers’ routines because their priorities are to transform teaching and learning, increase student productivity, and keep costs low.

A backward glance to lessons drawn from previous technological innovations, then, might help start-up entrepreneurs from being “necessarily naive” about MOOCs or the next new thing for K-12 classrooms. Will that happen? I doubt it.

 

12 Comments

Filed under school reform policies, technology use

Troubled Youth, Troubled Learning (Dave Reid)

Dave Reid is a high school mathematics teacher in his third year of teaching.   He received his MA in Education and credential in secondary mathematics and physics from Stanford University in 2011.  Dave spent a quarter of a century in high-tech primarily in the wireless and Global Positioning System (GPS) industries.  He earned a BS degree in electrical engineering from George Mason University, and an MBA in finance and marketing from Santa Clara University.  He also attended the U.S. Military Academy at West Point. He blogs as Mr. Math Teacher and tweets as @mathequality.

While the title for this post does not always ring true, in my few years teaching at Title I schools, it often reflects reality.  In fact, rarely does a day go by where no student disrupts the classroom learning environment for one reason or another.  As a fifty-something, I knew this going into teaching; what I did not know was how deleterious these disruptions are to continuity, sanity, and in the limit: opportunity, for my students, not me.  As someone in the classroom every day, hoping above all hope that my students can break out of their behavioral binds, it challenges my every fiber of existence to keep the class focused on our learning objective(s) for the day.

Troubled youth make for troubled learning, not only for themselves, but also for everyone in the classroom.  It is a huge force multiplier of the negative type.  In spite of what is heralded as the balm for these troubles, compassion, empathy, and other soft moves are frequently insufficient to overcome years of ingrained indifference, frustration, anger, resentment, or a host of other emotions, feelings, or attitudes that have overtaken an adolescent overwhelmed by his or her circumstances.  The older the youth, the more deeply embedded the issue or issues.  Now, extend these to one or more adolescents in a classroom, and you get a snapshot of teaching in a Title I school.

A few days ago, for instance, I taught three block periods: two of which are split into two sections apiece of algebra 1 and remedial mathematics, and one AP Calculus section.  The split sections are my attempt to support students who do not possess the arithmetic skill or understanding needed to succeed in algebra.  Fortunately, my administration and the district office support me in this effort.

The AP Calculus students are rarely “egregiously” troublesome, aside from the fact that they have yet to realize that frequent side conversations among the eight groups of four students each frequently distracts others.  At times, when teaching these students, it feels as if I am an onstage performer at a dinner theater with the audience commenting back and forth to each other about their meal, the show, or what not.  Periodically, I tell them that the classroom is not their living room, or a movie theater, where they freely watch or chat as they see fit during “the show.”  They seem a bit startled when I make them aware of their behavior, which puzzles me even more; it is as if I am the first and only teacher to ask them to consider their impact on a classroom.  Notwithstanding their surprise, I persist, as I do not believe college professors will tolerate their behavior any more than I do, for the majority of my calculus students are college bound this fall.

Yet, this is not a post about my privileged students, who make up most of my calculus students.  For they, mostly, are buffered, or far removed, from the intense psychosocial trauma faced by many low-income families.  Simply put, they live free from most of the burdens of poverty.  Burdens, which manifest themselves in low-income families, that inhibit attaining outcomes at the same level as those more privileged for the same level of effort.

My most challenged students, behaviorally and academically, frequent my algebra sections.  Their presence cannot be missed: whether visually or aurally.  While it only takes one student to derail the trajectory of a class, it is a rare day, indeed, when only one student in a class acts to call attention to themselves.  The duration, intensity, and frequency of the derailments vary based on the class composition.

In the face of these ever-present disruptions, I have to: keep students’ attention focused on moving forward with their learning; address the momentary outburst and its subsequent ripples throughout the classroom; all the while doing my best to stay passionate, motivated, and encouraging without having a mental breakdown.  I say that somewhat tongue in cheek.  However, it is not too far from reality.  Whoever mentioned that a teacher has nearly as demanding a job as an air traffic controller was pretty close to the truth.

Which brings me to the student who inspired this post.  John rarely participates positively in class. He seems to possess a boundless ability to draw negative attention to himself throughout a class period. He failed first semester and is on track to do the same this semester. I hope with all of my heart that he wakes up soon and understands how important it is to his future that he pay attention in class, attempt some of his homework, and learn as much as is humanly possible, for he is quite intelligent in spite of what he may believe.

John reminds me of how my younger brother, now deceased, might have been in school.  My brother was often truant.  He ran with the wrong crowd, experimented with things I never knew existed at his age, and dropped out of high school shortly after starting.  My brother may have been one of the silent ones, the student who attempts to disappear among the thirty or so classmates.  He might have giggled frequently chatting away with his classmates.  Regardless, he did not learn.  He missed out on that opportunity, as he was deeply troubled.  I will not go into details except to say that his burdens were too much for him.  They may have been too much for his teacher, if they manifested themselves while in school: I simply do not know.  What I do know is that I became a teacher, in part, to help those like my younger brother, of whom this one student reminds me.  .

I will not hold my breath for John.  I will encourage him as often as possible, in between addressing his behavioral shortcomings, for they do impact the class.  His mother is at her wits end and unsure what to do about him.  I believe my parents felt similarly some thirty plus years ago.  Life is amazingly complex.  Teaching is crazy hard.  It drains me nearly every day.  Yet, there is rarely a day I leave home headed to my classroom not eager to teach. Yes, some troubled youth await me; they are whom I most hope to help.  Yet, I only can do my part to work toward keeping them on a path to graduate; they need to do their part as well.  Time will only tell.

26 Comments

Filed under how teachers teach

Cartoons about Families and Schools

For this monthly feature of cartoons, I pulled together a bunch that got me smiling, chuckling, and occasionally laughing out loud. About half of them are about parent-child relationships in the home; the other half are about teachers and students from ex-middle school teacher Diana Bledsoe whose work I have featured before.

I met Diana through my blog. I read hers and saw that she did cartoons about a fictitious middle school. She told me that she is a “cartoonist who has been in the education field for over 15 years: first as a volunteer, then a teacher and currently as an administrator. My cartoons are inspired by my daily interactions with students and educational professionals.

In these Bledsoe’s cartoons, she features Stewart, a student with a mop of yellow hair who has only passed Gym and has given Mrs. Banks, a teacher, a hard time. Enjoy.

 

140303_cartoon_063_a18065_p465

girlscouts

images

an-a-in-gym1

healthy-lunch

halloween-costume-contest1

if-teaching-were-an-olympic1

resolution2

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

resolution-pt-4-chai-tea-la

images-2

images

parents kid blog

6 Comments

Filed under how teachers teach, raising children

Politics, Research, and School Reform: Letting Teens Sleep in

Teaching high school students, first period of the school day, say, 7:30 or 8 AM is tough. Why? Students from both affluent and working class families shuffle into the room, sometimes carrying wake-up food and drink, and sit down at their desks giving the teacher the 1000-yard stare or closing their glazed eyes. They are sleepy.

Recent research (see here, here and here) has established that adolescent bodies and minds are still developing and getting five or less hours of sleep a night when doctors recommend nine means sluggish lessons in the mornings and sleepy afternoons in class.

Citing such research, some school boards (e.g., Long Beach, California; Glen Falls, New York, and Stillwater, Oklahoma), after many open meetings with parents and experts on sleep and teenagers initiated later start times for middle and high school students. Research tied to solving a problem–sleepy and non-involved teenagers in academic classes– supporting a tidy solution such as a later school starting time in morning–seemed, thus far, to work in these communities. However, in other communities, raw politics, and coalitions built by sleep-deprived teenagers allied with parents and teachers made the changes.

Consider 17 year-old Jilly Dos Santos who tries again and again to get to her 7:50 AM class on time at Rock Bridge High School in Columbia (MO). And failed. She is an academically strong student, works at a fast food restaurant after school and interned in a get-out-the-voter campaign earlier in the year. She heard that the school board was meeting in a few weeks to approve a half-hour earlier starting time. Yes, 7:20 AM. Santos, a sleep-deprived teenagers morphed into a political “sleep activist.”

JP-SLEEP-tmagArticle

Santos created a Facebook page and Twitter account telling hundreds of fellow  students that the school board was going to start school at 7:20 AM. She contacted a non-profit group about sleep that gave her the scientific studies about how teenagers needed more, not less, sleep. She emailed all teachers in the district and started an online petition. She brought other students together and they made posters. She tweeted everyone that “If you are going to be attending the board meeting tomorrow we recommend you dress up.”

You guessed it. The school board turned down the earlier start time. A few months later, the coalition that Santos had pulled together worked successfully to get the school board to start high school at 9 AM. The superintendent said after the board voted 6-1 in favor of the later time: “Jilly kicked it over the edge for us.”

Who said that schools are apolitical institutions?

I use the example of Santos to underscore how an issue as school start times, so often driven by efficiency–scheduling a limited number of buses for both elementary and secondary schools, when teachers have to be in their classrooms in the morning, parents’ demands for child care, and other factors–gets turned around when a group of teenagers, teachers, and parents coalesce into a political group pressing the school board to alter its policy. Rowdy democracy in action.

So here is an incontrovertible fact: schools are political institutions. This fact means that teachers, principals, superintendents, students, and parents are political actors also. Not in the partisan sense of Democrats and Republicans but in the fundamental sense that politics are about relationships over power, resources, and to achieve goals.

Of course, reformers in every generation have known that schools are political institutions subject to popular pressures to adopt or reject policies. With the state and federal centralization of authority for school policies over the past half-century–think No Child Left Behind, state charter school laws, and Common Core Standards–the political nature of schooling becomes self-evident. Although the word “politics” continues to have a sour smell about it to many parents, teachers, principals, and superintendents, for Jilly Dos Santos, the fragrance of politicking the school board to adopt a later start time drove her on. She and like-minded citizens practiced democratic action.

Here is the second fact about the role research studies played into the political success of the coalition that Santos’s mobilized in favor of a later start time. As much as each of us believes that data compiled into evidence, especially from scientific studies, are essential to get a policy adopted–after all we see ourselves as rational and mindful creatures–in this instance of having teenagers come to school later in the morning–research studies became useful but clearly subordinate tools. Without the political muscle of  the coalition Santos and others mobilized, ho-hum responses from the school board would have occurred.

Political muscle at the federal, state, and local levels, using research as a shield and lance, continues to dominate the current reform debate over what teachers should teach, how they should teach, choice in schools, and, yes, what time Jilly Dos Santos has to wake up and go to Rock Bridge High School tomorrow morning.

10 Comments

Filed under school reform policies

Arranging Classroom Furniture: An Unobtrusive Glimpse into How Teachers Teach

How teachers arrange the furniture in classrooms gives a peek into how teachers teach. Look at these photos taken last year of elementary and secondary classrooms that have different furniture arrangements.Science+room

photo 9

images-4

Note the different arrangements of  desks. In the first photo, rows of movable desks face the front of the classroom where the teacher’s desk is located. The second photo has a horseshoe pattern of tablet armchairs across from one another. The third photo is of an elementary classroom that is chock-full of materials and children working on different activities with adults sitting on the rug and chair working with  individual pupils. And the final photo is one of a secondary classroom arranged in rows where each student has a tablet and a smart phone.

images-2

Now, take a look at photos of classrooms over the past century.

1890.0169

bostonlatin-001

1950s_classroom

Five decades later when movable desks and chairs replaced the traditional bolted down ones, a photo shows a typical classroom.

Note the regimented order of these classrooms a century ago and even five decades later. True, those desks were bolted down a century ago and were even a teacher then so inclined to arranging small groups of students–and such teachers were around–they could do it but had to overcome the furniture arrangement. But a half-century later, with movable desks, rows were still there in many classrooms but not others.

Are the changes in how classrooms are furnished and how students appear dramatically different? Yes and no.

The “yes” part is in how students are dressed and how there are more examples now of different ways to arrange desks and chairs over the decades. The “no” part is that while different ways of organizing furniture in elementary classrooms is evident and apparent for anyone who ventures into a kindergarten and first grade classroom, that is much less the case for secondary classrooms.

Do such photos of classroom furniture give observers a glimpse of how teachers teach? Yes, they do but only a hint. Here is my reasoning.

a.Furniture arrangement is seldom mandated by a school board, superintendent, or principal. The teacher decides how to use classroom space. Furniture placement, consciously or not, expresses the teacher’s views of how best to teach, maintain order, and how students learn. Thus, an observer gets a clue to whether teacher-centered and student-centered instruction* (including mixes of both) will prevail.

b. When all students face the teacher’s desk or teacher at the blackboard (now whiteboard or “smart board”) where directions, daily homework, textbook readings and quizzes are registered, whole group instruction is encouraged including class discussions (recitation was the word used in the early 20th century). Teacher-talk  gains higher priority and legitimacy than exchanges between and among students.

c. Surveillance is easier for a teacher when rows or tables are in rows. Threats to classroom order can be seen quickly and dealt with expeditiously.

d. Such a configuration of classroom space limits students’ movement within a classroom to that which the teacher permits.

e. If desks are arranged into a hollow square, horseshoe, or tables are scattered around the room permitting students to face one another and talk, student-centered instruction where student talk and decision-making are prized becomes a much stronger possibility.

Note, however, that furniture arrangements do not determine how teachers teach. Classroom rows, tables, or horseshoe configurations are no more than clues to what teachers believe and practice in their lessons. Keep in mind  that for the early decades of this century when desks were fixed to the floor, there were still teachers who ingeniously and with much energy overcame that obstacle and introduced student-centered practices into the classroom.Such furniture may have discouraged many teachers but it did not prevent some from altering their teaching practices.

So a glimpse of classroom furniture is useful as a starting point in assessing how teachers teach but it is only a small part of how teachers structure lessons and carry out activities. Far more information about what happens in the classroom would be needed since teacher-centered instruction can, and often does, occur even when seating arrangements look student-centered.

Furniture arrangements and the placement of students, then, are not random affairs.

images-3

They are the result of teacher decisions stemming from beliefs in keeping order and how students learn best in the age-graded school within which teachers work. So when I enter a classroom, the first thing I note and record is how desks and chairs are arranged in any classroom.

_____________________

*In using the language of  “teacher-centered-” and “student-centered” instruction, I need to be clear that I do not favor one over the other. Both forms of instruction and hybrids can be effective with different students at different times in different contexts. Classroom arrangements offer only a hint of what teachers believe and how they teach. That visible sign is only that, not the full picture of daily lessons.

25 Comments

Filed under how teachers teach