Category Archives: comparing medicine and education

“Good” Doctors and Teachers (Part 2)

1. Has the definition of “good” teachers changed over time as has the one about “good” physicians?

2. Are there many different versions of “good” teaching as there were for “good” doctors?

3. Even with the differences in definitions over time and setting, are their core characteristics that transcend both as there were among “good” doctors?

4. Are “good” teachers dependent for success on their students, as doctors are on their patients?

The answer to each of the four questions is yes.

1. Has the definition of “good” teachers changed over time as has the one about “good” physicians? 

From the 1960s, researchers laid out the following personal traits and behaviors that “good” teachers exhibit:

[E]ffective teachers carefully monitor learning activities and are clear, accepting and supportive, equitable with students, and persistent in challenging and engaging them.

In the 1980s and 1990s, researcher findings added up to the following attributes of “effective” teachers. They:

*are clear about instructional goals;

*are knowledgeable about their content an strategies for teaching it;

*communicate to their students what is expected of them and why:

*make expert use of existing instructional materials in order to devote more time to practices that enrich and clarify the content;

* are knowledgeable about their students, adapting instruction to their needs….;

*address higher- as well as lower-level cognitive objectives….;

*accept responsibility for student outcomes;

*are thoughtful and reflective of their practices.

Then there are the features of “good” teachers that progressives then and now hold dear:

*A classroom that is student-centered:

*Teaching methods that are inquiry driven and organized around problem-solving and investigation:
*Instructors who are passionate about their subject’s real world significance.
*Metacognition—critical reflection about content
and pedagogy—is an integral part of the classroom
experience.

Lists of attributes and behaviors of “good” teachers appear every decade. Some lists overlap, some do not.

good-teacher-mind-map-Medium

 

 

 

 

 

 

 

 

2. Are there many different versions of “good” teaching as there were for “good” doctors?

Surely, there are. Consider that since the 1990s, policymakers have rushed to raise academic standards, hold teachers and administrators accountable for student outcomes, and expanded testing. In that push, a narrowed view of what constitutes “good” teaching has unfolded that focuses more on direct instruction and teacher-centered behaviors.

cartoon2

 

 

 

 

 

Yet there are students who see “good” teaching as different than this current mainstream view (e.g., “What makes a great teacher is being kind,” “A great teacher is someone that cares for his or her students,” “Someone who can make learning fun and someone who can be funny and focused at the same time”).

And for many, but not the majority, there are parents, practitioners, and researchers, who define a “good” teacher as going beyond high test scores. They want their children’s teachers—reflecting another age-old tradition of teaching—to work daily for the well-being of the child, see students as whole human beings, believe in active learning, create structures for students to collaborate and explore. In short, these folks embrace a progressive ideology of teaching believing with supreme confidence that students exposed to this tradition of teaching will do well on tests, graduate and go to college. They would point to Los Angeles teacher Rafe Esquith, kindergarten teacher Vivian Paley, and Foxfire teachers in rural Georgia as “good” teachers who nurture, inspire, and connect to students.

3. Even with the differences in definitions over time and setting, are there core characteristics that transcend both as there were among “good” doctors?

Yes, there are. Just as when medical staff, patients, professionals and non-professionals define “goodness” in physicians, two essential features crop up again and again for teachers: competence and caring.

the.modern.school.teacher

4. Are “good” teachers dependent on their students as doctors are on their patients for success.

Yes. they are. To see how the dependence works, one has to sort out the notion of “good” from the idea of “successful.” They are often seen as equivalent terms. They are not. Once sorted out, it becomes clear that both teachers and doctors depend on their students and patients to learn and heal.

Keep in mind that doctors and teachers using “good” practices do not automatically yield “good” results. Following the best practices in either job leads, from time to time, to failure, not success. Why? Because motivated students and patients have to participate fully for “good” teaching to turn into “successful” learning and the same is true for doctors and their patients.

images

 

 

 

 

Here is how the distinction works for teachers. Good” teaching pursues morally and rationally sound instructional practices. “Successful teaching,” on the other hand, is teaching that produces the desired learning. As Gary Fenstemacher and Virginia Richardson put it:

“[T]eaching a child to kill another with a single blow may be successful teaching, but it is not good teaching. Teaching a child to read with understanding, in a manner that is considerate and age appropriate, may fail to yield success (a child who reads with understanding), but the teaching may accurately be described as good teaching. Good teaching is grounded in the task sense of teaching, while successful teaching is grounded in the achievement sense of the term.”

Another way to distinguish between “good” and “successful” is when a 8th grade teacher teaches the theory of evolution consistent with the age of the child and best practices of science teaching (the “good” part) and then has her students complete three written paragraphs filled with relevant details and present-day examples that demonstrate their understanding of the theory of evolution (the “successful” part). These teaching acts are not the same nor does one necessarily lead to the other.

For the past quarter-century, however, policymakers and politicians have chopped, grated, and blended the goals of schooling into a concoction seeking to make education an arm of the economy. They scan international test scores, focus on achievement gaps, and boost teacher pay-for-performance plans. This policy direction has shoved the notion of “good” teaching into one corner of the ideological debate and thoroughly erased the distinction between the “good” and “successful” in teaching. Now “good” teaching means test scores go up and students go to college. A big mistake.

Why a mistake? Erasing the distinctions between “good” and “successful” teaching muddles policy prescriptions seeking to improve how teachers teach and what students learn. Best example of that muddle is evaluating teacher performance on the basis of student test scores. Consider, for example, the stark differences between Houston’s pay-teachers-for-performance and Denver’s ProComp plan.

**************************************************************************

The answers to the four questions are monotonously “yes.” The string of “yes” answers reveals that policymakers have, as so often they do, ignored the history of diverse teaching traditions and different ways of teaching that parents, practitioners, and researchers prize resulting in an unfortunate monopoly on only one way of teaching while students—in their glorious diversity–learn in many different ways.

images-1

 

 

 

 

 

 

 

__________________________________________

*The quote marks are there to signal that “good” (or “great,” “excellent,” “effective”) is an adjective that varies in meaning among parents, teachers, students, researchers, and policymakers.

 

9 Comments

Filed under comparing medicine and education

“Good” Doctors and Teachers* (Part 1)

During the 1930s, my grandmother saw a specialist about a melanoma on her face. During the course of the visit when she asked him a question, he slapped her face, saying, ‘I’ll ask the questions here. I’ll do the talking.’ Can you imagine such an event occurring today? Melanomas may not have changed much in the last fifty years, but the profession of medicine has.  Eric J. Cassel, 1985[i]

Today, a stinging slap to the cheek of a patient who asked a question of her doctor could lead to an assault charge. Doctor-centered practice–paternalistic authority is no more. Shared decision-making between doctor and patient has become the ideal. In short, the definition of a “good” doctor has changed dramatically in the past half-century.[ii]

Even with this 180 degree shift in defining “goodness,” there remains much variation even among former TV doctors Welby and Kildare and today’s Dr. House. All are seen as “good” in different ways as times change.

And that is why I put “good” in parentheses. Personal features (e.g., communication skills, empathy), expertise (e.g., credentials on walls, medical specialty), what others say, and context matter greatly in judging how “good” a doctor is.

Here is how one doctor puts the issue of defining “goodness” among physicians.

In my view, there are many ways a doctor can be good, so it’s difficult to know what someone means when he or she says a doctor is good.

For some people, being a good doctor is all about bedside manner, personality and communication skills. Other people value smarts, technical skills or expertise in a particular condition. Still others rely on credentials, such as where a doctor went to medical school or residency training. I’ve even known patients who care little about these other factors and instead care most about how the office runs, how quickly the phone is answered or how friendly the receptionist is.

The type of doctor may also determine how a person defines a good doctor. For example, many people I know say they don’t care about a surgeon’s bedside manner as long as his or her patients have outstanding results. Yet those same people might say that a good bedside manner is much more important for their primary care physician.

Then there are those magazines that list “best” doctors in their cities annually. How do they compile such lists? New York magazine, for example, depends upon a private firm that polls doctors for their recommendations:

The idea is that medical professionals are best qualified to judge other medical professionals, and if one recommendation is good (think of your doctor referring you to a specialist), multiple recommendations are better. Licensed physicians vote online (castleconnolly.com/nominations) for those doctors they view as exceptional.

So if the notion of a “good” doctor varies by time–doctor-centered then and patient-centered now– it also varies by what patients and doctors, each having quite different perspectives, value most in medical practitioners (e.g.,competence,  empathy, bedside manner). In short, there is not one single definition of a “good” doctor that covers all settings, perspectives, and times.

Yet even with all of this variation over what constitutes a “good” doctor, even with all of those lists of personal and technical features that patients want in their doctors, two generic characteristics emerge from the flow of words time and again. These basic features: competence and caring–turn up in studies (see here) and public opinion polls among both physicians and patients.

Keep in mind, however, that even the most competent and caring doctor depends upon the patient for any success in diagnosis and treatment. The truth is that expertise and caring are necessary ingredients for any definition of “goodness” in medical practice but, overall, insufficient in the helping professions without the patient’s cooperation.

While doctors can affect a patient’s motivation, if that patient is emotionally depressed, is resistant to recommended treatments, or uncommitted to getting healthy by ignoring prescribed medications the physician is stuck. Medical competence and empathy fall short when patients cannot or do not enter into the process of healing.

This basic predicament in the helping professions of being dependent upon the cooperation of the patient for any success–often unremarked upon–hobbles any definition of a “good” doctor.

Does the historical shift in definitions about “good” doctors and the fundamental dilemma they face apply to teachers? I answer that in Part 2.

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

[i]Epigraph story in Christine Laine and Frank Davidoff, “Patient-Centered Medicine,” JAMA, 1996, 275(2), p. 152.

[ii] Ronald Epstein, Md., et. al. “Communicating Evidence for Participatory Decision-making,” JAMA, 2004, 291(19), pp. 2359-2366; Simon Whitney, Md., et. al., “A Typology of Shared Decision Making, Informed Consent, and Simple Consent,” Annals of Internal Medicine, 2003, 140, pp. 54-59.

______________________

*Synonyms for “good” are “best,” “great,” “effective,” “stellar,” etc.

7 Comments

Filed under comparing medicine and education

Evidence Based Education Policy and Practice: A Conversation (Francis Schrag)

 

This fictitious exchange between two passionate educators over making educational policy and influencing classroom practice through careful scrutiny of evidence–such as has occurred in medicine and the natural sciences–as opposed to relying on professional judgment anchored in expertise gathered in schools brings out a fundamental difference among educators and the public that has marked public debate over the past three decades. The center of gravity in making educational policy in the U.S. has shifted from counting resources that go into schooling and relying on professional judgment to counting outcomes students derive from their years in schools and what the numbers say.

That shift can be dated from the Elementary and Secondary Education Act of 1965 but gained sufficient traction after the Nation at Risk report (1983) to dominate debate over innovation, policy, and practice. Although this is one of the longest guest posts I have published, I found it useful (and hope that viewers will as well) in making sense of a central conflict that exist today within and among school reformers, researchers, teachers, policymakers and parents.

Francis Schrag is professor emeritus in the philosophy of education at the University of Wisconsin, Madison. This article appeared in Teachers College Record, March 14, 2014.

A dialogue between a proponent and opponent of Evidence Based Education Policy. Each position is stated forcefully and each reader must decide who has the best of the argument.

Danielle, a professor of educational psychology and Leo, a school board member and former elementary school teacher and principal, visit a middle-school classroom in Portland Maine where students are deeply engaged in building robots out of Lego materials, robots that will be pitted against other robots in contests of strength and agility.  The project requires them to make use of concepts they’ve learned in math and physics.  Everything suggests that the students are deeply absorbed in what is surely a challenging activity, barely glancing around to see who has entered their classroom.

Leo:  Now this is exciting education. This is what we should be moving towards.  I wish all teachers could see this classroom in action.

Danielle:  Not so fast.  I’ll withhold judgment till I have some data.  Let’s see how their math and science scores at the end of the year compare with those of the conventional classroom we visited this morning.  Granted that one didn’t look too out of the ordinary, but the teacher was really working to get the kids to master the material.

Leo:  I don’t see why you need to wait.  Can’t you see the difference in level of engagement in the two classrooms?  Don’t you think the students will remember this experience long after they’ve forgotten the formula for angular momentum? Your hesitation reminds me of a satirical article a friend showed me; I think it came from a British medical journal.  As I recall the headline went: “Parachute use to prevent death and major trauma related to gravitational challenge: systematic review of randomized controlled trials.”

Danielle:  Very cute, but let’s get serious.  Spontaneous reactions can be misleading; things aren’t always what they appear to be, as I’m sure you’ll agree.  I grant you that it looks as if the kids in this room are engaged, but we don’t know whether they’re engaged in the prescribed tasks and we don’t know what they’re actually learning, do we?  We’ll have a much better idea when we see the comparative scores on the test.  The problem with educators is that they get taken in with what looks like it works, they go with hunches, and what’s in fashion, but haven’t learned to consult data to see what actually does work.  If physicians hadn’t learned to consult data before prescribing, bloodletting would still be a popular treatment.

Suppose you and I agreed on the need for students to study math and physics.  And suppose that it turned out that the kids in the more conventional classroom learned a lot more math and physics, on average, as measured on tests, than the kids in the robotics classroom.  Would you feel a need to change your mind about what we’ve just seen?  And, if not, shouldn’t you?  Physicians are now on board with Evidence Based Medicine (EBM) in general, and randomized controlled trials (RCTs) in particular, as the best sources of evidence.  Why are teachers so allergic to the scientific method?  It’s the best approach we have to determine educational policy.

Leo:  Slow down Danielle.  You may recall that a sophisticated RCT convincingly showed the benefits of smaller class sizes in elementary schools in Tennessee, but these results were not replicated when California reduced its elementary school class size, because there was neither room in the schools for additional classrooms nor enough highly skilled teachers to staff them.  This example is used by Nancy Cartwright and Jeremy Hardie in their book on evidence-based policy to show that the effectiveness of a policy depends, not simply on the causal properties of the policy itself, but on what they call a “team” of support factors (2012, p. 25).  If any one of these factors were present in the setting where the trial was conducted but is lacking in the new setting, the beneficial results will not be produced.  This lack of generalizability, by the way, afflicts RCTs in medicine too.  For instance, the populations enrolled in teaching hospital RCTs are often different from those visiting their primary care physician.

Danielle:  I have to agree that educators often extrapolate from RCTs in a way that’s unwarranted, but aren’t you, in effect, calling for the collection of more and better evidence, rather than urging the abandonment of the scientific approach.  After all, the Cartwright and Hardie book wasn’t written to urge policy makers to throw out the scientific approach and go back to so-called expert or professional judgment, which may be no more than prejudice or illicit extrapolation based on anecdotal evidence.

Leo:  You seem to be willing to trust the data more than the judgment of seasoned professionals.  Don’t you think the many hours of observing and teaching in actual classrooms counts for anything?

Danielle: If your district has to decide which program to run, the robotics or the traditional, do you really want to base your decision on the judgment of individual teachers or principals, to say nothing of parents and interested citizens?  In medicine and other fields, meta-analyses have repeatedly shown that individual clinical judgment is more prone to error than decisions based on statistical evidence (Howick, 2011, Chap. 11). And, as I already mentioned, many of the accepted therapies of earlier periods, from bloodletting to hormone replacement therapy, turned out to be worse for the patients than doing nothing at all.

Now why should education be different?  How many teachers have “known” that the so-called whole-word method was the best approach to teaching reading, and years later found out from well-designed studies that this is simply untrue?  How many have “known” that children learn more in smaller classes?  No, even if RCTs aren’t always the way to go, I don’t think we can leave these things to individual educator judgment; it’s too fallible.

And you may not need to run a new study on the question at issue.  There may already be relevant, rigorous studies out there, testing more exploratory classrooms against more traditional ones in the science and math area for middle-schoolers.  I recommend you look at the federal government What Works website, which keeps track of trial results you can rely on.

Leo:  I’ve looked at many of these studies, and I have two problems with them.  They typically use test score gains as their indicator of durable educational value, but these can be very misleading.  Incidentally, there’s a parallel criticism of the use of “surrogate end points” like blood levels in medical trials.  Moreover, according to Goodhart’s Law—he was a British economist—once a measure becomes a target, it ceases to be a good indicator.  This is precisely what happens in education: the more intensely we focus on raising a test score by means of increasing test preparation to say nothing of cheating—everything from making sure the weakest, students don’t take the test to outright changing students’ answers—the less it tells us about what kids can do or will do outside the test situation.

Danielle:  Of course we need to be careful about an exclusive reliance on test scores.  But you can’t indict an entire approach because it has been misused on occasion.

Leo: I said there was a second problem, as well.  You recall that what impressed us about the robotics classroom was the level of involvement of the kids.  When you go into a traditional classroom, the kids will always look at the door to see who’s coming in.  That’s because they’re bored and looking for a bit of distraction.  Now ask yourself, what does that involvement betoken. It means that they’re learning that science is more than memorizing a bunch of facts, that math is more than solving problems that have no meaning or salience in the real world, that using knowledge and engaging in hard thinking in support of a goal you’ve invested in is one of life’s great satisfactions.  Most kids hate math and the American public is one of the most scientifically illiterate in the developed world.  Why is that?  Perhaps it’s because kids have rarely used the knowledge they are acquiring to do anything besides solve problems set by the teacher or textbook.

I’m sure you recall from your studies in philosophy of education the way John Dewey called our attention in Experience and Education to what he called, the greatest pedagogical fallacy, “the notion that a person learns only the particular thing he is studying at the time” (Dewey, 1938, p. 48).  Dewey went on to say that what he called “collateral learning,” the formation of “enduring attitudes” was often much more important than the particular lesson, and he cited the desire to go on learning as the most important attitude of all.  Now when I look at that robotics classroom, I can see that those students are not just learning a particular lesson, they’re experiencing the excitement that can lead to a lifetime of interest in science or engineering even if they don’t select a STEM field to specialize in.

Danielle:  I understand what Dewey is saying about “collateral learning.”  In medicine as you know, side effects are never ignored, and I don’t deny that we in education are well behind our medical colleagues in that respect.  Still, I’m not sure I agree with you and Dewey about what’s most important, but suppose I do.  Why are you so sure that the kids’ obvious involvement in the robotics activity will generate the continuing motivation to keep on learning?  Isn’t it possible that a stronger mastery of subject matter will have the very impact you seek?  How can we tell?  We’d need to first find a way to measure that “collateral learning,” then preferably conduct a randomized, controlled trial, to determine which of us is right.

Leo:  I just don’t see how you can measure something like the desire to go on learning, yet, and here I agree with Dewey, it may be the most important educational outcome of all.

Danielle:  This is a measurement challenge to be sure, but not an insurmountable one.  Here’s one idea: let’s track student choices subsequent to particular experiences.  For example, in a clinical trial comparing our robotics class with a conventional middle school math and science curriculum, we could track student choices of math and science courses in high school.  Examination of their high school transcripts could supply needed data.  Or we could ask whether students taking the robotics class in middle school were more likely (than peers not selected for the program) to take math courses in high school, to major in math or science in college, etc.  Randomized, longitudinal designs are the most valid, but I admit they are costly and take time.

Leo: I’d rather all that money went into the kids and classrooms.

Danielle:  I’d agree with you if we knew how to spend it to improve education.  But we don’t, and if you’re representative of people involved in making policy at the school district level, to say nothing of teachers brainwashed in the Deweyian approach by teacher educators, we never will.

Leo:  That’s a low blow, Danielle, but I haven’t even articulated my most fundamental disagreement with your whole approach, your obsession with measurement and quantification, at the expense of children and education.

Danielle:  I’m not sure I want to hear this, but I did promise to hear you out.  Go ahead.

Leo:  We’ve had about a dozen years since the passage of the No Child Left Behind Act to see what an obsessive focus on test scores looks like and it’s not pretty.  More and more time is taken up with test-prep, especially strategies for selecting right answers to multiple-choice questions.  Not a few teachers and principals succumb to the temptation to cheat, as I’m sure you’ve read.  Teachers are getting more demoralized each year, and the most creative novice teachers are finding jobs in private schools or simply not entering the profession.  Meanwhile administrators try to game the system and spin the results.  But even they have lost power to the statisticians and other quantitatively oriented scholars, who are the only ones who can understand and interpret the test results.  Have you seen the articles in measurement journals, the arcane vocabulary and esoteric formulas on nearly every page?

And do I have to add that greedy entrepreneurs with a constant eye on their bottom lines persuade the public schools to outsource more and more of their functions, including teaching itself.  This weakens our democracy and our sense of community.  And even after all those enormous social costs, the results on the National Assessment of Educational Progress are basically flat and the gap between black and white academic achievement—the impetus for passing NCLB in the first place—is as great as it ever was.

Danielle:  I agree that it’s a dismal spectacle.  You talk as if educators had been adhering to Evidence Based Policy for the last dozen years, but I’m here to tell you they haven’t and that’s the main reason, I’d contend, that we’re in the hole that we are.  If educators were less resistant to the scientific approach, we’d be in better shape today.  Physicians have learned to deal with quantitative data, why can’t teachers, or are you telling me they’re not smart enough?  Anyhow, I hope you feel better now that you’ve unloaded that tirade of criticisms.

Leo:  Actually, I’m not through, because I don’t think we’ve gotten to the heart of the matter yet.

Danielle:  I’m all ears.

Leo:  No need to be sarcastic, Danielle.  Does the name Michel Foucault mean anything to you?  He was a French historian and philosopher.

Danielle:  Sure, I’ve heard of him.  A few of my colleagues in the school of education, though not in my department, are very enthusiastic about his work.  I tried reading him, but I found it tough going.  Looked like a lot of speculation with little data to back it up.  How is his work relevant?

Leo:   In Discipline and Punish, Foucault described the way knowledge and power are intertwined, especially in the human sciences, and he used the history of the school examination as a way of illustrating his thesis (1975/1995, pp. 184-194).  Examinations provide a way of discovering “facts” about individual students, and a way of placing every student on the continuum of test-takers.  At the same time, the examination provides the examiners, scorers and those who make use of the scores ways to exercise power over kids’ futures.  Think of the Scholastic Assessment Tests (SATs) for example.  Every kid’s score can be represented by a number and kids can be ranked from those scoring a low of 600 to those with perfect scores of 2400.  Your score is a big determinant of what colleges will even consider you for admission.  But that’s not all: Foucault argued that these attempts to quantify human attributes create new categories of young people and thereby determine how they view themselves.  If you get a perfect SAT score, or earn “straight As” on your report card, that becomes a big part of the way others see you and how you see yourself.  And likewise for the mediocre scorers, the “C” students, or the low scorers who not only have many futures closed to them, but may see themselves as “losers,” “failures,” “screw-ups.”  A minority may, of course resist and rebel against their placement on the scale—consider themselves to be “cool”, unlike the “nerds” who study, but that won’t change their position on the continuum or their opportunities.  Indeed, it may limit them further as they come to be labeled “misfits” “ teens at-risk,” “gang-bangers” and the like. But, and here’s my main point, this entire system is only possible due to our willingness to represent the capabilities and limitations of children and young people by numerical quantities.  It’s nothing but scientism, the delusive attempt to force the qualitative, quirky, amazingly variegated human world into a sterile quantitative straight-jacket.  You recall the statement that has been attributed to Einstein, don’t you, “Not everything that can be counted counts, and not everything that counts can be counted.” I just don’t understand your refusal to grasp that basic point; it drives me mad.

Danielle:  Calm down, Leo.  I don’t disagree that reducing individuals to numbers can be a problem; every technology has a dark side, I’ll grant you that, but think it through.  Do you really want to go back to a time when college admissions folks used “qualitative” judgments to determine admissions?  When interviewers could tell from meeting a candidate or receiving a letter of recommendation if he were a member of “our crowd,” would know how to conduct himself at a football game, cocktail party, or chapel service, spoke without an accent, wasn’t a grubby Jew or worse, a “primitive” black man or foreign-born anarchist or communist.  You noticed I used the masculine pronoun:  Women, remember, were known to be incapable of serious intellectual work, no data were needed, the evidence was right there in plain sight.  Your Foucault is not much of a historian, I think.

Leo:  We have some pretty basic disagreements here.  I know we each believe we’re right.  Is there any way to settle the disagreement?

Danielle:  I can imagine a comprehensive, longitudinal experiment in a variety of communities, some of which would carry out EBEP and control communities that would eschew all use of quantification.  After a long enough time, maybe twenty years, we’d take a look at which communities were advancing, which were regressing.  Of course, this is just an idea; no one would pay to actually have it done.

Leo:  But even if we conducted such an experiment, how would we know which approach was successful?

Danielle:  We shouldn’t depend on a single measure, of course.  I suggest we use a variety of measures, high school graduation rate, college attendance, scores on the National Assessment of Educational Progress, SATs, state achievement tests, annual income in mid-career, and so on.  And, of course, we could analyze the scores by subgroups within communities to see just what was going on.

Leo:  Danielle, I can’t believe it.  You haven’t listened to a word I’ve said.

Danielle:  What do you mean?

Leo:   If my favored policy is to eschew quantitative evidence altogether, wouldn’t I be inconsistent if I permitted the experiment to be decided by quantitative evidence, such as NAEP scores or worse, annual incomes?  Don’t you recall that I reject your fundamental assumption—that durable, significant consequences of educational experiences can be represented as quantities?

Danielle:  Now I’m the one that’s about to scream.  Perhaps you could assess a single student’s progress by looking at her portfolio at the beginning and end of the school year.  How, in the absence of quantification, though, can you evaluate an educational policy that affects many thousands of students?  Even if you had a portfolio for each student, you’d still need some way to aggregate them in order to be in a position to make a judgment about the policy or program that generated those portfolios.  You gave me that Einstein quote to clinch your argument.  Well, let me rebut that with a quotation by another famous and original thinker, the Marquis de Condorcet, an eighteenth century French philosopher and social theorist.  Here’s what he said:  “if this evidence cannot be weighted and measured, and if these effects cannot be subjected to precise measurement, then we cannot know exactly how much good or evil they contain” (Condorcet, 2012, p.138).  The point remains true, whether in education or medicine.  If you can’t accept it, I regret to say, we’ve reached the end of the conversation.

References

Cartwright, N & Hardie, J. (2012). Evidence-based policy:  A practical guide to doing it better.  Oxford and New York: Oxford University Press.

Condorcet, M. (2012). The sketch. In S. Lukes, and N. Urbinati (Eds.), Political Writings (pp. 1-147). Cambridge: Cambridge University Press.

Dewey, J. (1938/1973). Experience and education.  New York: Collier Macmillan Publishers.

Foucault, M. (1995).  Discipline and punish: The birth of the prison. (A. Sheridan, Trans.) New York: Vintage Books. (Original work published in 1975)

Howick, J. (2011). The Philosophy of evidence-based medicine. Oxford: Blackwell Publishing.

21 Comments

Filed under comparing medicine and education, school reform policies

What’s The Evidence on School Devices and Software Improving Student Learning?

The historical record is rich in evidence that research findings have played a subordinate role in making educational policy. Often, policy choices were (and are) political decisions. There was no research, for example, that found establishing tax-supported public schools in the early 19th century was better than educating youth through private academies. No studies persuaded late-19th century educators to import the kindergarten into public schools. Ditto for bringing computers into schools a century later.

So it is hardly surprising, then, that many others, including myself, have been skeptical of the popular idea that evidence-based policymaking and evidence-based instruction can drive teaching practice. Those doubts have grown larger when one notes what has occurred in clinical medicine with its frequent U-turns in evidence-based “best practices.”

Consider, for example, how new studies have often reversed prior “evidence-based” medical procedures.

*Hormone therapy for post-menopausal women to reduce heart attacks was found to be more harmful than no intervention at all.

*Getting a PSA test to determine whether the prostate gland showed signs of cancer for men over the age of 50 was “best practice” until 2012 when advisory panels of doctors recommended that no one under 55 should be tested and those older  might be tested if they had family histories of prostate cancer.

And then there are new studies that recommend women to have annual mammograms, not at age  50 as recommended for decades, but at age 40. Or research syntheses (sometimes called “meta-analyses”) that showed anti-depressant pills worked no better than placebos.

These large studies done with randomized clinical trials–the current gold standard for producing evidence-based medical practice–have, over time, produced reversals in practice. Such turnarounds, when popularized in the press (although media attention does not mean that practitioners actually change what they do with patients) often diminished faith in medical research leaving most of us–and I include myself–stuck as to which healthy practices we should continue and which we should drop.

Should I, for example, eat butter or margarine to prevent a heart attack? In the 1980s, the answer was: Don’t eat butter, cheese, beef, and similar high-saturated fat products. Yet a recent meta-analysis of those and subsequent studies reached an opposite conclusion.

Figuring out what to do is hard because I, as a researcher, teacher, and person who wants to maintain good health has to sort out what studies say and  how those studies were done from what the media report, and then how all of that applies to me. Should I take a PSA test? Should I switch from margarine to butter?

If research into clinical medicine produces doubt about evidence-based practice, consider the difficulties of educational research–already playing a secondary role in making policy and practice decisions–when findings from long-term studies of innovation conflict with current practices. Look, for example, at computer use to transform teaching and improve student achievement.

Politically smart state and local policymakers believe that buying new tablets loaded with new software, deploying them to K-12 classrooms, and watching how the devices engage both teachers and students is a “best practice.” The theory is that student engagement through the device and software will dramatically alter classroom instruction and lead to improved  achievement. The problem, of course–sure, you already guessed where I was going with this example–is that evidence of this electronic innovation transforming teaching and achievement growth is not only sparse but also unpersuasive even when some studies show a small “effect size.”

Turn now to the work of John Hattie, a Professor at the University of Auckland (NZ), who has synthesized the research on different factors that influence student achievement and measured their impact on learning. For example, over the last two decades, Hattie has examined over 180,000 studies accumulating 200, 000 “effect sizes”  measuring the influence of teaching practices on student learning. All of these studies represent over 50 million students.

He established which factors influenced student learning–the “effect size–by ranking each from 0.1 (hardly any influence) to 1.0 or a full standard deviation–almost a year’s growth in student learning. He found that the “typical” effect size of an innovation was 0.4.

To compare different classroom approaches shaped student learning, Hattie used the “typical” effect size (0.4) to mean that a practice reached the threshold of influence on student learning (p. 5). From his meta-analyses, he then found that class size had a .20 effect (slide 15) while direct instruction had a .59 effect (slide 21). Again and again, he found that teacher feedback had an effect size of .72 (slide 32). Moreover, teacher-directed strategies of increasing student verbalization (.67) and teaching meta-cognition strategies (.67) had substantial effects (slide 32).

What about student use of computers (p. 7)? Hattie included many “effect sizes” of computer use from distance education (.09), multimedia methods (.15), programmed instruction (.24), and computer-assisted instruction (.37). Except for “hypermedia instruction” (.41), all fell below the “typical ” effect size (.40) of innovations improving student learning (slides 14-18). Across all studies of computers, then, Hattie found an overall effect size of .31 (p. 4).

According to Hattie’s meta-analyses, then, introducing computers to students will  fall well below other instructional strategies that teachers can and do use. Will Hattie’s findings convince educational policymakers to focus more on teaching? Not as long as political choices trump research findings.

Even if politics were removed from the decision-making equation, there would still remain the major limitation of  most educational and medical research. Few studies  answer the question: under what conditions and with which students and patients does a treatment work? That question seldom appears in randomized clinical trials. And that is regrettable.

 

 

29 Comments

Filed under comparing medicine and education, how teachers teach, technology use

Failure and Error among Doctors and Teachers

I had a conversation with a friend* last week about the two posts detailing my failures as a teacher with three students I have had over the years. He has practiced Family Medicine for over a half-century in Pittsburgh and for years helped resident physicians in doing medical research and now works with hospital residents in improving communication with patients.  He pointed out to me how similar teachers experiencing failures with students is to physicians erring in diagnoses or treatments (or both) of their patients.

I was surprised at his making the comparison and then began to think about the many books I have read about medicine and the art and science of clinical practice. In my library at home, I had two with well-thumbed pages authored by doctors who, in the  first dozen pages, detailed mistakes either they had made with patients or errors committed by other physicians on them or their families.

In one, Jerome Groopman, an oncologist, described what occurred with his 9-month old child after a series of doctor misdiagnoses that almost caused his son’s death. A surgeon, who was a friend of a friend, was called in at the last moment to fix an intestinal blockage and saved his son.

In the other book, surgeon Atul Gawande described how he almost lost an Emergency Room patient who had crashed her car when he fumbled a tracheotomy only for patient to be saved by another surgeon who successfully got the breathing tube inserted. Gawande also has a chapter on doctors’ errors. His point, documented by a paper in the New England Journal of Medicine (1991) and subsequent reports  is that nearly all physicians err.

If nearly all doctors make mistakes, do they talk about them? Privately  with people they trust, yes. In public, that is, with other doctors in academic hospitals, the answer is also yes. There is an institutional mechanism where hospital doctors meet weekly called Morbidity and Mortality Conferences (M & M for short) where, in Gawande’s words, doctors “gather behind closed doors to review the mistakes, untoward events, and deaths that occurred on their watch, determine responsibility, and figure out what to do differently (p. 58).” He describes an M & M (pp.58-64) at his hospital and concludes: “The M & M sees avoiding error as largely a matter of will–staying sufficiently informed and alert to anticipate the myriad ways that things can go wrong and then trying to head off each potential problem before it happens” (p. 62). Protected by law, physicians air their mistakes without fear of malpractice suits.

Nothing like that for teachers in U.S. schools. Sure, privately, teachers tell one another how they goofed with a student, misfired on a lesson, realized that they had provided the wrong information, or fumbled the teaching of a concept in a class. Of course,  there are scattered, well-crafted professional learning communities in elementary and secondary schools where teachers feel it is OK to admit they make mistakes and not fear retaliation. They can admit error and learn to do better the next time. In the vast majority of schools, however, no analogous M & M exists (at least as far as I know).

Of course, there are substantial differences between doctors and teachers. For physicians, the consequences of their mistakes might be lethal or life-threatening. Not so, in most instances, for teachers. But also consider other differences:

*Doctors see patients one-on-one; teachers teach groups of 20 to 35 students four to five hours a day.

*Most U.S. doctors get paid on a fee-for-service basis; nearly all full-time public school teachers are salaried.

*Evidenced-based practice of medicine in diagnosing and caring for patients is more fully developed and used by doctors than the science of teaching accessed by teachers.

While these differences are substantial in challenging comparisons, there are basic commonalities that bind teachers to physicians. First, both are helping professions that seek human improvement. Second, like practitioners in other sciences and crafts, both make mistakes. These commonalities make comparisons credible even with so many differences between the occupations.

Helping professions.

From teachers to psychotherapists to doctors to social workers to nurses, these professionals use their expertise to transform minds, develop skills, deepen insights, cope with feelings and mend bodily ills. In doing so, these helping professions share similar predicaments.

*Expertise is never enough. For surgeons, cutting out a tumor from the colon will not rid the body of cancer; successive treatments of chemotherapy are necessary and even then, the cancer may return.

Some high school teachers of science with advanced degrees in biology, chemistry, and physics believe that lessons should be inquiry driven and filled with hands-on experiences while other colleagues, also with advanced degrees, differ. They argue that naïve and uninformed students must absorb the basic principles of biology, chemistry, and physics through rigorous study before they do any “real world” work in class.

In one case, there is insufficient know-how to rid the body of different cancers and, in the other instance, highly knowledgeable teachers split over how students can best learn science. As important as expertise is to professionals dedicated to helping people, it falls short—and here is another shared predicament–not only for the reasons stated above but also because professionals seeking human improvement need their clients, patients, and students to engage in the actual work of learning and becoming knowledgeable, healthier people.

*Helping professionals are dependent upon their clients’ cooperation. Physician autonomy, anchored in expertise and clinical experience, to make decisions unencumbered by internal or external bureaucracies is both treasured and defended by the medical profession. Yet physicians depend upon patients for successful diagnoses and treatments. If expertise is never enough in the helping professions, patients not only constrain physician autonomy but also influence their effectiveness.

While doctors can affect a patient’s motivation, if that patient is emotionally depressed, is resistant to recommended treatments, or uncommitted to getting healthy by ignoring prescribed medications the physician is stuck. Autonomy to make decisions for the welfare of the patient and ultimate health is irrelevant when patients cannot or do not enter into the process of healing.

For K-12 teachers who face captive audiences among whom are some students unwilling to participate in lessons or who defy the teacher’s authority or are uncommitted to learning what the teacher is teaching, then teachers have to figure out what to do in the face of students’ passivity or active resistance.

Failure and error occur in both medical and teaching practices.
Both doctors and teachers, from time to time, err in what they do with patients and students. Patients can bring malpractice suits to get damages for errors. But that occurs sometimes years after the mistake. What hospital-based physicians do have, however, is an institutionalized way of learning (Mortality and Morbidity conferences) from their mistakes so that they do not occur again. So far, among teachers there are no public ways of admitting mistakes and learning from them (privately, amid trusted colleagues, such admissions occur). For teachers, admitting error publicly can lead directly to job loss).

So while doctors, nurses, and other medical staff have M & M conferences to correct mistakes, most teachers lack such collaborative and public ways of correcting mistakes (one exception might be in special education where various staff come together weekly or monthly to go over individual students’ progress).

Books and articles have been written often about how learning from failure can lead to success. Admitting error without fear of punishment is the essential condition for such learning to occur. There is no sin in being wrong or making mistakes, but in the practice of schooling children and youth today, one would never know that.

_______________________

* Dr. Joel Merenstein and I have been close friends since boyhood in Pittsburgh (PA)

32 Comments

Filed under comparing medicine and education

Best Practices and Bad Habits (Part 2)

Transfer of learning appears to be a simple concept. What you learn in the family or learn in school  can be applied in different situations outside of the family and the classroom. Learning Spanish, for example, helps later in learning Italian. Learning to get along with an older brother or sister helps in learning how to get along with others later in life. Learning math in middle school helps one in high school physics. It doesn’t always work that way, however.

images-2

In Part 1,  I described how I taught a two-week unit on specific critical thinking skills useful to understand history and use in daily life in the early 1960s. My theory was that teaching these thinking skills directly one after the other at the very beginning of the semester would lead to students applying them when I began teaching units on the American Revolution, Immigration, Sectionalism and the Civil War, and the Industrial Revolution.

The response of students to the stories and subsequent discussions and debates almost made me swoon. I was energized by students’ enthusiasm as we went from one specific skill to another using contemporary stories drawn from newspapers,  student lives, and Glenville high school. The two week unit was from all indications a success with student engagement high and even scores on the unit test were higher than I had expected.

Then, when I began my U.S. history units on the American Revolution through World War I, the skills I believed that I had taught my students weeks earlier were missing in action. Root canal work was easier than getting students to distinguish between a biased source and one less so or explain why certain statements were opinions, not facts. I was puzzled.  What had happened?

Years later, I discovered from reading psychologists a great deal about the ins-and-outs of transfer of learning (see, for example, here). Teaching specific critical thinking skills and expecting students to apply what they learned to different situations depended upon many conditions that were, I learned later, missing in my lessons. Even the concept of teaching these skills isolated from the historical content–as I did–undermines the very goal I wanted to achieve (see CritThink).

Nonetheless, puzzled as I was by the absence of students applying what they had learned in the later history units I taught, for the next few years I continued to teach that two week unit on critical thinking at the beginning of the semester, marching through the lessons one skill after another. I repeated again and again this unit because the students were engaged, loved to apply what they learned to their daily lives, and I felt good after each of the five periods I taught. An uncommon experience for a veteran teacher.

Even had a colleague I trusted grabbed me by the shoulders then and told me how I was way off in thinking that my students would transfer the skills they learned in the two-week unit to subsequent history units, I would not have believed that colleague.  I would have continued with what I considered a “best practice” that, in reality, had become a “bad” habit.

mooc+cartoon

Like Dr. Danielle Ofra, I would have given reasons to myself why what I was doing helped students. As I look back, I kept doing the same unit year after year and ignored the signs–the mysterious tug I felt every semester seeing repeatedly that students failed to apply the skills in subsequent history units that they had supposedly learned weeks earlier. I persisted even in light of the evidence of little transfer of learning.

Such “bad” habits, of course are common. From over-eating to smoking to excessive Internet surfing to watching far too much television, “bad” habits–destructive to one’s health and well-being–persist among substantial numbers of youth and adults.

images-1

Such habits are like ruts in road that get deeper and deeper through repetition of the behavior. It is hard to get out of the well worn groove. Yet people do break “bad” habits by replacing them with “good” habits that begin a new groove, and get practiced over and over again. It can be done and does occur.

As for me, my “bad” habit of ignoring evidence of my students not applying what they learned in that two-week thinking skills unit, eventually changed. The baffling lack of application got me to read more and talk to colleagues about what occurred in my teaching. I stumbled into new knowledge about transfer of learning. I made many attempts, some failed badly, to build new units in history where these thinking skills were embedded in the historical content. Eventually, I got into a new groove and created different units and taught them (e.g., Colonization, American Revolution, Causes of the Civil War, The Industrial Revolution, The Kennedy Assassination). See here.

But understanding transfer of learning was a hard road to travel in getting out of that rut I had made for myself as a history teacher many years ago.

11 Comments

Filed under comparing medicine and education, how teachers teach

Best Practices and Bad Habits in Practicing Medicine and Teaching Students

Listen to Danielle Ofri lamenting a fact she discovered about her work in New York City’s Bellevue hospital as a physician: “we often ignore good advice when it conflicts with what we’ve always done.”

Ofri was referring to the latest clinical guideline issued by the Society of General Internal Medicine that recommended against annual physical exams for healthy adults. The scientific evidence shows “the harm of annual visits — overdiagnosis, overtreatment, excess costs — can outweigh the benefits.”  These guidelines become  “best practices” for physicians to follow with patients; they are based upon analysis of many clinical studies.

Keep in mind that the body of evidence producing clinical guidelines for how doctors should practice is based on cumulative research and meta-analyses often involving tens of thousands of patients in control and experimental studies. “Evidence-based medicine”–even with all of the criticism of reversals in the advice doctors receive-is a reality at the fingertips of every doctor tapping keys and watching the computer screen as they take a patient’s history and conduct an annual exam.

Yet Ofri continues to have her patients return every year for an annual exam. How come?

She says: After the research was initially published last year, I grappled with the evidence, or lack thereof, reaching a conclusion that I mainly still supported the annual visit, if only because it establishes a solid doctor-patient relationship. But seeing these new, strongly worded recommendations, I may have to re-evaluate. At the very least, I should take a moment to think before I reflexively recommend the annual visit. But I know that I might still end up doing the same thing, despite the evidence.

She concludes: Humans are creatures of habit. Our default is to continue on the path we’ve always trod.

For some physicians, habit trumps evidence or what was once a “good” habit–annual exams for all of her patients–becomes a “bad” habit. True as well for K-12 teachers.

No such clinical research base, however, exists for recommending “best practices” in teaching reading, math, science, or history. Sure there are single studies, even groups of studies that point in a direction that teachers might consider in teaching long division or teaching six year-olds how to parse vowels and consonants. But for most teachers, “best practices” is a meld of what researchers say practitioners ought to do, what “experts” say should be done in classrooms, lessons learned from personal experiences in teaching, deeply-ingrained beliefs–call it ideology–about how best to teach and how students learn, and, yes, you guessed it: habit.

All of these ways of defining “best practice” for teachers came into play when I taught history to high school students many years ago. Let me explain.

In the fifth year of my teaching at Cleveland’s Glenville high school–it was the early 1960s–I had already introduced materials to my classes on what was then called “Negro history” (see here and here). I then began experimenting with the direct teaching of critical thinking skills. I believed that such skills were crucial in negotiating one’s way through life and understanding history. I wanted my students to acquire and use these skills every day. So I began teaching my U.S. history courses with a two-week unit on thinking skills. My theory was that the students learning these skills at the very beginning of the semester would then apply them when I began teaching units on the American Revolution, Immigration, Sectionalism and the Civil War, and the Industrial Revolution.

In the two-week unit, I selected skills I believed were important for understanding the past such as: figuring out the difference between a fact and opinion, making a hunch about what happened and sorting evidence that would support or contradict the hunch, judging how reliable a source of information is, distinguishing between relevant and irrelevant information in reaching a conclusion.

For each of these, I would go over the specific skill with the class and they and I would give examples from our daily lives, school events, and family happenings. Then,  I chose a contemporary event–a criminal case in the local newspaper, a national scandal that was on television, and occurrences in the school–and wrote out a one-page story that would require each student to apply the particular skill we were discussing such as making an informed guess, collecting evidence to support their hunch, and reaching a judgment. I also gave the class additional sources from which they could (or could not because of biases) select information to support their conclusion.

For the two weeks, each period–I was teaching five classes a day at the time–was filled with engaged students participating in flurries of discussion, debates over evidence, student questioning of each others’ conclusions, and similar excitement. I was elated by the apparent success of my critical thinking skills unit.

After the two weeks of direct instruction of skills, I plunged into the Coming of the American Revolution and subsequent history material. From time to time, over the course of the semester, I would ask questions that I felt would prompt use of those thinking skills we had worked on earlier in the year. Blank stares from most students with occasional “Oh yeah” from others. I designed homework that explicitly called for use of these thinking skills; few students applied what they had presumably learned.    I was thoroughly puzzled.

Which brings me to the concept of transfer. Why did students taught discrete thinking skills directly with a high degree of engagement and apparent learning for two weeks have a difficult time transferring those very same skills to history lessons later in the semester? I take up this issue and my “bad” habit in the next post. 

6 Comments

Filed under comparing medicine and education, how teachers teach