Questions To Ask When Reading Research and Blogs on School Reform and Classroom Practice (Paul Thomas)

Paul Thomas is a veteran high school English teacher and now academic. He is also a blogger. This particular post I liked because of the sensible questions he poses when readers inspect research studies, policy articles, and blogs that too often mix fact with opinion in unseemly ways.

I want to offer some guidelines for navigating the education debate based on my own experience as an educator for nearly three decades (almost two decades as a high school teacher and another decade in higher education/teacher education) and my extensive work as a commentator in print and on-line publications.

When you confront claims about education, and the inevitable counter-claims, what should you be looking for?

Are the claims and counter-claims framed within the perspective of the person making them? As a critical educator, I start my claims and conclusions with just that, essentially stating, “I am drawing this conclusion in the context of my critical commitments.” Claims or suggestions that a commentator or scholar is being objective should be met with deep skepticism….

Are the educational claims framed as “miracles”? This is likely the greatest red flag of all. Despite recurring claims of miracles—Texas miracle, Chicago miracle, Harlem miracle—these claims tend always to fall apart once examined closely…. [C]laims of miracles are a corrosive side effect of the growing need for advocacy among schools, particularly charter schools.

Are the claims of educational quality expressed in terms of correlation or causation? One of the most problematic elements of public discourse about research is the careless conflating of correlation and causation. While making claims that charter school X increased graduation rates may be a statistically accurate claim, suggesting or claiming that the status of the school as a charter school (compared with a public school, for example) caused the higher graduation rates is a tall order—statistically difficult to produce. Most research studies of quality make the distinction and offer caveats about causation that are ignored or hidden in public accounts, particularly when advocates choose the study primarily to promote an agenda instead of genuinely engaging in a discussion.

Do the claims address [specific] student populations…? Another careless aspect of public debates about education is the knee-jerk suggestion that school X did it so why can’t everyone else. Many claims about charter schools, for example, fail to reveal the disparity between student populations when compared to public schools. For example, many charter schools now serve high-poverty populations that tend to be primarily minority children. Claims of strong outcomes with these students seem to suggest that public schools that don’t produce similar results are failing—until you discover that the charter schools do not serve the special needs or ELL populations that public schools must serve. This does not necessarily discount the success of the charter schools with the high-poverty students, but these facts do discredit implications or claims about public schools functioning under much more complex and different circumstances….

Do claims of education success by non-public schools address issues of scalability, selection, attrition, stratification/re-segregation of students, and out-of-school factors? Each time claims are made about reforms or non-public schools producing positive results (high test scores, increased graduation rates, closing the gap among races or classes, etc.), those claims must factor in each of the parameters above, or the comparisons mean little. If charter A produces results that are not scalable to U.S. public education, then those results are credible for that situation only, but not for wide-scale reform policy. In the general population still, most people believe private schools outperform public schools (they don’t; it’s is much more complicated than that), but that misconception is driven by a failure to consider the populations of students served and the impact of out-of-school factors on measurable student outcomes.

Do counter-claims made about education commentaries start with fair and accurate characterizations of the positions being debated? Over the past two years, a disturbing pattern in the education reform debate has been the tendency to characterize anyone criticizing Secretary Duncan or Bill Gates as one of the following: anti-reform, defender of the status quo, union mouth-piece, or fatalistic (suggesting that somehow by acknowledging the influence of out-of-school factors, the person is suggesting that children in poverty cannot learn). Many pundits take this strategy: Mischaracterize the person’s position and then attack your own mischaracterization. This has become a common pattern in the education debate as well (I recommend scanning the comments sections of on-line commentaries about education to see how powerful this technique is)….

Are claims supported with evidence—citations, hyperlinks, or both? One of the commitments I have in my commentary/Op-Ed work is to bring the same qualities of scholarship to popular discourse that academia requires. I prefer on-line publications since I can insert hyperlinks and encourage readers to examine the basis for my claims themselves. This lends credibility to my claims, but it also provides me with learning opportunities … and new understandings about the topics I address.

Just as our public schools appear to be mired in conditions that never change, our public debates about education and education reform suffer from insular and unproductive cycles of monologues.

Our public schools need and our children deserve genuine school reform—reform that is nuanced and complex—and without the same nuance and complexity in an authentic dialogue about education and education reform, we are unlikely to reach the school reform we need.

7 Comments

Filed under school reform policies

7 responses to “Questions To Ask When Reading Research and Blogs on School Reform and Classroom Practice (Paul Thomas)

  1. Great post. I just added it as one of my 9/18 Net Nuggets at DrDougGreen.Com. Keep up the good work.

  2. An important reminder in this current age of education reform that appears to more about half truths and hoopla. An age when the only savoir of inequality in our schools is quantitative data. Children are more than test scores, and context counts, and talking to parents, students, and teachers about their experiences is data.
    Thank you for an excellent blog,
    Jesse

  3. Excellent piece….will distribute to my colleagues

    • larrycuban

      Jerry,
      After reading my report of Mr. Macauley’s lesson, you say: “I still don’t exactly know how to respond. If Mr. Macauley is the successful, charismatic figure that you say he is, what does that say about me?”
      Yes, Mr. Macauley has the students’ attention; yes, he is popular. But I did not say he was successful–that is what you said. If “success” means what students learned, I do not know what they took away from that class I observed.Nor do I know how his students did on benchmark tests given in World History a few times a year. That he engaged his class is certainly important (and most probably a precondition for learning) but it does not mean he was successful in getting students to think deeper about some of the concepts he was trying to get across. So I am puzzled why you are “reeling” from this description. Why do you question your own manner of teaching when you and I know that there are many ways to get students engaged and then learning, not just one. As for your feelings about colleagues and your reluctance to describe how you teach to them and administrators, I do feel bad that you work in such an environment that constricts teaching to “one best way.” I may be off-base in saying this, but I was moved by your comment.

  4. Thank you for this post. It is important to be reminded of the aspects of argumentation that play into the education reform debate. I so appreciate your work!

  5. Pingback: Vragen die je jezelf best stelt als je over onderwijsonderzoek en onderwijsblogs leest (Paul Thomas) « Is het nu generatie X, Y of Einstein?

  6. Reblogged this on From experience to meaning… and commented:
    This is maybe one of the most relevant blogposts I have ever read about education.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s