Is the Use of Untested Technologies in Classrooms Unethical?*

In a recent Teachers College Record commentary, M.O. Thirunarayanan, Associate Professor of Learning Technologies in the College of Education at Florida International University, argues the following:

When it comes to technology integration, no testing or research is done upfront to determine if they help students learn. Such products are typically purchased by schools and used in classrooms before educators start conducting research to determine if they are effective. This is akin to providing treatment using drugs whose effects have not yet been studied. Both are unethical. Not to mention the millions of dollars that are spent every year by schools to acquire these technologies, just so the leaders of these schools can claim that their schools are on the “cutting edge.”

Every time a new technological tool is developed, manufactured, and sold, its potential to improve learning is hyped up to a great extent. A tool that has a new or unique feature is sold as the tool that will help solve all problems in education. Such a product is overbought and underused in schools for a few years until the next best technological tool becomes the hottest product that has to be purchased and used in all schools….

He then concludes:

Those who develop technological products should be asked to spend their own funds to conduct research to find out the benefits of using their products in schools. They should be allowed to sell only those products that have been extensively researched and found to be helpful to learners and teachers. Any side effects should be noted on warning labels that are affixed to the products. 

One part of me is sympathetic to the argument that Professor Thirunarayanan makes. The hype associated with hardware and software raising academic achievement, engaging students like nothing else, and transforming instruction is well-known and seldom passes the sniff test nowadays. Moreover, many software products are (and have been) sold as beta versions to districts, meaning that they have not been fully tested. Teachers and students point out all of the glitches giving vendors very valuable data to improve the product. So I am sympathetic to points that he makes.

But, then, I think of the many instructional practices that have been used for centuries such as textbooks, worksheets, blackboards, and homework that underwent no tests then or now. Also I think about the flawed assumption that the author makes about technology, in of itself, determining academic achievement when that outcome results from many in-school and out-of-school factors. Finally, comparing clinical trials on medical treatments with the kinds of research that ought to be done on new hardware and software overlooks the very narrow questions that drive experimental/control testing and the recent questioning of the worth of the knowledge gained.

Decisions made by school boards and superintendents over buying and deploying new technologies may be unwise. But not unethical. Or even researching the worth of new technologies without capturing the complexity of the many factors determining academic achievement may be myopic.  But using untested technologies in classrooms “unethical,” I do not think so.

Why?

The best way for me to answer the question is to give a clear example of what I believe is (and has been) an unethical practic in the use of technology.

Readers over the age of 45 will remember Channel 1, a technological innovation where entrepreneur Chris Whittle gave hardware (TV monitors, VCRs, and satellite dishes) to schools that lacked the funds to buy equipment (mostly enrolling children from poor families) in exchange for secondary school students watching news programs that contained commercials for products that teenagers consume. Beginning in 1990, Channel 1 continues its 10-minute news broadcasts (of which two minutes are ads) in 10,000 schools across the country.

What makes this unethical (but not illegal) is that school boards and superintendents made the decision to contract with the for-profit firm of Channel 1 knowing that for two minutes a day, commercials would be shown to a captive audience. Under state law, parents are compelled to send their children to school until they are teenagers. Students must stay in their classrooms while commercials are shown. For Channel 1, public officials have abandoned their moral responsibility for children and youth’s  safety, health, and academic achievement by forcing them to watch ads.

For those who like to cite research studies, there is one field study with randomly assigned seventh and eighth graders to control and experimental groups that found those watching Channel 1 remembered the commercials more than news items and in the previous three months had bought over two items advertised on Channel 1.

For these reasons, I find requiring teenagers who are compelled to attend school to watch this program is an unethical act.

__________

I want to thank Jane David for a conversation about the ethics of  using untested technologies in classrooms. She raised points that I had not considered. I am, of course, responsible for what is written here.

About these ads

29 Comments

Filed under technology use

29 responses to “Is the Use of Untested Technologies in Classrooms Unethical?*

  1. Hi Larry,

    This is a great topic and one I’ve thought about a lot. A couple points that were not raised:

    Unlike flourishing research hospitals, lab schools have nearly died out, as far as I know. This pushes any experimentation to the actual school- and it is often “action” research without the strictures of formal studies.

    The other factor I see is the scale of the experiment. A school rolling out iPads or Chromebooks to the entire school population as an “experiment” seems unethical to me. A school that does the same thing with a smaller group and then expands the project based on the experiences of that group seems perfect fine.

    I’m not sure it is as cut and dried as “is experimentation ethical?”

    All the best,

    Doug

  2. Another great post, Larry. I enjoy reading your blog as it arrives in my inbox every week or so.

    I believe that the educational expert you quoted, M.O. Thirunarayanan, misses another point which to me is obvious: no technological innovation is likely to make up for poor teaching, and conversely, an excellent teacher could teach in an open field without any technology, if required to do so.

    Comparing educational technology to pharmaceutical drugs seems a little disingenuous; drugs have the capacity to kill the patient if misapplied. In fact, the same drug could kill one patient and cure another, making “experimental drug taking” highly risky. This cannot fairly be compared to educational technology, which at worst might cost a lot of money and switch students off learning, or bore them.

    If the argument is actually about the cost of technology, let’s have that debate. But to suggest that new technology should not be provided to teachers and students until it has been rigorously tested seems misguided.

    • larrycuban

      Thanks for the comments, Peter.

    • Thirunarayanan doesn’t seem to be suggesting that technology is either a silver bullet, or one stop shop solution. And I’m not certain that differeing levels of teching ability undermine his point.

      His point is that devoting time, resources and energy to untested solutions, and it appears he means on a large as opposed to test or pilot scale – is unethical.

      This point is, to a large degree, independent of the individual educator. Sure, educators adapt their tools, alter them, use them, or discard them, to greater or lesser degrees. But, although that might have a practical effect to some degree on how initiatives pan out on the ground, in some individual cases, it doesn’t mean that the decision to implement is not unethical. It merely means some educators will manage to work around something that doesn;t work, some will fail to achieve optimal outcomes with technology that does.

      Spending money, time,.resources, and shoehorning educators into solutions that undermine outcomes is probably a bad thing, regardless of how good an indoividual educator is.

      To put it another way, good educational research controls for variables. The actual teacher and teaching is one of them. Reseacrh projects will try to ensure that only one variable ois altered, so as to provode the clearest possible results, and meaningful conclusions. Thirunarayanan is probably very well aware of this, as, in this model of education research, the level of tewaching is, explicitly, one of the major things controlled for, and one of the major reasons why study results are discounted – when researchers fail to control for this.

      • larrycuban

        I believe you make a fine defense of Dr. Thirunarayanan’s position. Perhaps, we just disagree on the worth of the kind of research he and you recommend for testing the effectiveness of high-tech devices and software on children and youth in school settings.

      • Hi Larry,

        And thanks for the responses…

        We possibly do disagree on the level of utility here.

        This type of work does require fairly significant resources, specific training, and an iterative process. iT’shard, demanding, and time and resource consuming. And not every context has the resources to justify it.

        Iwould argue though, that Edtechh, and pedagogy is suffering a lack of these types of research projects. The literature has a huge bias towards case studies, lit reviews and theory papers. These have their place – and I’m not saying they don’t, but we have a distinct lack of the types of research that give hard evidence.

      • larrycuban

        Thanks for reply, Keith.

  3. Suzanne

    To me, it seems sad that the now-continual introduction of more and better technologies into schools (can’t believe we’re going to have to provide enough computers for all students to take tests on, for Common Core compliance, in NY State!) is going to fuel the cry of “you’re denying money to schools!” (or should I say, “…to the children!” ?).

    If a teacher is competent, I think the homework assignments and worksheets of the past have been field-tested; I know I’m constantly tweaking, rewriting, trying to improve the ones I’ve developed over the years. If a certain cohort, or subset thereof, isn’t succeeding with the current assignments, I try to develop new ones that are designed to help them master what they’re not understanding (I teach a foreign language).

    I think there are people who judge a school district on how up-to-date its technology is; I’m in a school district that’s experimenting (so I hear) with the classroom furniture–yup, if we just ‘rearrange the chairs’ we’ll solve all the intractable problems of the last few millennia–and I know there are those who hope that technology (and standardized curricula) will make the schools somehow teacher-proof.

    I feel sorry for today’s kids, for many reasons, and am glad that my own children are through K-12 by now.

  4. I asked myself this question as soon as I began working in the educational technology business and saw the gulf that existed between excellence in the classroom, and what leading technology companies wanted to happen in the classroom. I can assure you, it is an explosive question to raise in such an environment.

    So I was impressed recently when I met with Pearson’s Global Efficacy team in the UK, who have also, in essence, used the pharmaceutical industry as a model. Pearson are, as far as I’m aware, the only company attempting something as challenging and difficult to negotiate commercially, as this.

    http://efficacy.pearson.com/our-path-to-efficacy/our-team/

    I agree absolutely with your choice of “myopic” to describe the decisions made. As long as techno-zealots and “gurus” hold sway, then this kind of myopia will continue because the people who should be challenging such decisions, the expert classroom practitioners, are still either marginalised or disinterested.

    As an example, at the end of January I attended the BETT show in London (which is now a massive, international event) specifically to research literacy software resources, for a client who wanted something to support their efforts to rapidly improve the poor reading skills of children entering secondary schools. The leading, successful products in this field, all marketing themselves as helping to support literacy and aimed at what we would call SEN( special educational needs) teachers in the UK, rely essentially on text-to-speech software and company staff are not even aware that the products they are selling are replacing reading skills, not supporting or developing them. Many SEN teachers themselves will buy these products not because they are designed, tested and proven to improve reading skills, but because they make it easier for a child with reading difficulties to “read” something…by listening to it.

    • larrycuban

      Thanks for the comment, Joe. Your description of the text-to-speech literacy software at the recent BETT show in London aimed at special needs teachers raised a few questions in my mind. In the U.S. purchasing decisions of software in reading, math, and academic subjects are made by school boards, administrators, and occasionally IT officers. Rarely by teachers. Generally, what is the process for buying hardware and software in U.K.? Are teachers minimally, moderately, or heavily involved in such decisions?

      • Larry, there is a mixture. The situation used to be that only local authorities (city or town level administrators who ran schools) made these decisions for all the schools in their area, and in some cases that still happens. One of the problems was that ordinary teachers were never consulted and individuals running these procurements have been predominantly technology advocates, not highly skilled or experience educators in a strong position to evaluate technology educationally. Hence the exponential growth and widespread educational redundancy of VLEs, MLEs and learning platforms. But there has been a considerable move to give schools more autonomy through the Academies programme and many have seized the opportunity. Now a large number of schools belong to groups or “chains” of Academies, and failing or struggling schools are being encouraged to join chains, in an effort to improve them. The current government has also been encouraging the growth of Free Schools which are completely autonomous.

        However, even Free Schools are subject to the advice and influence of a central government body when they procure technology and the old issue of who is actually qualified or fit to advise them still remains.

        So the reality now is that many teachers with either an ICT role or a responsibility for ICT in their school, expect to, and do attend the BETT show. And they are increasingly able to access some kind of budget to spend on specific items.

        http://www.education.gov.uk/schools/leadership/typesofschools/academies

        http://www.education.gov.uk/schools/leadership/typesofschools/freeschools

      • larrycuban

        Many thanks, Joe, for describing teacher involvement in buying and deploying hardware and software in UK.

    • Hi Joe, Sorry to miss you at BETT….have you heard of quadblogging as a way of raising literacy standards? http://quadblogging.com/

  5. Pingback: Is the Use of Untested Technologies in Classroo...

  6. Pingback: Just Because It’s Efficient Doesn’t Mean It’s Effective | Consistently Contradictory

  7. Pingback: Is it unethical to use untested technology in the classroom?

  8. Thirunarayanan raises an interesting point. And i have a lot of sympathy with his take. It seems fairly reasonabl;e to say that if you want to make claims about a products efficacy, you need evidence to indicate that. And that evidence needs to be good, based on good methods, and trasnparent. Products which make unverified claims about their efficacy are probably unethical, and, in toher areas of our lives we have rules protecting us from such claims.

    In medicine, for example, we typically have rules making it illegal for manufacturers to claim health benefits for products where no evidence for such effects exists. So, I can;t offer a rock from my front garden for sale with the claim that owning one prevent’s you from contracting the common cold. These rules are there to protect us from huxters, aggressive marketing, and false and deceiving claims, and represent the expression of an agreed on set of rules that separate what can be legitimately claimed from things which can’t be legitmately claimed.

    In education, for example, Learning Styles theory has been going for…nearly 50 years at this point. It;s taught in all sorts of contexts, and all sorts of educators are told, by authorities they trust to implement it. Some educators are assessed on how well they implement it by their school authporities, and there are active pressures to shape experiences, curricula, and activities to conform to they theory.

    There is no meaningful evidence that learning styles effect learning. Recent metastdies of the literature show no evidence to support it as a theory, and what few sturdy studies there are show no benefit, or benefits that are statistically insignificant – they are too small to be attricbuited to anythiing other than the natural variation you would expect. Neuroscientific concepts of learning contradict it in many respects. And yet time, resources, educator grading, class time, lesson planning, money, effort and careers are poured into a set of techniques

    You post a couple of points to counter Thirunarayanan. One is that we habitually use practices which have not been subjected to this procedure. This is not really a reason to discard his argument. The fact that there are practices which don;t follow best practice is not a reason to abandon best practice. We practiced corporal punishment for millenia, but that’s not a reason to a) continue practising it or b) discard evidence based research suggesting other improvements or changes in other related fields – for example, evidence suggesting that priase for hard work, rather than for intelligence will yeild higher motivation.

    I’d also argue that you won;t have to look too hard to find studies that look at the efficacy of homework, the efficacy and guidelines for visual content presentation (which would cover blackboards, and textbooks). We have innvestigated all of these things, and that investigation is ongoing. There’s a three year study of flipped classrooms ongoing that compares them to conventional homework, for example. The fact is, we have tested, and do, test and look at those things.

    You also suggest the author has flawed assumptions,”lso I think about the flawed assumption that the author makes about technology, in of itself, determining academic achievement when that outcome results from many in-school and out-of-school factors.” and you ascribe to him a deterministic viewpoint on technology in education. Given the methods the author suggests, that assumption would not be implicit in the method. In fact, that assumption would rule a study out of the type of inquire suggested. Technology would be (and is in properly controlled, scientific educational research) considered to be one variable amongst many. Testing the efficacy of a particular technology does not involve assuming that technology is the sole, or even main component of learning. Such a study would fail the evidence bar set significantly. Rather, research strives to identify and control for all variables, while admitting that some may be unknown, and tries to describe the results with regard to itps narrow question. A good study would assume that technology may be, but is possib;ly one, one variable amongst numerous others, all of which would need to be identified and controlled for explicitly in the study design in order to determine what effect, if any, a specific technology had.

    It;s not unusual for good stusies, for example, to find that a technology has no negatove, or positive effect. It;s not unusual for studies to discover new and unexpected variables which meean more study needs to be done. It;s not unusual for studeis to discover that predicted for variables have interaceted unexpectedly and need study redesign to investigate.

    I think you may have done a disservioce to the author’s position there…

    The narrowness of the questions that such research asks are a function of variab;e control. If you don;t control for variables, then you can;t make meaningful cause and effect claims. And we know that the veracity of research claims is tied directly into how well they control. Studies with bad controls tend to have far less robust conclusions, and to make far more untrue claims than those with strict controls. Studies with terrible controls may be used to mkae claims that are patently untrue.

    The process is the reason why we understand how cholera works, it;s why we know that ulcers are bacterial. It’s why we understand cause and effect in so many cases.

    It is not perfect, but it is the least worst process that we have, and, though it has it;s disadvantages – time, limited breadth of scope in individual studies – it is far better than the options available, and a lack of accuracy is not one of them. Should we pursue paths of educational progress based on the assertions of salespeople, or on vested interests, theories with no reason to be implemented other than individual say so, or unforeseen bias, or should we commit to the use of the best tools we have available to understand what does and deosn;t work, and why?

    Evidence is the engine of truth. And truth is the engine of education.

    But I do think your distinction between unwise and unethical is a good one.

    • larrycuban

      Thank you very much for taking the time to rebut the points I made in disagreeing with Professor Thirunarayanan’s commentary in Teachers College Record.

  9. It seems the question is not whether or not untested technologies should be used in schools. We have first-hand experience that new technologies cannot be designed without being “tested” in real environments with real teachers and real students. The question is are we willing to put real effort into deploying and testing new technologies? Or do we make grand purchasing gestures and hope for miracles?

    http://blog.ponder.co/2013/09/17/certifying-edtech-educational-efficacy-fda-edtech/

    • larrycuban

      Thanks, Alex, for the comment and questions. If “we” means most software developers and vendors, my answer to your first question is “no.” My answer to the second question is “yes.”

  10. Billie J. McConnell, Ed.D

    We always want to blame the tool, when we should be focusing on the practice. No need to research, ask students what they get from low-level, “googleable” worksheets, both electronic and on paper. It’s not a problem with the tech, we need to have real conversations about what students should learn and how they should learn. We keep doing the same thing, with new tools, and expecting a different outcome. Hmm. Insanity?

  11. Well, this is unethical, but not illegal, since school boards and superintendents made the decision to contract with the for-profit firm of Channel One knowing that for two minutes a day, and commercials would be shown to a captive audience. To change that, please check proposition 1.011.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s