Why do districts keep buying new technologies when the evidence for success is so paltry? The political and psychological explanations offered in Part 2 offer plausible reasons for non-rational behavior. But plausible is a long stretch from being convincing
I doubt very much whether I can persuade readers that “resource dependence” theory and “inattentional blindness”–even in combination–can explain all of those scarce budget dollars flowing to vendors even when little evidence supports such purchases. Behavioral economics combines research on economics and psychological blind spots of individuals (their political and social biases and “rules of thumb”) to show again and again that people have “bounded rationality” in solving problems. Surely, there are other explanations that readers can offer. And some have in their comments to Parts 1 and 2.
Eliminating non-rational decisions in complex organizations is a fool’s errand. Think about the hasty, ideologically-driven decision to invade Iraq because Hussein supposedly had weapons of mass destruction or the greed-driven proliferation of “collateral derivatives” or bundled sub-prime mortgages that triggered financial panic and the Great Recession of 2008. Post-mortem analyses of these decisions can lead to greater understanding among decisionmakers about their hidden beliefs particularly when key evidence-driven questions are asked publicly. Yes, I do believe that policymakers can make better decisions, especially when they examine openly their assumptions when making policy. Which brings me full circle to the instance of non-rational behavior in buying electronic devices.
More and more evidence accumulates as new studies reveal that unbiquitious access and use of computers falls far short of producing the desired outcomes that policy elites, district officials and vendors promised patrons. Beyond the studies cited in the first post (July 4), additional ones have been published about children in Romania whose low-income families got vouchers to purchase computers to be used at home. The researchers, using an experimental and control design for the study, found “strong evidence that children in households who [had] the voucher received significantly lower school grades in math, English, and Romanian” than those children in low-income families who did not have a voucher (or computer) at home.
Or the North Carolina study that examined the availability of broadband service between 2000 and 2005 and effects on middle school test scores in that period. In this large correlational study, access to the Internet and home computers had a negative effect on children from poor families in math and reading but not for non-poor students, thus widening, not closing, the achievement gap between income groups.
Then there is the recent $20 million Texas study of 21 middle schools where students were permitted to take the machines home. Savvy tech people configured the machines to block email, games, and many questionable websites. In another 21 middle schools–the control group–students were not allowed to take the laptops home. In the 21 schools where students took the machines home, sure, you guessed it most of the students figured out how to get around the firewalls installed by experts. The conclusion of the study? “[T]here was no evidence linking technology immersion with student self-directed learning or their general satisfaction with schoolwork.” As for test scores, the evidence was decidedly mixed in the academic subjects with lower scores in writing in those schools with the highest school use of laptops.
For those readers who note the design of studies as a factor in giving weight to the conclusions, two of the above three studies were quasi-experimental and one was correlational.
These powerful “learning tools”–desktops, laptops, hand-held devices–give students and teachers unparalled access to information and, as tools, can be used in many ways individually and collaborativel, in and out of school, to advance knowledge in academic subjects. Yet the results in test scores and other measures are meager. The return on investment is atrocious. Still districts buy netbooks, iPads, laptops, and the next new thing. What to do?
As I have suggested in earlier posts, the chances of districts abandoning high tech purchases are close to nil. Whichever explanation readers find congenial–political, psychological, or their pet ones–new devices ( a robot teacher, perhaps) will continue to flow into schools even when district budgets are tight and evidence of effectiveness is scarce. The answer, at least to me, is the classic democratic one of policymakers being transparent with those who matter–parents, students, educators, voters, and taxpayers–in making decisions about technology. Transparent means letting all stakeholders in tax-supported public education know the assumptions, the desired outcomes, the strategies used, and, most important, the results over time of these educational investments. Technological Impact statements, similar to environmental impact ones, done before a decision is made would be a major step forward in transparency in decisionmaking. More public discussion, not less is required. Truth be told, however, I know of no districts that present to their patrons even this small gesture toward rational decision-making when it comes to putting technologies into schools.
Larry Cuban on School Reform and Classroom Practice 
Certainly there is not strong evidence supporting the purchase of technology by school districts in the form of test scores. Dr. Cuban’s statement near the end of the blog: “Yet the results in test scores and other measures are meager.” indicates such. But what are the other measures? Is success in schools and learning only measured by standardized tests of curriculum content and “traditional skills” or should we also be considering the problem solving skills, innitiative, and motivation of the students who are using the technology. Are they more creative? Are they more likely to obtain jobs in the “high tech sector”? If students are only “game playing” on the technology at home–should the schools not be introducing other learning skills via technology?
The question of evidence for 21st century skills is one that should be considered in assessment and instruction in our elementary and secondary schools.
Dear Larry,
As a longtime admirer, I appreciate your blog and your straightforward approach to the challenges of public education. As I continue to work in and with schools here in NYC, I suggest one more powerful reason for the computers and other gadgets despite the research results: motivation for reluctant students to come to and even perhaps stay in school. How about a quick study of the relationship between gadgets and graduation? Now there’s a title you can use.
Regards,
Jane Remer
I returned from the http://naf.org summer conference last night, so I am still ruminating on things.
Larry, I think you are making a case against “Cargo Cult Technology Use.” I hesitate to use the canard, but I think it’s true. Schools are fairly tribal and often treat innovation, whether it be innate language acquisition or technology, as an article of faith without understanding it. The worst part of it is that they build non-functional things that superficially resemble a real thing in the hope it will function like the real thing. You can chuckle here as it was intended to be understatement.
If we look at technology education “programs” and its associated staff, the outcomes are quite different from what we suppose to be true about schools (in general) that dump technology on populations as if they were delivering “smart pills”. Technology other than laptops has been integrated into career and technical education curriculum for quite a long time and there is good anecdotal evidence in its favor.
A career and technical education (CTE) teacher can use a t-shirt printing lesson to embed chemistry, economics, marketing, history, and art in a lesson in a way that is compelling from a narrative standpoint and that allows students to discover and build knowledge that stays with them.
Building that lesson is time-consuming and requires cross-disciplinary effort, but in a school where it is allowed to happen, outcomes can be impressive for the cohorts that are part of the effort.
Rather than examining failed efforts of technology integration, we have to look at design, programming, or the type of course Henry Jenkins or Howard Rheingold teach. (Disclaimer: it’s what I teach too.) These efforts are not trivial and have demonstrated favorable outcomes. That’s my belief. You may think Jenkins and Rheingold have useless courses or that my colleagues’ students that consistently win national design awards or go on to get computer science degrees are not getting an adequate education.
Using computer programming as an example; I think I can make some headway. Those of us that have had to find and hire a computer programmer for k-12 know they are a scarce commodity. Particularly good ones. The same is true, but to a lesser extent, for any course series that embeds technology. I can illustrate using the t-shirt example above. If you hire a silkscreen technician, the students will learn to produce beautiful shirts. But if your teacher is asking the chemistry teacher to provide standards that a silkscreen ink lesson can be built on, the outcome will allow the students that presumably take both classes to gain a deep understanding of the chemical processes inherent in a broad range of applications. (Superfine particles of heavy metal, mineral compounds, oils, polymers, polymerization, solvents and their manufacturing processes.)
This kind of curriculum is a bit expensive to develop, but I don’t think that’s the problem. Back at the first paragraph, I put “program” into quotes because “programs” are efforts educators use to enrich and supplement teaching. They generally are attached to a teacher and never become part of institutional memory. For example, teaching Shakespeare is part of the institutional memory but teaching kids why the HTML protocol was designed the way it was is not in the curriculum of web design though it is as important to the foundation of understanding how the global network functions as understanding why free speech is important to the U.S.
I am beginning to think some of these “programs” need to be moved to the center of our conversation about education reform because they are fundamentally solid methods for delivering engaging courses that are packed with academic value. This is directly connected to your concern with faith-based technology purchasing that has no pedagogic goal in the case of schools where there are not good outcomes. Schools that experiment with technology without examining process are in effect like a business buying inventory it will never use to produce output.
Whether a computer is necessary for delivering a richly layered, highly connected lesson in English Literature is not my area of expertise. I suspect that a laptop can facilitate access to resources only if those resources are KNOWN TO EXIST by teachers and students who also refrain from obsessive off task behaviors.
It seems to me The National Academy Foundation is actively pursuing lesson design that fulfills academic needs *and* connects students to professional models because they believe engaging in the learned behavior off campus is a way to activate learning.
Bob,
you raise some excellent points that I omitted. Particulary, those school programs where laptops and other devices are crucial to the outcomes (few measured by standardized tests) such as the career technical programs in high schools, augmentative communication devices used for students with disabilities and special programs where technologies are essential such as digital video, music, CAD, and photography. Thanks.
But you see I am advocating elimination of the word “program” and moving a method that resembles another curriculum design effort/program into a central position at the same time?
Bob Calder’s model of technology integration would work great at a school that put the structures in place necessary to implement professional learning communities (PLC). Unfortunately, the notion of a PLC sometimes gets lip service while business goes on as usual, e.g., expecting the level of collaboration necessary for PLC’s without somehow allowing for that to happen during school hours. It’s unfair and unrealistic to expect these types of systemic changes without addressing the organizational structures that inhibit them.
Bob, What are your thoughts about the actual instruction of technology skills? There seem to be two schools of thought on this. Today’s dominant philosophy of technology instruction is that students will already have the necessary tech skills or they will quickly pick it up on their own from friends. In other words, students don’t have a dedicated tech or computers class; all technology instruction should occur in the context of the content at hand. This is opposed to having a separate course dedicated to teaching students basic technology proficiency.
It seems that the promise of technology has thus far proved hollow for the masses. Technology has been very beneficial for governments, industries, and certain small subgroups of mankind.
An afterthought from my non-rational side.
We shouldn’t abandon technology just because we don’t have any empirical evidence that it has a demonstrable effect on learning. Teachers do a lot of “good teaching” that doesn’t translate into quantifiable results. If I think in terms of quantifiable results, then much of what teachers do on a daily basis is irrational. However, I can’t ignore the huge capital expenditures for technology. The business of America is education. Education is one of the only viable industries we have left; don’t let the market destroy it too.
Steve, I’m thinking PLC is the same as Small Learning Communities? I’m with you all the way on that because the success stories I see are organized that way. Inhibitory effects? Surely you jest. Seriously, horror stories abound of funding misuse.
On instructional skills: My personal experience is with mixed cohorts that have kids with both types of background. There is a critical mass effect that requires me to start lower on the skill ladder if there are not enough high skill students in a class. But institutionalizing those decisions (as has certainly been done in the past) is sheer “ignorance at work”. Perhaps if yellow road signs were required in front of these schools?
To tell you the truth, I think this issue plays into the previous subject (Larry’s topic) because smart management of class composition is very, very difficult in schools with high numbers of technology-challenged students. I mean that in the kindest and most careful way possible. For instance, I don’t think hard skills are really a problem even with these kids. The critical mass effect is rather a result of sea-change in attitude toward the curriculum in the entire cohort, high skills identified students included. The high skills students just get it quicker and provide modeling since the attitude change I require will move the students into different social communities online with some accompanying discomfort. They know it and resist leaving their comfort zone. There is a school of thought in Science education that says we are NOT asking students to adopt a new culture. I’m not sure if I can move in that direction since I am explicitly telling my students to adopt a new outlook on something they see as culture.
In your “two schools of thought” remark, I think you are pointing to the central fallacy of “computer skills” being a set of reflexes or magical passes (of the hand) that separate the classes because of arcane knowledge.
In fact as I said above, the knowledgable kids have to learn they don’t know what they think they know about the Internet. Computer knowledge is trivial if you consider the abysmal state of most programmers’ hardware knowledge. This brings me to mention that there are lots of “IT” programs and few of programming because hardware knowledge is valued over the meta-knowledge of programming. Yet more ignorance at work.
A final remark about less privileged classes. My reason to teach in a Title I school:
1. The Internet is the most powerful tool in the known universe. (Global reach et cetera)
2. Entry cost is trivial.
3. Nobody knows you’re a dog on the Internet.
This is the explicit message of my course series. I pray to the agnostic god of globalization that my students learn this simple message.
This Website addresses the similarities and differences between Professional Learning Communities (PLC) and small learning communities (http://www.allthingsplc.info/wordpress/?p=62)”even with the support and backing of one of the world’s greatest private philanthropic organizations, structural change alone will not reform schools. Those who pin their hopes on high school reform based on the size of the school are destined to be disappointed. Ultimately the culture must change to impact classroom practice and student and staff expectations, and the best strategy for improving schools at any level will focus less on the structure of the organization and more on building the capacity of people within the schools to create a new culture.”
While small learning communities or schools-within-schools (http://ed.stanford.edu/suse/faculty/displayFacultyNews.php?tablename=notify1&id=426)
“are typically theme- or career-based programs aimed at making school more relevant to teenagers,” Professional Learning Communities (PLC) are more akin to leadership models that promote true cross-curricular collaboration in terms of content, assessment and student advising (http://www.allthingsplc.info/pdf/articles/DuFourWhatIsAProfessionalLearningCommunity.pdf).
A small learning community would surely benefit from the implementation of a PLC but the former isn’t the same as the latter. A PLC may require the shifting of organizational structures; (block schedule, etc. to accommodate advisory period and collaboration) I don’t believe that the themes structure is necessary for a PLC’s success but it could conceivably enhance its success.
Thanks Steve. Strangely enough, the National Academy Foundation demands administration buy-in for success on account of scheduling needs and continuity. The net effect is a PLC with highly encouraged curriculum integration running in NAF academy schools without the PLC label.
It makes me wonder how many things contribute to good outcomes that don’t have convenient labels. It also makes me wonder about the social markers researchers use to determine what is happening at a school.
I would have thought this was obvious:
Legitimate schools give out PCs, and no school wants to be seen as illegitimate, so … laptops are sent home. It’s a status-legitimacy thing. Mimetic isomorphism.
Besides, I don’t know why the kids that found a way around the firewall weren’t given credit in the $20M study. Education reseachers need to lose their cognitive tunnel vision, and see what is going on in front of them.
Raising one’s hand: “I’m legitimate too!” 😉
We don’t acknowledge anything as proletarian as proxy filter avoidance. There are two ways students do it: 1. Use one of the free servers that go up every day under new names, or 2. install one at home for your private use. That’s what my kids do.
I think the fun part is seeing how we engage in the process in class. I can’t do anything about a student’s home machine and I don’t really care as long as there is no violation of the law. (The law addresses only pornography.) In return, students regulate their access to harmless material.
Your mention of tunnel vision is a good example of one of the actual problems which is refusal to discuss matters that are not in the “consultant du jour” category. For some reason, school districts are unable to take action without his or her imprimatur.
Part of it is the way school districts bring a given process into the realm of institutional memory. This is analogous to moving something from short term memory into long term memory in your brain.
There are probably a thousand “projects” that are short term (forgettable) items, yet produce improvements in educational outcomes. Yet these “projects” are never moved into permanent practice for various reasons. The principal reason is probably cost.
I find it vastly amusing to see school districts spend money on what you have pointed out as essentially a *signaling* effect that has no positive educational outcome. The really funny part is that it could produce those outcomes if fitted with a strategy.
There is nothing funny about inefficience and the wasting of money.
Seminal papers of neo-institutionalism, by John W Meyer and Brian Rowan [“Institutionalized Organizations: Formal Structure as Myth and Ceremony,” AJS 1977 83(2): 340-363. See link for twin-paper done at the same time.] point to the “myths” that pass for reality in highly institutionalized organizations. This goes far to explain your observations regarding social cognition and school districts:
“For some reason, school districts are unable to take action without his or her imprimatur.”
Installation of new technology in schools brings more information for the students. Your article reflects the will that institute who implement technological advancements. Teachers and students get an extensive approach of acknowledgment regarding their core subjects.