“Teachers Know Best” Survey of Technology in Classrooms

The rollout of the Gates Foundation 2015 survey of teacher opinion about technology sought to gain attention from entrepreneurs, policymakers, administrators, and teachers. I do not know if it did attract such attention but a brief analysis of what teachers told pollsters is in order because the report reveal strong biases that need attention.

CONTENT BIASES

1. Teachers still do not “personalize” instruction. On p. 2 of the report, under the heading “What Today’s Classrooms Look Like” the first finding of the survey is in the following  paragraphs:
Despite the proliferation of technology that enables student
learning experiences to be tailored to meet individual skills, needs,
and interests, … More than two-thirds (69 percent) of participating teachers
report teaching in classrooms where students generally
learn the same content, working at the same pace together
as a class.

The implied meaning is that over two-thirds of teachers teaching where students generally learn the same content, working at the same pace together as a class  is bad especially because there is a  proliferation of technology that enables student learning experiences to be tailored to meet individual skills, needs, and interests.

And if the reader did not get the message, the text on the same page continues:
Despite the availability of digital tools to assist with independent practice, assessment, and tutoring, most classroom time in these areas is still spent without using digital content. Teachers spend 16 percent of class time, on average, on independent practice without digital content, compared to 11 percent using it; another 16 percent of class time on paper-and-pencil assessment, compared to 9 percent on computer-based assessments; and 10 percent of class time on individual in-person tutoring, compared to 4 percent on online tutoring.
So most teachers, the text implies because no other explanations are offered, willfully ignore the technologies that would embrace a student-centered version of “personalized” instruction.As others have pointed out, this bias toward student-centered–call it individualized, “personalized,” or blended–instruction continues to drive techno-advocates (see here).
In the very next paragraph, the report says:

However, the majority of teachers (65 percent) report grouping students of similar abilities together for differentiated instruction or other supports. These groupings are also increasingly responsive to ongoing changes in student learning, with 73 percent of teachers changing the composition of student groups at least monthly.

The report suggests that teachers using multiple groupings to differentiate instruction is a “good” thing. Here nearly two-thirds of teachers are doing the right thing. Again there the history and organizational context for teachers using small groups are missing.

The fact of the matter is that Progressive reformers started grouping of students in the 1920s (following the use of IQ tests) and it has taken decades for the practice to become a mainstay in the nation’s schools. Why? Because organizational factors that teachers have faced (and do so now) account for the slow but steady adoption of an innovation a century old.

As I and many others have pointed out (see here and here) there are historical, political, and organizational reasons why teachers teach as they do that have little to do with whether new technologies are available. State standards, tests, accountability, class size, demographics, teacher turnover, historical patterns of instruction, and many other factors account for this stability in instruction, not the availability of new devices and software.

METHODOLOGICAL ISSUES WITH ONLINE SURVEYS OF TEACHER OPINION

The Gates Foundation-funded study depends upon online responses from teachers. Researchers know the dangers of unreliable estimates that plague such survey responses. For example, when investigators examined classrooms of teachers and students who reported high frequency of usage, these researchers subsequently found large discrepancies between what was reported and what was observed. None of the gap between what is said on a survey and what is practiced in a classroom is intentional. The discrepancy often arises from what sociologists call the bias of “social desirability,” that is, respondents to a survey put down what they think the desirable answer should be rather than what they actually do.

Online surveys are also plagued with selection bias. While the Boston Consulting Group that the Gates Foundation hired to survey over 3000 teachers claims that the respondents mirror the nation’s pool of teachers (p. 3), those who take the time to respond to an online survey may well differ from those teachers who cannot be bothered to answer the questions. That is called selection bias and flaws many online surveys (see here, here, and here). I could not find in the report whether the BCG corrected for selection bias or simply used the excuse of  teachers who responded as being representative of the nation’s teachers.

********************************************************************

The foundation’s report Teachers Know Best offers a biased view (both substantively and methodologically) of what should happen in the nation’s classrooms. Handle with care.

34 Comments

Filed under how teachers teach, technology use

34 responses to ““Teachers Know Best” Survey of Technology in Classrooms

  1. JoeN

    This exemplifies how tech companies regard teachers. They are an inconvenience to be managed: not a profession to be listened to.

    At one of the tech companies I worked for, the marketing department would disseminate results internally of an annual survey they carried out with head teachers, with absolute confidence. I recall how shocked and incredulous they were when I pointed out to them that no head teacher I had ever met would compete the survey personally. That at best, it might just be filled out by a conscientious but very busy deputy, but more likely by a head’s secretary. The latter, would of course, not seek any information to questions they couldn’t answer but would simply guess. Yet my marketing colleagues and every level of management throughout the company, would base major decisions on the information contained in this “survey.”

  2. It is disheartening that in 2015 our ed-tech discourse is still mired in a counterproductive “all-or-nothing” dichotomy. It seems odd that the survey is titled “Teachers Know Best.” There’s no acknowledgement there that teachers might deliberately and thoughtfully limit the use of digital tools based upon a sophisticated understanding of what works best in their fields and with their students.

  3. Pingback: "Teachers Know Best" Survey of Techno...

  4. Laura H. Chapman

    Sure looks like marketing research for the tech industry, paid for by Gates, aided by the Boston Consulting Co and some version of a survey monkey. This is an example of push surveys with cherry picked headlines that often contradict what the numbers show. In any case, thanks for disclosing the methodological flaws.

  5. Thanks Larry for this.
    (1) The only reason education exists is to empower ALL students with successful outcomes (engage, retain, relevant adaptive skills, graduate, relevant job placement, sustained relevant contributions to society)
    Where students = revenue; take away all students and imagine whats left for those in education…not much left to do
    (2) The next premise is that educators are professionals and their task as paid professionals is to empower students with successful outcomes (using proven advances in their field…Imagine what medicine would be like for all of us if medical professionals kept early 20th century methodologies to the disadvantage of their patients)
    (3) .Surveys are never perfect. Surveys with large sample populations are more reliable that surveys with smaller populations. BUT, surveys are better than no surveys, IF we desire to validate our assumptions of what may be occurring, with the goal of improving individual student success outcomes.
    (4) You can’t manage what you don’t measure. I sincerely hope we all agree that current student success outcomes are in need of improvement (I can provide validation for those of you that need it).
    (5) To improve student success outcomes, you must understand, change management methodology, what the problem is, why it needs fixing, who is empowered to fix it, what the proven fixes are, what the obstacles are to the fix.
    (6) The research is clear that individualized, ongoing, reinforced, professionally facilitated, student learning, is the key to 21st century deep adaptive learning outcomes, that advance individual, long term, student performance improvement outcomes
    (7) The research is clear that incumbent one size fits all teaching methodology does not empower students with long term deep adaptive learning
    (8) Individualized, ongoing reinforced, facilitated instruction cannot occur without technology (one teacher for each student is not economically viable)
    (9) We are seeing little if any advances in student success outcomes under the one size fits all teaching paradigm
    (10) Traditional educators (65% from the survey), who are tasked with advancing student success outcomes, are embracing a 20th century, flawed, methodology that is proven NOT to advance adaptive learning outcomes of their students. By embracing this old paradigm they are NOT embracing proven professional advances in their field to the disadvantage of their student and contrary to their employment mission/vision.
    Go figure…and we blame Gates and others who want to advance student success outcomes?

  6. Pingback: More on individualization | Blurts

  7. This stuff is always good for a chuckle. “Despite the proliferation of technology”. Not. The technology is not free. Most public schools are more concerned about keeping the boiler working in the winter and getting the semi operational wireless working than spending money on tenuously effective software and hardware. Setting up differentiated and personalized instruction takes time for an already overloaded teacher. When there are 30 kids in a class the teacher is more concerned about Johnny putting gum in Susie’s hair that anything else. The tech support to operate this technology does not exist for most schools.

    The people who write these reports seem to have never been in the trenches actually teaching in a “normal” school. They need to come down and watch my wife teach a middle school tech course with 30 kids in the class, 3 or 4 of which have major social issues, a couple others which “are having a bad day and should be excused for their misbehavior”, no administrator privileges on her computers (cannot have a non-tech working on the computers, might be a union thing), intermittent wireless even though the course requires internet access, and the usual balky laptop issues.

    Technology is not going to provide solutions to educational issues. Only experience teachers, educated in the uses and abuses of technology can do that. Getting educated in the uses and abuses of technology is a slow and often balky process. It takes lots of time, something the average classroom teacher is not overly endowed with.

    For me the most interesting page in the report was page 24 “Perceptions Differ in Tech-Forward Schools”. Lets be real here. Most schools in the US do not have the budget, tech support or training to be “Tech-forward”. I would love my school to be “tech-forward” but just looking at the staff training budget and hours is a mind boggler. The cost of the tech itself is a mind blower.

    I wish these technology advocates would come down to a normal classroom and figure a way of implementing tech in the classroom without spending more than the present budget, without requiring more time from the teacher and not increasing the load on the available school tech support. Now that would be interesting.

    • G Flint:
      Looks like you forgot the students success outcomes here, kinda the only reason education exists.
      The reason everyone is “overworked” is because the current one size fits all teaching methodology does not empower students to learn…at best you get superficial initial understanding that is soon forgotten…so you end up with effort with no long term student success results…Imaging digging a ditch by hand and then compare your results using a backhoe…Once size fits all teaching is digging the ditch by hand (ineffective and inefficient); Digging the ditch with a back hoe is using educationally innovative technology (effective AND efficient)
      I guess you need to get your head out the current shovel paradigm and into the backhoe paradigm.
      Not only will your “overworked” issue be solved, your students will love you because you will be advancing their deep learning outcomes…your only reason for being in education.
      Note that schools in droves are adapting the new paradigm. The out of toucgh naysayers think the current flawed paradigm still makes sense.

      • No argument. That analogy works really well. The cost of the backhoe, the training of the operator, the maintenance of the backhoe, the expense of just having the backhoe sit when other tools (shovel) is more appropriate, the cost of replacing the backhoe when it wears out, what happens to the job when the backhoe breaks, the cost of fuel, and so on and so on. I was literally a backhoe operator in my pre-teaching life. My backhoe and I were expensive. And I still had to have guys with shovels with me.

        My point is the technology is a very pricey tool. If it is not integrated properly with lots of training, for teachers, administrators, students and parents it is guaranteed to be a waste of school money and teacher time. I do not see “schools in droves” adopting the tech model. I see rich schools with large budgets adopting this tech model, not the small rural schools or the schools from poor tax bases .

        Student success is not dependent on technology. It s dependent on getting material to the student in a manner so they can retain a good percentage of that material. Used properly tech can facilitate that, but other times it is just a distraction. I personally am a tech advocate but as a school techie I am also responsible for getting the best bang for the buck. Right now that buck says shovel a lot.

    • JoeN

      I had to smile when I read the phrase “tech-forward.” I can just imagine the marketing team who dreamt that one up. A few years ago it was “e-confident” and then “e-mature” schools.

      When I told an international audience of educators at a conference in Turkey in 2011, they would never be “e-confident” because of course they will always be playing catch-up with tech companies, most of the audience stood and applauded.

    • larrycuban

      Thanks for comment, Garth.

  8. Alice in PA

    Thanks for digging into the language of the report. I agree with Laura that it is the equivalent of a push poll. And the methodological problems with self-report are myriad. They may be triangulated with additional types of data, but relying only on a survey makes this research very low level.
    But it does make for good talking points and I expect to see it show up in various places because the myth of personalized instruction via screens is a strong one. Of all the tech I have seen, I have seen none that truly personalizes the way a human can. Yes a student can retake the quiz or click through the video lesson again. There may be other programmed choices but they are limited and, I have found, of low instructional quality. For example, I would say teachers personalize instruction every day in the questions they ask and the responses they give to students. And the most interesting and productive questions are the more off the wall ones that stem from a student’s personal experience. Those are usually unforeseen and un-programmable. It is not flashy or sexy, but it is tailored to the students in the group or room.

    • G Flint:

      I’ll respect your position and opinions even though they go against all the best practices research out there

      If advancing individual student success outcomes is your goal (its my goal) using a backhoe is the proven solution.

      The backhoe doesn’t sit idle because it it perpetually used by each and every student to advance their student success outcomes (long term, relevant, adaptive learning skills). It makes individual learning more effective and efficient for the students and for the teachers.

      Classroom time is spent discussing how to use that information to advance individual perform

    • Alice in PA

      Personalized instruction is certainly NOT a myth….It is a research proven methodology. Shame on you for suggesting otherwise. it’s apparent you just don’t understand educationally innovative, truly personalized, adaptive learning technology. Its hard to be an expert at everything…I happen to be an expert in this (since 2000). i guess its time for you to brush up on the current research, outside of your personal bias. Plus technology CAN do what the human can’t, measurably advance individual learning outcomes, with large diverse student populations.

      http://mcdonaldsalesandmarketing.biz/3226/learning-transfer-5/

      I suppose that you are teaching to individual learning styles, feeling that that is advancing individual learning outcomes?

      • Alice in PA

        You are making a large assumption about my teaching and my expertise. Learning styles have been debunked, if they were ever really accepted by the research community. That is not what I was referring to. Teachers have hundreds of interactions with their students every day. It is through those interactions that learning is personalized. There is, of course, also the tailoring of specific activities/reading/discussions to specific students.
        I have looked at your site and found most of the research to not be peer reviewed but rather foundation reports. I see very little peer review literature supporting the truly personalized screen based learning systems as being superior, especially in my area of high school physics science.
        We obviously have a difference of opinion that probably can be traced back to theories of learning. Mine was explained in the previous post on this blog. It is based on 30 years of research in constructivism and the role of language in learning. It seems from your website that your business is more working with adults and job training – I may be wrong about that but it is the impression I get. Job training adults is a different job than compulsory education K-12.
        Again, I am not saying that screens have no place. They are a tool to be wielded by a competent teacher.

      • ” the myth of personalized instruction via screens is a strong one”. Alice you couldn’t be MORE WRONG with this statement.

      • Tom,

        “It’s apparent you just don’t understand educationally innovative, truly personalized, adaptive learning technology.” I myself definitely do not. As for brushing up on the current research, I would not even know where to look. That is why I follow blogs like this to try to find things that actually work in the classroom. I do not have time to find current research. I have to teach, fix computers, organize in-service training, replace servers, and figure out why some kid’s Google docs are doing a weird auto fill when he types a period. This weekend I plan on being in school for about 8 hours building a Powerschool server before the present sick one dies and leaves us with no grades, attendance or lunch software. When I actually read research I found 99% of it to be totally unusable in the classroom. It usually makes the horribly mistaken assumption that all kids want to learn. So point me at something that explains what “educationally innovative, truly personalized, adaptive learning technology” is and how to apply it for free. Because that is my budget.

      • I DO completely and fully understand educationally innovative, truly personalized adaptive learning technology. The example I presented is a real company I worked with. Not long ago I completed a research project for a NYC charter school on innovative, integrated SIS, LMS, LCMS. The only thing that matters is the best practices research, seamlessly integrated into technology that advances individual student success, that everyone’s too busy to access.
        Yes you are busy, but no more so than me.
        Looking for free truly personalized adaptive learning technology?
        https://www.edsurge.com/news/2015-08-28-finally-knewton-opens-adaptive-learning-platform-to-teachers-and-learners?utm_campaign=2cd7ff3bfc-Instruct+186&utm_medium=email&utm_source=EdsurgeTeachers&utm_term=0_3d103d3ffb-2cd7ff3bfc-291830477
        Now, integrate it into your learning methodology, or would you like me to do that for you as well?

      • I’m not sure I see how adaptive learning software could be at all useful in the humanities. It seems to be better suited for low-level mathematics or rote learning, such as vocabulary. I can see it being useful for the sort of job training/certificate modules that you have to complete in some industries.

        In the humanities, nothing works better than small class sizes and skilled teachers who are also experts in their fields. Students who write often, get prompt and substantive feedback, do research, discuss, debate, read, and think will make tremendous progress over the course of a year in content knowledge, critical thinking, intellectual sophistication and academic confidence. The presence of technology is almost irrelevant. Assuming teacher and students have good-quality laptops and wifi access, tech is a non-issue. It’s there to be used when it enhances, and it can be put away when it distracts. It’s not going to “personalize” learning any better than a good, effective teacher.

      • larrycuban

        Thanks for comment.

  9. mance improvement outcomes.

    Quite simply the shovel can’t accomplish what a backhoe can accomplish, when a backhoe is the right tool for the job.

    Sure, a shovel may come in handy and it remains available, when appropriate

    Knowledge is power. Knowing all about 21st century deep learning is key. We then can collectively agree on which tool is most appropriate for which task:

    http://mcdonaldsalesandmarketing.biz/category/learning-strategies-in-2012/

    http://mcdonaldsalesandmarketing.biz/36769/36769-plp-personalized-learning-plan/

    http://mcdonaldsalesandmarketing.biz/page/2/

    • larrycuban

      Tom,
      Am I correct in assuming that many of your comments push products that your company sells?

      • No Larry, you ARE NOT correct in that assumption.I am selling nothing. My positions have no personal bias. My information presented is supported by best practices documented research. Nice try to discredit what I present, but its just not true. The research is hard to discredit

      • larrycuban

        Thanks for the clarification.

  10. “Now, integrate it into your learning methodology, or would you like me to do that for you as well?” Opps, my mistake. I thought I was asking for some help and not a snide reply. I will take what I have learned and bail from this thread.

    Larry, I really appreciate your time in posting these tidbits on information. Keep up the good work. Every time I read your posts I learn something new.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s