Beginning last spring, I began publishing posts of classrooms in which I observed lessons (see here and here). These posts were one part of a larger research project on technology integration (see here).
Two questions have guided the case study design of the project:
- How have classroom, school, and district exemplars of technology integration been fully implemented and put into classroom practice?
- Have these exemplars made a difference in teaching practice?
In this and subsequent posts I will detail the methodology I use, what I mean by technology integration and describe models commonly used to determine its extent in schools.
The following posts are drafts that will be revised since I will be visiting more teachers and schools this fall. I welcome comments from readers who wish to take issue, suggest revisions, and recommend changes.
How I Got Started
In fall 2015, I wrote to district superintendents and heads of charter management organizations explaining why I was writing about instances of technology integration in their schools. At no point did these administrators ask me to define “technology integration” or even ask about the phrase; all seemed to know what I meant. In nearly all instances, the superintendent, school site administrator, technology coordinator, and CMO head invited me into the district. Administrators supplied me with lists of principals and teachers to contact. Again, neither my contacts nor I defined the phrase “technology integration” in conversations. They already had a sense of what the phrase meant.
I contacted individual teachers explaining how I got their names, what I was doing, and asked for their participation. More than half agreed. Because of health issues, I did not start the project until January 2016. For four months I visited schools and classrooms, observed lessons and interviewed staff. I resumed observations this fall and hope to complete all observations by December 2016.
In visiting classrooms, I interviewed teachers before and after the lessons I observed in their classrooms. During the observation, I took notes every few minutes about what both teacher and students were doing. I used a protocol to describe class activities while commenting separately about what both teacher and students were doing. I had used this observation protocol in previous studies. The point of the description and commentary was to capture what happened in the classroom, not determine the degree of teacher effectiveness. I avoided evaluative judgments about the worth of the lesson or teacher activities.
The major advantage of this approach is being in the room and picking up non-verbal and verbal asides of what is going on every few minutes as well as noting classroom conditions that often go unnoticed. I, as an experienced teacher familiar with schooling historically and the common moves that occur in lessons, can also assess the relationship between the teacher and students that other observers using different protocols or videos may miss or exclude. Teachers know that I will not judge their performance.
The major disadvantage of this way of observing lessons is the subjectivity and biases I bring to documenting lessons. So I work hard at separating what I see from what I interpret. I document classroom conditions from student and teacher desk arrangements through what is on bulletin boards, photos and pictures on walls, and whiteboards and which, if any, electronic devices are available in the room. I describe, without judging, teacher and student activities and behaviors. But biases, as in other approaches researching classroom life, remain.
After observing classes, I sit down and have half-hour to 45-minute interviews at times convenient to teachers. After jotting down their history in the district, the school, and other experiences, I turned to the lessons and asked questions about what teachers’ goals were and whether they believed those goals were reached. Then, I asked about the different activities I observed during the lesson. One key question was whether the lesson I observed was representative or not of how the teacher usually teaches.
In answering these questions, teachers gave me reasons they did (or did not do) something in lessons. In most instances, individual teachers told me why they did what they did, thus, communicating a map of their beliefs and assumptions about teaching, learning, and the content they teach. In all of the give-and-take of these discussions with teachers I made no judgment about the success or failure of different activities or the lesson itself.
I then drafted a description of the lesson and sent it to the teacher to correct any factual errors I made in describing the lesson. The teacher returned the draft with corrections.[i]
To provide context for the classrooms I observed, I collected documents and used school and teacher websites to describe what occurred within each school and district in integrating devices and software into teachers’ daily lessons.
All of these sources intersected and overlapped permitting me to assess the degree to which technology integration occurred. Defining what the concept of “technology integration,” however, was elusive and required much work. Even though when I used the phrase it triggered nods from teachers and administrators as if we all shared the same meaning of the phrase. I still had to come up with a working definition of the concept that would permit me to capture more precisely what I saw in classrooms, schools, and districts.
________________________________________________
[i] The protocol is straightforward and subjective. I write out in longhand or type on my laptop what teachers and students do during the lesson. Each sheet of paper or laptop screen is divided into a wide column and a narrow column. In the wide column I record every few minutes what the teacher is doing, what students are doing, and teacher-directed segues from one activity to another. In the narrow column, I comment on what I see.
Subsequent posts will deal with defining technology integration, common models describing its stages, and determining success of technology integration.
Larry Cuban on School Reform and Classroom Practice 
Larry, let me be sure I understand, this is nothing more than an anecdotal case study? Correct? Here’s why, I noted that in your observations, you used no research validated observation instruments. I know they are out there as I have used them in the past. Was there a reason for this? Is your interview protocol validated by research or through use by other researchers? As you noted, if not, your observations are very susceptible to bias. Thus, no matter how much you try to correct for it, your analysis will have some degree of your bias. Lastly, do your research questions have any basis within the literature? I ask this because bias can also creep into the questions that we ask. Having a basis in prior research helps reduce this. I think any study of this scale is commendable and I wish you luck. But with insufficient controls, this at face value looks like it may only serve the community with anecdotal evidence. Why not train and use additional data collectors?
Yes, Bob, it is an “anecdotal” case study and it will document instances of teachers, schools, and districts in a non-randomized sample. As for the protocol that I developed, it is not been tested for its reliability and validity. Every protocol, even those researched-approved for reliability and validity, have biases built into them. That is the nature of the struggle for objectivity in all research. The key, at least for me,is laying out the trade-offs on the design and methodology that I use and identify those biases. Thanks for the comment.
Here’s my second attempt – the login to WordPress to post a comment is maddening. If my other post comes out of the ether, please know that I didn’t mean to carry on so much.
I want to throw out to you a couple of things to consider as you go down this path. As you have noted in almost everything you write, change has happened; it’s just happened in increments. After having been an edtech coordinator for more than 16 years in a high performing high school in a high performing school division (APS), I have seen change. I have seen integration of technology. In the last 6 years, Google docs has been used almost exclusively over Word and PowerPoint. Why? because it is shareable and collaborative and the kids can “turn in their assignment” with the click of a button. Now with Google Classroom (Google’s attempt to do an LMS), teachers can distribute handouts and presentations and collect them the same way. Teachers are crazy about GC – it’s easy, we have it available and LOOK, they are using edtech daily!! What more could you ask for?
So what do you mean by integration – how often edtech is used, especially in a 1:1? or how collaborative is the work with edtech? or how innovatively is the technology being used to alter teaching and learning in some “disruptive” or “personalized” manner? Just the term “integration” needs to be clearly defined. Not sure you can do it.
The terms “disruptive,” “personalized” and “innovative” need to be defined and described. Go to NoRedInk.com and see first hand what “personalized” learning looks like for drill and kill. Are teachers using it? You bet! Is this integration? You bet! Warm-ups, Exit tickets, formative assessments! Yes. Teachers are using all these tools. But I think that the conclusion you will come to is the same as it ever was, same as it ever was… And, you know, I can’t find anything wrong with that. Teachers time and again use the tools that they are given to do the best they can to meet expectations. Those expectations are not always clearly defined.
So I say to you: get your definitions in order.
Thank you so much, Sandy, for your comments and especially the questions you ask. I have thought about the issue of definitions and what teachers mean when they say they have integrated technologies into their daily lessons. Can we continue this exchange after I post Parts 2 and 3 since those parts deal with definitions and stages of integration? Neither post (or both) may satisfy you in answering the questions you ask, Sandy, but the posts may give both us more to go on than Part 1. Again, thank you for raising the questions you do about incremental changes and even links to software that teachers use in APS.
Reblogged this on David R. Taylor-Thoughts on Education.