Abstract
Computer data systems have become a lynchpin to supporting school data use. However, successfully implementing such systems is no easy task. In this case, readers explore the ways in which “technology problems” and “people problems” can be intertwined. The case follows Dr. Molly Winters as she encounters social and organizational challenges relating to district vision, tensions around data use, and tensions involving technology implementation. Her exploration of these issues spans both the school and district levels. Her goal is to analyze and provide recommendations regarding how to support data system use and school improvement throughout her district.
Keywords
In the following case, readers explore some of the social and organizational issues associated with data system implementation. The narrative follows Dr. Molly Winter, a campus principal tasked by her new superintendent with generating a plan to bolster data system use within their district.
Case Narrative
Although the Elmhurst School District has traditionally done well according to state accountability measures, there has been growing concern about disparities in achievement among some students. The issues would seem to be nuanced—they are at the level of certain demographic subgroups and certain subject areas or grade levels. If these disparities are not remedied, Elmhurst and some of its schools might drop in state rankings. Despite these trends, one school in Elmhurst stands out for bucking these trends. This school, which is one of two middle schools in the district, is led by Dr. Molly Winter. Since her arrival in Elmhurst 5 years ago, Dr. Winter has worked hard to transform her school into a professional learning community focused on differentiated instruction and standards-based teaching. Students at her school seem to be flourishing. In comparison with other schools in the district, hers is narrowing achievement gaps and scores—particularly in math and science—continue a slow but upward trend even in the face of increasing state test rigor.
This year, Elmhurst has a new superintendent, Dr. Thomas Butters. After making a strong effort to get to know educators and schools in the district, Dr. Butters reported some of his concerns and plans to the school board. Specifically, he stated that although he found many examples of excellent teaching, quality throughout the district seemed to be inconsistent. One particular problem involved the district’s approach to assessment. He stated, “We have some really good tests in this district, but only certain schools or certain people in schools are using the data. If we’re going to close achievement gaps, Elmhurst needs to be more deliberate about improvement.” Subsequently, Butters indicated that some central office positions might be cut or redesigned to better serve campus needs. Reflecting about the achievements he saw at Molly Winter’s school, Dr. Butters decided to ask her help with his plans.
A New Leader, A New Emphasis on Data Use
During the meeting, Dr. Butters expressed his astonishment that teachers in Elmhurst rarely used DataTech, the district’s new computer data system. In an interview with the board preceding his hire, Butters proclaimed,
We bought DataTech in my last district—it certainly wasn’t the cheapest system out there, but we found it to be worth the investment. You’ve made a good choice. It’s easy to use. Teachers will be able to do everything from attendance, to longitudinal reports, to automatically assigning students to small groups. I know this system and I know they’ll love it.
During his school visits, however, Dr. Butters noticed that teachers and administrators spoke only vaguely about the system and its role in their work. His suspicions were confirmed that teachers were not using the data system when he viewed DataTech use records. With the exception of some at Dr. Winter’s school, few in the district had done more than log in during training.
Dr. Butters still felt that DataTech was worth the considerable costs (which included the initial purchase, plus annual licensing costs of approximately US$20 per student for an initial contract period of 5 years), but wanted Dr. Winter’s help in rethinking its implementation. He noted,
We could opt to take a loss and find an alternate system that the teachers like better; however, I’m not sure they’ve really given this one a chance. I’ve found it to be user-friendly and capable of informing day-to-day practice in my prior district. I really think if we create buy in around DataTech, we can reshape the culture of instruction around here, and test scores will take care of themselves.
Dr. Winter hesitated to agree, but added, “What I like about DataTech so far is that it starts good conversations. Regardless of if a student arrived yesterday from another classroom or another school, those new teachers still have something to talk about.”
Dr. Butters went on, “This gives us a safety net. But I don’t think our people have a clue about how powerful DataTech could be.” Encouraged by the conversation, Dr. Butters asked Molly to use the next 2 weeks to develop recommendations regarding how to jumpstart DataTech use. As Molly left, Butters added that this might be a good experience in case she was interested in applying for the new assistant superintendent position he was creating.
Dr. Winter Investigates: School and Central Office Perspectives
Later on, Dr. Winter reflected about the conversation. She was not surprised that her school was different. Unimpressed by the district’s training, she and her leadership team had created their own activities using DataTech. Dr. Winter knew she needed a better sense for why people might use (or not use) DataTech. Accordingly, she chatted with people at her school and around the district over the next week.
School-level perspectives
At her school, Molly brought up the topic of DataTech during a department chairs meeting. She asked for their thoughts—how, why, and if teachers might be using DataTech. Like Molly, most were dissatisfied with the district training. Lily Carpenter, the social studies chair, stated, “It’s always the same. They spin you through a Prezi. Thirty minutes later, you wake up and they are out the door.” Melissa Parma, the math chair, concurred: “We needed time talk to each other, to play with DataTech in the same room together, and to figure out what all this had to do with our focus areas.”
Overall, the chairs seemed to agree that their faculty probably used DataTech occasionally, such as when discussing the progress of a particular student during team meetings. Lloyd Fisher, the science chair, dissented, “I doubt my people are using it. The district science benchmarks stink, and we still can’t load our in-house common assessments. We’re sticking to the programs that will keep us on target for the spring.” Also chiming in was Jaime Del Rio, the Special Education chair. He added,
I’m glad the tech people finally got IEPs to couple with schedules. Scrambling to get hard copy IEPs to teachers with every schedule change was ridiculous. Plus, I’m concerned that the “Differentiate Small Groups” button might be dumping all my kiddos together. I need to figure this out.
Later that week, Molly met with the district improvement committee, which included faculty and staff from several other campuses. Molly spoke to members quickly and informally. One teacher had heard of DataTech, but also been told not to bother with it. Another teacher complained about the district’s direction regarding data use. “I already know when a kid is failing or going to fail. I don’t need a computer, more tests, or more pressure to pass the test.” In the parking lot, Molly also spoke to Rudy Walker, an elementary school principal. Rudy described activities at his school:
I’ve made it easy for my teachers. My aides print out reports so that teachers don’t have to. We’re looking at data all the time. For example, we have poster sheets throughout the halls. Some celebrate students who passed the district benchmark. Others tell students if they need afterschool tutorials or other intervention. But if you ran use logs it would probably only look as if only the aides ever used DataTech. So . . . are you going to be creating new policies for the system, or checking on other campuses’ use of DataTech, or what?
Although Molly had reservations about whether Rudy’s school was using data in appropriate ways, she tried to keep those reservations to herself. She was also a bit taken aback by Rudy’s parting question—she and Rudy had been friends and colleagues for several years now, but she sensed some apprehension about how her role with DataTech was emerging. Most disconcerting, she was unclear herself about how her role in providing recommendations to Dr. Butters on DataTech usage complemented or conflicted with her role as one of several (to this point, collegial) campus principals.
Central office perspectives
Molly contacted three acquaintances at central office to hear their opinions about DataTech’s implementation. The first call went to the district technology director, Paolo Strega. He described two sets of problems around DataTech. The first related to dealing with the vendor. For example, the district’s DataTech trainings were actually supposed to have been led by DataTech, but representatives had canceled at the last minute. Paolo and his small staff had filled in as best they could. Also, there were lingering technical problems. Paolo stated,
We’d been under the impression DataTech would help us enter and clean the data. Instead, we’re on our own, sometimes typing in data by hand. We still haven’t released half of this system’s report functions, because I can’t get DataTech to return my calls to explain what they actually do. One day, some district watchdog is going to ask the Board or Butters why we keep throwing money at systems we don’t use.
Paolo’s second set of problems related to dealing with Dr. Butters. As Paolo explained, “I kept trying to get into Dr. Butters’ office, but he was so busy this summer . . . Plus, it’s pretty apparent he loves those DataTech people. He thinks it’s a simple plug and go.”
The next two calls went quickly. One call went to Justine Baker, director for curriculum and instruction. Justine was one of the committee members who had helped select DataTech, and she was disappointed in the district’s progress with the system. She stated,
When DataTech gave their demo, the committee was floored. It’s so powerful and easy to use! But for some secret reason, the technology department won’t give anyone access to its best reports. Unlock those features, and I guarantee that you’d see teachers using data to drive instruction.
The final call went to Tommy Lee, director for accountability and assessment. He did not have much to say, but stated,
Talk to Paolo’s people. They handle the tech. Just like Justine’s people are supposed to handle the instruction. After all, she thinks she’s the test score wizard. I’m not paid to evaluate those two, but I can tell you that I’m doing my best to impress upon schools that they need to pass the test. Frankly, I’m a little offended that Butters is having you lean on me instead of calling me himself. Believe me: I know you think you want this job, but you don’t.
Seeking a Course for Improved Data Use
As Dr. Winter finally hung up the phone, she glanced at her calendar. She was initially excited to take on this assignment because she believed that using data and differentiating instruction would help students throughout the district. At the school level, she was struck by how few in the district could agree about the characteristics of quality teaching and data use. At the central office level—and even among some of her colleagues—she was surprised by the interpersonal tensions surrounding DataTech. These were people problems, and not simply technology problems. Dr. Winters began to ponder what ideas she might bring back to Dr. Butters, and how she could accomplish this in a way that would not burn bridges with colleagues.
Teaching Notes
The Elmhurst School District is similar to many across the United States. Aiming to support school improvement, district leaders are often tasked with supporting schools in using student data (Cho & Wayman, in press; Hamilton et al., 2009; Honig & Venkateswaran, 2012; Marsh, 2012). Recognizing the need for timely access to the “right data” (Lachat & Smith, 2005; Supovitz & Klein, 2003), central offices have often turned toward computer data system to remedy the technical burdens of data use (Burch & Hayes, 2009; Means, Padilla, DeBarger, & Bakia, 2009; Wayman, Stringfield, & Yakimowski, 2004). But at the same time, it is often unclear to district leaders how best to work with vendors and how to manage implementation overall (Cho & Wayman, in press).
Functionalities of such systems include analyzing data longitudinally, disaggregating data by various factors, and other instructional supports (Chen, Heritage, & Lee, 2005; Tucker, 2010; Wayman, Cho, & Richards, 2010; Wayman et al., 2004). Thus, scholars have commonly presumed that teachers will use these systems as long as they meet technical specifications (e.g., reliability, ease of use, interoperability). Unfortunately, this stance may be overly reductionist. Technological determinism refers to the assumption that the significance of a technology to work is self-evident and predictable (Brooks, 2011; Cho & Wayman, in press; Orlikowski & Barley, 2001). For example, both Dr. Butters and Justine Baker believe DataTech will, by the force of its own merits, prove undeniably valuable to school-level educators. Similarly, Lily Carpenter’s comment about the DataTech training would also suggest that trainers assumed that the value of the technology would be self-evident. Furthermore, some of the works by Rudy Walker, Jaime Del Rio, and Paolo Strega highlight how new technologies may unintentionally create (rather than eliminate) work.
Other comments highlight the ways in which DataTech was subject to sense-making processes. Interpretive flexibility refers to the ways in which the same technology may mean different things to different people (Cho & Wayman, 2014; Leonardi, 2009; Pinch & Weibe, 1984). For example, comments from various teachers illustrate how talking to each other and practicing the use of DataTech with each other led to very different opinions about what DataTech would (or would not) mean for them. Furthermore, Rudy Walker’s descriptions of his school serve as a reminder that how one views a data system is partly a function of one’s notions about data use itself. In this case, Rudy’s understandings led to some legally and ethically questionable practices. Altogether, these various differences of understanding point toward the need for clear vision and purpose when it comes to instruction and the roles of data in improving instruction (Anderson, Leithwood, & Strauss, 2010; Marsh, 2012; Wayman, Jimerson, & Cho, 2012).
Another issue for discussion may be the intersection of personnel issues with the path dependency from purchasing of a data system that requires significant financial investment. As Pierson (2000) explained, once a choice is made by leaders, investment in maintaining the direction indicated by the choice (in terms of expenditures, but also in terms of changing behavior patterns and human capital), becomes entrenched over time. Making changes becomes more difficult the longer the choice is maintained—the longer the organization has trod down a particular “path.” As Pierson noted, this can also apply to technology. Technology requires not only start up and fixed costs, but as persons in an organization attain comfort with a particular system, it can be difficult to accept the need for change (even if the newer system is ultimately better or more user-friendly) (Cho & Wayman, 2014; Wayman, Cho, & Shaw, 2009). Path dependency, then, may create tensions among experienced users, novice users, and central office supporters of the investment.
Finally, the conflicts among central office leaders shed light on the ways in which political and bureaucratic challenges may undermine district initiatives. Even apart from data systems and data use, central offices can, like any other organization, be fraught with non-rational behavior, competing interests, and interpersonal or political conflicts (Coburn, Honig, & Stein, 2009; Corcoran, Fuhrman, & Belcher, 2001; Cho & Wayman, 2014). Cho & Wayman (in press) described some of the ill consequences fragmentation and departmental isolation can have on data system implementation, as well as the potential benefits of bringing data, technology, and instruction under one roof. Comments from Justine and Paolo hint at just how fragmented some of the operations in Elmhurst have become, and Dr. Winter will have to use her tacit knowledge of the district as well as her professional skills to navigate potential conflicts in developing her recommendations.
Discussion Questions
What are the differences between data practices in Molly Winter’s school and those in Rudy Walker’s school? What might be her reservations about Rudy’s school?
Compare and contrast values and beliefs in Elmhurst regarding instruction, data use, and improvement. To what extent do policies and practices in Elmhurst align with the Dr. Butters’s hopes for DataTech?
Discuss the roadblocks to DataTech’s implementation. What were the “technology problems” and what were the “people problems?” How did these sets of problems interact?
What should Molly report back to Dr. Butters? Develop a plan addressing the following: values and beliefs about instruction and data use, issues relating to DataTech, disagreements within central office, and the role of central office in supporting district-wide improvement.
If you were in Dr. Winter’s shoes, how would you have carried out Dr. Butters’ request for help? What about her investigative approach might you have emulated or changed? How could she have handled the ambiguities around her role in the district?
Imagine that Dr. Butters were to create a new central office position dedicated to improving data use throughout the district. Create a list of qualifications (e.g., skills, experiences, traits) and responsibilities for this new position.
Field Work and Other Activities
Have students visit a school campus or other organization and conduct a brief evaluation of computer data system use (who uses the system, how, when, and for what purposes?), attitudes about data use, and supports for accessing and operating the system. Have students present findings to the group. Alternatively, have students explore these issues in a school or other organizational settings. Then, have them discuss areas of overlap or divergence, including what strategies or challenges from either sector could inform their future work.
Divide students into small groups to develop a 3-year implementation plan for a data system to replace the ones in their current contexts. They should attend to potential problems or issues addressed in the Elmhurst case (e.g., bureaucracy, communication, hardware issues, professional learning, vision) as well as any others they think important. Next, engage students in a “pitch contest” in which they present their plans to a panel representing a school board. Encourage panelists and guests to ask questions to help illustrate the most promising elements of the proposed plans. If possible, include practicing teachers, campus leaders, and school board members among potential panelists. Describe how progress according to the plan might be monitored, re-evaluated, or modified over time.
Develop plans for continuous school improvement. Students should imagine that they are school principals in a district with an emphasis on data use. As such, students should collect and become familiar with a variety of longitudinal qualitative and quantitative data about a particular school. Data points might include student achievement, school climate, and information about teachers or the community. After reviewing and analyzing these data, each student should prepare 10-min presentations describing (a) the vision and context of the school, (b) trends in the data, and (c) plans for improvement. Plans should be sure to describe priority areas, potential solutions, and the potential role of teachers in supporting improvement. Some might find it valuable to think about how to foster data use among teachers, what to do about reluctant teachers, and how progress might be re-evaluated over time. If students conduct their presentations on video, they can reflect about their presentation skills accordingly. Classmates would also be able to offer feedback and commentary.
Footnotes
Declaration of Conflicting Interests
The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
Funding
The author(s) received no financial support for the research, authorship, and/or publication of this article.