Abstract
Understanding students’ participation in collaborative classroom settings is important in a variety of educational contexts, with implications for teaching, research, and equity. Using data from a group-centered developmental mathematics class, this research brief illustrates novel quantitative representations of students’ group participation and their instructor’s classroom activity. Presenting an overview of classroom activities based on audio data, the diagrams show a variety of patterns, including each group’s progression through assigned tasks, some students’ exclusion from discussions of those tasks, and the teacher’s patterns of interaction with groups. The representations provide new ways that researchers might approach and present multilayered classroom data.
Keywords
Classroom audio and video data offer a rich source of information about student engagement with peers, teachers, and curricula. However, it can be difficult for researchers to take stock of the complex participation patterns in such data and to convey those patterns to others given that time, space, and human interactions all come into play. Additionally, collecting video data, in particular, is very labor-intensive.
In this research brief, we present a methodological technique that harnesses traditional, qualitatively coded, audio transcript data to create visualizations of collaborative classroom participation. These visualizations, which we call “classroom collaborative diagrams” (CCDs), track individuals through time, providing a bird’s-eye view of patterns in student and instructor actions in groups and across a classroom. Given that these representations are potentially useful in a variety of classroom-based studies, we focus here on what these diagrams can show us and discuss potential future uses, particularly for research audiences.
Exploring Collaborative Classroom Spaces
Strategies for making sense of classroom data are diverse and increasingly powerful. For example, machine learning, which uses computer algorithms to identify patterns in data, has been used with online classroom data to automatically identify group participation patterns (e.g., Dascalu et al., 2018) and group roles (e.g., Dowell et al., 2019; Gašević et al., 2019). Others have applied machine learning techniques to data from physical classrooms. For example, Kelly et al. (2018) developed a technique for automatically identifying questions in classroom audio data.
A second approach has focused on data visualizations, which provide succinct representations of raw data (Azzam et al., 2013). Methods for visualizing physical classroom data have tended to focus on how space is used by the teacher (e.g., Lim et al., 2012; Martínez-Maldonado et al., 2022) rather than interactions between individuals. Although individual interactions can be displayed separately, there has not been a good way to merge teacher location data and individual interactions into a single representation. Luckin’s (2003) scatterplots tracking qualitatively coded utterances over time in groups come closest to this goal. However, these plots have not been widely adopted, perhaps because it is difficult to track individuals in them. Our diagrams expand and improve Luckin’s diagrams and allow for identification of broad, within-group and classroom-level patterns. We argue that when combined with other techniques, our diagrams support research methods that involve classroom representations.
Study Background
Reform efforts in college developmental mathematics promote the use of problem-solving and group work rather than lecture (Cullinane & Treisman, 2010). However, K–12 research indicates that mathematics teachers struggle to move away from teacher-centered instruction (e.g., Drake & Sherin, 2006) and that students’ off-task talk can take up a large portion of group-work time (Wood & Kalinec, 2012). Given the findings from K–12, we were curious about whether the teacher and students in a problem-centered developmental classroom enacted the curriculum as intended and whether all students experienced the enactment equitably. To investigate these questions, we collected a variety of data in a single developmental mathematics classroom at a 2-year college. Collected data included classroom observations and audio, interviews, and curricular documents. This work resulted in over 80 hours of group work and instructor audio data, which facilitated close analysis of individuals but was difficult to parse for broader patterns.
CCDs emerged as a visualization strategy to identify student and instructor patterns in groups and across the classroom. Given our interest in the enacted curriculum, we used CCDs to document patterns in groups’ progression through the printed curriculum materials, the instructor interactions with students, and instructor movement in the classroom. We also used the CCDs to identify patterns that suggested inequitable student participation.
Data Collection and Preparation
Generating CCDs requires converting qualitatively coded audio transcripts into a quantitative data file that can be read by a statistical program. These data were converted into two types of CCDs: group-level and classroom-level. Group-level CCDs track individuals in a single group, whereas classroom-level CCDs track multiple groups in a classroom without disaggregating by individual. The online supplement (available on the journal website) includes a detailed description of the data preparation and annotated syntax used to create the diagrams.
Data
Instructor and student audio data were collected in a majority-White classroom during 12 class periods. Recording of student groups began when the class transitioned to small-group work, usually about 15 minutes into the 110-minute class period, and continued until class ended. The analysis presented here uses audio from 14 students in four groups during a single class period. The recorded students included nine White students, three African American students, one Asian student, and one Hispanic student (four males, 10 females). 1 Observation notes, lesson worksheets, and instructor audio were also used to create the visualizations.
Classroom Activities
CCDs plot both classroom-level and within-group activities. Classroom activities, which reflect how the instructor allocated time during the class period, fell into four categories: beginning of class activities, assessment, lecture, and group-work time. The start and end time of each class activity was recorded and used to create the colored background regions.
Group Activities
Group activities reflect the tasks and conversations students engaged in during group-work time and were selected based on our interests in how the students and the instructor engaged with the curriculum materials. These activities fell into the following five categories, with differentiation in categories as needed:
working on problems from lesson worksheets, coded at the problem level;
working on homework assignments, coded at the assignment level;
helping other groups on assignments;
planning and other class- or assignment-related talk not about solving assigned problems;
engaging in off-task talk (e.g., non-class- or non-mathematics-related discussions).
To code, the full audio of students in their groups was transcribed, attending to the speaker and timestamps. 2 Each speaker turn was then assigned a group activity code. Because these codes were of relatively low inference, the first author was the sole coder. In addition, for each speaker turn, we included an indicator identifying whether the instructor was present during the turn.
Creating and Learning From Group-Level Diagrams
Creating Group-Level CCDs
Figures 1 and 2 show group-level CCDs for two groups on the same day. To create the group-level CCDs, we imported the coded transcript data into a statistical program and plotted individuals’ utterances over time, noting which activity (e.g., the specific worksheet problem) was the focus of that utterance. Dots indicate the start time of an individual’s speaking turn plotted against the group activity. In sum:
Time is tracked on the x-axis.
Group activity is tracked on the y-axis. The ordering of activities reflects the lesson worksheet problems in ascending order, followed by homework, and then other class-related talk. Off-task talk is plotted highest on the y-axis.
Class activities are indicated using vertical colored regions.
Individuals are differentiated by color and tracked via dots. The instructor is always represented by a black dot.
Group-level classroom collaboration diagram.
Group-level classroom collaboration diagram for group with splintered participation pattern.
Group-level CCDs improve Luckin’s (2003) scatterplots of qualitative codes. Whereas Luckin’s plots show the distribution of codes over time, these graphs use the y-axis and color coding strategically to reveal the focus of that talk. This allows us to see, for example, whether all group members are discussing the same task around the same time.
What We Learned From Group-Level CCDs
For the larger study, we used group-level CCDs to examine whether research findings from K–12 classrooms transfer to the college mathematics population. In K–12 collaborative classrooms, off-task behavior can occupy a large portion of a group’s time. CCDs such as Figures 1 and 2 show that the adult learners remained relatively focused for upward of 80 minutes. When students engaged in off-task talk, they regularly cycled back to their assigned tasks without direct teacher intervention (and such talk may have contributed to fostering productive working relationships).
The group-level CCDs also helped identify groups that deviated from the more common group-interaction patterns. For example, while the group in Figure 1 works together through the assigned problems, the group in Figure 2 visibly splinters. Sarah (a White female), Dave (a White male), and Felicia (an African American female) generally progress through problems together, while Tyrone (an African American male) works through problems more slowly, often discussing problems with Sarah. In the larger study, these diagrams raised questions for us about the experiences of Tyrone and other students who experienced group splintering because Tyrone and another student whose group demonstrated splintering patterns ultimately did not complete the class. Notably, CCDs captured splintering patterns in Tyrone’s group as early as the first week of the class (with different group mates). A discussion of Tyrone and other students’ experiences can be found in Makowski (2017).
Creating and Learning From Classroom-Level Diagrams
Creating Classroom-Level CCDs
Figure 3 shows the classroom-level CCD for the same day diagramed in Figures 1 and 2 (Figures 1 and 2 are the group-level CCDs for Groups 1 and 2, respectively, in Figure 3). Classroom-level CCDs use similar conventions as group-level CCDs with two modifications. First, individuals in a group are assigned the same color and are no longer separated by spacing. Second, the y-axis is modified to track both groups and group activities. This is accomplished by compressing and repeating the group-level y-axis once for each group. These modifications mean that individuals can no longer be distinguished but that patterns across and between groups can be observed.
Classroom-level classroom collaboration diagram.
Larger markers highlight events of interest, adding additional information. For our study, we were interested in the instructor’s use of time. Using larger black dots, instead of small colored dots, to indicate the instructor’s presence during a speaker turn allowed us to observe visit frequency and length with each group.
What We Learned From Classroom-Level CCDs
In the larger study, the classroom-level CCDs were examined for evidence of where the instructor spent time and how that time was distributed. Cumulatively, her patterns of interaction visible on the CCDs, triangulated with other data sources, demonstrated that the instructor could let go of a teacher-centered role, which K–12 research has shown can be an issue for teachers (e.g., Drake & Sherin, 2006). For example, the CCD in Figure 3 shows groups working through the assigned worksheets while the teacher circulated, talking regularly with each group about the assigned content. Moreover, she went off topic at least once with each group.
The classroom-level CCDs also made it easier to compare groups in a single class period. For example, Figure 3 makes it clear that although Groups 1 and 2 regularly went off task, they still completed their work for the day. In contrast, Groups 3 and 4 worked more quickly and used their remaining time to work on class homework.
Discussion
CCDs offer an efficient way to display a large amount of classroom data in a single visualization, highlighting events and individuals of interest. Our group-level CCDs allowed us to quickly assess how individuals progressed through assigned tasks and which groups worked together versus splintering. Our group- and classroom-level CCDs showed how the teacher divided her time among groups and how groups progressed through the course material.
Group-level CCDs improve the scatterplots proposed by Luckin (2003), whereas the classroom-level CCDs go beyond them. In merging both spatial and social interactions into a single visualization, CCDs provide a bird’s-eye view of complex collaborative spaces using only audio (rather than video) data. We also found that CCDs were accessible to broad audiences, allowing us to quickly and effectively present patterns once analysis was complete.
Using CCDs in Research
In our larger study, CCDs highlighted places where deeper analyses seemed warranted. More generally, CCDs provide a mechanism for displaying some of the messiness of interactions among individuals, curriculum elements, and the classroom space using easily collected data. CCDs complement and can leverage emerging methodological techniques such as the automated coding of data (e.g., Dascalu et al., 2018; Gašević et al., 2019). Advances in automated transcription and coding of classroom audio data (e.g., Kelly et al., 2018) can further push the boundaries of what CCDs might be used for in representing results or informing analyses of collaborative classrooms. With these advancements, CCDs could, with additional development and programing, become a standard part of analytic software.
The methods presented here reflected our research interests in the enacted curriculum in a postsecondary classroom, which informed the codes used to create our CCDs. These elements can be modified to reflect different priorities or combined with other research tools to provide additional insight into classroom patterns—priorities that do not exclusively apply to developmental classrooms. For example, the representations can be adopted to include additional markers to highlight other aspects of interest (e.g., discourse moves, procedural vs. conceptual contributions). When used early in a deep transcript analysis, CCDs could help researchers identify transcript segments that warrant additional qualitative analyses. After analysis, CCDs can be used to help researchers represent the data, increasing the visibility of patterns in qualitative coding that otherwise require extensive text to present.
Limitations
CCDs can only highlight what has been coded and will therefore miss aspects outside the researcher’s focus. CCDs are one of many ways of exploring and presenting data that can enrich and enhance but not replace other forms of data analysis and presentation.
Currently, CCD creation requires substantial human processing, which limits current uses primarily to research. However, it seems reasonable to expect, as automated data processing and coding improves, that CCD creation could, in turn, become more automated, expanding the potential use of CCDs within and outside of research. For example, teachers could conceivably use it to examine interaction patterns in their own classroom, including whether they are allocating their time to students equitably and if some students are being left behind or otherwise marginalized during group work.
Broadening Horizons
The CCDs discussed here provide a glimpse into the potential uses of this form of data visualization. The two levels of diagrams provide a helpful way to display patterns for both individuals in groups and the classroom. CCDs can become a versatile and powerful tool in multiple stages of research and in a variety of multilayered, collaborative contexts. With educational spaces increasingly using technology and the development of machine learning, CCDs have the potential to become not just a research tool but also a tool for both understanding and enhancing collaborative educational practices.
Supplemental Material
sj-pdf-1-edr-10.3102_0013189X231158374 – Supplemental material for Classroom Data Visualization: Tracking Individuals During Group-Centered Instruction
Supplemental material, sj-pdf-1-edr-10.3102_0013189X231158374 for Classroom Data Visualization: Tracking Individuals During Group-Centered Instruction by Martha B. Makowski and Sarah Theule Lubienski in Educational Researcher
Footnotes
Notes
Authors
References
Supplementary Material
Please find the following supplemental material available below.
For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.
For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.
