Three Contrasting Pairs’ Collaborative Programming Processes in China’s Secondary Education

Abstract

Pair programming is a collaborative learning mode to foster novice learners’ computer programming. Previous empirical research has reported contrasting conclusions about the effect of pair programming on student learning. To further understand students’ pair programming, this study uses a mixed method to analyze three contrasting pairs’ collaborative behaviors, discourses, and perceptions from a multi-dimensional perspective. The analysis results show that the high-ranked student pair is characterized as the interactive, socially-supportive, and goal-oriented pair; the middle-ranked student pair is characterized as the highly-interactive, socially-supportive, and process-oriented pair; and the low-ranked student pair is characterized as the lowly-interactive, socially-unsupportive, and programming-distracted pair. The research reveals complex relations between collaborative behaviors, discourses, and performances, which have critical influences on students’ pair programming quality. Based on the results, this research proposes pedagogical, analytical, and theoretical implications for future instructional design, learning analytics, and empirical research of collaborative programming.

Keywords

pair programming collaborative learning mixed methods secondary education Minecraft

Introduction

The computer programming is one of the main learning modes to improve students’ computational thinking (CT) in K-12 schools (Brennan & Resnick, 2012; Lye & Koh, 2014; Wing, 2006). Although computer programming has potential to improve CT, empirical studies show that novice programmers easily become frustrated and bored when encountering challenges during the programming process (Falloon, 2016; García-Zubía et al., 2018; Liebenberg et al., 2012). Compared to solo programming, pair programming, as a collaborative learning mode, is a pragmatical strategy for students to solve challenging problems, generate creative ideas, and promote their learning experiences (Demir & Seferoglu, 2020a; Plonka et al., 2015; Umapathy & Ritzhaupt, 2017).

However, previous empirical research shows discrepancies about the effect of pair programming on student learning. While some empirical studies indicate that pair programming results in positive programming learning processes and performances (e.g., Hannay et al., 2009; Kalaian & Kasim, 2014; Salleh et al., 2011), other studies show that pair programming may undermine students’ problem-solving due to a lack of skills in organization, communication, and collaboration (e.g., Werner & Denning, 2009; Papadakis, 2018; Veerasamy et al., 2019). Primary factors that influence the effectiveness and quality of pair programming are individual’s programming ability (Salleh et al., 2011), social atmospheres (Werner et al., 2005), and the group configurations (Demir & Seferoglu, 2020b). Given the complex factors that may influence the pair programming, it is necessary to further conduct a fine-grained, multi-dimensional analysis of pair programming in order to provide research, analysis and practice implications.

To achieve this purpose, this study uses the pair programming strategy, supported with the Minecraft platform, to facilitate computer programming in China’s secondary education context. Using a mixed-method approach, this study deliberately chooses and analyzes three pairs’ collaborative behaviors, discourses, and perceptions during the pair programming processes. Students’ perceptions from the interview responses are used to triangulate the analysis results in order to better understand pair programming. In the following section, we describe literature review, research methodology and analysis results and discussions.

Literature Review

Grounded upon the social, cultural, situated perspectives of learning (Vygotsky, 1978), collaboration is defined as a group of people participate in coordinated activities to maintain shared understandings, to solve shared problems of a project, and to create new knowledge or relevant products (Goodyear et al., 2014). Pair programming, as a computer-supported collaborative learning mode, supports two programmers working together at one workstation to solve the same programming problem (Braught et al., 2011). Empirical studies have indicated that pair programming, under favorable conditions, is beneficial for students to develop the solution of ill-structured programming tasks (Liebenberg et al., 2012), to advance their programming knowledge (Umapathy & Ritzhaupt, 2017) and to foster their computational and creative thinking skills (Zhong et al., 2016).

Although pair programming can improve student learning under some conditions, previous empirical research shows discrepancy on students’ collaborative behaviors, discourses, and perceptions. For example, through analyzing and identifying code patterns, Hwang et al. (2012) investigated the relationships between students’ collaborative programming behaviors and their performances. The results showed that students had different coding patterns, including increasing, decreasing and no transition of six programming behaviors during the problem-solving processes. Satratzemi et al. (2018) examined students’ performance, confidence, and attitude during distributed pair programming. Results suggested that students had varied levels of programming performances, significantly related to a student’s previous programming experiences and confidences in programming. Using discourse analysis and epistemic network models, Wu et al. (2019) revealed that the high-performing team exhibited programming with a systematic approach, whereas the low-performing team used tinkering or guess-and-check approaches. Overall, although collaborative strategies are widely used to improve students’ programming, relevant research indicates different results in terms of students’ behaviors, discourses and perceptions.

During the educational practices, varied pedagogical strategies are used to improve collaborative programming during the instructional design, facilitation, and evaluation process. First, the groups’ social structure is a critical factor to consider during the instructional design. Social factors such as the role-playing strategy (Beck & Chizhik, 2013), heterogeneous group configuration (Mujeeb-u-Rehman et al., 2005), and mixed gender partnership (Zhong et al., 2016) are used to improve collaborative programming. Second, during the facilitation phase, the instructor’s intervention is critical to foster students’ programming. Because students with varied levels of programming ability need different instructional guidance and assistance during programming, the instructors should evaluate the students’ programming progresses before making interventions (Govender & Govender, 2014). Third, most studies tend to use summative outcomes to assess students’ programming, with little focus on the analysis of the process of collaborative programming (Maguire et al., 2014). Overall, it is necessary to further provide instructional strategies based on empirical research results in order to offer specific strategies for instructional design, facilitation, and evaluation for pair programming.

From an analytical perspective, multiple learning analytics methods have been used in previous empirical research in the collaborative learning field (e.g., Chen et al., 2018; Ouyang & Chang, 2019; Ouyang et al., 2020), which further lead to the use of mixed method for revealing a fine-grained picture of pair programming. For example, Berland et al. (2013) used the quantitative, statistical approach to analyze the aggregate measures across the whole class, and used the clustering method to analyze the features of novice programmers’ programming processes. Salinger et al. (2008) used a qualitative, grounded analysis to analyze video data of two learners to explain the detailed, multidimensional programming process between them. Pereira et al. (2020) used the behavior detection approach to analyze students’ effective, average and ineffective programming and used the statistic correlations to examine relations between different programming behaviors. Recent work started to use the mixed methods to gain a fuller picture of programming from both quantitative, sequential, and qualitative perspectives. For example, Wu et al. (2019) used a quantitative ethnography approach - combining statistical inference with the interpretation of qualitative, grounded analysis - to analyze the collaborative programming activities of a high-performing group and a low‐performing group. They also integrated the lag sequential analysis in this study to examine high-performing and low-performing students’ behavioral patterns in the programming process. Overall, following the research trend, this study uses the mixed method to better understand collaborative programming from the multi-dimensional perspectives.

This study uses a pair programming strategy to improve students’ programming within the Minecraft programming environment in China’s secondary education contexts. Using a multi-method approach (i.e., click stream analysis, classroom video analysis, quantitative content analysis, lag-sequential analysis, ethnographic interpretation approach), we investigate three contrasting groups’ collaborative behaviors, discourses and perceptions. Based on the results, this study proposes pedagogical, analytical, and theoretical implications for future instructional design, learning analytics, and empirical research of collaborative programming.

Method

Research Purpose and Question

The overarching research purposes are twofold: (1) to improve novice learners’ programming through using the collaborative pair programming strategy, and (2) to empirically investigate the pair programming behaviors, discourses, and perceptions. Among ten pairs of students, we specifically identify three contrasting pairs in terms of their discrepancies of the individual procedural performances. Our research question is: What are the differences of collaborative behaviors, discourses, and perceptions of three contrasting pairs during the pair programming process?

Research Context and Participants

The research context is a selective course titled “The Interactive Programming in Minecraft” offered at a junior high school during Spring 2019 in the Eastern area of China. This course is not a required course; students take the course autonomously based on their own choices. Twenty 7th graders (2 females, 18 males) enrolled in this 12-week course; they were all novice programmers with no text-based Python language programming experiences. Students were designated into ten pairs at the beginning under the instructor’s arrangement; students had an icebreaker activity at the beginning to know each other in order to better work together in the course.

This course focuses on learning the Python language, a programming language designed specifically for novice programmers and non-experts to learn computer programming (Kelleher & Pausch, 2005; Rossum, 1999). The Python course is one of the selective Information Technology Curricula for China’s high schools required by the China’s Ministry of Education (2017). Minecraft, a highly popular digital game among youth (Pellicone & Ahn, 2018) is used in this research for two reasons. First, previous review indicates the effect of gamifying programming education on youth programming (Lindberg et al., 2019); second, Python can be run and debugged through Minecraft. Therefore, the instructor (the first author) deliberately adopts Minecraft as the programming learning environment to facilitate students’ learning of the Python language (see Figure 1).

Figure 1.

Python Programming in Minecraft.

The instructor (the first author) designed three phases throughout 12 weeks, namely the initiation, the formal learning, and the pair programming phases (see Figure 2). All of the students previously had gaming experience in Minecraft but had no text-based programming experiences before this course. During the initiation phase (Phase I), the instructor taught students programming concepts in the lecturing format. During the formal learning phase (Phase II), ten pairs of students sat next to each other, completed a series of programming projects with their own computer, and discussed with the partner if they had problems. The series of programming projects tested students’ basic programming knowledge, such as sequential, selective, looping structure and creating methods. The instructor assessed students’ individual procedural performance for those tasks according to students’ degree of completion, integrity of codes, and accuracy of debugging, each accounting for 33 points out of 100 points. Students were required to complete several tasks during Phase II (see Appendix A), they were scored by the instructor in terms of whether their projects ran successfully with no errors (e.g., NameError, ValueError, SyntaxError), whether the structure was clear without unnecessary codes, and whether the essential annotations of debugging codes were added. During Phase III, the same pair of students collaborated together in one computer to complete a final programming project. The final project was comprised of three tasks; each task contained 3 to 5 problems for students to solve (see Appendix B). The instructor assessed pairs’ programming performances based on a rubric regarding correctness and completeness. Similar as the individual performance assessment, if the programming projects ran successfully with no errors, the pair was scored with full points; if the coding structure was clear without unnecessary codes, and with essential annotations of debugging processes, the pair was scored with full points.

Figure 2.

The Educational Procedures.

Among ten pairs, we identified three contrasting pairs based on students’ individual procedural performance scores (M = 63, SD = 15.37) at the end of Phase II. Two students of pair 1 achieved the scores of 86 and 70, identified as a high-ranked pair; two students of pair 2 had the scores of 70 and 65, identified as a middle-ranked pair; two students of pair 3 had the scores of 30 and 50, identified as a low-ranked pair (see Figure 3). After identifying those three contrasting pairs at the end of Phase II, we then collected collaborative process-oriented data in Phase III (see details in the next section). It is worth mention that in Phase III, Pair 1 and Pair 2 solved all the tasks for the final programming project with similar scores of 96 and 94, while Pair 3 failed in the second task of the project with a final score of 57. Here we also demonstrated the contrasting characteristics of three pairs in terms of programming behaviors and discourses that we analyzed post hoc (see Table 1). Although those characteristics were captured through post hoc analysis, the results to some extent validated the choice of three contrasting pairs based on their differences.

Figure 3.

Three Pairs’ Collaboration During a Class Session. Note. Pair 1 (the high-ranked pair) was comprised of Student 1 (male) and Student 2 (male); Pair 2 (the middle-ranked pair) was comprised of Student 3 (male) and Student 4 (female); and Pair 3 (the low-ranked pair) was comprised of Student 5 (male) and Student 6 (male).

Table 1.

The Quantities of Programming Behaviors, Discourse Units, and Discourse Words of Three Pairs.

	Quantity of behaviors	Quantity of discourse units	Quantity of discourse words
	(M = 677.00, SD = 53.70)	(M = 293.67, SD = 140.60)	(M = 3214.33, SD = 1646.26)
Pair 1	645 (Student 1: 311; Student 2: 334)	187 (Student 1: 100; Student 2: 87)	1,678 (Student 1: 890; Student 2: 788)
Pair 2	739 (Student 3: 360; Student 4: 379)	453 (Student 3: 243; Student 4: 210)	4,952(Student 3: 1,854; Student 4: 3,098)
Pair 3	647 (Student 5: 329; Student 6: 318)	241 (Student 5: 136; Student 6: 105)	3,013 (Student 5: 943; Student 6: 1,070)

Data Collection and Analysis Approaches

Focusing on three pairs’ programming processes, this study collected and analyzed data from three ways. First, we recorded students’ online and offline behaviors through the computer screen-running videos and a whole class video (without sounds). We used the click stream analysis (Filvà et al., 2019) and classroom video analysis (Kersting, 2008) to analyze those two types of data in order to identify pairs’ programming behaviors. Two coders first individually watched the computer screen-running videos and the classroom videos, identified initial codes of online and offline behaviors, then had discussions to achieve an agreement of the final coding framework. Eight behaviors were identified: Project Understanding (PU), Python Coding (PC), Minecraft Debugging (MD), Minecraft Gaming (MG), Programming Assistance (PA), Partner Discussion (PD), Instructor Communication (IC) and Classmate Communication (CC) (see Table 2). Then two coders independently coded the data again in a chronological order based on the coding framework, and reached an inter-rater reliability with the Cohen’s Kappa of .888. Students’ behaviors were reported in a summative way and further demonstrated in a temporal graph.

Table 2.

The Coding Framework for Online and Offline Behaviors.

Code	Description
Project understanding (PU)	A student transferred to the task window to understand the programming projects.
Python coding (PC)	A student wrote codes with the Python language in the system.
Minecraft debugging (MD)	A student debugged in Minecraft.
Minecraft gaming (MG)	A student played games in Minecraft.
Programming assistance (PA)	A student looked at the computer and assisted his/her partner to program.
Partner discussion (PD)	A student discussed with his/her partner during the programming.
Instructor communication (IC)	A student(s) talked with the instructor.
Classmate communication (CC)	A student(s) talked with other classmates in the class.

Note. The first four codes were identified through the clickstream data as online behaviors; the last four codes were identified through the video data as offline behaviors.

Second, we recorded students’ communicative discourses during the pair programming processes. We used the quantitative content analysis (QCA) (Grbich, 2007), lag-sequential analysis (LsA) (Faraone & Dorfman, 1987) as well as ethnographic interpretations (Wu et al., 2019) to examine and explain pairs’ discourse patterns and characteristics. We first manually transcribed the audio data in term of the time frame, double checked the transcripts, and used line-by-line coding to analyze audio transcripts. The unit of analysis was a sentence spoken by a student. Two raters individually coded the transcript data, had multiple meeting to solve discrepancy, achieved an agreement of a coding framework, and reached an inter-rater reliability with the Cohen’s Kappa of 0.802. This coding framework included eight codes: programming exploration (PEx), programming elaboration (PEl), asking question (AsQ), answering question (AnQ), instructor interaction (II), social expression-positive (SE-P), social expression-negative (SE-N) and social chatting (SC) (see Table 3). Then, complementing the “coding and counting” approach of QCA, we used LsA to examine the sequential contingencies of discourse codes (Chen et al., 2017). Here, we used the transitional frequencies to calculate the number of times a code transitioned immediately to another code. We also used ethnographic accounts to explain excerpts from groups’ communicative discourses that complemented the previous analysis results. We supported our analysis results with ethnographic interpretations demonstrated from one excerpt from each pair’s discourse.

Table 3.

The Coding Framework for Collaborative Discourses.

Code	Description	Example
Programming exploration (PEx)	A student explored programming with the partner, without detailed descriptions of Python coding.	“There is a problem. This was not modified.”
Programming elaboration (PEl)	A student elaborated programming with the partner, with detailed descriptions of Python coding.	“In the fourth line, we should write mc. postToChat.”
Asking question (AsQ)	A student asked questions to the partner.	“Something is wrong in “post”. What does it mean?”
Answering question (AnQ)	A student answered questions to the partner.	“The ‘t’ in postToChat should be capitalized.”
Instructor Interaction (II)	A student discussed with the instructor about programming problems.	“Instructor, why does here show “no mcpi”?”
Social expression positive (SE-P)	A student expressed positive emotions.	“I think we are doing perfect.”
Social expression negative (SE-N)	A student expressed negative emotions.	“I’ve got nothing to say to you.”
Social chatting (SC)	A student had irrelevant chatting with the partner.	“Hurry up, connect the multiplayer games and join the game.”

Finally, we collected and analyzed students’ self-reports about their psychological changes (e.g., anxiety) and students’ interview data regarding perceptions about the collaborative programming processes. Taken together, a mixed method was used to examine three pairs’ collaborative behaviors, discourses, and perceptions during pair programming from the summative, sequential, and interpretative perspectives.

Results

Pair 1: The Interactive, Socially-Supportive, and Goal-Oriented Pair

Pair 1, a high-ranked pair among three pairs, was identified as the interactive, socially-supportive, and goal-oriented pair. First, the most frequent behaviors of Pair 1 were Partner Discussion (PD; frequency = 191), Python Coding (PC; frequency = 136), and Programming Assistance (PA; frequency = 135), which all ranked at the middle level among three groups (see Figure 4). Moreover, the temporal graph showed that the programming behaviors of PD and PA distributed evenly across the time period, which indicated that students in Pair 1 consistently communicated with and assisted each other during the pair programming. In addition, among three pairs, Pair 1 had the highest frequency of Minecraft Debugging (MD; frequency = 124), which were interwoven with the Python Coding (PC) behavior. The interweaving behavior implied a typical, effective computer programming procedure (Wu et al., 2019).

Figure 4.

The Temporal Graph of Collaborative Behavior Changes. Note. The x-axis represents the time period; y-axis represents online behaviors (the number of the total frequency of the code).

Second, the results of discourse patterns showed that the most frequent discourse of Pair 1 was programming exploration (PEx; frequency = 79) (see Figure 5). Among three pairs, Pair 1 had the highest frequency of instructor interaction (II; frequency = 39), which implied that they requested the assistance from the instructor frequently during their debugging process. Moreover, the LsA results showed that in Pair 1, the most frequent transition was the transition from PEx to PEx (frequency = 41), from II to II (frequency = 25), and from AsQ to AnQ (frequency = 19) (see Table 4). Therefore, Pair 1 spent time on programming exploration without detailed explanations, conducted frequent programming debugging behaviors, and asked the instructor for help during the debugging process. An excerpt of Pair 1’s communicative discourses also demonstrated this collaborative characteristic: two students focused on the programming goal by exploring programming codes with partners without detailed explanations (e.g., line 3-7), followed with the request of help from the instructor (e.g., line 9-13) (see Table 5).

Figure 5.

Transitional Network Representation in Three Pairs of Collaborative Discourse. Note. A node represents one category of collaborative discourse (e.g. AsQ denotes Answering Question), the node size/number represented the frequency of the code, the direction of the arrow represented the transition from one code to the other, and the width represented the frequency of the transition.

Table 4.

LsA Transition Frequency of Collaborative Discourses of Three Pairs.

	Transition	Freq.		Transition	Freq.		Transition	Freq.
Pair 1	PEx->PEx	41	Pair 2	AsQ->AnQ	71	Pair 3	PEx->PEx	22
	II->II	25		PEx->PEx	54		SE-N->SE-N	21
	AsQ->AnQ	19		PEl->PEl	37		SC->SC	19

Note. Only the top three transitions with a high-level frequency were presented.

Table 5.

An Excerpt of Pair 1’s Communicative Discourses.

Line	Student	Utterance	Code
1	Student 2	What should we do?	AsQ
2	Student 1	Do as the task requires.	AnQ
3	Student 2	As long as it is not wrong, mcpi.minecraft.	PEx
4	Student 1	There is something wrong with this clock.	PEx
5	Student 2	Do we need to create a clock here?	PEx
6	Student 1	Yes, just right here, we need to do one step forward.	PEx
7	Student 2	We should ask teacher to judge if this is ok.	PEx
8	Student 1	Wow, we are so great.	SE-P
9	Student 2	Teacher, help, SOS.	II
10	Student 1	Why is this problem like this? mcdrawing.drawLine	AsQ/PEl
11	Instructor	Please look carefully at the code that I provided.
12	Student 2	Instructor, we cannot figure it out.	II
13	Instructor	There is something wrong with the description of hour and minute operation…

Finally, students’ self-reflection of pair programming indicated that both students in Pair 1 had a decreased level of anxiety after the pair programming. But students’ interview indicated contrasting perceptions about the pair programming process. The Student 1 perceived an unbalanced workload, “it was me who usually looked for the solutions for the problems when we found mistakes …”; due to the workload completed by Student 1, Student 2 felt a high-level efficiency of pair programming, “…the coding efficiency was much higher when we worked together. If I programmed alone, I had to double check the code by myself. But when I got a partner, he would help me”. Taken together, Pair 1, as a high-ranked pair in terms of the individual and group performances, was characterized as a goal-oriented pair and the programming process was relatively interactive and socially-supportive.

Pair 2: The Highly-Interactive, Socially-Supportive, and Process-Oriented Pair

Pair 2, the middle-ranked pair among three pairs, was identified as the highly-interactive, socially-supportive, and process-oriented pair. First, the most frequent behaviors of Pair 2 were Partner Discussion (PD; frequency = 245), Python Coding (PC; frequency = 174), and Programming Assistance (PA; frequency = 169), which also ranked highest among three pairs (see Figure 4). Therefore, compared to Pair 1, Pair 2 was initially labelled as highly-interactive and socially-supportive. According to the temporal graph, Pair 2 first attempted to understand the programming project (PU), discussed with the partner (PD), and then turned to the Python coding (PC) process. Pair 2 also debugged in Minecraft (MD) several times to modify the codes, and further continued the Python coding (PC) process. Like Pair 1, Pair 2’s students also constantly discussed with each other (PD) and helped each other (PA) to solve problems.

Second, regarding the discourse patterns, the most frequent discourse of Pair 2 was programming exploration (PEx; frequency = 119), asking questions (AsQ; frequency = 116), and programming elaboration (PEl; frequency = 99) (see Figure 5). Among three pairs, Pair 2 had the highest frequency on programming-related discourses PEx and PEl, questioning and answering AsQ and AnQ, as well as positive social expression SE-P. The LsA results showed that in Pair 2, the most frequent transitions were the transition from AsQ to AnQ (frequency = 71), from PEx to PEx (frequency = 54) and from PEl to PEl (frequency = 37) (see Table 4). Therefore, Pair 2 spent their time on exploring codes, asking and answering relevant questions, and explaining codes for problem-solving. An excerpt of Pair 2’s discourses also demonstrated this collaborative characteristic: two students focused on the programming process by exploring and elaborating programming codes with partners (e.g., line 5, 7, 9), supported with social supports and encouragements (e.g., line 10, 11) (see Table 6).

Table 6.

An Excerpt of Pair 2’s Communicative Discourses.

Line	Student	Utterance	Code
1	Student 4	What should we do?	AsQ
2	Student 3	We need to find out the mistake and debug.	AnQ
3	Student 4	The “t” in postToChat should be capital right?	AsQ
4	Student 3	Yes, it is.	AnQ
5	Student 4	import mcpi.minecraftstuff as minecraftstuff, is this a space?	PEl/AsQ
6	Student 3	I am not sure, there should be a space.	AnQ
7	Student 4	The second is mc.setBlock(ballPosX, ballPosY, ballPosZ, block.AIR.id,1), we should setBlock first	PEl
8	Student 3	You expand it first, is it like this?	PEx
9	Student 4	Yes, the next one is mc.setBlock (e.pos.x,e.pos.y,e.pos.z,block.WOOL.id,10), blocks, it is correct.	AsQ/PEl
10	Student 3	Yes, it should be right, let’s give it a try.	SE-P
11	Student 4	I think we have done a great job.	SE-P

Finally, students’ self-reflection of pair programming indicated that both students in Pair 2 self-reported a decreased level of anxiety after the pair programming process. In the interview, both of them perceived their pair programming processes as harmonious and effective. Student 3 said, “I would fell confused if I worked alone on the coding. It was hard to find all the mistakes by myself. She [student 4] programmed more carefully and thought more comprehensively than me”. Similar to Student 3’s perceptions, Student 4 said, “…when I made some mistakes in spelling, my partner helped me a lot”. Taken together, although Pair 2 was a middle-ranked pair in terms of their individual and group performances, Pair 2 was characterized as a progress-oriented pair and the process was highly-interactive and socially-supportive.

Pair 3: The Lowly-Interactive, Socially-Unsupportive, and Programming-Distracted Pair

Pair 3, the low-ranked pair among three pairs, was identified as the lowly-interactive, socially-unsupportive, and programming-distracted pair. First, the most frequent behaviors of Pair 3 were Minecraft Gaming (MG; frequency = 332), Python Coding (PC; frequency = 102), and Partner Discussion (PD; frequency = 71). Among three pairs, Pair 3 had the highest frequency of Minecraft Gaming (MG; frequency = 332) and the lowest frequency of Partner Discussion (PD; frequency = 71), Minecraft Debugging (MD; frequency = 48), and Programming Assistance (PA; frequency = 9). According to the temporal results (see Figure 4), Pair 3 first attempted to understand the programming project (PU), and then focused on Python coding (PC) through partner discussions (PD). However, from the middle to the end of the programming period, both of the students were constantly attracted by Minecraft Gaming (MG), which was a sign for giving up the programming problem-solving.

Second, among three pairs, discourse analysis results showed that Pair 3 had the highest frequency of negative social expression (SE-N; frequency = 41) and social chatting (SC; frequency = 34) and the lowest frequency of programming exploration (PEx; frequency = 64) and programming elaboration (PEl; frequency = 5) (see Figure 5). The LsA results showed that in Pair 3, the most frequent transition occurred from PEx to PEx (frequency = 22), from SE-N to SE-N (frequency = 21), and from SC to SC (frequency = 19) (see Table 4). An excerpt of Pair 3’s communicative discourses demonstrated that two students had disagreement about the programming procedures (e.g., line 2, 3) and later were distracted by game in Minecraft (e.g., Line 7, 8), accompanied by unsupportive social expressions (e.g., line 9) (see Table 7).

Table 7.

An Excerpt of Pair 3’s Communicative Discourses.

Line	Student	Utterance	Code
1	Student 6	Is it right? You must hurry up.	AsQ
2	Student 5	I know, we got it. But we have a problem with the post, the first one.	AnQ/PEx
3	Student 6	Yes, I know. My head is about to collapse, what is this?	AsQ
4	Student 5	Ah, instructor, I can’t debug here.	II
5	Instructor	You two should discuss with each to solve the problem.
6	Student 6	I don’t know how to write, what can I do?	AsQ
7	Student 5	Please hurry up, don’t play game here.	PEx
8	Student 6	I want to play game.	IC
9	Student 5	Forget about it, I don’t care anymore, whatever.	SE-N
10	Student 6	Don’t please, tell me what I can do?	AsQ
11	Student 5	At least you can set the clock for me.	PEx
12	Student 6	Well.	AnQ

Finally, students’ self-reflection of pair programming indicated a contrasting change in the level of anxiety, with Student 5 reported a decrease of anxiety while Student 6 reported an increase of anxiety. The interview also indicated both students had a negative perception to each other. Student 5 said, “For the first task, I told him [student 6] that he made mistakes, but he didn’t listen to me. The biggest divergence was the second task. He wanted to use codes taught by the teacher to get full score, but I chose to code in Minecraft.”. Student 6 also held a negative opinion, “I was not good at programming at all… both of us wanted to play game in Minecraft and hope the other person would finish the project…”. Taken together, as the low-ranked pair, Pair 3 was characterized as a programming-distracted pair, and the programming process was lowly-interactive and socially-unsupportive.

Discussions

Since previous empirical studies resulted in contrasting conclusions about the effect of pair programming on students’ learning quality, this study used a mixed-method approach to conduct fine-grained analysis of three pairs’ programming behaviors, discourses, and perceptions from a multi-dimensional perspective. The research results revealed discrepancies among three pairs: the low-ranked pair (Pair 3) was identified as a lowly-interactive, socially-unsupportive, and programming-distracted pair; the middle-ranked pair (Pair 2) was identified as a highly-interactive, socially-supportive, and process-oriented pair; and the high-ranked pair (Pair 1) was identified as an interactive, socially-supportive, and goal-oriented pair. Specifically, the low-ranked pair spent more time on distracted gamifications and social conversations; the middle-ranked pair made more conversations about programming explorations, elaborations and questioning-answering; and the high-ranked pair focused more on debugging programming codes to achieve goals without detailed explanations. Moreover, regarding student perceptions, the low-ranked pair perceived a negative, unsupportive learning atmosphere; students in the middle-ranked, process-oriented pair perceived a supportive and encouraging collaborative learning atmosphere; the high-ranked, goal-oriented pair perceived a contrasting feeling about the collaboration. Echoing previous studies (e.g., Hwang et al., 2012; Wu et al., 2019; Yang et al., 2015), this research reveals complex correlations between programming behaviors, discourses, and perceptions, which may have significant influences on the collaborative programming quality, performance and experience.

Based on the results, this study proposes pedagogical, analytical, and theoretical implications for future instructional design, learning analytics, and empirical research of collaborative programming. First, on the pedagogical level, the instructors should use the process-oriented interventions and evaluations to foster pair programming. The instructors should identify the specific learning situations that are appropriate to provide instructional interventions during the collaborative programming process. Our results showed that the low-performing groups could easily get distracted by irrelevant activities (e.g., gaming), such that the instructor needed to provide on-time assistances to guide students on the programming track (Wang & Hong, 2018). More incentives and assistances need to be given for low-achievers at the early stage of a problem-solving period to intervene their programming behaviors and motivations (Hwang et al., 2012; Pereira et al., 2020). Moreover, our results show that the process-oriented programming has potential to promote students’ knowledge building discourses with peers during the pair programming (Lewis, 2012; Wu et al., 2019). Although students’ final programming products and performances are usually the main focus and goal (Zhong et al., 2016), the instructors should take into consideration students’ programming process rather than merely focusing on the final performance, because the process-oriented scaffolding could facilitate question proposing and answering, knowledge sharing and construction, and the programming solution creation (Ouyang et al., 2020). Overall, the process-oriented instructional design is promoted based on the empirical results to increase pair programming quality.

Second, on the analytical level, the mixed method has been promoted to conduct multilevel analyses in collaborative learning research (Janssen et al., 2013; Medina & Stahl, 2020; Stahl, 2009). There has been a trend currently to apply the mixed method (e.g., clickstream analysis, behavior sequential analysis, statistical analysis, discourse analysis, network analysis) in the computer programming research (e.g., Pereira et al., 2020; Wu et al., 2019). This research integrates the statistical, sequential analysis with the qualitative, ethnographic approaches (e.g., discourse analysis) to examine the development of pair programming, which helps reveal fine-grained relevancies between the behavioral, cognitive, and temporal activities. However, due to the technical restriction, we were not able to capture pairs’ online, offline behaviors and their communicative discourses synchronously; such that we were not able to conduct a more integrated microanalysis of the moment-to-moment details of how students coordinate their communications, discourses, and behaviors to build programming knowledge and artifacts (Stahl, 2009). In addition, the qualitative-oriented conversation or discourse analysis is typically focused on a small fragment of students’ discourses over a short time duration, like what we did in this research. Future analysis should make a better integration of the quantitative analysis for discovering frequently observed regularities of interactions, behaviors, and communications, and the qualitative analysis for examining the delicate organization of prior, current, and subsequent practices over a time period. Overall, complementing each other, the qualitative and quantitative methods together can provide a more holistic, multilevel, multidimensional analysis of collaborative programming.

Finally, on the theoretical perspective, student agency is demonstrated by the students’ intentionality for and their action of taking learning initiations (Bandura, 2001), which should be promoted in novice learners’ programming process. The results showed that high-performing student pairs took actions to initiate questions, shared and negotiated knowledge and created the programming solutions. They together achieved cognitive accomplishments that exceeds the knowledge of any individual members for the higher-order programming work (Stahl, 2005). In other words, group members’ synergistic coordination of the peer interactions, programming behaviors, and knowledge discourses can lead to the high quality of the problem-solving process. Because this research merely analyzed a small size of sample, more studies on how to improve novice students’ agency of programming in collaborative contexts are needed.

Conclusions

Collaborative programming is a promising yet challenging learning practice for novice programmers. This research selects three contrasting collaborative pairs in China’s secondary education and conducts a fine-grained analysis of the pair programming processes from the multi-dimensional (i.e., summative, sequential, and interpretative) perspectives. The results reveal differences among three pairs in terms of collaborative behaviors, discourses, and perceptions during the pair programming processes. Pedagogical, analytical, and theoretical implications are provided for future instructional design, learning analytics, and empirical research of collaborative programming. A limitation of this research focuses on the small size of student sample, with a limited range of subjects’ demographic backgrounds and learning experiences. Future research should expand the sample size to further validate the research results and the proposed implications. Overall, since the intrinsic value of programming center on its process, relevant research and practice should take a process-oriented perspective to investigate, advance, and assess students’ programming in order to foster a sustainable learning.

Appendix

A. One example of the individual procedural projects

1. Connect python with Minecraft (40 points)

a) import Minecraft class (10 points)

Hint: import mcpi.minecraft as minecraft

b) create an object in Minecraft (10 points)

Hint: mc=minecraft.Minecraft.create()

c) print “hello world” in python and Minecraft (20 points)

Hint: print(“hello world”), mc.postToChat(“hello world”)

2. Print user’s position (60 points)

a) get user’s position (15 points)

Hint: pos=mc.player.getTilePos()

b) print user’s position in python and Minecraft (15 points)

Hint: print(pos.x), mc.postToChat(pos.x+pos.y+pos.z)

c) print user’s position in a certain format: (30 points)

Hint: mc.postToChat (“x=”+str(pos.x)+“, y=”+str(pos.y)+“, z=”+str(pos.z))

B. The Final Project

1. Task 1: Debugging the “final project’ python file, which consists of 5 missing parts (25 points)

a) Please print “welcome to my playground” in the Minecraft.

b) Please create an orange wood inside the ‘if'.

c) Please create a function which could build a playground.

d) Please complete the code when clicking face of 4.

e) Please complete the ‘else' to print ‘OUT' in the Minecraft.

2. Task 2: Adding ‘Clock’ in the ‘final project’ (30 points)

a) Using function of ‘drawLine’ to create a line (10 points)

import mcpi.minecraftstuff as minecraftstuff

mcdrawing = minecraftstuff.MinecraftDrawing(mc)

pos = mc.player.getTilePos()

mcdrawing.drawLine(pos.x, pos.y, pos.z, pos.x, pos.y+20, pos.z, block.WOOL.id,1)

mcdrawing.drawLine(pos.x, pos.y, pos.z, pos.x+20, pos.y, pos.z, block.WOOL.id,2)

mcdrawing.drawLine(pos.x, pos.y, pos.z, pos.x+20, pos.y+20, pos.z, block.WOOL.id, 3)

b) Using function of ‘drawCircle’ to create a circle (10 points)

mcdrawing.drawCircle(pos.x, pos.y + 20, pos.z, 20, block.WOOL.id, 4)

c) Using function of ‘drawSphere’ to create a sphere (10 points)

mcdrawing.drawSphere(pos.x, pos.y + 20, pos.z, 15, block.WOOL.id, 5)

3. Task 3: Create new feature to ‘final project’, which should be reasonable in ball game (45 points)

a) the correctness of the new feature (15 points)

b) the integrating of the basic structure and function taught in previous classes (15 points)

c) creativity and aesthetics – Color, size, location and etc. (15 points)

Footnotes

Declaration of Conflicting Interests

The authors declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Ethical Approval

The consent forms were sent to all participants; the data collection approaches were approved by all participants; participants’ names were anonymized.

Funding

The authors disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This research was supported by the National Natural Science Foundation of China (61977057; 61907038), and Major Program of New Generation of Artificial Intelligence of the National Science and Technology Innovation 2030 of China (2019AAA0105403).

ORCID iD

Fan Ouyang

Author Biographies

Dan Sun is a doctoral student of Educational Technology in the college of Education at Zhejiang University. Her research interests are computational thinking, K-12 programming education, learning analytics, and artificial intelligence in education.

Fan Ouyang, PhD, is an associate researcher (tenure-track) of Educational Technology in the College of Education at Zhejiang University. Her research interests are computer-supported collaborative learning, learning analytics and educational data mining, online and blended learning, and artificial intelligence in education.

Yan Li, PhD, is a professor of Educational Technology in the College of Education at Zhejiang University. Her research interests are e-learning, distance education, ICT education, STEM education, media education and diffusion of educational innovation.

Hongyu Chen is a master student of Science and Technology Education in the college of Education at Zhejiang University. Her research interests are STEM education, informal learning, and traveling education.

References

Bandura

(2001). Social cognitive theory: An agentic perspective. Annual Review of Psychology, 52, 1–26. https://doi.org/10.1111/1467-839X.00024

Beck

Chizhik

(2013). Cooperative learning instructional methods for CS1: Design, implementation, and evaluation. ACM Transactions on Computing Education, 13(3), 10–32.

Berland

Martin

Benton

Petrick

S.C.

Davis

(2013). Using learning analytics to understand the learning pathways of novice programmers. Journal of the Learning Sciences, 22(4), 564–599. https://doi.org/10.1080/10508406.2013.836655

Braught

Wahls

Eby

L. M.

(2011). The case for pair programming in the computer science classroom. ACM Transactions on Computing Education, 11(1), 1–21. https://doi.org/1921607.1921609

Brennan, K., & Resnick, M. (2012). New frameworks for studying and assessing the development of computational thinking. In Arnetha, F. B., & Cynthia, A. T. (Eds.), Proceedings of the 2012 annual meeting of the American educational research association (pp. 1–25). AERA.

Chen, B., Chang, Y. H., Ouyang, F., & Zhou, W. Y. (2018). Fostering discussion engagement through social learning analytics. The Internet and Higher Education, 37, 21–30. https://doi.org/10.1016/j.iheduc.2017.12.002.

Chen

Resendes

Chai

C. S.

Hong

(2017). Two tales of time: Uncovering the significance of sequential patterns among contribution types in knowledge-building discourse. Interactive Learning Environments, 25(2), 162–175. https://doi.org/10.1080/10494820.2016.1276081

Demir

Ö.

Seferoglu

S. S.

(2020a). A Comparison of solo and pair programming in terms of flow experience, coding quality, and coding achievement. Journal of Educational Computing Research. Advance online publication. https://doi.org/10.1177/0735633120949788

Demir

Ö.

Seferoglu

S. S.

(2020b). The effect of determining pair programming groups according to various individual difference variables on group compatibility, flow, and coding performance. Journal of Educational Computing Research. . Advance online publication. https://doi.org/10.1177/0735633120949787

10.

Govender

D. W.

Govender

T. P.

(2014). Using a collaborative learning technique as a pedagogic intervention for the effective teaching and learning of a programming course. Mediterranean Journal of Social Sciences, 5(20), 1077. https://doi.org/10.5901/mjss.2014.v5n20p1077

11.

Falloon

(2016). An analysis of young students’ thinking when completing basic coding tasks using Scratch Jnr. on the iPad. Journal of Computer Assisted Learning, 32(6), 576–593. https://doi.org/10.1111/jcal.12155

12.

Faraone

S. V.

Dorfman

D. D.

(1987). Lag sequential analysis: Robust statistical methods. Psychological Bulletin, 101(2), 312–323. https://doi.org/10.1037/0033-2909.101.2.312

13.

Filvà

D. A.

Forment

M. A.

García-Peñalvo

F. J.

Escudero

D. F.

Casañ

M. J.

(2019). Clickstream for learning analytics to assess students’ behavior with scratch. Future Generation Computer Systems, 93, 673–686. https://doi.org/10.1016/j.future.2018.10.057

14.

García-Zubía, J., Angulo, I., Martínez-Pieper, G., Orduña, P., Rodríguez-Gil, L., & Hernandez-Jayo, U. (2018). Learning to program in K12 using a remote controlled robot: RoboBlock. In Auer, M., & Zutin, D. (Eds.), Online engineering & internet of things (pp. 344–358). Springer.

15.

Goodyear

Jones

Thomson

(2014). Computer-supported collaborative learning: Instructional approaches, group processes and educational designs. In J. M. Spector, M. D. Merrill, J. Elen, & M. J. Bishop (Eds.), Handbook of research on educational communications and technology (4th ed., pp. 439–451). Springer.

16.

Grbich

(2007). Qualitative data analysis: An introduction. SAGE.

17.

Hannay

J. E.

Dyba

Arisholm

Sjoberg

D. I. K.

(2009). The effectiveness of pair programming: A meta-analysis. Information & Software Technology, 51(7), 1110–1122. https://doi.org/10.1016/j.infsof.2009.02.001

18.

Hwang

W. Y.

Shadiev

Wang

C. Y.

Huang

Z. H.

(2012). A pilot study of cooperative programming learning behavior and its relationship with students’ learning performance. Computers & Education, 58(4), 1267–1281. https://doi.org/10.1016/j.compedu.2011.12.009

19.

Janssen

Cress

Erkens

Kirschner

P. A.

(2013). Chapter 6: Multilevel analysis for the analysis of collaborative learning. In C. E. Hmelo-Silver, C. A. Chinn, C. K. K. Chan, & A. M. O’Donnell (Eds.), The international handbook of collaborative learning (pp. 124–137). Routledge.

20.

Kalaian

S. A.

Kasim

R. M.

(2014). Small‐group vs. competitive learning in computer science classrooms: A meta‐analytic review. In R. Queiros (Ed.), Innovative teaching strategies and new learning paradigms in computer programming (pp. 46–63). IGI Global.

21.

Kelleher

Pausch

(2005). Lowering the barriers to programming: A survey of programming environments and languages for novice programmers. Science, 37(2), 83–137. https://doi.org/10.1145/1089733.1089734

22.

Kersting

(2008). Using video clips as item prompts to measure teachers’ knowledge of teaching mathematics. Educational and Psychological Measurement, 68(5), 845–861.

23.

Lewis, C. (2012). The importance of students’ attention to program state: A case study of debugging behavior. In Alison, C., Kate, S., & Beth, S. (Eds.), Proceedings of the 9th annual international conference on international computing education research (pp.127–134). Association for Computing Machinery.

24.

Liebenberg

Mentz

Breed

(2012). Pair programming and secondary school girls’ enjoyment of programming and the subject Information Technology (IT). Computer Science Education, 22(3), 219–236. https://doi.org/10.1080/08993408.2012.713180

25.

Lindberg

R. S.

Laine

T. H.

Haaranen

(2019). Gamifying programming education in K‐12: A review of programming curricula in seven countries and programming games. British Journal of Educational Technology, 50(4), 1979–1995. https://doi.org/10.1111/bjet.12685

26.

Lye

S. Y.

Koh

J. H. L.

(2014). Review on teaching and learning of computational thinking through programming: What is next for K-12? Computers in Human Behavior, 41, 51–61. https://doi.org/10.1016/j.chb.2014.09.012

27.

Maguire

Hyland

Marshall

(2014). Enhancing collaborative learning using paired‐programming: Who benefits? AISHE‐J: The All Ireland Journal of Teaching and Learning in Higher Education, 6(2), 1411–14125.

28.

Medina

Stahl

(2020). Analysis of group practices. International handbook of computer-supported collaborative learning. Springer.

29.

Ministry of Education. (2017). General high school information technology curriculum standard (2017 Edition). Ministry of Education of the People’s Republic of China website. http://www.moe.gov.cn/jyb_xxgk/xxgk_jyta/jyta_kjs/202002/.html

30.

Mujeeb-u-Rehman, M., Yang, X., Dong, J., & Abdul Ghafoor, M. (2005). Heterogeneous and homogenous pairs in pair programming: An empirical analysis. In Billintion, R., & Alden, B. (Eds.), Canadian conference on electrical & computer engineering (pp. 1116–1119). IEEE.

31.

Ouyang, F., & Chang, Y. H. (2019). The relationship between social participatory role and cognitive engagement level in online discussions. British Journal of Educational Technology, 50(3), 1396–1414. https://doi.org/10.1111/bjet.12647

32.

Ouyang, F., Chang, Y. H., Scharber, C., Jiao, P., & Huang, T. (2020). Examining the instructor-student collaborative partnership in an online learning community course. Instructional Science, 48(2), 183–204. https://doi.org/10.1007/s11251-020-09507-4

33.

Papadakis

(2018). Is pair programming more effective than solo programming for secondary education novice programmers?: A case study. International Journal of Web-Based Learning and Teaching Technologies, 13(1), 1–16. https://doi.org/10.1177/1555412015622345

34.

Pellicone

Ahn

(2018). Building worlds: A connective ethnography of play in Minecraft. Games and Culture, 13(5), 440–458. https://doi.org/10.1177/1555412015622345

35.

Pereira

F. D.

Oliveira

E. H.

Oliveira

D. B.

Cristea

A. I.

Carvalho

L. S.

Fonseca

S. C.

Toda

Isotani

(2020). Using learning analytics in the Amazonas: Understanding students’ behaviour in introductory programming. British Journal of Educational Technology, 51(4), 955–972. https://doi.org/10.1111/bjet.12953

36.

Plonka

Sharp

Van der Linden

Dittrich

(2015). Knowledge transfer in pair programming: An in-depth analysis. International Journal of Human-Computer Studies, 73, 66–78. https://doi.org/10.1016/j.ijhcs.2014.09.001

37.

Rossum

G. V.

(1999). Computer programming for everybody. Python website. https://www.python.org/doc/essays/cp4e/

38.

Salinger

Plonka

Prechelt

(2008). A coding scheme development methodology using grounded theory for qualitative analysis of pair programming. Human Technology: An Interdisciplinary Journal on Humans in ICT Environments, 4(1), 9–25. https://doi.org/10.17011/ht/urn.200804151350

39.

Salleh

Mendes

Grundy

J. C.

(2011). Empirical studies of pair programming for cs/se teaching in higher education: A systematic literature review. IEEE Transactions on Software Engineering, 37(4), 509–525. https://doi.org/10.1109/TSE.2010.59

40.

Satratzemi

Xinogalos

Tsompanoudi

Karamitopoulos

(2018). Examining student performance and attitudes on distributed pair programming. Hindawi .

41.

Stahl

(2005). Group cognition in computer-assisted collaborative learning. Journal of Computer Assisted Learning, 21, 79–90. https://doi.org/10.1111/j.1365-2729.2005.00115.x

42.

Stahl

(2009). Studying virtual math teams. Springer.

43.

Sterling

(2001). Sustainable education: Re-visioning learning and change. Schumacher Briefings. Green Books.

44.

Umapathy

Ritzhaupt

A. D.

(2017). A meta-analysis of pair-programming in computer programming courses: Implications for educational practice. ACM Transactions on Computing Education, 17(4), 1–13. https://doi.org/10.1145/2996201

45.

Veerasamy

A. K.

D’Souza

Lindén

Laakso

M. J.

(2019). Relationship between perceived problem‐solving skills and academic performance of novice learners in introductory programming courses. Journal of Computer Assisted Learning, 35, 246–255. https://doi.org/10.1111/jcal.12326

46.

Vygotsky

L. S.

(1978). Mind in society: The development of higher psychological processes. Harvard University Press.

47.

Wang

S. L.

Hong

H. T.

(2018). The roles of collective task value and collaborative behaviors in collaborative performance through collaborative creation in CSCL. Educational Technology Research and Development, 66(4), 937–953. https://doi.org/10.1007/s11423-018-9593-y

48.

Werner, L., & Denning, J. (2009). Pair programming in middle school. Journal of Research on Technology in Education, 42(1), 29–49. https://doi.org/10.1080/15391523.2009.10782540

49.

Werner

L. L.

Hanks

McDowell

(2005). Pair-programming helps female computer science students. Journal of Educational Resources in Computing, 4(1), 1–8. https://doi.org/10.1145/1060071.1060075

50.

Wing

J. M.

(2006). Computational Thinking. Communications of the ACM, 49(3), 33–35. https://doi.org/10.1145/1118178.1118215

51.

Ruis

A. R.

Wang

(2019). Analysing computational thinking in collaborative programming: A quantitative ethnography approach. Journal of Computer Assisted Learning, 35(3), 421–434. https://doi.org/10.1111/jcal.12348

52.

Yang

T. C.

Chen

S. Y.

Hwang

G. J.

(2015). The influences of a two-tier test strategy on student learning: A lag sequential analysis approach. Computers & Education, 82, 366–377. https://doi.org/10.1016/j.compedu.2014.11.021

53.

Zhong

Wang

Chen

(2016). The impact of social factors on pair programming in a primary school. Computers in Human Behavior, 64, 423–431. https://doi.org/10.1016/j.chb.2016.07.017

54.

Zhong

Wang

Chen

(2016). An exploration of three‐dimensional integrated assessment for computational thinking. Journal of Educational Computing Research, 53(4), 562–590. https://doi.org/10.1177/0735633115608444