Real-Time Collaborative Programming in Undergraduate Education: A Comprehensive Empirical Analysis of Its Impact on Knowledge,Behaviors,and Attitudes

Abstract

Real-time collaborative programming (RCP), which allows multiple programmers to work concurrently on the same codebase with changes instantly visible to all participants, has garnered considerable popularity in higher education. Despite this trend, little work has rigorously examined how undergraduates engage in collaborative programming when using RCP. This study utilized a quasi-experimental design to investigate the impact of RCP on undergraduates’ programming knowledge and skills, behaviors, and attitudes toward collaboration. Eighty-two participants were randomly assigned to either the RCP group or the Traditional Collaborative Programming (TCP) group. The data analysis yielded noteworthy findings. First, RCP significantly improved learners’ problem-solving skills and led to higher knowledge test scores compared to TCP. Second, there were no significant differences in attitudes toward collaboration between the two groups. Third, analysis of programming behaviors indicated that the RCP group exhibited more efficient coding practices, including producing a greater volume of code and executing it more successfully. Additionally, the RCP group’s code-testing activities were predominantly concentrated at the first half of the class sessions. Based on the empirical research results, three major pedagogical implications were proposed. These findings highlight the benefits of integrating RCP into programming education and suggest avenues for future research into collaborative learning dynamics.

Keywords

real-time collaborative programming programming knowledge and skills behavioral patterns attitudes towards collaboration undergraduate students

Get full access to this article

View all access options for this article.

References

Al-Samarraie

Saeed

(2018). A systematic review of cloud computing tools for collaborative learning: Opportunities and challenges to the blended-learning environment. Computers & Education, 124, 77–91. https://doi.org/10.1016/j.compedu.2018.05.016

Arora

Goel

(2012). Learning to write programs with others: Collaborative quadruple programming. In IEEE 25th Conference on Software Engineering Education and Training (pp. 32–41). IEEE. https://doi.org/10.1109/CSEET.2012.27

Balijepally

(2005). Collaborative software development in agile methodologies-Perspectives from small group research. In Proceedings of the Eleventh Americas Conference on Information Systems (pp. 3138–3146). AIS Electronic Library. https://aisel.aisnet.org/amcis2005/511.

Barke

James

M. B.

Polikarpova

(2023). Grounded copilot: How programmers interact with code-generating models. Proceedings of the ACM on Programming Languages, 7(OOPSLA1), 85–111. https://doi.org/10.1145/3586030

Beck

(2000). Extreme programming explained: Embrace change. Addison-Wesley.

Berland

Martin

Benton

Petrick Smith

Davis

(2013). Using learning analytics to understand the learning pathways of novice programmers. Journal of the Learning Sciences, 22(4), 564–599. https://doi.org/10.1080/10508406.2013.836655

Bonwell

C. C.

Eison

J. A.

(1991). Active learning: Creating excitement in the classroom. ASHE-ERIC Higher Education Report No. 1.

Buitrago-Flórez

Casallas

Hernández

Reyes

Restrepo

Danies

(2017). Changing a generation’s way of thinking: Teaching computational thinking through programming. Review of Educational Research, 87(4), 834–860. https://doi.org/10.3102/0034654317710096

Buseyne

Rajagopal

Danquigny

Depaepe

Heutte

Raes

(2024). Assessing verbal interaction of adult learners in computer-supported collaborative problem solving. British Journal of Educational Technology, 55(4), 1465–1485. https://doi.org/10.1111/bjet.13391

10.

Celepkolu

Boyer

K. E.

(2018). The importance of producing shared code through pair programming. In Proceedings of the 49th ACM Technical Symposium on Computer Science Education (pp. 765–770). ACM. https://doi.org/10.1145/3159450.3159506

11.

Chang

C. J.

Chang

M. H.

Chiu

B. C.

Liu

C. C.

Fan Chiang

S. H.

Wen

C. T.

Hwang

F. K.

Y. T.

Chao

P. Y.

Lai

C. H.

S. W.

Chang

C. K.

Chen

(2017). An analysis of student collaborative problem solving activities mediated by collaborative simulations. Computers & Education, 114, 222–235. https://doi.org/10.1016/j.compedu.2017.07.008

12.

Da Silva Estácio

B. J.

Oliveira

Marczak

Kalinowski

Garcia

de Lucena

C. J. P.

Prikladnicki

(2015). Evaluating collaborative practices in acquiring programming skills: Findings of a controlled experiment. In Proceedings of 29th Brazilian Symposium on Software Engineering (pp. 150-159). IEEE. https://doi.org/10.1109/SBES.2015.24

13.

Da Silva Estácio

B. J.

Prikladnicki

(2015). Distributed pair programming: A systematic literature review. Information and Software Technology, 63, 1–10. https://doi.org/10.1016/j.infsof.2015.02.011

14.

De Carvalho

J. M.

de Magalhães Netto

J. F.

(2020). Currents trends in use of collaborative learning in teaching of robotics and programming–A systematic review of literature. In Proceedings of 2020 IEEE Frontiers in Education Conference (pp. 1–8). IEEE. https://doi.org/10.1109/FIE44824.2020.9273950

15.

Demir

Seferoglu

S. S.

(2020). A comparison of solo and pair programming in terms of flow experience, coding quality, and coding achievement. Journal of Educational Computing Research, 58(8), 1448–1466. https://doi.org/10.1177/0735633120949788

16.

Dombrovskaia

Barrera

P. G.

(2018). A collaborative learning experience in the programming course. In Proceedings of 37th International Conference of the Chilean Computer Science Society (pp. 1–4). IEEE. https://doi.org/10.1109/SCCC.2018.8705255

17.

Ellis

C. A.

Gibbs

S. J.

Rein

(1991). Groupware: Some issues and experiences. Communications of the ACM, 34(1), 39–58. https://doi.org/10.1145/99977.99987

18.

Ellis

T. J.

Hafner

(2007). Assessing collaborative, project-based learning experiences: Drawing from three data sources. In Proceedings of 37th Annual Frontiers in Education Conference-Global Engineering: Knowledge without borders, opportunities without passports (pp. 13–17). IEEE. https://doi.org/10.1109/FIE.2007.4418028

19.

Fan

Song

Zhang

Shi

(2019). CoVSCode: A novel real-time collaborative programming environment for lightweight IDE. Applied Sciences, 9(21), 4642. https://doi.org/10.3390/app9214642

20.

Fan

Sun

Shen

(2012). ATCoPE: Any-time collaborative programming environment for seamless integration of real-time and non-real-time teamwork in software development. In Proceedings of the 2012 ACM International Conference on Supporting Group Work (pp. 107–116). IEEE. https://doi.org/10.1145/2389176.2389194

21.

Fredrick

T. A.

(2008). Facilitating better teamwork: Analyzing the challenges and strategies of classroom-based collaboration. Business Communication Quarterly, 71(4), 439–455. https://doi.org/10.1177/1080569908325860

22.

Garvin-Doxas

Barker

L. J.

(2003). Creating learning environments that support interaction. In Proceedings of the 8th Annual Conference on Innovation and Technology in Computer Science Education (p. 276). ACM. https://doi.org/10.1145/961511.961641

23.

Goodyear

Jones

Thomson

(2014). Computer-supported collaborative learning: Instructional approaches, group processes and educational designs. In Spector

J. M.

Merrill

M. D.

Elen

Bishop

M. J.

(Eds.), Handbook of research on educational communications and technology (pp. 439–451). Springer.

24.

Graesser

A. C.

Fiore

S. M.

Greiff

Andrews-Todd

Foltz

P. W.

Hesse

F. W.

(2018). Advancing the science of collaborative problem solving. Psychological Science in the Public Interest, 19(2), 59–92. https://doi.org/10.1177/1529100618808244

25.

Hadwin

Järvelä

Miller

(2017). Self-regulation, co-regulation, and shared regulation in collaborative learning environments. In Handbook of self-regulation of learning and performance (pp. 83–106). Routledge.

26.

Hanks

Fitzgerald

McCauley

Murphy

Zander

(2011). Pair programming in education: A literature review. Computer Science Education, 21(2), 135–173. https://doi.org/10.1080/08993408.2011.579808

27.

Hernández-Sellés

Pablo-César

M. C.

González-Sanmamed

(2019). Computer-supported collaborative learning: An analysis of the relationship between interaction, emotional support and online collaborative tools. Computers & Education, 138, 1–12. https://doi.org/10.1016/j.compedu.2019.04.012

28.

Hohman

M. M.

Slocum

A. C.

(2002). Mob programming and the transition to XP. In Succi

(Ed.), EXtreme programming perspectives (pp. 323–334). Addison-Wesley.

29.

Hsu

T. C.

Abelson

Patton

Chen

S. C.

Chang

H. N.

(2021). Self-efficacy and behavior patterns of learners using a real-time collaboration system developed for group programming. In International journal of computer-supported collaborative learning (pp. 1–24). Springer.

30.

Hwang

W. Y.

Shadiev

Wang

C. Y.

Huang

Z. H.

(2012). A pilot study of cooperative programming learning behavior and its relationship with students’ learning performance. Computers & Education, 58(4), 1267–1281. https://doi.org/10.1016/j.compedu.2011.12.009

31.

Isaac

O. A.

Njoku Christian

Amana

Y. H.

(2021). Enhancing the teaching and learning of computer programming using collaborative method of delivery. International Journal of Advances in Scientific Research and Engineering, 7(1), 18–23. https://doi.org/10.31695/IJASRE.2021.33960

32.

Jaya

Lumu

Haryoko

Suhaeb

(2020). Development of remote laboratory for distance learning practicum online and real-time digital electronics subjects. Journal of Educational Science and Technology, 6(1), 56–64. https://doi.org/10.26858/est.v6i1.12006

33.

JetBrains . (2021). Code with me. https://www.jetbrains.com/code-with-me

34.

Jiang

Yin

Zhang

(2020). Knowledge tracing within single programming practice using problem-solving process data. IEEE Transactions on Learning Technologies, 13(4), 822–832. https://doi.org/10.1109/TLT.2020.3032980

35.

Kattan

H. M.

Soares

Goldman

Deboni

Guerra

(2018). Swarm or pair? Strengths and weaknesses of pair programming and mob programming. In Wagner

Hoda

Aguiar

(Eds.), Proceedings of the 19th International Conference on Agile Software Development: Companion (pp. 1–4). ACM. https://doi.org/10.1145/3234152.3234169

36.

Kong

S. C.

Chiu

M. M.

Lai

(2018). A study of primary school students’ interest, collaboration attitude, and programming empowerment in computational thinking education. Computers & Education, 127(3), 178–189. https://doi.org/10.1016/j.compedu.2018.08.026

37.

Korkmaz

Ö.

Çakir

Özden

M. Y.

(2017). A validity and reliability study of the computational thinking scales (CTS). Computers in Human Behavior, 72, 558–569. https://doi.org/10.1016/j.chb.2017.01.005

38.

Lacave

García

M. A.

Molina

A. I.

Sánchez

Redondo

M. A.

Ortega

(2019). COLLECE-2.0: A real-time collaborative programming system on Eclipse. In Proceeding of 2019 International Symposium on Computers in Education (pp. 1–6). IEEE. https://doi.org/10.1109/SIIE48397.2019.8970132

39.

Lai

Wong

G. K. W.

(2022). Collaborative versus individual problem solving in computational thinking through programming: A meta-analysis. British Journal of Educational Technology, 53(1), 150–170. https://doi.org/10.1111/bjet.13157

40.

Janssen

Wubbels

(2017). Collaborative learning practices: teacher and student perceived obstacles to effective student collaboration. Cambridge Journal of Education, 48(1), 103122. https://doi.org/10.1080/0305764x.2016.1259389

41.

Leopold

Smith

(2019). Implementing reflective group work activities in a large chemistry lab to support collaborative learning. Education Sciences, 10(1), 7. https://doi.org/10.3390/educsci10010007

42.

Kraemer

(2013). Programming with concurrency: Threads, actors, and coroutines. In Proceedings of IEEE International Symposium on Parallel & Distributed Processing, Workshops and Phd Forum (pp. 1304–1311). IEEE. https://doi.org/10.1109/IPDPSW.2013.193

43.

Lister

Fidge

Teague

(2009). Further evidence of a relationship between explaining, tracing and writing skills in introductory programming. ACM SIGCSE Bulletin, 41(3), 161–165. https://doi.org/10.1145/1595496.1562930

44.

Liu

C. Y.

Huang

J. Y.

Lei

L. Y.

Zhang

P. R.

(2023). Collaborative programming based on social shared regulation: An approach to improving students' programming achievements and group metacognition. Journal of Computer Assisted Learning, 39(5), 1714–1731. https://doi.org/10.1111/jcal.12828

45.

Wang

Fan

(2023). Integrating real-time and non-real-time collaborative programming: Workflow, techniques, and prototypes. Proceedings of the ACM on Human-computer Interaction, 7(GROUP), 1–19. https://doi.org/10.1145/3567563

46.

McDougall

(2005). Issues in the assessment of real-life learning with ICT. In Information and communication technologies and real-life learning: New education for the knowledge society (pp. 21–28). Springer. https://doi.org/10.1007/0-387-25997-X_3

47.

Nosek

J. T.

(1998). The case for collaborative programming. Communications of the ACM, 41(3), 105–108. https://doi.org/10.1145/272287.272333

48.

Ochoa

(2017). Chapter 11: Multimodal learning analytics. In Lang

Siemens

Wise

Gašević

(Eds.), Handbook of learning analytics (1st ed., pp. 143–150). Creative Commons License.

49.

Oeda

Kosaku

(2018). Development of a check sheet for code-review towards improvement of skill level of novice programmers. Procedia Computer Science, 126, 841–849. https://doi.org/10.1016/j.procs.2018.08.018

50.

Pietinen

Bednarik

Glotova

Tenhunen

Tukiainen

(2008). A method to study visual attention aspects of collaboration: Eye-tracking pair programmers simultaneously. In Proceedings of the 2008 Symposium on Eye Tracking Research & Applications (pp. 39–42). ACM. https://doi.org/10.1145/1344471.1344480

51.

Pietinen

Bednarik

Tukiainen

(2009, January). An exploration of shared visual attention in collaborative programming [Paper presentation]. 21st Annual Psychology of Programming Interest Group Conference (PPIG), Limerick, Ireland. https://www.ppig.org/files/2009-PPIG-21st-pietinen.pdf

52.

Pietinen

Bednarik

Tukiainen

(2010). Shared visual attention in collaborative programming: A descriptive analysis. In Proceedings of the 2010 ICSE workshop on cooperative and human aspects of software engineering (pp. 21–24). ACM. https://doi.org/10.1145/1833310.1833314

53.

Plonka

Sharp

Van der Linden

Dittrich

(2015). Knowledge transfer in pair programming: An in-depth analysis. International Journal of Human-Computer Studies, 73, 66–78. https://doi.org/10.1016/j.ijhcs.2014.09.001

54.

Polyak

S. T.

von Davier Peterschmidt

(2017). Computational psychometrics for the measurement of collaborative problem solving skills. Frontiers in Psychology, 8, 1–16. https://doi.org/10.3389/fpsyg.2017.02029

55.

Python . (2021). Python developers survey 2020 results. https://www.jetbrains.com/lp/python-developers-survey-2020/#Methodology

56.

Rantala

Soini

Kilamo

(2015). Gathering useful programming data; analysis and insights from real-time collaborative editing. In Proceedings of 38th International Convention on Information and Communication Technology, Electronics and Microelectronics (pp. 229-234). IEEE. https://doi.org/10.1109/MIPRO.2015.7160270

57.

Shadiev

Chen

Reynolds

B. L.

Song

Altinay

(2024). Facilitating cognitive development and addressing stereotypes with a cross-cultural learning activity supported by interactive 360-degree video technology. British Journal of Educational Technology, 55(6), 2668–2696. https://doi.org/10.1111/bjet.13461

58.

Shadiev

Reynolds

B. L.

(2024). The use of digital technology for sustainable teaching and learning. Sustainability, 16(13), 5353. https://doi.org/10.3390/su16135353

59.

Sharma

Jermann

Nüssli

M. A.

Dillenbourg

(2012, November). Gaze evidence for different activities in program understanding [Paper presentation]. 24th Annual Conference of Psychology of Programming Interest Group (PPIG), London, UK. https://infoscience.epfl.ch/server/api/core/bitstreams/fef0cbc0-5d7c-41b3-957d-c4733c4940bd/content

60.

Sherin

B. L.

(2001). A comparison of programming languages and algebraic notation as expressive languages for physics. International Journal of Computers for Mathematical Learning, 6, 1–61. https://doi.org/10.1023/A:1011434026437

61.

Silva

Mendes

A. J.

Gomes

(2020). Computer-supported collaborative learning in programming education: A systematic literature review. In Proceedings of 2020 IEEE Global Engineering Education Conference (pp. 1086–1095). IEEE. https://doi.org/10.1109/EDUCON45650.2020.9125237

62.

Song

Huang

T. Y.

(2017). Python programming basics (3rd ed.). Higher Education Press. https://xuanshu.hep.com.cn/front/book/findBookDetails?bookId=66536b5e74ce561611bd9fea

63.

Sun

Shute

V. J.

Stewart

Yonehiro

Duran

D’Mello

(2020). Towards a generalized competency model of collaborative problem solving. Computers & Education, 143, 103672. https://doi.org/10.1016/j.compedu.2019.103672

64.

Sun

Ouyang

Chen

(2021). Three contrasting pairs’ collaborative programming processes in China’s secondary education. Journal of Educational Computing Research, 59(4), 740–762. https://doi.org/10.1177/0735633120973430

65.

Sweller

(1988). Cognitive load during problem solving: Effects on learning. Cognitive Science, 12(2), 257–285. https://doi.org/10.1016/0364-0213(88)90023-7

66.

Tan

Jiang

Zhang

(2024). Understanding real-time collaborative programming: A study of visual Studio live Share. ACM Transactions on Software Engineering and Methodology, 33(4), 1–28. https://doi.org/10.1145/3643672

67.

Tissenbaum

Kumar

(2019). See the collaboration through the code: Using data mining and CORDTRA graphs to analyze blocks-based programming. In Lund

Niccolai

G. P.

Lavoué

Hmelo-Silver

Gweon

Baker

(Eds.), A wide lens: Combining embodied, enactive, extended, and embedded learning in collaborative settings (pp. 680–683). International Society of the Learning Sciences.

68.

Tohyama

Matsuzawa

Yokoyama

Koguchi

Takeuchi

(2018). Constructive interaction on collaborative programming: Case study for grade 6 students group. In Proceeding of IFIP World Conference on Computers in Education (pp. 589–598). Springer International Publishing. https://doi.org/10.1007/978-3-319-74310-3_59

69.

Umapathy

Ritzhaupt

A. D.

(2017). A meta-analysis of pair-programming in computer programming courses: Implications for educational practice. ACM Transactions on Computing Education, 17(4), 1–13. https://doi.org/10.1145/2996201

70.

von Davier

A. A.

(2017). Computational psychometrics in support of collaborative educational assessments. Journal of Educational Measurement, 54(1), 3–11. https://www-jstor-org-443.web.bisu.edu.cn/stable/45148408.

71.

Vygotsky

L. S.

(1978). Mind in society: The development of higher psychological processes. Harvard University Press.

72.

Watson

F. W.

Godwin

J. L.

(2013). Predicting performance in an introductory programming course by logging and analyzing student programming behavior. In Proceedings of IEEE 13th International Conference on Advanced Learning Technologies (pp. 319–323). IEEE. https://doi.org/10.1109/ICALT.2013.99

73.

Weintrop

Wilensky

(2019). Transitioning from introductory block-based and text-based environments to professional programming languages in high school computer science classrooms. Computers & Education, 142, 103646. https://doi.org/10.1016/j.compedu.2019.103646

74.

Williams

L. A.

Kessler

R. R.

(2000). The effects of “pair-pressure” and “pair-learning” on software engineering education. In Proceedings of 13th Conference on Software Engineering Education and Training (pp. 59–65). IEEE. https://doi.org/10.1109/CSEE.2000.827023

75.

Williams

Kessler

R. R.

(2003). Pair programming illuminated. Addison-Wesley.

76.

Wilson

(2015). Mob programming-what works, what doesn’t. In Proceedings of International Conference on Agile Software Development (pp. 319–325). Springer. https://doi.org/10.1007/978-3-319-18612-2_33

77.

Ruis

A. R.

Wang

(2019). Analysing computational thinking in collaborative programming: A quantitative ethnography approach. Journal of Computer Assisted Learning, 35(3), 421–434. https://doi.org/10.1111/jcal.12348

78.

Xia

B. S.

(2017). An in-depth analysis of teaching themes and the quality of teaching in higher education: Evidence from the programming education environments. The International Journal of Teaching and Learning in Higher Education, 29(2), 245–254. https://isetl.org/ijtlhe/pdf/IJTLHE2563.pdf.

79.

Correia

A. P.

(2024). Measuring mutual engagement in the context of middle-school pair programming: Development and validation of a self-reported questionnaire. Computers in Human Behavior Reports, 14, 100415 https://doi.org/10.1016/j.chbr.2024.100415

80.

Dong

Ouyang

(2024). The effects of three scaffoldings on computer-supported, robot-assisted collaborative programming in higher education. International Journal of Human-Computer Interaction, 5, 1–16. https://doi.org/10.1080/10447318.2024.2356399

81.

Ouyang

(2023). Multimodal learning analytics of collaborative patterns during pair programming in higher education. International Journal of Educational Technology in Higher Education, 20(1), 8. https://doi.org/10.1186/s41239-022-00377-z

82.

Zhan

Wang

Pan

Ding

Liu

(2022). Teaching model design of computer programming courses for digital media technology students. Wireless Communications and Mobile Computing, 2022, 7085914. https://doi.org/10.1155/2022/7085914

83.

Zhang

(2018). An investigation of technology design features for supporting real-time collaborative programming in an educational environment [Unpublished master’s thesis]. The Pennsylvania State University.

84.

Zhang

(2023). A smart interactive and collaborative online coding platform for programming education using machine learning and web socket. In Proceedings of 12th International Conference on Software Engineering and Applications (pp. 21–33). AIRCC Publishing Corporation. https://doi.org/10.5121/csit.2023.131703