The effectiveness of an activity to practise communication competencies: A case study across five European engineering universities

Abstract

Although accreditation bodies emphasise communication competencies development in engineering curricula due to its importance for future engineers, the focus is often mainly on oral presentation and written reports. A communication activity was created to practice and reflect on students’ communication competencies including describing information in a short time, listening skills, and ask and respond to questions. This activity was implemented at five different European engineering universities involving 393 students. This study explores the effectiveness of this activity as a case-study of game-based learning to practise and reflect on engineering students’ communication competencies. A mixed-method approach measuring students’ communication performance, their perceived communication competencies, and the benefits of this activity to students was used to explore the game characteristics that make the activity effective. The findings of the scoring rubric and the questionnaire used showed that this activity is effective. Because it is engaging and set by rules, students actively participated during the activity, reflected on their effective and ineffective communication competencies, the existing communication barriers and styles, gained awareness, experienced communication in teams, and cooperated with different people. This study highlights the effectiveness of a small intervention using a proven communication activity, which is freely available as OpenCourseWare through TU Delft for anyone to use.

Keywords

Game-based learning engineering curriculum communication competencies case study

Introduction

With the fast-changing demands of society, graduate engineers are required to possess both technical and transversal competencies.^1,2 Transversal competencies have gained importance in engineering curricula as engineering students equipped with these competencies are deemed more capable to enter the labour market and have a smoother transition from education to employment.³ Transversal competencies in this context are defined as “skills, values, and attitudes that are required for learners' holistic development and for learners to become capable of adapting to change”⁴ in different disciplines and professions.² They consist of competencies such as communication, teamwork, problem-solving, entrepreneurship, and lifelong learning. This study will focus on communication competencies which was previously defined by Leandro Cruz et al.⁵ as “the ability to show understanding and to carry technical/non-technical written/oral presentations and discussions depending on the audience where the feedback loop of giving and receiving opinions, advises and reactions is constant”.

Communication competencies are important for engineering graduates according to industry^6–8 and academia⁹ as they are required in engineer’s professional life. More nuanced communication competencies, considered important by industry and requiring higher competency levels for engineering graduates, were identified.⁸ Among the important competencies were adaptive communication style while competencies requiring higher levels of competence were writing, listening, and presentation skills. However, according to industry, engineering graduates still lack communication competencies such as listening skills¹⁰ and oral and written communication skills¹¹ when entering the workforce. For this reason, the introduction of communication competencies in the engineering curricula was considered a desirable outcome by accreditation bodies such as the ABET Engineering Criteria¹² in the USA and the European Network for Accreditation of Engineering Education¹³in Europe, as well as several engineering departments.¹⁴ However, in overcrowded curricula, to go from desire to a realisation is often difficult, as it requires finding space within a curriculum and a willingness to create and implement new activities, workshops and courses.¹⁵

So far, practises, such as oral presentations¹⁶ and written reports,¹⁷ were introduced mostly in project-based learning courses to enhance students’ communication competencies. However, communication competencies extending to informal listening and speaking are rarely addressed in the engineering curricula. In a Dutch university of technology, lecturers mentioned they practised listening skills during their courses through feedback and asking questions to students⁸ and students feel they are highly competent in listening skills.¹⁸ However, the same picture does not apply to pitching skills. Students feel they have low mastery levels in pitching skills¹⁸ and lecturers do not have students practise these skills as much. The same applies to the practising of writing skills in their courses.⁸ Therefore, as part of the European Erasmus+ project PREFER (Professional Roles and Employability of Future EngineeRs), which aims to reduce the transversal competency mismatch in the field of engineering, the Delft University of Technology created a ‘plug-and-play’ communication activity called Chinese Whispers with a Twist.

This activity is a game-based learning practise in which students cooperate in teams and experience other forms of communication rather than oral presentations and written reports. It intends to provide students with the opportunity to practise and reflect on effective verbal and visual communication. This includes active listening, describing information within a time-limit, effectively asking and answering questions, and drawing images. This study presents the outcomes of this activity as was implemented in five European engineering institutions - TU Delft (The Netherlands), KU Leuven (Belgium), TU Dublin (Ireland), IST and UMinho (Portugal) in a variety of courses and engineering degrees over a period of one-and-a-half years.

The aim of this study is to investigate the effectiveness of a game-based learning activity to practise and reflect on the communication competencies of students in different fields of engineering.

The main research question addressed in this study is: What are the characteristics of a game based-learning practice that stimulates engineering students to practise and reflect on their communication competencies?

To answer this question, the game characteristics that make this activity effective will be explored in a case-study using a mixed-method approach that investigates how students evaluate their communication competencies and areas for improvement in the game based-learning activity, how these correlate with their performance in the activity and finally the communication competencies that engineering students become aware of or experience in the game based-learning activity.

Communication competencies in engineering professions

To develop students’ communication competencies, it is essential to understand communication in the engineering profession. Novice engineers report that they spend 32% of their time using verbal communication with other people and 28% writing.¹⁹ Also, young engineers spend a great part of their time listening.¹⁹ In the study of Lievens,²⁰ engineers mentioned they spend on average 57% of their working hours on active communication such as writing e-mails and reports, making phone calls and having meetings. Lappalainen²¹ argues that, in technology sectors, engineers are constantly exchanging information between other engineering fields and society. They need to communicate effectively to show their vision, to put plans into practice, and to stimulate feedback mechanisms.

Regarding writing skills, differences between writing in academic and industry settings were observed.²² At university, students are required to write the content learned in an elaborative way, while in industry, because of limitations in time due to short deadlines, communication should be brief and concise. Also, in the study of Moore and Morton,²² engineering students are reported to have problems with adjusting their writing to their audiences and context. Often technical and academic language is used instead of simple language. Being able to adapt one’s communication style to a purpose and an audience was a key requirement indicated by industry.^8,22 Since communication is an active process of listening, adapting conversation styles, and using feedback in terms of giving and receiving opinions and responses,²³ engineering curricula must not limit communication training to just giving oral presentations and writing technical reports.

Learning by doing

Evidence of the importance of communication competencies for engineers is clear. The next step is to study which teaching strategies exist to practise communication competencies in engineering education.

Learning by doing as a practice has been well-known for more than two thousand years. The Greek philosopher Aristotle stated that “for the things we have to learn before we can do, we learn by doing”²⁴ and the Chinese Philosopher Confucius mentioned that “I hear and I forget. I see and I remember. I do and I understand”.²⁵ Active learning has gained even more attention over the past years. It is defined as a teaching and learning practice that engages students in doing things and think about what they are doing.²⁶ According to Kolb,²⁷ in experiential learning theory, knowledge is generated through direct experience. Learning happens concretely and actively, through interaction between people and their environment. Kolb promotes that abstract concepts cannot be learned with traditional educational techniques (e.g. books and lectures) because they are disconnected from the experience. According to Kolb's theory,²⁷ learning happens in a cycle where four modes are confronted: concrete experience (learners emerge themselves in the experience), reflective observation (learners reflect on what they experienced), abstract conceptualisation (learners conceptualise and draw conclusions from their experiences) and active experimentation (learners critically use the knowledge gained from the experience in future scenarios).

This study will consider game-based learning as an approach to active learning. Game is “a form of voluntary play that is structured by a set of rules, where players may make choices that can influence the actions of other players and the overall outcome”²⁸ and has a feedback system in which participants draw parallels between the game and real-world scenarios.²⁹ The type of games can go from live-action, board games to digital games.^28,30 Engineering education research²⁹ showed that games are effective to understand complex subjects, provide student engagement and increase their interest, motivation and confidence. A game element that positively impacted students’ motivation is cooperation.²⁸

In cooperative learning, students work in small groups towards a common goal.³¹ Students should have a role to be able to accomplish the task and the responsibility of the learning of the group is shared by all group members. In contrast, in collaborative learning students are responsible for their own individual work and learning.³¹ While cooperative and collaborative learning each have independent historical developments and philosophical roots, they have in common that students work in groups instead of individually.³² Research in engineering education^31,33 showed that cooperation improves academic achievement, interpersonal interaction and student attitude compared to students working as individuals. Cooperation was also compared to competition^31,33 and results showed that cooperation improves interpersonal interaction, social support and self-esteem. Cooperative learning also promoted communication and teamwork compared to lectures.³⁴ As engineers in their workplace need to think critically, solve problems and make decisions as a team, providing collaborative learning practices in engineering curricula is essential to develop transversal competencies and prepare students for their future workplace.

Game-based learning was also used to improve students’ communication competencies in engineering education.³⁰ In this study, a mixed-method approach was conducted to assess students’ perceptions and performance of oral and written communication skills. Findings showed that students improved their communication skills but did not perceive improvements. This study concluded that although students did not perceive an improvement of their communication skills, the use of games gave students a learning experience that stimulated communication.

However, a systematic literature review on game-based learning in engineering education has reported several limitations on game practices.²⁸ First, the majority of studies assess students’ perceptions and attitudes and do not consider the learning outcomes achieved by students. Second, issues were found with the reporting of the results in the research examined. Many studies did not provide validation evidence or procedures for instruments used. A final limitation was that in much of the research self-selected or small sample sizes were used.

This literature review showed the importance of experiential, cooperative and game-based learning to develop transversal competencies such as communication and teamwork and create students’ engagement and motivation in the learning process through active learning and reflective thinking. The points highlighted in this review were incorporated in the design and research of our communication activity.

Communication activity: Chinese whispers with a twist

Learning outcomes

The communication activity was designed to achieve the following learning outcomes. At the end of this activity, students will be able to:

Experience effective oral and visual communication through active listening, describing within a time limit, effectively asking and answering questions and drawing images

Understand the importance of effective oral and visual communication for engineers

Design of the activity

The communication activity is based on the universal children’s game, known as Chinese Whispers in the Commonwealth English world.³⁵ This activity is called Chinese Whispers with a Twist because instead of passing around a message and comparing it at the end as in the original game, in this version, the participants are given an image, have to pass instructions to the next person to draw it and compare the image at the end. This activity lasts one hour and allows students to practise their communication competencies by actively listening, describing information in a short time, and effectively asking and answering questions. It is performed in groups of 4–6 people and each group is divided into three subgroups, each of which has a set role (A, B and C). The rules and dynamics of this activity are shown in Figure 1. The vector image used as the drawing in the activity at all five universities is shown in Figure 2. This image was chosen so that it could be used in all university contexts. At the end of the activity, a 10 to 15-minute feedback session is held in which students are encouraged to reflect on their communication competencies and the performance of the team as a whole, as well as on how this activity can be related to a real-world engineering environment. The goal of triggering student reflection was also achieved by having students complete a questionnaire, which had not only a research function but also an educational purpose. From the experiential learning approach,²⁷ this study provides an activity in which students engage in an experience and actively reflect on it.

Figure 1.

Rules of the communication activity per role of each team.

Figure 2.

Image used in the activity and shown to students in role A.

In line with best practices at TU Delft, the activity, with supporting materials for lecturers, has been published as OpenCourseWare¹ at the OpenCourseWare Website of TU Delft.

Method

Participants

Data were collected between March 2018 and October 2019 over five European engineering universities: TU Delft, KU Leuven, TU Dublin, IST and UMinho. The activity was implemented at the first three universities because they were part of the PREFER project and in the last two as researchers at these institutions had shown an interest to implement it in their institutions after learning about its existence.

The communication activity in TU Delft was integrated into the Forensic Engineering course, an elective for Aerospace Master students in the faculty of Aerospace Engineering and was delivered to a multinational class of first-year Master students in March 2018 and May 2019.

In KU Leuven, the activity was implemented twice. First in September 2018, in a one-week summer school with international engineering Master students of KU Leuven and FH Dortmund, and second in April 2019, with Flemish Master in engineering students at different KU Leuven campuses.

The activity in TU Dublin was carried out in March 2019 with a group of first-year Bachelor students in a project-based learning course in the Civil engineering faculty.

At IST, ten lectures were given in October 2019 to first-year Master students of Computational Engineering in the course Independent Studies that focuses on communication.

Finally, at UMinho the activity took place with students in the first semester (October 2019) of the first year of the Master in Engineering Project Management.

The activity was conducted in English at TU Delft, TU Dublin, in the first implementation at KU Leuven and two lectures in IST. English was used when at least one non-native speaker was present in the activity. In the second implementation at KU Leuven, the activity was carried out in Dutch and at UMinho and IST, the activity was delivered in Portuguese.

In total 393 students participated in the activity (53 at TU Delft, 27 at KU Leuven, 6 at TU Dublin, 282 at IST and 25 at UMinho). Ethics approval was granted for this study by the university’s Institutional Review Board and 385 participants consented to be part of this research. Data were analysed anonymously.

Data analysis

This study took a mixed-method approach utilising the Chinese Whispers with a Twist as a case study to investigate what characteristics make the game-based learning exercise effective for students of different engineering fields to practise and reflect on their communication competencies. The characteristics of this game-based learning activity are first analysed based on the performance of the groups using the scores given to each of the outcome drawings produced by each group during the activity. The rubric (Table 1), developed based on Stevens³⁶ and on the information present in the image in Figure 2, was used to score the drawings on the number, position, and colour of the objects. This way, the drawings provide an objective quantitative measure of the communication performance of the groups.

Table 1.

Rubric used to compare the group outcome drawings to the image in Figure 2.

Objects	Amount	Colour	Position	Details
□ Tree	□ 3	□ Black	□ Middle	□ The tree on the left is the biggest (at the front)□ The tree in the middle is the smallest (at the back)□ The tree on the right is medium size (compared to the other two)□ The trees on the left and middle have triangular tree tops □ The tree on the right is circular
□ Bull/cow	□ 1	□ Black	□ Bottom right	□ Tail□ Horns□ Hair□ Head to the right
□ Fence	□ 1	□ Black	□ Bottom left	□ Horizontal alignment□ 4 horizontal wooden sticks□ 2 vertical wooden sticks
□ Grass		□ Black	□ Bottom	□ ¼ of the image
□ Ears of wheat	□ 2 bunches	□ Black	□ 1 bunch between the 2 trees on the left □ 1 bunch on the right	□ Each bunch has 3 ears of wheat
□ Mountain	□ 2	□ Orange and yellow	□ Middle	□ Rounded mountains□ Orange on the top and yellow on the bottom
□ Sky		□ Orange and yellow		□ Orange on the top and yellow on the bottom

Note: One point was given to each item (objects, number, colour, position and details) of the rubric (maximum points = 43).

In addition, a paper-and-pencil questionnaire, delivered at the end of the activity, collected information on students' perceptions of their communication performance in the activity (“1.1 A. How good did you feel your communication skills were in this activity?” on a 5-point Likert scale - very good, good, neither good nor bad, bad and very bad, and “1.1B. Explain briefly why.”), on points of improvement (“1.2. What do you feel you can improve on?” given a range of options [pay attention to details, listen to others, ask questions, reply to questions, describe information, write down information and other] and students could select more than one response) and on the importance of communication competencies (“2 A. Do you feel that this activity helped you to understand the importance of communication?” on a 5-point Likert scale - strongly agree, agree, neither agree nor disagree, disagree, strongly disagree, and “2B. Explain briefly why.”).

This study intended to investigate how effectively students performed as a group but also as individuals. The aspects that the students focused on in the rubric as a group and the aspects expressed by students individually are compared to see the effectiveness of students’ communication competencies.

In this paper, the open questions (1.1B and 2B) will be referred to as OQ1 and OQ2 from here on. Unlike OQ1, which was present for all implementations of the activity, OQ2 was added to the questionnaire only in the last three implementations (at TU Delft 2019, IST and UMinho) to gain deeper understanding of the quantitative data. The quantitative data was originally the only method of assessment used to understand whether students gained awareness of the importance of communication competencies in the first implementations (TU Delft 2018, TU Dublin, KU Leven 2018 and 2019). The analysis of the data in such implementations suggested the need to add OQ2.

The open responses (i.e. students’ explanations) were analysed by a multidisciplinary team of researchers with backgrounds in psychology, engineering and education. The analysis followed the steps described in Figure 3. A General Inductive Analysis was used, which is similar to Grounded Theory but instead of drawing a theory as in the Grounded Theory approach, it provides an understanding of how students experienced the communication activity.³⁷ Quotes are provided to explain themes that emerged from the analysis. They are between quotation marks and have been labelled using the role, group and university of the students, e.g. ‘C1.TUDublin’ is a student from TU Dublin with role C of group 1. As more than one lecture was conducted at IST per day, those student codes for IST were created as follows: ‘A6.ISTTh1’ is a student with role A, group 6, participating in the lecture on the first Thursday at IST.

Figure 3.

Process followed by the authors during the qualitative data analysis of the responses students provided in open question 1 and 2 (OQ1 and OQ2).

To ensure consistency within the findings, Fleiss’ kappa was calculated in R to measure agreement between the three raters.³⁸ After the first round of coding Fleiss’ kappa = 0.571 and 0.630 for OQ1 and OQ2, respectively. After discussion among authors 1, 2 and 3 about their individual attribution of the codes, the Fleiss’ kappa improved to 0.988 and 0.954 for OQ1 and OQ2, respectively. Agreement was not achieved when there was not a specific code linked to attribute (e.g. a code regarding the outcome drawing: “Because we got a good score” A6.ISTTh1) or when students’ responses were not well written grammatically.

Results

This section first presents the communication performance of students according to the group drawing scores. This is followed by reporting on the analysis of the perceptions of students, based on the quantitative and qualitative data provided in the questionnaire. Finally, it describes the benefits of the activity according to the quantitative and qualitative responses given by students in the questionnaire.

Students’ performance

Seventy-two outcome drawings were produced by the groups over the five universities. The drawings’ scores per category (objects, number, colour, position and details), as present in the rubric, are shown in Table 2. These scores give the aspects that the groups focused on during the activity and as a result, how effective each group of students performed.

Table 2.

Drawing scores, assessed using the rubric, of the groups who score higher than 75% of the points (highlighted in bold) and less than 50% of the points.

University	Year	Group ID	Rubric scores
University	Year	Group ID	Objects	Number	Colour	Position	Details	Group total
TU Delft	2018	4	7	4	7	7	11	36
IST	2019	5 Mo3	7	5	7	7	9	35
IST	2019	2 Th1	7	5	7	5	11	35
TU Dublin	2019	1	7	4	7	6	9	33
IST	2019	2 Mo2	6	3	0	3	9	21
IST	2019	2 Mo3	5	3	4	2	7	21
IST	2019	5 Mo2	6	3	0	3	8	20
IST	2019	3 Mo3	5	4	3	5	3	20
IST	2019	3 Fr1	5	4	3	4	4	20
IST	2019	3 Th1	5	4	0	4	7	20
IST	2019	2 Tu2	5	3	4	3	5	20
IST	2019	2 Tu1	5	3	2	3	7	20
IST	2019	4 Th3	4	3	2	3	8	20
IST	2019	5 Mo1	6	2	3	3	5	19
IST	2019	4 Th1	5	3	4	1	6	19
IST	2019	1 Mo4	6	3	1	3	4	17
UMinho	2019	6	5	3	1	3	5	17
IST	2019	4 Tu1	5	3	0	2	7	17
UMinho	2019	5	4	4	0	3	6	17
KU Leuven	2018	2	4	3	1	4	5	17
IST	2019	6 Mo3	4	2	2	2	7	17
KU Leuven	2018	3	3	3	3	3	5	17
IST	2019	4 Mo2	5	2	0	4	5	16
IST	2019	4 Th2	4	3	1	2	6	16
UMinho	2019	1	6	4	0	4	7	15
IST	2019	5 Tu1	6	2	4	1	2	15
UMinho	2019	3	5	2	1	1	6	15
IST	2019	1 Mo2	3	2	0	3	6	14
UMinho	2019	4	4	2	0	1	5	12
Rubric Total			7	5	7	7	17	43

Note: The maximum possible score was 43 points (with Mo - Monday, Tu - Tuesday, We - Wednesday, Th - Thursday and Fr - Friday).

The results showed that four groups: 4.TUDelft (drawing on the left in Figure 4), 1.TUDublin, 5.ISTMo3 and 2.ISTTh1 scored more than 75% of the points, meaning that the flow of communication between these participants worked effectively. These groups met most of the aspects present in the picture. They identified all the objects, number, colours, positions and even small details such as the ears of wheat. Conversely, 25 groups, as indicated by the drawing on the right in Figure 4, did not even score half of the points (Table 2), meaning that they had issues communicating with each other. They missed several aspects in the picture such as objects and their colours and number, as well as most of the details.

Figure 4.

Drawings of groups who score higher than 75% (left), between 50% and 75% (middle) and lower than 50% (right) of the points of the rubric.

The average groups, who met between 50% and 75% of the aspects of the picture, as shown by the example drawing in the middle of Figure 4, performed well on the identification of objects. However, some of these groups missed the colours of the objects, their number and positions, and other groups missed out the details.

Students’ perceived performance

Communication competencies

Looking at students’ perceptions of their performance in the communication competencies during the activity (Figure 5), 14% of the students mentioned they felt they were very good communicators and 63.1% of the students reported they felt they were good communicators. A small percentage of students, 20.5 and 2.3% perceived they were neither good nor bad or bad communicators, respectively.

Figure 5.

Students’ perceptions (N = 385) on their communication competencies on a 5-point Likert scale (very good, good, neither good nor bad, bad, and very bad).

To further investigate students’ performance during the activity, students were asked to explain their perceptions of their communication competencies performance (OQ1). Two categories emerged from the qualitative data analysis of OQ1: C1) effective communication competencies and C2) ineffective communication competencies. The themes and the number of times students mentioned them are present in Table 3.

Table 3.

Themes attributed to students’ responses to OQ1 of the questionnaire (N = 385).

Themes	# Students
Themes	Effective (+)	Ineffective (–)
Conveying/explaining/answering	167	36
Asking questions	48	28
Details	46	82
Planning/strategy	44	13
Listening	20	3
Managing	20	34
Taking notes	11	6
Noise	11	27
Observing	8	1
Participating	3	2
Remembering	2	16

Note: # Students is the number of students who mentioned each theme.

The two categories and their associated themes are used to summarise how students perceived their communication competencies performance. Most students provided balanced experiences of effective and ineffective communication competencies during the activity. Since effective and ineffective experiences were reported by students for the same themes, they will be presented together.

(In)Effective communication competencies: Themes in these categories focused on the ineffective and effective communication competencies perceived by students and how that hinders or benefits their performance during the activity.

Conveying/explaining/answering+

Almost half of the students perceived they were effective communicators in the activity when they provided clear explanations and responses to the questions posed and conveyed the message to their peers. Examples that expressed these were:

“Everything was explained accurately.” (A4.TUDelft2018)

“I tried to convey information as efficiently as possible by visual (charades) and descriptive (talking) means.” (A1.TUDublin)

“I answered all questions with details known to me, without expanding into aspects not specifically asked for.” (B1.KULeuven2018)

Conveying/explaining/answering-

This theme was not the most selected theme under the category of ineffective communication competencies, contrary to its opposite above. However, many students recognised they could have explained and answered better, be clearer and more coherent. Students also mentioned difficulties to convey, explain and respond to questions because they were disorganised and rushed, did not gather nor receive enough information, did not know how much information to provide and because time was limited. Also in this theme are comments in which students expressed that they did not describe or answer with detail. Examples of these were:

“Because I was ineffective in explaining everything, I had seen in 2 minutes.”² (A2.ISTMo2)

“Did not explain clearly the information to role C.” (B5.ISTMo1)

“I could've responded more specific and detailed.” (B4.ISTFr1)

Asking questions±

This was the second most referred to theme under the category of effective communication competencies. Students mentioned what kind of questions they posed. They go from specific, in-depth and detailed to open and broad questions. Other adjectives students used were good, relevant, clear, useful and right questions. Students also stated how they formulated questions. They either asked a lot of questions, with a specific structure or asked questions based on the answers given. Examples in this theme were:

“Asked both in-depth and broad, general questions” (C1.KULeuven2018)

“As C, I went straight to the point and started by ”what did A describe to you?“. After that, I asked questions like sizes, colours and relative positions.” (C2.ISTMo3)

“We overloaded the B's with open questions in a structured manner, getting a ton of information.” (C5.TUDelft2019)

Asking questions-

Issues with asking questions were identified by many students in the questionnaire. Some students mentioned they did not ask enough questions or that they asked questions that were too specific or too general. Students also stated they were not able to ask questions because they were nervous and limited by the time constraints. Examples provided by students were:

“I asked very specific and straightforward questions, instead of asking something broader.”^b (C1.UMinho)

“I should have asked more specific questions” (C5.ISTTu1)

“Because I was nervous, I didn't know how to ask the right questions.”^b (C3.UMinho)

Details±

This theme contained all the comments under effective communication competencies that students stated the word detail or similar words to that effect. They mentioned that they wrote down details, paid attention to them, explained with enough detail, received detailed information and ask and answered with detail. Examples of these were:

“I focused on the details.” (B1.ISTTu1)

“I explained the details as much as I could.”^b (A2.UMinho)

“I pay attention to details and transmit this information to others.” (A5.ISTMo1)

Details-

This theme was the most referred to under the category of ineffective communication competencies. As in the theme “Details+”, every time details were mentioned in a negative context, it was considered in this theme. Several students in this theme stated they were able to describe information, answered and responded to questions effectively, however, they mentioned they forgot, mixed up, and missed details. In contrast, other students mentioned specifically that they did not describe some details, ask what details to draw, respond with detailed answers nor paid attention to details. Also, in this theme comments of students can be found that express the causes for not being able to communicate the details. Causes listed were time constraints, wrong focus, and large amounts of information received. Examples given by students in this theme were:

“We could communicate the most important aspects of the image but forgot to mention some details.” (A3.ISTMo1)

“When trying to get a lot of information given by my colleagues, I was not able to pay much attention to details.”^b (B1.ISTMo2)

“I tried to communicate with detail in the simplest way possible but wasn't fast enough so some details were missing from my explanation.” (A4.ISTMo1)

Planning/strategy±

Under this theme, comments were found that showed that students had plans and strategies during the activity to describe, listen, ask, respond and draw the image. Some students only mentioned they were organised and had a structure in place. Others stated specifically their strategies, as seen in the following examples:

“We had a good strategy going from left to right and bottom to top. One person speaking.” (A6.TUDelft2019)

“We have grouped the information in different categories (trees, background, etc.)” (B7.ISTTh3)

“We were able to divide tasks and prepare what we were going to say.” (A5.ISTMo3)

Planning/strategy-

On the other hand, some students stated that they lack planning and organisation during the activity. Most students stated these issues as a point for improvement. They realised they should have been more organised when taking notes and describing the image, planned what to say in the short time and to have a strategy of how to ask questions. Students' responses showing these issues were:

“More organization was enough. We both tried to speak, repeating many things and leaving others.”^b (A4.ISTTh2)

“I could have asked more questions and not assume as much, and try to begin from left to right.” (C3.ISTTh1)

“Next time I would take this into account in the organization of the notes and in the way, I pass information.” (A1.ISTMo2)

Listening±

In this theme, comments of students that referred to listen, receive, pay attention, focus, capture and catch information were considered. Students mentioned they were able to receive the information described, listen to the questions posed and pay attention to details. One student also mentioned the importance of listening and show interest in the contribution of their colleagues. Examples in this theme were:

“I listened to both A's descriptions and C's questions attentively.” (B2.ISTMo3)

“Listened for most details” (B1.TUDublin)

“Listening to my colleague so that it is not just from my viewpoint”^b (A2.UMinho)

Listening-

In constrast, three students reported encountering problems when listening either because of teamwork or because of time management issues. Examples provided by students were:

“Didn't listen/take into account the suggestion of another team player “C”.” (2C.KULueven2018)

“It was hard to listen while in a hurry.” (5B.ISTTu1)

“When we started, we realized we only had 1:15 minutes left so we had to rush a bit, and we ended up causing group B to mishear “vaca” as “barra”.” (5A.ISTMo2)

Managing±

This theme referred to time management during the activity. Few students perceived that time was not a limitation to complete their tasks during the activity and others were more specific, saying they were efficient using their communication competencies. Examples in this theme were:

“There was enough time for it.” (B1.ISTTu2)

“Fast and effective communication.”^b (B2.ISTTh1)

“We managed to tell most of it in two minutes.” (A6.TUDelft2019).

Managing-

Students mentioned “Managing-” more frequently than “Managing+” which means that more students found that managing the limited time of the activity harder. Due to the time constraints, students stated they forgot information, could not take all the notes nor listen attentively and were not able to describe things clearly, because they were either too fast or too slow, focused too much on details, or failed to mention them all. Students expressed time management issues as follows:

“Passing all the information was difficult in 2 minutes.”^b (A2.ISTTu1)

“Time constraint makes you forget the simplest questions.” (C2.TUDelft2019)

“We spent too much time in discussing detail on a particular part of the sketch and that resulted in explaining badly in a rushed manner the other elements.” (A4.ISTMo1)

Taking notes±

Most students in this theme mentioned that they took notes to be able to transmit the information they saw. Few of them also took notes while listening to the description of the image. Students expressed these as follows:

“I think I wrote down all the right details.” (A2.TUDelft2018)

“Everything that was in our notes was successfully passed to group B.”^b (A5.ISTTh3)

“When listening, I tried to balance sheer memory and note-taking.” (B1.ISTTu1)

Taking notes-

Under this theme, students recognised they should have taken notes and not only memorised the information received. Also, students who took notes mentioned they were slow or did not take the right notes. Examples given by the students were:

“I should have written the details rather than memorize everything.” (3B.ISTTh3)

“I couldn't write down all the information I got.” (3B.ISTTu1)

“Because the notes I had taken weren't very helpful.” (5B.ISTMo1)

Noise±

Comments in this theme referred to the understanding and interpretation of information. Some students stated their colleagues understood what they described and asked. Other students realised that they understood the information they received. Examples in this theme were:

“For my part, everything I described was well interpreted.”^b (A6.ISTMo4)

“I managed to make myself understood in the questions I asked and I understood everything I was told.”^b (C1.ISTFr1)

“I didn't start drawing until I understood exactly what and where everything in the picture is.” (C1.TUDublin)

Noise-

More students have identified this theme than “Noise+”. Comments here were references to barriers in students’ communication, such as the assumptions they made and their lack of understanding. Students mentioned that their predefined ideas made them wonder about what to and how much to describe, what kind of questions to ask, and how detailed they should have answered. Another barrier pointed out by students was the lack of understanding of the activity itself, and of what others described and asked during the activity. Examples of this theme were:

“Found myself somewhat limited by my own pre-conceptions of what I would and could not do.” (B4.UMinho)

“Assumed things that weren't said (assumed sky was blue, grass was green…)” (C2.ISTTu2)

“Lack of understanding in the questions”^b (B6.UMinho)

Observing±

Comments here referred to observing and analysing the image to capture the necessary information to be transmitted for the next group. Not surprising, this theme was only referred to by students in role A as it was their given function. Examples provided by students were:

“I was able to observe a lot of details.”^b (A5.UMinho)

“I think we did a good job at analysing the image.” (A5.ISTMo1)

“We managed to capture more of the important aspects.” (A6.ISTMo1)

Observing-

Only one student mentioned not observing the image effectively as follows: “I did not observe attentively the smallest details of the images.” (6 A.ISTMo1)

Participating±

Under this theme, students stated that they participated actively in the activity. One student referred to their participation and the other two to the work in groups, as follows:

“Interventional, interested, commented.”^b (C5.ISTh2)

“Because we were all working for the same thing and there was a commitment from everyone to be able to draw.”^b (C3.ISTMo1)

Participating-

Only two students mentioned that they did not participate actively during the activity. Examples of these were:

“I didn't speak too much.” (1B.ISTTu2)

“I felt I could've interacted more in answering the questions.” (2B.ISTFr1)

Remembering±

In this theme, only two students in role B mentioned that they were able to keep the information described to them (“I was basically able to retain the necessary information to help describe what the drawing would become.”^b B5.ISTMo4).

Remembering-

Many more students referred to this theme than “Remembering+”. They recognised that their memory was not good enough because they forgot something. Also, students specifically mentioned they forgot to describe the details of the image and to provide all the details received and ask certain questions. Students mentioned that time constraints and assumptions were the cause for forgetting to explain and ask. Examples provided by the students were:

“My memory could be better.” (7B.ISTTu1)

“We forgot some details when answering C's questions.”^b (5B. ISTMo2)

“Time constraint makes you forget the simplest questions.” (2C.TUDelft2019)

Points for improvement

The researchers were also interested in investigating whether students recognised points for improvement (Figure 6). Pay attention to details and describe information were points for improvement frequently indicated by students in the three roles (Figure 6). Other points for improvement indicated by students were describing information clearly, slower or faster, with more detail and from general to detail, asking for clarification and more questions, managing time, organising speech and planning a strategy to describe information, listening before rushing to draw and take notes and not assuming or ignoring information.

Figure 6.

Points of improvement indicated by students (N = 385) per role (A, B and C). More than one option was allowed.

Importance of communication perceived by students

Students were also asked whether this activity helped them to understand the importance of communication and why. The results of the questionnaire showed that 38.7% and 52.5% of students strongly agreed and agreed, respectively, and that this activity helped them to understand the importance of communication (Figure 7). Only 7.8%, 0.8% and 0.3% responded neither agree nor disagree, disagree and strongly disagree.

Figure 7.

Students' perceptions (N = 385) on whether this activity helped them to understand the importance of communication on a 5-point Likert scale (strongly agree, agree, neither agree nor disagree, disagree, strongly disagree).

To further explore the quantitative data, the written explanations of the students were analysed. Three categories emerged from the qualitative data of OQ2: C1) Awareness, C2) Experience and C3) No gain. Themes arising from the analysis were grouped into one of these categories (Table 4) and were used to summarise the benefits of the activity perceived by students. The findings showed that this activity created awareness and experience of communication competencies for most students.

Table 4.

Themes and categories that emerged from students' responses to OQ2 of the questionnaire (N = 385).

Categories	Themes	# Students
C1) Awareness	Barriers in communication	67
	Details	49
	Conveying/explaining/answering	46
	Team/group communication	42
	Importance of communication	37
	Asking questions	26
	Managing/planning/organising	22
	Understanding	21
	Strengths and points for improvement	20
	Listening	19
	Assumptions	10
	Taking notes	8
	Communication styles	8
	Observing	1
C2) Experience	Communication process	15
	Team/group communication	7
	Engagement/fun	5
	Interaction between people	2
C3) No gain	Have awareness already	18
	Nothing new	6
	Have experience already	4

Note: # Students is the number of students who mentioned each theme.

C1) Awareness: This category concerns the awareness students gained during the communication activity. Most students perceived they gained awareness of this activity.

Barriers in communication

This theme was the most selected by students. General comments about the difficulties in the communication process were pointed out by students here. Students perceived that effective communication is hard because information can be easily misinterpreted or lost in the communication chain. These difficulties impacted communication outcomes according to students. For that reason, they recognised that communication should be as concise and clear as possible. Examples given by students were as follows:

“I had no idea how hard it could be to transmit information between sources.” (A2.ISTTh3)

“It was a good way to show how information can get lost, twisted and wrongly interpreted in a verbal description.” (A4.ISTFr1)

“Because we realize the impact that communication failures can have on a project's final result for example.”^b (A6.ISTTh1)

Details

Under this theme, comments that referred to details were grouped. They go from paying attention to details, providing detailed explanations and responses and asking detailed questions. Students also mentioned how details can be easily lost and forgotten and how selecting and communicating the fundamental details can impact the communication outcome. Examples provided by students in this theme were:

“This activity helps us to understand how important it is to pay attention to details, ask the right questions and in detail.”^b (A1.ISTMo4)

“Interesting to see the amount of detail that's lost when communicating, especially the thing that seem obvious like it's a silhouette.” (C3.TUDelft2019)

“It's a good example of how missing one detail/getting something wrong can make a huge difference in the outcome.” (C4.ISTMo3)

Conveying/explaining/answering

This theme encompassed comments in which students were specific about providing clear and concise information. Some students expressed how this activity helped them to gain awareness of this and others mentioned the consequences of not doing it effectively. Examples in this theme were:

“It's important to understand what to answer when you need to pass some information.” (B7.ISTMo3)

“You should really try to explain something with a neutral base and think about possibilities how it may be understood wrong.” (A5.ISTTu1)

“The hurried speech was responsible for communication failures.”^b (C5.ISTTh1)

Team/group communication

Under this theme were general comments in which students referred to gaining awareness of the difficulties of working in teams, the communication flow between team members, and the engineering role in the team. Students’ responses in this theme were:

“The information required strongly depended on the efficiency of the communication between each group.” (C2.TUDelft2019)

“Because it helped understand how communication can be lost between two groups.” (C1.ISTMo4)

“It’s important for us to be aware of what happens in the “real” world, as we will play roles that may be included in any of these roles.” (A5.ISTTh3)

Importance of communication

It is not surprising that the importance of communication was mentioned by students, given that they were prompted (in the questionnaire) to explain how the activity helped them to understand the importance of communication. Comments in this theme were generic in which students stated that communication is key to achieving objectives, is essential for engineers and generates interaction with others and relationships. Also, in this theme, comments were given that referred to the positive impact of communicating effectively, especially in engineering professions. Students expressed the importance of communication as follows:

“Communication is the basis of all relationships. It is important to know how to communicate to get the best results, especially if there are intermediaries.” (B4.ISTMo2)

“Because good communication is essential when trying to describe things to others, otherwise the information transmitted might be misunderstood.” (A5.TUDelft2019)

“This activity is definitely associated with the computer software production cycle. This process is only possible if there is effective communication.” (A1.ISTTh1)

Asking questions

Under this theme, students stated that this activity helped them to gain awareness of effectively asking questions. Students mentioned that making all the necessary questions (general and in-depth) help their understanding, acquisition of information and transmission of the message received. They expressed these as follows:

“It showed me the importance of details and questioning the aspects we usually take for granted.” (A1.ISTMo1)

“The straightforward question of team C: “What did they describe you?” made me realize that there are sometimes “out of the box” ways to achieve goals through communication. It allowed me to give a much more comprehensive description than with more closed questions.”^b (B2.ISTFr1)

“Being able to express ideas well and try to ask the important questions help to pass information better between people.”^b (C5.ISTMo3)

Managing/planning/organising

Comments here referred to the importance of structuring and selecting information and having a strategy or plan to transmit the message. Students referred to this often due to the time constraints. Examples in this theme were:

“Explaining things more succinctly and organised leads to better results” (C5.ISTMo1)

“Because it is through efficient (time) and effective (results) communication that you can communicate in the best way.”^b (A2.ISTTh1)

“Because not always we have all the information to give or the time to give it correctly and we must learn how to deal with that.” (A3.ISTMo2)

Understanding

In this theme, students mentioned that this activity helped them to think about the perspective of others, to recognise that not everybody has the same information, and how one should be concise and clear when communicating to create understanding in others, otherwise, communication is affected. Students showed these with the following examples:

“If the communication is not done properly, then the other's understanding will be affected.” (B1.ISTMo1)

“It was important to understand each phase of communication and remember that not all groups had access to all the information.”^b (C1.ISTTu1)

“It's very important to know to describe what we want to describe and to make sure the person on the receiving end is setting the idea we want to transmit.” (B2.ISTMo2)

Strengths and points for improvement

This theme referred to comments of students showing that this activity helped them to recognise where students failed during the activity and to reflect on their strengths and points for improvement. Examples of students’ responses were:

“It forces each group to think where we failed and to rethink.”^b (C1.ISTTu2)

“It helped me to understand that I'm really bad at writing down the important information.” (B4.ISTMo1)

“I already knew how to ask questions. In fact, I did it well.”^b (C4.ISTTh1)

Listening

Under this theme, students were specific about the importance of listening and paying attention to others. Students often mentioned that without this skill, one is not able to understand others nor effectively transmit the information received. That was why this theme often appeared with the theme conveying/explaining/answering. Examples of students were:

“It proved that listening to others is important in order to be able to later describe better.” (B6.ISTTh3)

“It's important to listen to other people and to ask specific questions in order to get a better understanding of a particular situation.” (A3.ISTMo3)

“It helps to realize the importance of paying attention and writing information in order to be able to convey it clearly.” (B5.ISTMo4)

Assumptions

Responses in this theme were specific to assumptions that people make and the importance of not making them by paying attention to what is transmitted and to ask questions when there is not enough information to make conclusions. Students showed these with the following responses:

“We can conclude that we often start from assumptions (like the colour of the elements of the image) instead of really thinking. We are pre-formatted to communicate taking this information into account.”^b (B2.UMinho)

“Pay attention to details and, mainly, do not assume information that was not said.”^b (C2.UMinho)

“It made me realize that when lacking information, ask questions first, rather than assuming my point of view is correct.” (C7.ISTTh3)

Taking notes

Under this theme, students mentioned how important taking notes is, along with listening, to be able to remember information and provide clear messages. Examples of how students showed these were:

“Helps to understand the importance of paying attention and writing information in order to be able to transmit it clearly.”^b (B5.ISTMo4)

“It is extremely important to listen carefully before writing any notes.”^b (B5.ISTMo4)

“It also showed that I need to write stuff down, memory is not enough.” (B6.ISTTh3)

Communication styles

Students indicated that this activity helped them to gain awareness that people are different as they see, think and perceive differently. This theme is often linked with themes like conveying/explaining/answering and understanding because students realised the need to put themselves in the other person’s shoes and adapt communication to them and to be clear and concise to convey information. Examples considered in this theme were:

“People interpret things the way they want to, not like its actually said.” (A6.TUDelft2019)

Because with this activity I realized that not everyone imagines the same things by just saying the word 'tree' for example. (A2.UMinho)

“Often people have to realize that we are all different, which will imply that we all have different perspectives and that if we don't know how to explain it properly there will be misinterpretations.”^b (A2.UMinho)

Observing

This theme was only mentioned once and the student stated they gained awareness of several ways of communicating through the activity and one of them was observing (“Because the dynamics showed the complexity of communication in several ways, from observing, speaking, paying attention, listening, knowing how to ask the right question for a less distorted conclusion as possible.”^b A6.UMinho).

C2) Experience: This category is characterised by the experiences students had during the communication activity.

Communication process

Responses under this theme were general comments in which students practised and improved their communication competencies and experienced the communication process existing in the engineering real world. Examples in this theme were:

“I believe it was a good exercise to develop other ways of communication.” (A4.UMinho)

“It's a good basic exercise to show how important small things can do with wrong information.” (B4.TUDelft2019)

“It was important to showcase communication in a similar way to what might happen in a more hierarchic, professional setting.” (A5.ISTMo3)

Team/group communication

Different from the previous theme, comments here were specific when students experienced teamwork within this activity and related to engineering work experiences. Examples in this theme were:

“As it evolved 3 teams and each one had their responsibilities, the only way to achieve the goals was to make sure everyone communicated well to one another.” (A3.ISTTu1)

“This is a brilliant example/exercise of what transmitting core information is, between working groups, which in engineering is essential.” (B1.ISTTu2)

“This activity showed us an example of what we are going to find after starting to work. Clients provide information on what they want to the project manager, which by their turn forward such information to the developers. It is not long for the three parties to explain/receive information.” (A7.ISTTh3)

Engagement/fun

Under this theme, comments were listed that showed that students enjoyed the activity. For example:

“I found the dynamics amazing!”^b (B3.UMinho)

“It's a really simple activity which can be understood from everybody, that in a short time evolves from the very beginning to the finished product.” (C4.UMinho)

“It was a fun experience.”^b (B6.ISTTu2)

Interaction between people

Only two students identified that this activity allowed them to work with people they did not know (“Good initiative to interact with strangers, one of the situations that do not occur often in our lives” A2.ISTTu2).

C3) No gain: This category is defined by themes in which students mentioned they did not gain any awareness, experience or anything else by taking part in the activity.

Have awareness already

Students in this theme stated they had awareness of the importance of communication already before the activity. However, most of them still mentioned that this activity improved their communication awareness or experience. Examples given by students were:

“I already knew how important communication is. This only makes it more clear.” (B7.ISTTu1)

“I already knew the importance but more at the theoretical level, here I was able to practice the consequences of good and bad communication.”^b (A2.ISTFr1)

“I knew it. It did prove it.” (A2.TUDelft2019)

Nothing new

Few students mentioned that they did not gain anything from this activity either because it was too short or too simple or because it was not useful for them. Examples of how students expressed this theme were:

“One hour isn't enough to learn anything new about a topic that important as communication.” (B7.ISTTu2)

“In the meantime, I don't really feel that I gained or lost anything from this activity.” (C2.ISTMo4)

“I don't really see the point of the exercise; it is too artificial to be useful, plus, regularly we don't have such time constraints when we're doing important communication activities.” (B7.ISTTu2)

Have experience already

Comments in this theme were similar from the theme Have awareness already but, in this case, students mentioned they already had experience with communication. Examples provided by students were:

“Before this activity, multiple life events had already shown me that communication is key, not just for work-related activities, but for general life purposes.” (B1.ISTMo4)

“I already knew how to ask questions.” (C4.ISTTh1)

Discussion

The main research question of this study was: “What are the characteristics of a game based-learning practice that stimulates engineering students to practise and reflect on their communication competencies?” Having used a mixed-method approach and the Chinese Whispers with a Twist as a case-study, it was found that the characteristics that make the activity effective are active participation, engagement, rules, reflection, risk taking and cooperation. This section will discuss the evidence provided by the data analyses for the identification of these characteristics.

As present in the literature,^28,29 games engage students because they are interesting and enjoyable. This activity is effective as a game-based learning activity because students actively participate and engage in the activity as they reported in the qualitative data of both open questions under themes “participation+” and “engagement/fun”. Through engagement and active participation students showed that they gained awareness and experienced communication competencies.

Rules are an element of games, as they are set by rules.²⁹ The rules in this activity were that students had limited time to describe the image, listen, ask and answer questions, and that they could not ask questions for clarification. Some students did not find the rules a constraint to their communication competencies as evidenced by the themes “planning/strategy+” and “managing+”. Because students had a strategy in place to observe the image, transmit information and ask questions, they were able to manage their time well during the activity. On the other hand, for some students, the rules imposed caused difficulties in communication. This was verified by the ineffective communication competencies perceived by students under themes “planning/strategy-” and “managing-”. Here, students had issues with planning and managing time during the activity and they perceived points for improvement including the organisation and time management of their speech, their listening skills and taking notes.

Another characteristic that makes this activity an effective game is reflection. This finding can be seen in two perspectives. First, it was found that students showed “effective and ineffective communication competencies” when they reflected on their communication competencies asked in the questionnaire. The quantitative part of the questionnaire showed that most students perceived they were effective communicators, however, many identified points for improvement. The qualitative part of the questionnaire corroborated these findings as students perceived that their communication competencies were both effective and ineffective during the activity.

It is evident from the questionnaire that students reflected on issues with details. In the quantitative part of the questionnaire, “pay attention to details” was the second most frequently indicated point for improvement by students. In the qualitative part, the theme “details” was frequently mentioned by students, either as effective but mostly as ineffective communication. Under this theme, students stated they did not pay attention to details, describe with detail, nor answer and ask in detail. This finding was also verified in the rubric as the category “details” obtained fewer scores compared to the other categories. Another point for improvement often indicated by students was “describing information”. This outcome was also found in the qualitative data in which students perceived they did not convey, explain and answer effectively. Other themes that emerged from the qualitative analysis, which were also present in the quantitative data were “listening”, “asking questions” and “taking notes”.

Evidence was found in the qualitative analysis of additional findings which was not present in the quantitative data. Students reflected on the barriers in communication, the need to be clear and concise when communicating with other people, and think about the others’ perspectives by putting themselves in the shoes of others to avoid misinterpretations when they communicate. These findings, under themes “Noise-” and “Barriers in communication”, showed that this active learning activity provided students with reflection on communication competencies such as adaptive communication style²² and pitching skills¹⁸ which were identified in previous studies as not yet mastered by engineering students.

The other perspective in which the occurrence of reflection was verified is in the feedback session at the end of the activity. This session was intended to link what happened in the game and the engineering real-world.²⁹ In these feedback sessions, students were able to draw a parallel between the activity and engineering work as present in the theme “team/group communication”. This is an interesting finding as this activity was designed to “plug-and-play” in a free context, although created to mimic an engineering environment of communication in teams. According to the contextual learning approach, activities should be contextualised to help students to connect academic content to the context of real-life.³⁹ In this activity, the connection between the activity and the real engineering environment was possible because the reflection was guided by lecturers to the context of each engineering field. The outcomes of the qualitative analyses can guide educators in this feedback session at the end of the activity, which was intentionally positioned to allow students to learn from their mistakes and reflect what they have done and learned.

As in previous game-based studies,²⁹ this activity gave students a training environment in which they could take risks and make mistakes without consequences. This is evident by the drawing scores measured with the rubric that showed that many students failed to communicate in this activity, as many groups did not obtain even half of the total number of points for their drawing outcome, and the qualitative analysis that demonstrated that students identified ineffective communication competencies, however, students still produced an outcome without consequences.

Finally, another characteristic of the activity that makes it effective is its cooperative nature. Students perceived that this cooperative activity allowed them to interact with other people, experience teamwork and become aware of team communication as evidenced in the second open question of the questionnaire under themes “Interaction between people”, experience and awareness: “Team/group communication”, respectively. Similar results were found in previous research where students’ interpersonal interaction increased within teams compared to individual work³¹ and communication and teamwork were developed in cooperative learning environments compared to lectures.³³

The main limitation of this study is that it is a case study and the results may be different for other cases.⁴⁰ However, this case can be seen as representative for a wider set of cases because it was applied and studied over five different engineering universities. Therefore, this is indicative that the results can be generalised to other engineering fields.

A further limitation is that the last question of the questionnaire (“Do you feel that this activity helped you to understand the importance of communication?”) may have biased the students to presume that communication is important. The authors recommend rephrasing this question in future uses of the questionnaire to a more neutral question, for instance: “what did you learn from this activity?”.

In future work, there is value in conducting interviews with past participants to investigate the benefits students perceived with the activity on the long term, i.e. whether students remember the activity and how it shaped their communication competencies in the period after they took part in the activity.

Conclusion

This study investigated the outcomes of the implementation of a communication activity called Chinese Whispers with a Twist as a case-study of a game-based learning activity over five engineering institutions. The activity was designed to practice and reflect on students’ communication competencies such as describing information in a short time, listening skills, and ask and respond to questions. This study provides support for the effectiveness of this activity as a game-based learning practice in engineering education because students actively participated and engaged in the activity, had to follow rules, reflected on their communication competencies and the existing communication barriers and styles, gained awareness and experienced communication in teams and cooperated with different people. The authors encourage educators to use activities, such as the one presented in this study, to put students in active learning activities where they work in teams and practise and reflect on their communication competencies.

This study also contributes to engineering education with the robust methodology used by the authors. They applied a mixed-method approach to analyse the learning outcomes. They also used an objective scoring rubric, using a ratio scale on which the elements displayed in the image can be quantified, to measure students’ communication performance. Students’ perceptions were analysed using both quantitative and qualitative approaches. The quantitative analysis allowed for defining a precondition that the activity is effective to practice and stimulate reflection on communication competencies. The qualitative analysis was used to complement and enrich the data of the quantitative approach and extensive and continuous iterations were applied by the authors during this qualitative phase to provide accuracy to the results. Also, the material necessary to replicate the research study and re-implement the activity in other universities was made freely available. Finally, a large sample size of students from different engineering universities was used. The authors wholeheartedly recommend researchers and educators to work in multidisciplinary teams and provide transparency during the research process as conducted in this study.

Footnotes

Author’s Note

Diana Mesquita is now affiliated to Centre for Human Development (CEDH), Faculty of Education and Psychology, Portuguese Catholic University, Porto, Portugal.

Acknowledgements

The authors would like to thank all students who participated in this study. Another big thank you goes to Prof. Marc de Vries of Delft University of Technology for his extremely valuable and constructive feedback on this paper. This work is part of the PREFER project ().

Declaration of conflicting interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

This work was partially supported by FCT–Fundação para a Ciência e Tecnologia within the R&D Units Project Scope UIDB/00319/2020 and by the Erasmus+ program of the European Union (grant agreement 575778-EPP-1-2016-1-BE-EPPKA2-KA).

ORCID iDs

Rui Lima

Gillian Saunders-Smits

Notes

References

Shuman

Besterfield‐Sacre

McGourty

The ABET “professional skills” – Can they be taught? Can they be assessed?

J Eng Educ 2005; 94: 41–55.

Cassidy

Developing employability skills: peer assessment in higher education. Educ Training 2006; 48: 508–517.

Mason

Williams

Cranmer

Employability skills initiatives in higher education: what effects do they have on graduate labour market outcomes?

Educ Econ 2009; 17: 1–30.

Care

Luo

Assessment of transversal competencies: policy and practice in the Asia-Pacific region. Paris: UNESCO, 2016.

Leandro Cruz

Saunders-Smits

Groen

Evaluation of competency methods in engineering education: a systematic review. Eur J Eng Educ 2020; 45: 729–757.

Passow

CH.

Which ABET competencies do engineering graduates find most important in their work?

J Eng Educ 2012; 101: 95–118.

Spinks

Silburn

Birchall

Educating engineers for the 21st century: the industry view. UK: Henley Management College, 2006.

Leandro

Saunders-Smits

GN.

Using an industry instrument to improve the transversal competency learning outcomes of engineering graduates. Eur J Eng Educ. DOI: 10.1080/03043797.2021.1909539

Beagon

Bowe

The academic perspective: a study of academic's perceptions of the importance of professional skills in engineering programmes in Ireland. In: 46th SEFI annual conference, Copenhagen, Denmark, 17–21 September 2018.

10.

Meier

Williams

Humphreys

MA.

Refocusing our efforts: assessing non‐technical competency gaps. J Eng Educ 2000; 89: 377–385.

11.

Nair

Patil

Mertova

Re-engineering graduate skills – a case study. Eur J Eng Educ 2009; 34: 131–139.

12.

Engineering Accreditation Commission. Criteria for accrediting engineering programs. Baltimore, MD: Accreditation Board for Engineering and Technology, 2000.

13.

ENAEE. EUR-ACE framework standards for the accreditation of engineering programmes. Brussels: European Network for Accreditation of Engineering Education, 2008.

14.

Donnell

Aller

Alley

, et al. Why industry says that engineering graduates have poor communication skills: what the literature says. In: 118th ASEE annual conference and exposition, Vancouver, Canada, 26–29 June 2011.

15.

Jennings

Ferguson

Focusing on communication skills in engineering education. Stud High Educ 1995; 20: 305–314.

16.

Berjano

Sales-Nebot

Lozano-Nieto

Improving professionalism in the engineering curriculum through a novel use of oral presentations. Eur J Eng Educ 2013; 38: 121–130.

17.

Drury

Langrish

O’Carroll

Online approach to teaching report writing in chemical engineering: implementation and evaluation. Int J Eng Educ 2006; 22: 858–867.

18.

Leandro Cruz

Saunders-Smits

Van den Bogaard

MED

, et al. Testing the validity and reliability of an instrument measuring engineering students perceptions of transversal competency levels. IEEE ToE 2020; 64: 180–186.

19.

Trevelyan

Mind the gaps: engineering education and practice. In: Proceedings of the 21st annual conference for the Australasian Association for Engineering Education, Sydney, Australia, 2010, pp. 383–390.

20.

Lievens

Are communication skills important for engineers? A relevant research question for engineering students and curriculum designers. In: 41th SEFI annual conference, Leuven, Belgium, 16–20 September 2013.

21.

Lappalainen

Communication as part of the engineering skills set. Eur J Eng Educ 2009; 34: 123–129.

22.

Moore

Morton

The myth of job readiness? Written communication, employability, and the ‘skills gap’ in higher education. Stud High Educ 2017; 42: 591–609.

23.

Wilkins

Bernstein

Bekki

JM.

Measuring communication skills: the STEM interpersonal communication skills assessment battery. J Eng Educ 2015; 104: 433–453.

24.

Bynum

Porter

Oxford dictionary of scientific quotations. Oxford: Oxford University Press, 2005.

25.

The Quotation Page. Confucius, www.quotationspage.com/search.php?homesearch=I+hear+and+I+forget&startsearch=Search (assessed 16 October 2020).

26.

Bonwell

Eison

JA.

Active learning: creating excitement in the classroom. ASHE-ERIC Higher Education Reports, ERIC, 1991.

27.

Kolb

DA.

Experiential learning: experience as the source of learning and development. Englewood Cliffs: Prentice-Hall, 1984.

28.

Bodnar

Anastasio

Enszer

, et al. Engineers at play: games as teaching tools for undergraduate engineering students. J Eng Educ 2016; 105: 147–200.

29.

Garris

Ahlers

Driskell

JE.

Games, motivation, and learning: a research and practice model. Simul Gaming 2002; 33: 441–467.

30.

Bodnar

Clark

RM.

Can game-based learning enhance engineering communication skills?

IEEE Trans Profess Commun 2017; 60: 24–41.

31.

Johnson

DW.

Active learning: cooperation in the college classroom. 2nd ed. Edina, MN: Interaction Book Co., 1998.

32.

Prince

Does active learning work? A review of the research. J Eng Educ 2004; 93: 223–231.

33.

Johnson

Smith

KA.

Cooperative learning returns to college what evidence is there that it works?

Change: The Magazine of Higher Learning 1998; 30: 26–35.

34.

Terenzini

Cabrera

Colbeck

, et al. Collaborative learning vs. lecture/discussion: students' reported learning gains. J Eng Educ 2001; 90: 123–130.

35.

Oxford Dictionary. Oxford University Press, www.oxfordlearnersdictionaries.com/definition/english/chinese-whispers?q=chinese+whispers (assessed 16 October 2020).

36.

Stevens

Levi

AJ.

Introduction to rubrics: an assessment tool to save grading time, convey effective feedback, and promote student learning. Sterling, Virginia: Stylus Publishing, LLC, 2005.

37.

Thomas

DR.

A general inductive approach for analyzing qualitative evaluation data. Am J Eval 2006; 27: 237–246.

38.

Fleiss

JL.

Measuring nominal scale agreement among many raters. Psychol Bull 1971; 76: 378–382.

39.

Johnson

EB.

Contextual teaching and learning: what it is and why it's here to stay. USA: Corwin, 2002.

40.

Cohen

Manion

Morrison

Research methods in education. London: Routledge, 2013.