Using Scenario-Based Assessment in the Development of Students’ Digital Communication Skills and Professional Competence

Introduction

The COVID-19 crisis in March 2020 caused a major worldwide disruption in education, as teaching was abruptly moved online for many teachers and students with little or no advanced planning. We believe the transferable skills that students can develop as a result of engaging in scenario-based assessment (SBA) are of relevance for business communication practitioners around the world, as digital communication endures, most especially as many of us are involved with students who may have limited workplace experience. We also believe that SBA can be a useful tool in harnessing the positive effects of using generative (AI) tools, as well as mitigating against its inappropriate use, as SBA can be designed to maximize the integrity of an assessment. Our hope is that our account of SBA will provide practitioners with a motivating and effective way of helping learners to develop the skills they will need to be effective communicators in a postpandemic professional world.

In February 2020, both higher education and the business world abruptly went online as a result of the COVID-19 lockdown. Spoken and written forms of digital communication proliferated as a result and continue to be the norm, both in business and academia. In addition, the explosion of generative AI tools, most famously ChatGPT at the end of 2022, means that we now have an increasingly complex set of opportunities and threats that can either help or hinder the development and assessment of our students’ communication skills. Three years on from the beginning of the COVID crisis, we consequently find ourselves in a situation where the skills our students need to excel in the classroom are the same as those that they will need to communicate effectively in business, while at the same time, we also need to find ways of adapting our assessment methods to harness the positive aspects of the generative AI tools that our students will undoubtedly need to use in their professional lives. These parallel developments provide us with an unprecedented opportunity to refocus our teaching in business communication both in identifying the battery of communication skills they will need for the workplace, and the the most effective ways in which to assess them.

In this discussion, we will argue that the use of SBA can provide students who have little or no work experience, with an effective means of developing the communication skills they will eventually need for the workplace, alongside relevant aspects of disciplinary knowledge that may help them to understand what it means to be a member of the business profession. This is because SBA engages students in real-life performance tasks in a situated context or scenario (Deane & Sparks, 2019). For business communication training in particular, this is useful, because an SBA can be constructed in such a way as to replicate an authentic, real-world task, similar to the type of tasks that students are likely to perform in the workplace (Davidson, 2023; Nickerson & Davidson, 2023). For example, this could involve students writing a text message, participating in a meeting, developing a business plan, writing a press release, or working with a team to write a proposal, with all of the activities typically being framed within a larger overarching project.

In addition, SBA is a type of learning-oriented assessment that can have a significant positive washback effect on learners, that is, the effect that the assessment has on teaching and learning, and it therefore has the potential to facilitate learning for students (Davidson & Coombe, 2022b). Moreover, as we will discuss below, with the incorporation of certain tasks, SBA can also mitigate against academic misconduct, which has been identified as a growing concern, most especially, in online and writing assessment (Davidson & Buchanan, 2023; Davidson & Coombe, 2022a), while at the same time, SBA can provide learners with an opportunity to learn how to use generative AI appropriately and to develop both their AI literacy and their writing skills in preparation for entering the workplace (P. Cardon et al., 2023a; P. W. Cardon et al., 2023b). For instance, a scenario-based group project within an SBA that involves constant interaction among the members of a group, alongside multiple drafts and feedback from the teacher, is less likely to have been written by an AI generative tool. Moreover, an easily accessible generative AI tool such as ChatGPT can help learners to complete certain tasks that can help them to develop their professional competence, such as accessing model texts for the genres they will need to use in their professional life, for example, an audit report, as well as helping them to understand what it means to be a part of a specific business profession, for example, the primary roles and responsibilities of an auditor. Finally, as an SBA is likely to be made up of a number of tasks that are thematically linked, this is particularly useful for aspiring business people, as it provides them with an opportunity to experience the intertextuality and interdiscursivity that is a common feature of communication in business contexts in creating links between different genres (Albers et al., 2020; Nickerson, 2018; Warren, 2016).

In the sections that follow, we will first give a brief overview of SBA and show how this lends itself to the assessment of digital communication skills in particular. We will look at scenario-based approaches that have been used in the business communication literature and we will also show how SBA can be used to incorporate the use of generative AI tools in a productive way into student assessment. We will consider several aspects of professional competence that have been identified as crucial for learners as they start their business careers, and then flourish within them, and we will show how SBA can be used in various ways to help them to understand more about the different skill sets that underlie that competence, as well as to develop them. We will illustrate our arguments with three examples of thematically linked SBAs, drawn from our own experience as business communication practitioners working with undergraduate learners with little or no working experience in the multicultural, multilingual, context of the United Arab Emirates.

Digital Communication, Professional Competence, and Scenario-Based Assessment

The origins of SBA suggest that it could be a useful tool in teaching business communication in classes where there is a focus on the development of digital communication skills, alongside other aspects of professional competence, and in particular, in situations where those classes are being given online. Din and Jabeen (2014), for instance, provide a useful overview, which frames the development of SBA as an offshoot of the increasing influence of information communication technology (ICT) on education from the beginning of this century. SBA is an extension of scenario-based learning, in which learners are assessed on their completion of a set of activities that replicate a real-life situation, such as solving a problem typical of the profession that they are aspiring to join. Scenario-based learning and SBA emerged as a result of the burgeoning use of e-learning and the need to ensure that learners engaged in online learning would also be given an opportunity to apply the concepts they were being presented within a real-time situation (Din & Jabeen, 2014, p. 93). They note that, among other things, using a scenario-based approach (a) allows learners engaged in online education to learn more about the practical implications of a subject (e.g., Tanali, 2012), (b) enhances their learning (e.g., Battersby, 2008; Oliver & Carr, 2009), and (c) motivates students engaged in the learning activity (e.g., Limson et al., 2007) . In addition, Din and Jabeen’s (2014) own empirical study of the effects of using SBA in an online Mass Communication course indicates “that scenario-based exercises enhance the students’ subject-related knowledge as well as helping them to sharpen their analytical skills” (p. 100). Finally, Deane and Sparks (2019) underline the usefulness of using SBA as a formative form of assessment, meaning that it encourages learners to reflect on their performance and to identify areas that need improvement, while at the same time it provides instructors with an opportunity to give feedback and to work with their students over a period of time to develop their skills. In summary, SBA would seem to provide students with an opportunity to deepen their disciplinary knowledge, to apply that knowledge in an authentic context, to reflect on the skills that they need in order to do this effectively, and to be engaged in a motivated way.

A search of the Business and Professional Communication Quarterly issues from the past decade reveals that scenario-based approaches have been used previously in various ways in business communication assessment as a way to help students practice a specific set of skills and/or to develop a specific aspect of disciplinary knowledge. For example, Sharma (2021) shows how he developed an assessment tool based on Service-Learning, which allowed students to practice their organizational, leadership, and interpersonal communication skills. Cyphert et al. (2016) discuss how they adapted their assessment methods to encourage their students to reflect on their own experience as a way of developing their managerial perspective on team communication. And Williams et al. (2022) describe a role-playing exercise in which the students refer to a large set of data to comply with a request, designed to simulate similar requests they will encounter in the workplace. In turn, several studies have focused on the efficacy of online communication courses, including McCool (2023), who looks at social presence, team cohesion, and collaborative writing in online teams, and Swartz et al. (2020), who note an increase in intercultural competence as the result of a virtual team collaboration, across three different countries; other studies explore the effects of using a flipped classroom approach, incorporating various forms of online interaction, for example, Sherrow et al. (2016), Balzotti and McCool (2016), and Cummings (2016). Studies such as these point to the potential usefulness of SBA for business communication teaching, not only because of the ease with which it can be used in e-learning contexts incorporating digital forms of communication but also because of its adaptability in assessing a wide range of knowledge and skills.

Studies on business discourse more broadly, and business English in particular, provide us with an indication as to the disciplinary knowledge and accompanying skills that our students will need on entering the business professions. This information on knowledge and skills, in turn, provides us with a frame for creating appropriate SBAs. For example, a number of studies have identified the underlying skills that business people need in order to communicate effectively in digital modes of communication in the workplace. Darics and Gatti (2019) show, for instance, that effective digital business transactions are characterized by collaboration and collegiality. Albers et al. (2020) point to the importance of intertextuality in authentic business emails, but also their informality. And Gimenez (2014) emphasizes the need for multitasking skills and an ability to combine different (digital media) with different audiences in order to communicate successfully using computer-mediated communication. Findings like these provide an indication as to where the focus should be in the topics discussed and the materials included in business communication courses with a focus on digital skills, in order to provide students with an opportunity to develop the skills they will need on entering the workforce. We believe that these topics and materials would work well within an SBA framework that would require students to use their online communication skills, while at the same time, also requiring them to reflect on the different characteristics of effective digital communication identified in the literature, and then assessing them on their proficiency in demonstrating these skills.

Alongside the obvious need for effective digital communication skills within contemporary workplaces, other aspects of professional competence have also been identified in previous studies that could be built into an SBA. This could be especially useful in helping students acquire specialized knowledge and skills, as “there is often a disconnect between the content in commercially published business English materials and textbooks, and what the research tells us actually happens in the business workplace” (Davidson, 2023, p. 198). For example, students may need to understand which types of communicative genres are typical of a particular business profession, together with learning about the processes surrounding their construction (e.g., Flowerdew & Wan, 2010, for the audit profession; Cheng & Mok, 2008, for land surveyors), or they may need to develop expertise in very specific business genres, such as an investment presentation, which are not adequately covered in commercial textbooks (e.g., Evans, 2013; Crawford Camiciottoli, 2020). In addition, other aspects of understanding what it means to be a business professional may be incorporated into an SBA, such as becoming a member of the specific community of practice associated with a business profession (e.g., Zhang, 2017), understanding how to manage intercultural communication in international meetings (e.g., Rogerson-Revell, 2008; Spencer-Oatey, 2000), or looking ahead to the additional skills that a business professional may need as their career progresses (e.g., Chan, 2019; Lichterfeld, 2023). As Davidson (2023) notes, SBA is a “possible way to reduce this gap between what happens in the business English classroom and what happens in the business workplace” (p. 198).

Finally, in a recent discussion on using generative Artificial Intelligence (AI) to assist in teaching and assessing student writing, Davidson and Buchanan (2023) note a number of different ways in which instructors can attempt to mitigate against the misuse of AI by learners in their writing assignments. These include using the type of scenarios and case studies that are already characteristic of using an SBA approach, and they also suggest using personalized or localized assignments, reflective assignments, tasks that require multiple drafts with constant monitoring and feedback from the instructor, making students use GoogleDocs to ascertain what students wrote and when (which can then be checked using Draftback), and the implementation of pair and group assignments. Several of these suggestions can be linked directly with what recent business communication research has told us are the hallmarks of how people communicate within the workplace and construct their professional identities over the course of their careers, such as working effectively in intercultural teams, understanding how to select different media and being reflective as a way of developing coaching and leadership skills (see Nickerson & Chan, 2024, for further discussion on these points).

Our suggestion here is that instructors should construct SBAs that also incorporate one or more of these additional characteristics, as this will create a powerful approach that combines the maximization of integrity in the assessment, while at the same time building in a set of AI-enabled tasks that recent research has identified as characteristics of the workplace, that is, researching a topic or generating ideas for work, drafting messages, drafting reports, and editing text (P. W. Cardon et al., 2023b) and therefore allowing students to develop the battery of skills that lead to professional competence. In order to do this, the instructor must first identify the knowledge and skills that the assessment should focus on, they should create a scenario specifically designed to practice and assess that knowledge and set of skills, and they should build in a set of tasks to ensure the integrity of the assessment, together with a set of appropriate AI-enabled tasks. We believe that this process will go some way toward answering Getchell et al.’s (2022) recent call that the business communication community should “continuously consider how digital transformation can change the nature of communication and collaboration in the workplace” (p. 8). In the next section, we will provide three examples of SBAs targeted at three different sets of knowledge and skills to illustrate this process.

Some Examples of SBAs

Future directions

A useful development in the future could be to set up an online assessment portal, for example through the Teaching and Learning Resources sections of Association for Business Communication website, in order to provide examples of different types of SBAs alongside relevant research studies, with suggestions for mitigating against the inappropriate use of generative AI tools in each case, and recommendations on how generative AI could be leveraged to help students learn and improve their business writing skills.