Abstract
Background. Because
Aim. This correlational study aimed to examine the effects of
Method. The setting for this study was an undergraduate logistics and supply chain class. The class uses a serious simulation game which is designed to realistically mimic the business transactions within an enterprise resource planning system (ERP). Cross-sectional surveys captured individual
Results. Two sets of analyses were conducted using a sample of 100 undergraduates performing in 42 teams. At the group-level, OLS regression results suggest that, while there was no effect on objective
Conclusion. Our findings suggest that
Keywords
Computer simulation games have long been a staple in high-stakes training environments, such as military teams, surgical units, and emergency first responders (Hays, 2005). These sophisticated simulation tools provide a realistic, yet safe, learning environment; one in which mistakes can be made without incurring expensive equipment loss or endangering human life (Bell et al., 2008). As the popularity of video gaming has exploded, business educators have also seen the merit of incorporating instructional content into an interactive simulation framework, thereby capitalizing on the entertainment value inherent in games. Indeed, as technology has evolved and cost barriers have lowered, simulation-based training (SBT) tools have become ubiquitous in workplace training and university settings (Bell & Kozlowski, 2008; Faria et al., 2009; Faria & Wellington, 2004). Often delivered in the form of computer-based simulation games, the power of SBT lies in its ability to mimic reality. In the field of workforce training, this aspect of SBT is invaluable, as it enables individuals to become proficient at their job without risk to themselves or others. Consequently, one particularly promising approach to SBT training has been to view the individual as an active participant in the learning process. In this context, the focus is on achievement motivation and self-regulatory learning behaviors as people explore and experiment with the simulation game, learning through trial and error (Bell & Kozlowski, 2009).
In addition to individual learning, computer-based simulations are also conducive to developing the cognitive structures and relational interfaces necessary for effective team functioning (Kozlowski & DeShon, 2004). We live in an era of technology, and most jobs are positioned within the realm of a knowledge economy. As such, work has become increasingly complex. This complexity requires people to work together, interact with each other, share information, and commit their combined talents and energies to the accomplishment of a single goal (Kozlowski & Bell, 2013). Learning resides within an individual. However, people who work closely and intensively with each other will often develop special group-level cognitions whereby they share understandings and mental representations of the team’s task environment (Klimoski & Mohammed, 1994; Lewis et al., 2005). Group level cognitions reduce the mental load on any one member of the team. This is particularly beneficial in situations with intense informational demands. Consequently, when team coordination is highly developed, team cognitions can ultimately improve team performance (DeChurch & Mesmer-Magnus, 2010). In terms of development, these team-level cognitive properties “emerge” as individuals within the group interact with each other over time. This is a complex evolution that requires multilevel theoretical conceptualizations (Kozlowski & Bell, 2008; Kozlowski & Klein, 2000). As a result of this complexity, the way in which team interactions develop and unfold is still not well understood (Salas et al., 2015).
Moreover, while a large body of research has been devoted to understanding the role of computer-based simulation games in the training of dynamic decision-making teams (Kozlowski & DeShon, 2004; Salas et al., 2009), less emphasis has been paid on effective ways to incorporate these learning tools into the university setting (Salas et al., 2009). The university classroom is unique in that students must complete a core curriculum, (e.g., management, accounting, finance, and logistics). While this battery of coursework is beneficial for an individual’s overall understanding of the business environment, it may contain certain classes that can be challenging or even intimidating for some students. In these situations, simulation games, with their ability to provide a realistic, yet harmless, learning environment, can be a powerful learning aide to help students master difficult material and develop critical thinking skills (Lovelace et al., 2016; Salas et al., 2009). However, in addition to delivering task-relevant material, the simulation gaming environment can also be leveraged to develop some of the softer skills needed for teamwork and collaborative problem-solving (Marlow et al., 2016). An educational tool that could help simultaneously develop task and team competencies within the undergraduate curriculum would fulfill a pressing need. To illustrate, a recent report from the National Association of Colleges and Employers (NACE, 2016) indicated that, while employers consider critical thinking, problem-solving, teamwork, and collaboration to be essential, there continues to be a significant deficit in recent new hire proficiencies and readiness. Indeed, this has been a consistent and troubling trend over the last few years as educators, employers, and researchers have noted the need to incorporate interpersonal, collaborative, and team-based skills into the business curriculum (Bedwell et al., 2014; Chen et al., 2004; Ritter et al., 2018). As a teaching and learning tool, simulation games can provide the relevant instructional content which promotes learning while also fostering collaborative, team-based behaviors. However, while simulation games hold much promise, there remains a lack of research on the specific motivational mechanisms, group interactions, and causal pathways through which these dual outcomes can be fostered (Marlow et al., 2016).
In this study, we examine the impact of individual achievement motivation and team knowledge sharing behaviors on satisfaction variables and team performance outcomes in the context of a complex and serious simulation game. Our goal is to advance an understanding of determinants of student success in computer-based simulation games, at the individual and team levels. We recognize that the repetitive rounds of play inherent in computer-based simulation games allow for team formation and team processing. This corresponds to the general framework of the IMOI (input-mediator-output-input) model of teamwork (Ilgen et al., 2005). Within the IMOI frame, we look at individual learning goal orientation and group-level knowledge sharing norms as our input variables. To understand our group-level, mediating variable, we base our thinking in the foundations of social exchange theory (SET) and the premise of reciprocity (Cropanzano & Mitchell, 2005). SET predicts that high-quality knowledge exchange and interpersonal communications will result in the formation of team cognitive structures such as team mental models or transactive memory systems and that these team-level cognitions result in positive team and individual outcomes (Bachrach et al., 2019; Fiore et al., 2001). Therefore, congruent with the concepts of social exchange, we expect that reciprocal knowledge sharing interactions will result in the formation of transactive memory systems. Furthermore, we expect that these transactive memory systems will enhance team performance and positively influence individual reactions to the game.
Theoretical Background
As technology has become more advanced, business simulation games have emerged as a popular learning tool. A survey from the late 1990s found that over 97% of business schools used simulation games (Faria, 1998), while a later survey found that a substantial number of faculty in AACSB institutions had used a business simulation game in the classroom at least once (Faria & Wellington, 2004). The current generation of business simulation games provides an interactive and experiential learning environment where students and trainees can be immersed in a realistic situation and learn from the consequences of their decisions. Cognitive structure refers to memory and knowledge bases, while affective structures involve motivations and attitudes. Simulation games can be effective training tools because, through an interaction of external and internal mechanisms, they target both cognitive and affective structures (Sitzmann, 2011; Tennyson & Jorczak, 2008). Although research has typically focused on individual learner outcomes, an understudied aspect of simulation games is their potential to encourage teamwork and cooperative behaviors such as knowledge sharing (Marlow et al., 2016).
Knowledge Sharing in Teams
Work has become complicated and dependent on technology, thereby making it difficult for an individual to function alone. As such, companies depend on teams to solve problems and deal with sudden and unexpected contingencies and events (Kozlowski & Bell, 2013). Because of this increasing complexity, teams are being thought of, not merely as vehicles to perform tasks, but as information processing units (Hinsz et al., 1997). Effective teams develop through the emergence and coalescence of individual knowledge, goals, efficacy, and skill (DeShon et al., 2004; Kozlowski & Bell, 2013). When teams effectively share and combine information and knowledge, they are able to achieve superior outcomes (Mesmer-Magnus & DeChurch, 2009).
Outcomes of team knowledge sharing behaviors
Knowledge and time are valuable commodities, and, unless there is a compelling reason, people are often reluctant to take the time and make an effort to share what they know. Yet we know that, over time, people who work together in teams often develop highly cohesive bonds, and that these relationships, which are built on trust and mutual liking, can result in team synergies which are beneficial to the organization (Mathieu et al., 2017). One explanation for the development of team cohesion lies in the reciprocity rules that are at the heart of positive social exchange (Cropanzano & Mitchell, 2005). Reciprocity is conceptualized as a series of interdependent exchanges whereby an action on the part of one person leads to a response by the other. If these exchanges are positive in nature, quality relationships develop over time, which can facilitate knowledge sharing and ultimately enhance performance (Bartol et al., 2009; Quigley et al., 2007). The nature of reciprocal response can be thought of as both behavioral and relational (Cropanzano et al., 2017). Positive individual outcomes such as organizational citizenship behaviors and trust have been examined in conjunction with relational, reciprocal exchange (Dirks & Ferrin, 2001; Eisenberger et al., 2001; Molm et al., 2000). Since knowledge sharing is an activity based on reciprocity and an integral part of team functioning, we would expect knowledge sharing norms and reciprocal exchanges to encourage an overall positive climate and an environment for cooperation. As a result, team members should have a better social experience, and this enhanced social experience should make members happier and more satisfied with their respective teammates.
However, the establishment of knowledge sharing norms should have positive effects beyond affective reactions to others on the team. Knowledge is often shared in the form of information, and there is a rich body of research that looks at the patterns, effects, and outcomes of information sharing in groups (De Dreu et al., 2008; Stasser & Titus, 1985, 1987). The original assumption was that the quality of decision-making was a mathematical function of the way in which information was distributed amongst group members (Stasser & Titus, 1985, 1987; Wittenbaum et al., 2004). Later, researchers began to acknowledge that the quality of information exchange was also a function of member motivations: in particular, epistemic and social motivations (Nijstad & De Dreu, 2012). In these models of group information sharing, the group members’ motivation to learn and acquire a deep understanding of the task at hand, along with their willingness to cooperate with others, has a significant impact on the quality of information exchange and group decision-making (De Dreu et al., 2008). So, in team settings, research findings suggest that such factors as the prosocial proclivities, perceived expertise, and social status of group members; the quality of leadership exchanges; and the type of team communications have a significant effect on performance outcomes (De Dreu et al., 2008; Marlow et al., 2018; Mohammed & Dumville, 2001). In terms of performance outcomes, the effects of information and knowledge sharing have been shown to affect both the task and socio-emotional functioning of the team (Mesmer-Magnus & DeChurch, 2009; Wittenbaum et al., 2004). In other words, members who share knowledge are not only happier with each other; the information exchange also boosts task satisfaction and actual task performance. Therefore, in this study, we would expect knowledge sharing norms to affect team and task satisfaction, as well as the overall performance outcomes of the team.
Hypothesis 1a: Knowledge sharing norms will be positively related to a) satisfaction with the team and b) satisfaction with the task.
Hypothesis 1b: Knowledge sharing norms will be positively related to a) perceived team performance and b) actual team performance.
Knowledge sharing norms and the development of transactive memory systems
When people are working closely together on a task, they will often develop special cognitive structures to achieve their common goals. In general, these cognitive structures are labeled team cognitions, and they are usually conceptualized in two different ways: team mental models and transactive memory systems (Fiore et al., 2001; Ilgen et al., 2005). Team mental models (TMM) refer to shared cognitions, where knowledge is communal and redundant. TMM is particularly useful in dynamic situations where coordination and backup behaviors are essential (e.g., emergency response teams and military units) (Cannon-Bowers et al., 1993; Klimoski & Mohammed, 1994). The second type of team cognition, transactive memory systems (TMS), describes the development and utilization of individual team member expertise. Initially conceived as a type of specialized team cognitive strategy, TMS explains how group members can, together, achieve a complex task that would be difficult, if not impossible, for one person, working alone (Ren & Argote, 2011).
In essence, TMS involves two components: a group level memory structure, (who knows what) and transactive processes to utilize that structure (Ren & Argote, 2011). Since the concept was first proposed, TMSs have been observed and studied in a wide variety of laboratory and field settings (Hollingshead, 1998; Lewis, 2004). In terms of antecedents, studies have examined the attributes of team members and have found such personal characteristics as critical team member assertiveness to be instrumental in the formation of TMS (Pearsall & Ellis, 2006). Through laboratory studies, we know that TMSs will naturally occur when people are trained together on a specific task (Lewis et al., 2005; Liang et al., 1995). Cooperative group behaviors, such as communication have also been connected to the development of transactive memory systems. For example, Kanawattanachai and Yoo (2007) found that task-oriented communication led, not only to expertise location, but also to cognition-based trust in virtual teams. He et al. (2007) found that communication in the form of calls or face-to-face meetings led to the formation of specialized team cognitions, but email exchanges did not.
By definition, a team that has established a group norm for knowledge sharing is engaging in cooperative behaviors. These positive and reciprocal behaviors should result in high-quality relationships, marked by a sense of mutual respect and liking (Blau, 1964; Molm et al., 2000). Moreover, in training situations, when everyone is a novice and in the initial stages of learning, where roles are differentiated, and informational requirements are complex, the establishment of knowledge sharing norms should result in two outcomes: first, individual team members will volunteer to acquire specialized knowledge, and, second, team members trust each other enough to allow that specialization to occur (Marlow et al., 2016). In other words, the team should develop a transactive memory system.
Outcomes of transactive memory systems
Transactive memory systems represent a division of cognitive labor, thereby reducing the mental load on individual team members (Lewis & Herndon, 2011). Teams with a well-developed TMS are able to locate and take advantage of individual talent and expertise. Hence, teams who develop these specialized structures generally perform better on complex, interdependent tasks (Bachrach et al., 2019). In laboratory studies, teams who take advantage of group member expertise perform better on experimental tasks (He et al., 2007; Lewis et al., 2005; Pearsall & Ellis, 2006). Generalizing to a broader base, studies looking at the effects of TMS have been conducted in a variety of settings from knowledge workers in consulting and product development teams (Akgün et al., 2005; Lewis, 2004) to national security teams and EMTs (Jarvenpaa & Majchrzak, 2008), with an overall consensus that groups who can develop and maintain effective TMSs achieve superior performance outcomes (DeChurch & Mesmer-Magnus, 2010; Lewis et al., 2005). This positive effect extends to a variety of performance outcomes, including perceived team performance (Bachrach et al., 2019; Zhang et al., 2007). Therefore, since we expect the high-quality relationships that develop from knowledge sharing to result in the formation of transactive memory systems, it would follow that these TMSs would result in improved team performance outcomes. Consequently, we expect transactive memory systems will mediate the relationship between knowledge sharing norms and team performance outcomes.
Hypothesis 2: Transactive memory systems will mediate the relationship between (a) knowledge sharing norms and actual team performance and between (b) knowledge sharing norms and perceived team performance.
Moreover, being in an environment where information is efficiently and effectively flowing between team members should have benefits beyond those of team performance. People who are part of such a reciprocal exchange should feel a sense of inclusion; they should derive a certain sense of self-worth as they find themselves actively contributing to the success of their team. Indeed, this is consistent with prior research findings, where we see that informational diversity is positively related to satisfaction levels when group differences and conflicts are minimized (Jehn et al., 1999), when information flows are efficient (Janz et al., 1997), and when goals are reached as complex, interdependent tasks are competently planned and executed (Saavedra et al., 1993).
Research findings continue to support the notion that participating in a specialized knowledge network lends itself to a certain sense of satisfaction. For example, in a recent metanalysis, Bachrach and colleagues (2019) found that well-developed transactive memory systems have beneficial effects on the affective aspects of team performance. In particular, teams with a developed TMS tend to have a more positive assessment of the team’s future viability (Lewis, 2004). Transactive memory systems have been shown to bolster the effects of positive intangible team factors. For example, when engaged in knowledge-intensive tasks, trust is positively related to team member satisfaction; this relationship flows through the development of a transactive memory system (Gockel et al., 2013). Other studies have shown transactive memory systems to be a mediator between leadership behaviors and team satisfaction, as transactive memory systems mediate the relationship between critical team member characteristics and satisfaction levels (Pearsall & Ellis, 2006). In a field study looking at nurse and physician anesthetists who were working under pressure and time constraints, transactive memory had a positive effect on work attitudes, such as job satisfaction and team identification (Michinov et al., 2008).
These findings make intuitive sense. Since transactive memory systems bolster goal attainment, we would expect team members to experience more satisfaction when they are part of a specialized team and reap the benefits of that specialization. Consequently, because team cognitions originate from reciprocal knowledge exchange, we would expect transactive memory systems to mediate the relationship between knowledge sharing norms and affective team outcomes.
Hypothesis 3: Transactive memory systems will mediate the relationship between (a) knowledge sharing norms and satisfaction with the team and between (b) knowledge sharing norms and satisfaction with the task.
Achievement Motivation and Learning Goal Orientations
One of the most useful concepts in understanding how people perform during the learning process is that of goal orientation. Goal orientation refers to the way in which people view and approach learning and performance in achievement situations. Its earliest conception came from educational psychology, where researchers noted differences in the way children would approach educational achievements (Dweck, 1986). In certain situations, some children would take a deep approach to learning, wanting to internalize and gain personal mastery over the material; whereas, other children were more interested in obtaining the external approval of others, (e.g., the teacher or parent). Educational psychologists began to refer to these differences as goal orientations. The people who viewed learning as a way to acquire or increase personal competence were considered to have learning goal orientations; whereas, the people who were more concerned with demonstrating competence and meeting performance expectations were considered to have performance goal orientations (Dweck, 1986).
Goal orientation, although malleable, can be viewed as a stable, individual difference, much like personality (Dierdorff et al., 2020; Porter, 2012; Steele-Johnson et al., 2000). Since the concept of goal orientation was introduced into the organizational literature in the 1990s, researchers have examined its effect in multiple areas such as job performance, evaluation, and feedback seeking (VandeWalle, 1997); training effectiveness and trainee reactions (Bell & Kozlowski, 2002); team cohesiveness and cooperative behaviors (Dierdorff & Ellington, 2012); and leadership effectiveness (Dragoni, 2005; Porter et al., 2016). The overwhelming consensus from all this research suggests that learning goals are correlated with positive adaptive behaviors, such as goal establishment, self-monitoring, and persistence in the face of failure (Payne et al., 2007).
Overall, research findings suggest that higher levels of learning goal orientations are linked to a motivation to absorb instructional material and gain proficiencies at a given task. For example, people with higher levels of learning goal orientation are more likely to stay focused on the task at hand and persevere when encountering difficulties (Brown, 2001; Fisher & Ford, 1998). Higher levels of learning goal orientations are linked to an increased use of learning strategies and increased use of self-regulatory mechanisms, such as metacognition (Dierdorff & Ellington, 2012; Payne et al., 2007). In a classroom setting, learning goal orientations were linked to higher motivations to learn, which, in turn, were linked to more positive course outcomes, such as grades and satisfaction (Klein et al., 2006).
Because people with learning or mastery goal orientations are intent on improvement, they are likely to invest in resources that will optimize their outcomes. In today’s environment, where work is complex and interdependent, it is reasonable to think that people with mastery goal orientations would see their peers as a valuable source of assistance. In line with the rationale that people view coworkers, colleagues, and peers as assets, scholars have surmised that people with mastery goals are likely to invest in exchange relationships, see the value of reciprocal norms, and seek out avenues to gain and integrate new sources of information (Poortvliet & Darnon, 2010). Because people with higher learning goal orientations are intently focused on learning the material and gaining mastery of the task, learning goal orientations should be postively related to task satisfaction. Moreover, when working on a complex, interdependent task, people with mastery orientations are likely to recognize the difficulties of operating alone. As a result, they are likely to seek out and form positive alliances with their teammates (Poortvliet & Darnon, 2010). Therefore, we would expect levels of trait learning goal orientation to have an effect on their reactions to an instructional tool, as well as their perceptions of the people assigned to work with them (see Figure 1).

Hypothetical model.
Hypothesis 4: Individual learning goal orientations will be positively related to a) satisfaction with the task and b) satisfaction with the team.
Method
Participants and Procedure
Approval for this correlational study to examine student reactions to computer-based simulation games was obtained from our university’s Institutional Review Board. Participants were students enrolled in an undergraduate logistics and supply chain class in a medium-sized public university in the southeastern part of the United States, and data were collected in a total of 5 classrooms, across two semesters. The team-based simulation game for this study mimics the functionality of operations management software, (e.g., enterprise resource planning, or ERP, systems). Large companies typically rely on these sophisticated and expensive computer systems to manage their business. Available from ERPsimLab HEC Montreal (Léger et al., 2007), the game was developed under the auspices of SAP. SAP provides software to 80% of the Fortune 500 companies. This particular simulation game is utilized both in university classrooms, and also in businesses, to teach the conceptual foundations of the ERP and to provide specific training on how to use the system. The game is designed to be played by teams of two to four participants. Because the system is complex and requirements differentiated, if team members specialize in certain areas, team performance should improve (Léger et al., 2010).
Standard instructions were generated and utilized when introducing the simulation to the students. In the introduction and just before each round of play, students were reminded that it was okay to make errors during the learning process. Consequently, there were no grades linked to actual performance in the simulation. Instead, students received an individual participation grade for being present the day of the simulation play. Moreover, team members did not evaluate each other in any way. During the simulation, students were placed into teams and introduced to the enterprise resource planning software in a set of rounds. All three rounds of the simulation utilize the same market environment. Each round of play became increasingly more complex. Initially, students conducted pricing and marketing within a market. In the second round, students were required to continually restock via a purchasing function. The students needed to work together to correctly sell product, but not run out before their reordered product arrived. In the third round students also used the SAP system to complete materials requirements planning which, if completed correctly, signaled to purchasing that more product needs to be ordered. In total, students interacted with five different SAP screens to correctly complete the planning process, procurement process and sales process. Furthermore, students had to obtain information from 6 reports to understand their market environment, sell at a profit, and remain stocked with the products that best match the market. So, the game is complex, it simulates customer and vendor behaviors, as well as the passage of time. This is a serious business simulation, without an entertainment aspect, and requires the students to engage in strategic decision making and dynamic problem-solving.
Data collection for this study occurred at two distinct time points. The variables that were chosen for this analysis are a subset of a larger survey. This is the sole study resulting from that survey. At the start of the semester, a survey was conducted, collecting information about trait learning goal orientations. At this time, students were randomly assigned to work together in teams. Then, toward the latter part of the semester, the simulation game was introduced. Approximately three weeks of class time was reserved for the students to work together and engage in repetitive rounds of play. After the final round was completed, another survey was administered; this time asking about impressions of knowledge sharing behaviors, transactive memory systems, individual satisfaction levels, and perceived team performance. A total of 131 students participated in the final round of play. Out of that 131, seven students declined permission for their data to be used in the analysis and six records were list-wise deleted because of missing data. This left a total of 118 complete responses. Because of absenteeism, 18 of these 118 students participated in the game alone. This left 100 students working together in 42 teams as the basis for our analysis.
Measures
Learning goal orientation
Learning goal orientation was assessed using 4 items adapted from VandeWalle (1997). Sample items include: “I enjoy challenging and difficult tasks where I’ll learn new skills,” and “I often look for opportunities to develop new skills and knowledge.” Items were rated on a six-point scale ranging from 1 = “Strongly Disagree” to 6 = “Strongly Agree”. The internal reliability coefficient for learning goal orientation was 0.78.
Knowledge sharing norms
Knowledge sharing norms was assessed using 10 items adapted from Quigley et al. (2007). Items were rated on a seven-point scale ranging from 1 = “Almost Never True” to 7 = “Almost Always True.” When asked the extent it seemed that “you and your teammates developed a mutual understanding that each other on the team would. . .”, students responded to such sample questions as: “share information on hints when you thought it might help the others on the team,” “share information on strategies that seemed to work well,” and “go out of your way to help the others on the team with a problem or question.” The internal reliability for this measure was 0.96.
Transactive memory systems
Transactive memory systems (TMSs) was adapted from Lewis (2003). Using 15 items, rated on a 5 point scale ranging from 1 = “Strongly disagree” to 5 = “Strongly agree,” sample questions include: “Different team members were responsible for expertise in different team areas,” and “I have knowledge about an aspect of the exercise that no other team member has.” The measure includes dimensions of team specialization, coordination, and credibility. Internal reliability was 0.85.
Satisfaction with the team
Team satisfaction was adapted from Spector (1994). Using 4 items, rated on a 7 point scale ranging from 1 = “Disagree very much” to 7 = “Agree very much,” sample questions include: “I liked the people I worked with,” and “I found I had to work harder because of the incompetence of the people I worked with (reverse scored).” The internal reliability for this measure was 0.72.
Satisfaction with the task
Task satisfaction was adapted from Spector (1994). Using 5 items, rated on a 7 point scale ranging from 1 =”Disagree very much” to 7 = “Agree very much,” sample questions include: “I liked doing the things I did on the simulation game exercise,” “The simulation game exercise was enjoyable,” and “I felt the simulation game exercise was meaningless (reversed scored).” The internal reliability for this measure was 0.91.
Perceived team performance
Perceived team performance was assessed using 3 items, rated on a 6 point scale ranging from 1 = “Disagree very much” to 6 = “Agree very much.” Sample questions include: “My team performed very effectively on this exercise,” and “My team made a quality decision.” The internal reliability for this measure was 0.95.
Actual team performance
Actual team performance is a calculated variable, generated by the ERPsim game. The game simulates revenue and costs based on player decisions. Throughout the game, the players see the financial impact of their actions. At conclusion of each round, a set of financial metrics, such as total sales and gross margin, are calculated and displayed. We chose the natural logarithm of cumulative net income as our performance metric, but the variable remained non-normally distributed with skewness = -1.45 and kurtosis = 0.20.
Analyses
Aggregation and Measurement Analyses
To justify aggregating our individual response data to the team level, we examined proportions of within and between group variance, as well as indicators of rater reliability. We calculated ICC variables to check on the proportion of between and within group variance. The ICC(1) indicates the proportion of variance that is attributable to group membership, while the ICC(2) indicates the reliability of group means (Bliese, 2000; Hox, 2002). To ensure that team member assessments were similar, we computed interrater reliability scores (LeBreton & Senter, 2008). Using the tool from Biemann et al. (2012), we calculated the multiple-item estimator of rwg(j) with a uniform distribution, as well as measure specific distributions.
Table 1 contains the results of our analyses, which yielded support for our aggregation decisions. For example, with knowledge sharing norms, we found the following: mean rwg(j), uniform = .87; rwg(j), slight skew = .81; ICC(1) = .30; ICC(2) = .50, with an F ratio = 2.02, p<.01. For transactive memory systems, we found a mean rwg(j), uniform = .95; rwg(j), normal = .69; ICC(1) = .29; ICC(2) = .49, with an F ratio = 1.95, p<.05. Finally, for perceived team performance, we found the mean rwg(j), uniform = .78; rwg(j), slight skew = .69; ICC(1) = .36; ICC(2) = .57, with an F ratio = 2.32, p<.01.
Aggregation Results for Study Predictor Variables.
Notes. SD = standard deviation of rWG(J) values; shape = the shape of an alternative null distribution; σ ²e = variance of the alternative null distribution. Excel tool from Biemann et al. (2012).
p<.01
p<.05
Our ICC(2) results were generally below .60, which is considered on the low side. Our average team size was small at 2.38. Low ICC(2) values can be attributed to small team sizes and, while low ICC(2)s might adversely affect statistical power, they do not preclude the use of multilevel analytical techniques (Bliese et al., 2018). However, as an additional precaution, we conducted an rwg(j) sensitivity analysis to make sure these variables truly represent group constructs (Biemann et al., 2012). We eliminated four teams with low rwg(j) uniform scores and reran our analysis. We found little impact on the results and nothing that would alter our substantive conclusions. Consequently, we aggregated data and created our team-level variables.
Analytical Strategy
Our research question involves outcomes at two levels of analysis. At the lowest level, we are interested in the effects of trait learning goal orientations on individual perceptions of team and task satisfaction, (i.e., satisfaction with the team and satisfaction with the task). At the group level, we are interested in the effects of knowledge sharing norms and transactive memory systems on perceptions of team performance and actual team performance, as well as their effect on individual (level-one) perceptions, (i.e., satisfaction with the team and satisfaction with the task). Since our sample size is small, with just 100 observations, we opted to analyze our data in two parts. To analyze our group level variables, we conducted an OLS regression using SAS PROC REG and assessed the mediated, or indirect, effects using PROCESS (Hayes, 2017). To examine our individual level outcomes, we conducted random coefficients modeling using SAS PROC MIXED (Bliese, 2002; Singer, 1998). Our multilevel model contains a second-level mediator. Consequently, we used MSEM and MPLUS version 8 (Muthén & Muthén, 1998–2017) to assess our mediation paths, as this technique allows for higher level outcome variables (Preacher et al., 2010). We used a Monte Carlo Method for Assessing Mediation (MCMAM) to create confidence intervals for the indirect effects (Selig & Preacher, 2008). Prior to conducting our multilevel analysis, we needed to determine the best fitting model, so we followed the build-up procedure from Hox (2002). Using this method, we looked to see if the predictors, random intercepts, random slopes, and/or cross-level interactions were helpful with model fit. After this analysis, we determined that, for both satisfaction with the team and satisfaction with the task, our best fitting multilevel model, as written below, has a single level-1 predictor, random intercepts, two level-2 predictors and fixed slopes.
Yij = β
0j
+ β
1j
(TMLGO) + eij β
0j
= γ
00
+ γ
01
(KSNorms) + γ
02
(TMS) + u0j β
1j
= γ
10
Simplified model: Yij = γ
00
+ γ
01
(KSNorms) + γ02(TMS) + γ
10
(TMLGO) + eij + u0j
Our model does not include cross-level interactions, and we are not assessing an individual’s standing relative to his or her group. As such, we opted against group-mean centering. However, to facilitate interpretation of the intercepts, we did grand-mean center our predictor variables.
Results
Table 2 provides the means, standard deviations, reliabilities and correlations for the study variables. The information in 2.1 pertains to the individual level of analysis. The information in 2.2 pertains to the team-level variables.
Descriptive Statistics.
Team Level n=42; Individual Level n=100; Cronbach alpha reliabilities are listed on the diagonal.
p < 0.01 level; * p < 0.05 level; τ p < 0.10 level. (All 2-tailed tests).
Natural Logarithm of Cumulative Net Income
Multilevel Examination of Individual Outcome Variables
In order to test our hypotheses, we first had to ensure that significant team variance in in our dependent variables existed. Two null models were evaluated. The interclass correlation (ICC) for team satisfaction indicated that the 34% of the variance is between teams and 66% within teams. The interclass correlation (ICC) for task satisfaction indicated that the 33% of the variance is between teams and 67% within teams. Both ICCs provide strong support for continuing with our multilevel analyses (Bliese, 2000; Hox, 2002).
As depicted in Table 3, the first model presents information about the null model. The second model presents the impact of adding our individual-level predictor, trait learning goal orientation. Model 3 of this table includes the individual-level predictor and both team-level predictors, knowledge sharing norms and transactive memory systems. Model 3 was employed to test our hypotheses.
Estimates From Random Coefficient Models Predicting Level-One Satisfaction Outcomes.
Note: * p<.05; ** p<.01. Values based on SAS PROC MIXED with grand mean centered predictors. Entries show unstandardized parameter estimates with standard errors in parentheses. Standardized predictor coefficients are in italics. Estimation method = ML. Degrees of freedom method is between-within. aICC = [τ00 / (τ00 + σ
According to hypothesis 1a, we expected to find a positive relationship between knowledge sharing norms and both the satisfaction variables. Looking first at the fixed effects, we see that the relationship between knowledge sharing norms and satisfaction with the team is positive and significant (
Looking at the random part of our model, for team satisfaction we find that knowledge sharing norms and transactive memory systems explains the variance in team intercepts, ICC = 0.00. The combination of individual trait learning goal orientation, team knowledge sharing norms, and team transactive memory systems explains most of the intercept variance in task satisfaction, ICC = 0.04. However, a significant amount of variance remains unexplained within teams for both outcomes. The level-1 variance for satisfaction with the team is (σ2 = .40, p<.01). The level-1 variance for satisfaction with the task is (σ2 = 1.26, p<.01). This makes sense, as other factors, such as individual differences, might affect an individual’s satisfaction level.
To evaluate the overall impact of including the level-2 predictors, we looked at the analysis from a model fit perspective. We evaluated the difference in -2 Log Likelihood between the model with the individual predictor only and the final model, which included our two team level predictors. For the satisfaction with the team variable, we found a significant decrease in deviance for the final model (χ2 statistics 35.1, p<.01). A similar result was noted for satisfaction with the task, with an overall significant improvement in model fit (χ2 statistics 20.6, p<.01). These results indicate that our level-2 predictors explained a significant amount of variance in our model.
According to hypotheses 3, we expected team transactive memory systems to mediate the positive relationships between knowledge sharing norms and our level-1 satisfaction outcome variables. To assess our mediation paths, we used MPLUS version 8 (Muthén & Muthén, 1998–2017) and used a Monte Carlo Method for Assessing Mediation (MCMAM) to create confidence intervals for the indirect effects (Selig & Preacher, 2008). Confidence intervals were set to 95% and bootstrapping was conducted 20,000 times. Results indicate a positive relationship between knowledge sharing norms and satisfaction with the team; this relationship is mediated by transactive memory systems [(β = 0.07, p < 0.05, (CI 95%: 0.019, 0.129)]. However, the mediated relationship between knowledge sharing norms, transactive memory systems and satisfaction with the task is not significant [(β = 0.07, n.s., (CI 95%: -0.005, 0.149)], therefore hypothesis 1a and hypothesis 3 are only partially supported.
Analysis of Group Level Variables
To assess the relationships between our team-level variables, we used a regression-based path analysis known as conditional process modeling (Hayes, 2017). This technique employs nonlinear bootstrapping (Preacher et al., 2007) to evaluate the effect of a causal variable on an outcome through one or more intermediary variables, (i.e., an indirect or mediated effect). In this part of our analysis, we examined the indirect effects of knowledge sharing norms on perceived team performance and actual team performance through the development of specialized team cognitions, (i.e., transactive memory systems). Because we hypothesized that transactive memory systems are the result of knowledge sharing norms, in step one we examine the direct effect of knowledge sharing norms on transactive memory systems. In step two, we assess the direct effect of knowledge sharing norms on the team performance variables. In the third step, we enter in the effects of transactive memory systems and assess the mediation paths. After each step, we assess model fit using F-values and R2. Results from steps 1 through 3 are shown in Table 4.
Multiple Regression Results for Aggregated Team Level Variables.
N=42 teams. B=unstandardized beta; β=standardized beta; SE=Standard Error. Values based on SAS PROC REG. Indirect effects calculated using PROCESS version 3.3.
<.10; *p<.05; **p<.01;
In step 1, overall model fit results were acceptable at (F (1,40) = 17.05, p<.01) with 30% of the variance in transactive memory systems explained. Knowledge sharing norms was a significant predictor of transactive memory systems, with (β = .55, p<.01). In step 2, looking at perceived team performance, model fit results were also acceptable at (F (1,40) = 19.37, p<.01) with 33% of the variance in perceived team performance explained. Knowledge sharing norms was a significant predictor of perceived team performance, with (β = .57, p<.01). However, also in step 2, looking at actual team performance, model fit results were poor at (F (1,40) = .09, n.s.) with none of the variance in actual team performance explained. Knowledge sharing norms was not a significant predictor, with (β = .05, n.s.). In step 3, looking at perceived team performance, model fit results remained at acceptable levels with (F (2,39) = 29.32, p<.01) with 60% of the variance in perceived team performance explained. In this step, transactive memory systems was a significant predictor, with (β = .63, p<.01), while knowledge sharing norms approached, but did not reach statistical significance with (β = .23, p<.10). For actual team performance, model fit results were again very poor at (F (2,39) = 1.00, n.s.) with only 5% of the variance in actual team performance explained. Neither of the predictors were significant, with knowledge sharing norms at (β = -0.09, n.s.), and transactive memory systems at (β = .26, n.s.).
After running the basic regression models, we employed process modeling and bootstrapping techniques to assess the conditional indirect paths. Because we did not find statistical significance with our actual team performance variable, we focused on perceived team performance. Bootstrapping was invoked 5,000 times. Using p-values and biased-corrected bootstrapped confidence intervals less than .05 as our guide, results indicate that the transactive memory systems variable serves as statistically significant mediating mechanism between the focal predictor (knowledge sharing norms) and the outcome variable (perceived team performance). The mediated pathway from knowledge sharing norms to perceived team performance via transactive memory systems was significant with [(B = 0.34, p<.05); (CI= 0.19, 0.51)]. Because we did not find statistical significance with our actual team performance variable, these results lend partial support to hypothesis 1b, which states that knowledge sharing norms will be positively related to a) perceived team performance and b) actual team performance. The results also lend partial support to hypothesis 3, which states that transactive memory systems will mediate the relationship between knowledge sharing norms and a) perceived team performance and b) actual team performance.
In sum, our results indicate that, at the individual level, trait learning goal orientation is related to satisfaction with the task, but not to satisfaction with the team. At the team level, knowledge sharing norms is positively and significantly related to satisfaction with the team, but not satisfaction with the task, while transactive memory systems has a significant and positive effect on both satisfaction with the team and satisfaction with the task. Transactive memory systems partially mediate the relationship between knowledge sharing norms and satisfaction with the team and fully mediate the relationship between knowledge sharing norms and perceived team performance. We found no significant predictive relationships to actual team performance.
Discussion
Our results reveal an interesting pattern of outcomes. Drawing on the framework from the IMOI model, we see that, as an input variable, higher levels of learning goal orientation led to higher levels of satisfaction with the task. This makes sense, as a person intent on mastering the material would embrace the challenge of the simulation game, and see the exercise as a way to build personal competencies. This finding is congruent with other studies utilizing computer-based simulation games. For example, researchers looking at ways to enhance learning in workforce training have adopted an active learning approach. In this methodology, instructional design components specifically encourage trainees to adopt mastery orientations. Errors are framed as learning opportunities, and trainees are encouraged to explore and experiment with the computerized task. Although immediate performance suffers, deeper learning patterns are achieved through this technique and that deeper learning transfers more readily to the job (Bell & Kozlowski, 2009). While individual learning goal orientation had a positive effect on satisfaction with the task, we found no statistically significant effect between learning goal orientations and satisfaction with the team. In other words, while the students focused on mastering the material enjoyed the simulation game more, the positive effect did not extend to their teammates. Perhaps, these students were engrossed in their own cognitions and not as engaged in team-related communications. This was not expected and would be a fruitful area for more research.
The variables that had the most explanatory effect in our model were related to social exchange and interactions between teammates. In our model, we cannot explain the origin of knowledge sharing behaviors. Those were not explicitly encouraged during the exercise. However, consistent with the predictions of social exchange theory (Cropanzano et al., 2017), when teams engaged in reciprocal knowledge exchange behaviors and established norms for knowledge sharing, the students were happier with their teammates. Moreover, and again consistent with the IMOI model and social exchange (Cropanzano et al., 2017; Ilgen et al., 2005), through repetitive iterations of team processing, these knowledge-sharing norms led to the establishment of transactive memory systems. As transactive memory systems were developed, the team thought they performed better, they liked the simulation game more, and they appreciated each other more. While we did not find a relationship to objective team performance, these affective outcomes might speak to the willingness of the students to engage in additional future coursework or continue to work with their peers in future team settings. Certainly, reporting a positive team experience would be beneficial as these students begin the job search and begin careers in organizations. While our study was confined to a computerized simulation game, the benefits of encouraging achievement motivations and reciprocal knowledge sharing behaviors within teams would extend to more traditional group-based projects as well.
Implications for Teaching
This study illustrates the potency of simulation games to cultivate behaviors that could lead to the development of team competencies. We found that knowledge sharing behaviors led to the development of specialized team cognitions. In this way, by lessening the cognitive load on any one person, participants not only enjoyed the learning task more, but they also enjoyed working with each other. This enjoyment was evidenced not only by the satisfaction variables but also through enhanced perceptions of team performance. In the context of an undergraduate game, perceptions of team performance would likely be predictive of a willingness to engage in future team interactions. A positive experience, lending itself to future team engagements would likely lead to the development of team competencies, competencies that are so valuable to future employers. Individuals’ learning goal orientations, on the other hand, led to an enhanced sense of satisfaction with the task, but not necessarily to an appreciation of one’s teammates. Since knowledge sharing behaviors can be encouraged and learning goal orientations are malleable, the results of this study should be of interest to educators (Bell & Kozlowski, 2009; Steele-Johnson et al., 2000).
Learning goal orientations can be encouraged through the use of instructional design techniques that encourage self-regulatory learning, e.g. encouraging exploration, experimentation, and positive framing of errors (Bell & Kozlowski, 2009). To facilitate norms for knowledge sharing, the students should be encouraged to work together, ask each other questions, and converse during the simulation rounds. By de-emphasizing evaluations and stressing the development of abilities, students should adopt “state” learning goal orientations (Steele-Johnson et al., 2000). Instead of seeing a question and answer exchange as a potential exposure of incompetence, it should frame the knowledge exchange as a way for all members on the team to benefit. Moreover, it should encourage all members of the team to engage in the simulation game.
During the debriefing, instructors should remind team members of how their ability to share understandings and their mental representations of the team’s task environment generally reduced the mental load on any one team member. Thus, through reflection, this should reinforce the concept that no one person can do it all; moreover, when knowledge sharing occurs among team members, it makes the learning task much more enjoyable. In this way, through the use of simulation games, educators can emphasize critical thinking and problem-solving skills, while also encouraging the development of behaviors that lead to team competencies.
Limitations and Suggestions for Future Research
Our study was a correlational study. We captured survey information from students enrolled in a logistics and supply chain class in a single, medium-sized university. To gain more insight into the formation of student team structures and cognitions, future studies should include control groups and experimental conditions. Future studies might also look at the antecedents of team knowledge sharing norms, as well as the effects of encouraging learning goal orientations in tandem with knowledge sharing behaviors. None of our independent variables were predictive of actual team performance. This was an unexpected finding and could be due to our research design. Before we measured performance, we allowed the students several rounds of play over the course of three weeks. During this time, some of the students may have had time to develop more advanced levels of skill, while others may not. Also, we may not have captured the variables relating to actual performance. Another explanation might be related to statistical power. A post-hoc power analysis suggests that the number of teams in our sample size was small. However, even with a small sample size, we found significant effects in support of prior research and a-priori theorized relationships. In the future, studies might compare student reactions from a computer-based simulation game with those from other, more traditional, group projects, such as research reports or in-class presentations. Also, longitudinal studies should follow students through their respective program completions and see if the experiential nature of the simulation games is helpful for upper-level course work, as well as future job opportunities and future job performance.
It is also worth noting that, in our study, knowledge sharing norms led to the formation of transactive memory systems. Our participants were novices and most likely volunteered to acquire, rather than share, respective areas of expertise. However, with a more professional sample, the sequencing of transactive memory systems and norms for knowledge sharing might be reversed, i.e., having an established TMS would lead to knowledge sharing, which would, in turn, lead to better team performance (Choi et al., 2010)
Conclusion
The goal of our study was to advance an understanding of determinants of student success in computer-based simulation games, at the individual and team levels. Consistent with theories of achievement motivations, our findings suggest that students with higher levels of trait learning goal orientations are intent on the game itself, and they enjoyed the learning exercise. From a social exchange perspective, we find that team member interactions and reciprocal knowledge exchanges were instrumental in the development of specialized team cognitions, or transactive memory systems. In our study, when teams formed transactive memory systems, they liked working on the game, they perceived that they were performing better on the game, and they enjoyed their teammates more. Future studies should look at the factors which are instrumental in encouraging team knowledge sharing norms, as well as the effects of encouraging learning goal orientations in tandem with knowledge sharing behaviors.
Footnotes
Declaration of Conflicting Interests
The authors declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
Funding
The authors received no financial support for the research, authorship, and/or publication of this article.
Author Biographies
Contact:
Contact:
Contact:
Contact:
