Abstract
This paper presents a rationale for establishing collaboration between corporate IT projects and those conducting empirical research on teamwork. A review of teamwork research reveals gaps in many teamwork models: the influence of context of performance, the mechanisms of team development over time, and data generated by practitioners in naturalistic settings. The paper describes the Scrum framework for executing projects in IT departments in terms of input artifacts, output artifacts, team processes, and the role of organizational context. Given the lack of quantitative project outputs, the method describes how Thematic Analysis and qualitative research are needed to convert project outputs to insights on team processes. This method of conducting research on team performance aligns closely with the Macroergonomics framework for research and its attention to team, task, context, and study with practitioners in naturalistic settings.
Introduction
Human factors research on teams, teamwork, and effectiveness has a long history across a wide range of complex domains such as military mission planning, corporate decision making, power plant control rooms, and medical emergency rooms. From this empirically-based research have come frameworks, theories, and models that have been successfully applied to problems of design, education, and training. For a comprehensive summary of research on teams and teamwork see Salas, Cooke, and Rosen (2008).
Robust Frameworks: Teamwork Competencies
Perhaps no framework is so robust as that of the ABC’s of teamwork competencies: attitudes, behaviors, and cognitions (Salas, Rosen, Burke, & Goodwin, 2009, p. 50).
Attitudes
Beliefs that influence behaviors and cognitions within a given context. Well-studied constructs include team efficacy, self efficacy, interpersonal trust, and psychological safety (Cannon-Bowers, Tannenbaum, Salas, & Volpe, 1995), as well as meaningfulness of work and psychological safety (May, Gilson, & Harter, 2004). Additional variables for investigation include willingness to experiment and openness to feedback.
Behaviors
Individual team members are dependent on one another to complete their own tasks. Team behaviors are those actions taken to aid others’ taskwork and satisfy team goals. These include categories of behaviors such as task organization, information exchange, and coordinating activities (Cannon-Bowers et al., 1995).
Cognitions
Effective teams must understand the team’s mission and goals, roles, and the abilities of their members, as well as the nature of the problems and solutions needed to deliver on goals (Cannon-Bowers et al., 1995). The interpretation of others’ expressed attitudes and behaviors in context may also influence team performance.
The IPO Model
Input-Process-Output (IPO) models represent team functioning as a set of team processes that accept inputs from an environment to produce intended outputs. The general framework is flexible enough to produce a number of variations that fit easily within the framework. “The IPO framework has proven highly robust and adaptable. It has been scaled both up and down in terms of temporal frames to better explain team performance” (Goodwin, Burke, Wildman, & Salas, 2009, p. 6).
Directions for Future Research
As with topics so complex and closely studied, researchers often note the domains that deserve further study. Extending the current scope of teamwork research into new domains may yield new insights. In reviewing the calls-to-action among researchers, three themes emerge:
1) The impact of
2) The relevance of
3) The importance of working with
Teamwork in Organizations: The IT Project Domain
Information Technology (IT) projects are ubiquitous around the globe as companies compete with their rivals to reduce the costs of doing business, improve quality, and scale their operations. The advantages of collaborating with practitioners in the IT project world are obvious. IT projects are complex, staffed with different skill sets and levels of experience. IT projects last from weeks to months, sometimes years. Companies spend a lot on training and coaching, actively seeking to improve their investments in technology projects. Approaches to teamwork improvement are typically based on what works in practice, and not informed by theory.
Given the number, complexity, and potential richness of teamwork data from IT projects, and the keen interest among practitioners in improving team processes, it would seem that researchers have found in IT projects a perfect test bed for theories and frameworks of team performance and effectiveness. However, working with IT projects poses significant challenges. For example, IT projects are not simply long and complex – they may be poorly controlled, subject to be changed or canceled at any time, thwarting even the bestdesigned study. Companies are often unwilling to disclose detailed information of how a project was conducted. Researchers are not typically familiar with IT projects and its processes. Finally, other than using opinion surveys and retrospective interviews, it’s not obvious how to measure behaviors of interest as they occur.
Despite these challenges, two recent developments in the corporate world offer a means of overcoming the barriers to collaborating with IT projects on teamwork research. The first development is the rise of remote work, the second is a software creation and teamwork improvement framework known as Scrum.
The Rise of Remote Work
Before the pandemic began in 2019, projects were usually run in the office, with team members co-located – physical proximity was thought to enhance team communication. Progress was tracked on whiteboards and printed cards in a shared space visible to team members and stakeholders. Today, many IT project teams operate online and team members are fully remote, communicating through electronic channels like Teams, Zoom, and Slack. Progress on tasks is tracked in tools like Jira. Remote work facilitates recording and near-real time analysis of conversations, chats, links to valuable documents, and task completion metrics.
The Scrum Framework
For 20 years, the Scrum framework has become an increasingly recognized means of executing IT projects. IT managers who regard Scrum only as a software engineering process for writing code faster are surprised to find no wellspecified methodology within the definitive Scrum Guide (Schwaber & Sutherland, 2020). Instead, the Guide begins with a discussion of the basis of Scrum theory: commitment to transparency, inspection, and adaptation.
Scrum projects embrace experimentation. Scrum process relies on a skilled team leader, usually in the role of scrum master or coach, to develop individuals and teams. Team leaders employ practice-based research methods of measurement, analysis, and experimental design to improve performance over time.
Scrum: A Primer
Scrum is, at its base, a method of improving teamwork as well as increasing the value of IT projects. Well-run Scrum IT projects are not, as many believe, “anything goes,” but are disciplined and focused. The discipline, predictability, and commitment to transparency, inspection, and adaptation make Scrum IT projects excellent candidates for research into team processes, leadership, and team problem solving.
Scrum Artifacts as Sources of Insight
The activities in a Scrum IT project are built around four types of events within a 2-4 week interval, or Sprint. The following is not a full description of Scrum and its components, but is provided as a means of identifying the events and artifacts available for inspection. For a full description of Scrum events and artifacts refer to Schwaber and Sutherland (2020).
At a minimum, all Scrum project teams collaborate to plan and build three primary artifacts: a Product Backlog, Sprint Backlogs, and Increments of value (see Table 1). The Definition of Done (DoD) is a detailed, objective description of what is needed to judge an Increment complete. If adopted by multiple projects it allows comparison across projects.
Scrum Artifacts Created and Maintained by a Scrum Team.
There are four types of events conducted within a 2-4 week Sprint (see Table 2). Events are conducted on a regular schedule. Each event is an opportunity for the team to inspect the teams’ goals and artifacts and to make changes as necessary.
Scrum Events Conducted Within a Sprint.
The IPOs of Scrum Events
Scrum events take well-defined artifacts (inputs), process them, and generate specific outputs. Each event reinforces the team’s commitment to transparency, inspection, and adaptation. The format of Scrum events are consistent and well-structured, and lend themselves to presentation in familiar IPO diagrams.
Sprint Planning
The Sprint Planning event is seen in Figure 1. The artifacts listed as Inputs are reviewed and analyzed as described in Table 2. The Sprint Planning Transcript (in gray) is not a traditional Scrum artifact but could be produced from a recording of the event using commercially available speech-to-text software and inmeeting text chats. Planning is a primary source of understanding (cognitions) of each team member as goals and plans are discussed.
The Sprint Planning IPO.
Daily Scrum
Each day for 15 minutes the team reviews progress toward the Sprint Goal in terms of item completion and makes agreed-upon updates to the Daily Plan if necessary (see Fig. 2). As with the Sprint Planning outputs, the Daily
The Daily Scrum IPO.
Scrum Transcript is not a traditional Scrum artifact, but could be produced from recordings. The discussions and online chats in the Scrum are a rich source of visible teamwork behavior as team members respond to requests for information, offer advice, and alert others that an item of work they were responsible for is ready for use.
Sprint Review
At the end of a Sprint, the team presents the output of their work in terms of the Increment delivered during the Sprint and progress toward the Product Goal (see Fig.3). Project stakeholders are involved in the Review to provide feedback, solve problems, and offer direction. If a team’s understanding of the Product Goal is incomplete, or the Goal needs to change, the discussion during the Sprint Review will reveal that.
The Sprint Review IPO.
Sprint Retrospective
The Sprint Retrospective allows the team to discuss the previous Sprint and make adjustments with respect to internal communications, process or technical challenges, and support from the organization or team leaders (see Fig. 4). The Retrospective is a primary source of insight on the attitudes of team members toward their own performance and progress toward team goals. Often, estimates of relative effort for the work required in the Sprint are reexamined and incorrect assumptions identified. The event facilitator often solicits topics for discussion from team members, and uses its feedback to create an agenda for the meeting.
The Sprint Retrospective IPO.
Although it is not a traditional Scrum artifact, an Attitude Survey (in gray) sent prior to the meeting can help structure the meeting agenda, and provide quantitative data of changes in team attitudes over time. The Contextualized Attitude Survey includes the reasons given for survey responses.
Macroergonomics
The discipline of macroergonomics provides both a framework and methodology for studying project teams in the context of organizations (Kleiner, 2008). For any study of IT projects, an analysis of team functioning would be incomplete without describing the organizational and technical context within which teams must operate. The macroergonomics perspective takes a large-systems view of team development, and emphasizes field studies over formal lab studies.
In an IT setting, organizational support refers to 1) the technical environment: technology stack, technical infrastructure, development and test environments, the maturity of the integration and deployment process and 2) the social environment: the stakeholders, managers and corporate offices the project must negotiate with to establish direction and goals, funding, staffing, training, and other support.
Experienced Scrum team leaders establish the types and degree of support required from the organization to meet a Project Goal. They engage with technical leads or senior developers, and monitor changes in the technical environment (or delegate the responsibility). They also monitor the organizational environment for potential impacts to the team, take action to remove barriers to success, and communicate results to the team. The last step is important; perceived organizational support has an impact on team commitment and therefore to performance (Rhoads, 2002; Rhoades, Eisenburger, & Armeli, 2001).
In terms of incorporating hard-to-quantify data in naturalistic field studies, the macroegonomics approach values quantitative and qualitative research in the study of teams. “Research in the field and especially the use of qualitative methods, perhaps, take some human factors and ergonomics researchers out of their comfort zones, but this does not make the pursuit any less valid or important” (Kleiner, 2008, p. 462).
Qualitative Research: Thematic Analysis
Most of the team performance data of interest in projects are found in the discussions, online chats, and emails between team leaders, team members, and stakeholders, appropriate for qualitative data analysis. Reflexive Thematic Analysis (RTA) is a flexible, powerful qualitative method for “identifying, analyzing and reporting patterns (themes) within data” (Braun & Clark, 2006, p. 79). More than just a coding scheme for tagging text, it can help provide a “rich description of (a) data set, or detailed account of one particular aspect” of observations of interest (p. 83).
RTA proceeds in either theoretical (deductive) or inductive fashion. Theoretical RTA is applied top down; researchers have identified themes prior to analysis and code incidents based on previous theory. Inductive RTA is applied bottom up; researchers sort and code incidents and attempt to discover themes as the analysis is conducted. In a domain as theory-rich as teamwork processes there is no lack of themes or patterns to which coded incidents can be mapped. However, there is no reason the same data could not be analyzed both top down and bottom up for the purpose of validating existing themes and generating unexpected insights.
There are a number of accessible resources created by practitioners that explain how to execute RTA, discuss its benefits and drawbacks, and make recommendations on software applications that support the process (Rosala, 2022).
Research Method
The outputs of Scrum events as tracked over the lifetime of a project serve as the inputs to the analysis of team performance and effectiveness: context, competencies, and project output.
Measuring Context: Organizational Support
Technical Environment
A line item list of things the project is dependent on. Pre-project checklists are commonly made available to projects by the Project Management Office (PMO) or other administrative function. Such a list could include a project-specific list of external dependencies needed for project completion and managed by team leaders who review changes over time. It could be inspected and updated in Sprint Planning or Sprint Retrospective events (as a source of blockers).
Social Environment
Less commonly measured but no less important, this is often handled as part of a project’s risk management approach. The team identifies risks, common among those being team member reassignment, loss of sponsorship, or lack of priority for the project. If executed as part of the agenda for a Sprint Retrospective, this procedure can generate Support Requests to stakeholders and managers.
Measuring Competencies
Attitudes
Quantitative measures of interest to the team and researcher may be administered as surveys in preparation for a Sprint Retrospective and help form the meeting agenda (see Fig. 4). The team can review the results as part of a discussion of what gaps, if any, need to be addressed in the next sprint. A survey may include attitudes towards organizational support (May et al., 2004; Rhoades, 2002).
Behaviors
The transcripts and online chats of recorded Daily Scrum meetings reveal teamwork behaviors as team members provide information and documents to others who need them. Sprint Retrospectives also provide evidence of good team behaviors as team members acknowledge help received from others. Evidence from spoken behaviors should correlate with the status of work items in the Sprint Backlog.
Cognitions
The Scrum team’s understanding of their project and team are apparent in all four Scrum events. Sprint Planning reveals the team’s understanding of the Product Goal, individual Sprint Goals, and plans for delivering Increments as discussions and problem solving with team leaders occur (see Fig. 1). Sprint Review shows how accurately the Scrum team did (or did not) understand their stakeholders’ needs when inspecting and comparing the completed Increment with the Sprint Goal. The Sprint Retrospective shows the Scrum team’s awareness of its shortcomings during the preceding Sprint. If the team inspects and re-estimates the Sprint Backlog items, differences with the original Estimates of Effort will reveal misunderstandings and incorrect assumptions about the nature of the work. As seen in practice, teams improve their ability to estimate by adjusting their estimating procedure; the ability to accurately estimate effort for work items is a feature of wellfunctioning teams (Cohn, 2006).
Measuring Project Output
The content of the Sprint Backlog and the Sprint Increment define what has been completed at the end of each Sprint. Comparison of what was planned with what was delivered is a valid measure of team effectiveness. If the Definition of Done was defined properly there should be no subjectivity in what work was considered complete.
Measuring Change Over Time
The development of a team is a dynamic process; team performance and effectiveness should be tracked over time to be fully understood (Santoro, Dixon, Chang, & Kozlowski, 2015). Scrum events happen on a regular cadence as an essential feature of the method. If researchers attend the events and keep pace with the team by analyzing event outputs and reviewing emerging themes as they are available, they can feed insights back to the team and assist with the team’s development. They would also experience the team’s improvement first hand, gaining insight into how to more accurately capture team changes.
Discussion
Scrum IT projects have everything a teamwork researcher might want: team interdependencies, domain complexity, problem solving, context, and unambiguous work outputs produced on a regular schedule over weeks or months. A Scrum project’s intentional commitment to transparency ensures that team attitudes, behaviors, and cognitions are available for inspection and adaptation.
It should be noted that not all Scrum teams are alike. The effectiveness of a Scrum team depends on the quality of the team leaders, support from management, and a robust technical infrastructure. Researchers interested in collaborating with Scrum teams would need to evaluate an organization’s maturity and support for the Scrum framework: management, infrastructure, and team leadership.
