Abstract
This paper explores some challenges and opportunities in designing visual interfaces for deep space habitats with communication delays to Earth. The study first examines how Mission Control for the International Space Station (ISS) operates in detecting and responding to anomalous events. Interviews with participants who have held roles as flight controllers, backroom engineers, astronauts, or flight instructors for ISS and shuttle missions were conducted to gain insights into the responses to anomalous events. The collected anomalous scenarios were then re-considered under the context of deep space missions, where communication delays and other environmental factors are substantially different. Findings and insights on data analysis of the anomalous scenarios were then presented subsequently. Based on the findings, four areas of challenges and opportunities for visual interfaces design were identified. The paper suggests that these design challenges and opportunities be considered in more detail in future research.
Keywords
Introduction
Ground support is critical to the safe and effective execution of space exploration missions, both during nominal operations and when anomalies occur. As we move closer to the reality of human deep space exploration, communication delays caused by the physical distance between Mission Control and crew members will become increasingly significant (Love, 2019). Currently, with the International Space Station (ISS), support from ground is available almost continuously in real time. Synchronous communication allows for collaborative, real-time problem solving between the ground and crew. Under such conditions, addressing anomalies can involve several dozens of people having expertise in various systems and processes. For future missions beyond low-Earth orbit, however, synchronous communication will not be possible, and round-trip delays of 10-40 minutes will become the norm. Communication delays on this scale will require fundamental shifts in the ways in which cognitive work is done. For instance, a small crew alone may need to respond to anomalies that have historically been handled by distributed teams 20 times larger (McTigue et al., 2021). Such delays can bring a large degree of uncertainty to command and control and anomaly response, having implications for the resilience of mission operations in general.
From hundreds of human space missions over in the past 60 years (Chamitoff & Vadali, 2021), we have accumulated mission control knowledge and experiences in low-Earth orbit. Specifically, the uninterrupted operation of the ISS over the past two decades has led to a wealth of insights and lessons-learned that could inform the development of future resilient extraterrestrial habitats. As we start to think about the design of decision aids, visual displays, and other artifacts for deep space missions, we face the challenge of anticipating new demands and constraints that we have not seen before. It is well known that the impacts of design changes on future practice are difficult to predict, due to often unforeseen complexities and opportunities that are created within sociotechnical systems (Dekker et al., 2002). This is particularly true in entirely new fields of practice, such as deep space missions with significant communication delays to Earth.
In this paper, we report preliminary findings from a cognitive task analysis with mission support practitioners from the ISS and shuttle missions, using the theoretical lens of macrocognitive processes and functions (Klein et al., 2003). Our goal is to understand the nature of current practice in order to envision future challenges and opportunities, especially at the intersection of anomaly response, data visualizations, and visual displays.
Research Approach
Inspired by previous research on the envisioned world problem (Dekker et al., 2002; Miller & Feigh, 2019) and the challenges of using theory for generative design purposes (Beaudouin-Lafon et al., 2021), we relied on our cognitive task analysis to support generative design thinking. Specifically, the first step involved gathering expert-reported data on anomalous scenarios encountered during prior operations of the ISS, and conducting detailed analysis on these incidents—specifically with a focus on how displays and data visualizations were used. The data was collected from a series of semi-structured interviews with practitioners having extensive experience in ISS Mission Control and related similar space missions, including shuttle missions and trainings.
Subsequently, we “recast” these collected scenarios by asking a series of “what if” questions under the context of deep space missions, where communication delays and other environmental factors are substantially distinct from those experienced in low-Earth orbit operations. In these re-casted scenarios, we look for new macrocognitive challenges, relating to communication, function allocation, planning and replanning, maintaining common ground, and so on, that are affected by decreased support from ground control.
Method
As part of an ongoing effort to study human habitats for deep space (Dyke et al., 2020), a series of interviews was conducted with participants with ISS and shuttle experience. Our target population included current and past ISS and shuttle flight controllers, managers, and trainers, as well as current or previous astronauts who have flown ISS or shuttle missions. Participants were recruited through a mix of purposive and snowball sampling. A recruiting email was sent internally to NASA personnel through a liaison on our project, providing information about the study and asking for volunteers. Some participants recommended others they thought would be good for the study. Participants first completed a short questionnaire asking about their role, experience, degree, and additional training or certification.
The interviews focused on the responses of ISS flight controllers and astronauts to anomalous events, with an emphasis on the decision-making strategies and the utilization of visual displays for telemetry data. Using techniques for cognitive task analysis—specifically relying on general knowledge elicitation and critical decisions (Hoffman & Militello, 2008)—participants were questioned on topics such as data acquisition, display functionality, and situation awareness.
Interview structure
The interview format was semi-structured, with a target time of 90-120 minutes. The interviewer first introduced the overall objectives of the research and contextual information for the rest of the conversation. Interviewers then clarified participants’ relevant experiences and job roles while they were in such experiences. The next section focused on situation awareness and the use of visual displays. Participants were asked about how they used displays to monitor, predict, and respond; what they needed to keep track of for the ‘big picture’ of their scope of responsibility; and a how they configured their multiple screens and why. The third section focused on eliciting participants’ expertise and their adaptation in their work. We focused on what makes their work challenging, and how they learned to work ‘smart’ and get things done. Following this, we asked participants to recount particularly challenging situations they faced. Timelines of the incidents were constructed with the help of interviewers. Detailed causes and resolutions of such incidents were explored.
In the last section, which was more open-ended than the others, we asked participants about their concerns and expectations for future deep space missions. Discussions were focused on what could be potentially challenging and in what aspect the mission could be fundamentally different. We also asked participants about the role of automation in their current practice, and their opinions about future automation and artificial intelligence in space exploration.
Participants
We recruited a total of 23 participants (12 females and 11 males) from highly relevant areas of work. Participants had an average of 7.9 years of experience in their current or related roles (ranging from 0.5 to 27), not including time spent for their training and certification. The mean time of the career length for the participants is 13.5 years, ranging from 3 to 35.
The study was approved by the Institutional Review Board (IRB) of Purdue University and informed consent was collected from each participant prior to the interviews. Table 1 shows the roles of each participant recruited for the interviews.
List of participants and their relative experiences.
Data Collection and Analysis
Interviews were conducted with each participant for approximately 90-120 minutes. Interviews were conducted remotely, using either Zoom or WebEx, and recorded for later transcription and analysis. Recordings were transcribed automatically using the Dovetail platform. Each transcript was then iteratively checked for errors and then corrected manually by the authors.
Transcripts were analyzed using a hybrid thematic analysis approach (Fereday & Muir-Cochrane, 2006), incorporating both data-driven inductive coding and top-down a priori coding. The authors first read through the transcripts multiple times, and then engaged in several rounds of coding. Our top-down approach was guided by previous literature on cognitive engineering and naturalistic decision making. Based on the theoretical backings of macrocognitive processes and functions the authors have listed out themes and codes that are relevant to the focuses of this study. We specifically looked for data relating to (1) known macrocognitive challenges such as maintaining common ground, handling cascading effects, avoiding fixation, managing attention and uncertainty; and (2) design challenges and opportunities in relation to automation and artificial intelligence, data visualization, and interactive artifacts more generally. We went through several rounds of bottom-up coding independently with these high-level categories in mind. We met regularly to discuss the codes, eventually coming into alignment on the final codes.
Anomalous scenario collection
Based on established norms for probing critical decisions (Klein et al., 1989), we asked each participant to recount a situation that was particularly challenging to deal with. Most of the stories surfaced by participants were in relation to anomalies they encountered either while sitting ‘on-console’ or during simulations (‘sims’). The scenarios covered a wide spectrum of anomalous events. At the time of writing, anomalous scenarios from four participants have been cataloged and analyzed. The selected anomalous scenarios were considered characteristic in different aspects in terms of their types of anomalies, types of communication involved, and the types of decision making and planning required to respond to such scenarios.
ISS cabin ammonia leak scenario
In this scenario, a potential ammonia leak occurred onboard ISS and was later discovered to be a false alarm. The anomalous situation caused multiple occurrences of communication between mission control and the crew, as well as an emergency evacuation of the US crew members to the Russian side. The false alarm was later discovered to be caused by a radiation hit which caused damage to the onboard computer. In the process of anomaly response, mission control once has determined that it was a false alarm and let the crew members to get back to work, and then further observed a consequential interior pressure raise due to cabin cooling shut off. The raised pressure caused them to suspect that there was indeed an ammonia leak and has eventually let the crew members to evacuate to the Russian side. The back-and-forth process of identifying and addressing anomalous situation has provided us with a lot of useful information.
ADCO gyro rate sensor failure scenario
In this anomalous scenario, our participant described an ‘e-log’ (electronic logging system) message indicating that one of the attitude control rate sensors had failed, and they were already in a state of degraded redundancy. The rate sensor is a key component of attitude control, so the flight controllers must react to a sensor failure immediately. Our participant let the flight director know about the situation as soon as the anomaly was identified and started the process of turning on the backup rate sensor. After configuring to turn on the back up sensor, the participant cross checked with multiple data sources to make sure the new sensor worked as it should.
C2 computer update error scenario
In this scenario, an onboard command and control computer experienced a failure during a planned updating process. The failure prevented the crew members or mission control from making commands to the ISS. After the failure had been identified, the participant communicated the resolution with crew members and quickly reverted the computer with a working version software update to solve the anomaly.
ISS cabin coolant leak scenario
In this scenario, an in-cabin coolant leak was caused by a failed quick disconnect port when ISS crew members were fixing a treadmill. When the incident happened, ISS and Mission Control were in an anticipated loss-of-signal. When the ground regained communication with ISS, the issue became known to the flight controller and he had to act quickly to address the situation. The crew were able to quickly help diagnose the problem, but as the consequence of coolant being leaked, the coolant loop was shut down automatically by the computer. This cascading event had more potential effects, since critical systems could be impacted with the cooling not functioning properly. Our participant devised a novel workaround, remembering there was a spare container that could be reconfigured for use by the coolant loop, restoring the cooling in time to keep critical systems from shutting down.
After reconstructing the timelines for the scenarios, the problem-solving process of these anomalous events and consequences of the actions taken by our participants have been documented also in detail within Dovetail platform. Further analysis on how visual interfaces were used by the crew member or the ground controller in such scenarios was conducted and then documented.
Scenario “recasting”
In an attempt to address challenges in envisioning future practice (Dekker et al., 2002), and to support theory-driven generative design thinking (Beaudouin-Lafon et al., 2021), we took the anomalies, as described to us by participants, and re-cast them in the context of future space missions having significant communication delays. In doing so, we focused on the shift that would have happened in function allocation, communication, decision making priorities, and command and control, should there be significant communication delays and limited support from ground. These counterfactual simulations around scenarios that were selected and mentioned in the previous sections were then documented in detail separately (Figure 1).
Recasting existing anomalies into future deep space mission contexts
Identifying challenges for design
According to the documentation synthesized in the previous steps, careful analysis and identification on the visual interface design challenges posed by communication delay in future deep space missions have been conducted. The findings are collected, compared, summarized and shown in the section below.
Findings
Before discussing the identified design challenges, we would like to highlight some important findings about ISS Mission Control that support previous research findings and are relevant to future deep space missions. First, we found that anomalies are common and inevitable—even in a well-established system like the ISS. Anomaly response is a core activity of flight controllers currently, and likely will be for any future deep space mission. This finding is in line with what is known about complex systems—even those that are well established—and is important to keep in mind for engineering resilient systems (Hollnagel et al., 2006)
Second, our analysis revealed that anomaly detection and problem-solving in the ISS context is highly distributed. Flight controllers are system experts and work together to coordinate information sharing and decision making. Ground control communicates frequently with the crew members when there is an issue, and the crew members trust the support and advice from ground control. Subsequently, ground controllers rely heavily on their training and protocols, communication with the team, and the information and data displayed on their screens when responding to anomalies. This echoes findings from space shuttle investigations (Watts-Perotti & Woods, 2007).
Third, the operation of the ISS is characterized by a wide and dynamic range of intensities and tempos. Once an anomaly has been identified by either ground control or crew members, they quickly switch their working pace to address the issue. Ground controllers use their displays to access mission-critical information, such as sensor readings and system status, in a more focused way than during normal monitoring. In this high-tempo and mission-critical context, they make decisions based on their experience gained from extensive training and system knowledge. Although such responses rely on established routines and simulated scenarios in training, the anomalies encountered are often unique and do not have pre-determined procedures for addressing them. Yet, practitioners are able to generate unique solutions relying on their tacit knowledge (Polanyi, 1966; Klein, 2011).
Design opportunities and challenges
Based on our cognitive task analysis, we have identified some design opportunities and challenges regarding visual interfaces and the management of prolonged communication delays in deep space missions:
Knowing the “time”
The time indicated here is not simply referring to the time at current time instances. It represents the effects time could cause due to communication delays. Communication delay is likely to fluctuate constantly during a deep space mission. For example, one-way transmission from Earth to Mars takes anywhere from 3 to 20 minutes, depending on the relative positions of the planets in the solar system. As the crew members travel, the transmission time also changes due to the varying distance. In a high-paced anomaly response, it is crucial for ground controllers to know when the command is expected to be received by the crew members and when to potentially expect a confirmation back. If commands are being sent to a habitat to be executed remotely, it is also important to track their states and project arrival and response times. This presents several opportunities to design new visual representations that can be used by the crew and mission control to coordinate and engage in remote, distributed macrocognitive work.
Uncertainty visualization
Following on from the above insight, when a stream of telemetry is deemed as unusual or problematic by a ground controller, it may represent the system status from several minutes in the past. It would be highly valuable to help the ground controllers understand the possibilities of how this stream of telemetry could be developing in the time since the data was sent. Being able to visualize future projections requires careful attention to communicating uncertainty clearly to the practitioners. Because of the high possibility for incorrect projections, visualizations that do not convey uncertainty effectively will likely become untrusted and unused.
Attention and expectation management
During an anomaly event in a potential future deep space mission, the function allocation across ground support, crew members, and automation will dramatically change from the current operation of the ISS. When an anomaly has been identified, there will likely be a very different procedure for the crews and the ground to decide who should be taking care of which part. Prioritizations of tasks are also going to change dramatically due to communication delays. The crew and the ground support both need to clearly know what is being handed over, what is the top action item on their own list as well as on their counterparts’. The demands and constraints on macrocognitive work will be significantly different also, and directing attention via visual displays and other artifacts will be critically important. This poses great challenge for future visualization designers to properly coordinate, allocate, and manage attention and expectations of both the crew and the ground.
Explainability
Limited reliance on ground support will necessitate increased reliance on automation. Explainability will become increasingly important, as trusting automated suggestions or decisions is critical for effective human-AI teaming. Although explainable AI is a hot topic, strategies and principles for avoiding automation surprises and ironies (Bainbridge, 1983; Sarter et al., 1997) are not well established. This presents opportunities to investigate how visualizations and other interactive artifacts can enhance understanding and trust in the context of human-AI teaming.
Conclusion
Communication delays in future deep space missions will be inevitable and will significantly change the nature of mission support and distributed macrocognitive work. Responding effectively to anomalies is already a major challenge, with large teams of experts available at all times. In missions beyond low-earth orbit, where ground support will not be available in real-time, anomaly response will be exceedingly challenging. Understanding the nature of macrocognitive work in Mission Control currently will aid in the design of artifacts for future deep space missions. A cognitive task analysis was done targeting ISS mission control operations. Four anomalous response scenarios were collected from the interviews with participants. The scenarios were then analyzed and recast into future deep space mission contexts. Doing this type of recasting can aid in identifying design challenges and opportunities. We suggest 4 areas of design challenges and opportunities based on our findings from the interview data analysis, focusing on visual displays and opportunities for creating new visual representations.
