A Framework for Relating Crew Member Performance Measures to Spacecraft Design and Operations

Introduction

Crew members play a critical role in human space missions; they are responsible for executing complex tasks that are difficult to automate, maintaining and repairing onboard hardware, and taking manual control of automated systems in emergency scenarios. As spacecraft systems become more complex in their design and operations, and missions become longer in duration and extend further from Earth, the need for ensuring human and vehicle integration plays an increasingly important role in the design process.

Processes for integrating humans into complex systems have been described with various terms ranging from human systems integration (HSI) to human-centered design.^1,2 With human spaceflight, such a process has been designated as human-rating of the spacecraft. The term “human-rating” (also known as “man-rating”) of vehicles first appeared in the early suborbital X-Plane program and later progressed through the Apollo and Space Shuttle eras. The basic intent of human-rating vehicles was to ensure the spacecraft could be safely operated within the range of the crew member's capabilities and limitations. As technology and terminology evolved, the National Aeronautics and Space Administration (NASA) captured a more inclusive definition to describe a human-rated vehicle as one that accommodates human needs, effectively utilizes human capabilities, controls hazards, and manages safety risk associated with human spaceflight (…). ³

Based on this definition, certifying a human-rated vehicle suggests that all three components (vehicle safety, accommodation, and utilization) should be quantifiable. While potential safety risks and hazards can be quantified using probabilistic risk assessment techniques, a similar, standardized comprehensive technique has not been identified for characterizing and quantifying crew accommodation or utilization. The typical method for incorporating astronauts into a spacecraft has historically been governed by design requirements found in documents such as NASA-STD 3001 (Volume 1, Crew Health; Volume 2, Human Factors and Habitability) and NASA/SP 2010-3407 Human Integration Design Handbook. However, using requirement verification as an assessment method has limited ability for predicting design impacts on the crew. Often, requirements only dictate functional needs, which can be static with limited flexibility to changes in the mission objectives. Additionally, tracing downstream impacts of specific requirements can become a convoluted and impractical process.

Other indirect means for assessing crew accommodation and utilization can be considered using various measurements of crew performance. The assumption is that if the crew is performing at a high level overall, they must be well accommodated and efficiently utilized by the vehicle design and operations. However, measuring overall crew performance is challenging and not yet a standardized process.

While a few models have been successfully utilized to quantify and predict human performance, such as the Army's Improved Performance Research Integration Tool ⁴ or NASA's Man–Machine Integration Design and Analysis System, ⁵ they lack the ability to aggregate the information into one comprehensive estimate of crew performance that can be compared with design tool counterparts such as the use of equivalent system mass. ⁶ Other approaches such as tracking human error probability rates^7–9 provide powerful methods for evaluating the safety-related performance aspects of the design, but lack data measures for human error rates.

Attempts have been made to explore the idea of an all-inclusive crew performance metric, including Cohen and Houk's ¹⁰ work developing a crew productivity figure of merit and Crocker's ¹¹ work in defining flight operability of spacecraft. Cohen and Houk's ¹⁰ approach relies upon Maslow's hierarchy of human needs, mapping those needs to NASA's HSI requirements. This approach allowed them to identify missing requirements represented by Maslow's needs. However, the method was only applied to the Altair Lunar Lander design and not extended further. Meanwhile, Crocker focused on crew operations and defined the term, spacecraft operability, using six themes: simplicity, margin, robustness, flexibility, situational awareness, and control. Leveraging a similar philosophy to the Cooper–Harper Rating Scale for assessing aircraft handling qualities, three question-based metrics were used to assess spacecraft operability across the six themes: Can the mission be accomplished? Can it be accomplished within tolerable limits? Can it be accomplished within normal limits? A qualitative rating scale was then offered to help designers assign an operability score for the system. Operability is an important measure for usability of the system, but this assessment assumes a healthy and fully functional crew member without adjusting for changes due to fatigue and modified physiology due to the spaceflight environment.

In all cases, these methodologies address relevant aspects of crew performance, but are constrained by either subjective measures with no clear path toward rigorous quantification or lack of ability to provide comprehensive information to the designer about how well the astronauts are accommodated and/or utilized. Our study addresses this challenge by proposing a foundational framework that links spacecraft design impacts to crew accommodation and utilization outcomes. The development of this framework is motivated by the need to provide a clear path toward development of a comprehensive quantitative model through rigorous empirical data collection. Specifically, this article focuses on the development of an input–output system that couples spacecraft design choices to measurable performance output variables that can be obtained with currently accepted methods.

Methodology for Framework Development

To create a framework for connecting crew performance to spacecraft design, a systematic approach was developed using an inquiry-based technique often applied in design thinking processes. Such an approach utilizes a series of seven high-level questions aggregated from a number of project management methodologies to define and constrain the framework's purpose and scope.

Defining and Characterizing Crew Accommodation and Utilization

A broad definition for crew accommodation and utilization is leveraged here from the study by Klaus et al., ¹² where accommodation is defined as what the vehicle provides to support the crew and utilization as what the crew can operate to support the mission. Extreme boundaries of these values can be defined to indicate their applicability here. For a high value of crew accommodation, it is expected that the crew members are alive, healthy, and happy, while a low value indicates that the crew may be deteriorating in health and psychological state, heading toward potential death. For crew utilization, a high value suggests the crew members are effectively conducting the mission with minimal issues, while a low value indicates under- or overutilization of the crew, leading to errors and/or detrimental crew impacts.

Accommodation can be further decomposed into measures of the crew's health and well-being. For this work, crew health and happiness are categorized into three components of well-being: (1) physiological, (2) cognitive, and (3) psychological. Each of these components is envisioned as a measurable resource reserve that the crew member has for maintaining life, health, and happiness (or mental well-being). The assumption is that a healthy and happy crew member starts with 100% of these resources, and throughout the mission, the crew member's quantified status can be affected by numerous design and operational factors that deplete or replenish these resources. The amount of resource remaining in each of these three components indicates the amount of reserve the crew has left at a given point in time. If these resource reserves fall to zero, it would indicate the loss of a crew member. Since well-being is not a straightforward metric, the components were decomposed to more basic relevant elements, referred to as human capabilities, as listed in Table 1.

For quantitatively assessing crew utilization, two types of metrics are used as indicators: (1) task performance and (2) design efficiency. Task performance can be measured with an objective set of metrics, including the accuracy, speed, and total number of mission tasks accomplished. The second type of metric is design efficiency, which is meant to indicate the quality of the design of a spacecraft or its specific interfaces with the astronaut, and is inherently more subjective. The more inefficient the design, the more energy or resources a crew member must expend to accomplish the task. These resources can be thought of as the same human capabilities that are used to measure a crew member's well-being and level of accommodation in the spacecraft. Therefore, a good overall spacecraft design would provide a neutral or net positive change in human capabilities, while a poorly designed spacecraft would cause the capabilities to degrade over time.

This concept of using human capabilities as the currency for measuring crew accommodation and utilization is the backbone of this framework, which are collectively captured in the physiological, cognitive, and psychological resource categories.

Framework Outputs

With this approach, spacecraft design choices can be linked to their influence on crew accommodation and utilization through the status of the crew member's capabilities, as measured by individual quantifiable elements. A subset of spacecraft design choices was then selected from Mindock's ⁷ list of 172 performance-shaping factors (PSFs). While Mindock's list of PSFs encompasses a wide range of influences such as social factors, mission control, team dynamics, and management structure, this work is currently focused on reviewing design impacts on performance. Each of these design choices was analyzed with expert guidance and literature review for how it could affect specific human capabilities. The list of impacts is by no means comprehensive, but provides a qualitative description for how capabilities might be impacted. However, given that no comprehensive database exists with measured human capabilities under mission conditions, this linkage can only be presented in a notional manner at this time. The results of this effort are shown in Tables 2–4. Each table represents a different aspect of spacecraft design, including the physical environment, architecture of the habitat, and user interfaces.

Parameters shown in these tables are meant to establish a foundation for our proposed model-based framework. Quantitative measures or mathematical modeling can then be integrated into this framework through different strategies, including literature aggregation and/or experimental validation. However, even without clearly defined mathematical relationships, the framework can still be utilized to provide insight to the spacecraft designer as to how design choices can alter crew performance. For example, a mission designer can qualitatively assess which design choices affect which capabilities. Additional insight can be gained by estimating the severity of those impacts by the design. By linking a given design choice to the specific capability impacted, the designer can include this as part of the trade space along with the typical mass, power, volume, and system complexity parameters.

This is just one example of the application that can be derived using the tables from this work to understand the impacts of different design choices and their correlation with specific crew performance metrics. Adding quantitative data and better-defined relationships will continue to increase the fidelity and usefulness of the model.

Conclusions

This work has demonstrated a candidate framework for analyzing how well spacecraft design choices affect crew capabilities. Terms and their relationships were defined and analyzed in an effort to identify metrics that can assist designers in evaluating spacecraft options in terms of crew accommodation and utilization. In characterizing these terms, human capabilities were identified as the currency of exchange for monitoring and tracking changes to physiological, cognitive, and psychological status levels used as quantifiable metrics for assessing overall accommodation or utilization effectiveness. This highlights the underlying philosophy for this framework, in which changes to human capabilities are an indication of impacts from the design and operations of the spacecraft, including changes stemming from the space environment as well. In particular, spacecraft that have been poorly designed to accommodate or utilize the crew would be reflected by reduced measures in the crew's physical, cognitive, and psychological well-being, while an optimally designed system should maintain, if not improve, the crew's well-being. To highlight these design impacts, tabularized data were presented identifying specific spacecraft design choices and their expected resultant impacts on the crew's capability. While the current framework lacks quantitative outputs at this time, the ability to develop future mathematical modeling and integrate empirical data has been considered as an integral next step in this process.

Ultimately, this framework provides a systematic methodology for restructuring design concepts around how to quantify crew accommodation and utilization in a spacecraft. The focus here was to use a human-centric approach where designers can account for impacts to the crew's performance as part of the decision-making process. This framework does not yet offer completeness or precision, rather it is meant to provide a straightforward logical method to help designers, managers, and operators better identify and assess the overall system design impacts on the crew to ensure mission safety and success.