The Difficulty to Break a Relational Complexity Network Can Predict Air Traffic Controllers’ Mental Workload and Performance in Conflict Resolution

Abstract

Objective

To test the network disentangling model for explaining air traffic controllers’ (ATCos) conflict resolution performance. The network rigidity index (NRI), and the steps to break the relational complexity network following a central-available-node-first rule, was hypothesized to explain the overall task demand, whereas marginal-effort-decrease rule was expected to explain the actual operational outcome.

Background

Understanding the conflict resolution process of ATCos is important for aviation safety and efficiency. However, linear models are insufficient. We proposed a new model that ATCos behavior can be largely considered as a process to break the relational complexity network, in which nodes represent the aircraft while links represent the cognitive complexity to understand the aircraft dyad relationship.

Method

Twenty-one professional ATCos completed 27 conflict resolution scenarios that varied in the NRI and other control variables. Multilevel regression analyses were performed to understand the influence of the NRI on the number of interventions, mental workload, and unresolved rate. A cross-validation was performed to evaluate the predictive power of the model.

Results

NRI influenced ATCos intervention number in a curvilinear manner, which further leads to ATCo’s mental workload. The deviance between the number of interventions and the NRI was strongly linked with unresolved rate. Cross-validation suggests that the models predictions are robust.

Conclusion

The network disentangling model provides a useful theory-driven way to explain controllers’ conflict resolution workload and other important performance outcomes such as intervention probability.

Application

The proposed model can potentially be used for workload management, sector design, and intelligent decision support tool development.

Keywords

relational complexity network workload air traffic controllers conflict resolution network disentangling error operation

Introduction

Understanding the factors that drive mental workload and performance of air traffic controllers (ATCos) is vital for aviation safety and efficiency, especially in an era of an unprecedented increase in air traffic volume (International Civil Aviation Organization [ICAO], 2013). However, existing models are insufficient for understanding ATCos workload and performance in a complex environment with specific operational constraints (e.g., the presence of a hard-to-maneuver aircraft) (Gianazza & Guittet, 2006; Laudeman, Shelden, Branstrom, & Brasil, 1998; Loft, Sanderson, Neal, & Mooij, 2007; Mogford, Guttman, Morrow, & Kopardekar, 1995; Prandini, Piroddi, Puechmorel, & Brazdilova, 2011). This study aims to contribute by proposing a novel computational model that depicts ATCos conflict resolving behavior as a function of a network disentangling process.

The main goal for ATCos is to maintain minimum separation between aircrafts (ICAO, 2005). ATCos must monitor air traffic flow to detect potential conflicts. When necessary, they instruct pilots to change altitude, direction, and/or velocity to resolve conflicts. Thus, relational information among aircrafts is important to decision making. Relational complexity (RC) is defined as the number of different parameters (speed, altitude, headings, etc.) that an ATCo needs to consider simultaneously in any dyad-based conflict detection task (Boag, Neal, Loft, & Halford, 2006). The RC is very low when two-level flight aircraft are on separate levels (RC = 0). Higher RC arises when two flights fly on converging paths and/or cross levels. Previous studies have found that the linearly aggregated total RC can have a strong impact on ATCo mental workload in situations with few aircrafts (Boag et al., 2006). However, the explanatory power of the RC index is attenuated when more aircrafts are taken into account (Zhang, Yang, & Wu, 2015), because a nonlinear interaction might exist (Kuk, Arnold, & Ritter, 1999; Loft, Bolland, Humphreys, & Neal, 2009; Loft et al., 2007).

Graph theory can help solve this problem, as a relational complexity network (RCN) can be constructed by combining all of the relational information (Zhang & Du, 2015; Zhang et al., 2015). In the RCN, the aircrafts are represented as nodes, whereas the RCs are represented as links. We previously found that, after controlling for the effect of the total RC and control variables, the centrality of the RCN (a measure of network structure) was an additional predictor of a lower workload and fewer errors (Zhang et al., 2015). However, it is unclear why the network structure has a significant influence on ATCo workload and whether this model can be applied to more generalized situations.

In this paper, we propose a more generalized modeling approach that follows two basic ideas. First, we conceptualize ATCos as active problem solvers who will fully utilize the pattern of the RCN, which provides both shortcuts and constraints for conflict resolution decision making. Second, we conceptualize ATCos as ecologically efficient agents, who, given their cognitive capabilities, will adopt certain strategies to manage their workload while achieving an acceptable performance goal. The model can be used to better understand factors that drive human performance outcomes beyond conflict resolution workload (such as unresolved rates, task completion times, number of interventions [NOI], etc.), and under varied conditions (e.g., when there are certain operational constraints).

The general strategy for ATCos for detecting and resolving conflicts is to make early interventions to eliminate perceivable potential conflicts between aircrafts (i.e., the RC > 0 situations). Ideally, ATCos should only intervene (by giving orders to the pilots of certain aircrafts) when actual conflicts are detected (vertical separation < 1,000 feet and horizontal separation < 5 miles). However, many studies have suggested that this is not the case. ATCos often intervene without precisely assessing the conflict potential, even when the safety margin is large (Hopkin, 1995; Loft et al., 2009; Seamster, Redding, Cannon, Ryder, & Purcell, 1993). This is because making accurate assessments of potential conflicts requires significant ATCo cognitive capacity, and non-intervened pairs in potential conflict must be monitored, which creates substantial ongoing task demands (Metzger & Parasuraman, 2005). As it is advantageous to preserve cognitive resources to cope with future demands, making interventions when facing any potential conflicts (the RC > 0 conditions) is often a preferred option for experienced ATCos. In this way, most of the RC > 0 pairs will undergo intervention to become RC = 0 conditions by changing certain properties of a node (e.g., changing the aircraft’s altitude to establish vertical separation). This process involves removing a link between two connected nodes. Collectively, we propose that ATCo behavior reflects their attempt to disentangle all links in an RCN by altering node properties.

In this process, two issues are of the highest importance, namely, the sequence of node selection and the stop rule. Obviously, the node selection sequence is important, as different sequences can lead to different consequences. A network can be disentangled in few intervention steps when following an effective strategy. For example, only one step is required to unlock the entire network by changing the property of a central node in a centralized star-like network. Figure 1 provides an illustration. In the upper left, the three-dimensional (3D) visualization of the air traffic pattern is presented. The red aircraft is ascending, and its trajectory is in conflict with the other three aircrafts. In the middle left panel, the two-dimensional (2D) visualization (what the ATCos actually see) of the same situation is shown. A1 is ascending from 24,000 feet to 28,000 feet high, while the other three aircrafts are cruising (i.e., on level flight) at 25,000; 26,000; and 27,000 feet, respectively. In the bottom left panel, the RCN topology of this situation (a centralized network) is presented. On the right, an intervention is made to change the flight plan of A1 (preventing it from ascending), which therefore leads to a fully broken RCN network using a one-step operation.

Figure 1.

Air traffic pattern and RCN before and after intervention. RCN = relational complexity network.

However, ATCos can also use less effective strategies when facing the same scenario if they first choose to intervene with the nodes at the periphery of the network. Based on previous findings (Zhang, Ren, & Wu, 2014; Zhang et al., 2015), it is reasonable to hypothesize that ATCos generally use the most effective method to break the network (i.e., larger-degree nodes; aircrafts with more RC > 0 links), and intervention will occur earlier. This also likely reflects that these nodes with higher degrees are more likely to be noticed during a visual search for potential conflicting pairs, so they are given higher priority (Zhang et al., 2014). In addition, changing the property of a central node can sharply reduce cognitive demands, as there is no need to assess the risks of all of the other relationships (the once-for-all benefit, Zhang et al., 2015). This strategy is the fastest way to remove all the links in the network so that the overall NOI can be minimized (Callaway, Newman, Strogatz, & Watts, 2000). With this larger-degree-node-first strategy, it is easier to understand why star-like networks can result in lower conflict resolution workload. That is, the best solution for breaking a centralized network is simpler than breaking a decentralized network even if they have the same numbers of nodes and links.

This framework can be further generalized by incorporating a common yet important operational constraint—the controllability of an aircraft. In practice, aircrafts are not always under full control, for example, if an aircraft is on a military mission or is running low on fuel (Hopkin, 1995; ICAO, 2005). As ATCos cannot change the trajectories of these aircrafts, they must focus on other aircrafts that have potential conflicts with the uncontrollable one. This can be handled in our model by simply treating those nodes as unavailable in the disentangling processes so that ATCos select other nodes when intervening. Therefore, the rule of node selection can be reformulated as a central-available-node-first rule: a node will be selected and intervened if it is both available and has the most links.

Based on this node selection rule, we can illustrate the disentangling process and its impact on performance outcomes. We introduce the network rigidity index (NRI) and define it as the NOI required by the ATCo to remove all RC > 0 links in the RCN following the central-available-node-first rule. In other words, the NRI describes the optimal NOI needed to achieve the lowest risk level. We postulate that the NRI serves as a more precise way of quantifying task demands and that it will be an effective predictor of the NOI, which will in turn predict ATCo conflict resolution workload and task completion times. We provide examples of the NRI and its relationship with the network structure and the position of uncontrollable aircraft (UA) in Figure 2: when an UA is at the margin of the network, the disentangling process (as indicated by the NRI in Figure 3) will not be influenced by its presence (the left and the middle column); however, if such an aircraft is at the center of the network, it significantly increases the NRI, indicating that ATCos must make a greater effort to deal with the situation (the right column). More specifically, network rigidity can serve as a more useful and more generalizable concept for understanding the influence of task complexity on ATCo workload and even actual performance such as the NOI.

Figure 2.

Uncontrollable aircraft at different network positions. NR = network rigidity; UA = uncontrollable aircraft.

Figure 3.

The steps (NRI) to break networks with medium centralization (the middle in Figure 2). NRI = network rigidity index; UA = uncontrollable aircraft.

Such a model is useful for accounting for the influence of UA on ATCo performance. Although the negative influence of UA is recognized, their effects on ATCo performance are far from clear (Hopkin, 1995; Loft et al., 2007; Moray, 1997). Although UAs are considered a major workload driver in formal regulations and expert opinions, they have not been found to have effects on performance in many empirical investigations (Laudeman et al., 1998; Mogford et al., 1995). We believe that our model may be able to explain this discrepancy between expert opinion and prior empirical data, because our model predicts that the effect of UA should depend on the network structure and the aircraft position in that network.

In delineating the stop rule, it is important to note that human cognition often deviates from what is optimal. Numerous studies have indicated that humans use bounded rationality and satisficing when making decisions (Kahneman, Slovic, & Tversky, 1982; Klein, 1999; Simon, 1979, 1990; Todd & Gigerenzer, 2007). When only considering the relationship between task demands and NOI in the ATC domain, we suggest that a marginal-effort-decrease pattern can best describe the relationship between the NRI (the optimal operation in theory) and the NOI. This is because ATCos are more likely to establish workload homeostasis rather than always minimizing future risks (Desmond & Hoyes, 1996; Sperandio, 1978). By taking an ecological rationality perspective (Todd & Gigerenzer, 2007), such an approach can help ATCos maintain a certain level of cognitive surplus so that they can be ready for any emergencies in the uncertain future (also see Metzger & Parasuraman, 2005). As a result, if the task demand (as suggested by the NRI) is too high, ATCos may only attend to the most prominent relationships and refrain from making too many interventions. In this way, this marginal-effort-decrease phenomenon will result in a curvilinear relationship between the NRI and AIA: the NOI might be higher than NRI when the overall task-load is very low (as we mentioned before, ATCos are overcautious when making interventions), then it may increase linearly as the NRI increases at first, but its increasing rate will reduce when the NRI reaches a certain level (the turning point). After this turning point, there will be a gap between the NRI and AIA, which indicates the degree to which the actual NOI are deviant from optimal NOI. Figure 4 illustrates such a relationship. Based on this, we can make the following hypotheses:

Hypothesis 1: There will be a curvilinear relationship between NRI and the NOI, mental workload, and task completion time.

Hypothesis 2: The NOI further mediates the influence of NRI on mental workload and task completion time.

Hypothesis 3: The deviation between the NRI and NOI will be linked to reduced performance (a higher unresolved rate).

Figure 4.

The hypothetical curvilinear relationship between NRI and the number of interventions. NRI = network rigidity index; NOI = number of interventions.

Taken together, we propose that a network-breaking process can provide a new way to understand the dynamicity of ATCo behavior. To provide empirical evidence for our network disentangling model, we developed scenarios in which number of aircraft and total RCs were held constant, whereas the NRI index and other important workload-driving factors were varied, including the number of conflicting pairs, the average number of minimum relative vectors, and so on. We used professional controllers to test whether the network disentangling model can explain the observed number of ATCo interventions, workload measures, and unresolved rates after controlling for other potentially confounding variables.

Method

Participants

Twenty-one professional en-route ATCos were recruited from a Provincial ATC center in Northern China. They were paid 200 yuan (approximately 35 U.S. dollars) for completing this and two other experiments. Their ages ranged from 23 to 51 years (M = 30.8, SD = 8.2), and their work experience ranged from 1 to 26 years (M = 6.97, SD = 7.77). All participants were men. This research complied with the tenets of the Declaration of Helsinki and was approved by the Institutional Review Board at the Institute of Psychology at the Chinese Academy of Sciences. Informed consent was obtained from each participant.

Conflict Detection and Resolution Task

Participants were asked to perform a series of conflict detection and resolution tasks using a medium fidelity air traffic control simulation platform (see Figure 5). At the beginning of each scenario, 8 aircrafts would appear within a boundary of a 220 nm (nautical miles) × 220 nm in the en-route sector. Each aircraft had an information block containing the call sign, aircraft type, current and targeted flight levels (in hundred feet), altitude change (↑ indicating climbing and ↓ descending), heading (in degrees), and ground speed (in nautical miles/hr). This information was updated every second, and the climbing/descent rate was set at 1,000 feet/min.

Figure 5.

Simulator interface.

The participants were asked to detect and resolve all potential conflicts. They were able to use supporting tools, including a scale maker, range bearing line, and a distance/time calculation to help them make assessments. They could change the flight parameters (altitude, speed, and direction) by pressing the corresponding figures within the flight information window on the screen. However, aircrafts with the label Emergency could not be controlled. There was no time limit imposed, such that the participants were told to press a “Finish Current Scenario” button after they had resolved all potential conflicts. The task completion time was measured as the time from the start of each scenario to the time at which “Finish” was pressed. The participants rated their workload using the NASA-TLX scale after they had pressed the Finish button (Hart & Staveland, 1988). The unresolved rate was evaluated using the ratio of unresolved conflicts to all presented conflicts.

Control Variables

Centralization

Centralization describes the degree to which the network is organized around one focal node. It has been found that central networks are easier for controllers to handle (Zhang et al., 2015). This index is calculated using the following formula:

C = \frac{\sum_{i = 1}^{n} (C_{m a x} - C_{i})}{\max [\sum_{i = 1}^{n} (C_{m a x} - C_{i})]}

(1)

In formula (1), C_i refers to the centrality degree (the number of linked edges for any given node i), and C_max is the greatest centrality degree among all of the nodes in a given network.

Minimum separation (MS)

This variable describes the minimum possible distance between two aircrafts in the future (Boag et al., 2006; Vuckovic, Sanderson, Neal, Gaukrodger, & Wong, 2013). Rather than using the Euclidean distance, this variable is weighted using horizontal and vertical separation standards:

MS = \min_{t \geq 0} \sqrt{\begin{array}{l} {(\frac{lateral s eparation at t_{i} (nm)}{5 n m})}^{2} \\ + {(\frac{vertical separation at t_{i} (ft)}{1, 000 ft})}^{2} \end{array}}

As previous research has observed, this variable should be negatively associated with controllers’ workload ratings and task completion times.

Violation count

This variable is the number of aircraft pairs that would violate the minimum separation standard in a certain period of time in the future given their current flight plans (Boag et al., 2006; Zhang et al., 2015). Generally, more potential conflicts mean more required ATCo interventions, which results in higher workload.

Scenarios

Twenty-seven scenarios were designed in accordance with Figure 2. First, we created 9 scenarios for the non-UA condition following the method described in the study by Zhang et al. (2015) in which each network structure (low, medium, and high centralization) had 3 scenarios with varying features in other complexity dimensions. We consulted experienced controllers (these same controllers did not participate in the actual study) to ensure that our scenarios were representative of ATC and therefore had the required ecological validity. Next, the UA-margin condition was created by rotating the non-UA scenarios 90 degree clockwise and changing all aircraft call signs. More specifically, the call sign of one plane at the margin of the RCN network was changed from an ordinary one (e.g., MU1881) to “Emergency.” Finally, the UA-center condition was created by rotating the non-UA scenarios 180 degree clockwise and changing the flight call signs again. This time, the call sign of the plane at the center of the RCN network was changed to “Emergency.”

Procedure

Upon arrival, the participants were asked to sit by a computer with a 23-inch-wide LED monitor. After they became familiar with the simulator, they completed three practice scenarios. During the task, each participant completed 27 formal and 5 filler scenarios in a random manner with a 2-min break after every 8 scenarios.

Results

Preliminary Analyses

On average, the participants successfully resolved 84.0% (SD = 28.2%) of the conflicts, spent 81.2 s (SD = 56.9) completing each scenario, and reported a medium workload level (M = 30.5, SD = 13.6) on the NASA-TLX scale (0–100). As predicted by our theoretical approach, more than 94% RC > 0 links were removed in our study by the controllers.

It might come as a surprise to readers at first glance that professional ATCos “missed” 16% of the conflicts. It is crucial to note, however, that the projected time to conflict for many of these pairs of aircrafts was greater than 10 min, and in such cases, some ATCos prefer to monitor aircraft pairs before intervening due to likely uncertainties in the environment (Loft et al., 2009). If the participants spent more time monitoring each scenario, the resolve rate might have significantly improved.

By averaging the data at the scenario level, we first plotted the relationship between the NRI and the NOI (see Figure 6). The curvilinear model had a better model fit (R² = .92) than the linear model (R² = .76). We then investigated the relationship between the deviance (the difference between NRI and NOI) and the unresolved rate (see Figure 7). Deviance was linked to the unresolved rate (R² = .72). These findings provided initial evidence that our proposed new model was effective.

Figure 6.

The relationship between network rigidity index and the number of interventions: Testing the marginal-effort-decrease principle. NR = network rigidity; NOI = number of interventions.

Figure 7.

The relationship between the deviation and the unresolved rate.

Although this averaging and plotting method can provide useful information, it may inflate the fit index (R²), as it removes individual differences from the analysis. As a result, we made a more precise evaluation by using the multilevel regression method.

Multilevel Regression Analyses

We followed the approach used by Zhang et al. (2015) to conduct a series of multilevel regression analyses (Raudenbush & Bryk, 2002) using the HLM 6.08 package. The null models suggested that 7.7%, 18.4%, 61.5%, and 12.0% variances were present at the between-individual level for the NOI, task completion time, perceived workload, and unresolved rate, respectively. Thus, it was appropriate to conduct a multilevel analyses.

We used four nested regression models (see Table 1). First, using NOI as the dependent variable, Model 1-a suggested that controllers would intervene with more steps when the minimum separation was lower (B = −0.06, p < .05), the network was less central (B = −1.77, p < .001), and the UA was at the center of the network (B = 1.46, p < .001), which together explained approximately 43.2% of the total proportional reduction in variance (PRV, the so-called pseudo R², and an indicator of variance explained in multilevel modeling). Next, we incorporated the NRI and NRI² into Model 1-b and found that both terms had significant effects (B = 2.03, p < .001, and B = −0.20, p < .01) and increased the PRV by 11.5%.

Table 1:

Hierarchical Linear Modeling Analyses on the Key Variables in Controllers’ Conflict Resolution

Parameters	Number of Interventions		Mental Workload (NASA-TLX: 0–100)			Task Completion Time (Seconds)			Unresolved Rate (%)
Parameters	M1-a	M1-b	M2-a	M2-b	M2-c	M3-a	M3-b	M3-c	M4-a	M4-b
Intercept	2.34 (0.07)***	−1.02 (0.74)	27.8 (2.5)***	14.2 (4.5)**	24.4 (5.2)***	63.3 (3.2)***	−67.8 (26.3)*	23.4 (24.1)	7.31 (3.2)*	14.3 (3.1)***
1. Violation counts	−0.03 (0.03)	−0.01 (0.03)	−0.58 (0.32)	−0.44 (0.30)	−0.41 (0.28)	3.40 (1.1)**	4.53 (1.1)**	4.81 (0.73)***	−0.06 (0.62)	−1.11 (0.51)*
2. Minimum separation	−.06 (0.02)*	−0.09 (0.02)***	−0.07 (0.19)	−0.27 (0.16)	−0.03 (0.17)	−0.47 (0.96)	−2.08 (0.93)	0.07 (0.74)	1.06 (0.52)*	0.74 (0.48)
3. Centralization	−1.77 (0.11)***	0.48 (0.57)	−2.43 (0.83)**	−6.83 (3.6)	5.54 (3.4)	−4.48 (4.0)	84.1 (21.0)*	72.8 (18.1)***	2.55 (3.0)	−0.67 (2.6)
4. UA-center	1.46 (0.20)***	−0.19 (0.20)	6.61 (2.0)**	3.60 (2.3)	4.12 (2.2)	60.3 (9.0)***	16.2 (9.9)	20.8 (8.4)*	21.8 (3.0)***	0.60 (3.4)
5. UA-margin	0.00 (0.05)	0.00 (0.05)	0.84 (0.83)	0.84 (0.83)	0.83 (0.82)	0.60 (1.6)	0.67 (1.6)	0.62 (1.8)	−1.28 (1.8)	−1.24 (1.8)
6. NR		2.03 (0.52)***		8.79 (3.2)**	3.28 (3.2)		82.4 (18.4)***	34.2 (16.1)*
7. NR²		−0.20 (0.07)**		−0.99 (0.40)*	−0.46 (0.39)		−8.71 (2.3)***	−4.07 (1.9)*
8. Number of intervention					2.66 (0.60)***			23.7 (3.3)***
9. Deviation										9.49 (1.0)***
PRV (Level 1)	43.2%	54.7%	12.5%	14.1%	23.3%	24.0%	36.0%	55.3%	20.2%	37.8%
△PRV(Level 1)	43.2%	11.5%	12.5%	1.6%	9.1%	24.0%	12.0%	19.3%	20.2%	17.6%

Note. TLX = Task Load Index; UA = uncontrollable aircraft; NR = network rigidity; PRV = proportional reduction in variance.

p < .05. **p < .01. ***p < .001 (numbers in parentheses are robust standard errors).

Models 2 and 3 were established using two indicators of workload (perceived workload and task completion time) as dependent variables. In Model 2-a, a larger perceived mental workload was related to lower centralization (B = 2.43, p < .01) and UA being in the center of the network (B = 6.61, p < .01); the coefficients of NRI and NRI² were significant (B = 8.79, p < .01; B = −.99, p < .5) when they were added into Model 2-b and increased the PRV by 1.6%. Moreover, when the NOI was added into Model 2-c, it was found to be a significant factor in the model (B = 2.66, p < .05), increasing the PRV by 9.1%.

In Model 3-a, a longer task completion time was related to higher violation counts (B = 3.40, p < .01) and a UA-center arrangement (B = 60.3, p < .001); the coefficients of NRI and NRI² were significant (B = 82.4, p < .001; and B = −8.71, p < .001) in Model 3-b and explained an additional 12.0% of the variance. Moreover, when the NOI was added to Model 3-c, it became a significant factor (B = 23.7, p < .001), and the PRV increased by 19.3%.

Finally, in Model 4-a, we found that a higher unresolved rate was related to a larger minimum separation (B = 1.06, p < .05) and UA-center arrangement (B = 21.8, p < .001); in Model 4-b, the coefficient of deviance between NRI and the NOI was significant (B = 9.49, p < .001), and it explained an additional 17.6% of the variance.

Cross Validation

We conducted a cross-validation to examine whether our models can potentially be used to make predictions. To do so, we first collapsed the data at the scenario level, thus creating 27 data points. Next, we adopted a leave-one-out procedure. Specifically, we trained our linear regression models using 26 data points and used the estimated parameters to make predictions regarding the dependent variables in the unused scenario. We repeated this procedure 27 times. We used two measures to evaluate the performance of the models. The first is the root mean squared error (RMSE), which was calculated by taking the square root of the average of the squared differences between forecasts and observed values; RMSE is a common, sensitive, and valid accuracy measure that can make internal comparisons over models on the same dependent variables. The second is the mean absolute percentage error (MAPE), which was calculated as the mean of the absolute values of the percentage errors in the forecasts; MAPE is a highly reliable and valid performance measure that has a unique advantage: it is unit free. As a result, we can form an overall picture of the general performance of our models over the four different dependent variables. Whereas there is no consensus on adequate cutoff values for RMSE and MAPE, MAPE < 10% is considered as an indicator of acceptable predictions in economics and other areas (Armstrong, 2001).

We then made extensive comparisons across alternative models (see Table 2). The major findings were as follows. First, the comprehensive models (all variables were used in the model) had the best performance (i.e., M8 for NOI, M11 for mental workload and task completion time, M14 for unresolved rate). Second, simply using NR and NR² (M2) can perform equally well or even outperform the models based on previous theories (the models containing all control variables). Third, the performance of our models was quite good across all dependent variables (the MAPEs were below 10% for NOI, mental workload, and task completion time; only for resolved rate it was about 40%, which might be an inflation caused by the fact that the unsolved rate was very close to zero (Armstrong, 2001).

Table 2:

Cross-Validation Analysis of the Model Performance

Dependent Variables	Number of Interventions			Mental Workload (NASA-TLX: 0–100)			Task Completion Time (s)			Unresolved Rate (%)
Models: Independent Variables Included in Each Model	R ²	MAPE (%)	RSME	R ²	MAPE (%)	RSME	R ²	MAPE (%)	RSME	R ²	MAPE (%)	RSME
1. NR	.82	17.86	.478	.70	5.26	1.977	.83	11.06	13.237	.63	53.27	.077
2. NR + NR²	.91	11.36	.326	.70	5.23	1.976	.86	9.53	11.960	.73	48.10	.065
3. VC + MS	.02	49.33	1.104	.05	9.72	3.518	.02	34.35	31.647	.01	106.87	.125
4. VC+ MS +centralization + UA-center + UA-margin	.71	21.88	.314	.73	5.07	1.869	.72	17.25	17.058	.71	46.16	.067
5. M3 + NR	.82	17.83	.474	.75	4.78	1.789	.84	10.46	12.560
6. M3 + NR + NR²	.93	9.32	.298	.76	4.80	1.780	.87	10.08	11.570
7. M4 + NR	.92	9.97	.314	.79	4.39	1.643	.88	9.61	10.942
8. M4 + NR + NR²	.94	9.13	.283	.82	4.25	1.546	.91	8.39	9.714
9. NOI				.67	4.98	2.064	.66	15.82	18.655
10. NOI + NR + NR²				.74	4.93	1.845	.88	9.63	10.926
11. M4 + NOI + NR + NR2				.86	3.66	1.356	.93	7.05	8.506
12. Dev										.75	47.56	.063
13. M3 + Dev										.75	47.37	.063
14. M4 + Dev										.79	38.81	.058

Note. TLX = Task Load Index; RMSE = root mean squared error; MAPE = mean absolute percentage error; NR = network rigidity; VC = violation counts; MS = minimum separation; UA = uncontrollable aircraft; NOI = number of interventions.

Discussion

The present study proposed a new modeling approach to understand the performance of ATCos. Based on consideration of network theories regarding task demands placed on ATCos, we proposed that the task of resolving potential conflicts is analogous to removing all links in a RCN. More specifically, the cognitive difficulty as well as the overt operational behaviors to perform this task can be predicted by the NRI, which is defined as the number of ATCo interventions required to break the RC network following a simple central-available-node-first rule.

Consistent with previous studies (Zhang et al., 2014; Zhang et al., 2015), our findings first suggest that a centralized network is easier to handle (as shown in its link with fewer interventions and lower perceived workload). However, the positions of UA in the RCN also matter: although UA at the margin of the network did not significantly change conflict resolution workload and performance, UA at the center of the RCN resulted in a significant increase in the ATCos conflict resolution workload; their performance was worse when compared to the non-UA condition. These results indicate that although network structure is important, how ATCos interact with a specific structure is more important.

These findings can be explained by our network disentangling model. The UA-margin condition would not result in any change in performance because it does not alter the network-breaking process. However, the UA-center condition blocks the nodes with the largest degrees to be controlled, in other words, the aircraft with the greatest effects on control of the situation. As a result, the network-breaking dynamics change from a once-for-all process to a step-by-step process; thus, the NOI and mental workload should increase.

More direct evidence is provided by the finding that the NRI can effectively predict the NOI beyond controlled variables. The NRI alone explained 54% of the variance in the NOI. To the best of our knowledge, this study is the first to provide an effective framework for predicting this overt operational consequence of ATCos facing complex situations. Moreover, the NRI influenced workload (perceived workload and task completion times) through its influence on the NOI. Therefore, the network disentangling framework explained changes in ATCos’ conflict resolution workload, a well-researched but still little-understood construct.

Furthermore, the marginal-effort-decrease principle was corroborated, as the data showed that (1) the relationship between the NRI and the NOI can be best described as curvilinear and (2) the gap between the task demand (NRI) and NOI was the best predictor of errors (the deviance between the NRI and NOI alone can also explain 41.7% of the variance in the unresolved rate). This evidence suggests that, rather than adopting a minimum risk criterion in performing tasks, ATCos tend to adopt ecologically reasonable behavior by preventing their workload from exceeding a certain level (Loft et al., 2007).

Our holistic network representation of a relational structure seems to be useful for understanding the interplay between environmental information structures and human performance in the domain of ATC. We conceptualized ATCos as shrewd network breakers who utilize structural information to achieve a performance level that is safe enough while preventing their cognitive effort from exceeding limits. In this study, we successfully show that the combination of these two new perspectives, namely, global-level representation and ecological rational hypotheses, can effectively predict the actual performance of controllers, at least for the interesting range of scenarios studied here.

This study provides a theory-driven approach to model ATCos conflict resolution workload and performance in a nonlinear way, which is different from previous data-driven approaches (e.g., Averty, Collet, Dittmar, Athenes, & Vernet-Maury, 2004). With its computational nature, such a model can be potentially used to predict ATCos conflict resolution workload in a more dynamic manner. In addition, the current study helps resolve a discrepancy concerning the influence of UAs. Our study provided an important explanation: the effect of UA depends on the structure of RCN and its position within that network. In addition, as the network structure can influence ATCos decision-making processes and performance, one way to improve performance is to design task aids that can show the network positions of certain aircrafts.

Limitations and Future Research

Several limitations of the present study must be mentioned. First, our model is not deterministic. It is true that the NRI is constant for each specific task, and there will be a general trend between the NRI and the NOI (a curvilinear relationship), and the NOI can be used to predict consequences of ATCo behavior. However, both the NOI and workload can be influenced by many other exogenous factors. For example, fatigue and corresponding attention deficits can prevent ATCos from correctly detecting the RCN, so they may not choose the same number of nodes for interventions that are predicted by the NRI. Future studies may benefit from investigating when and how ATCos may deviate from disentangling processes. Second, our model focuses on the conflict detection and resolution process, while ATCos have other tasks to perform, including monitoring, communication, and data entry. It is likely that the strategies controllers use to detect and resolve conflicts are shaped by the nature and difficulty of these concurrent tasks, which would also effect the time to resolve conflicts.

It is important to note that our normative model attempts to characterize optimal ATCo conflict resolution performance. We would expect expert ATCos to be responsive to the parameters specified by our model, and thus our model has the utility to predict which factors will influence ATCo performance and conflict resolution workload in operational settings. However, our model may not necessarily reflect important characteristics of the cognitive processes that controllers use to make conflict resolution decisions, and therefore, they may be contexts in ATC in which our normative model will not be predictive. We emphasize here then that our model can be used to estimate task demands that influence ATCo cognition and performance, rather than be used to infer the cognitive processes underlying that performance.

Conclusion

The present research provides evidence concerning the usefulness of the network disentangling framework to quantify ATCo mental workload and conflict detection and resolution performance. This research extends previous RC network frameworks (Boag et al., 2006; Zhang et al., 2014; Zhang et al., 2015). Theoretically, this study indicates that it is useful to combine the holistic information structure and ecological rational strategy together to understand controllers’ behavior. Practically, the study provides a cognitively sound, computational, yet easy-to-understand indicator for predicting controllers’ behavior and workload.

Key Points

We proposed that the conflict resolution behavior of professional air traffic controllers can be modeled as a network disentangling process of the relational complexity network.

In a relational complexity network, aircrafts are represented by nodes, whereas relational complexities are represented by links.

Two main conflict resolution rules were identified: the central-available-node-first principle and the marginal-effort-decrease principle.

Data from 21 professional air traffic controllers suggested that the model can effectively predict number of interventions, mental workloads, task completion time, and unresolved rate.

Footnotes

Acknowledgment

This research was supported by National Key Research and Development Plan [grant number: 2016YFB1001203] and the National Natural Science Foundation of China [31671148]. The corresponding author of this paper was Jingyu Zhang.

ORCID iD

Jingyu Zhang

Author Biographies

Jingyu Zhang is currently an associate professor at CAS Key Laboratory of Behavioral Science, Institute of Psychology and Department of Psychology, University of Chinese Academy of Sciences. He received his doctoral degree in applied psychology in 2011 at Institute of Psychology, Chinese Academy of Sciences.

Xiaotian E is a master’s student of applied psychology at CAS Key Laboratory of Behavioral Science, Institute of Psychology and Department of Psychology, University of Chinese Academy of Sciences.

Feng Du is currently a professor of applied psychology at CAS Key Laboratory of Behavioral Science, Institute of Psychology and Department of Psychology, University of Chinese Academy of Sciences. He received his doctoral degree in psychology in 2010 at Washington University in St. Louis.

Jiazhong Yang is currently a professor of engineering psychology at Aviation Human Factors and Ergonomics Lab, Civil Aviation Flight College of China. He received his doctoral degree in applied psychology in 2007 at Institute of Psychology, Chinese Academy of Sciences.

Shayne Loft is an associate professor at The University of Western Australia. He received his PhD in psychology in 2004 from The University of Queensland.

References

Armstrong

(2001). Principles of forecasting: A handbook for researchers and practitioners. Boston, MA: Kluwer Academic Publishers.

Averty

Collet

Dittmar

Athenes

Vernet-Maury

(2004). Mental workload in air traffic control: An index constructed from field tests. Aviation, Space, and Environmental Medicine, 75, 333–341.

Boag

Neal

Loft

Halford

G. S.

(2006). An analysis of relational complexity in an air traffic control conflict detection task. Ergonomics, 49, 1508–1526.

Callaway

D. S.

Newman

M. E.

Strogatz

S. H.

Watts

D. J.

(2000). Network robustness and fragility: Percolation on random graphs. Physical Review Letters, 85, 5468–5471.

Desmond

P. A.

Hoyes

T. W.

(1996). Workload variation, intrinsic risk and utility in a simulated air traffic control task: Evidence for compensatory effects. Safety Science, 22, 87–101.

Gianazza

Guittet

(2006). Selection and evaluation of air traffic complexity metrics. In IEEE/AIAA 25th Digital Avionics Systems Conference (pp. 1–12). New York, NY: IEEE.

Hart

S. G.

Staveland

L. E.

(1988). Development of NASA-TLX (Task Load Index): Results of empirical and theoretical research. Advances in Psychology, 52, 139–183.

Hopkin

V. D.

(1995). Human factors in air traffic control. London, England: Taylor & Francis.

International Civil Aviation Organization. (2005). Global air traffic management operational concept (No. Document 9854). Montreal, Québec, Canada: Author.

10.

International Civil Aviation Organization. (2013). Global air transport outlook to 2030 and trends to 2040 (No. Cir 333, AT/190). Montréal, Québec, Canada: Author.

11.

Kahneman

Slovic

Tversky

(1982). Judgment under uncertainty: Heuristics and biases. Cambridge, MA: Cambridge University Press.

12.

Klein

G. A.

(1999). Sources of power: How people make decisions. Cambridge, MA: MIT Press.

13.

Kuk

Arnold

Ritter

F. E.

(1999). Effects of light and heavy workload on air traffic tactical operations: A hazard rate model. Ergonomics, 42, 1133–1148.

14.

Laudeman

I. V.

Shelden

S. G.

Branstrom

Brasil

C. L.

(1998). Dynamic density: An air traffic management metric (No. NASA-TM-1988-11226). Moffet Field, CA: NASA Ames Research Center.

15.

Loft

Bolland

Humphreys

M. S.

Neal

(2009). A theory and model of conflict detection in air traffic control: Incorporating environmental constraints. Journal of Experimental Psychology: Applied, 15, 106–124.

16.

Loft

Sanderson

Neal

Mooij

(2007). Modeling and predicting mental workload in en route air traffic control: Critical review and broader implications. Human Factors, 49, 376–399.

17.

Metzger

Parasuraman

(2005). Automation in future air traffic management: Effects of decision aid reliability on controller performance and mental workload. Human Factors, 47, 35–49.

18.

Mogford

R. H.

Guttman

J. A.

Morrow

S. L.

Kopardekar

(1995). The complexity construct in air traffic control: A review and synthesis of the literature (No. DOT/FAA/CT-TN95/22). Atlantic City, NJ: Federal Aviation Administration, William Hughes Technical Center.

19.

Moray

(1997). Human factors in process control. In Salvendy

(Ed.), Handbook of human factors and ergonomics (pp. 1944–1971). New York, NY: Wiley.

20.

Prandini

Piroddi

Puechmorel

Brazdilova

S. L.

(2011). Toward air traffic complexity assessment in new generation air traffic management systems. IEEE Transactions on Intelligent Transportation Systems, 12, 809–818.

21.

Raudenbush

S. W.

Bryk

A. S.

(2002). Hierarchical linear models: Applications and data analysis methods (2nd ed.). Thousand Oaks, CA: SAGE.

22.

Seamster

T. L.

Redding

R. E.

Cannon

J. R.

Ryder

J. M.

Purcell

J. A.

(1993). Cognitive task analysis of expertise in air traffic control. The International Journal of Aviation Psychology, 3, 257–283.

23.

Simon

H. A.

(1979). Rational decision making in business organizations. The American Economic Review, 69, 493–513.

24.

Simon

H. A.

(1990). Invariants of human behavior. Annual Review of Psychology, 41, 1–20.

25.

Sperandio

J.-C.

(1978). The regulation of working methods as a function of work-load among air traffic controllers. Ergonomics, 21, 195–202.

26.

Todd

P. M.

Gigerenzer

(2007). Environments that make us smart: Ecological rationality. Current Directions in Psychological Science, 16, 167–171.

27.

Vuckovic

Sanderson

Neal

Gaukrodger

Wong

B. L. W.

(2013). Relative position vectors: An alternative approach to conflict detection in air traffic control. Human Factors, 55, 946–964.

28.

Zhang

(2015). Relational complexity network and air traffic controllers’ workload and performance. In Harris

(Ed.), Lecture Notes in Computer Science (EPCE 2015): Engineering psychology and cognitive ergonomics (Vol. 9174, pp. 513–522). Cham, Switzerland: Springer.

29.

Zhang

Ren

(2014). Modeling air traffic controllers’ decision making processes with relational complexity network. In 2014 IEEE 17th International Conference on Intelligent Transportation Systems (ITSC) (pp. 2663–2668). New York, NY: IEEE.

30.

Zhang

Yang

(2015). From trees to forest: Relational complexity network and workload of air traffic controllers. Ergonomics, 58, 1320–1336.