Abstract
Objective:
We explore whether the visual presentation of relative position vectors (RPVs) improves conflict detection in conditions representing some aspects of future airspace concepts.
Background:
To help air traffic controllers manage increasing traffic, new tools and systems can automate more cognitively demanding processes, such as conflict detection. However, some studies reveal adverse effects of such tools, such as reduced situation awareness and increased workload. New displays are needed that help air traffic controllers handle increasing traffic loads.
Method:
A new display tool based on the display of RPVs, the Multi-Conflict Display (MCD), is evaluated in a series of simulated conflict detection tasks. The conflict detection performance of air traffic controllers with the MCD plus a conventional plan-view radar display is compared with their performance with a conventional plan-view radar display alone.
Results:
Performance with the MCD plus radar was better than with radar alone in complex scenarios requiring controllers to find all actual or potential conflicts, especially when the number of aircraft on the screen was large. However performance with radar alone was better for static scenarios in which conflicts for a target aircraft, or target pair of aircraft, were the focus.
Conclusion:
Complementing the conventional plan-view display with an RPV display may help controllers detect conflicts more accurately with extremely high aircraft counts.
Applications:
We provide an initial proof of concept that RPVs may be useful for supporting conflict detection in situations that are partially representative of conditions in which controllers will be working in the future.
Keywords
Introduction
The demand for air travel is growing rapidly, with traffic volume expected to triple within the next 20 years (SESAR Consortium, 2006). However, the displays that air traffic controllers (ATCos) use have changed relatively little in the past 50 years (Nolan, 2010). ATCos still use the plan view or radar display to build a mental picture of the traffic and detect conflicts between aircraft. The requirement to scan the display and check separation between aircraft is a significant source of controller workload (Gaukrodger et al., 2009; Loft, Sanderson, Neal, & Mooij, 2007). Studies suggest that ATCos will not be able to handle the traffic volumes expected within the next 20 years using current displays (Metzger & Parasuraman, 2005; SESAR Consortium, 2007; Thomas & Rantanen, 2006).
To further complicate matters, the role of the ATCo is expected to change dramatically in the future (International Civil Aviation Organization [ICAO], 2005). According to the Global Air Traffic Management Operational Concept (GATMOC), a system is envisaged in which trajectories are negotiated collaboratively among airspace users and air traffic management service providers. There will be an increased emphasis on strategic and pretactical planning to minimize the requirement for tactical interventions that cause aircraft to deviate from their planned trajectories. There will also be an emphasis on so-called risk-based conflict management, by which the risk of collision is to be managed dynamically, on the basis of aircraft performance and conflict geometry, rather than through the application of static separation minima based on predefined distances and/or times. The ATCo’s role will therefore shift from tactical to strategic control (Neal et al., 2011). An important question is how one develops displays that support the ATCo’s shift from tactical to strategic control while still allowing the ATCo to intervene tactically if required, especially if traffic loads are at twice or three times current levels.
As a result, international initiatives are under way to find new tools and systems to help controllers handle greater traffic loads while maintaining safety and acceptable workload. Many tools and algorithms for helping ATCos detect conflicts have already been developed but, unfortunately, few are published and evaluated in the open literature (Stankovic, Mooij, Hasenbosch, Hannah, & Neal, 2011).
Two tools for conflict detection that have been discussed in the open literature are the Medium Term Conflict Detection tool (MTCD; Kauppinen, Brain, & Moore, 2002), developed in Europe, and the User Request Evaluation Tool (URET; Brudnicki & McFarland, 1997). Both have been designed to automate aspects of the ATCo’s job and to reduce mental workload (Galster, Duley, Masalonis, & Parasuraman, 2001).
Evaluations of MTCD and URET show that contrary to intention, they may reduce situation awareness and increase workload, and ATCos may not fully trust them (Corker, Howard, & Mooij, 2005; Kauppinen et al., 2002; Kirwan, 2001; Parasuraman & Riley, 1997; Sollenberger, Willems, Della Rocco, Koros, & Truitt, 2004). A major problem of such tools is nuisance alerts that result from problems with planned trajectories rather than actual trajectories. Controllers must conduct their own checks on the radar display to verify that the alert is valid, which can add extra workload and result in poor performance (Rovira & Parasuraman, 2010). Overall, it is still not clear how best to translate the information required for air traffic control (ATC) into a visual display (Kirwan, 2001; Landry, 2011).
In this article, we introduce a display intended to support the ATCo’s strategic control of large volumes of air traffic while retaining the capacity to draw the ATCo’s attention to actual or potential breakdowns of separation. We then report a broad-based evaluation of the display with professional ATCos. Finally, we comment on the display’s strengths and weaknesses and outline issues that would have to be resolved before it could be used operationally.
Background of the Multi-Conflict Display (MCD)
In previous work, Wong, Rozzi, Amaldi, Woodward, and Fields (2006) drew from cognitive systems engineering (Rasmussen, Pejtersen, & Goodstein, 1994; Vicente, 2002), representation design (Flach & Bennett, 1996; Woods, 1995), and a cognitive task analysis of ATCos (Rozzi et al., 2006) to identify the key information needs of ATCos. Wong et al. identified a spatial dimension consisting of the principal objects, constraints, and relationships that ATCos must handle and a temporal dimension consisting of the behavior of those elements in the past, present, and future with respect to the goals and priorities of the ATCo.
Wong et al.’s (2006) analysis suggested that controllers need not only basic information about aircraft but also (and more importantly) information about relationships between aircraft and how those relationships change over time. For conflict detection, the most important spatial relationship is the distance separating two aircraft, and the most important temporal relationship is how aircraft separation behaves over time: whether it increases or decreases and at what rate. If controllers had direct access to such information, rather than having to scan for it and deduce it, they might be able to perform air traffic management more quickly and effectively. Research in many other domains indicates that human operators perform better when given relationship information (Bennett & Flach, 2011; Burns & Hajdukiewicz, 2004). Although the same conclusion probably holds for ATC, seldom have efforts been made to show higher-order relationships directly in an ATC display, rather than using auditory and text-based alerts and on-screen advisories of significant state changes.
These insights guided development of an MCD (in Gaukrodger et al., 2009; originally discussed as the scalable workflow concept in Wong, Gaukrodger, Han, Loomes, & Shepherd, 2008). The MCD would complement the current radar display and help ATCos handle conflict detection with massively increased levels of air traffic. The MCD is based on the following three crucial insights that are elaborated in the next sections:
Develop a common metric for horizontal and vertical separation—the safety unit—that lets aircraft separation be measured on a single scale.
Represent the degree of separation of any two aircraft as their distance from zero safety units, where zero safety units represents physical contact.
Represent zero safety units as a single point on a display and plot all separations relative to that point.
Calculating Relationship Information in the MCD
The goal was to find an invariant way of describing spatial relationships between aircraft. The most commonly used separation standard in the current operational environment requires aircraft to be separated by at least 5 nm horizontally or 1,000 feet vertically. Separation must always be expressed in terms of both units. An alternative is to integrate and equate these measures by expressing 5 nm horizontal separation or 1,000 feet vertical separation each as one (1) safety unit. The distance of any two aircraft in three-dimensional space can then be represented as the Euclidean distance between those aircraft in safety units. Physical contact between aircraft would be zero (0) safety units.
The Euclidean distance between aircraft in safety units is what Gaukrodger et al. (2009) term the relative position vector (RPV). For example, if Aircraft A is 11.18 nm horizontally separated (H) and 1,500 feet vertically separated (V) from aircraft B, then the RPV in safety units for the pair would be calculated as follows:
Representing Relationship Information in the MCD
The next challenge was to find a way to display aircraft separations in safety units. Wong et al. (2008) and Gaukrodger et al. (2009) show early variants of RPV visualizations that led eventually to the MCD. The most effective visual form represented zero safety units at a single point on the display and aircraft separations as distances from that zero safety unit point.
Figures 1 and 2 illustrate the MCD display principles. In each row, the left image shows a radar display of three or four aircraft with their flight levels, whereas the right image shows the corresponding MCD based on RPVs of all pairs of aircraft in the radar display.
Conventional radar display (left images) and corresponding Multi-Conflict Displays (MCDs; right images). Route lines are shown here for clarity and are not all shown in the versions of the radar display used for the experiment. Area inside the red circle of the MCD is the conflict zone. The numbers in the data block attached to each aircraft in the radar display represent the current and cleared flight levels, respectively (reading from left to right). In the MCD, all aircraft pairs are labeled for illustration, whereas in the version tested, only selected pairs were labeled, as in Figure 4.
Conventional radar display (left images) and corresponding Multi-Conflict Displays (MCDs; right images). (a) Aircraft D has been selected in the radar display, and significant relationships of D with other aircraft have been highlighted in the MCD display. (b) Aircraft pair AB has been selected in the MCD display, and the AB pair has been highlighted in the radar display. In the MCD, all aircraft pairs are labeled for illustration, whereas in the version tested, only selected pairs were labeled, as in Figure 4.
Aircraft pair distances in safety units
Each symbol in the MCD at the right of Figure 1a is not an individual aircraft but the separation between a pair of aircraft. For example, symbol AB in the MCD display represents the separation between aircraft pair A and B in the radar display. The distance of symbol AB from the center of the red circle in the MCD is the separation between aircraft A and B in safety units. If an aircraft pair symbol is at the absolute center of the circle, then the aircraft are separated by zero safety units and are in physical contact with each other. If the aircraft pair symbol is inside the red circle, then the aircraft are separated by less than one safety unit and are in the conflict zone. If the aircraft pair symbol is touching any point on the red circle, it means that the aircraft pair is separated by exactly one safety unit. In this way, the MCD offers stable regions of relative danger or safety around the single invariant point representing collision.
Note that if two aircraft are one safety unit apart in accordance with Equation (1), then under the current separation standards, they are not necessarily legally separated in both horizontal and vertical dimensions. If a pair of aircraft are separated by exactly 5 nm and 1,000 feet, then their RPV is
The radar display of Figure 1a shows that aircraft C is at a different flight level from A and B and is traveling in the opposite direction from them at a higher speed, with no possibility of conflict with A or B on its current track. In the corresponding MCD, the symbol for aircraft pair AC is drawn farther away from the red circle than AB, because aircraft A and C are separated by more safety units than are aircraft A and B. The situation is similar in the MCD of Figure 1b for aircraft pairs AC, BC, BD, and CD.
Locations of aircraft pair symbols in the MCD
The MCD imposes no hard constraint on the bearing on which symbols for aircraft pairs should be drawn. To some extent, however, relative spatial positions can be preserved with respect to a reference point at the center of the conflict zone, or origin. For example, in the radar display in Figure 1b, aircraft A is above aircraft B. Therefore, in the corresponding MCD, the symbol for aircraft pair AB can be drawn either above the conflict zone to emphasize the relative position of A with respect to B if B were at the origin or below the conflict zone to emphasize the relative position of B with respect to A if A were at the origin. At present, the MCD chooses at random which aircraft is the reference point for the display. However, if two symbols overlap, the MCD will declutter the display by choosing the alternative aircraft as the reference point and redrawing.
Temporal relationships
Note that the MCD display also allows temporal relationships to be shown. A pair of aircraft can move faster or slower toward or away from the conflict zone, depending on the rate at which they are closing on each other or moving away from each other. For example, in the MCD of Figure 1a, history dots for aircraft pair AB are close to the conflict zone, indicating that aircraft A and B were recently separated by little more than one safety unit. In contrast, in the MCD of Figure 1b, the symbol for the AB aircraft pair is heading for the center of the conflict zone at some speed, representing an imminent collision. Changes in the rate of separation or closure are shown as different spacing between history dots. Using this information, ATCos can infer aircraft relationships at different points in the past and future.
Color coding of aircraft pair symbols
In the MCD, pairs of aircraft that are safely vertically separated have green symbols with their history traces dimmed to reduce clutter. Pairs that are, or that may become, separated by less than 1,000 vertical feet have white symbols and history traces.
For example, in Figures 1a and 1b, aircraft A and B are at the same flight level, so the symbol for the AB aircraft pair is painted white in the MCD to indicate that fact. Similarly, in Figure 1b, aircraft C and D are traveling at the same speed at the same flight level on the same route, so the symbol for aircraft pair CD is painted white to indicate that the two aircraft are at the same flight level. The history dots for aircraft pair CD are present but not visible, because although the two aircraft are moving in space, their distance in safety units is not changing and therefore the location of the aircraft pair symbol is not changing. The symbols for the aircraft pairs BC and AC in Figure 1a and for AC, AD, BC, and BD in Figures 1b and 1d are green because in each case, the two aircraft are at different flight levels. Although they may move toward the conflict zone, they will not enter it.
Links across radar display and MCD
ATCos must integrate information across the radar display and MCD, so steps were taken to promote visual momentum by creating links across the two displays (Bennett & Flach, 2012; Woods, 1984). In Figure 2a, the ATCo has selected aircraft D in the radar display. In the MCD, all pairs for which the other aircraft is at the same flight level as D are highlighted in yellow, and all pairs for which the other aircraft is at a different fight level are highlighted in orange. In Figure 2b, the ATCo has selected aircraft pair AB in the MCD. The corresponding radar display highlights the two aircraft symbols A and B.
Simulated Radar and MCDs for Experiments
For the actual experiment, rather than simulating a specific radar system in operational use, we created a display that included features typical of radar or plan-view displays used in Europe, Australia, and elsewhere. The radar display in Figure 3 is similar to those in Figures 1 and 2, but a full data block now accompanies each aircraft, indicating its call sign, current and proposed altitude, velocity, and heading information. When an aircraft is selected, the symbol becomes orange and the data block yellow, and the aircraft’s route is displayed with a white line that shows its prescribed waypoints (small upright triangles with a three-letter label).
Conventional radar display. Aircraft CS0839 and CS0659 have crossing paths. In the corresponding Multi-Conflict Display (Figure 4), the air traffic controller has selected the symbol for the CS0839 and CS0659 aircraft pair, so that the individual aircraft symbols are highlighted here on the radar display and their routes have been shown in gray.
Figure 4 shows the MCD corresponding to the radar picture in Figure 3. In addition to the conflict zone indicated by the red circle at one safety unit, a second zone of operations has been defined with a much larger circle in brown. In the example in Figure 4, aircraft pair CS0064 and CS0957 is moving toward the red circle and therefore toward a potential conflict.
Multi-Conflict Display corresponding to the radar display in Figure 3. The air traffic controller has highlighted an aircraft pair to visualize it in the conventional radar display (see Figure 3). Aircraft pairs at the same flight level are displayed in white with history dots. The selected aircraft pair is shown in yellow, and the aircraft are the same flight level. To avoid clutter, data blocks are suppressed. Aircraft call signs are provided for the selected aircraft pair.
Experimental Tasks
Many different tasks and procedures have been used in conflict detection studies. Some studies involve static scenarios (Bisseret, 1981), whereas others involve dynamic scenarios (Boudes & Cellier, 2000; Galster et al., 2001). Some studies present or highlight a single pair of aircraft (Loft, Bolland, Humphreys, & Neal, 2009; Neal & Kwantes, 2009; Rantanen & Nunes, 2005; Vuckovic, Kwantes, & Neal, 2011), whereas others present up to 20 or 30 aircraft on a display (Boudes & Cellier, 2000; Galster et al., 2001; Remington, Johnston, Ruthruff, Gold, & Romera, 2000). These tasks place different demands on the participants. They are either representative of different kinds of operational scenarios or have been constructed to explore specific aspects of cognition.
For our experiment, we selected tasks that represent different operational scenarios. They are explained next, and their fundamental properties and methods of operationalization are listed in Table 1.
Details of the Four Conflict Detection Tasks Used in the Study
Aircraft on screen for conflict trials are shown here, but nonconflict trials included a trial with 23 aircraft.
In the static target-pair judgment task, two aircraft out of all the aircraft were highlighted on the screen (for example, A and B in Figure 2b). Controllers judged whether the highlighted pair of aircraft was in conflict on the basis of their assessment of the separation between the aircraft in the frozen screenshot (Bootsma & Oudejans, 1993; DeLucia & Liddell, 1998; Law et al., 1993). In an operational environment, it is not particularly common for controllers to have a specific pair of aircraft highlighted to them and to be asked to judge whether they are in conflict or not. However, it does occur occasionally when ATCos coordinate with each other.
In the static one-target search task, a full set of aircraft was shown on paper. One target aircraft was identified and controllers had to identify all aircraft that conflicted with the target aircraft (Figure 2b provides a simplified example). Using the radar display, a controller can systematically check all same, crossing, or opposite-direction tracks and verify whether any of the aircraft on these tracks are at the same level as the target aircraft. Alternatively, they can search for aircraft at the same level as the target aircraft and check whether they are on same, crossing, or opposite-direction tracks. Both types of search tasks are common in the current operational environment. As an aircraft approaches the sector, the controller checks it against existing aircraft to see whether it creates a conflict. The controller may also receive a request from a pilot to change tracks or levels, which would also require a check against all other aircraft tracks and levels.
The static multi-pair search task and the dynamic multi-pair search task required complex visual search because the controller was not given any target aircraft or pair to focus on among a full set of aircraft (Figures 1a and 1b provide simplified examples). The static task was presented on paper, whereas the dynamic task ran on the computer in real time and so included time pressure and a changing air picture. In the dynamic task, controllers also needed to interact with the display to reveal aircraft routes. Controllers might scan clockwise or counterclockwise, focusing on an individual aircraft and checking for other aircraft on converging tracks or levels, and so on. Alternatively, they may focus on aircraft that are in closest proximity to a given aircraft and move on from there. This type of task is also common in the current environment. Controllers often scan their display to detect new conflicts that may have appeared. This procedure is important if the air situation changes rapidly or if traffic volume and density preclude assuring separation for all aircraft before they enter the sector. The ability to perform this type of search will become even more important as traffic levels grow.
Hypotheses
For all four tasks, we expected that performance in the condition involving both MCD and radar displays (MCD+radar) would be superior to performance in the radar condition. As the number of aircraft increased, we expected performance to worsen overall, but we expected performance to worsen more for the radar display condition than for the MCD+radar display condition.
Method
Participants
Participants were 17 volunteer current and retired ATCos. They were recruited through an advertisement posted in a union newsletter and by word of mouth. There were 3 females and 14 males. The sample included en-route, approach, and tower controllers. Some were also in management or in supervisory roles. Although participant age and experience were not captured formally, all had substantial professional experience. Participants received a $50 store or credit card voucher plus travel reimbursement in recognition of their participation. The study had ethics approval from The University of Queensland, and participants gave written informed consent.
Design
The evaluation had a within-subjects, fully-crossed design. All participants completed all tasks in both the radar and MCD+radar conditions. In all tasks, participants would complete all scenarios in either the radar or the MCD+radar condition before switching to the other condition. The order of presentation of conditions was counterbalanced across participants.
Development of Experimental Scenarios
Scenarios for each task were generated in advance quasirandomly by a simulator. From these scenarios, we selected a set that represented the traffic load that controllers might encounter in the current operational environment as well as a set that represented conditions envisaged in the future operational environment. So that we could assess the impact of traffic load with adequate statistical power, for each task we divided scenarios into a lower and higher traffic load group. Traffic loads of 7 or 8 to 20 aircraft ranged from normal to extremes that would be encountered currently only in rare conditions, whereas traffic loads of 23 up to 36 aircraft would be exceptionally rare in present practice. Across the four tasks, there were no scenarios with 21 or 22 aircraft, which allowed a clear separation between lower and higher traffic load groups while preserving a broad range of scenarios within each group.
To promote representativeness of the test scenarios within each traffic load group, we reviewed prior conflict detection studies and consulted relevant subject matter experts from the research team. The final set of scenarios showed a range of aircraft speeds, altitudes, orientations, minimum separation distances, bearings, and times to minimum separation. Scenarios also varied in complexity and workload and, as much as possible, varied the number of aircraft on screen, density of aircraft within a period of time, and angles of convergence. Generally, the more aircraft on screen, the more conflicts there were. We excluded communications with pilots and other controllers from the scenarios as well as coordination with other controllers.
Target stimuli
In tasks in which a pair or an aircraft was highlighted, the target aircraft were highlighted in orange in the radar screen. In the static target-pair search task, the dot symbol on the MCD representing the target pair was highlighted. In the static one-target search task, the MCD highlighted all dots involving the target aircraft. The dot symbol in the MCD was yellow (representing a same altitude pair) or orange (different altitude pair). Nonhighlighted aircraft were always shown in blue in the radar and either white (representing same altitude pair) or green (different altitude pair) in the MCD.
Procedure
Controllers were tested individually in 2-hr sessions in a quiet computer room at the university campus. For familiarization, controllers watched an audiovisual presentation that explained the logic underlying the MCD and its interface and gave instructions and examples of each task. Participants then had 5 min to interact freely with an in-the-loop, dynamic simulation of the MCD and radar displays working together.
Participants then completed the four experimental tasks (see Table 1). Depending on the task, participants were asked either to classify highlighted pairs as conflicts or nonconflicts or to identify all the conflicts on the display. A conflict was defined as a pair of aircraft that would violate the separation standard if the aircraft continued according to their currently approved flight plans. In the radar condition, participants used the radar display to perform the task, whereas in the MCD+radar condition, they used the radar display in conjunction with the MCD. Participants completed the four tasks in the same order, starting with paper-based tasks that required the most basic understanding of the MCD and continuing to the dynamic computer-based task that required a more advanced understanding of the MCD. As noted, presentation of display conditions was counterbalanced across participants.
In each task, scenarios were presented once in each of the two display conditions. In the radar conditions, controllers were given only the paper screenshot or computerized display of the radar. In the MCD+radar conditions, controllers were given both MCD and radar displays, shown side by side with radar at left and MCD at right. The displays were always equal in size. In the dynamic multi-pair search task, controllers could zoom in on either display by scrolling with their mouse.
The static target-pair judgment task and dynamic multi-pair search task were conducted on the computer, which allowed us to capture response times electronically for these two tasks only. The other two tasks were paper based and no response times were captured.
After the dynamic multi-pair search task, participants were asked to reflect on their performance during the task and rate their subjective workload during the radar versus MCD+radar trials using the 10-point Air Traffic Workload Input Technique (ATWIT) rating scale (Stein, 1985). The ranges were 0 to 2 (low workload), 3 to 5 (moderate), 5 to 7 (high), and 8 to 10 (extreme workload).
After all tasks, we conducted a 10-min interview probing (a) how individual conflicts were identified, (b) subjective evaluations of workload in the dynamic task, (c) ratings of the ease of use of the tool and its user interface, and (d) suggestions for further improvements to the tool.
Results
The key dependent variable was hit rate. For each scenario with each participant, the number of correctly identified conflict pairs was divided by the total number of conflict pairs for that scenario. This calculation could be performed only for scenarios that had scripted conflicts, so trials with no scripted conflicts were excluded from the hit rate analysis. In addition to hit rates, response time analyses were conducted for the computer-based static multi-pair judgment and dynamic multi-pair search tasks.
The false alarm rate was generally extremely low. There were many nonconflict pairs in each scenario (up to 621 per scenario), but ATCos generally avoided false alarms. Means were as follows: for the static one-target search task (radar = .002, MCD+radar = .002); static multi-pair search task (radar = .003, MCD+radar = .003); and dynamic multi-pair search task (radar = .024; MCD+radar = .023). For the simplest task, the static target-pair judgment task, false alarm rate was somewhat higher (radar = .36; MCD+radar = .41) as here alone the false alarm rate was calculated with the 15 nonconflict scenarios for the task as the denominator. Preliminary analyses of false alarm rate with and without the Macmillan and Kaplan (1985) correction (which adjusts for floor or ceiling effects) indicated that regardless of how opportunities for false alarms were computed, there were no differences in false alarm rates across display conditions. Given the lack of difference in false alarm rate across displays, the concern about the best way to calculate false alarms, and the fact that hit rate is the critical performance measure for ATC (because the ATCo is responsible for assuring separation among all aircraft), we relied solely on the hit rate as the most sensitive and cogent measure of performance rather than performing signal detection analyses.
Static Target-Pair Judgment Task
Hit rate was analyzed with a one-way within-subjects ANOVA, with display condition (two levels: radar vs. MCD+radar) as the independent variable. Participants detected more conflicts in the radar condition than in the MCD+radar condition, F(1, 16) = 6.18, p = .024, ηp2 = .279 (see Figure 5).
Hit rate across the four experimental tasks. Rad = radar only; MCD = MCD plus radar; Low = number of aircraft on frequency fewer than 23; High = number of aircraft on frequency more than 30. Error bars are SEM. Asterisks highlight significant differences between display conditions with large number of aircraft. Other results are in the text. Numbers on bars are mean and SEM.
Response time was analyzed with a two-way within-subjects ANOVA. Independent variables were display condition (two levels: radar vs. MCD+radar) and type of scenario (two levels: conflict vs. nonconflict pair). Participants made decisions faster in the radar than in the MCD+radar condition, F(1, 16) = 8.80, p = .009, ηp2 = .355 (see left panel of Figure 6). Participants also made decisions faster about nonconflict pairs than about conflict pairs, F(1, 16) = 14.94, p = .001, ηp2 = .483. The interaction of display condition and conflict or nonconflict pair was not significant, F(1, 16) = 1.52, p = .236, ηp2 = .087.
Response times for the static target-pair judgment (left) and the dynamic multi-pair search task (right). Rad = radar only; MCD = MCD plus radar; No Conflict = time to judge no conflict present; Conflict = time to judge conflict present; Low = 20 aircraft on frequency; High = 34 aircraft on frequency. Error bars are SEM. Numbers on bars are mean and SEM.
Static One-Target Search Task
Hit rate was analyzed with a two-way within-subjects ANOVA, with display condition (two levels: radar vs. MCD+radar) and number of aircraft (two levels: 7 to 20 vs. 24 to 35) as the independent variables. There was no significant difference in hit rate across the two display conditions, F(1, 16) = 3.486, p = .08, ηp2 = .179, although there was a trend for a higher hit rate in the radar condition (see Figure 5). Hit rate decreased as number of aircraft increased, F(1, 16) = 58.747, p < .001, ηp2 = .786. The interaction of display condition and number of aircraft was not significant, F(1, 16) = .012, p = .915, ηp2 = .001.
Static Multi-pair Search Task
Hit rate was analyzed with a two-way within-subjects ANOVA, with display condition (two levels: radar vs. MCD+radar) and number of aircraft (two levels: 14 to 17 aircraft vs. 31 to 36 aircraft) as independent variables. There was no significant difference in hit rate across the two display conditions, F(1, 16) = 0.963, p = .341, ηp2 = .057 (see Figure 5). Hit rate decreased with the larger number of aircraft, F(1, 16) = 106.18, p < .001, ηp2 = .869. The interaction between display condition and number of aircraft was significant, F(1, 16) = 10.42, p = .005, ηp2 = .394. As the number of aircraft increased, the hit rate reduced much more markedly when participants were using the radar display than when using the MCD+radar display. Bonferroni-corrected t tests show no difference between display conditions for small number of aircraft, t(16) = 0.651, p = .524, but a significant superiority for the MCD+radar display condition for the large number of aircraft, t(16) = 3.200, p = .006.
Dynamic Multi-pair Search Task
Hit rate and response time were analyzed with two-way within-subjects ANOVAs, with display condition (two levels: radar vs. MCD+radar) and number of aircraft (two levels: 20 vs. 34) as independent variables. Participants had a higher hit rate in the MCD+radar condition than in the radar condition, F(1, 16) = 16.17, p = .001, ηp2 = .503 (see Figure 3). Hit rate decreased when the number of aircraft increased from 20 to 34, F(1, 16) = 29.68, p < .001, ηp2 = .650. The interaction of display condition and number of aircraft was significant, F(1, 16) = 5.36, p = .034, ηp2 = .251, indicating that the hit rate declined more rapidly as number of aircraft increased when using the radar display than when using the MCD+radar display. Bonferroni-corrected t tests show no difference between display conditions for the small number of aircraft, t(16) = 1.383, p = .186, but a significant superiority for the MCD+radar display condition for the large number of aircraft, t(16) = 6.433, p < .001.
Participants detected conflicts faster in the radar condition than in the MCD+radar condition, F(1, 13) = 9.90, p = .008, ηp2= .432 (see Figure 5). Participants took longer to detect conflicts when number of aircraft was 34 rather than 20, F(1, 13) = 30.39, p < .001, ηp2 = .70. The interaction of display condition and number of aircraft was also significant, F(1, 13) = 5.28, p = .039, ηp2 = .289. Bonferroni-corrected t tests show that with the smaller number of aircraft, participants detected conflicts faster in the radar condition than in the MCD+radar condition, t(16) = −2.722, p = .016, whereas with the larger number of aircraft, participants’ time to detect conflicts did not differ significantly across the two display conditions, t(16) = –.124, p = .906.
ATWIT workload ratings
A t test for dependent samples was run on ATWIT workload ratings that participants made after performing the dynamic multi-pair search task. Participants rated their workload lower in the MCD+radar condition (M = 5.029, SD = 1.932) than in the radar condition (M = 7.206, SD = 1.562), t(16) = 5.028, p < .001.
Interview Data
Representative comments from the controllers are collated in this section.
Advantages
First, participants noted that the MCD allowed them to detect conflicts faster, especially during busy periods and in unfamiliar airspace. Second, they felt it lowered workload by automating some basic calculations and minimizing cognitive processing. Third, they liked the fact that the MCD provided immediate safety assurance. Finally, they felt the MCD could be used to build situation awareness.
Concerns
First, participants commented that to use the MCD, one would require solid training and a “change in mindset.” Some participants felt they could not move easily between the radar display and the MCD, especially during high workload. The MCD may also require further interactive tools and filters to help controllers interpret it, such as a common altitude filter, a look-ahead time filter, and so on. Second, controllers noted that MCD does not have a conflict resolution component and needs a “probe” to evaluate resolution options and the likelihood of potential conflicts. Third, they commented that the MCD cannot currently handle ascending and descending traffic or simultaneous loss of vertical and horizontal separation. Finally, the participants noted various usability issues related to screen clutter, the color scheme, clarity of the history trail, and so on.
Broader issues
Some participants felt that the focus on detecting short-term conflicts without the ability to forecast future positions limited projection and planning. Some participants found that using the MCD took time away from “building the picture” in the radar and so could potentially lead to a loss of situation awareness. They also cautioned against an overreliance on the tool as it could hinder performance on other important tasks such as communication, coordination, and planning. Finally, they noted that the MCD’s focus on pairwise conflicts could prevent the choice of resolution strategies that handle multiway conflicts.
Discussion
Our hypothesis that the MCD would lead to better conflict detection was partially supported. ATCos detected conflicts better with the MCD-plus-radar display in heavy traffic conditions in the static multi-pair search task and better overall with the MCD-plus-radar display in the dynamic multi-pair search task. When ATCos were monitoring 34 aircraft in the dynamic multi-pair search task, the MCD-plus-radar display let them increase their hit rate for detecting conflicts by 37 percentage points (28% to 65%) at the cost of just 7 s increase in scanning time (from 78 s to 85 s). This result was associated with an overall drop in rated mental workload with the MCD of 2.2 units, from 7.2 to 5.0, on the ATWIT 10-point rating scale (Stein, 1985).
Clearly, when potential or actual conflicts had to be discovered, and especially when a dynamic ATC perception-action cycle was simulated, adding the MCD to the radar display led to more accurate conflict detection and lower workload. These findings suggest that the three insights behind the MCD—representing separation using the common metric of safety units, representing separation as distance from zero safety units, and representing zero safety units as a single point on a radial display—have some merit. Safety unit values of 0 and 1 are invariant properties relevant for fundamental ATC goals that are given spatial form in the MCD (Bennett & Flach, 2011; Sanderson, Flach, Buttigieg, & Casey, 1989). Displays based on these insights may better support the kind of risk-based conflict management envisaged with the GATMOC, whereby risk of collision is managed dynamically and the ATCo becomes a strategic rather than tactical controller.
In contrast, in the static target-pair judgment task, the ATCos detected conflicts more accurately with the radar display alone, and in the static one-target search tasks, ATCos showed a trend towards more accurate conflict detection with the radar display alone. If the MCD does not produce superior conflict detection for all tasks, then one reason might be that it does not capture the spatial and temporal relationships of conflict detection in a way that helps ATCos. In the present case, the evidence differed in different tasks, with the MCD supporting the most challenging tasks. Therefore another reason might be the unrepresentative nature of some of the experimental tasks.
In the static target-pair judgment task, participants saw two aircraft selected in a frozen configuration, which does not happen in the operational context. The task preempted the process of selecting the pair of interest, whereas the MCD was intended to support the process of locating aircraft pairs of interest out of a large array of possibilities. In the static target-pair judgment task, the radar display may well be sufficient for a quick, accurate response by controllers experienced with radar displays. The MCD added to processing time and led to some conflicts being missed, possibly because in static form, some configurations were not fully understood.
It is harder to explain the findings for the paper-based static one-target search task, even though conflict detection superiority with the radar display only approached significance. In this task, one aircraft was selected in the radar display, and all pairs were highlighted in the MCD display according to the MCD’s conventions. Again, it is possible that in a static form, some configurations were not fully understood. In addition, many emergent features of the MCD emerge best dynamically (Haskell, Sanderson, & Flach, 1992), which was not possible for the paper-based tasks.
Comparisons With Other Tools
It is important to evaluate whether the MCD represents an advance compared with tools such as URET and MTCD (Brudnicki & McFarland, 1997; Kauppinen et al., 2002). Comparisons are difficult because the latter tools have been designed for use by two controllers performing strategic (“D-side”) and tactical (“R-side”) functions separately. In contrast, the MCD was designed for use by one controller performing both D-side and R-side functions. Nonetheless, some comparisons can be made. First, URET, MTCD, and similar tools are principally memory aids—they help to identify conflicts and they store the information in a list for the controller. Usually, controllers still check each pair in the list to verify and calculate likely separation, and eliminate nuisance alerts, before action is taken. As the number of aircraft grows, it is harder for ATCos to gain a rapid appreciation of the picture; instead, they must perform a serial search of the display to find actual or potential conflicts. By comparison, the MCD lets the ATCo see all potential conflicts at once and rapidly identify not only whether a pair will be in conflict if there is no intervention but also the exact size of current separation, which helps prioritize activity, and whether they are on a direct collision course. Second, because the MCD shows state information rather than predicted information, it does not forecast conflicts that may not happen.
Limitations and Future Work
At present, there are three main challenges for the MCD that must be addressed in future work. First, the radar-plus-MCD arrangement requires ATCos to move their eyes between the two displays, reducing visual momentum (Bennett & Flach, 2012; Woods, 1984). Greater visual momentum could be achieved by adjustments to the color coding, by adding lines linking selected aircraft or aircraft pairs across the two displays, and by letting ATCos mark aircraft pairs and select RPV orientations on the MCD. Note, however, that the cost of switching between displays has not yet been overcome by the URET or MTCD tools.
Second, at present the MCD offers no conflict resolution support. Future work should address whether such support can be integrated with the MCD or whether the integration of conflict detection and resolution requires a fundamentally different approach altogether. Either way, a broader analysis from first principles will be required (Bennett & Flach, 2011; Burns & Hajdukiewicz, 2004; Rasmussen et al., 1994; Vicente, 2002). The URET or MTCD tools currently support the assessment of alternative suggested resolutions, but the approach taken in those tools can itself add memory load because the controller must work through all possible options via trial and error to solve a problem pair (Kirwan, 2001; Parasuraman & Riley, 1997). A means should be found for the MCD to direct the ATCo’s attention to resolution options in an arrangement where the relative merits of those options are simultaneously evident.
Third, the MCD is not optimized for present separation rules but instead is the start of a new way of representing the risk of collision (Brooker, 2011a, 2011b; Parasuraman, Masalonis, & Hancock, 2000). For the kind of dynamic risk-based conflict management discussed in the GATMOC (ICAO, 2005), appropriate separation depends on many situational factors. In future work, the MCD could be extended to include such factors by applying concepts from fuzzy signal detection theory (Parasuraman et al., 2000) as long as the display still makes conflict detection simple by revealing the increasing or decreasing separation between aircraft.
Limitations in our evaluation of the RPV concept itself are as follows. First, we tested only one kind of display based on the RPV concept. Further evaluation is needed to test whether our design decisions were fully effective. Variants may show further benefits, particularly if they address concerns that ATCos raised during the study. Second, the MCD should be tested in a more advanced simulation environment where ATCos can actively control air traffic while dealing with all aspects of their job. Third, we do not know how attention is allocated across the radar and MCD displays and how it might change with alternative display arrangements or with experience. Future research with eye tracking could help to answer such questions.
Conclusion
This study underscores the importance of testing novel display concepts across a variety of tasks. New tasks that emerge as a domain evolves—or as the domain undergoes revolutionary change—may be performed better with a novel display, as long as the display effectively captures and represents domain properties that are critical for performing the task. Overall, our study suggests that the concept of displaying relationship information for ATC holds promise, certainly when ATCos must rapidly appreciate a dynamic situation under extreme task load. Although ATCos saw that introducing a tool such as the MCD would require a complete “change in mindset,” such challenges are certainly not new to ATC display developers (Nunes, 2004; Wickens & May, 1994). They may be overcome with provisions discussed earlier, accompanied by appropriate training.
Further work is needed to extend the RPV concept, but if displays based on it show clear benefits compared with radar displays and other conflict detection aids, it may offer a better way to help controllers maintain safety requirements and manage the greater levels of traffic predicted for the near future. In summary, the RPV concept holds promise as one way to address the capacity and demand issue in ATC. By allowing controllers to maintain acceptable levels of workload with greater traffic count, displays based on it should increase safety and efficiency.
Key Points
New displays are needed that allow air traffic controllers to handle larger volumes of traffic.
The Multi-Conflict Display (MCD) was developed to facilitate conflict detection.
In the MCD, relative position vectors (RPVs) display relationships among aircraft rather than display actual aircraft.
The MCD was evaluated with four tasks differing in complexity, and results indicated that it enhanced performance on complex search tasks and with high traffic volume.
Footnotes
Acknowledgements
William Wong acknowledges the funding provided by the EUROCONTROL CARE INO III Innovation Research Programme, EEC Contract No. C06/12399BE, awarded to Middlesex University, where the original design of the relative position vector and Multi-Conflict Display was developed, and partners Space Applications Services and NEXT Ingegneria dei Sistemi. The evaluation was subsequently carried out jointly between Middlesex University, The University of Queensland, and National Information and Communication Technology Australia (NICTA). NICTA is funded by the Australian government, as represented by the Department of Broadband, Communications, and the Digital Economy and the Australian Research Council through the ICT Centre of Excellence program.
Anita Vuckovic is a PhD candidate in organizational psychology and human factors in the School of Psychology at The University of Queensland. She received her bachelor of psychological science (Hons) in 2008 from The University of Queensland.
Penelope Sanderson is Professor of Cognitive Engineering and Human Factors at The University of Queensland, where she has appointments in the Schools of Psychology, of ITEE, and of Medicine. She received her PhD in 1985 from the University of Toronto.
Andrew Neal is Professor of Organizational Psychology and Human Factors in the School of Psychology at The University of Queensland. He received his PhD in 1996 from the University of New South Wales.
Stephen Gaukrodger performed this research while a researcher at the Interaction Design Centre at Middlesex University. He has a MSc in vision and cognitive psychology and a graduate diploma in computer science from the University of Canterbury in New Zealand. His main interest is augmented reality applications.
B. L. William Wong is Professor of Human-Computer Interaction, and is head of the Interaction Design Centre at the School of Science and Technology, Middlesex University. He received his PhD in information science in 1999 from the University of Otago, New Zealand.