Abstract
This article investigates different approaches to the interpretation of eye-tracking video records of pedestrians walking outdoors to determine the apparent importance of fixation on other pedestrians and how this is influenced by the frequency of occurrence. The three approaches were as follows: the proportion of time that fixations were on pedestrians (14%), a common approach to interpretation; the proportion of fixations at critical moments that were on pedestrians (23%), critical moments being defined by a delayed response to a dual task; and the probability of an approaching pedestrian being fixated at least once (86%). These data were compared against the number of pedestrians encountered during the trials; the proportion of all fixations and the probability of fixating people were affected by the number of people encountered – only the critical-fixations data did not exhibit a trend.
1. Introduction
In residential roads, it is normal to provide lighting that focuses more, but not exclusively, on the needs of pedestrians compared to those of drivers. 1 Following Caminada and Van Bommel, 2 the key visual needs are typically suggested to be perceived safety, obstacle detection, recognition of the intent and/or identity of other road users, and lighting meeting these needs must also offer an acceptable appearance. 3 However, there is, as yet, no empirical evidence to support these assumptions, whether these tasks are indeed the most appropriate for characterising lighting, whether there are other essential visual tasks that need to be considered and the relative importance of each task. This paper investigates the importance of visually fixating on other people. One approach to identifying critical visual tasks is to find out what pedestrians look at, and eye-tracking offers one method for establishing the objects fixated. There is reason to have some confidence that distribution of gaze and cognitive processes are related4–6 to the extent that a study investigating pedestrians’ fixations in a virtual environment found that specific tasks could be predicted from fixation data. 7
Two studies used eye-tracking to record fixations on other pedestrians in laboratory trials. Kitazawa and Fujiyama 8 had test participants walk repeatedly forward and back across a 15.6 m long × 3.6 m wide platform alongside up to three target pedestrians: Jovancevic-Misic and Hayhoe 4 had test participants and five target pedestrians walk 48 laps around an oval track. In these studies, the repeated exposure to the same target pedestrians may have led to a learning effect and thus to a misleading understanding of interpersonal fixations for natural outdoor settings where we frequently do not have such advance knowledge of another person’s likely behaviour. Evidence for this can be found in the fixation durations reported by Jovancevic-Misic and Hayhoe. 4 Their target pedestrians were instructed to follow one of three behaviours: safe (no collisions), rogue (veering towards a potential collision with the test participant) or risky (equally safe and rogue). For the first 12 laps, all types of pedestrians were fixated for the same duration, approximately 500 ms, but with continued laps, the duration of fixation on safe pedestrians reduced while that for rogue pedestrians increased, these being approximately 200 ms and 900 ms, respectively, for the last four of the 48 laps.
Further laboratory-based studies have used eye-tracking to investigate gait, balance and motion.9–12 These do not report data for fixation on other pedestrians, but what they do indicate is that task difficulty and visual interest may affect allocation of fixations. Consider Patla and Vickers 12 who recorded visual fixations when walking three short (10 m) paths in a laboratory where test participants were required to step on 17 footprints on the floor. They found that travel gaze fixation (i.e. fixation held on the path a fixed distance slightly ahead of the pedestrian and carried along at the speed of locomotion) occurred for 59% of the duration and footprint fixations for 16%. More difficult visual tasks, or tasks where action has safety implications, modify the proportional allocation of fixations. In an alternative study where participants were required to step on raised narrow wooden blocks, thus posing a greater danger to stability, travel gaze fixation duration was reduced to about 40% of travel time. 13 The 8.5 m artificial path in a laboratory used by Marigold and Patla 9 included a middle section comprising a patchwork array of surfaces of different irregularity, firmness and friction; only 0.27% of fixations among the participants were considered travel gaze fixations, and fixations were predominantly directed to surfaces that were eventually stepped on. An experiment carried out in a natural outdoor setting also revealed that eye movements were affected by terrain, in this case the surfaces being either irregularly placed steps or a cobbled road. 14 A study of cyclists’ visual fixations when cycling along a short (15 m) path in a gymnasium, with three lane widths and three velocities, revealed that less demanding situations (i.e. the wider path) led to more task-irrelevant fixations. 15
Laboratory studies have a number of limitations regarding estimation of pedestrian fixations when walking naturally in outdoor settings. Although walking is relatively simple, it entails a variety of subtasks (maintaining a heading, keeping track of one’s surroundings and footing, avoiding potential collisions) 16 and in real outdoor pavements these might demand more cognitive attention than in the laboratory. Laboratory studies tend to have purposeful visual targets; when Marigold and Patla 9 found that their test participants tended to look predominantly at the artificially irregular path, this is perhaps because it was an unusual surface and there was nothing else of interest to look at in the laboratory. Internal environments generally have smooth floor surfaces and there are no distractions such as dogs, buildings or vehicles; in many studies there are no other pedestrians, and even when they are present, it is unlikely that they would be perceived as potentially threatening. There is no account in these studies for the influence, if any, of reassurance17,18 on visual search behaviour. Finally, and of importance to evidence for road lighting, the lighting conditions are not described.
Two studies used eye-tracking to investigate pedestrians’ visual behaviour during natural walking activity outdoors.19,20 In this situation, test participants are more likely to adopt their natural gait and must be prepared to respond to irregular events – uneven pavement surfaces, other obstacles and other pedestrians. Participants in the study by Foulsham et al. 19 carried out a 5–10 minute outdoor walk to a café in daytime. Fixations were categorised as being directed to people, the path or other objects, and these by near or far distance. The majority of fixations were to the near path (29%) and far objects (27%); fixations to pedestrians were 7% when near and 14% when far. Davoudian and Raynham 20 examined visual fixations for pedestrians walking along three residential roads during the day and after dark. Again, the majority of fixations were on the footpath, and in this study only 3% of fixations were on other people.
A limitation of studying fixations when walking in an uncontrolled outdoor setting is that each test participant has a different experience, encountering different samples of pedestrians and vehicles. Hence, one possible reason why Davoudian and Raynham report a smaller fixation on people (3%) than did Foulsham et al. (21%) is that fewer people were encountered during their trials. An alternative approach to interpretations of eye-tracking data is to examine the probability that a pedestrian appearing in the field of view is fixated at least once. A greater probability of fixation may reflect greater importance as it increasingly demonstrates that visual information about that object is required. For example, Jovancevic-Misic and Hayhoe 4 showed participants learn to attend to important items in the environment, and the probability of fixating an item increases with its importance.
Eye-tracking studies tend to count visual fixations in every frame of the video record.12,19,20 Foulsham et al. 19 used their data to compare fixations in the real-world study with fixations whilst watching a video of the same route; they purposefully did not seek to compare the absolute frequency of gazes to different items in recognition that these depend on the frequency with which they occur in the visual environment. However, others 15 appear to have done so, using the results from Foulsham et al. to state that the near region is frequently fixated (∼30%) during walking and only few fixations (∼10%) are made to the distant path.
Although it is clear from eye-tracking to what point gaze is directed, the inference about what is being processed is not so easily accessible; gaze location does not uniquely specify the information being extracted. 7 Visual fixations may not always reflect the focus of attention. 21 Fotios et al. 22 used a dual task (response to an auditory stimulus) in an attempt to reveal the critical fixations from within the total fixations recorded in their eye-tracking study, carried out during daytime and after dark in an outdoor environment. Using this dual-task approach, it was concluded that fixation on the near path and distant pedestrians were the critical tasks.
There are two reasons for suspecting that the dual-task approach aids identification of critical moments. The first reason is associated with attention capacity. People often have trouble in concurrently performing two apparently simple tasks. 23 Attention is defined as the information processing capacity of an individual; attention capacity is limited for any individual and performing any task requires a given portion of that capacity. 24 Thus, in a dual-task situation, when the attentional demands exceed the capacity of an individual, the performance of one or both tasks is impaired. 25 It was therefore assumed that impaired performance of the dual task, i.e. delayed reaction time, indicated moments when the primary task of safe walking demanded greater than usual attention. A pilot study demonstrated that reductions in performance on the dual task analogous to the one used in this study were caused by visual distraction. 26 The second reason is that the dual task is expected to reduce instances of daydreaming (and hence meaningless fixations) by increasing perceptual load.27,28 Attention capture by irrelevant visual features is in part determined by the availability of cognitive resources, 29 and thus reducing such resources by adding a dual task is expected to decrease fixations not relevant to the task of walking.
The aim of this paper is to explore three approaches to the interpretation of eye-tracking data to determine the importance of fixation on other pedestrians. The first approach is to quantify all fixations that can be captured from the video record (all fixations), the approach used in the majority of eye-tracking studies. The second approach is to use a dual task to better identify the objects observed in moments of cognitive attention (critical fixations). The third approach is to estimate the probability by which pedestrians are fixated. This aim was addressed through further interpretation of the eye-tracking data reported by Fotios et al. 22 A specific question is whether these data are dependent on the frequency with which pedestrians were encountered during trials.
2. Method
Eye-tracking was used to record the visual fixations of test participants walking outdoors in daytime and after dark. The method is reported in detail elsewhere 22 and hence described here only briefly. The eye-tracking system (SensoMotoric Instruments iView X HED) comprised two cameras mounted on a cycle helmet worn by the participant. One camera recorded the scene facing the participant and the second captured an image of the right eye. A calibration task enabled the eye-tracking software to mark the participants’ gaze position on the video of the scene facing the participant. With this, equipment gaze position accuracy is between 0.5° and 1.0°. The dual task was the response to an auditory stimulus, a beep emitted from a speaker attached to the underside of the eye-tracking helmet, close to the left ear. These beeps were programmed to occur at random intervals between 1 s and 3 s. The test participants were instructed to press a hand-held button immediately upon hearing this signal. The timing of each beep and each press on the response button were recorded.
Forty participants walked a 900 m route, circumnavigating the University of Sheffield campus. Each participant carried out the walk twice, once during daylight (08:00 to 16:00) and once after dark (17:00 to 20:00). The orders of the light condition (daylight or after dark) and route direction (clockwise or anti-clockwise) were counterbalanced. On attending the first trial, participants completed a Landolt ring acuity test and an Ishihara colour perception test under normal office lighting conditions, these confirming that all test participants had normal, or corrected to normal, vision. They were then set up with the eye-tracking and dual-task equipment and were given an opportunity to practice responding to the auditory stimuli. During the walk, they were instructed to press the button in response to every beep as quickly as possible. At the beginning of each route section, participants were given a description of where to walk for that section and were shown a schematic map of the route.
To analyse the collected data, fixations were placed into one of eight categories, based on the type of object or area fixated: Path, person, goal, general environment, vehicle, latent threat, trip hazard and large object. A ninth category (‘unknown’) was used to record instances when fixation data were not available (e.g. the fixation point was off-screen or missing). The all-fixations analysis requires fixation in each single frame to be recorded. Frame-by-frame coding of visual fixations is a demanding task, which is perhaps one reason why past studies (e.g. Foulsham et al. 19 ) have examined only discrete sections of their video records. Hence, the current analysis used data from only 10 (25%) of the 40 test participants and 120 s segments from three of the four route sections. These 10 test participants were those having high eye-tracking validity (few missing fixation data) and were balanced across gender, trial order (daytime or after dark being the first trial) and route direction. Of the 10 participants selected, six were male, five were aged under 30, three were aged 30–49 and two were aged over 50 years old. Three participants wore their normal corrective lenses.
The route comprised four sections chosen to provide different characteristics of surface irregularity, pedestrian encounter and reassurance and is described elsewhere. 22 Section C was shorter than the other sections and used mainly as a transition between sections B and D, and has not been included in the current analysis. Fixation coding was carried out for 120 s continuous segments of routes A, B and D, giving a total data segment of 360 s. These segments were chosen to include significant features of the section such as steps or road crossings, and they were approximately identical for all test participants.
For each test participant, three 120 s segments of their eye-tracking videos were analysed, and this was done using three different approaches to analysis:
The 10 test participants used here were selected from those having high availability of fixation data; hence, their frequencies of fixations in the unknown category were small and this was excluded. Examination of data for all three approaches did not suggest that they were drawn from normally distributed populations.
3. Results and analysis
3.1. Measures of fixation
Figures 1 and 2 show the proportions of fixations on the different categories of object as determined using the critical-fixations and all-fixations methods for the daytime and after-dark trials, respectively. In daytime and after-dark trials, critical fixations indicate a higher proportion of fixations on people and vehicles than do all fixations. All fixations tend to suggest a higher proportion of fixations in the path, latent threat, goal and object categories than do critical fixations.
Median proportions of all fixations and critical fixations per category during daytime sessions. Error bars represent interquartile range; median proportion for critical fixations on trip and object categories = 0% Median proportions of all fixations and critical fixations per category during after-dark sessions. Error bars represent interquartile range; median proportion for critical fixations on latent threat and object category = 0%
Differences between the all-fixations and critical-fixations data were examined using the Wilcoxon signed-rank tests. This did not suggest differences between fixations on people to be significant for either the daytime or after-dark trials (p values were 0.846 and 0.232, respectively). For the daytime trials, the difference between all fixations and critical fixations for the object category was suggested to be significant (p = 0.049). Other categories were not suggested to be significant (p values ranged from 0.275 to 0.846). For the after-dark trials, a significant difference was suggested for the latent threat category (p = 0.049), and hinted at for the path category (p = 0.064) but not the remaining categories (p values ranged from 0.105 to 0.922).
Overall, 727 pedestrians were visible in the video records. Averaged across the 10 test participants, median fixation probability was 0.87 in daytime, 0.86 after dark and 0.86 overall. These data are of a similar order to that reported by Foulsham et al. 19 (0.83) and Davoudian and Raynham 20 (1.0) in their studies of outdoor walking. In turn, the fixation probabilities found when walking outdoors are higher than those found when fixations were examined using a virtual reality display (0.6) (Jovancevic et al. 16 : their Figure 5, no-leader data). This may relate to expectations of possible behaviour. Data from Jovancevic-Misic and Hayhoe 4 show that probability of fixating the rogue pedestrians was higher (∼0.9) than for the safe pedestrians (∼0.6). When walking outdoors, the intentions of other pedestrians are likely to be unknown, and this leads to a high probability of fixation.
3.2. Influence of target frequency
It is acknowledged that an all-fixations approach to analysis is dependent on the frequency by which a type of object occurs during the experiment, 19 a stimulus bias. If very few people were present during an eye-tracking study, then an all-fixations approach can reveal only a small proportion of fixations on people, and this may mislead interpretation of how important it was to look at those people. This paper uses two further methods of analysis that may overcome this: critical fixations established using the dual task and probability. One way to determine whether these alternative methods are of benefit is to examine the trend between the number of people within the visible field against the measure of fixation; a preferred method of analysis would not exhibit a trend.
Table 2 summarises the data used in this analysis. Figure 3 shows regression of pedestrian fixation against the number of pedestrians encountered. These data show the day and after-dark trials for the 10 test participants. Analysis of the critical-fixations data and probability data using the Wilcoxon test did not suggest significant differences between daytime and after-dark trials. With the all-fixations data there were fewer fixations (p = 0.049) on people after dark. Fixation proportion, fixation probability and number of pedestrians encountered are different kinds of measures using different scales. For example, the all-fixation and critical-fixation values are proportions based on total fixations, the fixation probability value is a proportion based on total number of pedestrians present and the number of pedestrians encountered is an absolute frequency. The values could be more usefully compared in relation to each other by standardising them, and this was done by transforming all data sets to Z-scores.30,31 This was done for each value in the daytime and after-dark conditions by subtracting the sample mean from the individual value and dividing by the sample standard deviation (SD). Transformation to Z-scores means all three distributions have a mean of 0 and a SD of 1.0.
32
Analysis of the Z-score distributions suggested that they are drawn from normally distributed populations except for the all-fixations data.
Regression of measures of pedestrian fixation against the number of pedestrians encountered. This shows data for daytime and after-dark trials Relationship between number of pedestrians encountered and values of pedestrian fixation for three analytical approaches Note: Correlations determined using data transformed using Z-scores and with Pearson’s correlation except for all fixations which were determined using Spearman’s rho.
With the all-fixations data, the fixation proportion increases as the number of pedestrians encountered increases, confirming expectation that this approach suffers from stimulus bias. Spearman’s test suggests this correlation to be significant (p < 0.01). With the probability approach there is a negative relationship, in that there is a decrease in the probability of fixation as the number of people encountered increases, and the degree of correlation here is close to significant (p = 0.08) according to Pearson’s test. This may be because with larger numbers of people it is not possible to fixate all of them, or, alternatively, deemed not necessary to fixate on all others. The horizontal line for critical fixations shown in Figure 3 indicates that this approach does not have a relationship with the number of people encountered and the Pearson’s test does not suggest correlation to be significant (p = 0.87). Thus, the critical fixations established using the dual task lead to a more robust measure of the importance of fixating on other people as it is less effected by the number of other people encountered during trials in a natural setting.
4. Discussion
The aim of this paper is to investigate the importance for pedestrians of fixating on other people. The conclusion drawn from the all-fixations data (Figure 2) is that path is the most important category of object as it has the highest proportion of fixations; observing other people appears less important. The critical-fixations approach reveals higher proportions of fixations on people and vehicles than did all fixations, although these differences did not reach significance. This increase in apparent importance reflects the increase in visual attention expected for objects of whose behaviours are less predictable than typically static items such as objects and goals. Jovancevic-Misic and Hayhoe 4 found that pedestrians walking in an unpredictable way were more likely to be fixated, and fixated for a longer duration, than pedestrians who were predictable in their movements. Other research has shown that an unpredictable feature of an environment produces greater fixation durations, 33 and more frequent fixations 34 than a predictable feature. It might simply be that the need to perceive the motion (speed and direction) of moving objects leads to frequent fixations; for example, the walking direction of other people provides critical information about their disposition and intention. 35 Regarding fixations on people, the human tendency for social attention means there is a bias towards fixation on other people when they appear in a scene19,36,37 and this may be regardless of their apparent movement or behaviour.
This paper reports the probability of fixating on other people, contributing to discussion of the needs of interpersonal judgements when setting standards for lighting. 38 The probability of fixating different types of object when they are present would give an idea of their relative importance. However, the probability approach is only meaningful for objects such as people and vehicles which are encountered irregularly; if they appear, a high prospect of being fixated indicates that some value is placed on their observation. Objects such as pavements and goals are single entities likely to be continually present in the field of view; one fixation on the pavement, for example, would yield a 100% probability of fixation. Applying the probability analysis to continuous objects would require disaggregation of the instances when fixation may be a necessity for safe walking from those instances when fixation was not likely to be a necessity but the object was there anyway. Such disaggregation was the aim of the dual task, used to identify critical moments. Of practical interest, the probability approach would be extremely time-consuming if applied to all possible objects, and thus difficult to justify without offering substantial gain above the critical fixations approach. One possible alternative is to combine the dual-task and probability approaches by calculating the probability of fixation during moments identified as critical by the dual task. This would first demand further consideration as to how the duration of a critical period is defined.
It may also be useful to explore whether reaction time occurs evenly throughout the environment or varies in response to particular factors, for example, whether the presence of other people leads to slower reactions even in non-critical moments.
According to the proportion of occurrence, fixation on other pedestrians accounts for 14% of fixations. Using a dual task to better identify the important visual fixations, fixation on other pedestrians accounts for 23% of the critical fixations; the relative proportion of fixations on pedestrians has increased because the dual task leads to less-important fixations (e.g. objects of more predictable behaviour) being ignored. A count of the number of pedestrians fixated compared with the total number of people encountered suggests 86% probability of fixating other people. Of these three estimates of the relative importance of fixating on other pedestrians, with the former estimate one might be less inclined to consider it is important but with the latter one would be more inclined to suggest it is important. The high probability of fixation suggests that fixating on other people is important, and this is better captured by the critical fixations than all fixations. Comparison of the proportion and probability of fixations against the number of other pedestrians encountered suggests that the critical-fixations approach is less affected than are the all-fixations and probability approaches. We therefore conclude that using a dual task to identify critical fixations provides a reasonable approach to estimating the importance of fixating other pedestrians.
Footnotes
Funding
This work was carried out through support from EPSRC (EP/H050817) as part of the MERLIN project, a collaboration with UCL and City University London, and the EPSRC funded E-Futures Doctoral Training Centre for Interdisciplinary Energy Research, University of Sheffield.