Abstract
The current study examines relations between individual differences in attention measured in laboratory and real-world settings. In-laboratory computer-based assessments of orienting, phasic alertness, and executive attention as well as self-report measures of temperament and attention-related problems were administered to 111 undergraduate participants. Participants later completed a walking tour of a multistory building while wearing video recording glasses. The recordings were subsequently coded for orientation and capture of overt attention. Overt attention was correlated with attention-related traits only in high-demand walking conditions (stairways). Our findings suggest that individual differences should be a more important consideration in attention research. The novel methodology piloted here may be especially practical for studying overt attention in social settings.
Introduction
William James (1890) famously defined attention as “. . . the taking possession by the mind, in clear and vivid form, of one out of what seem several simultaneously possible objects or trains of thought” (pp. 403–404). Although it now appears that James’s definition oversimplified this topic, there is widespread agreement that attention is an integral part of human cognition. Attention is also a primary component of temperament (Rothbart, 2007) and a necessary condition for social learning (Bandura, 1977). Nevertheless, the degree to which temperament and personality relate to attention in daily life is not well understood (Kaspar & König, 2012). Further complicating matters is the theoretical premise that cognition, temperament, and social learning are dynamic, interactive processes (Bandura, 1977). Gaining an ecologically relevant understanding of attention is likely to require some consideration of relations among attention during cognitive tasks, social contexts, and individual differences in attention-related temperament traits.
Much of the empirical research in attention has concentrated on covert attention, defined as the internal processes involved in focusing on and processing specific information. A second body of research has been devoted to the study of overt attention, which involves the orientation of sensory receptors toward a particular spatial location (e.g., the direction of eye gaze or head orientation). Although overt attention can be observed in another individual’s nonverbal behavior, it does not always reflect what the individual is thinking about (e.g., strategic self-presentation; Jones, 1990; Schlenker & Pontari, 2000). Covert attention generally is more closely related to cognition, or in James’s (1890) vernacular, the mind. Because these processes are not directly observable, covert attention researchers historically have used ingenious laboratory experiments to study them. Such experiments are frequently designed with the intent to minimize or eliminate the influence of overt attention.
The drawback of this strategy is that overt attention is unconstrained in ever-changing real-world contexts where oculomotor movement and head orientation are continually responding to dynamic stimuli. Consequently, laboratory assessments of covert attention alone may not reflect attention during everyday activities (Smilek & Frischen, 2013). This problem underscores the tension between micro- and macrolevel measures of attention. If one is attempting to identify the neurocognitive processes that underlie specific attention mechanisms, the precision of measurement afforded by microlevel measures (e.g., eye-tracking devices) are indispensable. However, if the goal is to understand observable attention-related behavior in everyday settings, then the ecological validity of midlevel and macrolevel measures are of primary importance (Funder, Furr, & Colvin, 2000). School teachers, criminal investigators, and clinical psychologists seldom use eye trackers to determine whether an individual is “paying attention.” To gain a full understanding of human attention, multiple levels of assessment will be required. In the current study, we will examine the degree to which controlled, in-laboratory attention measurements predict overt attention in daily life while taking into account individual differences in attention-related temperament traits.
Covert attention
A survey of the many processes related to attention exceeds the scope of this paper. Interested readers can refer to several excellent reviews (see Nobre & Kastner, 2014; Posner, 2012; Wright & Ward, 2008). Herein, we focus on specific domains of covert attention, described in Posner and Petersen’s (1990) widely cited three Network Attention Models that include (1) the ability to orient and reorient attention; (2) the ability to become and remain alert for impending events; and (3) the ability to control attention (executive attention). Orienting, alerting, and executive attention are viewed as distinct processes, with associated neural networks developing at different rates across childhood (Fan, McCandliss, Sommer, Raz, & Posner, 2002; Posner, 2012; Raz, 2004). Controlled laboratory experiments have advanced our understanding of each of these three covert attention domains. However, it should be noted that these networks are highly interactive during most cognitive tasks (Callejas, Lupiánez, & Tudela, 2004; MacLeod et al., 2010). Complimenting controlled experiments with research designs that allow the dynamics of orienting, alerting, and executive attention to function unconstrained is crucial to understanding attention processes during real-world daily life activities.
Each of these three covert attention domains has been subdivided into separate processes. Orienting can be a function of the exogenous demands of a stimulus or the endogenous intention of the individual (Jonides, 1981; Posner, 1980). This distinction is important because attention in real-life settings requires both exogenous and endogenous orienting, and it may even involve competition between the two (Berger, Henik, & Rafal, 2005). Exogenous and endogenous orienting have been studied intensively with computer-administered reaction time (RT) spatial cueing tasks (Posner, 1980). Exogenous orienting is assessed using RTs to targets preceded by cues appearing in the potential target location. Endogenous cueing tasks involve examining RTs to targets proceeded by centrally located cues that, for most of the trials (e.g., 80%), predict the likely target location. Some RT differences emerge when the temporal lag between the cues and targets is also varied. Nonetheless, researchers consistently find that faster responses to the target are found for exogenous and endogenous trials. Furthermore, this orienting effect has been found to be present when the orientation of gaze is fixated and when eyes are free to respond to the cue. This has widely been interpreted to suggest that covert and overt orienting represent distinct processes (Posner, 1980). Measurable individual differences in both types of orienting have been observed (Frischen, Bayliss, & Tipper, 2007; Matthews & Zeidner, 2012). Deficits in both exogenous and endogenous orienting have been linked with a variety of developmental disorders, including autism spectral disorder and attention deficit hyperactivity disorder (ADHD; Gupta & Kar, 2009; Renner, Grofer Klinger, & Klinger, 2006).
Similarly, alerting can be divided into two broad categories. Phasic alertness refers to the degree to which a stimulus evokes short-term alertness, typically lasting seconds at most. This form of alertness is examined with spatial cueing tasks by comparing RTs to targets that are uncued with RTs for targets that are proceeded by cues that provide no meaningful spatial information. In exogenous cueing tasks, “neutral” trials typically include bilateral cues appearing simultaneously on opposite sides of the screen. In endogenous cueing tasks, a nonpredictive cue (e.g., dashes or asterisks) may appear in the center of the screen. Posner and Peterson (1990) attributed the resulting facilitation from the nonspatial cues to phasic alertness. Tonic alertness (i.e., vigilance) describes a state of alertness that persists. A wide variety of assessments, most notably continuous performance tasks (Rosvold, Mirsky, Sarason, Bransome, & Beck, 1956), have been used to measure linear declines in performance across time. Recently, research has shown that intraindividual variability in RTs during any task involving controlled responses may serve as a reasonable index of vigilance (Castellanos et al., 2005). Differences between individuals in levels of tonic alertness and phasic alertness are associated with levels of cognitive ability (Carter & Swanson, 1995; Edley & Knopf, 1987). Deficits in phasic and tonic forms of alertness have also been associated with symptoms of ADHD (Cao et al., 2008; Chhabildas, Pennington, & Willcutt, 2001; Collings, 2003; Collings & Kwasman, 2006; Huang-Pollock, Karalunas, Tam, & Moore, 2012).
Executive attention involves the intentional regulation of the stimuli that are responded to (activation) and the stimuli that are ignored (inhibition), in accordance with the demands of the task. A variety of experimental paradigms have long been employed to study execute attention, including flanker tasks (Eriksen & Ericksen, 1974) and the Stroop’s (1935) Test. Executive attention has been associated with individual differences in many aspects of cognitive, emotional, and social functioning (Chuderski & Necka, 2010; Eysenck, 2010; Matthews, Warm, Reinerman, Langheim, & Saxby, 2010; Posner & Rothbart, 2005).
Importantly, each of these attentional domains (i.e., orienting, alerting, and executive attention) are related to measures of temperament during childhood, adolescence, and adulthood. For example, executive attention is associated with the temperament trait effortful control, the ability to inhibit affect-related behaviors so that the individual can execute goal-oriented behaviors (Rothbart, Ellis, Rosario, Rueda, & Posner, 2003; Rothbart, Sheese, & Posner, 2007). Problems with orienting and executive attentional control have been related with facets of temperament measured during adolescence and adulthood, in particular, negative affect and rumination (Checa, Rodríguez-Bailón, & Rueda, 2008; Tortella-Feliu et al., 2014). Relations among laboratory attention measures and temperament, however, appear to be dependent on the particular demands of the experimental task. For example, Garner et al. (2012) found that behavioral inhibition was related with visual orienting, but only until attention was captured by a spatial cue. Furthermore, individuals with relatively higher behavioral inhibition scores showed a leftward attentional bias in the control condition, but not when attention was captured by a peripheral cue.
Researchers often have relied on variables such as reaction time and accuracy of responses during computer-based assessments to study specific facets of attention. Experimental manipulations in these tasks typically involve presenting stimuli under specific conditions (e.g., targets preceded by a predictive or nonpredictive spatial cue, targets flanked by similar or competing distractors). Such tasks usually are cognitively demanding, and they frequently bear little resemblance to real-world experience. The ecological relevance of these paradigms has been the focus of much debate in the literature (MacLeod et al., 2010; Willcutt, Doyle, Nigg, Faraone, & Pennington, 2005). However, it is not clear how one might conduct similar measurements of covert attention outside of the laboratory, especially if the goal is to do so in a way that is ecologically relevant. To understand the generalizability of findings such as these, studies of temperament and real-world attention would need to include contextual manipulations of attention capture (i.e., conditions of low- and high-attention demands). This presents a formidable challenge for anyone attempting to study covert attention outside of the laboratory.
Overt attention
The coordination between overt and covert attention has been of particular interest to attention researchers. Specifically, the relationship between eye movements, as one measure of overt attention, and the orienting of covert attention has been widely studied (R. M. Klein & Pontefract, 1994; Posner & Petersen, 1990). One perspective, the premotor theory, suggests that exogenous orienting of covert attention results from activation of the oculomotor system (Büchel et al., 1998; Rizzolatti, Riggio, & Sheliga, 1994). However, empirical research has not entirely supported this proposition. For example, Hunt and Kingstone (2003) found that preparing for purposeful eye movements do not necessarily result in covert attention shifts, and reorienting covert attention does not necessarily result in oculomotor shifts. On the other hand, covert attention orienting and oculomotor control do seem to be complimentary systems that select specific targets and guide saccades (Awh, Armstrong, & Moore, 2006; MacLean, Klein, & Hilchey, 2015). Modeling the specific ways in which covert and overt attention function as separate systems is an important topic of ongoing research and debate, one that is beyond the scope of the present study. However, it is relevant, because the degree to which real-world overt attention behavior can be predicted from in-laboratory attention assessments rests on the assumption that the covert and overt attention are interactive processes.
Reconciling in-laboratory (microlevel) attention research with real-world (macrolevel) studies of overt attention is complicated by several methodological challenges. Overt attention involves coordination of eye movements and head orientation. When oculomotor shifts are narrow, they can work independently of head shifts. However, oculomotor shifts of 15°–20° or more involve reorienting both the eyes and the head toward the target. It appears that eye–head coordination involves distinct, but interactive motor systems (Pelisson & Guillaume, 2009). The two motor systems become more tightly coupled as the task demands are high, but this coordination does not seem to be coupled tightly during relatively low-demand activities, such as unconstrained walking and picking up objects (Grasso, Prévost, Ivanenko, & Berthoz, 1998; Pelz, Hayhoe, & Loeber, 2001). Interestingly, Guitton and Volle (1987) found that eye movement becomes limited when head movements are unexpectedly constrained, indicating that the oculomotor program is initiated by head movement. These findings indicate that coordination between eye/head programs is an integral aspect of overt attention when individuals navigate through daily life activities. Because studies of oculomotor movement (i.e., eye tracking) often involve constraining head movement, our current understanding of overt attention functioning in the real world may likewise be limited.
Finally, studies of overt attention in the real world face a substantial threat to ecological validity: intrusiveness. Advances in portable technologies have made it possible to conduct eye-tracking assessments both in and out of the laboratory (see Molina, Redondo, Lacave, Ortega, 2014; Tien et al., 2014). However, overt attention in real-world settings involves orienting both the eyes and the head. It is possible to combine eye tracking with measures of head orientation, but these devices are obvious to other people in a social environment. Consequently, wearing such technologies can fundamentally change behavior in social settings (Gobel, Kim, & Richardson, 2015).
The current study
In this study, we examine relations between attention measured in and out of the laboratory, individual differences in attention-related temperament traits, and attention-related problems in daily life. Finding psychometrically sound self-report measures of temperament and global attention behavior was not difficult. However, we faced substantial challenges when seeking (or developing) ecologically relevant measures of covert and overt attention. For this exploratory study, we adopt the strategy of measuring attention abilities under controlled laboratory settings and later using a minimally intrusive measure of overt attention orientation in a real-life social setting. This strategy affords the opportunity to explore the degree to which individual differences in attention-related traits and attention in the laboratory predict overt attention behavior in the milieu of a social environment in the real world.
Controlled in-laboratory setting
During the initial laboratory session, we measured orienting, alertness, and executive attention using the Attention Network Test (ANT, adult version; Fan et al., 2002). We also collected self-report data on two attention-related facets of temperament, using the Adult Temperament Questionnaire (ATQ; Evans & Rothbart, 2007). The first of these temperament facets, effort control, involves the ability to voluntarily control behavior and attention associated with the executive attention network (Rothbart et al., 2007). The second, orienting sensitivity, relates to the degree to which stimuli in the environment evoke automatic responses (Evans & Rothbart, 2009). We also included a widely used assessment of attention-related problems in daily life, the Current Symptom Scale (CSS; Barkley & Murphy, 1998).
Real-life setting
The overriding aim of this study is to examine overt attention outside of the laboratory. For this purpose, we planned a walking tour that would be guided by a research assistant during the second session. The location of the tour would need to provide a reasonable balance between control and ecological relevance. We ultimately selected a public academic building, which is a social setting familiar to potential participants and representative of the world in which they live. Because prior research has found that relations among temperament and attention are dependent on situational demands (Garner et al., 2012; Grasso et al., 1998; Pelz et al., 2001), we chose a building in which we could vary the demands of the guided walking tour to include low-demand and high-demand conditions. Data collection was planned at times during the building’s regular operating hours. We developed a schedule to accommodate individual walking sessions, in which each participant would encounter the same walking challenges, maximizing constancy across conditions in this otherwise uncontrolled social setting. Research assistants were trained to minimize interactions with the participants during the walking tour and to follow the same route with all participants. Task instructions imposed two constraints: (1) follow the guide (explicit) and (2) do not trip or fall (implicit).
Similar to the logic of using eye-tracking devices to measure the orientation of overt attention, we used video glasses to record participants’ point of view (POV). This approach was intended to provide an unconstrained measure of “looking” that would involve the coordination of oculomotor and head orientation (Pelisson & Guillaume, 2009). We selected a model of video glasses that resemble traditional eyeglasses, to be as unremarkable and nonintrusive as possible. Later, each video recording was coded for two criterion behavioral variables: (1) frequency of head turns and (2) duration of fixations during head turns. We reasoned that the frequency of head turns would reflect the degree to which overt attention is initially reoriented in response to exogenous stimuli. Duration of fixations have been associated with top-down information processing of the stimuli (Buswell, 1935; Unema, Pannasch, Joos, & Velichkovsky, 2005). Consequently, one might associate the duration of fixations would reflect the degree to which overt (and perhaps covert) attention, is subsequently captured by that stimuli.
Hypotheses
Although the current study is exploratory, we planned to test several hypotheses, based on the following theoretical premises:
Premise 1 Based on current thinking, covert and overt attention appear to be interactive processes (Pelisson & Guillaume, 2009). Premise 2 These systems appear to become more closely linked as the attentional demands involved in the task are increased (Grasso et al., 1998; Pelz et al., 2001). Premise 3 The attentional demands of the task also will affect the relationship between temperament and attention (Garner et al., 2012).
Accordingly, we predicted that individual differences during the in-laboratory, high demand ANT would be more strongly correlated with overt attention (head turns) during the high demand portions of the guided tour than in the low-demand portions. In light of Unema et al.’s (2005) arguments, we anticipated that this would be most strongly reflected in the relation between executive attention and the duration of fixations (presumed to represent the capture of overt attention). We also suspected that because the ANT measures of exogenous orienting and phasic alertness both reflect automatic responses to stimuli, they would strongly predict the frequency of head turns, but not necessarily the duration of fixations. Finally, we anticipated that individual differences in attention control (i.e., the effort control temperament trait and symptoms of attention-related problems) would better predict overt attention behavior in the high demand condition than in the low-demand condition. We expected that for the low-demand portion of the tour orienting sensitivity would be related to overt attention.
Method
Participants
Given the exploratory nature of this study, we based our target sample size on two criteria. First, we applied Cohen and Cohen’s (1983, p. 61) recommendation of recruiting a minimum of 84 participants for suitable power (1 − β = .80) for a conservative effect size estimate, r = .30. Second, accounting for possible data loss due to common issues (e.g., attrition, compliance), we determined that n = 150 would be optimal. Initially, 153 students volunteered, recruited from undergraduate psychology courses through class announcements. Students were offered extra credit in their respective classes for participation. Random technology malfunctions and difficulty achieving adequate coding reliability reduced the sample by 18 participants. Students who reported a previous diagnosis of ADHD or a learning disability were excluded from these analyses (n = 24). Our final sample consists of 111 participants (90 women and 21 men), with a mean age of 20.4 years (SD = .86). A Snellen Eye chart was used to test for visual acuity problems. All participants had normal or corrected normal visual acuity. The protocols for this study were approved by the college’s Institutional Review Board and met the American Psychological Association ethical criteria.
Measures
Adult Temperament Questionnaire (77-item version)
This Likert-type rating scale measures several dimensions of adult temperament, although only the two subscales related to attention were used in the analyses reported here (Evans & Rothbart, 2007). The first of these included 19 items related to effort control. The second subscale included 15 items about orienting sensitivity. The reliability for each subscale was calculated using Cronbach’s alpha (α; see Table 1).
Descriptive statics and reliability indexes for in-laboratory measures.
CI: confidence interval; SD: standard deviation; ATQ: Adult Temperament Questionnaire; CSS: Current Symptom Scale; ANT: Attention Network Test; RT: Reaction time.
CSS Adult ADHD Self-Report Questionnaire (abridged version)
The CSS was used to assess the severity of potential attention-related problems (Barkley & Murphy, 1998). This scale includes 18 Likert-type items associated with the ADHD criteria in the Diagnostic and Statistical Manual, 4th edition (American Psychiatric Association, 1994). These items are scored for two subscales: one related to inattention and one related to hyperactivity/impulsivity. The reliabilities (α) for each CSS subscale are reported in Table 1.
Attention Network Test (adult version)
The ANT combines two well-documented experimental tasks to assess the functioning of multiple attention networks (Fan et al., 2002). A variation of Eriksen and Ericksen’s (1974) flanker task is used to assess the individual’s ability to inhibit erroneous responses to competing distractors while performing a simple choice task, as a measure of executive attention. The ANT also borrows from Posner’s (1980) widely used spatial cueing task, in which exogenous cues are presented in the periphery to automatically draw attention to specific spatial areas, in which the target will appear. Such cues have been demonstrated to orient covert attention, even when the eyes remain fixated at a central location (Deubel & Schneider, 1996; Hoffman & Subramaniam, 1995; Posner, 1980). 1 Because the task involves exogenous cueing, it provides a means of assessing the degree to which peripheral stimuli automatically orient attention during a cognitively challenging task. The ANT also provides a means of assessing the degree to which the phasic alertness is automatically activated when such stimuli suddenly appear.
The ANT was administered using E-prime v1.1 (copyright 2002, Psychology Software Tools, Inc.). The participants performed the ANT on a Dell Optiplex GX240 (1.8-GHz Pentium 4 processor), with a Dell™ 20.1 in. Active Matrix TFT LCD (1600 × 1200). Each trial involved the presentation of three visual stimuli (see Figure 1). The fixation stimulus consisted of a “+” that appears in the center of the screen. Cue stimuli consisted of a “*” presented in (1) the center of the screen (center cue); (2) above or below the center (spatial cue); (3) above and below the center simultaneously (double cue); or (4) the fixation “+” with no “*” (no cue). 2 Target stimuli consisted of the following: (1) five arrows all directed toward the right or left (congruent); (2) five arrows with the center arrow facing either to the right or to the left and the flanking arrows all facing in the opposite direction (incongruent); or (3) a single arrow facing either to the right or left flanked by two lines on either side of the arrow (neutral). A chin rest was used to minimize lateral head movement during the task.
Attention Network Test. Orienting effect = Mean RTCenter Cue − Mean RTSpatial Cue. Alerting Effect = Mean RTNo Cue − Mean RTDouble Cue. Flanker Effect (executive attention problems) = Mean RTNoncongruent Target − Mean RTCongruent Target. In the spatial cue condition, the “*” can appear above or below the fixation.
In the ANT, scores for orienting, alerting, and executive attention during the ANT for each participant were calculated using the following formulae
Large orienting effects reflect problems with shifting attention from the center of the display to the cued location. Large alerting effects reflect the degree to which the warning cue elicits phasic alertness. Excessive phasic alertness (either large or small) can interfere with responses to stimuli. Large executive attention scores reflect problems with ignoring the noncongruent flankers and difficulty with executive attention. RT and accuracy of responses (i.e., correctly identifying the direction of the center arrow) also were recorded for each participant. Finally, the intraindividual standard deviation in RT was recorded for each participant (Castellanos et al., 2005; C. Klein, Wendling, Huettner, Ruder, & Peper, 2006). Split-half reliabilities for each of the ANT measures were calculated using the average correlations among the three experimental blocks of trials (with Spearman–Brown corrections; see Rosenthal & Rosnow, 2008). Note that the reliabilities were substantially lower for the ANT orienting and alerting effects, a phenomenon that has been reported previously (Fan et al., 2002; MacLeod et al., 2010).
POV measure of overt attention behaviors
During the guided walking tour, participants wore black i-Kam Xtreme video glasses with clear lenses. Video recordings were made with a center-mounted 3 MP pinhole complementary metal-oxide semiconductor camera, with a resolution of 736 × 480, with a recording speed of 25 fps. The video recordings were transferred to a Dell OptiPlex 620 computer, with the image projected on an overhead screen using an Epson EMP-822H 3LCD projector for coding.
Behavioral coding
Each subject’s POV video was subsequently coded by a minimum of three trained behavioral coders, using training procedures and reliability criteria developed for similar types of behavioral coding (see Funder et al., 2000). Each of the five walking segments (see “Procedure” section) were coded for the frequency of head shifts, using a nine-point Likert-type scale (1 = very seldom to 9 = very frequently). The duration of fixation, occasions when the participant did turn their head, was rated using a nine-point scale (1 = generally very quick glances to 9 = generally very long inspections). This produced 10 codes per rater for each participant and rater. Average inter-rater reliabilities for the 10 ratings were calculated for each participant using the Spearman–Brown formula (rSB; see Rosenthal & Rosnow, 2008). When a coder’s ratings produced low reliability, the video was assigned to a different independent coder. Finally, the average reliability was computed across the four conditions (hallway frequency of head turns; stairwell frequency of head turns; hallway duration of fixation; and stairwell duration of fixation) for each participant. The minimum average reliability for individual participants was equal to .52 (median rSB = .82). The overall reliabilities for the four conditions were calculated using interclass correlations (ICC; see Table 2).
Descriptive statics and reliability indexes for POV ratings.
CI: confidence interval; SD: standard deviation; POV: point of view.
Procedure
During the initial in-laboratory session, participants were administered informed consent, the self-report questionnaires (CSS and ATQs), and the visual acuity screening. They then were seated at the computer station for the ANT, with the chin rest adjusted according to their height and comfort. Participants were asked to remain focused on the fixation stimulus throughout each trial of the task, to ignore the warning cue, and to respond to the target as quickly and as accurately as possible. Participants were instructed to press the right mouse button if the middle arrow pointed to the right and the left mouse button if the middle arrow pointed to the left. The fixation stimulus was presented at the onset of each trial, for between 400 and 1600 ms (randomized). One of the four potential cues (25% of trials each) was then presented for 100 ms, followed by a second presentation of the fixation stimulus for 400 ms. Finally, one of the three target stimuli was then presented until a response was given (or for 1700 ms if no response was given). The intertrial lag was also variable (3500 ms minus the duration of the fixation stimuli, cue presentations, and RT). After completing a block of 24 practice trials (with feedback on accuracy), participants performed two blocks of 96 scored trials. Equal numbers of the cue and target conditions were presented, with the sequence of conditions randomized across blocks and participants.
The guided walking tours were conducted with individual participants during a second session. 3 The route for the POV tours began on the third floor of a three-story academic building familiar to the participants. The tour progressed as follows: Segment 1—down a flight of stairs; Segment 2—across the second floor; Segment 3—down a second flight of stairs; Segment 4—across the main floor, up a short flight of open stairs and across the second floor; and Segment 5—up a final flight of stairs. The segments involving public stairwells (1, 3, and 5) were labeled high demand routes (see Figure 2(a)), and those conducted in public hallways (2 and 4) were labeled low-demand routes (see Figure 2(b)).
Examples of two walking tour conditions: (a) Stairwells (high demand condition) and (b) Hallways (low demand condition).
At the beginning of the second session, participants were administered a verbal set of instructions. They were informed that the RA would lead them on a silent walking tour of the entire building. To make the walking tour as comfortable as possible, participants were asked to leave backpacks and other belongings in the classroom during the tour. Participants were instructed to maintain a consistent distance (approximately 10 ft) behind the RA. Because we aimed to minimize constraints on the participants looking behavior, no additional instructions were given regarding where participants should focus their attention. The RA then activated the recording function on the POV video glasses and asked the participants to put the glasses on. The RA then walked 10 feet ahead and asked the participants to begin following. During the tour, RAs walked at a casual pace, maintained their head gaze straight ahead, and did not interact with the participants or bystanders. The RA did not wait for or hold stairwell doors for the participants. At the end of the tour, the RA retrieved the video glasses, and the session was concluded.
Results
Descriptive statistics, reliability indexes, and preliminary correlations
Descriptive statistics for each measure are reported in Tables 1 and 2. 4 An independent groups t test was used to test the difference in the number of pedestrians passed in the hallways and stairwells. As expected, participants passed significantly more pedestrians in the hallways (M = 13.6, SD = 8.0) than in the stairwells (M = 1.4, SD = 1.6), t (110) = 16.29, p = .001, reffect = .84.
Pearson correlation coefficients among the in-laboratory ANT and self-report attention measures are reported in Table 3. Not surprisingly, the ATQ measure of effort control was negatively correlated with the two CSS measures of ADHD-like symptoms (inattention and hyperactivity/impulsivity). None of the ANT measures of attention were correlated with the self-report measures, except a relatively weak but significant correlation between effort control and RT.
Correlations between ANT and self-reported attention variables.
ATQ: Adult Temperament Questionnaire; CSS: Current Symptom Scale; ANT: Attention Network Test; RT: Reaction time.
*Statistically significant correlations (p < .05).
Relations among ANT and self-report attention variables and POV measures of overt attention during the walking tour
Table 4 reflects the degree to which individual differences in attention measured in the laboratory predicted overt attention behavior during the walking tour. Although the ANT measures and the POV ratings were essentially uncorrelated, the self-reported attention behavior (ATQ and CSS) did predict overt attention during the guided tour. We had hypothesized that individual differences in attention would better predict overt attention under high demand contexts. To examine this, we conducted related-groups t tests to compare the differences in the absolute values of correlations among self-report attention and the POV ratings (Fisher transformed) between the low-demand (hallways) and high-demand (stairwells) walking conditions. The absolute values of the correlations among the self-report measures and frequency of head turns were significantly greater in the stairwells (Mr = .25) than in the hallways (Mr = .19), t (3) = 2.92, p = .031, reffect = .86. A similar pattern was observed for self-reported attention behavior and duration of fixation in the stairwells (Mr = .21) and hallways (Mr = .13), t (3) = 3.42, p = .021, reffect = .89.
Correlations between predictor variables and frequency of head turns and duration of fixation.
Frequency: frequency of head turns; duration: duration of fixation; hallways: low attentional demand walking; stairwells: high attentional demand walking. ATQ: Adult Temperament Questionnaire; CSS: Current Symptom Scale; ANT: Attention Network Test; RT: Reaction time.
*Statistically significant correlations (p < .05).
Although the time of day for participant walking tours was tightly constrained, we nonetheless thought it was prudent to examine whether the correlations reported in Table 4 were moderated by the number of pedestrians the participant passed in the hallways and stairwells (traffic). To address this potential concern, we calculated a series of partial correlations among the in-laboratory attention measures and the two measures of overt attention in each of the walking conditions controlling for traffic. There was no change in the magnitude of correlations, suggesting that the amount of traffic had little effect. 5
Discussion
As predicted by the literature that guided the premises for our hypotheses, self-reports of attention-related temperament traits and problems were related to overt attention behavior during the walking tour. The consistent pattern of these correlations is noteworthy. Both self-reported measures of attention control (ATQ and CSS) were associated with the POV measures of overt attention (frequency of head turns and duration of fixation) in the high demand stairway condition only. When walking up and down stairs, individuals who had self-reported relatively higher levels of effort control tended to be lower on frequency of head turns and duration of fixation. During the less demanding hallway portion of the tour, self-reported attention control was uncorrelated with shifts in overt attention. Although these findings are consistent with Pelisson and Guillaume’s (2009) contention, the coordination of covert and overt attention responds to context demands. It is important to note, however, that because both covert and overt attention processes were likely to have been engaged during the ANT and the POV assessments, it is not possible to test this speculation with the current data.
With reference to self-reported orienting sensitivity, we found a significant positive correlation with the frequency of head turns, but not with the duration of fixation. Similarly, POV measures of overt attention were correlated with ANT phasic alertness evoked by the sudden appearance of a stimulus, but not with ANT exogenous orienting. Based on this pattern of correlates, we tentatively offer the speculation that both orienting sensitivity and phasic alertness may be associated with the degree to which external stimuli evoke automatic responses, but not the degree to which attention is subsequently captured and held for further information processing. In other words, while shifts in overt attention may provide opportunities for subsequent information processing, they do not guarantee that such processing will occur. However, this is highly speculative, since distinguishing between covert and overt attention processes in either tasks exceeded the scope of the current study. Future research is needed to understand how orienting sensitivity and other temperament and personality traits interact with attention processes in everyday settings, as predicted by Smilek and Frischen (2013; also see Kaspar & König, 2012).
An important caveat must be raised about performance measures of covert attention, such as the ANT. It is possible that the relatively modest observed reliabilities with the ANT orienting effect may explain the weak correlations we observed among ANT performance measures of orienting and the self-report measures of orienting sensitivity. However, the reliability for ANT executive attention was quite strong, and yet this effect was also uncorrelated with self-report measures of effort control and inattention. Similarly, intraindividual RT variability on the ANT, also with strong observed reliability, was uncorrelated with self-reported sustained attention problems on the CSS. Further research is needed to determine whether or not the current findings are idiosyncratic of the ANT. However, the current findings do add to the growing list of concerns that have been raised about the ecological validity of the ANT and other performance measures of attention (see Collings, 2003; MacLeod et al., 2010; Willcutt et al., 2005). Attention-related processes measured during in-laboratory experiments are presumed to be generalizable to cognition outside of the laboratory. The current findings challenge this presumption, suggesting that attention processes (both overt and covert) may be highly sensitive to task demands and the physical or social context.
The POV methodology piloted in the current study represents an innovative approach to assessing overt attention. Although stringent criteria for coding reliability were used for this study, such ratings lack the precision of oculomotor measurements using eye trackers. However, eye-tracking technology is relatively intrusive in natural settings, and it has been found to influence the looking behavior of the person wearing the apparatus (Risko & Kingstone, 2011). Consequently, such approaches may have limited ecological validity in the study of “real-world” social behavior. For our participants, wearing POV video glasses was a similar experience to wearing eyeglasses with noncorrective lenses. Consequently, the POV glasses were less intrusive for the participant and less obvious to bystanders. For this reason, they may represent a relatively “naturalistic” means of measuring overt attention related to head orientation. As “smart glasses” technology and less-intrusive portable eye tracking and neuroassessments are refined, researchers may develop increasingly sophisticated approaches to studying and measuring overt attention during social interactions in a variety of situations (e.g., workplaces, athletic competitions, classrooms).
Conclusions
In conclusion, overt attention (i.e., the frequency of head turns and duration of fixation) was associated with self-report measures of both attention control and orienting sensitivity during high-demand walking conditions, but not during low-demand conditions. Interestingly, in-laboratory measures of attention were generally unrelated to our real world measures of overt attention. Our findings suggest that individual differences in attention and attention-related traits and contextual demands should both be important considerations in attention research.
Footnotes
Acknowledgments
The authors acknowledge Audrey Adams, Andrew Billups, Shannon Braddick, Andrew Leverton, and Philip Rascona for their assistance with the data collection and coding. The authors also thank their institution for the use of the facilities and equipment used in this study.