Do You See the Forest or the Tree? Neural Gain and Breadth Versus Focus in Perceptual Processing

Abstract

When perceiving rich sensory information, some people may integrate its various aspects, whereas other people may selectively focus on its most salient aspects. We propose that neural gain modulates the trade-off between breadth and selectivity, such that high gain focuses perception on those aspects of the information that have the strongest, most immediate influence, whereas low gain allows broader integration of different aspects. We illustrate our hypothesis using a neural-network model of ambiguous-letter perception. We then report an experiment demonstrating that, as predicted by the model, pupil-diameter indices of higher gain are associated with letter perception that is more selectively focused on the letter’s shape or, if primed, its semantic content. Finally, we report a recognition-memory experiment showing that the relationship between gain and selective processing also applies when the influence of different stimulus features is voluntarily modulated by task demands.

Keywords

neural gain perception attention memory neural network pupillometry

The ability to focus on particular cues while ignoring others is necessary for people to perform many daily activities (Arthur & Doverspike, 1992; Green & Bavelier, 2003); it is especially useful when a particular cue signals an available reward or imminent danger (e.g., seeing a bear in the woods) that requires a person’s full and immediate attention. Such high-stakes situations typically increase physiological arousal, which has long been thought to narrow attentional focus (Easterbrook, 1959). However, narrow focus on one or a few cues can compromise performance in situations that require integration of a broad range of cues (Baddeley, 1972)—even basic functions, such as recognizing a face, depend on simultaneous integration of multiple cues (Richler, Cheung, & Gauthier, 2011).

We propose that the balance between focus and breadth in perceptual processing is controlled by brain-wide levels of neural gain (Aston-Jones & Cohen, 2005; Eldar, Cohen, & Niv, 2013; Servan-Schreiber, Printz, & Cohen, 1990). Our hypothesis follows from the idea that gain enhances both excitation and inhibition and thus increases the contrast between weak and strong neural inputs (Fig. 1). As a result, perceptual processing may become dominated by the strongest inputs—those that reflect the most salient signals—at the expense of weaker sources of information that are effectively ignored. In contrast, with low gain, weak and strong inputs produce comparable levels of activity; therefore, perception may reflect a broader range of sources of information.

Fig. 1.

Input-output functions of model processing units (e.g., a neuron, or possibly a population of neurons) with low and high neural gain. Variations in neural gain can be captured in computational models by changing the gain of a standard nonlinear activation function (e.g., output = 1/(1 + e^{–gain × input}). The arrows highlight the differential activation of inhibited and excited units when gain is high versus low.

Converging evidence suggests that neural gain is modulated throughout the brain by the locus ceruleus–norepinephrine (LC-NE) system (Aston-Jones & Cohen, 2005; Einhäuser, Stout, Koch, & Carter, 2008; Eldar et al., 2013; Gilzenrat, Nieuwenhuis, Jepma, & Cohen, 2010; Jepma & Nieuwenhuis, 2011; Murphy, Robertson, Balsters, & O’connell, 2011; Servan-Schreiber et al., 1990; Waterhouse, Moises, & Woodward, 1980; Waterhouse, Moises, Yeh, Geller, & Woodward, 1984; Waterhouse & Woodward, 1980). In findings pertinent to the present study, small pupillary responses, which are indicative of high baseline LC-NE activity (Aston-Jones & Cohen, 2005; Joshi, Li, Kalwani, & Gold, 2016), have been shown to be associated with signatures of high gain in functional magnetic resonance imaging, including, in particular, a higher contrast between weak and strong activations (Eldar et al., 2013). Smaller pupillary responses were also associated with a more locally focused configuration of neural dynamics and with learning behavior that focused with greater selectivity on particular aspects of experimental stimuli (Eldar et al., 2013). These latter findings are in line with our hypothesis concerning gain and focused attention.

Building on this previous work, in the current experiments, we investigated the effects of gain on the balance between focus and breadth in perceptual processing. We began by simulating the effects of gain on perception in a neural-network model to demonstrate that with high gain, processing is selectively dominated by the most salient stimulus features, whereas with low gain, other features are taken into account as well. We then tested for this effect experimentally by manipulating feature saliency via subliminal priming and examining how the effect of this manipulation on perception varied with gain, indexed using pupillometry. In a second experiment, we extended our hypothesis to the domain of memory and to circumstances in which differences in feature saliency arise from explicit task demands.

Simulation and Experiment 1: Letter Perception

Method

To test the degree to which perception reflects sources of information that differ in salience, we used ambiguous stimuli; specifically, each character was a hybrid of two English letters. Each character was closer in shape to one of its two precursor letters, but when presented as part of a potential word, the character was more likely to be perceived as the other precursor letter. For example, the middle character in “” resembles the letter H, but the resemblance of the whole string to the word CAT favors perception of the ambiguous character as the letter A (see Fig. 2). Perception of such a stimulus involves competition between the letter’s shape and its potential to form a familiar word with the adjoining letters. Because processing of words relies on and is thus secondary to processing of character shapes (McClelland & Rumelhart, 1981), we assumed that information about character shape is more immediately salient than is information about word context. As a result, we predicted that participants with high levels of neural gain would perceive the letter that better matched the character’s shape, whereas participants with low gain would integrate the shape and word information more equally. To further test the relationship between gain and salience, we attempted to manipulate the relative salience of the character shape and word context by semantically priming half of the stimuli, thereby increasing the salience of the word context. We predicted that increasing word salience would reverse the relationship between gain and letter perception, such that with high gain, perception of primed stimuli would more strongly reflect the letter that matches the word context.

Fig. 2.

Example of an ambiguous character used in Experiment 1. Perception of this character is affected by its shape as well as the letters that surround it. Resemblance of the strings to known words favors perception of the ambiguous character as an H in the string shown at the top and as an A in the string shown at the bottom.

It is not possible to directly measure gain, or the norepinephrine activity thought to regulate gain, in human participants. However, pupil diameter has been shown to closely correlate with LC-NE activity in nonhuman primates (Aston-Jones & Cohen, 2005; Joshi et al., 2016) and with behaviors hypothesized to be associated with LC-NE activity in humans (Einhäuser et al., 2008; Gilzenrat et al., 2010; Jepma & Nieuwenhuis, 2011; Murphy et al., 2011). Furthermore, we recently showed that pupil-dilation response, which is thought to be inversely related to baseline LC-NE activity (Aston-Jones & Cohen, 2005), correlates inversely with hallmarks of brainwide fluctuations of gain in functional MRI (Eldar et al., 2013). The stimulus-locked phasic pupil-dilation response, which is anticorrelated with baseline pupil diameter, is particularly useful for between-subjects comparisons, because it can be normalized to the baseline diameter and thereby dissociated from factors that confound between-subjects baseline comparisons. We therefore used mean pupil-dilation responses to task stimuli as an inverse measure of gain—that is, high pupillary responses were assumed to reflect low gain and vice versa.

Participants

Eighty-six participants (mean age = 21.7 years, age range = 18–61 years; 69 women) performed the main experiment. A sample size of 80 participants was chosen a priori on the basis of previous studies of semantic priming effects (Lucas, 2000), and data collection continued until the desired sample size was reached. Six participants who had fewer than 20 trials in which at least half of the baseline pupil-diameter and pupil-response measurements were free of artifacts had to be excluded from the sample. Participants were from the Princeton University area and gave written informed consent before taking part in the study, which was approved by the university’s institutional review board. Participants received either monetary compensation ($10) or course credit for participation.

Experimental task

Participants were presented with 88 three-letter strings, 52 of which included an ambiguous character that was a hybrid of two English letters. If the ambiguous character was interpreted as one of its two precursor letters, the string formed a word, whereas if the character was interpreted as the other precursor letter, the string did not form a word. We manipulated the salience of the potential word using subliminal presentation (33 ms) of a semantically related word before half of the letter strings, whereas the other half were preceded by subliminal presentation of a nonword of similar size (semantic priming was counterbalanced across participants). We used semantic rather than repetition priming because the latter would involve priming of both the visual shape of one of the letters and the semantic meaning of the potential word.

After the priming stimulus, the three-letter target stimulus was presented for 225 ms, flanked by “%%%” on both sides to mask the priming stimulus, which could consist of more than three letters. The three-letter string then disappeared from the screen and an arrow pointed to where the target letter had previously appeared. Participants had 5 s to choose, from a list of four letters, which letter the target letter most resembled. The list always included the two letters from which the ambiguous character was derived and two other letters that did not appear in the letter string, which allowed us to verify that participants were not choosing letters randomly. Choices of one of the two letters that did not appear in the letter string were infrequent (less than 5% of trials) and were not included in the analyses. The intertrial interval varied randomly (within a uniform distribution of real numbers) between 6 and 10 s—long enough to allow the pupil-dilation response to resolve after each trial (Hoeks & Levelt, 1993).

Participants were explicitly instructed to try to choose the letter that most resembled the target character and to disregard whether the letters formed words. It is possible that participants sometimes deliberately chose letters that form words; we cannot rule this possibility out, but we have no reason to expect that such behavior would correlate with indices of neural gain. To account for possible response biases that may have resulted from conscious awareness of the priming manipulation, we asked participants during debriefing whether they saw any words appearing immediately before any of the letter strings. Ten participants reported that they saw such words. The results presented include the data from these participants, but analyses performed with and without these data produced similar results.

Stimuli

We used the Processing programming environment (Reas & Fry, 2007) to design 52 ambiguous characters, each created by morphing one letter halfway into a different letter. Each ambiguous character was then embedded in a three-letter string that either formed or did not form a word, depending on which of the two possible letters was perceived. To counteract the word’s contextual effect on perception of the ambiguous character, the characters were morphed so that their shape was slightly closer to the letter that did not form a word. Ambiguous characters were positioned at either the beginning or the end of the letter string, and participants were directed to fixate at the center. This ensured that the distance between the ambiguous letters and the focus of gaze remained constant throughout the experiment while allowing variability in the location of the ambiguous letter. The words that letter strings could form were all medium- to high-frequency words (above 10 per million; Kucˇera & Francis, 1967) picked using the MRC Psycholinguistic Database (Coltheart, 1981).

To prime the words that ambiguous characters could form, we used semantically related words, three to seven letters long. To avoid shape-related priming effects, we made certain that priming words did not include the two letters that the ambiguous letter resembled or other visually confounding letters (e.g., because of visual resemblance, F could favor perception of E). To ensure that participants were paying attention to all three letters of each string and not just to the ambiguous letter, we designed 36 additional three-letter strings in which one letter was somewhat morphed, but participants were asked to identify one of the nonmorphed letters.

To maximize the ambiguity of the ambiguous characters, we conducted several iterations of a preliminary experiment, the results of which were used to adjust the stimuli so as to equalize the probability that the ambiguous character would be perceived as the word-forming or non-word-forming letter. On each iteration, 4 to 6 participants performed the task described earlier. Then, every ambiguous character that was perceived as one particular letter at least 80% of the time was morphed slightly toward the other letter. This process was repeated six times, for a total of 30 participants (mean age = 20.4 years, age range = 18–23 years; 25 women). Participants in this preliminary experiment were also from the Princeton University area, gave informed consent, and were compensated with $10 or course credit. To minimize luminance-related changes in pupil diameter, we adjusted all stimuli to be isoluminant with the background using the flicker-fusion procedure (Lambert, Wells, & Kean, 2003) on the display system used in the experiment.

Pupillometry

A remote table-mounted eye tracker (ASL Series 5000; Applied Science Laboratories, Bedford, MA) was used to measure the diameter of participants’ left pupil as they were performing the task. At the be-ginning of the experiment, a baseline measurement of pupil diameter at rest was taken for a period of 45 s. Pupil-diameter data were processed in MATLAB (The MathWorks, Natick, MA) to detect and remove blinks and other artifacts. For each trial, baseline pupil diameter was computed as the average diameter over a period of 1 s before the beginning of the trial (at the end of the intertrial interval, at which point pupil activity from the trial itself should have subsided). Pupil-dilation response was computed as the difference between the baseline diameter and the peak diameter recorded during the 4 s that followed the beginning of the trial. All pupil-dilation responses were normalized according to a given participant’s preexperiment baseline pupil diameter. Horizontal displacement of gaze during stimulus presentation was quantified for all participants but one (for whom gaze data were not recorded because of a technical problem). Because gaze displacement might affect pupil-diameter measurements, we used a control covariate indicating gaze displacement to verify that all reported correlations with pupil diameter could not be explained by differences in gaze displacement.

Neural-network model of the task

To formalize our hypothesis, we first simulated perception of the stimulus “” using an established neural-network model of letter and word perception (McClelland & Rumelhart, 1981). The network consisted of three layers: a “visual” input layer, a letter layer, and a word layer (Fig. 3a). Because C and T are unambiguous, their respective letter-layer units received maximal input (i.e., an input value of 1). In contrast, because the middle letter is ambiguous, the H and A letter-layer units received submaximal input (i.e., input values less than 1). To reflect the fact that the shape of the ambiguous letter was closer to H, we simulated stronger input to the H unit compared with the A unit (precise values are given at the end of this section). Because the task required participants to decide on a single percept for the ambiguous letter, in our simulation, the A and H units competed through mutual inhibition, such that only one prevailed on any given trial. Finally, the unit representing the word CAT was connected with excitatory connections to the letters C, A, and T, with which it is consistent.

Fig. 3.

Our neural-network simulation of the effect of neural gain on perception of ambiguous letters. The diagram in (a) shows the structure of the network, which was applied to a stimulus similar to the one at the bottom of Figure 2. Lines ending in arrowheads indicate excitatory connections, and lines ending in filled circles indicate inhibitory connections. The circles in the input row contain assigned input values (see the Neural-Network Model of the Task section). Simulated choice of the word-forming letter (A) rather than the non-word-forming letter (H) is graphed in (b) as a function of neural gain and the degree of priming (1 = maximal priming; 0 = no priming) of the “” word-layer unit. One thousand simulations were conducted at each setting of gain (0–13) and priming. In (c), the graphs show the trajectories of activation of the letter units (H, A, C, and T) and word unit (CAT ), with low gain (4) and high gain (10), without priming and with maximal priming. Iteration 0 indicates onset of the “CAT” stimulus input. Shaded areas indicate the relative timing of word priming.

To simulate the limited exposure time used in the experimental task, we applied input to the letter units for 225 iterations. At each time step t, the activity αti of every network unit i built up gradually according to a weighted sum of its inputs:

α_{i}^{t} = 0.9 α_{i}^{t - 1} + 0.1 f (b_{i} + \sum_{j} w_{i j} α_{j}) + n,

where b_i refers to the bias to unit i (initially set to −0.5 for all units), w_ij refers to the connection weight from unit j to unit i (set to +1 for excitatory connections and −1 for inhibitory connections), f is the sigmoid activation function,

f (x) = \frac{1}{1 + e^{- g . x}},

and n is a normally distributed random-noise variable. The gain parameter (g) in the sigmoid function was used to simulate the level of neural gain in the network, which was the same for all units.¹ Finally, we simulated semantic priming of the word information by adding excitatory input to the CAT word unit for 33 iterations immediately before the stimulus input.

Because the network’s task was to reach a decision between perception of the middle character as A or H, we simulated two mutually inhibitory decision units (activity was initialized to 0, and bias was always set at 0), each of which had a bidirectional excitatory connection with its corresponding letter unit. After presentation of the stimulus, the network switched to a decision mode, in which the biases of the letter-layer A and H units were increased from their resting state of −0.5 to 0, simulating the allocation of attention to the letter-decision task (Cohen, Dunbar, & McClelland, 1990). Activity continued to be updated using Equation 1 until (a) one of the decision units reached an activity level of 0.9 or (b) 1,000 iterations were completed, at which point the probability of choosing the word-forming letter (A) was computed as the activity of the A decision unit divided by the sum of the activity of both decision units. In addition, to simulate the relationship among letter choice, reaction time, and a noisy pupillary index of gain, we computed reaction time as the number of iterations the network needed to reach a decision (with a maximum of 1,000 iterations) and pupil response as the true level of gain used in the simulation plus randomly distributed noise (SD = 0–10). The strength of the inputs to the H (0.52) and A (0.25) letter-layer units and the level of noise (SD = 0.035) were adjusted to make the network equally likely to decide in favor of H or A under conditions of low gain (≤ 4).

Statistical analysis

Analyses were carried out using MATLAB. All predictions concerning individual differences were tested using correlation and regression analyses across the whole group of participants. Median splits were used only for complementary analyses and for visualization of results. Reported correlation values are Pearson correlation coefficients. The significance of across-participants correlations was computed using the Student’s t distribution. Averaging of correlation coefficients was preceded by Fisher’s r-to-z transformation and followed by Fisher’s z-to-r transformation to mitigate the problem of the nonadditivity of correlation coefficients. Group-level significance of within-participant correlations was computed using a one-tailed one-sample Student’s t test on the vector of correlation coefficients after the Fisher’s r-to-z transformation. To account for potential outliers, we also tested correlations and interactions using robust regression analysis with default MATLAB options (bisquare weighting, tuning constant = 4.685; Holland & Welsch, 1977; Rousseeuw & Leroy, 2005). All statistical tests were two tailed except for within-participants tests that were used to validate between-participants results.

Results

Simulation

We used the neural network shown in Figure 3a to simulate perception of the stimulus “” with different levels of neural gain. With low gain, the shape of the ambiguous character initially drove the network to perceive the letter H, but as the surrounding letters activated a representation of the word CAT, perception of the letter A increased. As a result, the network was equally likely to perceive the ambiguous character as A or H (Fig. 3b and Fig. 3c, leftmost graph). In contrast, with high gain, the effect of the ambiguous character’s shape was enhanced, and thus the network settled on the non-word-forming letter before the word representation had a chance to influence the outcome (Fig. 3b, red line, and Fig. 3c, second graph from left). Thus, a higher level of gain, despite being applied similarly to all network units, focused processing on the ambiguous character’s shape.

Our hypothesis suggests that the focusing effect of high gain acts in favor of the character’s shape because the shape information has a stronger and more immediate impact (for additional simulations investigating the distinction between strength and immediacy, see Fig. S2 in the Supplemental Material). To test this explanation, in a second set of simulations, we preactivated (i.e., primed) the word representation, and thereby strengthened the word’s immediate impact relative to the character-shape information. In this case, high gain had the opposite effect: When the word was primed, higher gain became associated with a higher frequency of word-congruent letter perception (Fig. 3b, blue line, and Fig. 3c, right two graphs).

Experiment

To test for similar effects of neural gain on perception in humans, we showed participants letter strings such as “” and asked them to indicate which letter the ambiguous character resembled most. As we predicted, both a correlation analysis, r = .30, t(78) = 2.78, p < .01, and a robust regression analysis, t(78) = 3.3, p < .005, revealed that participants with a lower mean pupil response (indicating higher sustained neural gain) were more likely to perceive the ambiguous character as the letter that did not complete a word (Fig. 4a). These results indicate that high gain was associated with perceptions that more strongly reflected the ambiguous character’s shape and are consistent with our simulations. Moreover, the relationship between pupil response and letter perception changed in the opposite direction when the words were subliminally primed using semantically related words (e.g., the stimulus “” was preceded by subliminal presentation of the word “DOG”), r = −.12 with priming versus r = .30 without priming. Thus, pupil response and the difference between the priming and no-priming conditions were correlated, r = −.28, p = .01, and a robust regression analysis showed an interaction between pupil response and condition, t(156) = 2.7, p < .01 (Fig. 4b).

Fig. 4.

Results from Experiment 1. The scatterplots (with best-fitting regression lines) show the relationship between the percentage of choices of the word-forming letter and mean pupil response in (a) the no-priming condition and (b) the priming condition. In (c), the mean percentage of choices of the word-forming letter is shown separately for participants with low and high mean pupil-dilation responses in the no-priming condition and the priming condition. Low and high pupil-dilation responses were those below and above, respectively, the median pupil-dilation response (9.88%). The graph in (d) presents correlations from the simulation. Correlations between pupil response and letter choice, between pupil response and reaction time, and between reaction time and letter choice are graphed as a function of noise in the simulated pupil measurement. One thousand simulations were conducted with each level of noise and neural gain, with and without priming. In (e), the mean reaction time of human participants is shown for choices of the non-word-forming letter and choices of the word-forming letter in the priming and no-priming conditions. a.u. = arbitrary units. Error bars indicate ±1 SEM.

Although semantic priming generally increased word-congruent letter choices, mean increase = 4.6%, SE = 1.8, t(79) = 2.6, p = .01, it did so only in participants whose pupil responses indicated high gain (i.e., mean pupillary response below the median; Fig. 4c): When the word was not primed, perception primarily reflected character shape, but when the word was primed, perception primarily reflected the word context, mean increase = 7.7%, SE = 2.8, t(39) = 2.80, p < .01. In contrast, participants whose pupil responses indicated low neural gain (i.e., whose pupil responses were higher than the median) were relatively unaffected by the saliency manipulation, exhibiting almost equal sensitivity to letter shape and word in both conditions, mean increase = 1.5%, SE = 2.3, t(39) = 0.69, p = .50. Together, these findings suggest that participants whose pupil diameter indicated high gain predominantly processed the most salient aspect of the stimuli, irrespective of its source.

Although we observed the predicted relationship between pupil response and letter perception across participants, we did not find a similar within-participants relationship between perception and trial-by-trial variations in pupillary response. One reason for this may be that neural gain did not vary sufficiently within individual participants over the course of the experiment for such a relationship to be detectable. Consistent with this possibility, the difference in pupil response between the first and second halves of the experiment was significantly lower within participants (M = 2.3%) than across participants (M = 5.1%), t(79) = −8.6, p < 10⁻¹³. In addition, the high level of noise associated with pupillometric measurements makes it difficult to detect trial-by-trial within-participants effects.

To circumvent this problem, we used reaction time as an alternative index of neural gain. High gain leads to faster reaction times because all signals are amplified and thus a network settles on a decision more quickly. This is evident in the model (see Fig. 4d) as well as in the experiment, where the trial-to-trial correlation between pupil response and reaction time was significant in both the no-priming condition (mean r = .08), t(79) = 2.7, p < .01, and the priming condition (mean r = .07), t(79) = 2.3, p < .05. Our simulations suggested that if pupillometric noise exceeds a certain level, the effects of neural gain on perception might be more robustly evident when using reaction time instead of pupil response as an indirect index of gain (Fig. 4d). In results consistent with this suggestion, we found that reaction times were faster for shape-related letter choices (i.e., choices of the non-word-forming letter) in the no-priming condition but were faster for word-forming letter choices in the priming condition (difference between conditions: M = −0.05, SE = 0.02), t(79) = 1.9, p < .05 (Fig. 4e), which indicates that high gain was associated in both cases with perceptions that more strongly reflected the more salient feature. This within-participants interaction between priming and reaction time mirrors the between-participants interaction between priming and pupil response (shown in Fig. 4c), which indicates that variations in gain between and within individuals had similar impacts on perception.

Experiment 2: Recognition Memory

In Experiment 1, we used subliminal priming to manipulate salience in order to investigate the interaction between salience and neural gain in the domain of perception. In Experiment 2, we tested whether these interactions extend to the domain of memory and to salience that arises from voluntary allocation of attention in accord with task demands. We directed participants’ attention toward the visual shapes of words by asking them to rate how easy it was to read each word and then used a memory test to assess the degree to which memory for the words relied primarily on the words’ visual shape (i.e., font). On the basis of our hypothesis that high gain focuses processing on the most salient information, we predicted that high gain would be associated with word memory that is more specific to the particular font in which the words originally appeared, if participants were directed to focus on word shape.

Method

Participants

Forty-five participants (mean age = 19.8 years, age range = 18–22 years; 28 women) performed the recognition-memory experiment and received course credit for participation. The sample size was chosen on the basis of sample sizes used in similar studies (Graf & Ryan, 1990; Morris, Bransford, & Franks, 1977). Data collection continued until the desired effective sample size was reached. Participants were Princeton University students who gave written informed consent before taking part in the study, which was approved by the university’s institutional review board.

Experimental task

Participants were presented with 72 words, each for 2 s, in one of two highly dissimilar fonts. Half of the words were coupled with a task that focused participants’ attention on word shape. Specifically, participants were asked to rate each word’s readability on a scale from 1 (very hard to read) to 4 (very easy to read). The other half of the words, which served as a control, were coupled with a semantic task that required processing both a word’s shape (to read the word) and its meaning. Specifically, participants were asked to report, for each word, whether it referred to an object that was human made (e.g., buildings) or not (e.g., trees).

Words were presented in four 18-word blocks, and the two tasks were performed in separate blocks. Task order was counterbalanced both within and between participants. To mitigate primacy and recency effects, we started and ended each block with 4 words that were not included in the later recognition-memory test. Words that participants indicated they were not able to read (M = 1.0 words per block, SE = 0.18) were excluded from further analysis. Words were separated by an intertrial interval randomly selected from a uniform distribution from 7 to 9 s.

After an average period of 19.0 min (SE = 0.18), during which participants performed an unrelated decision-making task, they were tested on their recognition memory for the words. In this test, half of the words were foils, a quarter had previously appeared in the same font as they had previously (in either the readability or the semantic task), and a quarter had previously appeared in a different font than they had previously. Participants were asked to indicate whether each word had appeared in the first part of the experiment (regardless of font). Recognition-memory performance was quantified by participants’ hit rate.

Stimuli

For each participant, 176 words were randomly assigned to different blocks or to be used as foils. All words were of medium to high frequency (> 10 occurrences per million; Kucˇera & Francis, 1967) and were five to seven letters long. They were presented in an isoluminant color in capital letters in one of two fonts, 86-point Old English Text MT or 80-point Matura MT Script. These fonts were chosen because they are dissimilar and relatively difficult to read.

Pupillometry

We again used mean pupil-dilation responses to task stimuli as an inverse measure of gain. A desk-mounted eye tracker (RED 120; SensoMotoric Instruments Inc., Boston, MA) was used to measure the diameters of each participant’s pupils at a rate of 60 samples per second. To allow pupil diameter measurement, we had each participant perform the tasks with his or her head fixed on a chinrest. Pupil-diameter data were processed using the same methods as in Experiment 1. Mean pupil-dilation response was computed separately for the readability task and the semantic task.

Statistical analysis

Analyses were carried out using the same methods as in Experiment 1.

Results

The hypothesis that high gain focuses processing on the most salient information predicts that for words from the readability task (in which participants focused on words’ visual shape), recognition memory would be more strongly degraded by font change among participants with high gain than among those with low gain. To examine the effect of font change, we calculated the difference in hit rates as the hit rate for words in the same font minus the hit rate for words in a different font. We found that during the readability task, although font did not affect word recognition for participants with a large pupillary response (i.e., a mean pupil response above the median, indicating low gain), hit-rate difference = −3.1% (SE = 3.6), t(21) = −0.9, p = .39, it did have a significant effect on memory in participants with a low pupillary response (indicative of high gain), hit-rate difference = 12.5% (SE = 3.3), t(21) = 3.7, p < .005. Likewise, both a correlation analysis, r = −.43, p < .005, and a robust regression analysis, t(43) = 3.4, p < .005, revealed a significant correlation between pupil response and hit-rate difference across all participants (Figs. 5a and 5c). These results suggest that higher gain was associated with greater selectivity of processing, and thus memory, of word shape.

Fig. 5.

Results from Experiment 2. The scatterplots (with best-fitting regression lines) show the relationship between hit-rate difference and mean pupil response for (a) the readability task and (b) the semantic task. The hit-rate difference was calculated as the same-font hit rate minus the different-font hit rate. In (c), the mean hit-rate difference is shown separately for participants with low and high mean pupil-dilation responses for the readability task and the semantic task. For each task, participants were divided into low and high pupil-response groups using a median split on the mean pupillary response during the task. Error bars indicate ±1 SEM.

Moreover, this effect was not evident for words for which participants performed the control semantic task; in this case, the correlation between pupil response and hit-rate difference was not significant, r = −.05, p = .76. Thus, we found a difference between the correlations with pupil response measured for the readability and semantic tasks, z = 1.92, p = .05, and a robust regression analysis revealed a significant interaction between pupil response and task in predicting the hit-rate difference, t(86) = 3.2, p < .005 (Figs. 5a and 5b). Finally, pupil response did not correlate significantly with general recognition performance, r = −.01, p = .97, which indicates that pupillary indices of gain primarily reflected an interaction with the distribution of attention rather than overall task engagement. Together, these results indicate that high gain amplified the specificity of memory to the stimulus features to which participants’ attention was directed by the experimental task.

Discussion

In the experiments reported here, the degree to which perception and memory were selectively focused on the most salient sources of information was correlated with variations in pupillary indices of neural gain. Our priming results demonstrate that gain increases the focus on the most salient information regardless of the source of that information (i.e., visual or semantic). Our recognition-memory results further suggest that gain interacted with saliency regardless of whether saliency was determined by automatic processes, as in the case of priming, or by voluntary attention in response to task demands.

We jointly manipulated the strength and immediacy of particular sources of information. However, our simulations suggest that high gain should similarly favor stronger inputs and earlier inputs. We note also that the focusing effect of gain in our simulations was driven mainly by amplification of lateral inhibition, which has been suggested to underlie winner-takes-all dynamics (Coultrip, Granger, & Lynch, 1992). However, lateral inhibition cannot explain the faster reaction times associated with pupillary indices of higher gain; these faster reaction times suggest amplification of excitatory signals as well (amplified excitation could be feed-forward or recurrent, as in Usher & Davelaar, 2002). Future work could clarify the contributions of these different factors to selectivity in information processing.

Our conclusions require several qualifications. The relationship between pupil diameter and behavior in our experiments was also evident in changes across time for individual participants, but only indirectly, through the relationship of both measures with reaction time. In addition, although there is considerable evidence to support our pupillary measure of gain, the precise relationship between pupil-dilation responses, central norepinephrine activity, and brain-wide fluctuations of gain has yet to be fully established. In particular, we cannot rule out the possibility that effects that are associated with high pupil dilations are driven by transient LC-NE activity, though we note that anticorrelations between pupil dilation and baseline pupil diameter are not thought to reflect variations in transient LC-NE activity (Joshi et al., 2016). Additional research is needed to dissociate the transient and sustained components of pupillary or LC-NE activity.

In sum, our findings suggest that neural gain modulates the trade-off between focus and breadth in information processing: High gain causes people to focus on the most salient features of stimuli, whereas low gain favors more even-handed processing of all available features. Appropriate control over this trade-off is essential in people’s daily activities, and its failure could underlie several neuropsychiatric conditions. In particular, impaired ability to focus is a hallmark of learning disabilities (Richards, Samuels, Turnure, & Ysseldyke, 1990; Tarver, Hallahan, Kauffman, & Ball, 1976), whereas the opposite end of the spectrum, an excessively narrow focus, is thought to be a fundamental feature of autism spectrum disorders (Happé, 1996). Our findings implicate gain as a potential underlying mechanism for this important dimension of individual differences and provide a practical way of measuring it.

Footnotes

Action Editor

Eddie Harmon-Jones served as action editor for this article.

Declaration of Conflicting Interests

The authors declared that they had no conflicts of interest with respect to their authorship or the publication of this article.

Funding

This project was supported by the Howard Hughes Medical Institute (E. Eldar) and the John Templeton Foundation (J. D. Cohen, Y. Niv).

Supplemental Material

Additional supporting information can be found at

Notes

References

Arthur

Jr. Doverspike

(1992). Locus of control and auditory selective attention as predictors of driving accident involvement: A comparative longitudinal investigation. Journal of Safety Research, 23, 73–80.

Aston-Jones

Cohen

J. D.

(2005). An integrative theory of locus coeruleus-norepinephrine function: Adaptive gain and optimal performance. Annual Review of Neuroscience, 28, 403–450.

Baddeley

A. D.

(1972). Selective attention and performance in dangerous environments. British Journal of Psychology, 63, 537–546.

Cohen

J. D.

Dunbar

McClelland

J. L.

(1990). On the control of automatic processes: A parallel distributed processing account of the Stroop effect. Psychological Review, 97, 332–361.

Coltheart

(1981). The MRC psycholinguistic database. The Quarterly Journal of Experimental Psychology Section A, 33, 497–505.

Coultrip

Granger

Lynch

(1992). A cortical model of winner-take-all competition via lateral inhibition. Neural Networks, 5, 47–54.

Easterbrook

J. A.

(1959). The effect of emotion on cue utilization and the organization of behavior. Psychological Review, 66, 183–201.

Einhäuser

Stout

Koch

Carter

(2008). Pupil dilation reflects perceptual selection and predicts subsequent stability in perceptual rivalry. Proceedings of the National Academy of Sciences, USA, 105, 1704–1709.

Eldar

Cohen

J. D.

Niv

(2013). The effects of neural gain on attention and learning. Nature Neuroscience, 16, 1146–1153.

10.

Gilzenrat

M. S.

Nieuwenhuis

Jepma

Cohen

J. D.

(2010). Pupil diameter tracks changes in control state predicted by the adaptive gain theory of locus coeruleus function. Cognitive, Affective, & Behavioral Neuroscience, 10, 252–269.

11.

Graf

Ryan

(1990). Transfer-appropriate processing for implicit and explicit memory. Journal of Experimental Psychology: Learning, Memory, and Cognition, 16, 978–992.

12.

Green

C. S.

Bavelier

(2003). Action video game modifies visual selective attention. Nature, 423, 534–537.

13.

Happé

F. G.

(1996). Studying weak central coherence at low levels: Children with autism do not succumb to visual illusions. A research note. Journal of Child Psychology and Psychiatry, 37, 873–877.

14.

Hoeks

Levelt

W. J.

(1993). Pupillary dilation as a measure of attention: A quantitative system analysis. Behavior Research Methods, Instruments, & Computers, 25, 16–26.

15.

Holland

P. W.

Welsch

R. E.

(1977). Robust regression using iteratively reweighted least-squares. Communication in Statistics: Theory and Methods, 6, 813–827.

16.

Jepma

Nieuwenhuis

(2011). Pupil diameter predicts changes in the exploration–exploitation trade-off: Evidence for the adaptive gain theory. Journal of Cognitive Neuroscience, 23, 1587–1596.

17.

Joshi

Kalwani

R. M.

Gold

J. I.

(2016). Relationships between pupil diameter and neuronal activity in the locus coeruleus, colliculi, and cingulate cortex. Neuron, 89, 221–234.

18.

Kucˇera

Francis

W. N.

(1967). Computational analysis of present-day American English. Providence, RI: Dartmouth Publishing Group.

19.

Lambert

Wells

Kean

(2003). Do isoluminant color changes capture attention? Perception & Psychophysics, 65, 495–507.

20.

Lucas

(2000). Semantic priming without association: A meta-analytic review. Psychonomic Bulletin & Review, 7, 618–630.

21.

McClelland

J. L.

Rumelhart

D. E.

(1981). An interactive activation model of context effects in letter perception: I. An account of basic findings. Psychological Review, 88, 375–407.

22.

Morris

C. D.

Bransford

J. D.

Franks

J. J.

(1977). Levels of processing versus transfer appropriate processing. Journal of Verbal Learning and Verbal Behavior, 16, 519–533.

23.

Murphy

P. R.

Robertson

I. H.

Balsters

J. H.

O’connell

R. G.

(2011). Pupillometry and P3 index the locus coeruleus–noradrenergic arousal function in humans. Psychophysiology, 48, 1532–1543.

24.

Reas

Fry

(2007). Processing: A programming handbook for visual designers and artists. Cambridge, MA: MIT Press.

25.

Richards

G. P.

Samuels

S. J.

Turnure

J. E.

Ysseldyke

J. E.

(1990). Sustained and selective attention in children with learning disabilities. Journal of Learning Disabilities, 23, 129–136.

26.

Richler

J. J.

Cheung

O. S.

Gauthier

(2011). Holistic processing predicts face recognition. Psychological Science, 22, 464–471.

27.

Rousseeuw

P. J.

Leroy

A. M.

(2005). Robust regression and outlier detection. Hoboken, NJ: John Wiley & Sons.

28.

Servan-Schreiber

Printz

Cohen

J. D.

(1990). A network model of catecholamine effects: Gain, signal-to-noise ratio, and behavior. Science, 249, 892–895.

29.

Tarver

S. G.

Hallahan

D. P.

Kauffman

J. M.

Ball

D. W.

(1976). Verbal rehearsal and selective attention in children with learning disabilities: A developmental lag. Journal of Experimental Child Psychology, 22, 375–385.

30.

Usher

Davelaar

E. J.

(2002). Neuromodulation of decision and response selection. Neural Networks, 15, 635–645.

31.

Waterhouse

B. D.

Moises

H. C.

Woodward

D. J.

(1980). Noradrenergic modulation of somatosensory cortical neuronal responses to iontophoretically applied putative neurotransmitters. Experimental Neurology, 69, 30–49.

32.

Waterhouse

B. D.

Moises

H. C.

Yeh

H. H.

Geller

H. M.

Woodward

D. J.

(1984). Comparison of norepinephrine- and benzodiazepine-induced augmentation of Purkinje cell responses to gamma-aminobutyric acid (GABA). Journal of Pharmacology and Experimental Therapeutics, 228, 257–267.

33.

Waterhouse

B. D.

Woodward

D. J.

(1980). Interaction of norepinephrine with cerebrocortical activity evoked by stimulation of somatosensory afferent pathways in the rat. Experimental Neurology, 67, 11–34.

Supplementary Material

Please find the following supplemental material available below.

For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.

For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.

0.00 MB

0.23 MB