Abstract
It has been debated whether human visual working memory is limited by the number of items or the precision with which they are represented. In the research reported here, we show that the precision of working memory can be flexibly and willfully controlled, but only if the number of retained items is low. Electroencephalographic recordings revealed that a neural marker for visual working memory (contralateral delay activity, or CDA) that is known to increase in amplitude with the number of retained items was also affected by the precision with which items were retained. However, willfully enhanced precision increased CDA amplitude only when the number of retained items was low. These results show that both the number and the (willfully controlled) precision of retained items constrain visual working memory: People can enhance the precision of their visual working memory, but only for a few items.
Human visual working memory (the system that actively retains visual information over a short-term delay) is limited in the amount of information it can retain (Cowan, 2001; Luck & Vogel, 1997; McNab & Klingberg, 2008; Vogel & Machizawa, 2004; Vogel, McCollough, & Machizawa, 2005). Several different accounts to explain this fact have arisen. Some of these accounts propose that visual working memory has a discrete limit on the number of items it can retain, or a limited number of available working memory “slots” (Anderson, Vogel, & Awh, 2011; Awh, Barton, & Vogel, 2007; Barton, Ester, & Awh, 2009; Fukuda, Awh, & Vogel, 2010; Luck & Vogel, 1997; Vogel & Machizawa, 2004; Zhang & Luck, 2008). Other accounts conceptualize visual working memory as a more dynamic resource (Bays, Catalao, & Husain, 2009; Bays & Husain, 2008; Gorgoraptis, Catalao, Bays, & Husain, 2011; Huang, 2010; Wilken & Ma, 2004). For instance, it has been suggested that the precision with which items are retained in visual working memory, rather than only their number, may be limited. Hybrids of discrete-slot and dynamic-resource models that envisage visual working memory resources as flexibly allocated but constrained by both the number of items and the precision of their representation might therefore be possible (Alvarez & Cavanagh, 2004; Buschman, Siegel, Roy, & Miller, 2011; Machizawa & Driver, 2011; Xu & Chun, 2006).
To date, there is no decisive evidence indicating whether people can willfully vary the precision with which they retain particular visual items and, if so, whether this precision is constrained by the number of items retained. It has been envisaged that when items differ in appearance, both their number and their visual complexity may influence visual working memory performance (Alvarez & Cavanagh, 2004; but see Awh et al., 2007, and Barton et al., 2009). In the research reported here, we examined whether people can vary the precision of representation in visual working memory when the physical properties (e.g., complexity) of the items to be retained are held constant. We did so by manipulating expectancies about the precision of retained information that would likely be required to perform an orientation discrimination after a delay.
We used a new orientation-discrimination paradigm in which participants could anticipate whether a fine (15°) or coarse (45°) discrimination would likely be required after a delay (see Fig. 1). 1 In our initial, purely behavioral study (Experiment 1), the color of memoranda in the initial sample display correctly indicated on 67% of trials (congruent trials) whether the subsequent discrimination would require fine or coarse precision. On the remaining 33% of trials (intermediate trials), discrimination of the change in the orientation of the probe (a 30° change) required intermediate precision, regardless of the color of memoranda in the sample. Participants were informed that 15° and 45° rotations would occur but were not told about this intermediate rotation. We tested whether performance improved on these intermediate trials when participants had anticipated having to make a judgment that required fine, rather than coarse, precision. Improved performance in this case would imply that people can vary the precision with which they retain information about objects’ orientation, whereas unchanged performance would imply that there is a discrete limit on the number of items that can be retained in visual working memory, regardless of the precision of the retained items.
Trial sequence (a) and schematic illustrating the design of the study (b). On each trial, a precue indicated which hemifield in a subsequent sample display would contain items to be retained. After an interstimulus interval, the sample display was presented. The sample display contained either four bars (not shown) or eight bars (shown) in varying orientations; the bars were either all red (shown) or all green (not shown). After a short delay, a probe display was presented. The probe display contained the same number of bars in the same locations as the sample display did; all but one of the bars were black, and the nonblack, target bar was rotated. Participants judged whether the target was rotated clockwise or counterclockwise relative to its counterpart in the sample display. On congruent trials (b), the color of the bars in the sample display correctly indicated whether the target in the probe display would be rotated 15° (fine discrimination) or 45° (coarse discrimination). (In this illustration, green bars indicate coarse precision, and red bars indicate fine precision.) On intermediate trials, the target was rotated 30°, regardless of the color of the bars in the sample display.
In a subsequent electroencephalography (EEG) study (Experiment 2), we collected EEG data while participants performed a similar orientation-discrimination task. We tested whether contralateral delay activity (CDA; McCollough, Machizawa, & Vogel, 2007; Vogel et al., 2005; Vogel & Machizawa, 2004), which is known to increase in amplitude with the number of items retained in visual working memory, also increased in amplitude when information about the items’ orientation was retained with more precision because participants anticipated making a fine discrimination after the delay.
Experiment 1
Method
Participants
Twelve healthy young adult participants (ages 19–35 years, M = 21.02 years) took part in this experiment. All participants reported having normal or corrected visual acuity, passed the Ishihara test for color blindness, and gave informed consent
Procedure
Experiment 1 comprised eight blocks of 48 trials each; there were a total of 96 trials for each set size at each level of expected precision. A given trial type occurred the same number of times in each block; the order of conditions in each block was randomized across participants. Figure 1a depicts the time course of the trials. On each trial, a 200-ms precue indicated which hemifield in the upcoming sample display would contain items to be retained. The sample display appeared 300 to 500 ms later, for 200 ms. There were either four or eight bars in the sample display (two or four bars, respectively, in each hemifield); the bars were either all red or all green. Note that because participants attended to only one hemifield, the set size was 2 items for four-bar displays and 4 items for eight-bar displays. Each bar was randomly assigned an orientation from a total of 12 orientations (orientations ranged from 5° to 170° in 15° intervals; thus, canonical vertical, horizontal, and diagonal orientations were excluded). The color of the bars in the sample display indicated whether fine or coarse precision would likely be required for the upcoming orientation discrimination (the assignment of the two colors to level of expected precision was counterbalanced across participants).
After a 1,300-ms delay, the probe display appeared for 2,500 ms, followed by a blank display for 300 to 500 ms. (The stimulus onset asynchrony between the sample and the probe was fixed at 1,500 ms.) The probe display contained the same number of bars as the sample display. One bar (the probe target) was the same color as the corresponding bar in the sample display; the other bars in the probe display were black. The nontarget, black bars retained the orientations of the corresponding bars in the sample display. The probe target was rotated clockwise or counterclockwise (equiprobable across trials) relative to the corresponding bar in the sample: 15° rotation for fine discriminations, 45° rotation for coarse discriminations, or 30° rotation for intermediate discriminations (see Fig. 1b). Two thirds of the trials were congruent, and the remaining one third were intermediate. Participants had to judge whether the probe target was rotated clockwise or counterclockwise.
In each hemifield, the oriented bars were presented in two symmetrical sectors measuring 20° of polar angle (one in the upper quadrant and the other in the lower quadrant), as in a prior study (Machizawa & Driver, 2011). Each item was presented between eccentricities of 4° to 8° of visual angle from central fixation, and items were separated from each other by more than 2° visual angle. Each bar was 1.5° long and 0.5° wide and had rounded ends. Either one or two bars were presented within one quadrant of each hemifield.
Results
Figure 2a shows the mean proportion of correct responses for congruent trials. Performance showed the usual effect of set size, such that performance was better for the set size of 2 items than for the set size of 4 items, F(1, 11) = 8.58, p < .05. In addition, performance was better on trials requiring coarse discrimination than on trials requiring fine discrimination, F(1, 11) = 14.31, p < .005. There was no interaction between expected precision and set size, F(1, 11) = 0.48, p = .50.
Results from Experiment 1: mean proportion of correct responses on (a) congruent and (b) intermediate trials. Results are shown as a function of set size and the expected level of precision required for the task (fine or coarse). On congruent trials, participants were cued to expect the correct level of precision required for the task, and on intermediate trials, they were cued to expect that discrimination would require either fine or coarse precision, rather than the intermediate precision that the discrimination actually required. Asterisks indicate significant differences (p < .05 or better) between conditions. Error bars indicate standard errors of the mean.
Our more important results were for the trials that required intermediate precision (see Fig. 2b). We again found the expected effect of set size, such that performance was better for the set size of 2 items than for the set size of 4 items, F(1, 11) = 27.89, p < .001. Critically, however, there was an interaction between expected precision and set size, F(1, 11) = 8.08, p < .05. For the set size of 2 items, performance was better when participants had anticipated making a judgment that required fine precision rather than coarse precision, t(11) = 3.47, p < .001, but performance did not differ between the two levels of expected precision when the set size was 4 items, t(11) = 0.65, p = .55.
The fact that performance was better for intermediate discriminations with the set size of 2 items, but not with the set size of 4 items, when participants anticipated that fine rather than coarse precision would be required implies that people can willfully vary the precision with which they retain visual information, provided the number of items is not too demanding for working memory capacity (i.e., precision can be varied for 2, but not 4, items). In Experiment 2, we addressed the same issue, but we used EEG data to examine CDA amplitude over the delay period between the offset of the sample and the onset of the probe, during which encoded items are retained in memory. The CDA, an interhemispheric amplitude difference that emerges during the delay period (while observers maintain visual representations), has been shown to increase in amplitude with the number of items retained in visual working memory (Vogel & Machizawa, 2004; see also Ikkai, McCollough, & Vogel, 2010; McCollough et al., 2007; Vogel et al., 2005). We tested whether the CDA also increased when participants retained a given number of items with greater precision and whether, as in Experiment 1, this effect of precision would emerge only when the set size was small (i.e., 2 items).
Experiment 2
Method
Participants
Twenty young adults (ages 19–35 years, M = 24.68 years) were separately recruited for Experiment 2. All participants reported having normal or corrected vision and gave informed consent.
Behavioral procedure
The paradigm of Experiment 2 was similar to that of Experiment 1, with the following exceptions. The experiment comprised 16 blocks of 48 trials each; there were a total of 192 trials for each set size at each level of expected precision. We did not include intermediate discriminations because CDA amplitude was assessed during the delay period before the actual discrimination on each trial. Hence, all trials were congruent trials (i.e., participants correctly anticipated the precision that would be required for discrimination on each trial). To maximize participants’ certainty about the level of precision that would be required on each trial, and thereby to better determine any effect of expected precision on the CDA, we blocked trials by level of required precision (fine or coarse). Note that because the CDA is a lateralized difference waveform between hemispheres that depends on cued side, which varied from trial to trial, all raw event-related potentials (i.e., all brain activity tied to the onset of the sample) are subtracted out. Thus, compared with raw event-related potentials, the CDA better isolates related cognitive components that should be sensitive to the number of retained objects from the attended side (see McCollough et al., 2007).
Electrophysiological procedure
EEG was continuously recorded with a sampling rate of 512 Hz using an ActiveTwo system (BioSemi, Amsterdam, The Netherlands). Recordings were taken from 64 active electrodes placed in accord with the international 10-20 layout. Two mastoid channels, as well as vertical and horizontal electrooculogram (EOG) channels, were also placed in accordance with standard procedures (McCollough et al., 2007; Vogel et al., 2005; Vogel & Machizawa, 2004). Each participant’s data were filtered off-line at 0.05 Hz with a high-pass finite-impulse-response filter, resampled at a rate of 125 Hz, rereferenced to bilateral mastoid channels, extracted in epochs from 200 ms before to 1,600 ms after sample onset, and normalized relative to a 200-ms time window prior to sample onset. Trials with large eye blinks measured as greater than 50 µV on a vertical EOG channel under the left eye were rejected. Trials on which losses of fixation (> 2° of visual angle) occurred and on which the horizontal EOG amplitude was greater than 25 µV were also rejected. Horizontal EOG data of retained trials did not differ between conditions as a function of set size or expected precision (all Fs < 2.50, all ps > .13). The average CDA component was obtained from the P5/6, P7/8, PO3/4, PO7/8, and O1/2 channels.
Results
Behavioral results
As expected, performance was again better for the set size of 2 items than for the set size of 4 items, F(1, 11) = 94.31, p < .05. In addition, performance was better for trials requiring coarse discrimination than for trials requiring fine discrimination, F(1, 11) = 30.83, p < .001. There was no interaction between expected precision and set size, F(1, 11) = 0.32, p = .58. The mean proportion of correct responses was .76 (SD = .11) for coarse-discrimination trials with a set size of 2 items, .72 (SD = .10) for fine-discrimination trials with a set size of 2 items, .66 (SD = .09) for coarse-discrimination trials with a set size of 4 items, and .61 (SD = .07) for fine-discrimination trials with a set size of 4 items.
CDA results
Figure 3a shows grand-averaged CDA waveforms for the four conditions (2 discriminations × 2 set sizes). Figure 3b shows averaged CDA amplitude in each condition during a time window 400 ms to 1,400 ms after the onset of the sample. Critically, there was an interaction between expected precision and set size, F(1, 19) = 7.90, p < .05. Consistent with prior research (Ikkai et al., 2010; Vogel et al., 2005; Vogel & Machizawa, 2004), results showed that on coarse-discrimination trials, CDA amplitude was greater for the set size of 4 items than for the set size of 2 items, t(19) = 2.39, p < .05; however, on fine-discrimination trials, the CDA did not differ between the two set sizes, t(19) = −0.94, p > .35. The CDA was enhanced by anticipation of fine discrimination for the set size of 2 items, t(19) = 2.19, p < .05, but not for the set size of 4 items (p > .18). There were no main effects of either precision or set size (both Fs < 1).
Results from Experiment 2. The grand-averaged waveforms (a) show contralateral delay activity (CDA) amplitude at the two expected levels of precision, separately for the set size of 2 items (left) and the set size of 4 items (right). The black rectangles along the x-axes mark the time period of the sample display. The gray rectangles along the top of the graphs indicate the duration for which CDA amplitude was averaged. The graph in (b) shows mean CDA amplitude as a function of expected precision (coarse vs. fine) and set size (2 items vs. 4 items). Asterisks indicate significant differences (p < .05) between conditions. Error bars indicate standard errors of the mean.
Discussion
It has been debated whether the number of items to be retained or the precision with which they are retained constrains visual working memory (Alvarez & Cavanagh, 2004; Bays & Husain, 2008; Machizawa & Driver, 2011; Zhang & Luck, 2008). Many accounts of visual working memory capacity consider these two alternatives to be mutually exclusive. We provide a new approach to investigating this issue with a paradigm that varies not only the number of items to be retained in memory, but also the precision with which participants anticipate they need to retain information.
As expected, in Experiment 1, performance was worse for the set size of 4 items than for the set size of 2 items (cf. Luck & Vogel, 1997; Zhang & Luck, 2008), and was also worse for fine discriminations than for coarse discriminations (cf. Bays & Husain, 2008). Critically, on intermediate trials with a set size of 2 items, performance was better when a fine rather than coarse discrimination had been anticipated. This finding implies that people can willfully vary the precision with which they retain visual information, provided that the number of memoranda falls within the limits of working memory capacity (typically estimated to be approximately 3 to 4 items in such tasks; cf. Cowan, 2001; Luck & Vogel, 1997; Vogel & Machizawa, 2004).
To extend our behavioral findings from Experiment 1, we used EEG measures in Experiment 2 to examine the neural signatures of willfully varied precision during the delay period. We designed this experiment to test whether the well-known CDA component (Vogel et al., 2005; Vogel & Machizawa, 2004), which is known to vary with the number of items retained, also varies as a function of the precision of their representation, and, if so, whether willfully varied precision interacts with set size in affecting CDA amplitude.
As in prior research (Ikkai et al., 2010; McCollough et al., 2007; Vogel et al., 2005; Vogel & Machizawa, 2004), CDA amplitude was greater for the set size of 4 items than for the set size of 2 items when coarse precision was required for the orientation discrimination (see note 1). Critically, for the set size of 2 items, but not for the set size of 4 items, CDA amplitude was greater when participants anticipated that the discrimination would require fine precision than when they expected that it would require coarse precision (see Fig. 3b). This pattern of results concerning CDA amplitude is analogous to our behavioral findings in Experiment 1.
Our CDA results go beyond our behavioral results by showing that the flexible control of working memory precision influences the very neural signature of visual working memory previously linked to the number of items retained, and by showing that the neural consequences of enhanced precision extend throughout a delay (see Fig. 3a). Our findings conflict with accounts that assume CDA amplitude reflects only the number of items retained in working memory, because CDA amplitudes were different in the fine-precision and coarse-precision conditions when set size (2 items) was held constant. We can therefore reject a strict slot model of visual working memory (one slot per retained item), according to which the CDA should increase as more slots are utilized, regardless of the precision with which items are retained. If more capacity (reflected by higher CDA amplitude) can be allocated to a given number of items, or slots, then hybrid models of flexible capacity allocation may be correct (Alvarez & Cavanagh, 2004; Buschman et al., 2011; Machizawa & Driver, 2011; Xu & Chun, 2006).
A recent report (Anderson et al., 2011) suggested that the precision of retained information in visual working memory reaches asymptote as the capacity limit for the number of items is approached. This account may accord with our finding that the precision of retained information could be varied at will only when the number of retained items was low. However, our paradigm marks a departure from prior work because it varied expectancies about the precision that would be required for a subsequent probe discrimination while holding the physical properties of stimuli constant. Future research should test whether our findings regarding the precision of information about orientation extend to the precision of information about other visual properties (e.g., location: Bays & Husain, 2008; color: Zhang & Luck, 2008); in addition, future research should combine measures of EEG with behavioral measures (like those used in Experiment 1) to assess individual differences in the willful control of precision and the number of items that can be retained in visual working memory.
It has been vigorously debated whether visual working memory is constrained by the number of items to be retained (Luck & Vogel, 1997; Vogel & Machizawa, 2004; Zhang & Luck, 2008) or by the precision with which they are retained (Bays & Husain, 2008). Our data show conclusively that both factors are critical for visual working memory capacity, as reflected by both performance in a discrimination task and neural activity. People can willfully control the precision with which they maintain visual information, but only if the number of retained items is low (i.e., well within working memory capacity). Thus, both the quality and the quantity of information retained in visual working memory affect CDA amplitude. The quantity of retained items constrains the willfully varied quality, or precision, of their representation in working memory.
Footnotes
Acknowledgements
All authors designed the studies and discussed the findings. M. G. M. and C. C W G. collected and analyzed data. M. G. M. and J. D. wrote the manuscript. J. D. held a Royal Society Anniversary Research Professorship. He passed away on November 28, 2011.
The authors declared that they had no conflicts of interest with respect to their authorship or the publication of this article.
This research was supported by a Wellcome Trust Programme Grant to J. D. and by a UCL Graduate School Award to M. G. M.