Look Before You Leap

Abstract

Simple decisions require the processing and evaluation of perceptual and cognitive information, the formation of a decision, and often the execution of a motor response. This process involves the accumulation of evidence over time until a particular choice reaches a decision threshold. Using a random-dot-motion stimulus, we showed that simply delaying responses after the stimulus offset can almost double accuracy, even in the absence of new incoming visual information. However, under conditions in which the otherwise blank interval was filled with a sensory mask or concurrent working memory load was high, performance gains were lost. Further, memory and perception showed equivalent rates of evidence accumulation, suggesting a high-capacity memory store. We propose an account of continued evidence accumulation by sequential sampling from a simultaneously decaying memory trace. Memories typically decay with time, hence immediate inquiry trumps later recall from memory. However, the results we report here show the inverse: Inspecting a memory trumps viewing the actual object.

Keywords

decision making memory perception cognitive neuroscience better decisions

Traditionally, theories regarding the mechanisms underlying sensory decision making have focused on the processes that occur during the presentation of an actual stimulus. In the laboratory, these decision processes are often studied through the use of noisy sensory stimuli, which subjects are asked to make a simple decision about. These stimuli provide a particularly robust system for studying perceptual decision making, as they allow for the amount of evidence available to be controlled by varying the signal-to-noise ratio of the stimulus.

Experiments using such stimuli have provided further support for models of decision making, which propose that decisions are based on the gradual accumulation of evidence (Diederich, 1997; Leite & Ratcliff, 2010; Ratcliff & McKoon, 2008; Ratcliff & Rouder, 1998; Smith & Vickers, 1988; Usher & McClelland, 2001). Decision accuracy (proportion of correct responses) and processing time show an interdependent relationship: Decisions tend to be more accurate if subjects are exposed to the stimulus for longer periods of time (Britten, Shadlen, Newsome, & Movshon, 1992; Burr & Santoro, 2001; Mateeff et al., 2000). An important limitation of these models is that they do not address the role of memory in the decision-making process. Considering that it has been shown that stimulus information can be automatically retained in a high-capacity memory store for several hundred milliseconds (Bradley & Pearson, 2012; Coltheart, 1980; Coltheart, Lea, & Thompson, 1974; Dick, 1974; Long, 1980; Sperling, 1960), it stands to reason that it is possible for information to continue accumulating from a memory representation of the stimulus. In the research reported here, we developed a task to explore how short-term sensory memory might affect the decision-making process.

Method

Ten subjects, 8 of whom were naive to the purpose of the study, participated in this study. All participants had normal or corrected-to-normal vision and provided informed written consent prior to participation. This study was approved by the University of New South Wales School of Psychology ethics committee. Participants were seated at a distance of 57 cm from a 20-in. Sony Multiscan G520 CRT monitor (resolution = 1,024 × 768, refresh rate = 100 Hz). Participants’ heads were stabilized by a chin rest. Stimuli were presented using The Psychophysics Toolbox, Version 3, for MATLAB (The MathWorks, Natick, MA; see Kleiner, Brainard, Pelli, Ingling, Murray, & Broussard, 2007) on a Macintosh MacPro running Mac OS X.

The stimuli used were dynamic random-dot-motion (RDM) displays, commonly used in decision-making research (Churchland, Kiani, & Shadlen, 2008; Roitman & Shadlen, 2002; Shadlen & Newsome, 1996). The RDM stimulus consisted of white dots, each a 2 × 2 pixel square, moving at a speed of 6° per second on a black background, with an average dot density of 2.7 dots/deg². On each trial, the direction of motion was randomly chosen from a pool of an equal number of leftward and rightward directions. The motion coherence of the stimulus was randomly selected from one of five possible coherence levels (3%, 9%, 16%, 24%, and 32%). This range of coherence levels was determined during pilot testing and was chosen to span the range of behavioral ability. Three uncorrelated sequences of dot movement were generated, and frames were interleaved so that each frame was correlated only with a frame that was either three frames back or three frames ahead in the sequence and not the subsequent frame (Roitman & Shadlen, 2002; Shadlen & Newsome, 2001; see Fig. 1).

Fig. 1.

Trial design. In all conditions, each trial began with the presentation of a central fixation point for 1,000 ms. This was followed by a random-dot motion (RDM) stimulus presented for 250 ms. Participants were instructed to indicate the direction of the coherent motion in the stimulus as quickly and as accurately as they could after a tone sounded. In the (a) no-delay condition, the tone was played concurrently with the stimulus onset. A response window of 750 ms followed stimulus presentation, and missed trials were repeated at the end of each block. In the (b) masked-delay condition, the stimulus presentation was followed by a 0% coherent RDM mask for 100 to 800 ms, followed by the tone and response. In the (c) blank-delay condition, the stimulus presentation was followed by a blank-delay period (100–800 ms), and then the tone sounded, indicating the time to respond.

Trials were arranged in blocks of 100, and each block consisted of a given delay period. All conditions were pseudorandomized to avoid possible learning effects. Each participant completed six blocks for each time condition (100, 200, 400, 600, and 800 ms) for both the blank- and masked-delay conditions, as well as eight blocks of trials for the no-delay condition. To avoid any confusion between the stimulus and mask, we had the center of the bull’s-eye fixation point change color—from yellow during the stimulus presentation to black during the mask presentation. This was done to aid subjects in identifying in the stimulus. Subjects were further instructed to base their decision on the RDM that was displayed in the presence of the yellow fixation point only.

In a second experiment, the dots in the mask stimulus were increased in size to 4 times that of dots in the test stimulus. The range of direction of motion of these dots was restricted, in that they could not travel within a 20° band surrounding the horizontal axis. The RDM stimulus was followed by the presentation of either a blank delay or a sensory mask for 50, 100, 200, 300, or 500 ms. In addition, the masked- and blank-delay conditions were pseudorandomly interleaved and blocked by delay time. At the end of the delay period, a short tone was sounded, and participants had 750 ms to respond. Subjects completed five experimental blocks, with each block consisting of 100 trials of a single delay duration. The order of presentation of the blocks of trials was pseudorandomized.

In a subsequent experiment, the display time of the RDM stimuli was manipulated. In the display condition, the RDM stimulus was presented for 150, 200, 250, 400, or 800 ms, after which a short tone was sounded and participants had a 750-ms window to make a response. For the blank-delay condition, the RDM stimulus was presented for 250 ms, followed immediately by the presentation of a black screen for 50, 100, 200, or 400 ms. At the end of this delay period, a short tone was sounded, and participants had 750 ms in which to respond. The parameters were arranged in blocks of 100 trials with the five coherence levels, randomly interleaved, for a given condition at a given stimulus duration. Each session comprised an average of 10 experimental blocks, with each block consisting of 100 trials in a single condition. The order of presentation of the blocks of trials was pseudorandomized. Note that the two conditions were not tested in an interleaved manner (all other parameters were pseudorandomly interleaved within each condition). In all experiments, any data directly compared were collected during the same time period to avoid any effects of learning.

In a final experiment, each trial was preceded by the presentation of either a 5-digit number string (low cognitive load) or a 10-digit number string (high cognitive load). An RDM stimulus was then presented for 250 ms, followed by either a blank or a masked delay for 250 ms. The mask-stimulus dots were 4 times larger than the RDM dots, and they did not travel within a 20° band surrounding the horizontal axis (see the previous description of the method). Following the delay, a short tone was sounded and participants had 750 ms in which to respond. After a response, the previously presented number was presented again either unchanged or with one digit changed. Participants then had to indicate whether the number was the same as or different from the one that they had previously seen.

Data Analysis

A Weibull psychometric function, which describes the relationship between the level of a stimuli’s motion strength and a subject’s probability of a correct response, was fitted to the data. An estimate of each subject’s threshold was obtained from the acquired functions at 81.6% correct. Because individual threshold means tended to naturally vary, with some participants displaying greater sensitivity to the motion stimuli in all conditions, individual thresholds were normalized by dividing each participant’s threshold for each condition by his or her overall average threshold for the relevant conditions. Reaction times were normalized by dividing each participant’s mean reaction time for each coherence by his or her average reaction time for the lowest coherence (3%; see Fig. S4 in the Supplemental Material available online). This procedure was performed to compare the reaction times across the different motion coherences. Otherwise, reaction times were normalized by dividing each participant’s times by his or her overall mean (see Fig. S5).

Results

Psychometric functions were fitted to the data to attain an estimate of the discrimination threshold (see top row of Fig. 2a for example data fits; see Fig. S2 for individual subjects’ data). These functions demonstrate that the probability of a correct decision in the blank-delay condition was generally higher than the probability of a correct decision in the no-delay and masked-delay conditions for most levels of motion strength. Although no new visual information was available during the blank delay, we found that decision accuracy was significantly higher in the blank-delay condition compared with the no-delay condition (p = .009; see Fig. 2b). We also found that decision accuracy was significantly lower when a noncoherent RDM mask was presented in place of the blank screen (p = .002; see bottom row of Fig. 2a and blue triangles in Fig. 2b); the difference in accuracy levels between no-delay responses and masked-delay responses was not significant (p = .21; see Fig. 2). The same pattern of significance was found with the nonnormalized raw data. These data suggest that in the absence of any new visual input, a representation of the motion stimulus held in memory allows for the continued extraction of evidence, thereby improving decision accuracy almost twofold (see Fig. 2c for ratios of thresholds). A control experiment, in which we compared performance in a condition with the same parameters as the no-delay condition with performance in a condition in which no tone was played (with all other parameters identical) demonstrated that the presence of the tone did not significantly affect performance, p = .335 (see Fig. S5).

Fig. 2.

Results showing the effect of sensory memory on decision accuracy. The graphs in (a) show the mean proportion of correct responses for 5 subjects, plotted for the five different levels of motion coherence. The top row shows psychometric functions for both the blank-delay (red) and no-delay (black) conditions, whereas the bottom row shows functions for blank-delay (red) and masked-delay (blue) conditions, separately for delays of 100, 200, and 400 ms. Error bars indicate standard errors of the means. (Here, the data were fitted with a Weibull function; see Fig. S2 in the Supplemental Material for individual functions.) The graph in (b) shows mean normalized thresholds for 5 subjects as a function of delay condition (see Fig. S1 for raw-data plots). Error bars indicate standard errors of the mean. The graph in (c) shows the ratio of thresholds in the blank-delay condition (400-ms delay) compared with the no-delay condition, plotted for 5 participants (black bars). Each participant’s threshold for the no-delay condition was divided by the participant’s threshold in the blank-delay condition. The white bar represents the overall mean (1.85).

To avoid stimulus-mask confusion in the masked-delay condition, we made the fixation point change color during the stimulus presentation. However, we were still concerned that participants might be mistakenly basing their decisions—at least in part—on the 0% mask stimulus, given that the test and mask stimulus were similar. Accordingly, we conducted a second experiment in which the dots in the mask stimulus were increased in size to 4 times that of dots in the test stimulus. Increasing the visual size of the dots in this manner changed the visual appearance of the mask stimulus categorically, greatly reducing the probability of any stimulus confusion. Furthermore, the range of random-motion directions of the mask dots was restricted so that the mask stimulus did not contain any dots travelling within a 20° band surrounding the horizontal axis (see Fig. 3a for the mask schematic). This procedure was used to prevent the decision-making mechanism from continuing to accumulate decision-relevant information (horizontal motion) from the mask stimulus. Thus, the mask did not explicitly contain motion energy along the decision-relevant axis, and the mask stimulus therefore should not drive the same sensory neurons that were involved in making the direction decision. Hence, any reduction in performance due to the presence of the mask stimulus should be related to disruption of a memory trace as opposed to accumulation of erroneous decision information.

Fig. 3.

Controlled motion-direction sensory mask, quantification of evidence-accumulation rates, and a simple model of evidence accumulation from memory. The schematic in (a) shows the 20° band of motion directions around the horizontal midline that was removed in the random-motion mask. The actual mask had more directions than are shown in the diagram. The graph in (b) shows mean normalized thresholds for 5 subjects as a function of time since stimulus offset and delay condition. (See Fig. S3 in the Supplemental Material for raw-data plots.) Error bars indicate standard errors of the means. The graph in (c) shows normalized thresholds for 4 subjects as a function of time since stimulus onset and type of experimental manipulation, plotted for different evidence-accumulation times (total time = stimulus display plus any memory of the display). In the condition in which display time was manipulated, memory time equalled 0. The slopes of the lines fitted to the data for both conditions represent the rate of change with increased processing time (i.e., the rate of evidence accumulation). Error bars indicate standard errors of the means. Note that the two conditions were not tested in an interleaved manner because the comparison of interest was the rate of changes in accuracy within each condition, and accuracy in the two conditions could not be compared directly due to strong perceptual-learning effects. (For an additional graph showing results found using an alternative normalization method, see Figure S4.) The information-flow diagram (d) illustrates the proposed relationship between the amount of evidence available in the sensory-memory store and the rate of evidence accumulation. A visual stimulus is presented for 250 ms, followed by a rapidly decaying sensory memory trace (blue plot). Information is available while the stimulus is present and during the memory trace after the stimulus offset. The red plot shows the accumulation of noisy information toward a decision variable. Evidence accumulation is delayed for 200 ms from stimulus onset (Kiani, Hanks, & Shadlen, 2008; Roitman & Shadlen, 2002). The decay of the memory trace was modeled by an exponential decay function (Lu, Neuse, Madigan, & Dosher, 2005): EvidenceA = Se^-τ as the fast-decaying representation that reflects the initial sensory stimulus, in which τ is the time constant showing the rate of decay for sensory memory and EvidenceA is the amount of evidence available at any time point. The evidence accumulates as a function of the area under the plot (evidence available ± a random noise variable).

Additionally, in this control experiment, we pseudorandomly interleaved the masked- and blank-delay conditions, blocking by delay time. Thus, in each block, an RDM stimulus was presented for 250 ms, followed by the presentation of either a blank delay or a sensory mask for 50 to 500 ms. This procedure eliminated potential biases that may have occurred as a result of the predictability of the condition in the first experiment. Consistent with the findings of the first experiment, a repeated measures analysis of variance revealed that decision accuracy in the blank-delay condition was significantly greater than in the mask condition (p = .007; see Fig. 3b).

We wondered whether high-level cognitive mechanisms might contribute to this memory boost or whether it might be a sensory-based, automatic representation. To investigate this question, we manipulated concurrent cognitive load using a number-based working memory task at high and low load. The decision-making task in this experiment was the same as that previously described: Participants had to hold a short (5 digits in length) or long (10 digits in length) number string in mind while performing the decision-making task with a blank delay or the presentation of a sensory mask. We found that accuracy for the masked-delay condition was the same under both low and high cognitive load, p = .83, whereas accuracy in the blank-delay condition was significantly higher in the low-cognitive-load condition, p = .01 (see Fig. S6). Performance on the high-cognitive-load number working memory task did not differ between the blank-delay (76.4% correct) and masked-delay (75% correct) conditions, p = .40. These results are consistent with findings from a recent study by Persuh, Genzer, and Melara (2012), who demonstrated that information could be successfully consolidated in iconic memory only under low-attentional-load conditions.

These results provide further evidence that the mask stimulus does not reduce accuracy by polluting the sensory evidence stream with erroneous decision information. If this were the case, we would expect accuracy in the masked-delay condition to be lower than in the blank-delay condition irrespective of cognitive load; however, we found that this was true only for the low cognitive-load condition—low cognitive load: p = .01; high cognitive load: p = .83. Furthermore, we would similarly expect there to be no differences in accuracy between the two levels of cognitive load in the blank-delay condition if the difference in performance was attributable solely to the presence of the mask. Rather, these data suggest that performance in the blank-delay condition was enhanced by the opportunity to further accumulate evidence from a representation of the stimulus held in memory.

Longer viewing periods of noisy sensory stimuli result in greater decision accuracy (Britten et al., 1992; Burr & Santoro, 2001; Mateeff et al., 2000), and this can be seen as a proxy of the rate of evidence accumulation. Likewise, we found that longer memory times led to greater decision accuracy. To compare the rate of evidence accumulation for decisions based on perception versus those based on memory, we conducted a fourth experiment to see how accuracy benefited from extra processing time, whether based on perception or memory.

In this experiment, participants were presented with either an RDM stimulus that varied in duration from 100 to 800 ms or a 250-ms RDM stimulus in which the subsequent blank delay varied from 0 to 400 ms. This procedure allowed us to compare decision-accuracy rates as a function of total accumulation time (perceptual-display presentation time plus any memory time; see Fig. 3c). Figure 3c also shows data for manipulations in the display time or blank poststimulus delay time (data were normalized to the mean of each condition).

We performed a linear regression analysis comparing the slopes of the lines fitted to the data of both conditions. The slopes of the linear fits to the data for both conditions represent the rate of change with increased processing time. We found that accuracy improved at a similar rate with more accumulation time (with similar slopes to the linear fits) regardless of whether the information was sourced from a perceptual stimulus (slope = −0.0011 ± 0.0006) or from the representation of one in memory (slope = −0.0013 ± 0.0003). A comparison-of-fits analysis indicated that the slopes of the two lines were not significantly different (p = .73), and neither were the elevations (p = .67). These results suggest that visual information can be retained in a form of memory that, at least over these short periods, allows increases in accuracy similar to those attained by extending the presentation of the sensory stimulus itself.

Given the current data, we interpreted our findings in terms of a sequential-sampling model (Laming, 1968; Link & Heath, 1975; Stone, 1960; Townsend & Ashby, 1983), in which evidence continues to accumulate from a simultaneously decaying memory trace. The model incorporates a period of noisy accumulation from a short-term sensory memory store after the offset of a physical stimulus (see Fig. 3d). The sensory memory store is modeled by an exponential decay over a given period of time (Lu, Neuse, Madigan, & Dosher, 2005). The rate of evidence accumulation is given by the area under the curve (amount of evidence available) and flattens out as the evidence approaches zero. We propose that this simple mnemonic module can be coupled with existing models of decision making (Diederich, 1997; Leite & Ratcliff, 2010; Ratcliff & McKoon, 2008; Ratcliff & Rouder, 1998; Smith & Vickers, 1988; Usher & McClelland, 2001) to incorporate the continued accumulation of evidence from memory.

Discussion

Our results indicate that decision-relevant information can continue to be extracted from a sensory memory store for several hundred milliseconds following stimulus removal. When this memory trace was corrupted by the presentation of a visual mask, or high-level cognitive mechanisms were otherwise occupied, performance gains were lost. Furthermore, we found that the rate of evidence accumulation from the mnemonic trace matched that of actual perception. Consequently, decision accuracy can be greatly improved by simply delaying responses.

Resulaj, Kiani, Wolpert, and Shadlen (2009) developed a task that allowed for the monitoring of changes of mind during decision making and suggested that information can be processed after the brain has already committed itself to a decision. The current results dovetail nicely with this work. We found that with no new sensory information, accuracy can continue to increase for periods of almost half a second, but only when the sensory memory trace or working memory are left undisturbed. The temporal dynamics of this memory facilitation are consistent with the considerable body of research that has demonstrated that after the presentation of a visual stimulus, an observer has access to a rapidly fading stimulus trace, or icon, for several hundred milliseconds or more (Bradley & Pearson, 2012; Coltheart, 1980; Coltheart et al., 1974; Dick, 1974; Long, 1980; Sperling, 1960). This icon retains vestiges of stimulus information that are accessible to the observer but vulnerable to visual masking (Enns & Di Lollo, 2000; Smithson & Mollon, 2006; Spencer, 1969; Turvey, 1973). Hence, iconic memory is a good source candidate for a mechanism that holds information in the system, allowing further decision evidence to be passed on to higher-level cortical areas (Philiastides, Auksztulewicz, Heekeren, & Blankenburg, 2011).

This study represents the first demonstration, to the best of our knowledge, that the short-term advantage of iconic-sensory memory is not only limited to item recollection but also can be used to improve decision accuracy. Further, contrary to previous findings, our results showed that this boost in accuracy increases with longer durations (up to 200–400 ms), as the decaying stimulus memory trace allows for the continued accumulation of evidence in the absence of the stimulus itself. This account is consistent with a primate study conducted by Lemus et al. (2007) in which decisions on a tactile sensory-discrimination task were delayed. The Lemus et al. study suggested that postponing the decision results in the maintenance of the sensory information in working memory during the delay period. This maintenance allows for the continued evaluation and revision of the decision-relevant information for up to 1 s, but it did not lead to higher accuracy. We propose that a similar mechanism allowed for the continued accumulation (beyond maintenance) of sensory evidence in our task: The original stimulus information was encoded in the middle temporal visual area (Gold & Shadlen, 2007) and was maintained in an iconic memory store. The final decision was continuously updated by this information during the blank delay, which improved decision accuracy. However, Lemus et al. (2007) reported a reduction in accuracy with longer delay times, which suggests that the accuracy advantage of short-term sensory memory might not be evident for longer delay periods, such as those equivalent to working memory storage periods.

The improved decision accuracy due to the use of memory cannot be attributed to a speed-accuracy trade-off, given that reaction times in the memory data were actually faster (see Fig. S7). In addition, we do not see any a priori reason why the mask stimulus would alter internal parameters for speed or accuracy, and the use of the tone did not significantly affect performance. Furthermore, these results could provide insight into the mechanisms that drive the speed-accuracy trade-off (Pachella, 1974; Wickelgren, 1977; Wood & Jennings, 1976). In such decision-making tasks, slower responses are thought to be more accurate, in part, because longer periods of evidence accumulation give more evidence for a decision outcome (Palmer, Huk, & Shadlen, 2005). Such a model can easily be extended to include periods of accumulation from a memory store. If short-term sensory memory did not play a central role in this process, then we would expect similar benefits from slower responses in the presence of the sensory mask. Because we found that masked delays did not produce significant improvements in performance, we conclude that sensory memory plays a pivotal role in the increase of accuracy associated with longer processing times.

The current results also cannot be attributed to the sensory mask acting as a backward mask that might impair the visibility of the stimulus (Enns & Di Lollo, 2000; Smithson & Mollon, 2006; Spencer, 1969; Turvey, 1973). If this were the case, then we would expect to see lower accuracy in the masked-delay condition than in the no-delay condition. However, this was not the case: There were no significant differences between accuracy rates in the masked- and no-delay conditions. As the current data show, the use of a sensory mask immediately following a dynamic stimulus decreases the accuracy of responses compared with using a blank delay. This decrease is likely due to the sensory mask disrupting the sensory stimulus trace and thereby preventing the further accumulation of evidence. This result could have important implications for the use of backward masking as a method of controlling the perceptual availability of sensory stimuli (Enns & Di Lollo, 2000; Smithson & Mollon, 2006; Spencer, 1969; Turvey, 1973).

The current results are also not adequately explained by a delay-of-sensory-processing account, which, on the basis of nonhuman primate studies of motion discrimination, proposes that there is an approximately 200-ms delay of decision-related activity in the lateral intraparietal area (Churchland et al., 2008; Roitman & Shadlen, 2002). This account suggests that conditions with a delay should simply benefit from additional response time because the start of processing or accumulation was delayed. However, in this scenario, a sensory mask stimulus should also be delayed and, hence, no difference between the masked- and blank-delay conditions would be predicted. Furthermore, in our second experiment, we demonstrated that when a physically dissimilar sensory mask fills the delay period, performance accuracy is reduced. The only way a delay-of-processing account might explain these data is if the test stimulus was delayed but not the mask stimulus. We find no a priori reason to propose such an account.

Furthermore, given that the mask stimulus in our second experiment did not contain motion travelling along the horizontal axis, it would not drive the same sensory neurons as the test stimulus and, therefore, would not be incorporated into the decision-relevant sensory stream of evidence. We believe that the results from this additional experiment provide further evidence for a sensory-memory-based explanation of our findings. By demonstrating that it is highly unlikely that the mask was either confusing or corrupting the sensory stream of evidence, given that there is no reason that the information stream for the masked- and blank-delay conditions would be processed differently in time, we have also shown that it is unlikely that the advantage of having a blank delay was due solely to the delayed processing of the evidence. Our manipulation of cognitive load during this task provided further evidence against a delay-of-processing account. If a processing delay were the mechanism driving the difference in accuracy between the masked- and blank-delay conditions, then the introduction of a working memory task should not eradicate this effect, as the information in the processing pipeline would still be incorporated into the decision during the delay period. However, we obtained differences in accuracy only under low-cognitive-load conditions. This divergent effect on performance is consistent with the pattern of data we would expect if information were being extracted from an iconic memory store during the blank delay (Persuh et al., 2012).

The recollection of an event from memory pales in comparison with the richly detailed and full-featured immersive sensory experience of the actual event. Hence, short- and long-term memory are often characterized by their decay functions: how much information is lost over time (Loftus, Duncan, & Gehrig, 1992; Pasternak & Greenlee, 2005; Wickelgren, 1972). Here, we reported the counterintuitive finding that subjects were more accurate when they made their responses from memory than when responding at the time of the actual event. This could have important implications for the understanding of memory function, a fundamental aspect of cognition and behavior. We propose that the sensory stimulus was maintained in memory and that this allowed for the continued extraction of the relevant information. There was a marked increase in performance due to memory in all the motion-coherence levels except for the lowest level of 3% (see Fig. S8). This suggests that for motion strengths of 9% coherence and above, the memory trace was capable of holding a representation of the relevant motion and its direction. This may be surprising, given that up to 91% of the stimulus was perceptual noise. It remains to be seen whether both the noise and the signal were stored in the same manner and to the same degree.

It would be interesting for future studies to investigate what other types of ambiguous or noisy sensory and cognitive information can benefit from postpresentation memory representations. The present results demonstrate that short-term sensory memory can hold a representation of a noisy sensory stimulus to the degree that important signal information can be extracted from the representation. These results suggest that performance accuracy can be almost twice as good when decisions are made from memory than when they are made at the time of the actual event.

Footnotes

Acknowledgements

The authors thank Duje Tadin, Matthew Finkbeiner, and Joseph Lappin for helpful discussion or comments on the manuscript, Franco Caramia for technical assistance, and anonymous reviewers for helpful comments.

Declaration of Conflicting Interests

The authors declared that they had no conflicts of interest with respect to their authorship or the publication of this article.

Funding

This work was supported by National Health and Medical Research Council C. J. Martin Fellowship 457146, Career Development Fellowship APP1049596, and Project Grants APP1024800 and APP1046198, awarded to J. Pearson.

Supplemental Material

Additional supporting information may be found at

References

Bradley

Pearson

(2012). The sensory components of high-capacity iconic memory and visual working memory. Frontiers in Psychology, 3, 355. doi:10.3389/fpsyg.2012.00355

Britten

K. H.

Shadlen

M. N.

Newsome

W. T.

Movshon

J. A.

(1992). The analysis of visual motion: A comparison of neuronal and psychophysical performance. Journal of Neuroscience, 12, 4745–4765.

Burr

D. C.

Santoro

(2001). Temporal integration of optic flow, measured by contrast and coherence thresholds. Vision Research, 41, 1891–1899. doi:10.1016/s0042-6989(01)00072-4

Churchland

A. K.

Kiani

Shadlen

M. N.

(2008). Decision-making with multiple alternatives. Nature Neuroscience, 11, 693–702. doi:10.1038/nn.2123

Coltheart

(1980). Iconic memory and visible persistence. Perception & Psychophysics, 27, 183–228.

Coltheart

Lea

C. D.

Thompson

(1974). In defence of iconic memory. Quarterly Journal of Experimental Psychology, 26, 633–641. doi:10.1080/14640747408400456

Dick

A. O.

(1974). Iconic memory and its relation to perceptual processing and other memory mechanisms. Perception & Psychophysics, 16, 575–596.

Diederich

(1997). Dynamic stochastic models for decision making under time constraints. Journal of Mathematical Psychology, 41, 260–274. doi:10.1006/jmps.1997.1167

Enns

J. T.

Di Lollo

(2000). What’s new in visual masking? Trends in Cognitive Sciences, 4, 345–352. doi:10.1016/s1364-6613(00)01520-5

10.

Gold

J. I.

Shadlen

M. N.

(2007). The neural basis of decision making. Annual Review of Neuroscience, 30, 535–574. doi:10.1146/annurev.neuro.29.051605.113038

11.

Kiani

Hanks

T. D.

Shadlen

M. N.

(2008). Bounded integration in parietal cortex underlies decisions even when viewing duration is dictated by the environment. Journal of Neuroscience, 28, 3017–3029. doi:10.1523/jneurosci.4761-07.2008

12.

Kleiner

Brainard

Pelli

Ingling

Murray

Broussard

(2007). What’s new in Psychtoolbox-3? Perception, 36(ECVP Abstract Suppl.). Retrieved from http://www.perceptionweb.com/abstract.cgi?id=v070821

13.

Laming

D. R. J.

(1968). Information theory of choice-reaction times. London, England: Academic Press.

14.

Leite

F. P.

Ratcliff

(2010). Modeling reaction time and accuracy of multiple-alternative decisions. Attention, Perception, & Psychophysics, 72, 246–273. doi:10.3758/app.72.1.246

15.

Lemus

Hernández

Luna

Zainos

Nácher

Romo

(2007). Neural correlates of a postponed decision report. Proceedings of the National Academy of Sciences, USA, 104, 17174–17179. doi:10.1073/pnas.0707961104

16.

Link

S. W.

Heath

R. A.

(1975). A sequential theory of psychological discrimination. Psychometrika, 40, 77–105.

17.

Loftus

G. R.

Duncan

Gehrig

(1992). On the time course of perceptual information that results from a brief visual presentation. Journal of Experimental Psychology: Human Perception and Performance, 18, 530–549. doi:10.1037/0096-1523.18.2.530

18.

Long

G. M.

(1980). Iconic memory: A review and critique of the study of short-term visual storage. Psychological Bulletin, 88, 785–820. doi:10.1037/0033-2909.88.3.785

19.

Z.-L.

Neuse

Madigan

Dosher

B. A.

(2005). Fast decay of iconic memory in observers with mild cognitive impairments. Proceedings of the National Academy of Sciences, USA, 102, 1797–1802. doi:10.1073/pnas.0408402102

20.

Mateeff

Dimitrov

Genova

Likova

Stefanova

Hohnsbein

(2000). The discrimination of abrupt changes in speed and direction of visual motion. Vision Research, 40, 409–415. doi:10.1016/S0042-6989(99)00185-6

21.

Pachella

R. G.

(1974). The interpretation of reaction time in information-processing research. In Kantowitz

(Ed.), Human information processing: Tutorials in performance and recognition (pp. 41–82). Hillsdale, NJ: Erlbaum.

22.

Palmer

Huk

A. C.

Shadlen

M. N.

(2005). The effect of stimulus strength on the speed and accuracy of a perceptual decision. Journal of Vision, 5(5), Article 1. Retrieved from http://www.journalofvision.org/content/5/5/1.full

23.

Pasternak

Greenlee

M. W.

(2005). Working memory in primate sensory systems. Nature Reviews Neuroscience, 6, 97–107. doi:10.1038/nrn1603

24.

Persuh

Genzer

Melara

R. D.

(2012). Iconic memory requires attention. Frontiers in Human Neuroscience, 6, 1–8.

25.

Philiastides

M. G.

Auksztulewicz

Heekeren

H. R.

Blankenburg

(2011). Causal role of dorsolateral prefrontal cortex in human perceptual decision making. Current Biology, 21, 980–983. doi:10.1016/j.cub.2011.04.034

26.

Ratcliff

McKoon

(2008). The diffusion decision model: Theory and data for two-choice decision tasks. Neural Computation, 20, 873–922.

27.

Ratcliff

Rouder

J. N.

(1998). Modeling response times for two-choice decisions. Psychological Science, 9, 347–356. doi:10.1111/1467-9280.00067

28.

Resulaj

Kiani

Wolpert

D. M.

Shadlen

M. N.

(2009). Changes of mind in decision-making. Nature, 461, 263–266. doi:10.1038/nature08275

29.

Roitman

J. D.

Shadlen

M. N.

(2002). Response of neurons in the lateral intraparietal area during a combined visual discrimination reaction time task. Journal of Neuroscience, 22, 9475–9489.

30.

Shadlen

M. N.

Newsome

W. T.

(1996). Motion perception: Seeing and deciding. Proceedings of the National Academy of Sciences, USA, 93, 628–633.

31.

Shadlen

M. N.

Newsome

W. T.

(2001). Neural basis of a perceptual decision in the parietal cortex (area LIP) of the rhesus monkey. Journal of Neurophysiology, 86, 1916–1936.

32.

Smith

P. L.

Vickers

(1988). The accumulator model of two-choice discrimination. Journal of Mathematical Psychology, 32, 135–168. doi:10.1016/0022-2496(88)90043-0

33.

Smithson

Mollon

(2006). Do masks terminate the icon? Quarterly Journal of Experimental Psychology, 59, 150–160.

34.

Spencer

T. J.

(1969). Some effects of different masking stimuli on iconic storage. Journal of Experimental Psychology, 81, 132–140. doi:10.1037/h0027472

35.

Sperling

(1960). The information available in brief visual presentation. Psychological Monographs, 74(11, Whole No. 498), 1–29.

36.

Stone

(1960). Models for choice-reaction time. Psychometrika, 25, 251–260. doi:10.1007/bf02289729

37.

Townsend

J. T.

Ashby

F. G.

(1983). The stochastic modeling of elementary psychological processes. Cambridge, England: Cambridge University Press.

38.

Turvey

M. T.

(1973). On peripheral and central processes in vision: Inferences from an information-processing analysis of masking with patterned stimuli. Psychological Review, 80, 1–52. doi:10.1037/h0033872

39.

Usher

McClelland

J. L.

(2001). The time course of perceptual choice: The leaky, competing accumulator model. Psychological Review, 108, 550–592. doi:10.1037/0033-295x.108.3.550

40.

Wickelgren

W. A.

(1972). Trace resistance and the decay of long-term memory. Journal of Mathematical Psychology, 9, 418–455. doi:10.1016/0022-2496(72)90015-6

41.

Wickelgren

W. A.

(1977). Speed-accuracy tradeoff and information processing dynamics. Acta Psychologica, 41, 67–85. doi:10.1016/0001-6918(77)90012-9

42.

Wood

Jennings

(1976). Speed-accuracy tradeoff functions in choice reaction time: Experimental designs and computational procedures. Attention, Perception, & Psychophysics, 19, 92–102. doi:10.3758/bf03199392

Supplementary Material

Please find the following supplemental material available below.

For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.

For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.

0.00 MB

1.14 MB