Abstract
Past events, particularly emotional experiences, are often vividly recollected. However, it remains unclear how qualitative information, such as low-level visual salience, is reconstructed and how the precision and bias of this information relate to subjective memory vividness. Here, we tested whether remembered visual salience contributes to vivid recollection. In three experiments, participants studied emotionally negative and neutral images that varied in luminance and color saturation, and they reconstructed the visual salience of each image in a subsequent test. Results revealed, unexpectedly, that memories were recollected as less visually salient than they were encoded, demonstrating a novel memory-fading effect, whereas negative emotion increased subjective memory vividness and the precision with which visual features were encoded. Finally, memory vividness tracked both the precision and remembered salience (bias) of visual information. These findings provide evidence that low-level visual information fades in memory and contributes to the experience of vivid recollection.
Episodic memories are marked by the recollection of a unique event, including specific perceptual details (Tulving, 2002). Much research has characterized the processes supporting recall of high-level visual-event features, such as the location, people, and objects involved. Yet episodic memories vary not only in their content but also in their quality; some experiences continue to burn bright in memory, whereas others seem to fade. To what extent is forgetting associated with changes to the low-level visual quality of a memory, and how does this map on to our subjective experience of remembering?
Subjective judgments, such as vividness, have been frequently used as an overall measure of episodic memory, reflecting an ability to evaluate the richness of a particular composition of memory details (Bonnici, Richter, Yazar, & Simons, 2016; Ford & Kensinger, 2016; Kuhl & Chun, 2014; St-Laurent, Abdi, & Buchsbaum, 2015). Yet the vast majority of studies have not assessed which specific characteristics underpin vivid recollection. Research has traditionally relied on discrete measures, such as the quantity of high-level event details (e.g., Horner & Burgess, 2014; Uncapher, Otten, & Rugg, 2006; see Fig. 1), to indicate the objective complexity of episodic memory. However, these measures have not captured the fine-grained variability of recollected information, which can be considered in terms of two components: precision, reflecting the fidelity of memory, and bias, representing the systematic deviation of memory content from its true value. Measuring the precision of episodic features, such as color or spatial location, has provided novel insights into the cognitive and neural processes involved in recollection (Brady, Konkle, Gill, Oliva, & Alvarez, 2013; Harlow & Yonelinas, 2016; Nilakantan, Bridge, Gagnon, VanHaerents, & Voss, 2017; Richter, Cooper, Bays, & Simons, 2016). For instance, the precision of visual information contributes to perceived memory vividness over and above quantitative retrieval success (Richter et al., 2016; Xie & Zhang, 2017). However, to our knowledge, no study has assessed biases in retrieval of continuous, low-level features, such as visual salience, nor how they relate to memory vividness.
The hypothesized relationships between subjective memory vividness and encoded high-level visual features (e.g., objects) and low-level visual features (e.g., visual salience), accounting for positive modulatory effects of emotional salience (orange stars). Tested relationships are shown with solid arrows, whereas relationships that are predicted but not tested here are shown with dashed arrows. We predicted that negative emotion would enhance the encoding and subsequent fidelity of both low-level and high-level visual representations, even as these features are forgotten (↓). We also predicted that emotional salience might transfer to represented visual salience (indicated by the orange arrow), creating a positive bias relative to neutral images, but we remained agnostic (?) about whether bias in represented visual salience would change with forgetting. We further predicted that subjective memory vividness would be positively influenced (+) by the bias or magnitude of represented visual salience, in addition to the fidelity of visual features.
An important factor in driving episodic memory quality is affective salience, namely, the intensity of an emotional experience. Negative emotional events are consistently more likely than neutral events to be subjectively recollected with enhanced vividness (Dolcos, LaBar, & Cabeza, 2005; Ritchey, Dolcos, & Cabeza, 2008; Sharot, Delgado, & Phelps, 2004). However, greater emotional memory vividness is not driven by a general increase in the number of details remembered (Kensinger, Addis, & Atapattu, 2011; Phelps & Sharot, 2008; Rimmele, Davachi, Petrov, Dougal, & Phelps, 2011) but, rather, by the magnitude of negative affective salience and greater recall of central visual information (Kensinger, 2009; Mather & Sutherland, 2011; Yonelinas & Ritchey, 2015). Emotional visual details not only are more likely to be recalled but also are recalled more precisely (Kensinger, Garoff-Eaton, & Schacter, 2007; Xie & Zhang, 2017). Such enhanced visual precision could explain why vivid and emotional memories are accompanied by greater neural recapitulation of visual details than less vivid (Lee, Samide, Richter, & Kuhl, 2018), nonemotional (Bowen, Kark, & Kensinger, 2017; Kark & Kensinger, 2015) events. It is also possible that subjectively vivid, emotional memories are remembered as more perceptually vibrant. That is, there may be a transfer of affective salience that creates a positive bias in visual salience, indexed by low-level visual features such as color or luminance (see Fig. 1). Could subjectively vivid recollection, enhanced by emotion, be characterized by a more visually salient memory representation?
Prior research has suggested that emotional salience can bias perception of low-level visual features. In one set of experiments, participants viewed emotional and neutral images, overlaid with varying noise, and were immediately asked to judge the magnitude of noise in relation to a scrambled image. Negative emotion enhanced the images’ signal-to-noise ratio, causing them to appear clearer than neutral images (Todd, Schmitz, Susskind, & Anderson, 2013; Todd, Talmi, Schmitz, Susskind, & Anderson, 2012), which is in line with research showing that affective images are perceived as more visually complex (Madan, Bayer, Gamer, Lonsdorf, & Sommer, 2018). Interestingly, the degree to which emotion influenced perception was predictive of subsequent subjective memory vividness (Todd et al., 2013). However, it remains unclear how participants reconstruct low-level features in long-term memory and whether emotionally enhanced memory vividness is accompanied by a positive bias in recollected visual salience.
To this end, we developed a novel paradigm in which participants studied negative and neutral images presented in varying levels of visual salience, manipulated by changing luminance and color saturation. Participants reconstructed the images’ visual salience either immediately (perceptual task) or in a subsequent test (memory task), providing a measure of precision and bias. We first predicted that emotional images would be perceived and remembered as more visually salient than neutral images. We also expected that negative images would be reconstructed more precisely because of the benefits of emotion on encoding central visual features. Finally, we examined the relationship between remembered visual salience and subjective memory vividness, predicting that overall vividness not only would relate to the precision of recalled information but also would positively track bias in remembered visual salience.
Experiment 1
Method
Participants
Thirty-four participants took part in Experiment 1 (10 men, 24 women). The sample size was selected from a power analysis (G*Power; Faul, Erdfelder, Lang, & Buchner, 2007), which indicated that this number of participants was sufficient to detect a medium within-subjects effect (Cohen’s d = 0.5) 80% of the time using an alpha (α) of .05. A Cohen’s d of 0.5 is similar to effect sizes reported in a recent study investigating the effect of emotion on memory for continuous visual features (Xie & Zhang, 2017). All participants were between 18 and 35 years of age, had normal or corrected-to-normal vision, and had no current diagnoses or history of psychological or neurological disorders. One participant withdrew from the study (final N = 33 for all analyses; mean age = 19.46 years, SD = 1.25). Informed consent was obtained from all participants, who received course credit for their time. Procedures were approved by the Boston College Institutional Review Board.
Materials
A total of 288 images were selected from the Nencki Affective Picture System (NAPS; Marchewka, Zurawski, Jednoróg, & Grabowska, 2014), 144 of which contained negative emotional content and 144 of which were neutral. Using the NAPS norms, we selected negative images to have an arousal rating greater than 5 (range = 1, low, to 9, high) and a valence rating less than 4 (range = 1, low, to 9, high). Neutral images had an arousal rating less than 5 and a valence rating between 4.5 and 6.5. Twelve versions of each image were created by linearly manipulating both the luminance and color saturation, key parameters influencing attention and perceived salience (Engmann et al., 2009; White, Rojas, Mappes, Rautiala, & Kemp, 2017), in Commission Internationale de l’Éclairage (CIE) L*a*b* color space, where euclidean distance is approximately perceptually uniform. The lowest visually salient version of each image was allocated a luminance of 0.68 times the original image value and a color saturation of 0.05 times the original values. These values were increased in equal increments up to a maximum of 1.12 times the original luminance value and 1.7 times the original color saturation (see Fig. 2). Importantly, there were no significant differences in the luminance (mean pixel value of each gray-scale image) and color saturation (mean absolute values of a* and b*) of the original negative and neutral images (ps > .607), which also did not differ on contrast or entropy (ps > .279). Contrast is the standard deviation of pixel values within each gray-scale image, and entropy, H, is calculated from a histogram of the 8-bit gray-scale intensity values x, H = −Σp(x)log p(x), and represents the image randomness (for further details, see Marchewka et al., 2014). All images were scaled to 600 pixels × 450 pixels.
Example of an image at each of the 12 levels of visual salience. The luminance and color saturation of each image were increased or decreased in proportion to the original values. Excluding this example, all images used in the present experiments were taken from the Nencki Affective Picture System (Marchewka, Zurawski, Jednoróg, & Grabowska, 2014).
Procedure
Participants completed six study-test blocks, displayed using MATLAB (The MathWorks, Natick, MA) for the Psychophysics Toolbox (Kleiner, Brainard, & Pelli, 2007). During each study phase, participants were presented with 24 negative images and 24 neutral images, displayed for 4 s each. Participants studied the visual details of each image for the duration of its presentation. For both negative and neutral conditions, 2 images from each of the 12 visual-salience levels were presented per study phase. For half of the trials, participants were tested immediately on the brightness of each image (perceptual task). For the other half of the trials, participants were tested on their memory for the brightness level in a subsequent memory test (memory task). An equal number of negative images and neutral images, at each level of salience, were studied for the memory and perception tasks. Presentation order was randomized for each participant.
Within a perception trial, participants studied an image and were then presented with a scrambled mask for 0.5 s, followed by the reappearance of the image at a pseudorandom level of visual salience. The same mask was used for every perception trial and was generated from a random combination of the experimental stimuli at random levels of visual salience. Participants were instructed to move up and down a brightness scale using the left and right arrow keys to change the brightness of the image back to the level in which it had just been presented and to press the space bar to confirm their response. Immediately after each study phase, participants completed a memory test in which they were presented with the 24 images (12 negative and 12 neutral) that they had previously studied but were not tested on immediately as part of the perceptual task. Participants were shown each image in a pseudorandom level of visual salience and were asked to reconstruct the original (studied) brightness of the image by moving up and down the scale between low and high. The average time between studying an image and seeing it again in the memory test was approximately 5 min. All responses were self-paced and equal numbers of each visual-salience level were used as the probe (test) images in the perceptual and memory tasks and for negative and neutral images. Therefore, the average distance between studied salience levels and test-probe salience levels was always zero for each condition, with a median absolute difference of 3.5. All trials were separated by a 1-s fixation cross.
Analyses
For each trial, we first calculated salience error, which is the difference between the participant’s response and the target (studied) image salience level. Errors were analyzed separately by task (memory, perception) and emotion (negative, neutral), and two measures of interest were calculated to reflect distinct aspects of reconstructed visual salience: (a) salience bias and (b) salience precision. Salience bias is simply the average error across trials. No bias in responses along the visual-salience (brightness) scale would reflect an average error of zero, whereas a bias away from zero would reflect a shift in reconstructed salience, allowing us to test whether images were perceived or remembered as more or less visually salient than they actually were. Salience precision is the standard deviation of errors so that precision is equal to SD−1. A smaller standard deviation would reflect a narrower error distribution and thus more precise visual-salience memory or perception, regardless of participants’ bias (see Fig. 1). All means are reported alongside a 95% confidence interval (CI), and all significant test statistics are accompanied by generalized eta-square (ηG2) or Cohen’s d effect sizes.
Results
First, we tested whether negative images were associated with a positive bias (mean error) in reconstructed visual salience, which would mean that negative images were perceived or remembered as being more visually salient than their original level. Salience bias was not significantly influenced by emotion, F(1, 32) = 0.20, p = .661, but interestingly, bias in the memory task was significantly more negative than in the perceptual task, F(1, 32) = 4.45, p = .043, ηG2 = .03. This latter effect was not moderated by emotion, F(1, 32) = 0.44, p = .511. Specifically, we found, unexpectedly, that memory salience was significantly negatively biased, t(32) = −2.58, p = .015, d = 0.45, whereas perceptual-salience bias did not significantly differ from zero, t(32) = −1.23, p = .229. Therefore, although images were not perceived as any more or less visually salient than their true value, they were remembered as being less visually salient than they had been encoded, demonstrating memory fading (see Fig. 3b). Importantly, the magnitude of salience bias was not driven by inherent variation in the original (unmanipulated) luminance and color saturation levels of the images (see Paragraph 1 in the Supplemental Material available online).
Salience bias and precision in Experiment 1. Probability density plots (a) show the aggregate salience errors across participants within the memory and perception tasks, separately for negative- and neutral-emotion trials. The smoothed density curve is overlaid on each plot. The violin plots show (b) salience-bias estimates and (c) salience precision−1 estimates, separately for each task and trial type. Salience-bias estimates were calculated as the mean salience error shown in (a), and salience precision−1 estimates were calculated as the standard deviation of salience errors shown in (a). Colored circles show means for each participant, white circles represent the means for each combination of task and trial type, and error bars indicate 95% confidence intervals.
Second, we analyzed how emotion might modulate the precision (standard deviation of errors) with which low-level visual salience is perceived and remembered. Not surprisingly, participants were more precise in the perceptual task compared with the memory task, F(1, 32) = 208.97, p < .001, ηG2 = .53. Moreover, the salience of negative emotional images was reconstructed more precisely than for neutral images (see Fig. 3c), F(1, 32) = 5.75, p = .022, ηG2 = .01, and this emotion effect did not vary by task, F(1, 32) = 0.10, p = .753 (see Fig. 3c). Therefore, negative emotion enhanced the precision with which continuous, low-level visual features were perceived and remembered. Moreover, salience bias and precision appear to be relatively independent measures, as indicated by a lack of correlation between these two variables in any condition—task (memory, perception) by emotion (negative, neutral)—across participants, |rs| < .07, ps > .713. Thus, variation in bias is unlikely to be explained simply by worse memory for the image features, as would be reflected in lower precision.
Experiment 2
Experiment 1 revealed that scenes are biased to be remembered as less visually salient than originally experienced and that emotion can increase the precision with which this information is represented. Experiment 2 built on this study by investigating the relationships among visual salience, emotion, and subjective memory vividness: Are vivid memories simply more precise than weaker memories, or are they also recalled as more vibrant by showing attenuated fading?
Method
Participants
Thirty-four participants took part in Experiment 2 (9 men, 25 women; mean age = 19.35 years, SD = 1.77). Inclusion criteria were the same as for Experiment 1.
Procedure
The task for Experiment 2 used the same procedures as Experiment 1 except for one change: In this task, participants studied 20 negative images and 20 neutral images per block, with the remaining 8 images (48 images total across the six study-test blocks; half negative and half neutral) presented in each block’s test phase as novel, unstudied images. For each memory test trial, participants were first shown a probe image at a pseudorandom level of visual salience and were asked to judge whether it was new (response = 1) or old (response = 2–7); the latter judgment was combined with an overall memory-vividness rating on a 6-point scale. Specifically, when an image was recognized, participants were asked to also make an overall judgment reflecting how vividly they remembered studying it originally, on the basis of how clearly they could remember “the image’s specific visual appearance.” For this question, participants were further prompted to “evaluate the quality of your memory for each image and use a range of vividness ratings throughout the experiment.” Responses were self-paced. For recognized images (regardless of whether they were actually old or new), participants were then asked to reconstruct the original brightness of the image as before.
Analyses
Measures of visual-salience bias and precision were computed using the same method as before. However, for the memory condition, now only studied items that were successfully recognized could be analyzed. Additional analyses for Experiment 2 included recognition memory accuracy (hits – false alarms) and mean memory vividness for successfully recognized images, comparing negative items with neutral items. We then analyzed how trial-specific subjective-vividness ratings related to visual-salience memory bias and precision. First, separate within-subjects linear regression analyses were used to estimate (a) precision (absolute error; 0–11) and (b) bias (raw error, accounting for positive or negative shift; −11–11) at each level of vividness (2–7). Second, within-subjects multiple regression analysis was used to assess the unique influence of bias and precision on vividness ratings. One-sample t tests were used to test whether the average beta estimates differed from zero across subjects. Overall, participants responded with more high vividness ratings than low vividness ratings, and so we restricted this analysis to participants who showed sufficient variability in their vividness responses, removing participants whose variance of vividness ratings fell below 1 standard deviation from the group mean (n = 7).
Results
First, we compared negative trials and neutral trials in terms of recognition accuracy and overall memory vividness. As expected, negative emotional images (M = .88, 95% CI = [.844, .908]) were more likely to be correctly recognized than neutral images (M = .84, 95% CI = [.800, .878]), t(33) = 3.07, p = .004, d = 0.53, and memory vividness was also higher for successfully recognized negative images (M = 5.50, 95% CI = [5.198, 5.802]) compared with neutral images (M = 5.26, 95% CI = [4.944, 5.576]), t(33) = 3.83, p < .001, d = 0.66. Moreover, a continuous measure of emotional salience within negative items alone positively correlated with subsequent memory (see Paragraph 2 in the Supplemental Material).
Analyzing visual-salience bias for all perceptual trials and for successfully recognized images, we again found that bias in the memory task was significantly more negative than in the perceptual task, F(1, 33) = 48.39, p < .001, ηG2 = .21, with no main effect of emotion, F(1, 33) = 3.31, p = .078. Replicating Experiment 1, results showed that remembered salience was significantly negatively biased, t(33) = −5.87, p < .001, d = 1.01 (see Fig. 4b), yet perceptual-salience bias did not significantly differ from zero, t(33) = −0.68, p = .502. However, we now found that emotion modulated the memory-fading effect, F(1, 33) = 8.56, p = .006, ηG2 = .01, and recognized emotional images showed an attenuated-salience bias relative to neutral images, t(33) = 2.59, p = .014, d = 0.44. Of note, though, the magnitude of salience bias did not correlate with a continuous measure of emotional salience within negative images (see Paragraph 2 in the Supplemental Material). Because of the restriction of analyses to successfully recognized trials, it is possible that apparent memory fading is influenced by an imbalance in the number of images recognized from each salience level, which would occur if the level of studied visual salience influenced the likelihood of subsequent recognition. Although we did observe a positive relationship between studied visual salience and subsequent memory (mean z = 0.08, 95% CI = [0.037, 0.130]), t(33) = 3.54, p = .001, d = 0.61, controlling for this relationship still revealed significant memory fading (see Paragraph 3 in the Supplemental Material).
Salience bias and precision in Experiment 2. Probability density plots (a) show the aggregate salience errors across participants within the memory and perception tasks, separately for negative- and neutral-emotion trials. The smoothed density curve is overlaid on each plot. The violin plots show (b) salience-bias estimates and (c) salience precision−1 estimates, separately for each task and trial type. Salience-bias estimates were calculated as the mean salience error shown in (a), and salience precision−1 estimates were calculated as the standard deviation of salience errors shown in (a). Colored circles show means for each participant, white circles represent the means for each combination of task and trial type, and error bars indicate 95% confidence intervals.
With regard to salience precision, participants were again more precise at reconstructing the target salience in the perceptual task compared with the memory task (see Fig. 4c), F(1, 33) = 205.55, p < .001, ηG2 = .58, but this analysis revealed no significant effects of emotion on salience precision, Fs(1, 33) < 1.35, ps > .255. Therefore, emotion did not enhance the precision with which successfully recognized images, specifically, could be reconstructed.
Next, we tested how memory vividness was related to visual-salience precision and bias. Trial-specific vividness ratings were strongly negatively related to absolute error of visual-salience memory (mean β = −0.262, 95% CI = [–0.323, –0.201]), t(26) = −8.39, p < .001, d = 1.61; higher vividness was associated with higher precision within subjects, and this relationship was significantly stronger for neutral images (mean β = −0.33, 95% CI = [–0.418, –0.250]) than for negative images (mean β1 = −0.19, 95% CI = [–0.273, –0.107]), t(26) = 2.46, p = .021, d = 0.47. Additionally, vividness ratings were positively related to trial-to-trial memory bias (mean β = 0.251, 95% CI = [0.137, 0.365]), t(26) = 4.30, p < .001, d = 0.83; images remembered as less salient than they were encoded were associated with lower memory vividness, and the most vividly remembered images did not show signs of fading (see Fig. 5). This effect did not significantly differ between negative trials (mean β = 0.19, 95% CI = [0.038, 0.340]) and neutral trials (mean β = 0.31, 95% CI = [0.174, 0.454]), t(26) = 1.36, p = .185. The relationship between visual-salience bias and memory vividness remained, even when analyses controlled for variation in precision (mean β = 0.029, 95% CI = [0.007, 0.050]), t(26) = 2.60, p = .015, d = 0.50, with precision also uniquely contributing to vividness (mean β = −0.102, 95% CI = [–0.130, –0.074]), t(26) = −7.17, p < .001, d = 1.38. The reported results were limited to participants with sufficient variability in vividness responses (see the Method section), but results were similar when the full sample was included.
Relationship between trial-specific memory vividness and remembered visual salience in Experiment 2. Frequency histograms (a) show the number of responses per vividness rating across participants in this analysis, separately for negative- and neutral-emotion trials. Aggregate probability density plots (b) show salience error for all trials on which the image was judged as “old.” Results are shown for each level of memory vividness, ranging from 2 (lowest) to 7 (highest). Mean predicted salience bias (c) and mean predicted salience precision (absolute error; d) are shown for each level of memory vividness, separately for negative- and neutral-emotion trials. Error bars represent 95% confidence intervals.
Experiment 3
The results of Experiment 2 replicated those of Experiment 1 by showing that memories fade. Moreover, memory vividness was influenced both by the precision of remembered visual details and by the degree of bias. The effects of emotion, however, differed: Negative emotion enhanced precision and had no effect on bias in Experiment 1, but it had no effect on precision and attenuated memory fading in Experiment 2. To test whether these differences might be explained by including salience judgments for only remembered images (Experiment 2) or for all trials (Experiment 1), we ran a third preregistered experiment with one change: In this experiment, participants reconstructed the visual salience of every image, even those that they did not explicitly remember.
Method
Participants
Thirty-four participants took part in Experiment 3 (7 men, 27 women). Two participants were excluded from analyses—1 because of chance-level memory performance and 1 as a result of not meeting inclusion criteria—leaving a final sample size of 32 (mean age = 18.94 years, SD = 0.88).
Analyses
Visual-salience bias and precision were first analyzed across all studied items. Salience errors for the memory task were then further analyzed by looking only at successfully recognized images. If this comparison produced a pattern similar to that observed between Experiments 1 and 2, then the difference in previous results could be due to the trial types analyzed. In contrast, if the results did not differ between including and excluding forgotten images, then the difference between experiments would more likely be a result of additionally evaluating subjective memory vividness.
Results
As before, negative emotional images (M = .86, 95% CI = [.816, .899]) were more likely to be correctly recognized than neutral images (M = .81, 95% CI = [.779, .846]), t(31) = 3.37, p = .002, d = 0.60, and subjective memory vividness was also higher for successfully recognized negative images (M = 5.51, 95% CI = [5.146, 5.874]) compared with neutral images (M = 5.34, 95% CI = [4.976, 5.704]), t(31) = 3.92, p < .001, d = 0.69.
We first replicated the memory-fading effect by finding that salience bias in the memory task was significantly more negative than in the perceptual task, F(1, 31) = 12.87, p = .001, ηG2 = .07, whereas effects of emotion did not reach significance, Fs(1, 31) < 3.97, ps > .055. As in Experiment 1, remembered salience across all studied images was significantly negatively biased, t(31) = −3.33, p = .002, d = 0.59, whereas perceptual-salience bias did not differ from zero, t(31) = −0.56, p = .578. Analyzing salience bias for only successfully recognized images, as in Experiment 2, also revealed a significant negative salience bias, t(31) = −4.46, p < .001, d = 0.79. However, we no longer observed any attenuation of bias by negative emotion, t(31) = 0.47, p = .644. Thus, the effects of emotion on the magnitude of visual-salience bias were not consistent across the three experiments (see Figs. 6a and 6b). In contrast, we again observed that negative emotion enhanced the precision with which visual salience was perceived and remembered, across all studied images, F(1, 31) = 8.74, p = .006, ηG2 = .01, as seen in Experiment 1. However, as found in Experiment 2, this difference was not significant when only memory hits were analyzed, t(31) = 0.75, p = .461 (see Figs. 6a and 6b), which suggests that the effects of emotion on precision were evident only when remembered and forgotten trials were combined.
Salience bias and precision, as well as their relationships to memory vividness, in Experiment 3. The violin plots (a, b) show salience bias (upper panels) and precision−1 (lower panels), separately for each task and trial type, calculated (a) across all studied images and (b) within correctly recognized trials only. Colored circles show means for each participant, white circles represent the means for each combination of task and trial type, and error bars indicate 95% confidence intervals. In (c), mean predicted salience bias (upper panel) and mean predicted salience precision (absolute error; lower panel) are shown for each level of memory vividness. Error bars show 95% confidence intervals.
Lastly, we analyzed the relationships between (a) subjective memory vividness and salience precision and (b) subjective memory vividness and salience bias for successfully recognized images. Because of low variability in vividness responses, data from 4 participants were excluded from this analysis. Trial-specific vividness ratings were strongly negatively related to absolute error (precision) of visual-salience memory (mean β = −0.334, 95% CI = [–0.410, –0.258]), t(27) = −8.64, p < .001, d = 1.63, but this was not modulated by emotion, t(27) = 0.49, p = .627. Additionally, vividness ratings were positively related to trial-to-trial memory bias (mean β = 0.210, 95% CI = [0.077, 0.343]), t(26) = 3.09, p = .005, d = 0.59 (see Fig. 6c), which was also not influenced by emotion, t(27) = 0.29, p = .772. The relationship between visual-salience bias and memory vividness was marginally significant even when analyses controlled for variation in precision (mean β = 0.021, 95% CI = [–0.002, 0.044]), t(26) = 1.92, p = .066, d = 0.36, with precision uniquely contributing to vividness (mean β = −0.125, 95% CI = [–0.157, –0.093]), t(26) = −7.58, p < .001, d = 1.43. These results were similar to those reported for Experiment 2, except that here, we no longer observed an effect of emotion on the relationship between precision and vividness.
Discussion
Episodic memories contain complex sensory features, which vary in how accurately they reflect the original event. Yet it remains unclear how well continuous visual features are remembered and to what extent these qualities underpin vivid recollection, which is enhanced for negative emotional experiences (Bowen et al., 2017). In three experiments, we tested how low-level visual salience is reconstructed, relates to subjective memory vividness, and is modulated by emotion. Unexpectedly, we first found that images were consistently remembered as less visually salient than originally experienced. Second, we found that memory vividness tracked the precision and bias of remembered salience, such that the fidelity and vibrancy of visual features were enhanced during vivid recollection. Finally, negative emotional content increased the precision with which visual salience was encoded but did reliably influence memory fading.
The finding that memories literally fade has, to our knowledge, not been demonstrated previously. Bias in remembered visual salience was related to subjective memory vividness; specifically, less vivid memories were measurably more visually dull than highly vivid memories. Given that systematic fading was specific to memory and not perception, this change could occur during memory consolidation. Prior research has suggested that, after encoding, high-level memory details (e.g., the central semantic and emotional content) are prioritized, whereas peripheral low-level details (e.g., the specific sensory properties) are weakened (Mitchell & Cusack, 2016; Sekeres et al., 2016). We showed that these changes may reflect a genuine transformation of visual details rather than simply a loss of information. An alternative explanation is that a weaker recollective experience might lead to the heuristic that an experience must not have been visually salient to start with (cf. Johnson, 1997). Although the relationship between vividness and remembered visual salience may be bidirectional, a simple heuristic explanation failed to account for the finding that negative images were consistently remembered with greater subjective vividness but were not necessarily remembered as visually brighter than neutral images.
The effect of negative emotion on remembered visual-salience bias was inconsistent, with no significant emotional modulation in Experiments 1 and 3 and an attenuation of bias in Experiment 2. These findings ran counter to our initial hypothesis, in which we expected that affective salience might transfer to remembered visual salience. Considering the minor differences across the experiments, we note that the lack of a consistent effect highlights the importance of replication. Given that between-subjects variability appeared to be lowest in Experiment 2, it is possible that noise in subjective estimates of bias may have contributed to the different results. Alternatively, modulatory effects of emotion on memory for low-level visual features may be susceptible to individual differences in emotional experience, which we did not measure in this study. We consistently observed no effect of emotion on bias in the perceptual task, counter to our prediction and in contrast to findings on the memory task. Our hypothesis was inspired in part by findings from the work by Todd and colleagues (2013), who showed that emotional content enhanced the perceptual salience of images. However, there are substantial differences among our paradigms: In the studies by Todd et al. (2012, 2013), images were overlaid with noise and participants judged the relative signal-to-noise ratio. This judgment was likely influenced by the clarity with which the images appeared because visual perception is known to be enhanced by emotional arousal (Mather & Sutherland, 2011). In contrast, we directly changed image characteristics, such that any shift in visual-salience bias would involve participants perceiving the images as looking qualitatively different. We found that negative emotion enhanced the precision with which low-level visual information was perceived. Thus, emotion likely enhances the fidelity of visual details but does not appear to bias the salience with which this information is represented.
Emotional images were more likely to be recognized with higher subjective vividness than neutral images, as expected, and visual details of negative memories could also be recalled more precisely. Moreover, memory precision was strongly related to judgments of memory vividness, as predicted. These results support previous findings that negative emotion can enhance both the subjective vividness and precision with which continuous visual features, such as color, are remembered (Xie & Zhang, 2017, 2018). However, the benefit of emotion on precision was observed only when all studied trials were analyzed (thus mixing different proportions of recognized and forgotten images for negative and neutral conditions) and not when analyses were restricted to successfully recognized images. Therefore, emotion may enhance the precision with which visual details are encoded, acting to increase subsequent recognition without necessarily increasing the fidelity of remembered events. Hence, emotional-memory vividness is likely to be independently influenced by affective salience (Todd et al., 2012), rather than being predominantly explained by modulation of low-level visual information.
The current results open several questions for future research, most significantly the need to unpack the mechanisms by which visual memory details are transformed. First, orthogonalizing components of visual salience, such as color and luminance, may elucidate which properties most contribute to memory fading. Relatedly, it is important to investigate the relative contribution of low-level visual information to subjective vividness judgments when multiple aspects of events are tested. Finally, an important research avenue is to test how memory for different visual properties changes over time. Given that the benefits of emotion on recollection are amplified during memory consolidation (Yonelinas & Ritchey, 2015), emotion effects may emerge with an increasing delay between encoding and retrieval. Overall, the discovery that visual characteristics of past events literally fade provides important insight into how memories are reconstructed: Memory representations do not simply lose high-level event details, but their low-level visual content can also become qualitatively transformed.
Supplemental Material
CooperOpenPracticesDisclosure_rev – Supplemental material for Memories Fade: The Relationship Between Memory Vividness and Remembered Visual Salience
Supplemental material, CooperOpenPracticesDisclosure_rev for Memories Fade: The Relationship Between Memory Vividness and Remembered Visual Salience by Rose A. Cooper, Elizabeth A. Kensinger and Maureen Ritchey in Psychological Science
Supplemental Material
CooperSupplementalMaterial_rev – Supplemental material for Memories Fade: The Relationship Between Memory Vividness and Remembered Visual Salience
Supplemental material, CooperSupplementalMaterial_rev for Memories Fade: The Relationship Between Memory Vividness and Remembered Visual Salience by Rose A. Cooper, Elizabeth A. Kensinger and Maureen Ritchey in Psychological Science
Footnotes
Acknowledgements
We thank Max Bluestone, Helen Schmidt, Samantha Murphy, Mary Nanna, Julia Napoli, and Maria Khoudary for assistance with data collection.
Action Editor
Caren Rotello served as action editor for this article.
Author Contributions
R. A. Cooper and M. Ritchey developed the study concept. R. A. Cooper analyzed the data under the supervision of M. Ritchey. All the authors contributed to the study design and data interpretation. R. A. Cooper drafted the manuscript, and E. A. Kensinger and M. Ritchey provided critical revisions. All the authors approved the final manuscript for submission.
Declaration of Conflicting Interests
The author(s) declared that there were no conflicts of interest with respect to the authorship or the publication of this article.
Funding
This research was supported in part by National Institutes of Health Grant No. R00MH103401 (to M. Ritchey).
Open Practices
All data have been made publicly available via the Open Science Framework and can be accessed at https://osf.io/cuz8g/. The materials used in these studies are freely available. Experiment 3 was preregistered and can be accessed at https://osf.io/nsj65/. The complete Open Practices Disclosure for this article can be found at http://journals.sagepub.com/doi/suppl/10.1177/0956797619836093. This article has received the badges for Open Data and Preregistration. More information about the Open Practices badges can be found at 
.
References
Supplementary Material
Please find the following supplemental material available below.
For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.
For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.
