Abstract
The human amygdala is sensitive to stimulus characteristics, and growing evidence suggests that it is also responsive to cognitive framing in the form of evaluative goals. To examine whether different evaluations of stimulus characteristics shape amygdala activation, we conducted a preregistered replication of Cunningham, Van Bavel, and Johnsen’s (2008) study demonstrating flexible mapping of amygdala activation to stimulus characteristics, depending on evaluative goals. Participants underwent functional MRI scanning while viewing famous names under three conditions: They were asked to report their overall attitude toward each name, their positive associations only, or their negative associations only. We observed an interaction between condition and rating type, identified as the effect of interest in Cunningham et al. (2008). Specifically, postscan positivity, but not negativity, ratings predicted left amygdala activation when participants were asked to evaluate positive, but not negative, associations with the names. These results provide convergent evidence that cognitive framing, in the form of evaluative goals, can significantly alter whether amygdala activation indexes positivity or negativity.
Keywords
Research practices in psychology have faced increased scrutiny following a large-scale replication project in which the results of only 39 of 100 studies published in prominent journals in 2008 were unambiguously replicated (Open Science Collaboration, 2015). Reproducibility in functional MRI (fMRI) research may be of particular interest given the high number of researcher degrees of freedom in data processing and analysis in that field (Poldrack et al., 2017; Poldrack et al., 2008). Unfortunately, few preregistered neuroimaging replications have been conducted or published to date (as noted by Gilmore, Diaz, Wyble, & Yarkoni, 2017; Poldrack et al., 2017). Here, we report a direct replication that used several analytic approaches to examine the robustness of a previously reported effect in which amygdala activation in response to affective stimuli was influenced by cognitive framing.
The amygdala, a subcortical brain region implicated in social, cognitive, and affective processes (Adolphs, 2003), is often characterized as responding to certain external stimulus characteristics. In many studies of amygdala function, researchers manipulate stimuli and report the dimensions associated with differential amygdala activation, including fear (Keifer, Hurt, Ressler, & Marvar, 2015; Whalen et al., 2004), positive and negative valence (implicating arousal; Bonnet, Comte, Tatu, Millot, Moulin, & de Bustos, 2015; Hamann, Ely, Hoffman, & Kilts, 2002), novelty (Blackford, Buckholtz, Avery, & Zald, 2010; Wright et al., 2003), and ambiguity (Whalen, 1998). Both forward- and reverse-inference maps generated using the automated neuroimaging meta-analysis tool Neurosynth (www.neurosynth.org) show a strong statistical association between the presence of terms such as “fear,” “emotion,” “negative,” and “arousal” in article abstracts and increased likelihood of reported activation in the amygdala. This reactive characterization of the amygdala is consistent with its broader role in responding to environmental cues by facilitating adaptive behaviors, quickly and potentially without conscious perception (Méndez-Bértolo et al., 2016; Whalen, et al., 2004).
Complementary research has also shown that amygdala activation is influenced by top-down, cognitive goals. The target of the present replication was an impactful, frequently cited study indicating that the amygdala not only responds passively to stimulus characteristics, but also is sensitive to the relationship between internal evaluative states and stimulus characteristics (Cunningham, Van Bavel, & Johnsen, 2008). Prior studies have demonstrated that amygdala activation is elicited by both positive and negative stimuli (Hamann et al., 2002; Kim, Pignatelli, Xu, Itohara, & Tonegawa, 2016). However, Cunningham et al. (2008) successfully demonstrated that a minimalist framing instruction can change the way in which the amygdala responds to positive and negative stimulus characteristics. Given its central role in organizing large-scale bodily responses, enhancing emotional memory, and acquiring and expressing affective associations (Phelps & LeDoux, 2005; Phillips & LeDoux, 1992), this powerful demonstration that internal, cognitive processes influence amygdala activation has broad implications.
In our preregistered replication of Cunningham et al. (2008), 1 we examined the effect of evaluative goals on affective responses in the amygdala. We designed our study to replicate the methods of Cunningham et al. as directly as possible, with appropriate changes given differences in context (e.g., stimuli were updated) and upgrades to fMRI technology and analysis software.
Method
Participants
The sample size in the original study was 12, and our target, based on a priori power analyses, was 35 (see Preregistered Data Collection and Analysis Plan, in the Supplemental Material available online). We recruited a total of 39 right-handed undergraduates from the Denver area, anticipating loss of data when we applied various a priori exclusion criteria (e.g., excessive motion). Participants were compensated $75 for completing the study. Data could not be collected from 1 participant because of an MRI contraindication (unidentified implant), and another participant was excluded from analyses because of blatant non-compliance with the task instructions. Thus, 37 participants provided usable data.
Our primary analyses were conducted using the original exclusion criteria of Cunningham et al. (2008): movement greater than 2 mm in any direction during scanning (n = 10) and recognition of less than 75% of the famous names presented during the study (n = 0). With these exclusions, 27 participants remained for analysis. Additionally, analysis of postscan ratings of some of our participants revealed unexpected collinearity errors, such that positivity and negativity ratings could not be used as unique predictors for one or more functional blocks (n = 5, 2 of whom were also excluded for excessive motion). Therefore, our final sample for our preregistered analyses consisted of 24 participants. The majority of this sample identified their gender as female (58%), as was true in Cunningham et al.’s (2008) sample (75% female before exclusions). The mean age of our sample was 21.2 years, which was comparable to the mean age of 22.8 years in Cunningham et al.’s sample before exclusions. (See Supplemental Method and Results, in the Supplemental Material available online, for analyses conducted with less restricted samples.)
Main task
The main task was administered using a modified version of the original E-prime script, provided to us by W. A. Cunningham and J. J. Van Bavel via the Open Science Framework (personal communication, July 28, 2014). Participants completed eight functional blocks during which they rated famous names in three conditions. In the overall-attitude condition, participants were asked to consider both the positive and the negative associations they had for each stimulus. In the positive condition, they were asked to focus only on positive associations and ignore any negative ones. Similarly, in the negative condition, they were asked to focus only on negative associations and ignore any positive ones. Additional details of the design, including features of the design based on scanning considerations (e.g., jittered timing in the trials), are reported in Preregistered Data Collection and Analysis Plan, in the Supplemental Material.
As in the original study, the stimuli were 96 famous names (e.g., Adolph Hitler, Rosa Parks, Britney Spears), and participants evaluated each name once in each condition while they were inside an MRI scanner. In the overall-attitude condition, participants were asked to evaluate both positive and negative aspects of each name. In the positive condition, they were asked to evaluate only positive associations of each name. And in the negative condition, they were asked to evaluate only negative associations of each name. Participants were instructed to rate the names using different response options in each condition: The 4-point scales ranged from very good to very bad in the overall-attitude condition, from none to very good in the positive condition, and from none to very bad in the negative condition. We conducted a pilot test of all the original stimuli to determine both their familiarity and their emotionality in the present population. Of the original 96 names, 44 (46%) were replaced because of low recognition or emotionality.
As in the original study, following the fMRI scan, participants were asked to indicate whether each name was known to them (yes/no) and to rate each known name for positivity, negativity, and emotionality (i.e., how ‘‘emotional’’ the name made them feel). These ratings were made on a Likert-type scale with more response options (1 = low, 8 = high) than the scale used for the ratings made in the scanner. Valence was calculated by subtracting negativity ratings from positivity ratings, so higher scores indicated more positivity. Extremity was calculated by squaring valence, to capture the overall magnitude of feelings (both highly positive and highly negative names had high extremity scores). Ambivalence was computed by multiplying the valence rating with the lower value by 3 and subtracting the valence rating with the higher value from that product (Thompson, Zanna, & Griffin, 1995).
Functional localizer task
A task known to elicit amygdala activation was included at the end of the scanning session to confirm the ability of our scan parameters and fMRI analyses to detect amygdala activation. Based on a task described by Hariri, Bookheimer, and Mazziotta (2000), this brief task compared activation during matching of emotional faces with activation during matching of shapes (see Supplemental Method and Results, in the Supplemental Material, for details).
fMRI parameters
The original study used a Siemens 3-T scanner with slices parallel to the anterior commissure–posterior commissure (AC-PC) line (32 axial slices; echo time = 25 ms, repetition time = 2,000 ms, in-plane resolution = 3.5 × 3.5 mm). All scans in our replication study were conducted on a Siemens Tim Trio (3-T) scanner with software that was up-to-date in 2014. For whole-brain functional coverage, 56 prescribed axial slices parallel to the AC-PC line were collected using a multiband sequence with a 32-channel head coil (echo time = 29 ms, repetition time = 1,000 ms, in-plane resolution = 3 × 3 mm).
fMRI processing
We followed the original procedures Cunningham et al. (2008) used to process and analyze the data (i.e., their analysis pipeline), with minor modifications. Data were prepared for analysis using FSL (University of Oxford, Oxford, United Kingdom; https://fsl.fmrib.ox.ac.uk/fsl/fslwiki) and SPM12 (Wellcome Department of Cognitive Neurology, London, United Kingdom; http://www.fil.ion.ucl.ac.uk/spm/; note that SPM5 was used in the original study). Corrections for slice acquisition time, motion, and high-frequency noise were done in FSL. For group comparisons, data were then normalized into Montreal Neurological Institute (MNI) standardized space, interpolated to a 3- × 3- × 3-mm grid space, and smoothed at an 8-mm kernel in SPM. First- and second-level SPM models then analyzed blood-oxygenation-level-dependent (BOLD) signal using a blocked-trial design with a canonical hemodynamic response function. As in the original study, analyses of the primary region of interest (ROI), the amygdala, used an anatomical mask (MRIcro atlas; Rorden & Brett, 2000) with a voxel-level threshold of p < .05 and a 10-voxel minimum. (Results of whole-brain analyses are reported in Table S7 in Supplemental Method and Results, in the Supplemental Material available online.)
Modeling approach
Individual-level models
As in the original study, two individual-level models were used. The first, the valence model, predicted BOLD signal as a function of extremity, valence, condition, and the valence-by-condition interaction. Valence values near zero could reflect either low or high ratings of both positivity and negativity, so ambivalence was included as a covariate. Emotionality was also included as a covariate to help differentiate valence effects from more general emotionality effects. Activation in response to names reported as unknown in the postscan ratings was modeled separately (the results of these analyses are not reported here, as they are of no interest for the replication).
The second individual-level model, the positive-negative model, was run to differentiate the effects of positivity and negativity ratings in the positive and negative conditions. Subject-centered positivity and negativity ratings and their interaction term were used as parametric modulators of BOLD signal in the three conditions. Emotionality was again included as a covariate. Activation for unknown names was modeled separately (the results are not reported here).
Effect of interest
We ran a repeated measures general linear model (RMGLM) on the beta weights extracted from a functionally defined ROI to test for an a priori effect of interest (EOI), an interaction between condition and the type of rating. First, voxels displaying an overall effect of extremity and those showing an interaction between valence and condition were identified in the valence model. Next, the overlap of these two effects was computed to create a functional ROI mask, a subsection of the brain to be examined. Then, averaged beta weights from the positive-negative model were extracted from this functional mask. Finally, these beta weights were entered into an RMGLM to test whether positivity ratings better predicted amygdala activation in the positive condition whereas negativity ratings better predicted amygdala activation in the negative condition, as observed by Cunningham et al. (2008), who reported the following results for this effect: F(1, 11) = 9.67, p < .01. 2
Results 3
Postscan ratings
Summary statistics were computed for each participant’s ratings and then averaged across participants. To ensure that the stimuli in the original and replication studies were comparable, we conducted independent-samples t tests. The mean postscan ratings in the replication did not differ from the means in the original study (all 95% confidence intervals, CIs, overlapped, all ps > .250). Correlations among postscan ratings were also very consistent between the two studies (additional information is provided in Supplemental Method and Results, in the Supplemental Material). Additionally, recognition was very high in the replication sample (M = 97.01%, SD = 4.06).
Amygdala ROI
Functional localizer
To ensure adequate coverage of the amygdala, we first analyzed the data from the functional localizer task. For the face > shape contrast, we observed significant activation in the left amygdala, t(23) = 4.60, p < .001, and the right amygdala, t(23) = 3.95, p < .001. Therefore, our updated scan parameters resulted in sufficient functional coverage and supplied independent left and right functional amygdala masks for additional analyses (Fig. 1c and Table 1).

Maps of the amygdala region of interest (ROI): (a) amygdala activation associated with the extremity effect in the valence model, (b) left lateralized amygdala activation associated with the valence-by-condition interaction in the valence model, (c) amygdala activation for the faces > shapes contrast in the functional localizer task, and (d) illustration of the anatomically defined amygdala mask from MRIcro (Rorden & Brett, 2000). All coordinates are in Montreal Neurological Institute space. Right is right. The threshold for functional images was set at p < .05, with a minimum of 10 contiguous voxels.
Significant Results of the Amygdala Region-of-Interest Analyses
Note: The threshold for these analyses was p < .05, with a minimum of 10 contiguous voxels. The table reports peak coordinates in Montreal Neurological Institute (MNI) space.
EOI analyses
As in the original study, we observed an overall effect of extremity on BOLD signal in both the left amygdala, t(23) = 4.06, p < .001, and the right amygdala, t(23) = 3.58, p < .001 (Fig. 1a and Table 1); higher extremity was associated with more activation. Additionally, there was neither a significant main effect of valence nor an interaction of extremity and condition. Also as in the original study, a significant valence-by-condition interaction was observed in the left amygdala, F(2, 69) = 4.93, p = .010 (Fig. 1b and Table 1). This result supports the concept of affective flexibility. The replicated overlap of the extremity and valence-by-condition masks was a 19-voxel cluster in the left amygdala.
A 2 (rating type: positivity, negativity) × 2 (condition: positive, negative) RMGLM compared extracted beta weights from this region of functional overlap in the left amygdala. The results of this analysis were consistent with the EOI, showing a significant interaction between rating type and condition, F(1, 23) = 10.50, p = .004, η p 2 = .31. Positivity, but not negativity, ratings predicted amygdala activation in the positive condition, t(23) = 2.53, p = .019, dz = 0.37, 95% CI = [0.06, 0.67] (Fig. 2 and Table 2). Although results for negativity followed the predicted pattern of greater amygdala activation in the negative than the positive condition, t(23) = 1.48, p = .15, dz = 0.33, 95% CI = [−0.17, 0.83], this difference did not meet our significance threshold. Note that the original interaction was characterized by a full cross-over, in which negativity ratings were better predictors of amygdala activation in the negative condition and positivity ratings were better predictors of amygdala activation in the positive condition.

Extracted betas for the effect-of-interest analysis. The graph shows the average beta weights for the effect of positivity ratings and negativity ratings in the positive and negative conditions. The beta weights were extracted from the functional overlap mask including those voxels displaying both an overall effect of extremity and a valence-by-condition interaction. The asterisk indicates a significant pairwise comparison (p < .05). Error bars represent ±1 pooled SEM.
Results of the Repeated Measures General Linear Model Testing the Interaction of Condition and Rating Type
Note: The table reports results for regions of interest in which condition and rating type had a significant (p < .05, uncorrected) interaction effect on extracted beta weights; asterisks indicate p values that met the criterion for significance after Bonferroni correction for three comparisons per hemisphere (p < .0167). Within a row, parameter estimates that pairwise tests revealed to be significantly different (p < .05, uncorrected) are marked with the same subscript.
Additional analyses
Given that many decisions can determine the analysis pipeline for neuroimaging data, in several unregistered analyses we examined the robustness of our results to ROI selection, exclusion criteria, and processing choices (see Supplemental Method and Results, in the Supplemental Material). Because these analyses were not outlined in our preregistered analysis plan, the results should be interpreted with caution, but they do reflect the current, if often unreported, practice of ensuring that results are not dependent on specific analysis choices. As tests performed on extracted data from functional ROIs defined using the same data are likely to capitalize on nonindependence error (e.g., Kriegeskorte, Simmons, Bellgowan, & Baker, 2009), we examined the robustness of the EOI using two additional masks: an anatomical amygdala mask (Fig. 1d) and the independent mask derived from our functional localizer task (Fig. 1c). In recognition of the overlap of the functional ROI defined by the main task, the anatomical ROI, and the functional localizer ROI in each hemisphere, we calculated the Bonferroni correction for a family-wise error rate of .05 for three comparisons and determined that the corrected significance threshold for these additional tests was p < .0167.
Anatomical ROI
Average beta weights were extracted from both the left and the right anatomical amygdala in the MRIcro atlas (Fig. 1d; Rorden & Brett, 2000), to provide the most conservative test of the interaction identified as the EOI. We observed a significant interaction in both the left mask, F(1, 23) = 9.33, p = .006, η p 2 = .29, and the right mask, F(1, 23) = 7.34, p = .013, η p 2 = .24 (Table 2). The interaction in the left amygdala was again driven by differences between the beta weights for positivity and negativity ratings in the positive condition, t(23) = 2.80, p = .010, dz = 0.42, 95% CI = [0.10, 0.73]. The interaction in the right amygdala was driven by marginal differences (considering the Bonferroni correction) between the beta weights for positivity ratings in the positive and negative conditions, t(23) = 2.49, p = .021, dz = 0.55, 95% CI = [0.01, 1.08] (Table 2).
Functional localizer ROI
In the mask generated from the independent functional localizer task (Fig. 1c), we again observed the interaction defined as the EOI in the left amygdala, F(1, 23) = 7.71, p = .011, η p 2 = .25. The interaction was marginally significant (considering the Bonferroni correction) in the right amygdala, F(1, 23) = 5.04, p = .035, η p 2 = .18 (Table 2). As in the analyses reported earlier, the effect in the left amygdala was driven by differences between the beta weights for positivity and negativity ratings in the positive condition, t(23) = 2.52, p = .019, dz = 0.38, 95% CI = [0.06, 0.71]. As we found for the anatomical mask, the effect in the right amygdala was driven by a difference between the beta weights for positivity ratings in the positive and negative conditions, t(23) = 2.38, p = .026, dz = 0.51, 95% CI = [0.01, 1.02] (Table 2).
Discussion
This preregistered replication study confirmed a significant interaction effect of evaluative goals and stimulus characteristics on activation of the amygdala, first reported in Cunningham et al. (2008). We performed several additional analyses reflecting different key analytic choices to demonstrate that the effect is not dependent on choice of mask, exclusion criteria, covariates, and software package. We consider this study, along with other recent work (Beer, Rigney, & Flagan, 2016; Burunat et al., 2016), an important contribution to examining the reproducibility of neuroimaging studies. Our results confirm the importance of cognitive processes, such as evaluative goals, in determining how the amygdala responds to external stimuli. This work supports a growing, clinically relevant literature on effective ways to influence affective responding using cognitive processes such as emotion regulation (Ochsner, Silvers, & Buhle, 2012; Snyder, Miyake, & Hankin, 2015). Furthermore, our results provide compelling evidence that amygdala activation cannot be sufficiently described by the stimuli that elicit it, but is jointly determined by stimulus characteristics and cognitive context.
Limitation and future directions
One limitation of this study is the large number of participants who were excluded when we used the original exclusion criteria, which resulted in the preregistered replication analysis being underpowered. Additionally, although the original effect was observed bilaterally, the replication effect was limited to the left amygdala. We also did not observe the full cross-over interaction in the extracted beta weights that was observed in the original study. Rather, the replication effect was driven by a significant difference in the effect of positivity ratings compared with negativity ratings in the positive condition. We consider the positivity effect the more interesting component of the central interaction, as abundant previous work has demonstrated amygdala response to negativity (Blanchard & Blanchard, 1972; LeDoux, 2013). In addition, many of the results reported in the Supplemental Material (see Supplemental Method and Results) provide evidence that different analytic choices lessen or entirely remove these departures from the original findings. Therefore, future studies should explore whether any of these new findings are replicable and meaningful. Finally, future work should explore the generalizability of the interaction effect with other samples and other types of ambiguous emotional stimuli to more fully characterize the role of cognitive evaluations in responding to and resolving emotional ambiguity (Hirsch, Meeten, Krahé, & Reeder, 2016).
Conclusion
This fMRI experiment successfully replicated an identified EOI reported in Cunningham et al. (2008): Amygdala activation flexibly responded to evaluative goals. This replicated finding that cognition has a central role in affective neural systems could have wide-reaching implications for understanding broader processes, such as emotion regulation. The notion that the amygdala, and the coordinated bodily responses it modulates, is influenced not just by stimulus characteristics but also by internal cognitive processes is critical to the conceptualization of affective responses as malleable and controllable (Lindquist, Wager, Kober, Bliss-Moreau, & Barrett, 2012; Ochsner et al., 2012). Specifically, these results highlight the importance of cognitive context in interpreting physiological and neural responses elicited by affective stimuli. This study provides corroborative evidence that the amygdala not only passively responds to stimulus characteristics, but also works flexibly, adapting its response to external stimuli according to internal states and goals.
Footnotes
Acknowledgements
The authors thank the Center for Open Science, especially Brian Nosek and Mallory Kidwell, for facilitating this project; the original authors, especially Wil Cunningham and Jay Van Bavel, for providing their materials, input, and a collaborative approach to the project; Max Weisbuch for commenting on a previous version of this manuscript; the Intermountain Neuroimaging Consortium and the AACT Lab at the University of Denver for providing helpful feedback and support; and the University of Denver for providing the high-performance computing cluster for data analysis.
Action Editor
D. Stephen Lindsay served as action editor for this article.
Declaration of Conflicting Interests
The authors declared that they had no conflicts of interest with respect to their authorship or the publication of this article.
Funding
This project was funded by the Center for Open Science.
Open Practices
All data and materials have been made publicly available at the Open Science Framework and can be accessed at https://osf.io/bifc7/. The design and analysis plans were preregistered at the Open Science Framework and can also be accessed at https://osf.io/bifc7/. The complete Open Practices Disclosure for this article can be found at https://journals-sagepub-com.web.bisu.edu.cn/doi/suppl/10.1177/0956797617719730. This article has received badges for Open Data, Open Materials, and Preregistration. More information about the Open Practices badges can be found at
.
Notes
References
Supplementary Material
Please find the following supplemental material available below.
For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.
For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.
