Show Me the Money: A Systematic Exploration of Manipulations,Moderators,and Mechanisms of Priming Effects

Abstract

A major challenge for accumulating knowledge in psychology is the variation in methods and participant populations across studies in a single domain. We offer a systematic approach to addressing this challenge and implement it in the domain of money priming. In three preregistered experiments (N = 4,649), participants were exposed to one of a number of money manipulations before completing self-report measures of money activation (Study 1); engaging in a behavioral-persistence task (Study 3); completing self-report measures of subjective wealth, self-sufficiency, and communion-agency (Studies 1–3); and completing demographic questions (Studies 1–3). Four of the five manipulations we tested activated the concept of money, but, contrary to what we expected based on the preponderance of the published literature, no manipulation consistently affected any dependent measure. Moderation by sociodemographic characteristics was sparse and inconsistent across studies. We discuss implications for theories of money priming and explain how our approach can complement recent efforts to build a reproducible, cumulative psychological science.

Keywords

research methodology replication moderators money priming open data open materials preregistered

A critical assumption underlying the progression of science is its self-correcting nature. However, the emerging discussion about the quality of findings in psychological science, and the interest in developing and implementing practices that improve it (e.g., Ledgerwood, 2014; Lindsay, 2015; Open Science Collaboration, 2015; Simmons, Nelson, & Simonsohn, 2011), highlights how this process does not necessarily occur spontaneously but instead requires a more conscious and systematic effort (Nosek, Spies, & Motyl, 2012).

We join these efforts with the methodological approach we propose in this article, in which we aim to address a major challenge for the aggregation of knowledge across studies within the same domain: variance in methods and participant populations. In many areas of experimental research, different studies investigating the same phenomenon use different manipulations of the independent variable, different dependent measures, different populations of participants, or some combination thereof. When conceptually equivalent studies (e.g., a study and its attempted replication) lead to seemingly discrepant results, these variations can inhibit a clear understanding of whether the discrepancy reflects the low reliability of the documented effect, some meaningful variation between the studies (a “hidden moderator”; see, e.g., Van Bavel, Mende-Siedlecki, Brady, & Reinero, 2016), or something else.

To address this challenge, we implemented an approach that systematically evaluates the effects of different manipulations of the same construct on a constant series of outcomes while measuring a series of potential moderators, all within the same heterogeneous sample. Although this approach could be applied to any domain, we illustrate it in the domain of money priming, because our reading of the literature—as well as our personal experience conducting research in this area—leads us to believe that this domain has reached a point at which it could benefit from such a systematic assessment.

Money-Priming Effects

Research on money priming was fueled by a seminal article that introduced the self-sufficiency hypothesis, according to which people who are reminded of money “put forth effort to attain personal goals and prefer to be separate from others” (Vohs, Mead, & Goode, 2006, p. 1154). In less than a decade after the original publication, more than 165 experiments were published that generally support the self-sufficiency notion, and converging evidence spans a broad range of manipulations, participant populations, and dependent measures (Vohs, 2015).

Note that research on money priming has also found that manipulations that do not activate the idea of abundance—such as thinking about small sums of money—do not induce self-sufficient behavior (Baumeister & Vohs, 2015). In addition to moderation by the type of prime used, some published studies have documented moderation by participant demographics such as gender (Yong & Li, 2012) and socioeconomic status (Mogilner, 2010).

It has been argued that specific characteristics of the participant population may account for inconsistencies in recent attempts to replicate money-priming effects (Vohs, 2015). Indeed, previous results showing that reminders of money increased participants’ tendency to justify the existing social system in the United States (Caruso, Vohs, Baxter, & Waytz, 2013) were not replicated in other participant populations (Klein et al., 2014; Rohrer, Pashler, & Harris, 2015), whereas subsequent studies have found that the original effect is moderated by participants’ subjective wealth (Schuler & Wänke, 2016).

The existing research suggests that there may be both theoretical and empirical reasons to think that money-priming effects are highly sensitive to aspects of the experimental design and participant population. We believe the field is at a critical point for providing direct and systematic evidence for the source of such sensitivity to refine and advance the understanding of money-priming effects (Cesario, 2014; Locke, 2015).

Overview of Studies

To gather such evidence, we designed an experimental procedure that would (a) compare a series of money-priming manipulations that have been used in past work, (b) determine whether exposure to each of these primes actually activates the construct of money, (c) measure a series of outcomes that have been theoretically or empirically linked to money priming in previous research, (d) collect sociodemographic information, and (e) draw from heterogeneous participant pools. We implemented this procedure across three preregistered studies. In Studies 1 and 2, we implemented the procedure with large, diverse online samples, using self-reported dependent measures. In Study 3, we implemented the procedure with a sample of both students and community members in a laboratory study and included a behavioral measure of self-sufficiency.

Given the existing literature, we would expect that money primes (compared with control primes) would (a) heighten activation of the concept of money; (b) increase one’s sense of wealth, agency, and self-sufficiency, as well as self-sufficient behavior; and (c) reduce communal tendencies. We did not have a priori expectations about the relative strength of the effects of the different manipulations or about specific patterns of moderation by sociodemographic variables. We report the results of three experiments that implement our approach. Additional details of the methods and results of all three studies appear in the Supplemental Material available online. We report all data exclusions, conditions, measures, and how we determined our sample size in all studies (either in the main text or in the Supplemental Material).

Study 1

Method

Participants

Participants were adult residents of the United States who were invited to participate in a study about “social attitudes.” They were recruited through the Qualtrics Panels service (https://www.qualtrics.com/online-sample/) and completed the study online in exchange for $0.75. After attrition and predefined exclusions, the final sample consisted of 2,167 participants. The study was conducted in April and May of 2014.

Manipulations

We selected as our money primes four manipulations that (a) have been used in prior money research (by us or other researchers), (b) were assumed to activate the idea of having ample money (rather than, e.g., having little money), and (c) could be administered online. Because some of these primes were assumed to operate unconsciously, we also included a comparison condition with an explicit instruction to contemplate the meaning of money. We included, as another comparison, the high- and low-power conditions from a commonly used manipulation in which participants remember times when they felt they had high or low power (Galinsky, Gruenfeld, & Magee, 2003). This allowed us to examine whether any effects of the money primes were unique to activating the concept of money as opposed to activating concepts related to social status or command of resources in general. After providing informed consent, participants were randomly assigned to 1 of 12 experimental conditions.

Background-image conditions

Participants assigned to the background-image/money condition saw a faint image of $100 bills in the background of the initial instruction screen, whereas participants assigned to the background-image/control condition saw a blurred version of this image, such that the bills were unrecognizable (Caruso et al., 2013; for a similar manipulation, see Kushlev, Dunn, & Ashton-James, 2012).

Perceptual-estimation conditions

Participants assigned to the perceptual-estimation/money condition were presented with a sketch of seven rectangles in different sizes and proportions. They were asked to select the three options that represented the best new sizes and shapes for the $10, $20, and $50 bills of U.S. currency. Participants assigned to the perceptual-estimation/control condition were asked to select three options for new sizes of Post-it notes (Shapira, Molouki, Mead, & Caruso, 2014).

Scrambled-phrases conditions

Participants assigned to the scrambled-phrases/money condition were presented with 30 combinations of five words (including 15 neutral phrases and 15 phrases related to money—e.g., “He has the capital”) and instructed to form correct phrases using four of the listed words. The scrambled-phrases/control condition included 30 neutral phrases conceptually unrelated to money (Boucher & Kofos, 2012; Vohs et al., 2006, Experiment 1).

Imagine-life conditions

Participants were asked to take a few minutes to imagine their lives in 5 years and then to write a paragraph describing what they imagined. Those in the imagine-life/abundance condition were asked to imagine having ample access to money and never having to worry about paying their bills; those in the imagine-life/scarcity condition were asked to imagine having little access to money and having to constantly worry about paying their bills; those in the imagine-life/control condition were asked to imagine what they would do and how they would feel tomorrow. The imagine-life/control condition also served as the control condition for the explicit-thought/money and power conditions described in the next paragraphs. To create the three imagine-life conditions, we adapted an essay manipulation from previous work (Vohs et al., 2006, Experiment 5) so that we could administer it online and to make it more comparable with the explicit-thought/money and power conditions. In the original study, participants also wrote essays in the presence of a large or small amount of Monopoly money or with no money present.

Explicit-thought condition

Participants in the explicit-thought/money condition were presented with a clear image of $100 bills and were asked to describe what money meant to them, what things the idea of money brought to their minds, and how the thought of money made them feel.

Power conditions

Participants in the high-power condition were asked to recall and write about an incident in which they had power over another person, whereas participants in the low-power condition were asked to recall and write about an incident in which someone else had power over them (Galinsky et al., 2003; Smith & Galinsky, 2010).

Dependent measures

After the manipulation, participants completed four sets of items that assessed constructs that have been previously shown or theorized to be affected by money priming.

Money-activation measure

A basic assumption in money-priming research is that the primes operate by making the concept of money cognitively accessible (Vohs et al., 2006). To test this assumption directly for each of the manipulations, we used a money-activation measure that we adapted from a stem-completion task that has been found to be influenced by the scrambled-phrases money-priming manipulation (Boucher & Kofos, 2012; Vohs et al., 2006, Supplemental Material).

Subjective-wealth measures

We assessed subjective wealth because merely thinking about large sums of money has been theorized to activate psychological states that are similar to those experienced when one actually possesses abundant resources (Vohs et al., 2006; Vohs, Mead, & Goode, 2008). To measure absolute subjective wealth, or how wealthy participants felt when an explicit comparison to other people was not elicited, we used five items (e.g., “I feel wealthy right now”; Chance & Norton, 2013). Scores on the five items were averaged (α = .94). To measure relative subjective wealth, or participants’ perception of their own wealth relative to other people’s wealth, we used two items: (a) “Based on your family’s household income, how wealthy would you say you are compared to your peers?” and (b) the MacArthur Socioeconomic Status Ladder Measure (Adler, Epel, Castellazzo, & Ickovics, 2000), an accepted measure of subjective social status (e.g., Kraus, Piff, Mendoza-Denton, Rheinschmidt, & Keltner, 2012; Piff, 2014). The two items were strongly correlated, r (2041) = .64, p < .001, and were thus standardized and averaged to form one index.

Self-sufficiency measures

We chose to measure self-sufficiency because it has been the focal construct theorized and shown to be influenced by thoughts of money (Baumeister & Vohs, 2015; Vohs et al., 2006, 2008). To directly measure participants’ subjective sense of general self-sufficiency, we used a four-item measure that we adapted from previous research (e.g., “Currently, I think that I can obtain most things by myself”; Lammers, Galinsky, Gordijn, & Otten, 2012, Experiment 3). Three items were averaged (α = .81); the last item was omitted because including it reduced the scale’s reliability. As a more implicit measure of self-sufficiency, we adapted four items from a task used in previous research to assess preference for “being alone even when choosing leisure activities” (e.g., choosing between watching “my three favorite movies alone” or receiving “two tickets to a movie”; Vohs et al., 2006, Experiment 8).

Trying to gauge preference for solitary activities as sensitively as possible, we also asked participants, after making each choice, to rate the strength of their preference for their chosen option. The resulting scores from three of the items were averaged. The fourth item was omitted because it reduced the reliability of the scale, but this remained low even after the exclusion (α = .42). Thus, we had two measures of preference for solitary activities: number of solitary activities (calculated as a simple count of the solitary choices in all four dichotomous items) and strength of preference for solitary activities (an average of scores on three of the continuous items).

Adjective ratings

Money reminders have been found to promote personal agency (e.g., X. Zhou, Vohs, & Baumeister, 2009) and reduce interpersonal concerns (e.g., Pfeffer & DeVoe, 2009; Vohs et al., 2006). To measure the relevant constructs of agency and communion, we asked participants to rate the extent to which they identified with a series of 12 adjectives derived from a communion-agency scale (e.g., for agency, “helpful,” “understanding”; for communion, “competent,” “assertive”; Abele & Wojciszke, 2007, Study 4). Following the method of Gasiorowska (2014), we also added the items “self-sufficient” and “strong” to measure the construct of self-sufficiency, as well as five other items that we deemed, on the basis of previous research, to be potentially pertinent to money activation: “independent,” “powerful,” “competitive,” “cooperative,” and “selfish.” An exploratory principal component analysis with a varimax (orthogonal) rotation yielded a three-factor solution for these adjective ratings: communion (α = .91), agency (α = .82), and self-sufficiency (α = .84).

Composite of dependent measures

To provide an overall snapshot of the efficacy of each manipulation, we calculated a composite score of the dependent measures by standardizing scores on all nine dependent-variable indexes and averaging the resulting z scores for each participant (we reverse-scored communion ratings because we expected money primes to decrease endorsement of adjectives related to communion). This approach allowed us to assess the effect of each manipulation across all of our dependent measures; larger values indicated larger effects in the expected directions. This composite was not in our preregistered analysis plan; we decided to adopt it after observing that most of our effects were fairly weak and inconsistent and after receiving feedback during the review process that such summary results would help clarify the overall pattern of findings for readers.

Demographic measures

Participants completed sociodemographic measures of gender, socioeconomic status (i.e., income and education), and political ideology that would allow us to explore potential moderators of the effects of our money primes. We used their responses to examine whether the interaction effects found in previous research would be replicated in such a large-scale investigation and to statistically address alternative explanations for any differences we might observe between the money conditions. We also collected data on participants’ age, marital status, number of children, religiosity, ethnicity, and native language.

Question order

We chose to always present the money-activation measure first (right after the manipulation) because we considered the extent to which each manipulation activated the concept of money fundamental to interpreting any other findings. We also did not want responses on this measure to be influenced by any other measures (such as the subjective-wealth measures, which could reasonably activate the concept of money independent of the prime). The other dependent variables were presented in one of three orders; we used a Latin Square design such that each set of measures appeared in each temporal position. After these measures, participants completed tasks related to two other unrelated research projects, the demographic measures mentioned earlier, an attention check, debriefing questions, and several questions for a third unrelated research project.

Results

Main analyses

To test the effect of each of the manipulations designed to activate the concept of money or power, we conducted planned contrast analyses on each of the dependent measures defined in the Method section. For each of the contrasts, we computed the effect size (Cohen’s d ) and its 95% confidence interval using Smithson’s (2016) SPSS script for noncentral t distributions (IBM SPSS, Version 23), and we calculated the sampling variance (v) of each effect size on the basis of the cell sample sizes in each comparison (see Lipsey & Wilson, 2001; Wilson, n.d.). Effect sizes and variances for all comparisons can be found in Table 1. (We report full confidence-interval information in the Supplemental Material.) As Table 1 shows, four of the five manipulations designed to activate the concept of money (the exception was the background-image manipulation) did so successfully, as measured by the stem-completion task. However, effects on the other outcomes were weak and inconsistent across measures. Only the imagine-life/abundance manipulation significantly affected the composite score, but it did so in the direction opposite to our expectation.

Table 1.

Results From Study 1: Effect Size (Cohen’s d ) and Sampling Variance (v) for Each Manipulation and Dependent Measure

Conditions compared	Money activation	Subjective wealth		General self-sufficiency	Solitary activities		Adjective ratings			Composite
Conditions compared	Money activation	Absolute	Relative	General self-sufficiency	Number	Strength of preference	Communion	Agency	Self-sufficiency	Composite
Background image/control vs. background image/money	−0.14(0.0088)	0.02(0.0089)	0.02(0.0089)	0.04(0.0089)	−0.01(0.0088)	−0.07(0.0089)	−0.05(0.0089)	−0.03(0.0089)	0.18(0.0089)	0.02(0.0088)
Perceptual estimation/control vs. perceptual estimation/money	0.24(0.0088)	−0.27(0.0089)	−0.21(0.0088)	−0.20(0.0089)	−0.01(0.0088)	0.12(0.0088)	0.09(0.0088)	−0.17(0.0089)	0.03(0.0088)	−0.16(0.0088)
Scrambled phrases/control vs. scrambled phrases/money	0.38(0.0112)	−0.01(0.0110)	−0.01(0.0110)	−0.10(0.0110)	−0.01(0.0110)	0.00(0.0110)	−0.10(0.0110)	−0.17(0.0111)	−0.18(0.0111)	−0.01(0.0110)
Imagine life/control vs. imagine life/scarcity	0.17(0.0127)	−0.22 (0.0128)	−0.19(0.0128)	−0.06(0.0127)	−0.02(0.0127)	−0.07(0.0127)	0.03(0.0127)	−0.11(0.0127)	0.07(0.0127)	−0.12(0.0127)
Imagine life/control vs. imagine life/abundance	0.28(0.0124)	−0.24(0.0124)	−0.12(0.0123)	−0.09(0.0123)	−0.24(0.0124)	−0.26(0.0124)	−0.04(0.0123)	−0.12(0.0123)	−0.10(0.0123)	−0.22(0.0123)
Imagine life/scarcity vs. imagine life/abundance	0.11(0.0122)	−0.01(0.0122)	0.07(0.0123)	−0.02(0.0123)	−0.22(0.0123)	−0.18(0.0123)	−0.07(0.0123)	−0.01(0.0122)	−0.17(0.0123)	−0.10(0.0122)
Imagine life/control vs. explicit thought/money	0.40(0.0119)	−0.03(0.0116)	0.03(0.0116)	−0.11(0.0117)	−0.09(0.0116)	−0.16(0.0117)	0.04(0.0116)	0.13(0.0117)	0.03(0.0116)	0.05(0.0116)
All control conditions vs. all (high) money conditions^a	0.25(0.0023)	−0.08(0.0023)	−0.05(0.0023)	−0.09(0.0023)	−0.05(0.0023)	−0.05(0.0023)	−0.02(0.0023)	−0.07(0.0023)	−0.01(0.0023)	−0.04(0.0023)
Imagine life/control vs. low power	−0.10(0.0155)	−0.05(0.0155)	0.08(0.0155)	0.03(0.0155)	−0.16(0.0155)	−0.17(0.0155)	0.25(0.0156)	0.20(0.0155)	0.25(0.0157)	−0.04(0.0155)
Imagine life/control vs. high power	−0.09(0.0157)	−0.07(0.0157)	0.07(0.0157)	−0.01(0.0158)	−0.28(0.0159)	−0.38(0.0160)	0.09(0.0158)	0.11(0.0158)	0.00(0.0158)	−0.20(0.0158)
Low power vs. high power	0.02(0.0184)	−0.02(0.0184)	−0.02(0.0184)	−0.04(0.0185)	−0.12(0.0185)	−0.21(0.0185)	−0.16(0.0186)	−0.09(0.0185)	−0.26(0.0188)	−0.16(0.0185)
High power vs. imagine life/abundance	0.37(0.0155)	0.18(0.0153)	0.19(0.0153)	0.07(0.0154)	−0.04(0.0153)	−0.12(0.0153)	0.13(0.0154)	0.23(0.0154)	0.10(0.0154)	0.02(0.0152)

Note: Values in parentheses are sampling variances. Positive effect sizes indicate higher scores in the condition mentioned second. Boldface type indicates effect sizes for which 95% confidence intervals did not include zero.

In the imagine-life/scarcity condition, people were asked to recall a time when they had little money. Although this condition referred to money, we predicted that it would not increase scores on any of the dependent variables we measured. Hence, the contrast here included only money conditions that referred to high amounts of money.

Potential moderators and confounds

We tested moderation of the main results for each of our dependent measures by gender, objective socioeconomic status, political ideology, and question order. Overall, we found little evidence that the effects of the manipulations were moderated by these variables. In addition, when we controlled for these variables, there was no meaningful change in the patterns of findings reported in the main analyses.

Mediation analyses

For each of the significant effects that did emerge for the money primes, we tested for mediation by money activation to assess our theoretical assumption that the priming procedure would affect the outcomes by making the concept of money more accessible. We found no evidence for such mediation.

Discussion

Four of the five manipulations we tested successfully activated the concept of money. This activation was specific to the money-related manipulations: An established manipulation of perceived power did not increase activation of money. However, effects of these manipulations on our dependent measures were sparse and were inconsistent with what we would expect on the basis of previous theorizing. For the most part, our main findings were not moderated by gender, socioeconomic status, political ideology, or the order in which our dependent measures were completed, and the findings remained similar when we controlled for other demographic measures.

We chose to always administer the money-activation measure (which typically has not been included in previous research) immediately after the manipulation to ensure that it was not contaminated by prior completion of any of our other dependent measures. However, one explanation for the scarce findings on the dependent measures could be that the money-activation measure interfered somehow with the influence of the primes on subsequent measures. We addressed this possibility in Study 2.

Study 2

We conducted Study 2 to examine the same manipulations and measures as in Study 1, and this time we omitted the money-activation measure. All other elements were the same as in Study 1.

Method

The study was conducted in June 2014. The recruitment of participants and the procedure were similar to those in Study 1 (except that the money-activation measure was not included). After attrition and predefined exclusions, the final sample consisted of 2,150 participants. After the main study, participants completed tasks related to three other unrelated research projects. Finally, participants answered the same series of demographic questions, the attention check, and the debriefing questions.

Results

Main analyses

We used the same analytic approach as in Study 1. Effect sizes and variances for this study are presented in Table 2. As in Study 1, the effects of our manipulations were weak and inconsistent across measures. Two manipulations significantly affected the composite of our dependent variables, but these effects were in the direction opposite to our expectation.

Table 2.

Results From Study 2: Effect Size (Cohen’s d ) and Sampling Variance (v) for Each Manipulation and Dependent Measure

Conditions compared	Subjective wealth		General self-sufficiency	Solitary activities		Adjective ratings			Composite
Conditions compared	Absolute	Relative	General self-sufficiency	Number	Strength of preference	Communion	Agency	Self-sufficiency	Composite
Background image/control vs. background image/money	−0.19(0.0092)	−0.15(0.0092)	−0.06(0.0092)	0.12(0.0093)	0.14(0.0092)	−0.12(0.0092)	−0.13(0.0092)	−0.15(0.0092)	−0.08(0.0092)
Perceptual estimation/control vs. perceptual estimation/money	−0.26(0.0095)	−0.22(0.0095)	−0.18(0.0095)	−0.01(0.0095)	0.01(0.0094)	−0.17(0.0095)	−0.12(0.0094)	−0.16(0.0095)	−0.22(0.0095)
Scrambled phrases/control vs. scrambled phrases/money	−0.02(0.0117)	0.00(0.0118)	−0.04(0.0118)	−0.04(0.0118)	0.08(0.0118)	0.05(0.0117)	0.13(0.0118)	0.08(0.0118)	0.04(0.0117)
Imagine life/control vs. imagine life/scarcity	−0.12(0.0124)	−0.15(0.0124)	−0.20(0.0124)	0.08(0.0123)	0.25(0.0124)	0.16(0.0124)	0.01(0.0123)	−0.20(0.0124)	−0.12(0.0123)
Imagine life/control vs. imagine life/abundance	−0.25(0.0118)	−0.20(0.0118)	−0.16(0.0117)	−0.13(0.0117)	−0.13(0.0117)	0.10(0.0117)	−0.07(0.0117)	−0.25(0.0118)	−0.37(0.0119)
Imagine life/scarcity vs. imagine life/abundance	−0.13(0.0124)	−0.04(0.0124)	0.04(0.0124)	−0.21(0.0124)	−0.38(0.0126)	−0.06(0.0125)	−0.08(0.0125)	−0.06(0.0125)	−0.25(0.0125)
Imagine life/control vs. explicit thought/money	−0.05(0.0107)	−0.11(0.0107)	−0.06(0.0107)	0.20(0.0107)	0.29(0.0108)	0.04(0.0107)	−0.21(0.0107)	−0.11(0.0107)	−0.03(0.0107)
All control conditions vs. all (high) money conditions^a	−0.15(0.0023)	−0.13(0.0023)	−0.10(0.0023)	0.02(0.0023)	0.07(0.0023)	−0.04(0.0023)	−0.08(0.0023)	−0.11(0.0023)	−0.13(0.0023)
Imagine life/control vs. low power	−0.19(0.0135)	−0.21(0.0135)	−0.14(0.0134)	0.14(0.0134)	0.25(0.0135)	0.13(0.0135)	−0.03(0.0134)	−0.13(0.0135)	−0.13(0.0134)
Imagine life/control vs. high power	−0.17(0.0146)	−0.14(0.0146)	−0.18(0.0146)	0.07(0.0146)	0.05(0.0147)	0.31(0.0148)	−0.03(0.0147)	0.01(0.0147)	−0.20(0.0147)
Low power vs. high power	0.02(0.0164)	0.07(0.0164)	−0.04(0.0164)	−0.07(0.0164)	−0.20(0.0165)	0.18(0.0166)	0.00(0.0165)	0.13(0.0165)	−0.07(0.0164)
High power vs. imagine life/abundance	0.08(0.0147)	0.06(0.0147)	−0.02(0.0147)	0.19(0.0147)	0.19(0.0148)	0.21(0.0148)	0.04(0.0148)	0.26(0.0149)	0.17(0.0147)

Potential moderators and confounds

As in Study 1, we found little evidence for moderation of our main results by gender, objective socioeconomic status, political ideology, or question order, and the effects we did find were for different contrasts than in Study 1. In addition, our main findings changed only slightly when we controlled for potential confounds.

Discussion

Removing the possible interference by the activation measure did not meaningfully change the effects of money primes on the outcomes we measured. Two manipulations did display effects on the dependent measures that were consistent (at least conceptually) across Studies 1 and 2 (but in a direction opposite our a priori expectation). Specifically, the perceptual-estimation manipulation decreased absolute and relative subjective wealth, and the imagine-life/abundance manipulation decreased absolute subjective wealth and the composite of our dependent variables. We found almost no evidence that our main findings were moderated by participants’ demographics or the order in which our dependent measures were completed.

Study 3

The online recruitment method used in Studies 1 and 2 allowed for large and heterogeneous samples that enabled assessment of a variety of money primes, but this procedure produced substantial attrition of participants (see the Supplemental Material) and did not include a behavioral measure of self-sufficiency. Although the findings remained similar when we addressed differential attrition statistically, our confidence in the results would be increased by observing similar effects when attrition is minimal. In addition, the sole reliance on self-report measures is a departure from prior research that used behavioral dependent variables (e.g., Vohs et al., 2006, Experiments 1–7, 9) and is potentially problematic because priming a concept such as money might affect actual behavior without affecting conscious judgments of the sort that we examined in Studies 1 and 2 (see, e.g., Baumeister, Vohs, & Funder, 2007). To address these two issues, in Study 3, we implemented our approach in a laboratory setting with a behavioral dependent measure.

Method

Participants

Participants were students and community members recruited in two research laboratories run by the Center for Decision Research at The University of Chicago Booth School of Business (one located on the university’s Hyde Park campus that mainly recruits undergraduate students at the university and one located in downtown Chicago that recruits a more representative community sample of Chicago residents). After attrition and predefined exclusions, the final sample consisted of 332 participants. No participant who started the experiment requested that the session be terminated for any reason before its completion. As the voluntary drop-out rate was 0%, any potential concerns about attrition (particularly selective attrition; see H. Zhou & Fishbach, 2016) in Studies 1 and 2 are absent from Study 3.

Procedure

The study was conducted from August 2016 through November 2016. After participants consented to participate, a research assistant explained the rules of a puzzle task and demonstrated how to solve a simple example of the task. After ensuring that participants understood the rules, the research assistant left the room. Manipulations and dependent measures were administered on a computer.

Manipulations

Because of constraints on the size of the pools of laboratory participants, we included only three manipulations. These manipulations (a) were designed to activate the concept of money (rather than power), (b) were implicit, and (c) were found to successfully activate the concept of money in Study 1. Participants were therefore randomly assigned to one of six experimental conditions: perceptual estimation/control, perceptual estimation/money, scrambled phrases/control, scrambled phrases/money, imagine life/control, and imagine life/abundance. These conditions were as described in Study 1.

Dependent measures

Because we observed no consistent order effects in the prior studies, the dependent measures in this study were administered in a fixed order. The puzzle task was always presented immediately after the manipulation, followed by the subjective-wealth measures, the self-sufficiency scales, and the adjective ratings from Studies 1 and 2. These were followed by the same demographic measures used in the previous studies.

Following the method of Vohs et al. (2006; Studies 1 and 2), we implemented a behavioral measure of self-sufficiency: the time that participants spent on a difficult puzzle before asking for help. We modeled the procedure for this study after Study 1 in Vohs et al. (2006), but we elected to use the unsolvable puzzle from their Study 2 (rather than the solvable one from their Study 1) because we were concerned that solving the puzzle correctly might induce an experience of self-sufficiency, which could plausibly interfere with any effect of our manipulations on the subsequent self-sufficiency measures.

Results

Main analyses

Time spent on the puzzle task was skewed, Shapiro-Wilk statistic = .88, df = 332, p < .001. In accordance with our preregistered analysis plan, we log-transformed this variable before analysis.

We used the same analytic approach as in Studies 1 and 2, except that we calculated two different composite dependent variables: One included all of our dependent measures, and one excluded the puzzle task (i.e., it included only self-report measures to allow for direct comparison with the composite we used in Studies 1 and 2). Effect sizes and variances for this study are presented in Table 3. The effects of our manipulations were weak and inconsistent across measures. Only one contrast analysis that we conducted was statistically significant (in the direction consistent with our a priori expectations). Specifically, the scrambled-phrases manipulation increased general self-sufficiency. No manipulation significantly affected either of the composite dependent scores.

Table 3.

Results From Study 3: Effect Size (Cohen’s d ) and Sampling Variance (v) for Each Manipulation and Dependent Measure

Conditions compared	Persistence on puzzle task	Subjective wealth		General self-sufficiency	Solitary activities		Adjective ratings			Composite	Composite (self-report)
Conditions compared	Persistence on puzzle task	Absolute	Relative	General self-sufficiency	Number	Strength of preference	Communion	Agency	Self-sufficiency	Composite	Composite (self-report)
Perceptual estimation/control vs. perceptual estimation/money	−0.23(0.0357)	−0.05(0.0355)	−0.06(0.0355)	−0.01(0.0355)	0.05(0.0358)	0.13(0.0355)	0.06(0.0355)	−0.09(0.0355)	0.07(0.0355)	−0.06(0.0355)	0.00(0.0355)
Scrambled phrases/control vs. scrambled phrases/money	0.22(0.0371)	0.09(0.0369)	0.00(0.0368)	0.54(0.0382)	0.05(0.0369)	0.06(0.0369)	0.32(0.0373)	0.13(0.0369)	0.36(0.0378)	0.28(0.0372)	0.24(0.0371)
Imagine life/control vs. imagine life/abundance	−0.07(0.0364)	0.05(0.0364)	−0.13(0.0365)	−0.22(0.0366)	−0.11(0.0365)	−0.20(0.0366)	0.10(0.0365)	0.17(0.0365)	0.00(0.0364)	−0.17(0.0365)	−0.16(0.0365)
All control conditions vs. all (high) money conditions^a	−0.03(0.0121)	0.03(0.0121)	−0.06(0.0121)	0.09(0.0121)	0.00(0.0121)	0.00(0.0121)	0.16(0.0121)	0.07(0.0121)	0.14(0.0121)	0.02(0.0121)	0.03(0.0121)

Study 3 did not have an imagine-life/scarcity condition, so the contrast here involved only money conditions that referred to high amounts of money.

Potential moderators and confounds

As in Studies 1 and 2, we found little evidence for moderation of our main results by gender, objective socioeconomic status, or political ideology. We also tested for moderation by participant pool, because the population of University of Chicago students may differ from other populations, given the school’s strong association with the discipline of economics and pro-free-market ideology (Vohs, 2015). Scores on the dependent measures indeed showed that University of Chicago participants differed from the community participants in several notable ways: They spent longer on the puzzle task, they scored higher on the subjective wealth measures, and they gave themselves lower adjective ratings for communion, agency, and self-sufficiency (for statistics, see the Supplemental Material). However, none of the effects of the money-priming manipulations were moderated by participant pool. In addition, our main findings changed only slightly when we controlled for potential confounds, as in Studies 1 and 2.

Discussion

In a laboratory experiment that (a) used a behavioral measure of self-sufficiency taken from previously published research, (b) eliminated concerns over participant attrition, and (c) tested for differences in the results between a student and a community sample, the three manipulations of money activation we tested did not exert consistent effects on our dependent measures. The one significant effect that did emerge in Study 3 was different from the significant effects that emerged in Studies 1 and 2. Furthermore, although the student and community samples we tested did differ in potentially relevant dimensions, the effects of the money primes did not significantly differ between the two samples.

General Discussion

We illustrate in this section how the methodological approach we used in these studies can help to specify the conditions under which psychological effects—in money-priming research or in other domains—can be expected to appear (if at all) and, in doing so, contribute to a reproducible, cumulative psychological science.

Explaining variation in money-priming effects

At least two of our findings directly inform whether money-priming effects are qualified by the priming procedure or by participant population. First, we found evidence that four different money primes reliably activated the construct of money. However, these manipulations did not have consistent effects on our dependent measures, either within or across studies. Second, none of the sociodemographic factors we assessed consistently moderated the effects of the money primes on the constructs we measured.

Of course, it is possible that additional factors, which varied or could have varied among our studies and previously published studies (e.g., participants’ attitudes toward money) or among the online studies and laboratory study in this article (e.g., participants’ level of distraction), might account for these apparent inconsistencies. We did not aim to conduct a direct replication of any specific past study, and therefore we encourage special care when using our findings to evaluate existing ones (Doyen, Klein, Simons, & Cleeremans, 2014; Stroebe & Strack, 2014).

The promise of our methodological approach

Beyond the implications for money-priming research, we believe our approach shows promise for ongoing efforts to develop reliable and well-specified psychological theories. Variance in methods and participant populations across studies in the same domain is all but inevitable in psychological research given that it is typically impossible to conduct a study more than once with identical participants using the exact same methods. We suggest that large-scale preregistered experiments, comparing multiple manipulations (or variations of other methodological elements) and assessing multiple individual-difference moderators within the same heterogeneous sample, can serve as a useful tool for identifying the variations that are meaningful and incorporating them into theories of human behavior. Although our current investigation did not identify any factors that consistently qualified money-priming effects, we remain hopeful about future implementations and elaborations of this general approach in additional domains with even larger sample sizes. Specifically, the approach could be elaborated to include formal testing of moderation by additional elements of the experimental design within and across studies. For instance, manipulation type (e.g., scrambled phrases vs. imagine life) and experimental setting (online vs. laboratory) could be statistically tested as moderators in experiments that use this approach.

This approach could complement other approaches for estimating the reliability and reproducibility of findings in psychology, such as meta-analysis and direct replication. In meta-analysis, moderation by participant characteristics can be uncovered only if the specific potential moderator has been consistently recorded and reported in the original studies. Furthermore, the set of studies included in a meta-analysis may suffer from problems such as publication bias and experimenter bias, which may be difficult to adjust for (e.g., van Elke et al., 2015). Likewise, direct replications often yield inconclusive results; when an attempted replication fails to reproduce the original effect, the original researchers can often propose some difference in the method or participants to explain the discrepancy. The approach we promote here could help overcome such potential deadlocks by collecting a fresh set of data to validate moderators suggested by meta-analyses or that surface in discussions after direct replications. Science progresses not only by speculating about what might be, but also by testing what actually is. In addition to helping researchers update their collective understanding of the nature and empirical support for money-priming effects, we hope that the approach we have implemented here will help foster new collaborative efforts to make our theories richer, more accurate, and more reliable.

Footnotes

Acknowledgements

We thank Nicholas Epley, Daniel Gilbert, Daniel Kahneman, Nira Liberman, Nicole Mead, Devin Pope, Jane Risen, Kathleen Vohs, and Adam Waytz for feedback on previous versions of the manuscript, and Zach Bradley, Morgan Britt, James Camp, Michelle (formerly Jacob) Chambers, Sarah Molouki, Emory Richardson, Ariadne Souroutzidis, and Haotian Zhou for assistance with this project.

Action Editor

Bill von Hippel served as action editor for this article.

Declaration of Conflicting Interests

The authors declared that they had no conflicts of interest with respect to their authorship or the publication of this article.

Funding

This work was supported by The University of Chicago Booth School of Business and by a grant from the John Templeton Foundation. The opinions expressed in this publication are those of the authors and do not necessarily reflect the views of the John Templeton Foundation.

Supplemental Material

Additional supporting information can be found at

Open Practices

All data and materials have been made publicly available via the Open Science Framework and can be accessed at https://osf.io/x25ea/. The design and analysis plans were preregistered at the Open Science Framework and can be accessed at https://osf.io/vxj5c/ (Study 1), https://osf.io/976ne/ (Study 2), and https://osf.io/fj2gt/ (Study 3). The complete Open Practices Disclosure for this article can be found at https://journals-sagepub-com.web.bisu.edu.cn/doi/suppl/10.1177/0956797617706161. This article has received badges for Open Data, Open Materials, and Preregistration. More information about the Open Practices badges can be found at .

References

Abele

A. E.

Wojciszke

(2007). Agency and communion from the perspective of self versus others. Journal of Personality and Social Psychology, 93, 751–763.

Adler

N. E.

Epel

E. S.

Castellazzo

Ickovics

J. R.

(2000). Relationship of subjective and objective social status with psychological and physiological functioning: Preliminary data in healthy, White women. Health Psychology, 19, 586–592.

Baumeister

R. F.

Vohs

K. D.

(2015). The meanings of money: Behavioral, interpersonal, cognitive, motivational, and affective consequences of money-related thoughts. Unpublished manuscript, Department of Psychology, Florida State University, Tallahassee.

Baumeister

R. F.

Vohs

K. D.

Funder

D. C.

(2007). Psychology as the science of self-reports and finger movements: Whatever happened to actual behavior? Perspectives on Psychological Science, 2, 396–403.

Boucher

H. C.

Kofos

M. N.

(2012). The idea of money counteracts ego depletion effects. Journal of Experimental Social Psychology, 48, 804–810.

Caruso

E. M.

Vohs

K. D.

Baxter

Waytz

(2013). Mere exposure to money increases endorsement of free-market systems and social inequality. Journal of Experimental Psychology: General, 142, 301–306.

Cesario

(2014). Priming, replication, and the hardest science. Perspectives on Psychological Science, 9, 40–48.

Chance

Norton

M. I.

(2013). I give, therefore I have: Giving and subjective wealth. Retrieved from http://faculty.som.yale.edu/zoechance/documents/zchancepaper1-jcr2ndround-givingandsubjectivewealth.pdf

Doyen

Klein

Simons

Cleeremans

(2014). On the other side of the mirror: Priming in cognitive and social psychology. Social Cognition, 32, 12–32.

10.

Galinsky

A. D.

Gruenfeld

D. H.

Magee

J. C.

(2003). From power to action. Journal of Personality and Social Psychology, 85, 453–466.

11.

Gasiorowska

(2014). Psychologize znaczenie pienie˛dzy: Dlaczego pienie˛dze wzbudzaja˛ koncentracje˛ na sobie [The psychological meaning of money: Why money induces self-focus]. Warsaw, Poland: Wydawnictwo Naukowe PWN.

12.

Klein

R. A.

Ratliff

Vianello

Adams

A. B.

Jr. Bahnk

Bernstein

N. B.

Nosek

B. A.

(2014). Investigating variation in replicability: A “Many Labs” replication project. Social Psychology, 45, 142–152.

13.

Kraus

M. W.

Piff

P. K.

Mendoza-Denton

Rheinschmidt

M. L.

Keltner

(2012). Social class, solipsism, and contextualism: How the rich are different from the poor. Psychological Review, 119, 546–572.

14.

Kushlev

Dunn

E. W.

Ashton-James

C. E.

(2012). Does affluence impoverish the experience of parenting? Journal of Experimental Social Psychology, 48, 1381–1384.

15.

Lammers

Galinsky

A. D.

Gordijn

E. H.

Otten

(2012). Power increases social distance. Social Psychological & Personality Science, 3, 282–290.

16.

Ledgerwood

(2014). Introduction to the special section on advancing our methods and practices. Perspectives on Psychological Science, 9, 275–277.

17.

Lindsay

S. D.

(2015). Replication in psychological science. Psychological Science, 26, 1827–1832.

18.

Lipsey

M. W.

Wilson

D. B.

(2001). Practical meta-analysis. Thousand Oaks, CA: Sage.

19.

Locke

E. A.

(2015). Theory building, replication, and behavioral priming: Where do we need to go from here? Perspectives on Psychological Science, 10, 408–414.

20.

Mogilner

(2010). The pursuit of happiness: Time, money, and social connection. Psychological Science, 21, 1348–1354.

21.

Nosek

B. A.

Spies

J. R.

Motyl

(2012). Scientific utopia II. Restructuring incentives and practices to promote truth over publishability. Perspectives on Psychological Science, 7, 615–631.

22.

Open Science Collaboration. (2015). Estimating the reproducibility of psychological science. Science, 349, 943.

23.

Pfeffer

DeVoe

S. E.

(2009). Economic evaluation: The effect of money and economics on attitudes about volunteering. Journal of Economic Psychology, 30, 500–508.

24.

Piff

P. K.

(2014). Wealth and the inflated self class, entitlement, and narcissism. Personality and Social Psychology Bulletin, 40, 34–43.

25.

Rohrer

Pashler

Harris

C. R.

(2015). Do subtle reminders of money change people’s political views? Journal of Experimental Psychology: General, 144, e73–e85.

26.

Schuler

Wänke

(2016). A fresh look on money priming: Feeling privileged or not makes a difference. Social Psychology and Personality Science, 7, 366–373.

27.

Shapira

Molouki

Mead

N. L.

Caruso

E. M.

(2014, February). I think I have, therefore I deserve: Thinking about money increases men’s sense of entitlement. Poster presented at the annual meeting of the Society for Personality and Social Psychology, Austin, TX.

28.

Simmons

J. P.

Nelson

L. D.

Simonsohn

(2011). False-positive psychology: Undisclosed flexibility in data collection and analysis allows presenting anything as significant. Psychological Science, 22, 1359–1366.

29.

Smith

P. K.

Galinsky

A. D.

(2010). The nonconscious nature of power: Cues and consequences. Social and Personality Psychology Compass, 4, 918–938.

30.

Smithson

(2016). Scripts and software for noncentral confidence interval and power calculations. Retrieved from http://www.michaelsmithson.online/stats/CIstuff/CI.html

31.

Stroebe

Strack

(2014). The alleged crisis and the illusion of exact replication. Perspectives on Psychological Science, 9, 59–71.

32.

Van Bavel

J. J.

Mende-Siedlecki

Brady

W. J.

Reinero

D. A

. (2016). Contextual sensitivity in scientific reproducibility. Proceedings of the National Academy of Sciences, USA, 113, 6454–6459.

33.

van Elke

Matzhe

Gronau

Q. F.

Guan

Vandekerckhove

Wagenmakers

E.-J

. (2015). Meta-analyses are no substitute for registered replications: A skeptical perspective on religious priming. Frontiers in Psychology, 6, Article 1365. doi:10.3389/fpsyg.2015.01365

34.

Vohs

K. D.

(2015). Money priming can change people’s thoughts, feelings, motivations, and behaviors: An update on 10 years of experiments. Journal of Experimental Psychology: General, 144, e86–e93.

35.

Vohs

K. D.

Mead

N. L.

Goode

M. R.

(2006). The psychological consequences of money. Science, 314, 1154–1156.

36.

Vohs

K. D.

Mead

N. L.

Goode

M. R.

(2008). Merely activating the concept of money changes personal and interpersonal behavior. Current Directions in Psychological Science, 17, 208–212.

37.

Wilson

D. B.

(n.d.). Practical meta-analysis effect size calculator. Retrieved from http://www.campbellcollaboration.org/artman2/uploads/1/2_D_Wilson__Calculating_ES.pdf

38.

Yong

J. C.

N. P.

(2012). Cash in hand, want better looking mate: Significant resource cues raise men’s mating standards. Personality and Individual Differences, 53, 55–58.

39.

Zhou

Fishbach

(2016). The pitfall of experimenting on the Web: How unattended selective attrition leads to surprising (yet false) research conclusions. Journal of Personality and Social Psychology, 111, 493–504.

40.

Zhou

Vohs

K. D.

Baumeister

R. F.

(2009). The symbolic power of money: Reminders of money alter social distress and physical pain. Psychological Science, 20, 700–706.

Supplementary Material

Please find the following supplemental material available below.

For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.

For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.

0.00 MB

0.59 MB

0.81 MB