Just Playing the Role of Good Study Participants? Evaluative Conditioning,Demand Compliance,and Agreeableness

Abstract

Evaluative Conditioning (EC) is the change in liking of stimuli due to their co-occurrence with other valenced stimuli. Recent research has shown stronger EC effects for more agreeable individuals. Because EC procedures are prone to demand characteristics, we hypothesized that more agreeable individuals might simply play the role of good study participants and therefore show stronger EC effects. We tested this in two preregistered experiments (N = 700). In Experiment 1, self-reported Agreeableness and a behavioral measure of Demand Compliance moderated EC. However, Agreeableness and Demand Compliance were uncorrelated, and the moderations were independent. Experiment 2 used an instructional EC paradigm, showing only a moderation by Demand Compliance but not Agreeableness. Our studies imply that although EC effects are related to Demand Compliance, more agreeable participants are not more likely to comply with demand characteristics in EC experiments.

Keywords

attitudes emotion individual differences personality social cognition

Evaluative Conditioning (EC), the change in liking of a conditioned stimulus (CS) resulting from its pairing with a positive/negative unconditioned stimulus (US), is a central effect in social psychology (De Houwer, 2007; Hofmann et al., 2010; Moran et al., 2023). EC effects occur in different social domains, such as advertising (Ingendahl, Vogel, Maedche, et al., 2023) or stereotype formation (French et al., 2013). For example, encountering an unknown stranger (CS) with a good friend (positive US) may lead to a more positive attitude toward the stranger.

Recent research has shown that EC effects are moderated by people’s personality, particularly Agreeableness (Ingendahl & Vogel, 2023; Vogel et al., 2019). Agreeable individuals, who are sympathetic, considerate, truthful, and supportive (Wilmot & Ones, 2022), show stronger EC effects. This moderation might seem surprising, as other personality traits—especially Neuroticism and Extraversion—have traditionally been associated with conditioning effects (Eysenck, 1962; Gray, 1981). Nevertheless, especially Agreeableness seems to reliably predict interindividual differences in EC (Ingendahl & Vogel, 2023; Vogel et al., 2019).

Previous research suggests that more agreeable individuals experience the USs more extremely, which is associated with¹ stronger EC effects (Ingendahl & Vogel, 2023; Vogel et al., 2019). In addition, they tend to have a more accurate memory of the stimulus pairings, which could contribute to the stronger EC effects (Ingendahl & Vogel, 2023). This article addresses one critical alternative explanation of why agreeable participants show stronger EC effects—compliance with experimental demand characteristics (Orne, 1962). Specifically, we test whether more agreeable people are just “nicer” study participants to the researcher and therefore show stronger EC effects.

Demand Characteristics in Evaluative Conditioning

Demand characteristics are “the totality of cues which convey an experimental hypothesis to the subject” (Orne, 1962, p. 779). They impose a severe threat on an experiment’s internal validity because they produce effects that appear to be caused by the construct of interest but are actually artifacts of the experimental situation. Despite being a standard component of methodological courses in undergraduate programs, demand characteristics are rarely considered in experimental research (Klein et al., 2012).

In a recent review, Corneille and Lush (2023) distinguished between three levels of demand characteristics: knowledge of the hypothesis, motivation to comply, and the strategy participants adopt to comply. In the following, we illustrate these three levels in the EC paradigm where Ingendahl and Vogel (2023) found a moderation by Agreeableness. In this paradigm, participants first evaluated highly positive/negative pictures. Next, participants saw a slideshow of the same positive/negative pictures alongside unfamiliar brand names. Afterward, the participants also evaluated the brand names.

In this paradigm, participants were likely aware that the experiment investigates how pairings with positive/negative pictures influence evaluations of brand names (knowledge of hypothesis): The study asks first for evaluations of highly positive/negative pictures, then shows these pictures with unknown brand names, and then asks for evaluations of the names. Ingendahl and Vogel (2023) did not assess participants’ hypothesis awareness, but other research suggests that participants are often aware of the underlying hypothesis in an EC experiment (Allen & Janiszewski, 1989; Corneille & Lush, 2023; Page, 1973).² Depending on participants’ motivation to comply with the inferred hypothesis, they may try to produce a hypothesis-consistent effect or show reactance and produce a hypothesis-inconsistent effect (Corneille & Lush, 2023). Participants might then exert different strategies for evaluating the CSs: conscious faking, conscious imagination of liking or disliking specific CSs, and phenomenological control (imagining but being unaware of it; see Dienes & Lush, 2023). If compliance is higher (lower), all three strategies lead to CS evaluations that are more (less) in line with the US valence—more positive (negative) evaluations of CSs paired with positive USs, and more negative (positive) evaluations of CSs paired with negative USs.

This EC paradigm and its proneness to demand characteristics are no exception in EC research (see Corneille & Lush, 2023, for a review). However, because EC is often studied from a functional perspective (De Houwer et al., 2013)—as the mere change in evaluations, independent of process assumptions—inferences drawn from stimulus pairings are unproblematic for studying EC. Some theories even propose that these inferences are a natural process underlying EC effects (De Houwer, 2018). However, it is important to distinguish between inferences that generalize beyond the EC paradigm (“This stimulus is shown with something positive, probably it’s also positive”) and those that will not (“The researchers probably test how stimulus pairings change evaluations”).³ Thus, to understand EC as a social psychological phenomenon in everyday life, it is crucial to understand to what extent EC effects arise just from demand characteristics.

To summarize, EC experiments like the one used by Ingendahl and Vogel (2023) are susceptible to demand characteristics. Participants are likely aware of the hypothesis and can exhibit behavior confirming it. What is unknown is to what extent participants are actually motivated to comply with the hypothesis they draw from an EC experiment. This is where the personality trait of Agreeableness might matter.

Agreeableness and Demand Compliance

Highly agreeable individuals are sympathetic, truthful, compassionate, cooperative, supportive, and considerate toward others (Wilmot & Ones, 2022). Agreeableness is a broad trait that subsumes the three aspects: compassion, trust, and politeness/respectfulness (Soto & John, 2017).

We are unaware of previous research investigating an association between Agreeableness and Demand Compliance in experiments. However, the core of Agreeableness is maintaining positive relationships with others (Graziano & Tobin, 2017). More agreeable individuals are thus more likely to comply with requests made by others (e.g., Carlo et al., 2005), which should include researchers conducting psychological experiments. Also, Agreeableness is associated with avoiding conflicts (Tehrani & Yamini, 2020), which should include potential conflicts with the researcher in an experiment. In addition, Agreeableness is related to the desire to please others and gain their approval (e.g., Leary et al., 2013)—including the researcher in an experiment. Finally, Agreeableness is associated with respect for authority figures and following social norms (Wilmot & Ones, 2022; but see Osborne et al., 2013). In a psychological experiment, the researcher is the authority figure, and the social norm is being a good study participant.

To summarize, even though there is no research on Agreeableness and Demand Compliance in experiments, previous research on Agreeableness suggests that such an association could exist.

Measuring Demand Compliance

Various strategies are employed for dealing with demand characteristics—also in EC research (see Corneille & Lush, 2023, for a review). For example, researchers could conceal the experiment’s purpose (Olson & Fazio, 2006) to keep participants from becoming aware of the hypothesis. Also, they could use less controllable measures of evaluation (Hütter et al., 2012), reducing participants’ ability to fake responses (but still allowing imagination or phenomenological control). Crucially, these strategies aim to avoid knowledge of the hypothesis or suppress specific behaviors, but they do not capture interindividual differences in participants’ motivation to comply with the researcher’s hypothesis.

One way of assessing such interindividual differences is a behavioral approach introduced by Nichols and Maner (2008). They told participants about an alleged hypothesis that people find pictures more pleasant when presented on the left side of the screen. Subsequently, participants saw several picture pairs and indicated their preference for the left or right picture. The authors found that the extent to which participants preferred the left pictures correlated with attitudes toward the experimenter and social desirability (Nichols & Maner, 2008).

This behavioral measure holds promise for our research question. First, the task resembles CS evaluations in an EC experiment. The extent to which participants show evaluations in line with a researcher’s hypothesis should influence the Demand Compliance task and the CS evaluations similarly (but see the “General Discussion” for a discussion of potential differences). Second, the task has been shown to capture interindividual differences. Third, the measure can be added to any experimental paradigm without modifying the paradigm itself.

Overview and Hypotheses

In this research, we tested whether the stronger EC effect for more agreeable individuals could be explained by higher Demand Compliance among these individuals. For that purpose, we first adapted the EC paradigm by Ingendahl and Vogel (2023) in Experiment 1 and also assessed participants’ Demand Compliance with a behavioral measure. Experiment 2 employed an instructional EC paradigm and will be introduced separately.

Experiment 1 should first of all show an EC effect, such that CSs paired with positive USs are evaluated more positively than those paired with negative USs. Based on our theorizing and previous findings, the EC effect should be stronger at higher levels of Agreeableness and Demand Compliance. Crucially, if more agreeable participants are simply more compliant, then Agreeableness should be positively associated with Demand Compliance, and the stronger EC effect for more agreeable individuals should be mediated by higher Demand Compliance.

Replicating and extending Ingendahl and Vogel (2023), we also tested whether these moderations occur simultaneously for US evaluations. Here, positive pictures should be evaluated more positively than negative ones. Again, this effect should be stronger at higher levels of Agreeableness and Demand Compliance, and the more extreme US evaluations for more agreeable individuals should be mediated by higher Demand Compliance.

We preregistered our hypotheses,⁴ methods, and analyses: https://aspredicted.org/zz4r8.pdf. All data, analysis scripts, and materials are in the following Open Science Framework (OSF) directory: https://doi.org/10.17605/OSF.IO/T9JXF. We report how we determined our sample size, all data exclusions, all manipulations, and all measures in the studies.

Experiment 1

Experiment 1 tested the role of Demand Compliance in the EC paradigm of Ingendahl and Vogel (2023) with an additional behavioral measure of Demand Compliance.

Method

Design and Participants

In a single-factor design, normed US valence (NUSV; positive vs. negative) varied within participants. Agreeableness and Demand Compliance served as continuous covariates. To determine the sample size, we conducted an a priori power analysis with G*Power (Faul et al., 2007). Based on Ingendahl and Vogel (2023), we expected r > .2 for the correlations between Agreeableness, EC, and Demand Compliance, requiring N = 255 for 90% power. We collected data from 350 native German speakers (213 male, 134 female, three non-disclosed, M_age = 33.95) via Prolific Academic, accounting for potential exclusions due to an attention check. More detailed demographic information for both experiments is provided in Online Supplement B.

Procedure and Materials

The experiment was adapted from Ingendahl and Vogel (2023) with only a few changes: Participants first filled out the 12 Agreeableness items of the German Big Five Inventory-2 (Danner et al., 2019). In contrast to Ingendahl and Vogel (2023), we did not assess the other Big Five but Nichols and Maner’s (2008) Demand Compliance measure. Afterward, the actual EC experiment started, where six (instead of 20) US pictures were drawn randomly from our stimulus pool and evaluated by the participants. These pictures were then paired with neutral CSs in a subsequent conditioning phase, followed by CS evaluations and a memory test for the stimulus pairings. In the following, we will explain each experimental task step by step. We provide a detailed list of all stimuli and screenshots of the tasks on the OSF.

Demand Compliance Task

As Nichols and Maner (2008), we told participants that this study tested whether presentations on the left/right side of the screen influence stimulus liking. The hypothesis was that stimulus presentations on the right (left) side led to higher liking, with between-participants counterbalancing which side was mentioned. Afterward, participants saw 20 stimulus pairs of neutral alien drawings and selected the alien they found more likable. Each alien pair actually consisted of the same stimulus twice, but we had debriefed participants that differences between the stimuli might be subtle. We assessed how often participants selected the stimulus consistent with the alleged hypothesis. After the 20 pairs, participants had to identify the hypothesis mentioned before (i.e., presentation on the left, presentation on the right, or don’t know). Following the preregistration, we excluded 18 participants who failed this attention check.

US Evaluation Task

We used the same 60 pictures from the Open Affective Standardized Image Set (OASIS; Kurdi et al., 2017) as USs as Ingendahl and Vogel (2023). Six US pictures (three per NUSV) were randomly drawn from the picture set and rated by the participants. Each picture was presented on its own slide, with the heading “Valence” and a labeled scale (very negative, moderately negative, somewhat negative, neutral, . . ., very positive).

Conditioning Procedure

After the US evaluations, participants were informed about upcoming presentations of unfamiliar brand names together with pictures. Next, six CSs were presented together with the US pictures. We used a random subset of the 36 fictitious brand names (e.g., STAREBO, DEMADOS) from Ingendahl and Vogel (2023) as CSs. Each conditioning trial started with a blank screen of 250 ms, followed by a CS-US pair presented for 2,500 ms. In our study, each CS was conditioned four (instead of five) times, leading to 24 trials presented in random order. Each CS was always shown with the same US.

CS Evaluation Task

Participants next evaluated the CSs on the same 7-point scale as the USs. On each slide, participants were asked, “How would you evaluate this brand name?” and presented with the brand name and the labeled scale (very negative, moderately negative, somewhat negative, neutral, . . ., very positive).

Pairing Memory Task

After the CS evaluations, participants should identify which specific picture had been paired with a brand name. Each CS was presented on a single slide with a matrix of four USs. Participants had to select the US picture the CS had been presented with among two pictures from each NUSV level. One picture was the correct US which counted as the correct response (see Ingendahl, Woitzel, Propheter, et al., 2023); one from each NUSV level had not been shown in the conditioning procedure.

Results

CS Evaluations (Preregistered)

We ran multilevel regression models in lme4 (Bates et al., 2019) using the highest converging random effect structure (Barr et al., 2013). We standardized all variables at the grand mean and coded NUSV with −0.5 (negative) and +0.5 (positive). This coding allows interpreting the effect of NUSV as Cohen’s d and the interactions of Agreeableness/Demand Compliance with NUSV as changes of this effect for ±1 SD of Agreeableness/Demand Compliance. The detailed regression results are displayed in Table 1 and visualized in Figure 1.

Table 1

Multilevel Regressions in Experiment 1

Predictors	(a) CS evaluations			(b) CS evaluations			(c) CS evaluations			(d) US evaluations			(e) US evaluations			(f) US evaluations			(g) CS evaluations
Predictors	b	95% CI	p	b	95% CI	p	b	95% CI	p	b	95% CI	p	b	95% CI	p	b	95% CI	p	b	95% CI	p
(Intercept)	−0.000	[−0.036, 0.036]	1.000	−0.000	[−0.036, 0.036]	1.000	−0.000	[−0.036, 0.036]	1.000	0.000	[−0.023, 0.023]	1.000	0.000	[−0.023, 0.023]	1.000	0.000	[−0.023, 0.023]	1.000	0.003	[−0.029, 0.035]	.869
NUSV	1.157	[1.057, 1.257]	<.001	1.157	[1.056, 1.258]	<.001	1.157	[1.058, 1.256]	<.001	1.780	[1.733, 1.828]	<.001	1.780	[1.733, 1.828]	<.001	1.780	[1.734, 1.827]	<.001	0.130	[−0.004, 0.265]	.058
A	−0.003	[−0.038, 0.033]	.889				−0.003	[−0.039, 0.032]	.850	0.031	[0.008, 0.054]	.009				0.030	[0.007, 0.053]	.010	−0.018	[−0.050, 0.014]	.277
A × NUSV	0.202	[0.102, 0.302]	<.001				0.196	[0.097, 0.295]	<.001	0.094	[0.047, 0.142]	<.001				0.091	[0.045, 0.138]	<.001	0.119	[0.035, 0.203]	.006
DC				0.018	[−0.017, 0.054]	.315	0.018	[−0.017, 0.054]	.311				0.014	[−0.009, 0.037]	.244	0.012	[−0.011, 0.035]	.296	0.016	[−0.015, 0.048]	.314
DC × NUSV				0.127	[0.025, 0.228]	.014	0.117	[0.017, 0.216]	.021				0.067	[0.019, 0.115]	.006	0.063	[0.016, 0.109]	.009	0.095	[0.009, 0.180]	.030
																			0.571	[0.492, 0.650]	<.001

Note. All variables were standardized, NUSV was coded with 0.5 (positive) and −0.5 (negative). The last model (g) was not preregistered. Bold p-values are significant at an alpha level of 5%. CS = conditioned stimulus; CI = confidence interval; NUSV = normed US valence; A = agreeableness; DC = demand compliance; US = unconditioned stimulus.

Figure 1

Predicted CS Evaluations and US Evaluations in Experiment 1

The first model showed an EC effect such that positive NUSV enhanced CS evaluations (p < .001, Table 1a). As expected, this effect was stronger at higher Agreeableness, as shown by the Agreeableness × NUSV interaction (p < .001). Likewise, the EC effect was stronger at higher Demand Compliance, as shown by the Demand Compliance × NUSV interaction (p = .014, Table 1b). Crucially, the Pearson correlation between Agreeableness and Demand Compliance was not significant (r = .05, p = .365). A Bayesian test (not preregistered) with the BayesFactor package (Morey & Rouder, 2018) showed moderate evidence for the null hypothesis (BF₁₀ = 0.19). Accordingly, a model with both Agreeableness and Demand Compliance showed no reduction in the Agreeableness × NUSV interaction (Table 1c), suggesting fully independent moderations and making the preregistered mediation analysis obsolete.

US Evaluations (Preregistered)

We analyzed the US evaluations with the same analytical approach (see Table 1d–f). Normatively positive USs were evaluated more positively than normatively negative USs (p < .001). In line with the findings on CS evaluations, this effect was more pronounced at higher levels of Agreeableness (p < .001) and Demand Compliance (p = .006). However, as for the CS evaluations, the moderations were entirely independent, and a mediation analysis was obsolete.

Moderated Mediation (Not Preregistered)

Given the similar moderations on CS and US evaluations, we examined whether individual US evaluations mediated the EC effect and whether the moderations on US evaluations mediated the moderations on CS evaluations, as found by Ingendahl and Vogel (2023). We thus conducted a multilevel moderated mediation analysis with the mediation package (Tingley et al., 2014), where Agreeableness and Demand Compliance moderated the effect of NUSV on US evaluations. When controlling statistically for US evaluations, the Agreeableness × NUSV and the Demand Compliance × NUSV interactions became smaller but were still significant (Table 1g). We computed conditional direct and indirect effects at different levels of Agreeableness and Demand Compliance (Figure 2). Both direct and indirect effects increased at higher levels, suggesting that more extreme US evaluations partially mediated the stronger EC effect for higher Agreeableness or Demand Compliance.

Figure 2

Conditional Direct and Indirect EC Effects in Experiment 1

Discussion

In Experiment 1, we found evidence against the idea that more agreeable individuals comply more with demand characteristics. Even though Agreeableness and Demand Compliance moderated the EC effect, they were uncorrelated and had independent moderations. Experiment 1 also showed similar moderations of Agreeableness and Demand Compliance on US evaluations, which partially mediated the moderations on the CS evaluations. Overall, these findings speak against the hypothesis that more agreeable participants are simply more compliant with experimental demand characteristics.

However, a test with a different methodology may be necessary to be more confident that the moderation by Agreeableness is unrelated to Demand Compliance. One possibility is a quasi-control procedure (Corneille & Lush, 2023). Here, participants are instructed to behave as if they were participating in the experiment but are not exposed to the experimental procedure. A similar approach is known in EC research as an instructional EC paradigm, where participants are only told that specific CSs will be paired with positive/negative USs, but do not actually experience any stimulus pairings (De Houwer, 2006; Hütter & De Houwer, 2017). Even though instructional EC paradigms are not necessarily more prone to demand effects (Corneille & Bena, 2023), effects in these paradigms are driven mostly by inferences participants draw from instructed pairings, including inferences regarding their own role as study participants. We therefore conducted a conceptual replication of Experiment 1, where participants were exposed to an instructional EC paradigm instead of actual pairings.

Experiment 2

Experiment 2 was identical to Experiment 1, except that participants were only told about the stimulus pairings instead of actual pairings. Experiment 2 was preregistered at https://aspredicted.org/vp55b.pdf.

Method

Design and Participants

Experiment 2 followed the same method as Experiment 1 with the following changes: After the US evaluation, participants underwent an instructional EC procedure adapted from Hütter and De Houwer (2017). Participants were told that they would be presented with unfamiliar brand names and positive or negative pictures later in the experiment. The CSs were then revealed on two screens for 15 s each, indicating which brand names would be paired with positive or negative pictures. The CS evaluation followed as in Experiment 1. Detailed instructions and screenshots can be found on the OSF.

We also modified the pairing memory task because the CSs were not shown with US pictures. Participants were instructed to recollect whether each CS would be later shown with positive or negative pictures. The memory test only contained two buttons: “positive pictures” and “negative pictures.”

We collected data from 350 native German speakers (178 male, 169 female, 3 non-disclosed, M_age = 32.11) via Prolific, excluding one participant without logged data. In line with the preregistration, 32 participants who failed the attention check were excluded.

Results

CS Evaluations (Preregistered)

We used the same analytical approach as in Experiment 1. First, the positive NUSV effect on CS evaluations showed a significant instructional EC effect (p < .001, Table 2a, Figure 3). In contrast to Experiment 1, the Agreeableness × NUSV interaction was descriptively in the opposite direction than expected, with descriptively weaker EC effects for more agreeable participants (p = .353). As in Experiment 1, the EC effect was stronger for higher Demand Compliance, shown by the Demand Compliance × NUSV interaction (p = .047, Table 2b). As in Experiment 1, there was no correlation between Demand Compliance and Agreeableness (r = −.06, p = .365, BF₁₀ = 0.22). Accordingly, a model with both Agreeableness and Demand Compliance showed fully independent effects, making the preregistered mediation analysis obsolete.

Table 2

Multilevel Regressions in Experiment 2

Predictors	(a) CS evaluations			(b) CS evaluations			(c) CS evaluations			(d) US evaluations			(e) US evaluations			(f) US evaluations
Predictors	b	95% CI	p	b	95% CI	p	b	95% CI	p	b	95% CI	p	b	95% CI	p	b	95% CI	p
(Intercept)	0.000	[−0.038, 0.038]	1.000	0.000	[−0.038, 0.038]	1.000	0.000	[−0.037, 0.037]	1.000	0.000	[−0.020, 0.020]	1.000	0.000	[−0.020, 0.020]	1.000	0.000	[−0.020, 0.020]	1.000
NUSV	0.778	[0.658, 0.898]	<.001	0.778	[0.659, 0.898]	<.001	0.778	0.659, 0.898]	<.001	1.780	[1.730, 1.830]	<.001	1.780	[1.730, 1.830]	<.001	1.780	[1.730, 1.830]	<.001
A	0.037	[−0.001, 0.075]	.055				0.038	[0.001, 0.075]	.043	0.004	[−0.016, 0.023]	.698				0.004	[−0.015, 0.024]	.677
A × NUSV	–0.057	[−0.177, 0.063]	.353				–0.050	[−0.170, 0.070]	.412	0.042	[−0.008, 0.091]	.101				0.040	[−0.010, 0.090]	.113
DC				0.012	[−0.026, 0.050]	.546	0.014	[−0.023, 0.051]	.458				0.005	[−0.015, 0.024]	.641	0.005	[−0.015, 0.024]	.625
DC × NUSV				0.121	[0.001, 0.241]	.047	0.118	[−0.002, 0.238]	.053				–0.024	[−0.074, 0.026]	.339	–0.022	[−0.072, 0.028]	.387

Note. All variables were standardized, NUSV was coded with 0.5 (positive) and −0.5 (negative). Bold p-values are significant at an alpha level of 5%. CS = conditioned stimulus; CI = confidence interval; NUSV = normed US valence; A = agreeableness; DC = demand compliance; US = unconditioned stimulus.

Figure 3

Predicted CS Evaluations and US Evaluations in Experiment 2

US Evaluations (Preregistered)

We analyzed the US evaluations with the same analytical approach (Table 2d–f). As in Experiment 2, normatively positive USs were evaluated more positively than normatively negative USs (p < .001). However, even though the Agreeableness × NUSV moderation went in the expected direction, it was not significant (p = .101). Also, inconsistent with Experiment 1, US evaluations were not more extreme for higher Demand Compliance (p = .339). Due to these findings, a mediation analysis was obsolete. We did not conduct further exploratory mediation analyses because the CSs were not actually paired with the US pictures in Experiment 2.

Discussion

Experiment 2 showed a dissociation between Agreeableness and Demand Compliance in an instructional EC paradigm. Agreeableness did not moderate EC effects—however, Demand Compliance did. As in Experiment 1, Agreeableness and Demand Compliance were entirely uncorrelated. Even though these findings cannot show whether a moderation by Agreeableness is statistically explained by Demand Compliance (because Agreeableness did not moderate EC), they show that different moderations of Agreeableness and Demand Compliance can emerge in instructional EC paradigms. In contrast to Experiment 1, we did not replicate the moderations on US evaluations.

General Discussion

In two experiments, we investigated whether the stronger Evaluative Conditioning (EC) effect for more agreeable individuals could be due to higher Demand Compliance. Experiment 1 showed that both Agreeableness and Demand Compliance independently moderate the EC effect in the procedure of Ingendahl and Vogel (2023). In addition, similar moderations of Agreeableness and Demand Compliance on evaluations of positive/negative USs partially mediated the moderations on the EC effect. Experiment 2 utilized an instructional EC paradigm. Here, only Demand Compliance but not Agreeableness was associated with stronger EC effects. None of the moderations on the US evaluations was significant. In both experiments, Agreeableness and Demand Compliance were uncorrelated. These findings offer important insights for research on EC and Agreeableness.

Implications for EC

One central implication of this research concerns the correlation between Demand Compliance and EC effects. Even though the influence of Demand Compliance in EC has been discussed in previous research, studies have focused primarily on participants’ knowledge of the hypothesis and how to suppress specific strategies when evaluating the CSs (for a review, see Corneille & Lush, 2023). Our studies show that EC effects correlate with participants’ Demand Compliance which, to some extent, questions the generalizability of EC effects beyond the lab. However, the correlation was modest, and EC effects emerged across all levels of Demand Compliance. Even if we consider only the participants with the lowest Demand Compliance (e.g., below −2 SD in Figures 1–3), the EC effect was still substantial.⁵ Therefore, we conclude that differences in EC effects are only partially related to Demand Compliance. Notably, the correlation between Demand Compliance and EC was not stronger in an instructional EC paradigm than in a picture-picture paradigm. This could imply that instructional EC effects are qualitatively similar to those in a standard EC procedure (Hütter & De Houwer, 2017), at least regarding how sensitive they are to Demand Compliance.

A second important implication of our research is that there is a substantive moderation by Agreeableness that seems independent of Demand Compliance. Even though previous research speculated that agreeable participants simply play the role of good study participants (Corneille & Lush, 2023; Ingendahl & Vogel, 2023), we do not find support for this explanation. Instead, the findings from Experiment 1 support the explanation by Ingendahl and Vogel (2023): Agreeable individuals perceive the USs as more extreme and might therefore show stronger EC effects. Even though the moderation by Agreeableness on US evaluations fell below significance in Experiment 2, the results from both experiments combined suggest a relationship between Agreeableness and more extreme US evaluations, r = .15, p < .001. Also, in Experiment 2, the CSs were not shown together with any US pictures, and the correlation between EC and Agreeableness vanished. This suggests that once interindividual differences in US valence cannot contribute to EC effects, Agreeableness is unrelated to EC.

Implications for Agreeableness

Our findings also offer valuable information about Agreeableness. First, our findings from both experiments combined support recent evidence that Agreeableness is associated with a more extreme perception of affective stimuli (Bresin & Robinson, 2015; Finley et al., 2017; Ingendahl & Vogel, 2022). It seems that not only Neuroticism or Extraversion but also Agreeableness is genuinely related to interindividual differences in emotional experiences.

Furthermore, our research shows that these interindividual differences are not merely a byproduct of Demand Compliance. Highly agreeable people seem to like positive stimuli more and negative stimuli less than disagreeable participants. Even though the core of Agreeableness is maintaining positive relationships with others (Graziano & Tobin, 2017), this does not necessarily extend to complying with researchers’ hypotheses in psychological experiments.

However, one further finding deserves mentioning: Agreeableness was unrelated to memory accuracy (Tables 3 and 4), in contrast to Ingendahl and Vogel (2023). Yet, there are methodological reasons for this. Experiment 1 had a strong ceiling effect with 95% memory accuracy, limiting the chances of finding a correlation with Agreeableness. Experiment 2 did not assess actual pairing memory but prospective memory on which CSs would be paired with positive/negative USs. Even though there was no ceiling effect here, no correlation with Agreeableness emerged, which could imply that the memory advantage among agreeable individuals is limited to experienced stimulus pairings in the environment.

Table 3

Correlations and Descriptive Statistics in Experiment 1

Variables	A	DC	EC	US_+-	MEM
A	(.80)	.05	.21	.21	.03
DC		(.71)	.13	.15	–.01
EC			(—)	.42	.24
US_+-				(—)	.00
MEM					(—)
M	3.65	0.54	2.00	4.11	0.95
SD	0.50	0.2	1.63	1.03	0.14

Note. Values in brackets represent Cronbach’s alpha. All correlations > |.12| are significant when applying the Holm–Bonferroni correction. Tests were conducted with the psych package (Revelle, 2022). A = agreeableness (scale ranging from 1 to 5); DC = demand compliance (% of trials); EC = difference score in CS evaluations for positive versus negative normed US valence; US_+- = difference score in US evaluations for positive versus negative normed US valence; MEM = pairing memory (% correct).

Table 4

Correlations and Descriptive Statistics in Experiment 2

Variables	A	DC	EC	US_+-	MEM
A	(.84)	–.06	–.05	.09	.02
DC		(.71)	.11	–.05	–.08
EC			(—)	.10	.36
US_+-				(—)	.02
MEM					(—)
M	3.68	0.53	1.23	4.16	0.77
SD	0.55	0.19	1.72	1.05	0.24

Note. Values in brackets represent Cronbach’s alpha. All correlations >|.11| are significant when applying the Holm–Bonferroni correction. Tests were conducted with the psych package (Revelle, 2022). A = agreeableness (scale ranging from 1 to 5); DC = demand compliance (% of trials); EC = difference score in CS evaluations for positive versus negative normed US valence; US_+- = difference score in US evaluations for positive versus negative normed US valence; MEM = pairing memory (% correct).

Limitations and Directions for Future Research

One conceptual limitation of our studies is that we focused exclusively on the motivation to comply (Corneille & Lush, 2023) because we considered it more related to the core of Agreeableness than awareness of the hypothesis or selecting a specific strategy. Nevertheless, future research might consider these other aspects of demand characteristics and their relation to personality traits.

A second limitation concerns generalizability. Experiment 1 suggests that the relationship between Agreeableness and EC found by Ingendahl and Vogel (2023) is incremental to Demand Compliance. However, other experimental paradigms, for instance, those that involve the direct interaction of researcher and participant, may show such an association.

A third limitation concerns the validity of our Demand Compliance measure. Even though the measure correlated with attitudes toward the experimenter and social desirability in the original study (Nichols & Maner, 2008) and correlated with the EC effect in our experiments, it is unclear how the measure relates for example to self-report measures of Demand Compliance as used in previous EC research (Bar-Anan et al., 2010; Kasran et al., 2022). Because responses on self-reports may themselves be biased by Demand Compliance (Corneille & Lush, 2023), we would argue that a behavioral measure using stimulus evaluations is closest to the effect of interest. Yet, one could argue that this closeness primarily applies to faking responses, whereas demand effects in the EC procedure might also arise from imagination or phenomenological control (Corneille & Lush, 2023). It might be easily imaginable for participants that neutral stimuli presented together with positive (negative) stimuli should be liked (disliked), but less imaginable that presenting stimuli on the left/right predicts liking. This might explain why participants’ sensitivity to demand effects as measured in our “left/right” procedure bears a weak relation to participants’ sensitivity to demand effects in the more straightforward “positive/negative US” procedure.

Finally, we tested in this research whether interindividual differences in EC are related to Agreeableness and Demand Compliance. Because we measured these constructs, any relationship except for the experimentally induced EC effect should not be interpreted as causal.

Conclusion

Even though Agreeableness is sometimes considered the least exciting personality trait in the Big Five, our findings suggest that this trait is substantially related to interindividual differences in emotional experiences and learning—and not simply because agreeable people are nicer study participants to the researcher.

Supplemental Material

sj-docx-1-spp-10.1177_19485506231198653 – Supplemental material for Just Playing the Role of Good Study Participants? Evaluative Conditioning, Demand Compliance, and Agreeableness

Supplemental material, sj-docx-1-spp-10.1177_19485506231198653 for Just Playing the Role of Good Study Participants? Evaluative Conditioning, Demand Compliance, and Agreeableness by Moritz Ingendahl, Johanna Woitzel and Hans Alves in Social Psychological and Personality Science

Supplemental Material

sj-docx-2-spp-10.1177_19485506231198653 – Supplemental material for Just Playing the Role of Good Study Participants? Evaluative Conditioning, Demand Compliance, and Agreeableness

Supplemental material, sj-docx-2-spp-10.1177_19485506231198653 for Just Playing the Role of Good Study Participants? Evaluative Conditioning, Demand Compliance, and Agreeableness by Moritz Ingendahl, Johanna Woitzel and Hans Alves in Social Psychological and Personality Science

Footnotes

Handling Editor: Yoav Bar-Anan

Declaration of Conflicting Interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The author(s) received no financial support for the research, authorship, and/or publication of this article.

ORCID iDs

Moritz Ingendahl

Hans Alves

Supplemental Material

Supplemental material for this article is available online.

Notes

Author Biographies

Moritz Ingendahl is a postdoctoral researcher at the Social Cognition lab of Ruhr University Bochum. His research examines the cognitive mechanisms underlying attitude change and decision-making in social and consumer settings.

Johanna Woitzel is a doctoral researcher at the Social Cognition lab of Ruhr University Bochum. Her primary research interest lies in understanding the processes of attitude formation within intergroup contexts.

Hans Alves is an associate professor for Social Cognition at the Ruhr-University Bochum. His research examines the cognitive and informational mechanisms underlying social psychological phenomena.

References

Allen

C. T.

Janiszewski

C. A.

(1989). Assessing the role of contingency awareness in attitudinal conditioning with implications for advertising research. Journal of Marketing Research, 26(1), 30–43. https://doi.org/10.1177/002224378902600103

Bar-Anan

De Houwer

Nosek

B. A.

(2010). Evaluative conditioning and conscious knowledge of contingencies: A correlational investigation with large samples. The Quarterly Journal of Experimental Psychology, 63(12), 2313–2335. https://doi.org/10.1080/17470211003802442

Barr

D. J.

Levy

Scheepers

Tily

H. J.

(2013). Random effects structure for confirmatory hypothesis testing: Keep it maximal. Journal of Memory and Language, 68(3), 255–278. https://doi.org/10.1016/j.jml.2012.11.001

Bates

Mächler

Bolker

Walker

(2015). Fitting linear mixed-effects models using lme4. Journal of Statistical Software, 67(1), 1–48. https://doi.org/10.18637/jss.v067.i01

Bresin

Robinson

M. D.

(2015). You are what you see and choose: Agreeableness and situation selection. Journal of Personality, 83(4), 452–463. https://doi.org/10.1111/jopy.12121

Carlo

Okun

M. A.

Knight

G. P.

de Guzman

M. R. T.

(2005). The interplay of traits and motives on volunteering: Agreeableness, extraversion and prosocial value motivation. Personality and Individual Differences, 38(6), 1293–1305. https://doi.org/10.1016/j.paid.2004.08.012

Corneille

Bena

(2023). Instruction-based replication studies raise challenging questions for psychological science. Collabra: Psychology, 9(1), Article 82234.

Corneille

Lush

(2023). Sixty years after Orne’s American psychologist article: A conceptual framework for subjective experiences elicited by demand characteristics. Personality and Social Psychology Review, 27(1), 83–101. https://doi.org/10.1177/10888683221104368

Danner

Rammstedt

Bluemke

Lechner

Berres

Knopf

Soto

C. J.

John

O. P.

(2019). Das big five inventar 2. Diagnostica, 65(3), 121–132. https://doi.org/10.1026/0012-1924/a000218

10.

De Houwer

(2006). Using the implicit association test does not rule out an impact of conscious propositional knowledge on evaluative conditioning. Learning and Motivation, 37(2), 176–187. https://doi.org/10.1016/j.lmot.2005.12.002

11.

De Houwer

(2007). A conceptual and theoretical analysis of evaluative conditioning. The Spanish Journal of Psychology, 10(2), 230–241. https://doi.org/10.1017/S1138741600006491

12.

De Houwer

(2018). Propositional models of evaluative conditioning. Social Psychological Bulletin, 13(3), 1–21. https://doi.org/10.5964/spb.v13i3.28046

13.

De Houwer

Gawronski

Barnes-Holmes

(2013). A functional-cognitive framework for attitude research. European Review of Social Psychology, 24(1), 252–287. https://doi.org/10.1080/10463283.2014.892320

14.

Dienes

Lush

(2023). The role of phenomenological control in experience. Current Directions in Psychological Science, 32(2), 145–151. https://doi.org/10.1177/09637214221150521

15.

Eysenck

H. J.

(1962). Conditioning and personality. British Journal of Psychology, 53(3), 299–305. https://doi.org/10.1111/j.2044-8295.1962.tb00835.x

16.

Faul

Erdfelder

Lang

A.-G.

Buchner

(2007). G*Power 3: A flexible statistical power analysis program for the social, behavioral, and biomedical sciences. Behavior Research Methods, 39, 175–191. https://doi.org/10.3758/BF03193146

17.

Finley

A. J.

Crowell

A. L.

Harmon-Jones

Schmeichel

B. J.

(2017). The influence of agreeableness and ego depletion on emotional responding. Journal of Personality, 85(5), 643–657. https://doi.org/10.1111/jopy.12267

18.

French

A. R.

Franz

T. M.

Phelan

L. L.

Blaine

B. E.

(2013). Reducing Muslim/Arab stereotypes through evaluative conditioning. The Journal of Social Psychology, 153(1), 6–9. https://doi.org/10.1080/00224545.2012.706242

19.

Gray

J. A.

(1981). A critique of eysenck’s theory of personality BT–A model for personality ( Eysenck

H. J.

, Ed., pp. 246–276). Springer. https://doi.org/10.1007/978-3-642-67783-0_8

20.

Graziano

W. G.

Tobin

R. M.

(2017). Agreeableness and the five factor model. In Widiger

T. A.

(Ed.), The Oxford handbook of the five factor model (Vol. 1, pp. 105–131). Oxford University Press.

21.

Hayes

A. F.

(2015). An index and test of linear moderated mediation. Multivariate Behavioral Research, 50(1), 1–22. https://doi.org/10.1080/00273171.2014.962683

22.

Hofmann

De Houwer

Perugini

Baeyens

Crombez

(2010). Evaluative conditioning in humans: A meta-analysis. Psychological Bulletin, 136(3), 390–421. https://doi.org/10.1037/a0018916

23.

Hütter

De Houwer

(2017). Examining the contributions of memory-dependent and memory-independent components to evaluative conditioning via instructions. Journal of Experimental Social Psychology, 71, 49–58. https://doi.org/10.1016/j.jesp.2017.02.007

24.

Hütter

Sweldens

Stahl

Unkelbach

Klauer

K. C.

(2012). Dissociating contingency awareness and conditioned attitudes: Evidence of contingency-unaware evaluative conditioning. Journal of Experimental Psychology: General, 141(3), 539–557. https://doi.org/10.1037/a0026477

25.

Ingendahl

Vogel

(2022). Stimulus evaluation in the eye of the beholder: Big five personality traits explain variance in normed picture sets. Personality Science, 3, 1–21. https://doi.org/10.5964/ps.7951

26.

Ingendahl

Vogel

(2023). (Why) do big five personality traits moderate evaluative conditioning? The role of US extremity and pairing memory. Collabra: Psychology, 9(1), Article 74812. https://doi.org/10.1525/collabra.74812

27.

Ingendahl

Vogel

Maedche

Wänke

(2023). Brand placements in video games: How local in-game experiences influence brand attitudes. Psychology & Marketing, 40, 274–287. https://doi.org/10.1002/mar.21770

28.

Ingendahl

Woitzel

Propheter

Wänke

Alves

(2023). From deviant likes to reversed effects: Re-investigating the contribution of unaware evaluative conditioning to attitude formation. Collabra: Psychology. https://osf.io/tpa3v/?view_only=efece7baf55940aa9707f3861f3369ca

29.

Kasran

Hughes

De Houwer

(2022). Observational evaluative conditioning is sensitive to relational information. Quarterly Journal of Experimental Psychology, 75(11), 2043–2063. https://doi.org/10.1177/17470218221080471

30.

Klein

Doyen

Leys

Magalhães de Saldanha da Gama

P. A.

Miller

Questienne

Cleeremans

(2012). Low hopes, high expectations: Expectancy effects and the replicability of behavioral experiments. Perspectives on Psychological Science, 7(6), 572–584. https://doi.org/10.1177/1745691612463704

31.

Kurdi

Lozano

Banaji

M. R.

(2017). Introducing the open affective standardized image set (OASIS). Behavior Research Methods, 49(2), 457–470. https://doi.org/10.3758/s13428-016-0715-3

32.

Leary

M. R.

Kelly

K. M.

Cottrell

C. A.

Schreindorfer

L. S.

(2013). Construct validity of the Need to Belong Scale: Mapping the nomological network. Journal of Personality Assessment, 95(6), 610–624. https://doi.org/10.1080/00223891.2013.819511

33.

Moran

Nudler

Anan

Y. B.

(2023). Evaluative conditioning: Past, present, and future. Annual Review of Psychology, 74, 245–269. https://doi.org/10.1146/annurev-psych-032420-031815

34.

Morey

R. D.

Rouder

J. N.

(2018). BayesFactor: Computation of Bayes factors for common designs. https://cran.r-project.org/package=BayesFactor

35.

Nichols

A. L.

Maner

J. K.

(2008). The good-subject effect: Investigating participant demand characteristics. The Journal of General Psychology, 135(2), 151–166. https://doi.org/10.3200/GENP.135.2.151-166

36.

Olson

M. A.

Fazio

R. H.

(2006). Reducing automatically activated racial prejudice through implicit evaluative conditioning. Personality and Social Psychology Bulletin, 32(4), 421–433. https://doi.org/10.1177/0146167205284004

37.

Orne

M. T.

(1962). On the social psychology of the psychological experiment: With particular reference to demand characteristics and their implications. American Psychologist, 17, 776–783. https://doi.org/10.1037/h0043424

38.

Osborne

Wootton

L. W.

Sibley

C. G.

(2013). Are liberals agreeable or not? Social Psychology, 44(5), 354–360. https://doi.org/10.1027/1864-9335/a000132

39.

Page

M. M.

(1973). On detecting demand awareness by postexperimental questionnaire. Journal of Social Psychology, 91(2), 305–323.

40.

Revelle

(2022). Psych: Procedures for psychological, psychometric, and personality research. Northwestern University. https://CRAN.R-project.org/package=psych

41.

Rosseel

(2012). Lavaan: An R package for structural equation modeling and more (Version 0.5–12 (BETA)). Journal of Statistical Software, 48(2), 1–36.

42.

Soto

C. J.

John

O. P.

(2017). The next Big Five Inventory (BFI-2): Developing and assessing a hierarchical model with 15 facets to enhance bandwidth, fidelity, and predictive power. Journal of Personality and Social Psychology, 113(1), 117–143. https://doi.org/10.1037/pspp0000096

43.

Staats

A. W.

(1969). Experimental demand characteristics and the classical conditioning of attitudes. Journal of Personality and Social Psychology, 11, 187–192. https://doi.org/10.1037/h0027026

44.

Tehrani

H. D.

Yamini

(2020). Personality traits and conflict resolution styles: A meta-analysis. Personality and Individual Differences, 157, Article 109794. https://doi.org/10.1016/j.paid.2019.109794

45.

Tingley

Yamamoto

Hirose

Keele

Imai

(2014). Mediation: R package for causal mediation analysis. Journal of Statistical Software, 59(5), 1–38. https://doi.org/10.18637/jss.v059.i05

46.

Vogel

Hütter

Gebauer

J. E.

(2019). Is evaluative conditioning moderated by big five personality traits? Social Psychological and Personality Science, 10(1), 94–102. https://doi.org/10.1177/1948550617740193

47.

Wilmot

M. P.

Ones

D. S.

(2022). Agreeableness and its consequences: A quantitative review of meta-analytic findings. Personality and Social Psychology Review, 26(3), 242–280. https://doi.org/10.1177/10888683211073007

Supplementary Material

Please find the following supplemental material available below.

For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.

For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.

0.00 MB

0.02 MB

0.01 MB