Abstract
Kim and Hommel (2015) provided an intriguing alternative explanation for conformity effects. Building on the theory of event coding (TEC; Hommel, 2009; Hommel, Müsseler, Aschersleben, & Prinz, 2001), they assumed that one’s own and others’ actions are represented in comparable ways, so that people may fail to distinguish between those two action categories. As a consequence, “people’s actions that have no social meaning should induce conformity effects” (p. 484).
Building on a paradigm previously used to investigate social conformity (Klucharev, Hytönen, Rijpkema, Smidts, & Fernández, 2009; Shestakova et al., 2013), Kim and Hommel tested their hypothesis in three experiments involving facial-beauty ratings. In a first block, participants rated the beauty of 220 faces on a scale from 1 to 8 using a computer keyboard. After each rating, participants were presented with an intervening event. Participants saw either a static slide showing a number between 1 and 8 or a short movie in which another person pressed the respective number key (1–8) on a computer keyboard. These numbers were equal to, 2 to 3 points lower than, or 2 to 3 points higher than the rating given by the participant (equal, lower, and higher conditions, respectively). In a second block, all faces were again rated on the same scale from 1 to 8. The hypothesis was that numbers lower than the initial rating should lead to a negativity shift, whereas numbers higher than the initial rating should lead to a positivity shift. The authors present support for this across all three experiments.
Critically, the paradigm confounded extremity of initial ratings and assignment of experimental condition (equal vs. lower vs. higher). Since the rating scale was restricted from 1 to 8, experimental condition could not be assigned independently of initial ratings. For example, it was not possible to assign an initial rating of 7 to the higher condition, but only to the equal or lower condition. Similarly, low ratings could not be assigned to the lower condition, but only to the equal or higher condition. As a result, the higher condition occurred for low (and moderate) but not for high values, and the lower condition occurred for high (and moderate) but not for low values. Figure 1 shows the distribution of experimental conditions across initial ratings and experiments in Kim and Hommel’s study.
Distribution of experimental conditions (trial type) across initial ratings in the three experiments reported in Kim and Hommel (2015). Note that while trials for initial ratings of 3 and 6 look as though experimental conditions are relatively equally distributed, they are still problematic because only a difference of 2 (not 3) toward the nearest scale end point can be shown for these trials.
This confound is problematic because regression to the mean (Campbell & Kenny, 1999; Galton, 1886) dictates that extreme values on a variable X (e.g., first measurement) will be closer to the mean on any variable Y (e.g., second measurement) when X and Y are less than perfectly correlated. While error is on average randomly distributed across all measurements of X, high values of X will on average have a more positive error, whereas low values of X will have a more negative error. In other words, the measured values of X correlate with the measurement error of X. Because the measurement error of Y is independent of the measurement error of X, any extreme value in X will be less extreme in Y. See Shanks (2017) for an extensive discussion on this topic.
Applied to Kim and Hommel’s data, this means that we would expect initially high values to be lower in the second measurement and initially low values to be higher in the second measurement. Regression to the mean combined with the confound described above can create the original pattern of results without assuming any psychological processes.
Reanalysis of the Original Data
Kim and Hommel provided us with the original data files, allowing us to analyze the extent to which there were effects beyond regression to the mean. We present two types of analyses, for all of which change from the initial rating to the final rating was the criterion. These are multilevel analyses using random intercepts for participants and stimuli (Judd, Westfall, & Kenny, 2012). Note that these analyses are on a trial level.
First, we ran analyses similar to the original (confounded) analyses on each data set, using trial type (lower vs. equal vs. higher, coded as −1, 0, and 1, respectively) 1 and, if appropriate, type of feedback (coded as −1 vs. 1) as predictors (Analysis 1). Second, we reran these analyses, this time using only the initially neutral trials (initial rating of 4 or 5). For these trials, there is no confound, and effects found here cannot be attributed to regression to the mean (Analysis 2).
Results
Results of the analyses are presented in Table 1. Note that for the sake of brevity, we do not present main effects of feedback type. Full results and additional detailed analyses can be found in the Supplemental Material available online. As expected, when not controlling for the confound (Analysis 1), we found reliable effects of trial type (lower vs. equal vs. higher) across all experiments and conditions. Hence, differences in the following analyses are not attributable to the multilevel approach. Because of the interaction of trial type and feedback type in Experiment 2, we present separate analyses for both feedback-type conditions.
Reanalysis of the Original Data From Kim and Hommel’s (2015) Study
Note: TT = main effect of trial type (lower vs. equal vs. higher), TT × FT = interaction of trial type and feedback type (movie vs. number).
Critically, the majority of corrected analyses did not show the original effect. Out of the five analyses we ran, three showed no significant effect of trial type, and one even showed a reversed effect (see b weights in Table 1). While there was still a small effect in Experiment 1, it is clear that the confound of initial rating and trial type led to a strong overestimation of the reliability and strength of the effects in the original analyses. 2 When confounded trials were removed, most of the effects vanished. Researchers interested in related phenomena should therefore be very careful when using a similar paradigm.
Footnotes
Action Editor
D. Stephen Lindsay served as action editor for this article.
Author Contributions
Declaration of Conflicting Interests
The author(s) declared that there were no conflicts of interest with respect to the authorship or the publication of this article.
Funding
The research reported in this article was supported by an Emmy Noether grant from the Deutsche Forschungsgemeinschaft (HU 1978/4-1) awarded to Mandy Hütter.
Notes
References
Supplementary Material
Please find the following supplemental material available below.
For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.
For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.
