Face Pareidolia Recruits Mechanisms for Detecting Human Social Attention

Abstract

Face pareidolia is the phenomenon of seeing facelike structures in everyday objects. Here, we tested the hypothesis that face pareidolia, rather than being limited to a cognitive or mnemonic association, reflects the activation of visual mechanisms that typically process human faces. We focused on sensory cues to social attention, which engage cell populations in temporal cortex that are susceptible to habituation effects. Repeated exposure to “pareidolia faces” that appear to have a specific direction of attention causes a systematic bias in the perception of where human faces are looking, indicating that overlapping sensory mechanisms are recruited when we view human faces and when we experience face pareidolia. These cross-adaptation effects are significantly reduced when pareidolia is abolished by removing facelike features from the objects. These results indicate that face pareidolia is essentially a perceptual phenomenon, occurring when sensory input is processed by visual mechanisms that have evolved to extract specific social content from human faces.

Keywords

pareidolia anthropomorphism face perception gaze perception sensory adaptation

The phenomenon of recognizing faces in everyday objects, called face pareidolia, is experienced widely. For example, collections of objects that look like faces are widespread on social media sites such as Twitter, Reddit, and Flickr, accumulating thousands of user-contributed photos (e.g., https://www.flickr.com/groups/facesinplaces/pool/). Interestingly, many pareidolia objects not only resemble faces but also evoke specific mental or communicative content; for example, the facade of a building might appear to be staring back at you, and a bell pepper might have a happy look (Fig. 1). The human brain has evolved to rapidly detect the presence of other people in our environment and to make inferences about qualities such as their personal identity, emotional state, and direction of attention on the basis of specific sensory cues (Adams, Ambady, Nakayama, & Shimojo, 2011). Pareidolia can be understood as a striking false positive in these systems, in which visual mechanisms that are specialized to detect and extract sensory cues from human faces are spontaneously recruited in the absence of a real human form. Alternatively, the facelike nature of pareidolia objects might simply be a cognitive or mnemonic association with only limited engagement of the visual pathways that process human faces. For example, common features shared between pareidolia objects and human faces might lead us to think of them as facelike, much like how we categorize objects semantically as animals or plants. Resolving this question is scientifically interesting because it speaks to how deeply and automatically our sensory experience of the physical world is shaped by the social specialization of our visual system.

Fig. 1.

Examples of objects that look like faces (i.e., pareidolia objects). None of these objects have a mind, but they can still evoke a sense of specific mental content, including emotional states and a direction of attention. (Most of these images are from the Faces in Places group on flickr.com; https://www.flickr.com/groups/facesinplaces/pool/).

How deeply are pareidolia objects processed as faces in the visual system? A recent discovery is that rhesus monkeys seem to experience face pareidolia, as reflected in the similarity of their looking behavior when viewing pareidolia objects and when viewing the faces of their conspecifics (Taubert et al., 2018; Taubert, Wardle, Flessert, Leopold, & Ungerleider, 2017). This indicates that the phenomenon is not simply a product of human culture (e.g., our mass exposure to anthropomorphic characters in cartoons) but is driven by features of the nervous system that are shared across primate species. When humans experience face pareidolia, there appears to be engagement within at least some visual pathways linked to face processing, more so than when we view regular objects. This is reflected in hemodynamic responses in the fusiform face area (FFA; Meng, Cherian, Singal, & Sinha, 2012; Wardle, Seymour, & Taubert, 2017) and in partial consistencies in the time course of electrical and magnetic evoked responses recorded from the scalp when participants view pareidolia objects and human faces (Hadjikhani, Kveraga, Naik, & Ahlfors, 2009; Proverbio & Galli, 2016). Although increased activation in the FFA is closely linked to face perception in human neuroimaging (Tsao & Livingstone, 2008), it does not by itself answer the question of how deeply pareidolia objects are processed as faces. For instance, FFA activation when pareidolia objects are viewed may reflect an initial detection stage in face processing (e.g., visual feature detectors tuned to basic facelike structure) without activation of other face-specialized processing, such as the extraction of social cues.

Here, we investigated whether face pareidolia reflects the activation of sensory mechanisms that extract social information from human faces. We focused on the specialization of the human visual system for encoding the direction of other people’s attention (on the basis of gaze, head, and body direction). There is evidence that pareidolia objects with eyelike features can trigger reflexive shifts in the spatial direction of an observer’s attention, similar to that which commonly occurs when viewing human faces with averted gaze (Takahashi & Watanabe, 2013). Here, we tested for overlap in the experience of face pareidolia and the perception of human gaze by taking advantage of the susceptibility of gaze mechanisms to sensory-adaptation effects (Clifford & Palmer, 2018). Adaptation is a pervasive sensory phenomenon in which the firing of sensory neurons becomes habituated in response to repeated exposure to their preferred stimulus, often with measurable consequences for subsequent perception (Webster, 2015). In the context of gaze perception, repeated exposure to faces with a specific direction of attention, such as a leftward gaze, causes a repulsive perceptual aftereffect, leading subsequently viewed faces to appear to be looking more rightward than they really are (Jenkins, Beaver, & Calder, 2006; Seyama & Nagayama, 2006). These perceptual effects are accompanied by changes in cortical responses recorded with functional MRI (Calder et al., 2007) and electroencephalography (Kloth & Schweinberger, 2010; Schweinberger, Kloth, & Jenkins, 2007) and are likely to reflect the selective habituation of face-selective cell populations in temporal cortex that have differential tuning to gaze direction (Carlin & Calder, 2013). Adaptation to gaze direction can be modeled computationally in terms of the adjustment of gain on gaze-selective sensory channels in response to the recent stimulus history, with the pooling of responses across channels determining the perceived gaze direction (Palmer & Clifford, 2017a; Palmer, Lawson, Shankar, Clifford, & Rees, 2018). Pertinently, there is evidence that gaze aftereffects reflect changes in higher-level coding of social attention in the visual system, abstracted from the specific face features that signal gaze direction in a given image (Palmer & Clifford, 2017b, 2018; Teufel et al., 2009).

Here, we tested for cross-adaptation between pareidolia objects and human faces by repeatedly exposing participants to pareidolia objects with a specific direction of attention and testing whether this alters their subsequent perception of where human faces are looking. The logic is as follows: If face pareidolia reflects the recruitment of visual mechanisms that are specialized to extract mental or communicative content from human faces, then adaptation to pareidolia images with a specific type of social content (e.g., leftward gaze direction) may have a systematic effect on the perception of human faces by altering the function of shared underlying neural systems. Experiment 1 was designed to test whether adaptation to pareidolia objects that are perceived as having a specific direction of social attention modulates the subsequent perception of where human faces are looking. Experiment 2 was designed to test whether this cross-adaptation effect depends on the facelike components of these images or can be explained instead by adaptation to more general object-rotation cues in the pareidolia images.

Experiment 1

Method

Participants

Participants were 30 adults (20 women, 10 men; age: M = 20 years, SD = 2.0 years) recruited from a research sign-up system open to university students. They received course credit for their participation. All participants provided written informed consent. The project was approved by the University of New South Wales (UNSW) Human Research Ethics Advisory Panel.

On the basis of the data from a previous study that used a comparable method for quantifying the effects of adaptation to human faces (Palmer & Clifford, 2018), we conducted a power analysis prior to data collection. The power analysis suggested that gaze aftereffects of the same magnitude as those observed in this previous study (~2° centroid shift, Cohen’s d = 1.5) would require a sample size of 7 to provide 95% power to detect an effect, and aftereffects of half this magnitude would require a sample size of 21 to provide 95% power to detect an effect. We chose to test 30 participants to err on the side of greater power, given uncertainty regarding the expected strength of cross-adaptation between pareidolia images and human faces, as well as other differences in the design of these studies.

Stimuli

Face stimuli

Images of human faces were generated using a combination of 3D scanning and scene-based graphical rendering, allowing precise control over the gaze direction of the faces while retaining a high degree of photorealism. We used 3D models of human faces that were created using an extensive camera array and photogrammetry. These models were produced by a professional 3D-scanning company, Ten24 (https://ten24.info/3d-scanning/). This technique results in realistic facial morphology and extremely detailed skin texture. We used scans of six different human models: four men and two women.

The faces were rendered as 2D images in Blender 2.79b (The Blender Foundation, Amsterdam, The Netherlands) using the Cycles rendering engine. This is a physically based engine that generates each pixel of the image by tracing light paths within a modeled 3D scene. This involves simulating interactions between light and the geometry of surfaces and using shaders to set surface characteristics such as reflectance and transmission. The shader properties of the faces incorporated a combination of diffuse reflectance, specular reflectance, and subsurface scattering. This used Blender’s “principled bidirectional scattering distribution function” shader, driven in part by texture maps created with the 3D photography of human models described above. The eyes, modeled separately from the rest of the face, were depicted as roughly spherical objects encased by a transparent surface that mimicked the cornea and the wet surface of the eyeball. The diffuse shading of the eyeball was driven by a high-resolution texture that depicted a detailed iris and sclera. Additional features such as eyelashes, eyebrows, hair, teeth, and clothing were also modeled separately for some identities.

The lighting within the scene was from a high-dynamic-range illumination map captured from a real-world outdoor setting by merging multiple camera images. This illumination map was created by the company Panocapture (www.panocapture.com). This style of lighting provided realistic illumination of the faces. An additional light source was added within Blender to mimic the soft-box lighting commonly used in portrait photography.

The images were rendered from the perspective of a camera situated 50 cm in front of the face, matching the viewing distance of the participant during the experiment. Frontal views of the face were used in all images. The gaze direction of the face was varied across images by rotating each eyeball such that its pupillary axis intersected the desired point of fixation in 3D space. The face images were presented on screen in approximately life-size proportions, as defined by an interocular distance of 6.3 cm (Fesharaki, Rezaei, Farrahi, Banihashem, & Jahanbkhshi, 2012). We were careful to control the geometry in Blender and the presentation size and location of the 2D renders on screen during the experiment. We also carefully controlled the position of the participant’s face so that the mapping between the fixation point of each face stimulus and the real-world location of the participant’s face could be estimated (described further below). This meant that when the face looked directly ahead, it would be fixating a point on the participant’s face between his or her eyes. Image production in Blender was controlled using custom scripts in Python (Version 3.5.3; Python Core Team, 2017). For each image generated, we also used a horizontally flipped version in the stimulus set, with the gaze direction recoded accordingly. This was done to control for any effect of left–right asymmetries in the image (e.g., in terms of face morphology, texturing, or illumination).

Pareidolia stimuli

A set of pareidolia-face images was collected from the public domain using a Google Images search. Perceptual aftereffects have been observed following adaptation to human faces with either direct gaze or averted gaze (e.g., Calder, Jenkins, Cassel, & Clifford, 2008), but they tend to be stronger in magnitude for faces with more averted gaze. This can be accounted for by the hypothesis that there is an increasingly uneven effect of adaptation on the sensitivity of sensory channels tuned to gaze direction along the horizontal dimension (Palmer & Clifford, 2017a). An initial pilot study was therefore conducted to find pareidolia-face images that had a perceived gaze direction significantly averted to one side. We chose six images on this basis, depicting a bin, a kayak, an interior car light, a sink, and two parking ticket dispensers (two are shown in Fig. 2b). We did not edit the collected images for Experiment 1, apart from horizontally flipping each image so that both leftward-oriented and rightward-oriented versions of the image could be presented. In each of the experimental tasks, the pareidolia images were presented on screen such that the “eyes” of the object were approximately matched to those of the human faces in terms of interocular distance (6.3 cm) and horizontal–vertical position.

Fig. 2.

Measuring the perceived gaze direction of pareidolia faces in Experiment 1. We generated images of human faces (a) whose gaze directions varied in the horizontal dimension. Examples of objects that evoke face pareidolia with averted gaze are shown in (b). Different objects were presented for both leftward- and rightward-gaze conditions. Participants rotated a spherical pointer that was displayed on screen (c) to report the gaze direction of both the human faces and pareidolia faces. The pointer responses to the pareidolia faces were compared with those for human faces (d) to estimate the perceived gaze direction of the pareidolia faces. Leftward-oriented and rightward-oriented pareidolia faces were analyzed separately. The shaded regions in the graph indicate the range of mean responses to the different pareidolia exemplars. Error bars indicate ±1 SEM. In (b), the sink photo is copyrighted by Jay Joslin, and the recycling-bin photo is copyrighted by Thomas Regembal.

Procedure

Pointer task

Participants first completed a task designed to measure the perceived gaze direction of the objects depicted in the pareidolia-face images (Fig. 2). Participants were required to report the gaze directions of a set of human faces and a set of pareidolia faces so responses could be compared between the two. In each trial, participants saw either a human face or a pareidolia face for 1 s, then reported the direction that the face was looking by rotating a spherical pointer that was displayed on screen (Fig. 2c). The pointer was rotated in the horizontal dimension by moving the mouse.

The human faces had gaze directions that varied in 5° intervals between 25° left and 25° right (Fig. 2a). The fixation depth of the faces, reflected in the convergence of the two eyes, matched the 50-cm viewing distance of the participant. For each gaze direction, participants completed 12 trials, consisting of both left- and right-flipped versions of six human identities. The pareidolia-face images were the same as those used in the adaptation task (Fig. 2b), described earlier in the Stimuli section. There were 12 trials presented for each of the leftward-oriented and rightward-oriented conditions: the six pareidolia exemplars, each repeated twice. Thus, there were 156 trials in total, which took approximately 10 min to complete.

Adaptation task

The effects of adaptation to the pareidolia faces were tested on the perception of gaze direction for human faces. This design was motivated by past research that tested for separable or overlapping neural representations of object viewpoints in the human visual system by examining cross-category adaptation effects in human observers who made judgments about object rotation (Fang & He, 2005). Specifically, this past research found that perceptual aftereffects occur when participants adapt to and are tested on different viewpoints of stimuli that belong to the same object category (e.g., faces or cars) but are absent or much reduced when participants adapt to stimuli from one category and tested on another. These findings suggest distinct neural coding of object rotation for these categories. In the present study, the adaptation task consisted of three stages: (a) a baseline eye-contact task, (b) an adaptation period, and (c) a postadaptation eye-contact task.

In the baseline eye-contact task (Fig. 3), participants viewed a series of human faces with different gaze directions. Each face was presented for 500 ms. Participants indicated whether each face was making eye contact with them or not by pressing one of two buttons on the keyboard. The focus of gaze of the stimuli varied in 2-cm intervals in the horizontal dimension between 8 cm left and 8 cm right, at a depth of fixation that matched the 50-cm viewing distance of the participant (Fig. 3c). The purpose here was to vary gaze direction between a series of points across the participant’s face to define a range of gaze deviations that were perceived as making eye contact. There were 108 trials in total, consisting of 12 trials for each of nine gaze deviations. The trials for each gaze deviation included both left- and right-flipped versions of different human identities.

Fig. 3.

Eye-contact task used to measure adaptation effects in both experiments. Models of human faces (a) produced using 3D scanning were used to generate images in which the eyes were fixated on different points in the environment. We then produced images of human faces (b) whose gaze direction varied in the horizontal dimension. Participants were positioned such that the faces presented on screen would appear to be looking at different locations across their own faces (c). Participants then indicated whether each face was making eye contact with them (d). In Experiment 1, responses on this task were compared before and after repeated exposure to the pareidolia images to test for cross-adaptation effects between human and pareidolia faces (e).

In the adaptation period, participants viewed a series of adapter images. The period consisted of 30 images, each presented for 4 s, for a total duration of 2 min. Participants were adapted to either leftward-oriented images or rightward-oriented images, and the orientation of the adapters was counterbalanced across participants.

The postadaptation eye-contact task was identical to the baseline task, except that in each trial, one of the adapter images was shown for 4 s before the human-face stimulus was presented. In each trial of both the baseline and postadaptation tasks, participants had 2 s from onset of the human-face stimulus to make a response; otherwise the next adapter appeared, and the trial was repeated at the end of the block. This time limit helped to produce consistent timing across participants’ exposure to the adapter images regardless of differences in their response times. The adaptation period was repeated midway through the postadaptation eye-contact task.

During the adaptation task, participants’ heads were positioned in a chin rest that was carefully situated relative to the monitor. Participants were instructed to keep their heads still throughout the experiment. This was important because a significant change in head position could change the particular stimulus-gaze directions that fell within participants’ eye regions. A camera situated below the monitor took photos of participants’ faces at intervals throughout the experiment so we could estimate the position of their faces relative to the monitor. Photos were taken at four times during each of the baseline and postadaptation blocks. Four participants were excluded because the photos indicated that they did not use the chin rest. We analyzed the photos of the remaining 30 participants to track the position of face landmarks. Specifically, we annotated the positions of the medial canthus and lateral canthus of each eye and converted them to estimates of real-world locations using landmarks of known size that were visible within the images. The estimated horizontal range of movement of the participants’ eye regions across the images was less than 1 cm on average (M = 0.74 cm, SD = 0.56 cm, range = 0.13–2.14 cm). The statistical effects reported in the Results section were also present when all participants were included in the analysis.

The camera images were also used to confirm that the central gaze points of the face stimuli were between the eyes of each participant (as depicted in Figs. 3c and 3d). The distance between the outer edges of each participant’s eyes varied from approximately 8 cm to 11 cm in our sample; thus, the face stimuli were fixating a series of points both within and outside of each participant’s eye region.

During the adaptation period, we asked participants to perform a detection task to encourage them to maintain their attention on the images. In this task, small targets (stars) would occasionally appear near the eyes of the adapter images for 200 ms. Participants were instructed to press a button each time a target appeared. Correct responses were counted if the participant responded within 1 s of each target’s appearance. Performance was high in this task (Mdn = 100% correct).

Results

Perceived gaze direction of pareidolia faces

Participants first completed a psychophysical task designed to measure the perceived gaze direction of a set of objects that evoked face pareidolia (Fig. 2). Participants reported the gaze direction of both human faces and pareidolia faces using an on-screen pointer, and responses for the two categories of images were compared. The mean pointer responses that participants made when reporting the gaze direction of human faces are shown in Figure 2d. We fitted a linear function to these data to capture the relationship between pointer responses and the gaze direction of the stimulus. The line of best fit had a slope of 1.7 and an intercept of −1.8°. The slope having a value greater than 1 reflects an overestimation of perceived gaze direction, which has been observed previously when both the same and other measurement methods have been used (see discussion in Balsdon & Clifford, 2018). To estimate the perceived gaze direction of the pareidolia faces, we calculated the intercept between the participants’ mean pointer response to the pareidolia faces and the line of best fit to the human-face data. This was 15.1° left for the leftward-oriented pareidolia faces and 15.8° right for the rightward-oriented pareidolia faces. In other words, the perceived gaze direction of the pareidolia faces was equivalent to that of human faces with a veridical gaze direction of 15° to 16°. Similarly, when this same analysis was performed for each participant separately, the pareidolia faces were, on average, perceived as having a gaze direction equivalent to that of human faces with eyes averted by 15.9° (SD = 4.1° across participants). The different pareidolia exemplars used in the experiment were perceived as looking in similar directions, as illustrated by the range shown in Figure 2d. Fitting a logistic function to the human-face data (rather than a linear function) produced similar results: The intercept between the participants’ mean pointer response to the pareidolia faces and the function was 13.3° left for the leftward-oriented pareidolia faces and 14.1° right for the rightward-oriented pareidolia faces.

The magnitude of perceptual aftereffects produced by adaptation to gaze direction varies systematically with the gaze direction of the adapting stimuli, with more pronounced effects occurring following adaptation to faces with more averted gaze (e.g., Calder et al., 2008; Palmer & Clifford, 2017a). In previous studies examining the effects of adaptation to gaze direction, researchers have tended to use adapter stimuli with gazes averted by 25° or more (for a review, see Clifford & Palmer, 2018). However, robust perceptual aftereffects have also been observed following adaptation to faces with gazes averted by approximately 10° (Calder et al., 2008; Palmer & Clifford, 2018). Thus, the perceived gaze direction of the pareidolia faces used in the current experiment (i.e., equivalent to 15°–16° for human faces) should be of sufficient magnitude to induce observable adaptation effects if there is overlap in the perceptual mechanisms processing gaze direction for pareidolia faces and human faces.

Participants tended to be just as precise in reporting the gaze direction of the pareidolia faces as when reporting the gaze direction of human faces. We calculated the average standard deviation of each participant’s pointer responses to pareidolia faces and compared this with the average standard deviation of that participant’s responses to human faces. Across the sample, the mean of the average standard deviation of responses was 9.4° both for human faces and pareidolia faces. Correspondingly, a paired-samples t test indicated no significant difference in this measure between the two stimulus types, t(29) = 0.04, p = .97, Cohen’s d = 0.01.

Cross-adaptation between pareidolia faces and human faces

The effects of adaptation to pareidolia faces were tested on the perception of gaze direction for human faces (Fig. 3). To characterize gaze perception in the eye-contact task, we calculated the centroid of responses for each participant. The centroid is a measure of the center of the distribution of eye-contact responses across gaze deviations and is calculated as follows:

centroid = \frac{\sum_{i = 1}^{n} (p_{i} g_{i})}{\sum p_{i}},

where g is a vector of stimulus gaze directions, and p is the proportion of eye-contact responses for each. We compared the centroid in the baseline and postadaptation sessions to quantify changes in the perception of eye contact produced by the adaptation procedure. Gaze-adaptation effects tend to produce a direction-dependent shift in perceived gaze direction away from the gaze direction of the adapter (e.g., leftward-looking human faces are perceived as looking more directly following adaptation to leftward-looking human faces; Clifford & Palmer, 2018), which corresponds to a shift in centroid in the current task toward the direction of gaze of the adapting stimulus.

Figure 4a shows response data averaged across the sample and box-and-whisker plots of the shift in centroid between the baseline and adapted conditions. One can see from these figures that adaptation to the pareidolia faces affected perceived gaze direction of the human faces in a systematic way. A one-sample t test was used to examine the hypothesis that the mean centroid of responses to the human faces was shifted in the direction of gaze of the adapting stimulus following adaptation to the pareidolia faces. This indicated a significant effect, t(29) = 5.37, p < .001 (one-tailed), Cohen’s d = 0.98. The assumption of normality was supported by a nonsignificant Shapiro-Wilk test (p = .50). The centroid shifted toward the side of the adapter for 24 of 30 participants (80%). These results indicate that adaptation to pareidolia objects that are perceived as having a specific direction of social attention modulates the subsequent perception of where human faces are looking.

Fig. 4.

Cross-adaptation between pareidolia-face images and human faces in Experiment 1 (a) and Experiment 2 (b). The top row shows an example of leftward- and rightward-oriented adapters in each experiment. In the middle row, the graphs show the proportion of trials in the baseline and adapted conditions for which participants responded that the stimulus was making eye contact with them, separately for each stimulus orientation and averaged across participants. The dashed lines show the centroids of the response data. Error bars indicate ±1 SEM. In the bottom row, the graphs on the left of each panel show the centroid shift between the baseline and adapted conditions, separately for leftward- and rightward-oriented adapters, and the graphs on the right show the shift toward the adapter, combined across both stimulus orientations. The central dot of each box-and-whisker plot indicates the median, the box ranges between the 25th and 75th percentiles of the data, whiskers indicate the full range of the data excluding outliers, and crosses indicate outliers (data that extend beyond the limits of the box by more than 1.5 times the box length). The photo used to create the stimuli in this figure is copyrighted by Thomas Regembal.

Experiment 2

Method

Participants

Participants were 30 adults (17 women, 13 men; age: M = 24 years, SD = 7.1 years) who did not participate in Experiment 1. Participants were recruited from a research sign-up system open to the general public and were compensated 20 Australian dollars for their participation. All participants provided written informed consent. The project was approved by the UNSW Human Research Ethics Advisory Panel.

The sample size was chosen to match that of Experiment 1. A power analysis indicated that this sample size would provide more than 95% power to detect perceptual aftereffects of the magnitude that occurred in Experiment 1. For a between-groups comparison of the magnitude of perceptual aftereffects across experiments, the power analysis indicated that sample sizes of 30 per group would provide more than 95% power for detecting a difference if aftereffects were absent in Experiment 2 and approximately 60% power if aftereffects were present in Experiment 2 with half the magnitude of Experiment 1.

Stimuli

The same human-face stimuli were used as in Experiment 1. However, for Experiment 2, an alternative version of each pareidolia image was created with the internal face features replaced with an image texture that roughly matched its surroundings. This was done manually in Adobe Photoshop using the Clone Stamp tool (which duplicates part of an image) and the Healing Brush tool (which blends pixels within a region of the image). We removed eyelike features from all images and mouthlike features from four out of six images. Examples of the edited images are shown in Figure 4b. The motivation here was to retain most of the object-rotation cues present in the pareidolia images but without the internal facelike features that engender a sense of these objects being faces. Previous research has indicated that face pareidolia depends primarily on the presence of eyelike and mouthlike object features, and pareidolia is much reduced when these features are removed (Omer, Sapir, Hatuka, & Yovel, 2019).

Adaptation task

Participants completed the same adaptation task as in Experiment 1, but with adapter images that lacked internal face features. The estimated horizontal range of movement of the participants’ eye regions during the task was less than 1 cm on average (M = 0.52 cm, SD = 0.35 cm, range = 0.13–1.37 cm). Performance on the detection task that participants completed when viewing the adapter images was high (Mdn = 92% correct). Although this is slightly lower performance than in Experiment 1, this was not unexpected given the lack of “eye” landmarks in the adapter images to precisely indicate where the targets would appear.

Results

First, we tested whether the median centroid of responses to human faces was shifted in the direction of orientation of the adapting stimulus following adaptation to the object-rotation images. A significant Shapiro-Wilk test suggested that the data were not normally distributed (p = .012), so a nonparametric Wilcoxon signed-rank test was used. This test indicated a significant shift in responses to the human faces, Wilcoxon W = 366, p = .003 (one-tailed), Cohen’s d = 0.57. The centroid shifted toward the side of the adapter for 19 of 30 participants (63%). Figure 4b shows response data averaged across the sample and box-and-whisker plots of the shift in centroid between baseline and adapted conditions.

Second, we tested the hypothesis that cross-adaptation effects would be stronger for the pareidolia-face images (Experiment 1) compared with the object-rotation images (Experiment 2), reflecting greater engagement of mechanisms that encode the direction of other people’s attention. The strength of perceptual aftereffects was compared between participants using a Mann-Whitney U test, which is a nonparametric alternative to the independent-samples t test. The shift in centroid toward the side of the adapter was significantly stronger for the pareidolia-face images (Mdn = 0.85 cm, interquartile range = 1.26 cm) compared with the object-rotation images (Mdn = 0.25 cm, interquartile range = 0.70 cm), U = 289, p = .008 (one-tailed), Cohen’s d = 0.66. Thus, adaptation to pareidolia-face images altered the perception of eye contact beyond that produced by adaptation to general object-rotation cues.

Discussion

In this study, we employed a stimulus-adaptation procedure to investigate the perceptual mechanisms that underlie face pareidolia. We found that repeated exposure to pareidolia objects that appear to have a specific direction of attention (e.g., an apparent leftward gaze) causes a systematic bias in the subsequent perception of gaze direction more generally, reflected in judgments about eye contact with human faces. Adaptation to gaze direction is thought to reflect plasticity in neural mechanisms that encode the perceptual features of a face; these cross-adaptation effects indicate overlap in the sensory mechanisms that underlie our experience of face pareidolia and human social attention. This is evidence that pareidolia objects not only resemble faces to us, in a cognitive or mnemonic sense, but also are processed by higher-level sensory mechanisms in our visual system that have developed to extract and encode specific social content from human faces. In this sense, face pareidolia can be understood as a kind of visual illusion that defies cognitive penetration: We have explicit knowledge that the object does not have a mind, but we cannot help but see it as having social qualities, such as gaze direction, because of mechanisms in our visual system that are spontaneously engaged by objects with facelike visual characteristics.

The perceptual effects of adaptation to gaze direction can be accounted for within a population-coding model of sensory processing. In this model, the perceived gaze direction of a face is coded in terms of the relative activity of a set of sensory channels in the visual system that are tuned to different angles of gaze (Clifford & Palmer, 2018; Palmer & Clifford, 2017a; Palmer et al., 2018). Stimulus exposure in this framework causes stimulus-dependent changes in the gain on sensory channels, altering their responses to subsequent visual input. This can account for the direction-dependent nature of gaze-adaptation effects, in which adaptation to leftward- and rightward-oriented faces, for example, produces opposite shifts in perceived gaze direction (Calder et al., 2008). Adaptation to gaze direction also produces direction-dependent changes in hemodynamic responses in the anterior superior temporal sulcus (STS) and inferior parietal lobule (Calder et al., 2007). This is consistent with electrophysiological evidence for face-selective neurons in the anterior STS that exhibit a degree of selectivity to gaze direction in macaques (De Souza, Eifuku, Tamura, Nishijo, & Ono, 2005; Perrett et al., 1990; Perrett, Hietanen, Oram, & Benson, 1992) and with functional-MRI research in humans that indicates that the pattern of responses across voxels in anterior STS carries graded information about gaze direction (Carlin & Calder, 2013; Carlin, Calder, Kriegeskorte, Nili, & Rowe, 2011). The direction-dependent effects of adaptation to pareidolia objects observed in the present study are consistent with the notion that these stimuli engage cell populations in the temporal cortex that typically encode the gaze direction of human faces, leading to stimulus-dependent habituation of gaze-selective sensory channels.

Does adaptation to pareidolia objects affect the perception of human gaze by modulating the function of visual mechanisms that are not face selective? In Experiment 2, we found that adaptation to rotated objects (which lack the eyelike features that engender face pareidolia) had a small but significant effect on the perceived gaze direction of human faces. Adaptation to rotated objects could in principle affect the perceived gaze direction of subsequently presented faces by altering the perceived rotation of the head (e.g., Otsuka, Mareschal, Calder, & Clifford, 2014). However, the magnitude of cross-adaptation effects was substantially greater for the pareidolia-face images (with the median shift in centroid more than 3 times the magnitude of that in the object-rotation condition), indicating that adaptation to the perceived gaze direction of the pareidolia images occurs over and above that which occurs to more general object-rotation cues or directional cues. In general, there is evidence for a relative lack of cross-adaptation between gaze direction and other directional cues (e.g., arrows or pointing gestures), indicating a degree of specificity in gaze-adaptation effects (Bayliss, Bartlett, Naughtin, & Kritikos, 2011; Jenkins et al., 2006; Seyama & Nagayama, 2006; for a review, see Clifford & Palmer, 2018). Another possibility is that adaptation occurs to visual characteristics of the eyelike features in the pareidolia images at a level that is not dependent on the facelike configuration of the stimulus. Arguing against this, however, is the fact that pareidolia images vary considerably in their low-level features, with the T-shaped spatial configuration of midlevel features being the most striking similarity between images and what leads us to define certain features as eyelike. Moreover, the eyelike features in pareidolia images tend to lack the characteristic luminance profile of the human eye (i.e., the white sclera, intermediate iris, and black pupil), so it is not obvious that low-level adaptation to the luminance distribution within pareidolia images predicts any particular effect on the perception of human gaze.

A key question in face-perception research is what determines whether face-selective sensory mechanisms are brought to bear on a pattern of visual input. One hypothesis is that there is a gating process, in which an obligatory face-detection stage precedes deeper face-specialized processing (Tsao & Livingstone, 2008). For example, visual detectors tuned to rudimentary patterns of spatial information, such as the configuration of the eyes and mouth in the human face, may determine in a bottom-up manner whether a stimulus is subsequently processed by mechanisms specialized for the extraction of specific social cues. The current results fit this type of model: Viewing objects with a rudimentary facial form appears sufficient to elicit social attention processing, suggesting that these objects pass through a face-detection stage and engage subsequent stages of face-specialized sensory processing, despite the participant’s explicit knowledge that the object is inanimate. This fits with evidence suggesting that pareidolia objects with eyelike features can trigger reflexive shifts in the spatial direction of the viewer’s attention, similar to what commonly occurs when observers view human faces in which gazes are averted (Takahashi & Watanabe, 2013). The phenomenon of face pareidolia thus speaks to how high-level sensory mechanisms are brought to bear on signals arriving from the retina, selectively imbuing our visual experience of the physical environment with social significance.

Footnotes

Acknowledgements

The authors thank Emma Leamon for her assistance with data collection.

Transparency

Action Editor: Alice J. O’Toole

Editor: D. Stephen Lindsay

Author Contributions

Both authors contributed to the research design. C. J. Palmer analyzed the data and wrote the manuscript; C. W. G. Clifford suggested edits, and both of the authors approved the final version for submission.

ORCID iD

Colin J. Palmer

References

Adams

R. B.

Ambady

Nakayama

Shimojo

(Eds.). (2011). The science of social vision. Oxford, England: Oxford University Press.

Balsdon

Clifford

C. W. G.

(2018). How wide is the cone of direct gaze? Royal Society Open Science, 5(8), Article 180249. doi:10.1098/rsos.180249

Bayliss

A. P.

Bartlett

Naughtin

C. K.

Kritikos

(2011). A direct link between gaze perception and social attention. Journal of Experimental Psychology: Human Perception and Performance, 37, 634–644.

Calder

A. J.

Beaver

J. D.

Winston

J. S.

Dolan

R. J.

Jenkins

Eger

Henson

R. N.

(2007). Separate coding of different gaze directions in the superior temporal sulcus and inferior parietal lobule. Current Biology, 17, 20–25. doi:10.1016/j.cub.2006.10.052

Calder

A. J.

Jenkins

Cassel

Clifford

C. W. G.

(2008). Visual representation of eye gaze is coded by a nonopponent multichannel system. Journal of Experimental Psychology: General, 137, 244–261. doi:10.1037/0096-3445.137.2.244

Carlin

J. D.

Calder

A. J.

(2013). The neural basis of eye gaze processing. Current Opinion in Neurobiology, 23, 450–455. doi:10.1016/j.conb.2012.11.014

Carlin

J. D.

Calder

A. J.

Kriegeskorte

Nili

Rowe

J. B.

(2011). A head view-invariant representation of gaze direction in anterior superior temporal sulcus. Current Biology, 21, 1817–1821. doi:10.1016/j.cub.2011.09.025

Clifford

C. W. G.

Palmer

C. J.

(2018). Adaptation to the direction of others’ gaze: A review. Frontiers in Psychology, 9, Article 2165. doi:10.3389/fpsyg.2018.02165

De Souza

W. C.

Eifuku

Tamura

Nishijo

Ono

(2005). Differential characteristics of face neuron responses within the anterior superior temporal sulcus of macaques. Journal of Neurophysiology, 94, 1252–1266. doi:10.1152/jn.00949.2004

10.

Fang

(2005). Viewer-centered object representation in the human visual system revealed by viewpoint aftereffects. Neuron, 45, 793–800. doi:10.1016/j.neuron.2005.01.037

11.

Fesharaki

Rezaei

Farrahi

Banihashem

Jahanbkhshi

(2012). Normal interpupillary distance values in an Iranian population. Journal of Ophthalmic & Vision Research, 7, 231–234.

12.

Hadjikhani

Kveraga

Naik

Ahlfors

S. P.

(2009). Early (M170) activation of face-specific cortex by face-like objects. NeuroReport, 20, 403–407. doi:10.1097/WNR.0b013e328325a8e1

13.

Jenkins

Beaver

J. D.

Calder

A. J.

(2006). I thought you were looking at me: Direction-specific aftereffects in gaze perception. Psychological Science, 17, 506–513. doi:10.1111/j.1467-9280.2006.01736.x

14.

Kloth

Schweinberger

S. R.

(2010). Electrophysiological correlates of eye gaze adaptation. Journal of Vision, 10(12), Article 17. doi:10.1167/10.12.17

15.

Meng

Cherian

Singal

Sinha

(2012). Lateralization of face processing in the human brain. Proceedings of the Royal Society B: Biological Sciences, 279, 2052–2061. doi:10.1098/rspb.2011.1784

16.

Omer

Sapir

Hatuka

Yovel

(2019). What is a face? Critical features for face detection. Perception, 48, 437–446. doi:10.1177/0301006619838734

17.

Otsuka

Mareschal

Calder

A. J.

Clifford

C. W. G.

(2014). Dual-route model of the effect of head orientation on perceived gaze direction. Journal of Experimental Psychology: Human Perception and Performance, 40, 1425–1439. doi:10.1037/a0036151

18.

Palmer

C. J.

Clifford

C. W. G.

(2017a). Functional mechanisms encoding others’ direction of gaze in the human nervous system. Journal of Cognitive Neuroscience, 29, 1725–1738. doi:10.1162/jocn_a_01150

19.

Palmer

C. J.

Clifford

C. W. G.

(2017b). The visual system encodes others’ direction of gaze in a first-person frame of reference. Cognition, 168, 256–266. doi:10.1016/j.cognition.2017.07.007

20.

Palmer

C. J.

Clifford

C. W. G.

(2018). Adaptation to other people’s eye gaze reflects habituation of high-level perceptual representations. Cognition, 180, 82–90. doi:10.1016/j.cognition.2018.07.005

21.

Palmer

C. J.

Lawson

R. P.

Shankar

Clifford

C. W. G.

Rees

(2018). Autistic adults show preserved normalisation of sensory responses in gaze processing. Cortex, 103, 13–23. doi:10.1016/j.cortex.2018.02.005

22.

Perrett

D. I.

Harries

M. H.

Mistlin

A. J.

Hietanen

J. K.

Benson

P. J.

Bevan

. . . Brierley

(1990). Social signals analyzed at the single cell level: Someone is looking at me, something touched me, something moved! International Journal of Comparative Psychology, 4, 25–55.

23.

Perrett

D. I.

Hietanen

J. K.

Oram

M. W.

Benson

P. J.

(1992). Organization and functions of cells responsive to faces in the temporal cortex. Philosophical Transactions of the Royal Society B: Biological Sciences, 335, 23–30. doi:10.1098/rstb.1992.0003

24.

Proverbio

A. M.

Galli

(2016). Women are better at seeing faces where there are none: An ERP study of face pareidolia. Social Cognitive and Affective Neuroscience, 11, 1501–1512. doi:10.1093/scan/nsw064

25.

Python Core Team. (2017). Python: A dynamic, open source programming language. Retrieved from https://www.python.org/psf/

26.

Schweinberger

S. R.

Kloth

Jenkins

(2007). Are you looking at me? Neural correlates of gaze adaptation. NeuroReport, 18, 693–696. doi:10.1097/WNR.0b013e3280c1e2d2

27.

Seyama

Nagayama

R. S.

(2006). Eye direction aftereffect. Psychological Research, 70, 59–67. doi:10.1007/s00426-004-0188-3

28.

Takahashi

Watanabe

(2013). Gaze cueing by pareidolia faces. i-Perception, 4, 490–492. doi:10.1068/i0617sas

29.

Taubert

Flessert

Wardle

S. G.

Basile

B. M.

Murphy

A. P.

Murray

E. A.

Ungerleider

L. G.

(2018). Amygdala lesions eliminate viewing preferences for faces in rhesus monkeys. Proceedings of the National Academy of Sciences, USA, 115, 8043–8048. doi:10.1073/pnas.1807245115

30.

Taubert

Wardle

S. G.

Flessert

Leopold

D. A.

Ungerleider

L. G.

(2017). Face pareidolia in the rhesus monkey. Current Biology, 27, 2505–2509.e2. doi:10.1016/j.cub.2017.06.075

31.

Teufel

Alexis

D. M.

Todd

Lawrance-Owen

A. J.

Clayton

N. S.

Davis

(2009). Social cognition modulates the sensory coding of observed gaze direction. Current Biology, 19, 1274–1277. doi:10.1016/j.cub.2009.05.069

32.

Tsao

D. Y.

Livingstone

M. S.

(2008). Mechanisms of face perception. Annual Review of Neuroscience, 31, 411–437. doi:10.1146/annurev.neuro.30.051606.094238

33.

Wardle

S. G.

Seymour

Taubert

(2017). Characterizing the response to face pareidolia in human category-selective visual cortex. bioRxiv. doi:10.1101/233387

34.

Webster

M. A.

(2015). Visual adaptation. Annual Review of Vision Science, 1, 547–567. doi:10.1146/annurev-vision-082114-035509