Use of Face Information Varies Systematically From Developmental Prosopagnosics to Super-Recognizers

Software

Tasks included to measure use of information were completed on a web platform created specifically for this purpose using PHP, HTML, and Javascript. Secure data management was carried out using MySQL databases. The stimuli were generated on a trial-by-trial basis using MATLAB (The MathWorks, Natick, MA).

Stimuli

Images of celebrity faces were used in this experiment because the ability to recognize them has several things in common with the ability to recognize a face from one’s social environment. Famous faces, such as the faces of personal acquaintances, are learned through repeated exposure during participants’ lifetimes in various visual conditions (e.g., static/dynamic, various visual angles and lighting conditions). Therefore, unlike the recognition of a face newly learned in the context of a study, the recognition of a famous face likely taps into mechanisms that are similar to those used when identifying an old friend.

We selected 100 celebrity faces (50 female faces) from the celebrity-faces database assembled by Butler, Blais, Gosselin, Bub, and Fiset (2010). These faces were selected because they were the best-recognized faces in a pilot study conducted with American and Canadian university students. All faces displayed either a happy or a neutral expression and were viewed from a frontal perspective. Faces were translated, rotated, and scaled to minimize the mean square of the difference among the positions of the eyes, the eyebrows, the nose, and the mouth of each face and the average positions of these features across faces. Importantly, these so-called Procrustes transformations preserve relative interattribute distances. A single elliptical mask was applied to all faces to hide external features of the face, such as hair or ears.

Identification of known celebrities

For the purpose of our bubbles experiment (Gosselin & Schyns, 2001), it was important that participants knew the faces presented to them. To determine the subsets of faces personally known by participants, we followed a two-step procedure. First, all 100 faces were presented sequentially, in random order, for 1 s. After each presentation, the celebrity’s name appeared along with four randomly selected names drawn from all the same-gender celebrities in the face database. The participant’s task was to select the name of the celebrity presented. Participants either (a) selected the name with the computer mouse or (b) used the up or down arrow keys to navigate through the names and select their response. Second, the same procedure was then repeated twice for the faces correctly identified in the first run. For each participant, the faces correctly identified on all three occasions—an event unlikely to have happened by chance—were chosen as the subset of faces used in the bubbles experiment. On average, participants correctly identified 54.3 (SD = 26.3) faces.

Bubbles

To reveal the visual features used by each participant to recognize celebrities, we used bubbles (Gosselin & Schyns, 2001). In a bubbles experiment, stimulus information is randomly sampled, and the multiple regression is done between the samples’ locations and the corresponding accuracy scores. The resulting classification image reveals which parts of the stimuli, on the dimensions sampled, are correlated with performance. Only the errors due to ineffective samples of information were of interest to us; it was, therefore, important to reduce the number of errors due to unknown celebrities (see Identification of Known Celebrities).

Each participant completed 1,000 trials. Such a large number of trials was necessary to enable reliable statistical inference about the correlations between face parts and accuracy at the individual level. Each trial began with a “bubblized” face stimulus, which remained on the screen for approximately 1 s. The stimulus was then immediately replaced by five response choices, consisting of the celebrity’s name along with four randomly selected names drawn from the same-gender celebrities in the personally known subset of the face database. Each participant had to select the celebrity’s name using either the mouse or the up and down arrow keys. No feedback was provided. The next trial started approximately 1 s after the participant’s response.

The sampling method is illustrated in Figure S1 in the Supplemental Material available online. First, the image was filtered using a Laplacian pyramid decomposition (Burt & Adelson, 1983) into six spatial-frequency bands (128–64, 64–32, 32–16, 16–8, 8–4, and 4–2 cycles per image, or 85–43, 43–21, 21–11, 11–5, 5–3, and 3–1 cycles per face width). Note that we do not report spatial-frequencies in cycles per degree because we did not control for the size of stimuli in degrees of visual angle. The sixth and coarsest spatial-frequency band was always entirely revealed and served as a background. A mask was applied to the remaining five spatial-frequency bands using pointwise multiplication. This mask contained Gaussian apertures (bubbles) at random locations, which partially revealed visual information. To equalize the number of cycles revealed in one bubble regardless of spatial-frequency band, we varied the size of the Gaussian windows depending on the spatial-frequency band. Therefore, higher spatial-frequency bands contained smaller bubbles, and lower spatial-frequency bands contained larger bubbles. More bubbles were applied to the high-spatial-frequency bands than to the low-spatial-frequency bands to equalize the total surface of information revealed. Ultimately, the five filtered images were summed, along with the coarsest band. The same procedure was applied on each trial to create a randomly sampled stimulus. Accuracy was maintained at 75% correct on average by adjusting the total number of bubbles on the stimulus on a trial-by-trial basis using QUEST (Watson & Pelli, 1983). On average, participants needed 150.9 (SD = 54.6) bubbles to reach a 75% accuracy rate.

Average use of information

To pinpoint the visual information used by participants to identify faces of familiar celebrities, we computed a classification image for each participant and for each spatial-frequency band. A classification image is a weighted sum of the bubble masks that were presented to a participant during the experiment, with the accuracy of the participant transformed into z-score values as weights. This procedure amounts essentially to a multiple linear regression on the bubble masks and on accuracy. The result of this analysis indicates which facial areas in each spatial-frequency band are correlated with accurate face identification. In other words, it shows what visual information was systematically present on trials that led to correct face identification. The classification images were transformed into z scores using the uninformative area surrounding the face stimulus as a reference noise distribution.

A group-classification image was then computed for each band of spatial frequencies by summing the individual classification images and then dividing the sum by the square root of 112 (i.e., the number of participants). Finally, the pixel test (Chauvin, Worsley, Schyns, Arguin, & Gosselin, 2005) was applied to the group-classification images to determine the critical z-score threshold for statistical significance (p < .05, Bonferroni corrected for five tests, S_r = 256 × 256 pixels; σ = 8, 16, 24, 32, and 40 pixels, respectively, from finest to coarsest scale; threshold z = 4.38, 4.05, 3.84, 3.69, and 3.58, respectively, from finest to coarsest scale). The statistical threshold provided by this test corrects for multiple comparisons while taking the spatial correlation inherent to smooth classification images into account. The group-classification images are available in Figure S4E in the Supplemental Material; participants, on average, made use of the inner face features, mainly the eyes, eyebrows, and mouth.

The statistically significant regions are quite similar to the ones found in previous bubbles studies on the use of information for face identification: Across the five spatial-frequency bands, the critical pixels in our group-classification images (see Fig. S4E) overlapped with 53.3% of those in the Schyns, Bonnar, and Gosselin (2002) classification images (see Fig. S4B in the Supplemental Material), 59.0% of those in the Caldara et al. (2005) classification images (see Fig. S4C in the Supplemental Material), and 50.1% of those in the Butler et al. (2010) classification images (see Fig. S4D in the Supplemental Material). Note that, prior to this analysis, the group-classification images from the different studies were aligned using Procrustes transformations to ensure that the locations of facial features were comparable. Furthermore, the number of critical pixels was equalized across these aligned group-classification images in each band of spatial frequencies.

Use of information as a function of face-identification ability

The main goal of the present study was to investigate how interindividual variation in the use of facial information is linked with variation in face-recognition ability. To do so, we ran a second-order multiple regression to predict our general index of face-recognition ability from use of information. The classification images computed for each participant in the analysis detailed in the previous section correspond to the participants’ use of information in faces; z scores in each region of the face represent how much each participant used this region.

To diminish the collinearity between the independent variables (i.e., the value of a pixel can be predicted from the value of neighboring pixels), we did the following. First, we used unsmoothed individual classification images (smoothing introduces spatial correlation). Second, we reduced the size of these unsmoothed individual classification images to 6 × 9, 5 × 9, 4 × 9, 4 × 6, and 3 × 5 elements, respectively, for the finest to coarsest spatial-frequency bands; unsmoothed individual classification images collapsed across spatial-frequency bands were reduced to 5 × 7 elements. Third, and finally, we used a regularized multiple linear regression (ridge function in MATLAB).

Regression coefficients for each of these elements are depicted in Figure 1a. The model predicted the general-ability index better in higher than in lower spatial-frequency bands (R² = .613, k = 43.10; R² = .530, k = 44.00; R² = .351, k = 68.90; R² = .281, k = 49.25; and R² = .141, k = 33.45, respectively, from high- to low-spatial-frequency bands). Statistically significant regression coefficients are marked with asterisks (p < .05, determined with a 5,000-iteration permutation test). The eyes or eyebrows were related to abilities at all spatial-frequency bands except the lowest, with a focus on the right eye or eyebrow from the observer’s point of view between 5 and 21 cycles per face width. Use of the mouth was also related to abilities in high to middle spatial-frequencies. When all spatial-frequency bands were collapsed (R² = .406, k = 45.25), use of both eyes or eyebrows as well as the mouth was a statistically significant predictor of face-recognition ability. We also ran the same second-order regularized multiple linear regressions on the use of facial information and CFMT scores only, CFPT scores only, and other face-recognition ability scores (see Fig. S4). In all cases, results were remarkably similar.

OPEN IN VIEWER

Fig. 1.

Results. Second-order regularized multiple linear regression coefficients (a) are shown for all participants (N = 112), separately for each of the five spatial-frequency bands as well as for all bands combined. The color of each element represents its regression coefficients or the level at which the particular area predicted the global index of performance. Statistically significant areas are marked with asterisks (p < .05, determined with a 5,000-iteration permutation test). Classification images of the developmental-prosopagnosics group (n = 5) and the super-recognizers group (n = 8) are shown in (b). Areas in which the use of spatial information was above the statistical threshold (p < .05; threshold z = 2.9; σ = 10; S_r = 15,204) are marked in white. The scatterplot (c; with best-fitting regression line) shows the relation between the actual and predicted global index of face-recognition abilities for all participants, including developmental prosopagnosics and super-recognizers. Predictions were made using the model built solely with data from participants with intermediate face-recognition abilities. Dashed lines delimit the 95% confidence interval. The inverted triangle represents the developmental prosopagnosic recruited from the general population. SF = spatial frequency; cpf = cycles per face width.

Use of information in developmental prosopagnosics and super-recognizers

There are two main hypotheses about how face processing differs in developmental prosopagnosics, super-recognizers, and individuals with intermediate face-recognition skills (for a recent review, see Barton & Corrow, 2016). According to the first hypothesis, these groups would process faces in qualitatively different ways. In other words, from developmental prosopagnosics to super-recognizers, observers’ use of facial information would not vary predictably (see, e.g., Bobak, Parris, Gregory, Bennetts, & Bate, 2017, in which atypical eye movements were observed only in extremely impaired developmental prosopagnosics). The alternate hypothesis states that developmental prosopagnosics, super-recognizers, and individuals with intermediate face-recognition skills process faces in a quantitatively different way (e.g., Russell et al., 2009). This translates to predictable variations in the use of facial information from developmental prosopagnosics to super-recognizers. Because we have evaluated the use of information during face identification across the entire spectrum of face-recognition abilities for the first time, we have the opportunity to test these two propositions directly.

To increase signal-to-noise ratio of the individual classification images, we reduced their dimensionality by pooling the data across spatial-frequency bands. Specifically, for each participant, we performed a multiple linear regression on the participant’s accuracy and the locations of the bubbles, irrespective of the spatial-frequency band, and smoothed the resulting image with a single Gaussian filter with a standard deviation of 10 pixels. Individual classification images of the five developmental prosopagnosics and the eight super-recognizers are available in Figure S5 in the Supplemental Material. Grouped developmental prosopagnosics’ and super-recognizers’ classification images (see Fig. 1b) were also computed by summing, respectively, the individual classification images of developmental prosopagnosics and the individual classification images of super-recognizers. Statistical significance was determined using the pixel test (Chauvin et al., 2005). Areas in which the spatial information used was above the statistical threshold (p < .05; threshold z = 2.9; σ = 10; S_r = 15,204) are marked in white in Figure 1b.

Seven of the eight individual super-recognizers used the left eye or eyebrow from the observer’s point of view, five out of eight used the right eye or eyebrow, and five out of eight used the mouth (see Fig. S5B). In the classification image of the super-recognizer group (see Fig. 1b), the highest z scores were observed in the eyes, eyebrows, and mouth. In contrast, none of the five individual developmental prosopagnosics used the left eye or eyebrow (see Fig. S5A). Interestingly, a single developmental prosopagnosic used the right eye and this individual happened to be the most competent at face recognition of our five developmental prosopagnosics. The classification image of the developmental-prosopagnosic group (see Fig. 1b) showed a single area attaining statistical significance—the leftmost part of the mouth. The use of this area was only weakly positively correlated with face-recognition ability, shown in Figure 1a.

If the use of information by developmental prosopagnosics and super-recognizers is predictable from that of individuals with intermediate abilities, we would expect (a) that super-recognizers’ use of facial information is similar to that of nonextreme but skilled face identifiers, whereas developmental prosopagnosics’ use of the information is similar to that of nonextreme but unskilled face identifiers, and more generally, (b) that a model predicting the global index of face-recognition ability from the use of information in nonextreme participants would generalize to developmental prosopagnosics and super-recognizers. Thus, we ran a regularized multiple linear regression similar to the one presented in Figure 1a but excluding developmental prosopagnosics and super-recognizers (R² = .419, k = 44.35). Regression coefficients for each of 35 elements are depicted in Figure S6 in the Supplemental Material. In this model, use of the mouth as well as the right eye or eyebrow was correlated significantly with face-recognition ability (p < .05, determined using a 5,000-iteration permutation test), and use of the left eye or eyebrow was linked marginally with face-recognition ability (p = .052).

Informally comparing these results with information used by super-recognizers and developmental prosopagnosics suggests, indeed, that super-recognizers’ use of facial information is similar to that of nonextreme but skilled face identifiers and that developmental prosopagnosics’ use of the information is similar to that of nonextreme but unskilled face identifiers. In Figure 1c, we plot the global index of face-recognition ability measured experimentally in all participants against the global index of face-recognition ability predicted from the model derived only on the nonextreme participants. First, we noticed that none of the 13 extreme participants fell outside the 95% confidence interval. Second, and more crucially, the model based on participants of intermediate abilities explains 65% of the variance in developmental prosopagnosics’ and super-recognizers’ face-recognition ability (n = 13). Together, our results favor the quantitative rather than the qualitative hypothesis about how face processing differs in developmental prosopagnosics, super-recognizers, and individuals with intermediate face-recognition skills.