Analysing Texture in Portraits

Abstract

This case study is an initial exploration as to whether the depiction of texture in a set of portraits, all portraying the same Sitter, is related to the familiar likeness assessments reported in a companion paper containing a principal component analysis (PCA) of the portraits’ depiction of shape. Somewhat unexpectedly, a texture PCA failed to discriminate the high from low likeness portraits, despite experimentation with different pre-processing methods to reduce the portraits’ high level of uninformative, image-level texture variability. There were some findings arising from these analyses, and while only indicative at this stage, include that linear histogram matching is effective in reducing variability in portrait brightness; that depicting, and perhaps exaggerating, shading relating to lighting direction may enhance portrait likeness; and, that whether minimised or exaggerated, lighting direction can be portrayed somewhat anomalously. Furthermore, and in agreement with findings from photographs, shape and texture were not found to be independent variables, and shape-free image registration, while very usefully enabling a comparison of closely corresponding pixel coordinate values, could itself be a confounding factor for undertaking a texture PCA with portraits produced under relatively ambient conditions.

Keywords

portraiture eigenpictures eigenfaces likeness image pre-processing shape-free shading

This research builds on two previous public science projects involving collaborations with local artists, where we each produced portraits depicting the same Sitters. On both occasions, the portraits were subsequently anonymised and exhibited in a local art gallery in the Illawarra region of New South Wales (Australia) so that members of the general public could assess each for likeness. The focus of these projects was the relationship of shape accuracy, as measured by a geometric morphometric principal component analysis (PCA), to the mean likeness assessments the portraits attained (Hayes et al., 2020; Hayes et al. 2018). We found that relative accuracy and/or exaggeration of facial shape depiction had a significant and positive correlation with the likeness assessments of both familiar and unfamiliar viewers. Furthermore, the statistical average of 10 portraits depicting the same Sitter was found to be more shape accurate than the portraits from which it was derived, and when two of these averages were exhibited as portraits in their own right, both were considered by viewers to be a very good—though not necessarily the best—portrait likeness.

This current study is a companion analysis to Hayes et al. (2020) and arose from the recommendations of two reviewers of the previous papers. Both suggested complementing the shape analyses with a PCA of the portraits’ facial textures, given studies of manipulated photographs have found texture to be more important than shape for recognition (for reviews, see Andrews et al., 2016; Burton et al., 2015), and face recognition in photographs is highly correlated with their familiar likeness assessments (Ritchie et al., 2018). A subsequent review of the literature indicated there are few, if any, precedents examining the relationship of texture depiction in portraits to likeness, and furthermore, only one set of portraits that we analysed for shape depicts the Sitter’s head and face in an unoccluded frontal view (Figure 1), which is required for undertaking a texture PCA with the MATLAB-based software, InterFace (Kramer et al., 2017). This, therefore, is an initial, and necessarily limited, case study primarily addressing how portrait textures can be quantitatively analysed, and if these findings can be related to their previously reported likeness assessments (Hayes et al., 2020). What follows is an overview of how texture (luminance, hue, and saturation; reflectance) has been explored within face perception and the visual arts, including where some of these studies have applied image processing, and what this indicates regarding the likely relationship of texture to portrait likeness.

Figure 1.

The exhibited images. Reference photograph (top left), portraits (A–J), and portrait average (K).

Early face perception research explored the recognisability of monochromatic, low resolution 3D laser scans lacking computer-generated shading properties (Bruce et al., 1991), and computer-generated line drawings depicting the luminance boundaries of the facial features (Bruce et al., 1992; Davies et al., 1978). The impact of shape from shading on identification has also been studied with photographic negatives (Galper, 1970), and the effect on recognition when these texture cues are inverted in photographs—and photographic negatives—depicting familiar faces lit from below (Johnston et al., 1992). All were found to significantly reduce image recognition, with the greatest effect involving manipulations of the facial textures relating to light and shadow. Other research has examined the effect of unrealistic colours in photographs and concluded that this has minimal impact on facial recognition, providing the patterns of light and shadow are retained (Kemp et al., 1996).

The different contributions of exaggerating shape and texture to facial recognition form the basis of more recent perception research. For example, using learned images of faces, Itz et al. (2014) exaggerated the texture values of luminance, hue, and saturation and found this to have a stronger recognition advantage than exaggerating facial shape. A different approach by Lee and Perrett (2000) involved participants separately manipulating the shape, colour (including luminance and saturation), and contrast values of photographs depicting familiar (famous) faces to produce what they considered to be the best likeness. On average, a veridical or slightly anticaricatured face shape, together with exaggerated colour and enhanced contrast values, was considered the best likeness, with contrast tending to be enhanced twice as much as colour was exaggerated. In addition to noting that colour exaggeration and contrast enhancement produces more vivid images, Lee and Perrett speculate that the advantage of caricaturing luminance, hue, and saturation is that these values include the texture cues of facial shadows.

Other studies have explored the effect of morphing colour photographs depicting familiar faces to an average facial shape while retaining the original reflectance properties, and vice versa. Russell and Sinha (2007) have found that reflectance (light, shadow, and colour values) within an average face shape is recognised as well as, if not better than, a face shape containing an average reflectance. Russell and Sinha also comment, however, that even though their stimulus photographs were captured using identical lighting conditions, it was likely that the images retained slight indications of differences in facial shape from shading. A variation of this experiment undertaken by Andrews et al. (2016) produced stronger results in favour of reflectance for face recognition. The stimuli for this study were hybrid faces, with the surface properties of one familiar (famous) face morphed to the shape of a different familiar (famous) face. Andrews et al. suggest that one of the reasons reflectance dominated recognition in this study is because facial shape is less stable, given, for example, the shape changes arising from variations in head pose.

Many of the experiments involving exaggerating, averaging, and hybridising texture and shape in photographs of faces were enabled by the development of photographic caricature software (Benson & Perrett, 1991); a development that has also contributed to enhancing digital image registration prior to undertaking a texture PCA. Texture PCAs are pixel-based coordinate systems (Nestor et al., 2013), where each pixel in each image constitutes a variable in luminance and/or colour values, and, in addition to the component loadings, statistical variance is visually displayed as an eigenpicture (Sirovich & Kirby, 1987). Eigenpictures, which are also referred to as eigenfaces or holons (Bartlett et al., 1999; Turk & Pentland, 1991), are coded so that the lighter pixels represent the positive component loadings, and the darker, the negative (Kirby & Sirovich, 1990). When a texture PCA is undertaken with relatively minimal image registration, such as only using the pupils of the eyes to align the faces, the eigenpictures have an elusive appearance (see, e.g., Figure 2) and are typically interpreted subjectively—and with difficulty (Hancock et al., 1996). To better align facial images, Craw and Cameron (1992) applied caricaturing software to transform each image to the same facial shape. This results in all of the images having similar locations for each of the facial features, and enables a more direct comparison of the corresponding pixel values across the data set. Craw and Cameron referred to these shape registered images as “shape-free faces” (p. 500), and subsequent evaluations (e.g., Hancock et al., 1996) have found that shape-free image registration greatly improves PCA performance. However, and as noted by Burton et al. (2001), shape-free is a misleading description, given shape-free faces typically retain residual texture cues of shape from shading.

Figure 2.

Eigenfaces.

For many perception studies that incorporate shape-free faces, texture is defined as all that remains following shape-free registration, and in addition to luminosity, hue, and saturation, this includes variations in lighting direction, cameras, and camera settings (e.g., Burton et al., 2015; Hancock et al., 1996; Jenkins & Burton, 2011; Kramer et al., 2017). When a texture PCA is also applied to shape-free faces, this is typically for automated facial identification (e.g., Burton et al., 2005; Jenkins & Burton, 2011). Identification is taken to occur when a known and unknown face share similar loadings within or across selected components (Moon & Phillips, 2001; Turk & Pentland, 1991), but one of the main limitations of a texture PCA is a marked sensitivity to variation in image lighting (Zhang & Turk, 2008). As a consequence of this sensitivity, it is common for ambient lighting conditions and/or camera settings to result in overall image brightness (e.g., exposure) dominating the texture variance (Kramer et al., 2017), and photographs of different people having more similar component loadings than those of the same person photographed under different lighting conditions (Jenkins & Burton, 2011). Reducing the impact of these uninformative, image-level pixel values can be achieved by either standardising the image capture protocols (e.g., Nestor et al., 2013; Russell et al., 2007) or by applying one or more methods of digital image pre-processing (e.g., Burton et al., 2005; Kirby & Sirovich, 1990; Moon & Phillips, 2001; O'Toole et al., 1999).

The 10 portraits being analysed in this study (Figure 1) were all photographed using the same protocols (f/20, ISO-100, focal length 29 mm), but due to the diversity of media, materials, and style of depiction, they differ noticeably in colour, brightness, and contrast, and likely to a greater extent than a collection of portrait photographs. Pre-processing reduces image-level variability, but which methods are applied depends on which aspects of texture need to be preserved and/or enhanced (Gonzalez et al., 2008). Converting images to greyscale reduces the impact of variance in colour values, and histogram equalisation has been applied within face perception research to standardise image brightness (e.g., Burton et al., 2005). Histogram equalisation redistributes an image’s greyscale values across the range of 256 intensities and enhances the contrast levels of both under- and overexposed images. Alternatively, image pre-processing can involve linear histogram matching, which involves shifting all of the greyscale values towards either a lighter or darker average intensity while retaining much of the image’s contrast pattern (Sedgewick, 2010). These two pre-processing methods are illustrated in Figure 3, and both, together with the effect of greyscale conversion, are evaluated in this case study for their effectiveness in reducing uninformative, image-level intensity values in the portraits, portrait average, and reference photograph.

Figure 3.

Image pre-processing. The greyscale version of Portrait A (left) has been pre-processed using two different techniques. Histogram equalisation (top) results in the tonal values spanning the range of intensities. Linear histogram matching (bottom) has darkened Portrait A by shifting the values while retaining much of the characteristic histogram shape.

Jenkins and Burton (2011) present a very different approach to pre-processing photographs of faces and, in so doing, provide an indication that texture cues associated with lighting direction likely impact photographic likeness. Rather than discrete images, Jenkins and Burton submitted a database of photographic averages (derived from 10 to 12 ambient photographs) to a texture PCA, because averaging has the effect of cancelling out differences in lighting direction. This approach was found to improve automated facial identification, but Jenkins and Burton note that, in lacking the “environmental noise” (p. 1681) of illumination, average faces appear to be unrealistic representations. Further research by Ritchie et al. (2018) has found that photographic averages—unlike the portrait average analysed in this study—tend to attract low likeness assessments from familiar viewers.

Since the Western European Renaissance (c.1400–1500), the illusion of light striking a surface is one of the fundamental artistic approaches to depicting texture, and this is exemplified by the technique of chiaroscuro, where a specific light source is designated to produce distinct patterns of light (chiaro) and shadow (oscuro) (Casati & Cavanagh, 2019; Gombrich, 1977). A study of 225 historical “master paintings” used a protractor to measure the depictions of lighting direction and found that this was typically 30° to 60° left of vertical (Sun & Perona, 1998, p. 184)—though it is not clear whether this study included portraits. Furthermore, and echoing the photograph colour manipulations of Kemp et al. (1996), while the depiction of light and shadow can involve colour and saturation (see, e.g., Figure 4), highly improbable colours can still retain the illusion of texture, providing the improbable colours have probable brightness values (Cavanagh, 2005; Cavanagh & Leclerc, 1989). Artists are, however, advised to work with a limited range of tones (brightness values). For example, Maughan (2004) recommends chiaroscuro portraits have a maximum of 9 tonal values for the total work, with the darkest reserved for the shadows cast by the facial features, and fewer (≤ 4) for the depiction of the form of a specific facial feature—otherwise, the “composition will fail” (p. 24). Maughan is not specific as to what is meant by this failure, but it is likely related to Arnheim’s (1974) recommendation that, to avoid visual confusion in an artwork, both the depiction of brightness values due to illumination, and those due to surface properties, should each constitute a “simple, unified system” (p. 304), and furthermore, these two systems need to be visually distinct from each other.

Figure 4.

Texture as tonal values. To the left is an extract from one of the colour portraits analysed in this study (Portrait E). To the right is this extract represented as seven grey levels, with each level matched to a region that was produced using colour to create the tonal variations.

Art manuals also recommend that artists depict cast shadows accurately (e.g., Maughan, 2004; Speed, 1917; Zaidenberg, 1944), though anomalies in shadow depiction, and an artist’s selection of which, if any, shadows to depict, may have minimal impact on how an artwork is perceived. Referring to experiments involving manipulated images and a range of historical artworks, Casati and Cavanagh (2019) report that, providing cast shadows appear formless and are darker than their surrounds, human perception is such that shadows do not need to agree with the shapes that cast them, nor with the lighting direction indicated by the shadows cast by other shapes depicted within the composition. Furthermore, an artist may selectively or completely avoid including attached and/or cast shadows. This freedom of shadow depiction, however, may not hold as strongly for highly familiar objects, such as the face. A shadow experiment using a manipulated photograph showing the cast shadow of the nose falling in the opposite direction to those of the rest of the face was found to be very rapidly perceived as anomalous (average 3 milliseconds), while similar stimuli involving the shadows cast by buildings were not (Ostrovsky et al., 2005). Computer measures of cast shadows and highlights have been applied to iconic paintings, including portraits depicting named and unnamed Sitters, to quantitatively measure the depiction of lighting direction, and thereby identify a particular artist’s style and/or to ascertain whether the portrait had likely been painted from life (reviewed in Stork, 2011). This suggests that the illusion of verism in portraiture—which may, or may not, be related to recognition and likeness—is enhanced by a relatively coherent depiction of lighting direction.

To summarise, while portrait texture does not appear to have been studied in relation to likeness (or recognition), research within face perception, the visual arts, and the application of digital image processing to photographs and artworks indicates texture, and in particular the depiction of lighting direction, will likely have a strong relationship to familiar likeness assessments. Of particular interest in this initial case study is whether a texture PCA can quantify the extent to which lighting direction is depicted within the portraits and portrait average, the level of coherence, visual clarity, relative accuracy and/or exaggeration of this depiction, and which of these texture characteristics, if any, are related to the likeness assessments undertaken by viewers highly familiar with the Sitter in life.

Methods

Images

All of the portraits were produced in reference to the same photograph of the Sitter (refer Figure 1). This reference photograph was taken during a preliminary life drawing session with the Sitter’s face lit by two studio lights: a primary light source located ∼ 60° above and to the right of the Sitter’s face (i.e., above left in the picture frame) and a secondary, highly diffuse source above and to the left. As with the shape and likeness analysis of these 12 images (Hayes et al., 2020), the reference photograph is taken to be the most accurate depiction.

Each artist applied different media (pencil, charcoal, compressed charcoal, oils, mixed media) to different materials (white paper, tinted paper, canvas) to achieve their portrait likeness. Six of the portraits are relatively monochromatic, the remainder (four portraits, the portrait average, and the reference photograph) vary in both colour and saturation. The portrait average is a statistical average of the 10 portraits achieved using the thin plate spline software created by James Rohlf (2015), and was exhibited as a portrait alongside the portraits from which it was derived—made possible by the images being only identified by a letter (A–K). Further information regarding the production and exhibition of the works can be found in Hayes et al. (2020).

Likeness Assessments

Public participation in the likeness assessments was both voluntary and anonymous. Following a brief explanation of the project, volunteers were provided with an assessment form, where they also recorded their age (in decades), sex, experience as a visual artist, and prior familiarity with the Sitter. All likeness assessments were recorded on a 7-point Likert scale represented by open circles, with the verbal cues of Very Low and Very High. Those with some prior familiarity assessed the likeness of the reference photograph, which was exhibited separately on a plinth near the entry to the gallery. No explanation was given as to what was meant by likeness. The volunteers were encouraged to view all of the portraits (A–K) prior to undertaking their assessments, though these could be undertaken in any order. Visitors posted their completed assessments into a purpose-built box, which were subsequently dated, coded, and filed. For more detail, see Hayes et al. (2020).

Our previous research compared shape depiction to the likeness assessments provided by all 153 volunteers. This companion texture analysis is only concerned with the assessments provided by those who reported a high level of prior familiarity with the Sitter in life (5–7 on the Likert scale), though whether this familiarity was solely due to the Sitter being the local Lord Mayor, or included a more personal familiarity, was not recorded. Of the 70 highly familiar viewers, most were women (n = 51, mean 57.2 years, range 30–80+ years), and their reported experience as a visual artist (3-point Likert scale) was evenly spread, from none (31%) to very (37%). The men (n = 19, mean 53 years, range 30–80+ years) had few very experienced visual artists (16%) and a higher proportion with no experience (42%).

InterFace Image Registration

To achieve shape-free image registration using the InterFace software, 82 homologous landmark coordinates constituting the vertices of a triangular grid are applied to each image, with the option to save each image with the background (the pixels falling outside of the grid) set to pure black (Figure 5). Using a graphics program, this background was deleted from each landmarked image (point sample, tolerance 0, contiguous pixel selection) and resaved as a TIFF (tagged image file format) file in preparation for pre-processing.

Figure 5.

InterFace image registration: landmarking. Centre is the standardised grid provided by the software. To the left is this grid modified to the Sitter’s reference photograph. To the right is the reference photograph with the area outside the grid replaced by pure black pixel values.

Image Pre-processing

The Original images were first converted to greyscale in a graphics program (image mode function) and duplicated. One set of the Greyscale images was pre-processed using the automated histogram equalisation tool (Figure 6, middle row). A second set was normalised to the same mean intensity (i.e., overall image brightness) using linear histogram matching. This was achieved using the graphics program levels tool to adjust each image until the mean intensity was within 0.5 of the average intensity of all 12 Greyscale images (153.9). The resulting Normalised mean intensity images (mean 153.8, range 153.5–154.0, variance 0.03, standard deviation [SD] 0.18) are illustrated in Figure 6, bottom row. The Greyscale images, Equalised histogram images, and Normalised mean intensity images were then resaved as RGB colour bitmaps, which is required for the InterFace PCA viewing tool to function, and entered into separate InterFace PCA folders, with each folder containing a copy of the Original image shape data (i.e., the 82 homologous landmark coordinate grids).

Figure 6.

Image pre-processing. Following deletion of the background, the Original images were converted to greyscale (top row). A set of the Greyscale images were pre-processed using histogram equalisation (centre row), and a further set were pre-processed to the same mean intensity using linear histogram matching (bottom row). The images that include colour in their Original (unprocessed) form are indicated by an asterisk. K is the portrait average.

Image Intensity Values

The intensity values for the Original, Greyscale, Equalised histogram, Normalised mean intensity images, and the shape-free versions of these images were measured using the biological image analysis software, Fiji (Schindelin et al., 2012). Following selection of the region of interest (i.e., the perimeter of the InterFace grid), Fiji outputs to a spreadsheet-selected measures, which for this analysis included (a) the area as represented by the total pixel count, (b) the mean intensity, and (c) the SD of intensity, which is commonly referred to as image contrast (Smith, 1997). Fiji also provides the numerical values of each image’s intensity histogram, which were converted to scalable vector graphics using the statistical software, PAST4 (v. 4.0, 2000; Hammer et al., 2001). To ensure that the image areas were consistent, the perimeter of the Original version of each image was also applied to the Greyscale, Equalised histogram, and Normalised mean intensity versions. For the shape-free images, the Original image shape-free average arising from the Original image texture PCA was applied to all as the common perimeter.

InterFace PCA

Texture PCAs were undertaken separately for the Original, Greyscale, Equalised histogram, and Normalised mean intensity images, and for each the PCA was undertaken twice—once with the full set of images including the reference photograph (N = 12), and once without the reference photograph (n = 11).

The output of an InterFace PCA includes the shape-free versions of the input images, the average shape-free image, the PC loadings (the z-projections of each image within each component), eigenvalues (the percentage each component is capturing of the total texture variance), and eigenpictures (the visual representation of the pixel variance being captured by each component). Using the InterFace PCA Viewer, images displaying the component texture variance at selected points along individual PC axes were saved as separate image files. Following the recommendation of MacLeod (2013), these were taken at regular (0.5) component intervals to more clearly indicate the component variance. Graphic illustrations of the InterFace results (e.g., PC plots) were also produced using PAST4.

Segmentation and Quantification of Dominant Shading Patterns

A segmentation process involving a series of computer graphic software filters was developed to quantify the approximate extent to which each shape-free portrait and the portrait average agrees with the dominant shading depicted in the shape-free reference photograph.

The Greyscale shape-free images were entered into a graphics program and the average intensity level of each Greyscale shape-free image adjusted to within 0.3 of the average of the 12 Greyscale shape-free images (mean 154.07, range 153.9–154.3, variance 0.02, SD 0.15; see, e.g., Figure 7.1). A posterisation filter was applied (Figure 7.2) resulting in each shape-free image having the same 7 equidistant intensity values (31, 63, 95, 127, 159, 191, 223). Following this, a threshold filter set to 128 (Figure 7.3) was applied. This conflates the darker intensity values (< 128) to black, and the lighter values to white, with the black values representing the dominant shading pattern of each shape-free image. Each image’s threshold pattern was then calculated as a proportion of black to white pixel counts.

Figure 7.

Shading segmentation process. All images are the portrait average (K). The segmentation is based on the Greyscale PCA shape-free images (far left). The shading segmentation process is as follows: (1) Mean intensity following background deletion, (2) Posterisation to seven equidistant intensities, (3) Thresholding (Level 128) to represent the dominant shading pattern as pure black pixels, (4) Difference in threshold shading from the reference photograph, and (5) Segmentation of the threshold shading pattern in relation to the threshold shading of the reference photograph.

A difference filter was applied to identify where each shape-free image differs, at the level of individual pixel values, from the threshold shading of the reference photograph. The shape-free threshold images of the 10 portraits and portrait average were converted to difference layers and inverted. Each image was then positioned, in turn, above the reference photograph threshold shape-free image. The resulting black pixels indicate where each image’s threshold intensity values differ from the corresponding pixel values in the reference photograph (Figure 7.4), enabling the extent of this difference to be calculated as proportional black/white values. The difference filter also indicated that the residue of the outer perimeter present in all of the shape-free images (see Figures 7.3–7.5), which is due to aliasing arising from interpolation to shape-free, is the same for each image (213 of 29,950 pixels) and therefore has the same contribution to the pixel counts.

An additional segmentation method was devised to provide a numerical estimate of how, as well as where, each of the 11 shape-free threshold images differs from/is similar to the threshold shading of the reference photograph. The shape-free reference photograph threshold shading was duplicated as a separate layer, set to a transparency level of 67%, and placed above each image’s difference filter (which had been saved as discrete images). This results in four distinct intensity values (0, 84, 171, and 255) that can each be calculated as a proportion of the total pixel count. As illustrated in Figure 7.5, these four intensity proportions indicate:

Black (0)—where the shape-free image is missing the reference photograph shading;

Dark grey (84)—where the image agrees with the reference photograph shading (replaced by green [#336666] with a similar tonal value [87] for visual clarity);

Light grey (171)—where the image has additional shading; and

White (255)—where the absence of shading agrees with the reference photograph.

Statistical Analyses

PAST4 was used to undertake Spearman’s rank correlations (r_s), Pearson’s linear correlations (r) with Bonferroni correction, descriptive statistics, Wilcoxon pairwise tests (W), and Mann–Whitney–Wilcoxon rank sum tests (U).

Results

Likeness Assessments: Participants, Image Shape, and Depiction of Colour

Neither the sex and age of the highly familiar viewers, nor their level of familiarity with the Sitter prior to the exhibition, had any impact on their likeness assessments of the reference photograph, portraits, and portrait average (r_s and U not significant), and the 70 viewers are therefore treated as one cohort. There was a slight tendency for older viewers to be more experienced visual artists (r_s = .27, p = .03) and for the likeness assessments of Portrait B to be positively correlated with visual art experience (r_s = .28, p = .02). The highly familiar mean likeness assessments are significantly correlated (r = –.70, p = .02) with the relative shape accuracy of the portraits and portrait average, and these values are listed in Table 1. Both likeness and shape accuracy are not, however, related to the presence or absence of colour in the images (U not significant). See Hayes et al. (2020, 2018) regarding how relative shape accuracy was determined using the Procrustes chord Distances.

Table 1.

Familiar Mean Likeness Assessments and Relative Shape Accuracy (PcD).

Image	Likeness	PcD	Image	Likeness	PcD
Photo*	6.03	na	F	3.30	0.038
A	3.99	0.054	G*	3.31	0.062
B	6.04	0.042	H	3.44	0.069
C*	4.49	0.069	I*	4.89	0.060
D	3.80	0.057	J	1.87	0.087
E*	4.84	0.035	K (mean)*	5.87	0.026

Note. The mean likeness assessments are from 70 viewers highly familiar with the Sitter’s facial appearance in life. Relative shape accuracy is the geometric morphometric PcD from the reference photograph. Photo is the reference photograph, Portrait K is the portrait average, and the Original colour images are indicated by an asterisk. The four highest likeness assessments and the four lowest PcD (most shape accurate) are underlined. PcD = Procrustes chord Distance.

Image Brightness (Mean Intensity), Contrast (SD), and Familiar Likeness

Figure 8 illustrates the differences in intensity values by extent and type of image pre-processing. In their Original form, the six relatively monochromatic portraits tend to be brighter and have a greater variability in image contrast, with the difference in contrast levels attaining significance (U = 4, p = .05). All of the images are brighter following conversion to Greyscale (W = 72, p = .007), but the brightness values strongly agree with those of the Original images (r = .93, p < .001), and this agreement is even stronger for contrast (r = .99, p < .001). High likeness images (Portraits B, E, and the reference photograph) have comparatively low mean intensity values, while low likeness images (Portraits A, D, and F) have relatively high mean intensity values. Low likeness Portrait H is an outlier with a very high level of image contrast, followed by high likeness Portrait B and the reference photograph.

Figure 8.

The mean brightness and contrast of the Original, Greyscale, Equalised histogram, and Normalised mean intensity images. The shape-free versions are indicated by dashed lines. The six Original colour images are indicated by an asterisk; Phot is the reference photograph; K is the portrait average.

Following both histogram equalisation and normalisation to the mean intensity, the brightness levels no longer correspond to those of the Original and Greyscale images. The Equalised histogram images are considerably darker (W = 78, p < .001), and the reference photograph becomes the brightest image. Histogram equalisation increases image contrast, but this increase varies according to each image, and to the extent that they no longer agree with the contrast patterns of the Original and Greyscale images. Normalising the mean intensity maintains the contrast values of the Original and Greyscale images (r ≥ .98, p < .001), and as can be seen in Figure 9, the histograms are suitably similar in shape. The brightness and contrast values by extent and type of preprocessing are listed in Supplemental Materials Table SM1, and the full results of the comparisons (Pearson’s r) of mean intensity and SD are provided in Supplemental Materials Table SM2.

Figure 9.

The intensity histograms arising from the Original images, Greyscale, Equalised histogram, and Normalised mean intensity images. The x axis is the tonal values from white (0) to black (255), and the y axis the frequency of these values within the image. The Original colour images are indicated by an asterisk, K is the portrait average.

There is no significant relationship between the mean familiar likeness assessments and either image brightness or contrast, and this holds regardless of the extent and type of image pre-processing. Relative shape accuracy is significantly correlated with image contrast, but only following histogram equalisation (r = .65, p = .03), and this does not sustain following Bonferroni correction.

Shape-Free Brightness (Mean Intensity), Contrast (SD), and Familiar Likeness

The shape-free versions of the images differ in the intensity values of the images from which they were derived. As can be seen in the dashed lines in Figure 8 and the statistical comparisons in Table 2, the extent and significance of these differences varies according to the extent and type of image preprocessing.

Table 2.

Significance of the Changes to Intensity Values Following Transformation to Shape-Free.

	Value	Image	Shape-free	Wilcoxon pairwise	Pearson’s r
Original (unprocessed)	X̅	149.6	150.1	W = 74, p = .003	r = 1.00, p < .001
Original (unprocessed)	SD	26.0	26.6	W = 66, p = .03	r = 1.00, p < .001
Greyscale	X̅	153.9	154.1	W = 54, n. sig.	r = 1.00, p < .001
Greyscale	SD	26.0	27.9	W = 67, p = .02	r = 1.00, p < .001
Equalised histogram	X̅	127.5	128.6	W = 69, p = .02	r = –.002, n. sig.
Equalised histogram	SD	73.6	72.4	W = 71, p = .009	r = –.41, n. sig.
Normalised mean intensity	X̅	153.8	154.5	W = 65, p = 002	r = .58, p = .05
Normalised mean intensity	SD	24.3	25.6	W = 55, n. sig.	r = 1.00, p < .001

Note. Results of Wilcoxon pairwise (W) and Pearson’s correlation (r) of the changes to image intensity values following transformation to shape-free. X̅ is the mean intensity (brightness), and SD is the standard deviation (contrast).

Not recorded on Table 2 is that, following shape-free image registration, there is a strong level of agreement between the brightness levels of the Equalised histogram and Normalised mean intensity images (r = .87, p = .007). The Equalised histogram shape-free images, however, continue to be significantly darker (W = 78, p < .001) and have higher levels of image contrast (W = 78, p < .001).

Following shape-free image registration, the brightness levels of the Normalised mean intensity images are significantly correlated with the familiar likeness assessments (r = .66, p = .02). These brightness levels are also, and more strongly, correlated with the relative shape accuracy (r = –.79, p = .001). The Equalised histogram shape-free image mean intensities are also significantly, though inversely, correlated with shape accuracy (r = –.73, p = .01). However, and as is illustrated in Figure 10, subtracting the shape-free mean intensity values from the mean intensity values of the images from which they were derived, shows that the more shape accurate images become brighter than the images that are less shape accurate when shape-free, and vice versa. This occurs with statistical significance for all of the shape-free images, regardless of the extent and type of image pre-processing (Original: Pearson’s r = .83, p = .002; Greyscale: r = .79, p = .004; Equalised histogram: r = .73, p = .01; Normalised mean intensity: r = .82, p = .002). The intensity values of the shape-free images are listed in Supplemental Materials Table SM3.

Figure 10.

The relationship of change in brightness after shape-free registration to shape accuracy. The images are ordered by their relative shape accuracy (refer Table 1), from the most to least accurate. K is the portrait average; the Original colour images are indicated by an asterisk. Listed below the portrait letters are the mean familiar likeness assessments each image attained.

When the reference photograph is excluded, the resulting shape-free images are consistently brighter and have a lower contrast regardless of the extent and type of image preprocessing (W = 66, p = .001), but still strongly agree with the brightness and contrast levels of the shape-free images including the reference photograph (r ≥ .98, p < .001). The relative brightness of the shape-free images is consistent with the reference photograph tending to be darker and with a higher level of image contrast than most of the images in the data set. The shape-free image values without the reference photograph are listed and compared in in Supplemental Materials, Tables SM4 and SM5.

Texture PCA

The level of agreement between the PCAs including the reference photograph (N = 12) and those excluding the reference photograph (n = 11) is very strong, and particularly strong for the components capturing the majority of the texture variance (r ≥ .90, p < .001; refer Supplemental Tables SM6–9).

The eigenpictures arising from the Original, Greyscale, and Equalised histogram image PCAs including the reference photograph (N = 12), the percentage variance associated with each eigenpicture, the relative strength of the agreements between the Original and Greyscale image PCA results, and between the Greyscale and Equalised histogram PCA results are illustrated in Figure 11. As can be seen, the component loadings of the Original and Greyscale images, and their corresponding eigenpictures, tend to strongly agree with each other, with the exception of the first component (PC1).

Figure 11.

InterFace eigenpictures for PC1–11: agreement between the Original, Greyscale, and Equalised histogram image texture PCA (N = 12). The top row shows the Original image eigenpictures, underneath which are the corresponding Greyscale eigenpictures, followed by the eigenpictures after the Greyscale images had been pre-processed using histogram equalisation. Listed with the eigenpictures are the eigenvalues (the percentage of texture variance being captured by each principal component). The black connectors are where there is a very high level of agreement between the components (r ≥ .80, p ≤ .001), and the grey connectors are where the agreement is less strong (r ≥ .58 ≤ .73, p ≤ .05).

There is a comparatively limited level of agreement between the Greyscale image PCA and the PCA following histogram equalisation, both in the strength of the correlation coefficients and in the eigenpicture variance. The Equalised histogram analysis is also less discriminating, with eigenvalues that are more evenly spread across the principal components. The correlation coefficients arising from these comparisons are given in Supplemental Materials Tables SM10 (Original to Greyscale) and SM11 (Greyscale to Equalised histogram).

Figure 12 compares the PCA results of the Normalised mean intensity images with the Greyscale and Equalised histogram images. The Greyscale image PC1 only weakly agrees with the Normalised mean intensity PC4, but otherwise there is a strong level of agreement across the components and the appearance of the associated eigenpictures. Comparison of the Normalised mean intensity PCA with the Equalised histogram PCA indicates they most strongly agree for PC1. The correlation coefficients arising from this comparison are given in Supplemental Materials Table SM12 (Greyscale and Normalised mean intensity) and Table SM13 (Normalised mean intensity and Equalised histogram).

Figure 12.

InterFace eigenpictures for PC1–11: agreement between the Greyscale, Normalised mean intensity, and Equalised histogram image texture PCA (N = 12). The top row shows the Greyscale image eigenpictures, underneath which are the corresponding Normalised mean intensity eigenpictures, followed by Equalised histogram eigenpictures. Listed with the eigenpictures are the eigenvalues (the percentage of texture variance being captured by each principal component). The black connectors are where there is a very high level of agreement between the principal components (r ≥ .80, p ≤ .001), and the grey connectors are where the agreement is less strong (r ≥ .58 ≤ .78, p ≤ .05).

Table 3 compares each of the Original, Greyscale, Equalised histogram, and Normalised mean intensity image PC loadings with the corresponding shape-free intensity values (listed in Supplemental Materials Table SM3). Brightness is strongly significant for the Original image PC2 and Greyscale image PC1, and agrees with the brightness of the associated eigenpictures (Figure 11). Both the Equalised histogram PCA and the Normalised mean intensity PCA result in low eigenvalues being weakly significant for overall image brightness.

Table 3.

InterFace Texture PCA and Shape-Free Histogram Data.

	Brightness		Contrast
Original	PC2	r = .89, p < .001	PC1	r = –.58, p = .05
Greyscale	PC1	r = .99, p < .001	PC2	r = .67, p = .02
Equalised histogram	PC10	r = –.65, p = .02	PC1	r = –.58, p = .05
Normalised mean intensity	PC6	r = .60, p = .04	PC1	r = .82, p = .001

Note. Comparison (Pearson’s r) of the Original, Greyscale, Equalised histogram, and Normalised mean intensity image PCA loadings with the intensity values of the corresponding shape-free images.

Image contrast is weakly significant for the Original image PC2, and not PC3, though the high-contrast Portrait H appears present in the PC3 eigenpicture (Figure 11). The Greyscale PC2, Equalised histogram PC1, and Normalised mean intensity PC1 also contain the visual traces of Portrait H (Figure 12), and all are significantly correlated with image contrast. Variability in contrast accounts for most of the variance across the Normalised mean images (PC1 26%).

Texture PCA, Likeness, and Shape Accuracy

Comparison of the mean familiar likeness assessments and relative shape accuracy of the portraits and portrait average to the InterFace PC loadings undertaken with, and without, the reference photograph results in weakly significant correlation coefficients with low component eigenvalues. None sustain following Bonferroni correction (refer Supplemental materials Table SM14), and none of these weakly significant components have associated eigenpictures that appear—subjectively—to be related to the depiction of shading: Original image PC4 (4% overall variance), Greyscale PC3 (8%), and Normalised mean intensity PC2 (24%) (refer Figures 11 and 12).

Segmenting Shading

The shading segmentation analyses indicate Portrait H has the greatest proportion of black to white pixels as a threshold shape-free image, followed by the reference photograph. Portrait J is most similar to the threshold proportions of the reference photograph, followed by Portrait B. The shape-free image that has the greatest level of threshold shading difference from the reference photograph is again Portrait H, followed by Portrait C, while Portrait B and the portrait average (K) have the least level of threshold shading difference. All proportional counts arising from the shading segmentations are listed in Supplemental Materials Table SM15, and the posterised, threshold and difference filter images are illustrated in Supplemental Materials Figure SM1.

As can be seen in Table 4, the extent of additional shading and the proportion of white pixels (highlights) are inversely correlated, and both are significantly related to the extent of threshold shading, and the degree of difference in threshold shading, from the reference photograph. There is also an inverse relationship between where the images agree with, and where they are missing, the reference photograph’s threshold shading. However, neither Agree shading or Missing shading is significantly correlated with any other segmentation measure. None of the shading estimates are related to the brightness levels of the images prior to segmentation, though image contrast is significantly related to the extent of threshold shading, the extent of additional shading, and the extent of image highlights.

Table 4.

Relationship of the Shading Segmentations and the Adjusted Greyscale Shape-Free Intensity Values.

	Threshold	Difference	Agree	Missing	Additional	Highlights	Adj. Mean	Adj. SD
Threshold		1	1	1	0.01	0.01	1	0.009
Difference	0.57		1	1	0.03	0.03	1	(0.01)
Agree	0.53	–0.39		>0.001	1	1	1	1
Missing	–0.53	0.39	–1.00		1	1	1	1
Additional	0.90	0.87	0.11	–0.11		>0.001	1	0.003
Highlights	–0.90	–0.87	–0.11	0.11	–1.00		1	0.003
Adj. Mean	–0.13	–0.01	–0.12	0.11	–0.09	0.09		1
Adj. SD	0.90	0.75	0.26	–0.26	0.93	–0.93	0.03

Note. Comparison (Pearson’s r) of the numerical estimations of shading, with Bonferroni correction. Significance that did not sustain following correction is indicated in brackets. Threshold is the dominant shading and shadows; Difference is where this differs from the reference photograph. Agree is where the shading matches the reference photograph; Missing is where the reference photograph shading is absent; Additional is where the image has added shading not present in the reference photograph; and Highlights are the white pixels that remain. The table includes the adjusted mean intensity (Adj. Mean) and standard deviation of the values (Adj. SD) of the Greyscale shape-free images prior to posterisation.

Further segmenting the threshold shape-free images into where, and how, they compare to the reference photograph threshold shading is illustrated in Figure 13. The segmentation images and bar graph are arranged from highest to lowest level of Agree shading. As can be seen, the highest level of shading agreement occurs with both high and low likeness images (Portraits B, J, the portrait average (K), and Portrait A), as does the extent of additional shading (Portraits H, J, A, and B). As a consequence, none of the shading segmentation proportions are significantly related to either the mean likeness assessments (r ≤ .40, p > .05) or relative shape accuracy (r ≤ .45, p > .05).

Figure 13.

Threshold shading segmentation. Agree (green/dark grey): where the images agree with the reference photograph; Missing (black): where the images do not contain shading present in the reference photograph; Additional (grey): where the images have shading that is not present in the reference photograph; Highlights (dashed line, secondary Y axis values): where a lack of shading agrees with the reference photograph. The images and bar graph are ordered by highest to lowest levels of shading agreement. Original colour images are indicated by an asterisk. K is the portrait average.

Texture PCA (n = 11) and the Segmented Threshold Shading Proportions

Table 5 compares the threshold shading proportional counts and the intensity values of the Greyscale shape-free images (following their normalisation to their average intensity) to the PC loadings resulting from the texture PCAs undertaken without the reference photograph (n = 11). As can be seen, the Greyscale PC3, Equalised histogram PC2, and Normalised mean intensity PC2 are each significantly correlated with the extent to which the images agree with the reference photograph’s threshold shading (Agree). The eigenpicture and texture variance being captured by the Greyscale PC3, Equalised histogram PC2, and Normalised mean intensity PC2 are illustrated in Figure 14, together with the eigenpicture and variance of the Original image PC4. Figure 14 also includes images illustrating each components’ variance, which have been extracted at intervals of 0.5 to display the variance within the component, together with threshold (level 128) versions of the component interval images.

Table 5.

Comparison (Pearson’s r) of the Shading Results and Shape-Free Intensity Values With the PCA Loadings (n = 11).

	PC	Original	PC	Greyscale	PC	Equal. histogram	PC	Normalised mean
Thr.	2	r = .64, p = (.04)		n.sig.	1	r = .64, p = (.03)	1	r = .80, p = (.003)
Diff.	3	r = –.82, p = (.002)	2	r = –.83, p = (.002)	1	r = .73, p = (.01)	1	r = .88, p = .05
Agree		n.sig.	3	r = –.85, p = (.001)	2	r = –.70, p = (.01)	2	r = .94, p = .002
Add.	3	r = –.62, p = (.04)	2	r = –.79, p = (.004)	1	r = .76, p = (.02)	1	r = .93, p = .004
Mean	2	r = –.89, p = .01	1	r = .97, p < .001		n.sig.		n.sig.
SD	1	r = –.69, p = (.02)	2	r = –.78, p = (.004)	1	r = –.67, p = (.02)	1	r = .89, p = .02

Note. PCA results of the Original, Greyscale, Equalised histogram, and Normalised mean intensity images (without the reference photograph) compared to the Threshold (Thr.), Difference (Diff.), Agree, and Additional (Add.) shading proportional pixel counts. The mean and standard deviation (SD) are the intensity values of each PCA’s shape-free images (n = 11). The PC and r values with significant relationships that sustain following Bonferroni correction are in bold. Uncorrected p values are in brackets. The underlined PCs are those that have associated eigenpictures that can be visually interpreted as displaying variance in form and cast shadows.

Figure 14.

PCA shading depiction variance (without the reference photograph, n = 11). The Normalised mean intensity (top row), Greyscale (upper middle row), Equalised histogram (lower middle row), and Original (bottom row) principle components that each have an eigenpicture (to the left of the axis) indicating a relationship to shading variance. Below each component axis are images of the variance extracted at 0.5 intervals. The grey dots on the axes are the locations of the portraits (A–J) and K, the portrait average. The Normalised mean intensity PC2 (top row) includes threshold versions of the component variance images.

The eigenpictures associated with the Normalised mean intensity PC2, Greyscale PC3, and Original PC4 arising from the PCAs undertaken without the reference photograph (Figure 14) are very similar in appearance to the eigenpictures associated with the same components arising from the PCAs including the reference photograph (Figures 11 and 12). The eigenpicture for the Equalised histogram PC2 is also similar, but is showing the inverse of the variance when the reference photograph is included. As can be seen in Table 5, only the Normalised mean intensity, Greyscale, and (to a lesser extent) Equalised histogram eigenpictures are statistically significant for variance in texture related to the extent to which the images Agree with the reference photograph’s shape-free threshold shading. However, it is only the Normalised mean intensity PC2 that retains this significance following Bonferroni correction.

Referring again to Figure 14, each principal component is similarly differentiating the texture variance of Portrait B at one extreme of the axis, and from the texture variance of Portraits C and/or G at the other extreme. The Normalised mean intensity PC2 loadings strongly correlate with those of Greyscale PC3 (r = –.95, p < .001), weakly agree with the Equalised histogram PC2 (r = –.70, p = .02), and moderately agree with the Original image PC4 (r = .81, p = .01). However, although there is a significant level of agreement, it can be seen that the loadings of the intervening images differ within each PC. It is likely, therefore, that while none of these components are significant for either image brightness or image contrast (Table 5), the Greyscale, Equalised histogram, and Original image principal components have higher levels of residual texture variance related to these image-level values. For the Original images there is the additional texture variance related to colour, and this has probably contributed to the component failing to attain a statistically significant relationship with the Agree shading segmentation.

When the reference photograph is included in the Normalised mean intensity PCA, the PC2 loadings strongly agree with the Normalised mean intensity PC2 undertaken without the photograph (r = 1.00, p < .001). As can be seen in Figure 15, the variance images extracted at 0.5 intervals along the axis are close to identical to those excluding the reference photograph (Figure 14). As can also be seen in Figure 15, the reference photograph is located between the low likeness Portrait J and the high likeness Portrait B, and closer to Portrait J than any other image. As with the PCA that excluded the reference photograph, the threshold versions of the PC2 interval images show that the texture variance is from relatively minimal shading with an ambiguous indication of lighting direction (negative values), to a strongly marked shading pattern with a less ambiguous shading pattern, and a stronger light source (positive values). Furthermore, all but one of the images have reduced the amount of shading relative to the reference photograph.

Figure 15.

Normalised mean intensity PC2 (including the reference photograph, N = 12). The Normalised mean intensity PC2 (24% variance) axis from the PCA undertaken including the reference photograph (N = 12). The variance images below the axis are at 0.5 intervals, and beneath are threshold versions of the variance. The grey dots are the locations of the portraits (A–J) and portrait average (K). The location of the reference photograph within the component variance is indicated by a smaller grey dot.

Discussion

The expectation of this experimental case study was that the results of an InterFace (Kramer et al., 2017) PCA of texture would show some correspondence with the familiar likeness assessments that 10 portraits and a portrait average (all depicting the same Sitter and referring to the same reference photograph) attained during a public exhibition (Hayes et al., 2020). A texture PCA is known to retain pixel values related to the influence of lighting direction (e.g., Burton et al., 2001; Hancock et al., 1996; Jenkins & Burton, 2011; Russell & Sinha, 2007), and although the relationship of likeness to portrait texture does not appear to have been previously analysed, there are indications in the literature that portraiture may be perceptually enhanced by a relatively coherent and clear depiction of the light source (Arnheim, 1974; Ostrovsky et al., 2005; Stork, 2011), which is most often depicted above and to the left of the composition (Sun & Perona, 1998).

The initial texture PCA involving the Original (unprocessed) portraits, portrait average, and reference photograph appeared to identify a slight degree of variance in the depiction of shading (Figure 11: Original images PC4, 4% variance), though none of the components attained a statistically significant relationship to the mean familiar likeness assessments. The 12 images differ according to the materials, media, and style, and an examination of the pixel intensity values indicated that, in addition to colour, they vary substantially in brightness and contrast, and it is well established that image-level variability—and in particular image brightness—will confound a texture PCA (e.g., Jenkins & Burton, 2011; Kirby & Sirovich, 1990; Moon & Phillips, 2001; Nestor et al., 2013; O'Toole et al., 1999; Russell & Sinha, 2007; Zhang & Turk, 2008). Therefore, and because the brightness and contrast values continued to result in image clusters unrelated to likeness following conversion to Greyscale (Figure 8), two different methods of image pre-processing were applied to see which would best reduce the influence of these uninformative pixel values.

Histogram equalisation was found to reduce the influence of both image brightness and contrast, but resulted in no eigenpicture indicating variance in light and shadow, and no clear relationship of texture to likeness. Linear histogram matching to a Normalised mean intensity only reduced the influence of brightness but also failed to result in a statistically significant relationship to likeness. Normalising the mean intensity of the images, however, resulted in most of the variables in image contrast being captured by PC1 (Figure 12: 28% variance), followed by an eigenpicture suggesting nearly a quarter of the overall image variance is due to the depiction of shading (PC2, 24% variance). All of the PCA loadings were found to be close to identical when the reference photograph was excluded from the analyses, and together these results indicate that, at least for this set of images:

Histogram equalisation reduces image brightness and enhances image contrast but disrupts the depiction of shading, and the eigenvalues indicate it is less effective in discriminating texture variance;

Linear histogram matching reduces uninformative variance due to image brightness while enhancing texture variance due to the depiction of shading; and

Although tending to be darker and with a high level of image contrast, the reference photograph was found to have minimal impact on the overall texture variance.

Shape-free image registration greatly improves PCA performance by enabling a relatively direct comparison of the corresponding pixel values across different digital images (Craw & Cameron, 1992; Hancock et al., 1996). After a series of experiments, this study found that shape-free image registration also facilitates the application of different image filters to result in numerical estimations of the dominant shading patterns present in each image (Figure 13). Although approximate due to the posterisation and threshold filters conflating much of the more subtle tonal variability in the portraits, nevertheless these estimates provided an objective interpretation of the PCA eigenpictures that had subjectively indicated they were capturing variance in the depiction of shading. This relationship only failed to reach significance with the Original images, and had the greatest effect (Pearson’s r = .94) with the Normalised mean intensity image PC2 loadings (Table 5).

For ease of reference, the Normalised mean intensity PC2 eigenpicture and images illustrating the interval variance (including the reference photograph) are reproduced here (Figure 16), together with the Original images, and the Normalised mean intensity images following shape-free image registration, ranked by their Normalised mean intensity PC2 loadings. As can be seen, the PC2 interval variance is differentiating the shading texture values related to the depiction of lighting direction. The reference photograph is located within values with a stronger light source above and to the right (to the left in the picture frame), indicated by a more pronounced shading on the left hemiface. The threshold versions of the PC2 interval variance indicate the reference photograph shading is most marked in three main facial regions: the left nasolabial fold (where the cheek brackets the mouth), across lower left cheek, and within the inner and lower regions of the left eye socket.

Figure 16.

Texture PCA: variance in lighting direction. All images are organised by the Normalised mean PC2 loadings resulting from the analysis including the reference photograph. Above the PC2 axis are the Original images prior to shape-free registration and the Normalised mean intensity images following shape-free registration. Beneath the PC2 axis is the PC2 eigenpicture (left), and images representing the component variance at 0.5 intervals (indicated by black dots along the axis). The Original colour images are indicated by an asterisk. Note that the PC2 axis has been truncated between +1 and +2.5.

The PC2 interval variance indicates that 9 of the 10 portraits, and the portrait average (K), have reduced the depiction of shading relative to the reference photograph, and that at both extremes of the PC2 axis the shading tends to be inconsistently represented. The largest negative interval variance (–1) indicates the lower face shading is associated with a weak light source above and to the right, while the upper face is shown lit from directly above, and by a weaker light source. The highest positive interval variance (+2.5) indicates an exaggeration of shading relative to the reference photograph, and while the direction of the lighting is fairly consistent, the extent of shading is far more exaggerated on the mid to lower left cheek. Towards the centre of the axis (0), the interval variance indicates that the depiction of shading, though reduced, is relatively coherent with regards to both direction and extent.

Referring to the likeness assessments (Table 1), the portraits that cluster towards the maximum negative values were assessed, on average, as low likeness by those familiar with the Sitter in life. The portrait average clusters with the middle value images and was considered a very good likeness. The only portrait to exaggerate shading attained the highest mean familiar likeness, and viewers who were experienced visual artists tended to give this portrait a higher likeness assessment. This suggests that portrait texture depiction agrees with previous findings from face perception studies: exaggerating the facial texture cues that include shading enhances recognition and likeness (Itz et al., 2014; Lee & Perrett, 2000), while removing texture cues relating to shading both reduces recognition (Bruce et al., 1991, 1992; Davies et al., 1978; Galper, 1970; Johnston et al., 1992) and the likeness ratings of photographic averages (Jenkins & Burton, 2011; Ritchie et al., 2018). However, this apparent agreement cannot be concluded from this limited case study.

As suggested by Lee and Perrett (2000), the degree of texture exaggeration in photographs likely differs according to the degree of texture distinctiveness in the depicted face, and this may also hold for portraits. Furthermore, while the relatively weak presence of shading in the majority of the portraits analysed here could be due to a low level of visual distinction between the illumination and surface properties (Arnheim, 1974), a minimal depiction of lighting direction is not unusual within the visual arts. Stork (2011) has observed that not all artists, including Vincent van Gogh, include shadows when depicting faces, and Casati and Cavanagh (2019) suggest that because the presence or absence of cast and/or form shadows tends not to register strongly on human perceptual systems, artists are under no obligation to depict them. In addition, human perception is insensitive to incoherent representations of shading and shadows (Casati & Cavanagh, 2019), and while a gross anomaly in facial shadows is very quickly perceived (Ostrovsky et al., 2005), the relatively subtle shading inconsistencies present in most of the portraits analysed here were unlikely to have been noticed by viewers. Therefore, while the results of this case study are indicative, further experimentation with a larger group of portraits depicting different Sitters displaying a range of facial textures is required.

A further finding indicated by this case study relates to the relationship of texture to shape. Although the extent to which this contributed to the results is not known, the less shape accurate portraits were found to be significantly darker following shape-free registration, with the least accurate undergoing the greatest degree of texture change (Figure 10), and this occurred regardless of the extent and type of image pre-processing. Furthermore, and as can be seen in Figure 16, the portrait that is closest to the PC2 loadings of the reference photograph, and is therefore the most similar to the reference photograph’s depiction of shading, is the least shape accurate and attained a low mean familiar likeness assessment (Portrait J). As noted by Burton et al. (2001), shape-free is a misleading term given it implies shape and texture are independent variables. Other researchers have also noted that it is very likely that texture cannot be separated from shape (e.g., Lee & Perrett, 2000; Russell & Sinha, 2007), but there is still a tendency for face perception studies to conclude that texture cues are more important than facial shape for familiar recognition. This does not, however, take into account that not all shape changes are the same. In Figure 16, it can be clearly seen that, as an Original image, Portrait J has compressed the height of the Sitter’s face. The relative accuracy of the shading following shape-free image registration indicates Portrait J’s depiction of shading is consistent with this vertical compression, and therefore -if texture is more important than shape- this portrait should have been considered a good likeness. Gilad-Gutnick et al. (2018), however, have found that even slight compressions along the vertical dimensions of the face deleteriously impact the familiar recognition of photographs, and by extension, their familiar likeness assessments. In essence, this case study not only affirms the interdependence of shape and texture but also raises the possibility that shape-free image registration could more usefully, and accurately, be referred to as shape-shift.

To conclude, this initial study of portrait texture has found that it is possible to apply a texture PCA to identify the extent and coherence of lighting direction depiction in portraits, and that if image preprocessing is required, linear histogram matching retains this information while automated histogram equalisation does not. Unfortunately, a relationship to likeness in portraiture was not able to be established using either principal component loadings or shading segmentation. This failure to attain a statistically significant relationship of likeness to portrait texture is very likely due to the limitations of the study. A larger data set of portraits depicting a range of different Sitters—and, ideally, all produced using a narrower range of media and materials—could result in a more meaningful contribution, as would knowing the likeness of the portraits after, as well as before, shape-free image registration.

Supplemental Material

sj-pdf-1-pec-10.1177_0301006620975705 - Supplemental material for Analysing Texture in Portraits

Supplemental material, sj-pdf-1-pec-10.1177_0301006620975705 for Analysing Texture in Portraits by Susan Hayes in Perception

Footnotes

Acknowledgements

This research acknowledges the Traditional Owners and Custodians of the land on which it has taken place, the Wodi Wodi people of the Dharawal Nation, and pays respect to Elders past, present, and emerging for their knowledge and care of country. The collaborating artists who contributed their thoughts, skills, and time to enable this research were Donna Abbati, Emma Calvert, Kim Christopher, Maggie Henderson, Elspeth McCombe, Emma Medwell, Odette Smith, Julie Telenta, Joyce Wilcock, T. S. Zaracostas, and Dulcie Dal Molin (Red Point Artists Association, Port Kembla, NSW, Australia). The Sitter was Lord Mayor Gordon Bradbery AM (Wollongong City Council) and the photographer Gerrit van den Bergh (Centre for Archaeological Science, School of Earth and Environmental Science, University of Wollongong, Australia). Sincere thanks to the two Perception reviewers who first suggested undertaking a portrait texture analysis, and to Thomas Hayes, Gerrit van den Bergh, Gillian Porter, Holger Wiese, Perception’s Chief Editors, and the three reviewers who greatly improved this article by providing detailed and highly relevant advice and recommendations.

Declaration of Conflicting Interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The author(s) received no financial support for the research, authorship, and/or publication of this article.

ORCID iD

Susan Hayes

Supplemental Material

Supplemental material for this article is available online.

References

Andrews

T. J.

Baseler

Jenkins

Burton

A. M.

Young

A. W.

(2016). Contributions of feature shapes and surface cues to the recognition and neural representation of facial identity. Cortex, 83, 280–291. https://doi.org/10.1016/j.cortex.2016.08.008

Arnheim

(1974). Art and visual perception: A psychology of the creative eye. University of California Press.

Bartlett

M. S.

Hager

J. C.

Ekman

Sejnowski

T. J.

(1999). Measuring facial expressions by computer image analysis. Psychophysiology, 36, 253–263. https://doi.org/10.1017/s0048577299971664

Benson

P. J.

Perrett

D. I.

(1991). Perception and recognition of photographic quality facial caricatures: Implications for the recognition of natural images. European Journal of Cognitive Psychology, 3, 105–135.

Bruce

Hanna

Dench

Healey

Burton

(1992). The importance of ‘mass’ in line drawings of faces. Applied Cognitive Psychology, 6, 619–628. https://doi.org/10.1002/acp.2350060705

Bruce

Healey

Burton

Doyle

Coombes

Linney

(1991). Recognising facial surfaces. Perception, 20, 755–769. https://doi.org/10.1068/p200755

Burton

A. M.

Jenkins

Hancock

P. J.

White

(2005). Robust representations for face recognition: The power of averages. Cognitive Psychology, 51, 256–284. https://doi.org/10.1016/j.cogpsych.2005.06.003

Burton

A. M.

Miller

Bruce

Hancock

P. J. B.

Henderson

(2001). Human and automatic face recognition: A comparison across image formats. Vision Research, 41, 3185–3195. https://doi.org/10.1016/s0042-6989(01)00186-9

Burton

A. M.

Schweinberger

S. R.

Jenkins

Kaufmann

J. M.

(2015). Arguments against a configural processing account of familiar face recognition. Perspectives on Psychological Science, 10, 482–496. https://doi.org/10.1177/1745691615583129

10.

Casati

Cavanagh

(2019). The visual world of shadows. MIT Press.

11.

Cavanagh

(2005). The artist as neuroscientist. Nature, 434, 301–307. https://doi.org/10.1038/434301a

12.

Cavanagh

Leclerc

Y. G.

(1989). Shape from shadows. Journal of Experimental Psychology: Human Perception and Performance, 15, 3–27.

13.

Craw

Cameron

(1992). Face recognition by computer. In Hogg D., Boyle R. (eds) BMVC92: British machine vision conference (pp. 498–507). Springer. https://doi.org/10.1007/978-1-4471-3201-1_52

14.

Davies

Ellis

H. D.

Shepherd

(1978). Face recognition accuracy as a function of mode of representation. Journal of Applied Psychology, 63, 180. https://doi.org/10.1037/0021-9010.63.2.180

15.

Galper

R. E.

(1970). Recognition of faces in photographic negative. Psychonomic Science, 19, 207–208.

16.

Gilad-Gutnick

Harmatz

E. S.

Tsourides

Yovel

Sinha

(2018). Recognizing facial slivers. Journal of Cognitive Neuroscience, 30, 951–962. https://doi.org/10.1162/jocn_a_01265

17.

Gombrich

E. H.

(1977). Art and illusion: A study in the psychology of pictorial representation (5th ed.). Phaidon Press.

18.

Gonzalez

R. C.

Woods

R. E.

Masters

B. R.

(2008). Digital image processing third edition. Pearson International Edition.

19.

Hammer

Ø.

Harper

Ryan

(2001). Past-palaeontological statistics. www.uv.es/∼pardomv/pe/2001_1/past/pastprog/past.pdf

20.

Hancock

P. J.

Burton

A. M.

Bruce

(1996). Face processing: Human perception and principal components analysis. Memory and Cognition, 24, 21–40. https://doi.org/10.3758/BF03197270

21.

Hayes

Caputi

Zaracostas

Henderson

Telenta

McCombe

Christopher

Calvert

Abbati

Smith

Medwell

Wilcock

(2020). Likeness, familiarity, and the ambient portrait average. Perception, 49, 567–587. https://doi.org/10.1177/0301006620905420

22.

Hayes, S., Rheinberger, N., Powley, M., Rawnsley, T., Brown, L., Brown, M., Butler, K., Clarke, A., Crichton, S., Henderson, M., McCosker, H., Musgrave, A., Wilcock, J., Williams, D., Yeaman, K., Zaracostas, T. S., Taylor, A. C., & Wallace, G. (2018, Jun). Variation and Likeness in Ambient Artistic Portraiture. Perception, 47(6), 585–607. https://doi.org/10.1177/0301006618770347

23.

Itz

M. L.

Schweinberger

S. R.

Schulz

Kaufmann

J. M.

(2014). Neural correlates of facilitations in face learning by selective caricaturing of facial shape or reflectance. Neuroimage, 102, 736–747. https://doi.org/10.1016/j.neuroimage.2014.08.042

24.

Jenkins

Burton

A. M.

(2011). Stable face representations. Philosophical Transactions of the Royal Society B: Biological Sciences, 366, 1671–1683. https://doi.org/10.1098/rstb.2010.0379

25.

Johnston

Hill

Carman

(1992). Recognising faces: Effects of lighting direction, inversion, and brightness reversal. Perception, 21, 365–375. https://doi.org/10.1068/p210365

26.

Kemp

Pike

White

Musselman

(1996). Perception and recognition of normal and negative faces: The role of shape from shading and pigmentation cues. Perception, 25, 37–52. https://doi.org/10.1068/p250037

27.

Kirby

Sirovich

(1990). Application of the Karhunen-Loeve procedure for the characterization of human faces. IEEE Transactions on Pattern Analysis and Machine Intelligence, 12, 103–108. https://doi.org/10.1109/34.41390

28.

Kramer

R. S.

Jenkins

Burton

A. M.

(2017). InterFace: A software package for face image warping, averaging, and principal components analysis. Behavior Research Methods, 49, 2002–2011. https://doi.org/10.3758/s13428-016-0837-7

29.

Lee

K. J.

Perrett

D. I.

(2000). Manipulation of colour and shape information and its consequence upon recognition and best-likeness judgments. Perception, 29, 1291–1312. https://doi.org/10.1068/p2792

30.

MacLeod

(2013). A picture may be worth 1,000 landmarks. Palaeontological Association Newsletter, 84, 20–34. https://www.palass.org/publications/newsletter/palaeomath-101/palaeomath-part-30-picture-may-be-worth-1000-landmarks

31.

Maughan

W. L.

(2004). The artist's complete guide to drawing the head. Watson-Guptill.

32.

Moon

Phillips

P. J.

(2001). Computational and performance aspects of PCA-based face-recognition algorithms. Perception, 30, 303–321. https://doi.org/10.1068/p2896

33.

Nestor

Plaut

D. C.

Behrmann

(2013). Face-space architectures: Evidence for the use of independent color-based features. Psychological Science, 24, 1294–1300. https://doi.org/10.1177/0956797612464889

34.

Ostrovsky

Cavanagh

Sinha

(2005). Perceiving illumination inconsistencies in scenes. Perception, 34, 1301–1314.

35.

O'Toole

A. J.

Price

Vetter

Bartlett

J. C.

Blanz

(1999). 3D shape and 2D surface textures of human faces: The role of “averages” in attractiveness and age. Image and Vision Computing, 18, 9–19. https://doi.org/10.1016/S0262-8856(99)00012-8

36.

Ritchie

K. L.

Kramer

R. S.

Burton

A. M.

(2018). What makes a face photo a ‘good likeness’? Cognition, 170, 1–8. https://doi.org/10.1016/j.cognition.2017.09.001

37.

Rohlf

F. J.

(2015). The tps series of software. Hystrix, the Italian Journal of Mammalogy, 26, 9–12.

38.

Russell

Biederman

Nederhouser

Sinha

(2007). The utility of surface reflectance for the recognition of upright and inverted faces. Vision Research, 47, 157–165. https://doi.org/10.1016/j.visres.2006.11.002

39.

Russell

Sinha

(2007). Real-world face recognition: The importance of surface reflectance properties. Perception, 36, 1368–1374.

40.

Schindelin

Arganda-Carreras

Frise

Kaynig

Longair

Pietzsch

Preibisch

Rueden

Saalfeld

Schmid

Tinevez

J.-Y.

White

D. J.

Hartenstein

Eliceiri

Tomancak

Cardona

(2012). Fiji: An open-source platform for biological-image analysis. Nature Methods, 9, 676–682. https://doi.org/10.1038/nmeth.2019

41.

Sedgewick

(2010). Scientific imaging with Photoshop: Methods, measurement, and output. Peachpit Press.

42.

Sirovich

Kirby

(1987). Low-dimensional procedure for the characterization of human faces. Journal of the Optical Society of America A, Optics and Image Science, 4, 519–524. https://doi.org/10.1364/josaa.4.000519

43.

Smith

S. W.

(1997). The scientist and engineer’s guide to digital signal processing. California Technical Publishing. http://www.dspguide.com

44.

Speed

(1917). The practice and science of drawing (3rd ed.). Dover.

45.

Stork

D. G.

(2011, January). Computer analysis of lighting style in fine art: Steps towards inter-artist studies [Paper presentation]. Electronic Imaging: Computer Vision and Image Analysis of Art II, (Vol 7869), San Francisco, CA, United States: SPIE.

46.

Sun

Perona

(1998). Where is the sun? Nature Neuroscience, 1, 183–184. https://doi.org/10.1038/630

47.

Turk

Pentland

(1991, Winter). Eigenfaces for recognition. Journal of Cognitive Neuroscience, 3, 71–86. https://doi.org/10.1162/jocn.1991.3.1.71

48.

Zaidenberg

(1944). Drawing the human figure. Bell.

49.

Zhang

Turk

(2008). Eigenfaces. Scholarpedia, 3, 4244.

Supplementary Material

Please find the following supplemental material available below.

For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.

For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.

0.00 MB

0.87 MB