Perception

Models such as that of Olshausen and Field (O&F, 1997 Vision Research 37 3311–3325) and principal components analysis (PCA) have been used to model simple-cell receptive fields, and to try to elucidate the statistical principles underlying visual coding in area V1. They connect the statistical structure of natural images with the statistical structure of the coding used in V1. The O&F model has created particular interest because the basis functions it produces resemble the receptive fields of simple cells.

We evaluate these models in terms of their sparseness and dispersal, both of which have been suggested as desirable for efficient visual coding. However, both attributes have been defined ambiguously in the literature, and we have been obliged to formulate specific definitions in order to allow any comparison between models at all. We find that both attributes are strongly affected by any preprocessing (eg spectral pseudo-whitening or a logarithmic transformation) which is often applied to images before they are analysed by PCA or the O&F model. We also find that measures of sparseness are affected by the size of the filters—PCA filters with small receptive fields appear sparser than PCA filters with larger spatial extent. Finally, normalisation of the means and variances of filters influences measures of dispersal. It is necessary to control for all of these factors before making any comparisons between different models.

Having taken these factors into account, we find that the code produced by the O&F model is somewhat sparser than the code produced by PCA. However, the difference is rather smaller than might have been expected, and a measure of dispersal is required to distinguish clearly between the two models.

The visual system employs a gain control mechanism in the cortical coding of contrast whereby the response of each cell is normalised by the integrated activity of neighbouring cells. While restricted in space, the normalisation pool is broadly tuned for spatial frequency and orientation, so that a cell's response is adapted by stimuli which fall outside its ‘classical’ receptive field. Various functions have been attributed to divisive gain control: in this paper we consider whether this output nonlinearity serves to increase the information carrying capacity of the neural code. 46 natural scenes were analysed with the use of oriented, frequency-tuned filters whose bandwidths were chosen to match those of mammalian striate cortical cells. The images were logarithmically transformed so that the filters responded to a luminance ratio or contrast. In the first study, the response of each filter was calibrated relative to its response to a grating stimulus, and local image contrast was expressed in terms of the familiar Michelson metric. We found that the distribution of contrasts in natural images is highly kurtotic, peaking at low values and having a long exponential tail. There is considerable variability in local contrast, both within and between images. In the second study we compared the distribution of response activity before and after implementing contrast normalisation, and noted two major changes. Response variability, both within and between scenes, is reduced by normalisation, and the entropy of the response distribution is increased after normalisation, indicating a more efficient transfer of information.

Fourier-phase information is important in determining the appearance of natural scenes, but the structure of natural-image phase spectra is highly complex and difficult to relate directly to human perceptual processes. This problem is addressed by extending previous investigations of human visual sensitivity to the randomisation and quantisation of Fourier phase in natural images. The salience of the image changes induced by these physical processes is shown to depend critically on the nature of the original phase spectrum of each image, and the processes of randomisation and quantisation are shown to be perceptually equivalent provided that they shift image phase components by the same average amount. These results are explained by assuming that the visual system is sensitive to those phase-domain image changes which also alter certain global higher-order image statistics. This assumption may be used to place constraints on the likely nature of cortical processing: mechanisms which correlate the outputs of a bank of relative-phase-sensitive units are found to be consistent with the patterns of sensitivity reported here.

The human visual system is sensitive to both first-order variations in luminance and second-order variations in local contrast and texture. Although there is some debate about the nature of second-order vision and its relationship to first-order processing, there is now a body of results showing that they are processed separately. However, the amount, and nature, of second-order structure present in the natural environment is unclear. This is an important question because, if natural scenes contain little second-order structure in addition to first-order signals, the notion of a separate second-order system would lack ecological validity.

Two models of second-order vision were applied to a number of well-calibrated natural images. Both models consisted of a first stage of oriented spatial filters followed by a rectifying nonlinearity and then a second set of filters. The models differed in terms of the connectivity between first-stage and second-stage filters. Output images taken from the models indicate that natural images do contain useful second-order structure. Specifically, the models reveal variations in texture and features defined by such variations. Areas of high contrast (but not necessarily high luminance) are also highlighted by the models. Second-order structure—as revealed by the models—did not correlate with the first-order profile of the images, suggesting that the two types of image ‘content’ may be statistically independent.

We have developed a protocol for testing experimentally the hypothesis that the human visual system is optimised for making visual discriminations amongst natural scenes. Visual stimuli were made by gradual blending of the Fourier spectra of digitised photographs of natural scenes. The statistics of the stimuli were made unnatural to varying degrees by changing the overall slopes of the amplitude spectra of the stimuli. Thresholds were measured for discriminating small amounts of spectral blending at different spectral slopes. We found that thresholds were lowest when the spectral slope was natural; thresholds were increased when the slopes were either shallower or steeper than natural. A number of spurious cues were considered, such as differences in mean luminance or overall spectral power or contrast between test and reference stimuli. Control experiments were performed to remove such spurious cues, and the discrimination thresholds were still lowest for stimuli that were most natural. Thus, these experiments do provide experimental support for the idea that human vision and the human visual system are optimised for processing natural visual information.

It has been suggested (Tadmor and Tolhurst, 1994 Vision Research 34 541–554) that the psychophysical task of discriminating changes in the slope of the amplitude spectrum of a complex image may be similar to detecting differences in the degree of blur. It has also been suggested that human observers may perform this discrimination by detecting changes in the effective contrast within single narrow spatial-frequency bands, rather than by detecting changes in the slope per se which would involve the use of contrast information across many different frequency bands. To distinguish between these two possibilities, we have developed an experiment where observers were asked to discriminate changes in the spectral slope while different amounts of random contrast variation were introduced, with the purpose of disrupting their performance. This disruptive effect was designed to be particularly manifest if the observer really was performing a single-frequency-band contrast discrimination but to be unnoticeable if the observer was discriminating the change of slope per se. Our results imply that the observers do not usually detect changes in contrast in just one narrow spatial-frequency band when they discriminate changes in the slope of the amplitude spectrum. Rather, they must compare contrast between bands or, at least, they use contrast information from more than one band. However, for edge-enhanced (whitened) pictures, there is some evidence to suggest that observers rely on contrast changes in only a limited low-spatial-frequency band.

Peripheral performance involving simple visual tasks and stimuli can be equated with foveal performance by spatial scaling, whilst more complex tasks and stimuli seem to need additional scaling of image contrast. We therefore determined whether the contrast manipulation needed to compensate for eccentricity-dependent performance changes is due to an increase in stimulus or task difficulty. We measured contrast sensitivities to determine foveal and peripheral ability to discriminate between an original and a distorted version of a polar – circular sinusoidal grating and a face image. Contrast sensitivities as a function of image size were spatially scaleable across eccentricities for both the face and grating. Furthermore, irrespective of stimulus, performance could be scaled with the same individual E₂ value. Thus task simplicity overrides the nature of the stimulus in determining scaling requirements, suggesting that it is the complexity of the task, not of the stimulus, that makes contrast scaling necessary in complex tasks.

In previous work (Campbell et al, 1997 Pattern Recognition 30 555–563) a vision system was developed which is capable of classifying objects in outdoor scenes. The approach involves segmenting the image into regions, obtaining a feature-based description of each region, and then passing this description on to an artificial neural network (ANN) which has been trained to label the region with one of eleven possible object types. The question addressed here is: how important is each of these features to overall performance, both in human and machine vision?

A set of experiments was conducted in which human subjects were trained in the same labelling task as the ANN. The stimuli, each depicting a single image region, were generated from a large database of urban and rural images. The subjects were then tested on both intact and degraded stimuli. The results suggest that certain features are particularly influential in mediating overall labelling performance.

An equivalent experiment was carried out with the ANN. A method is presented which allows individual features to be corrupted in such a way as to simulate the loss of certain forms of visual information. The results, which are broadly similar to those found in the previous experiment, imply that the ANN can provide a useful model of human image region labelling. It is anticipated that the methodology, which draws on both computational and psychophysical techniques, will be of use to other areas of investigation.