Abstract
The visual system adapts to the environment, changing neural responses to aid efficiency and improve perception. However, these changes sometimes lead to negative consequences: If neurons at later processing stages fail to account for adaptation at earlier stages, perceptual errors result, including common visual illusions. These negative effects of adaptation have been termed the coding catastrophe. How does the visual system resolve them? We hypothesized that higher-level adaptation can correct errors arising from the coding catastrophe by changing what appears normal, a common form of adaptation across domains. Observers (N = 15) viewed flickering checkerboards that caused a normal face to appear distorted. We tested whether the visual system can adapt to this adaptation-distorted face through repeated viewing. Results from two experiments show that such meta-adaptation does occur and that it makes the distorted face gradually appear more normal. Meta-adaptation may be a general strategy to correct negative consequences of low-level adaptation.
When our environment changes, our sensory systems change in tandem to keep us perceiving well. In the visual system, this adaptation produces changes in neural response at early stages of processing that are large, common, and likely improve the efficiency of encoding (for reviews, see Clifford et al., 2007; Kohn, 2007; Webster, 2015). Nevertheless, adaptation can also lead to misperceptions of the world as a result of what is known as the coding catastrophe (Fairhall, Lewen, Bialek, & de Ruyter van Steveninck, 2001; Schwartz, Hsu, & Dayan, 2007). Both perceptual and neurophysiological work have shown that adaptation arising in early visual areas propagates along the visual stream, changing responses in later stages to a fixed stimulus (for examples, see Dhruv & Carandini, 2014; Patterson, Wissig, & Kohn, 2014; Xu, Dayan, Lipkin, & Qian, 2008). These altered responses are sometimes interpreted as arising from a change in the stimulus rather than a change in the neural response pattern (or code), which produces incorrect, “illusory” percepts (Schwartz et al., 2007).
For example, a classic illusory misperception— the tilt aftereffect—is produced when an observer adapts to a high-contrast pattern of tilted stripes, for example, oriented at 20°, and is subsequently presented with a physically vertical pattern, which then appears to be tilted away from the adaptor pattern (Gibson & Radner, 1937). Adaptation to high-contrast patterns is known to cause a reduction in neural responsiveness in the primary visual cortex (V1; e.g., Fang, Murray, Kersten, & He, 2005; Movshon & Lennie, 1979), particularly for neurons that prefer orientations similar to that of the adaptor. This adaptation causes presentation of the vertical pattern to produce a distribution of responses across neurons in early visual cortex that is changed from the distribution in the unadapted state; in this example, the center of mass of the neural response distribution would be shifted away from 20° following adaptation. The accepted interpretation of the tilt aftereffect is that neurons in later visual areas respond to this altered distribution of input from V1 as if it were generated by a physical stimulus that was shifted away from 20°, producing the illusory tilt (Fairhall et al., 2001; Schwartz et al., 2007).
Adaptation can also occur at later processing stages. Many high-level percepts encoded later in processing change following adaptation, including the shape of a face (Webster & Maclin, 1999), the specific viewpoint for observing an object (Fang & He, 2005), and even the gender of a hand (Kovács, Zimmer, Bankó Harza, Antal, & Vidnyánszky, 2006). Given feedback connections in cortex and uncertainty regarding the neural bases of perceptual measures, strictly defining earlier and later stages of processing can be challenging. Here, we refer to later stages of processing as those that receive feed-forward input from earlier stages and lower-level and higher-level adaptation as changes in response properties of neurons at earlier and later stages of processing, respectively. When direct neural evidence is lacking, investigators take advantage of the hierarchical nature of the visual pathways and define lower-level adaptation as that affecting representations of features and higher-level adaptation as that affecting representations that integrate those features (e.g., see Xu et al., 2008).
Some higher-level adaptation can be explained as shifts in perceptual norms: For example, prolonged exposure to a masculine face causes a physically gender-neutral face to appear more feminine (Webster, Kaping, Mizokami, & Duhamel, 2004). Norm-based encoding accounts for this effect by proposing that adapting to a masculine face shifts a norm for faces toward male faces. This shift, in turn, causes the previously neutral face to appear more feminine, because it is encoded relative to the norm. Norm-based codes are also believed to play an important role in color perception (Webster & Leonard, 2008), perception of blur (Elliott, Georgeson, & Webster, 2011), body-identity perception (Rhodes, Jeffery, Boeing, & Calder, 2013), and other perceptual qualities.
Can this higher-level adaptation help overcome the negative consequences of lower-level adaptation? We hypothesized that norm-based encoding may help resolve the coding catastrophe if adaptation propagates along the visual stream and later-stage visual areas adapt to the altered input received from earlier stages. Whereas past work has made clear that lower-level adaptation can influence processing at later stages (e.g. Dhruv & Carandini, 2014; Patterson et al., 2014), the extent to which later areas can adapt to the altered input they receive remains widely unexplored (though for one example, see Webster & Mollon, 1995).
We tested our hypothesis by inducing lower-level adaptation that made a physically normal face appear distorted. We expected repeated viewing of this distorted face to produce adaptation at later stages of face processing, an example of what we term meta-adaptation. The meta-adaptation should shift perceptual norms toward the adapter—that is, toward the distorted face—and so cause it to appear more normal (Fig. 1). Our results supported our hypothesis, documenting meta-adaptation, showing its buildup over time, and advancing understanding of how adaptation at different levels can work together to optimize behavior (Webster & Mollon, 1995, 1997).

The meta-adaptation paradigm. To produce lower-level adaptation, we placed high-contrast checkerboard patterns at locations near the eyes of a subsequently presented undistorted face (a). Note that the face beneath the checkerboard was not presented during adaptation; it is shown here to illustrate the relative retinal locations of the checkerboard and the subsequently presented face. As shown in (b), this lower-level adaptation (black single-headed arrow) caused the eyes of the subsequently presented face to be represented as shifted outward, farther apart than normal, because of changes in neuronal responses at early stages of the visual system (black tick marks along upper double-headed arrow). The low-level representation, altered by adaptation, is inherited by later stages of processing (vertical dashed arrow). Meta-adaptation is higher-level adaptation driven by this altered input (gray single-headed arrow). Because the higher-level adaptation shifts the perceptual norm (gray tick marks along lower double-headed arrow), the face should appear to be more normal as it is viewed for a prolonged duration, correcting the effects of the coding catastrophe. The face images in (b) represent the content of lower-level and higher-level visual-processing stages.
Experiment 1
Method
Participants
Fifteen observers participated in Experiment 1. Sample size was determined from the prior literature on face adaptation (e.g., Leopold, O’Toole, Vetter, & Blanz, 2001; Webster et al., 2004), as well as an unpublished pilot version of the experiment. Data collection stopped when the sample size reached 15. The experimental procedures were approved by the University of Minnesota Institutional Review Board (IRB 1006M84457) and conformed to the principles of the Declaration of Helsinki.
Stimuli
To produce lower-level adaptation that distorted a face, we presented observers with a contrast-reversing checkerboard pattern and a familiar face, in interleaved fashion (Fig. 1a). The checkerboard patterns were placed near the eye position of the familiar face to produce shifts in the perceived location of the eyes, making them appear farther away from each other. The patches of checkerboard pattern were 0.5° tall and wide and were presented at 95% Michelson contrast, with checks alternating between positive and negative contrast at 8 Hz. Observers’ viewing distance was maintained at 52 cm.
The familiar face used in this experiment was of former U.S. president Barack Obama; the image subtended 15° by 9.6° of visual angle and was displayed at the center of a video screen. We created test images with intereye distances that differed by steps of 0.23° of visual angle. Mean luminance was 42 candela/m2, and stimuli were delivered in MATLAB (The MathWorks, Natick, MA) using the Psychophysics Toolbox (Brainard, 1997; Kleiner, Brainard, & Pelli, 2007; Pelli, 1997).
Procedure
The observers’ task consisted of judging a test face image centered 1.87° above fixation. Observers judged whether the distance between this face’s eyes was narrower or wider than that of an undistorted face image. Test faces were presented for 0.3 s. The position of the eyes of the test face was controlled by a one-up, one-down staircase procedure based on the observers’ response, which converged to an intereye distance that was equally likely to be judged as too narrow or too wide. We estimated this point of subjective equality (PSE) by fitting psychometric functions to the responses, as described below.
Observers participated in up to two practice sessions before the main experiment. The practice sessions began with familiarization to the undistorted face; it was shown for 0.5 s, followed by a 1-s presentation of a uniform mean-gray screen. This familiarization process continued for 60 s. During this time, observers were instructed to closely examine the face’s features.
Next, observers practiced the behavioral task in three training blocks, each consisting of 30 trials. Practice-session results were used to determine whether the observer had internalized an unbiased and consistent estimate of the intereye distance of the undistorted face. Observers were invited to participate in the main experiment if their PSE across blocks varied on average by one level of the intereye distance shift used to create the stimuli (i.e., the variation was smaller than 0.23° of visual angle). Otherwise, they completed an additional practice session.
In the main session, we measured observers’ perceptual norm for a test face prior to and following meta-adaptation. In preadaptation trials, which were identical to the practice sessions, observers saw a test face above the center of the screen for 0.3 s and had 2 s to determine whether the face had narrower or wider intereye distance than a normal face. Each observer completed 30 trials.
The main session began with 2 min of contrast adaptation, in which observers viewed the contrast-reversing checkerboard patterns that were placed near fixation. This contrast adaptation caused the eyes of the undistorted face to appear to be shifted outward, away from the contrast-reversing pattern. The likely mechanism for this is that adaptation caused a decrease in responsiveness in the neurons that represent the adapted location, which resulted in an overall shift in the neural-population response distribution to the eyes (i.e., the amount of neural response as a function of receptive-field location) away from the adaptor (Dhruv & Carandini, 2014). This, in turn, caused the perceived location of the eyes to shift away from the adapted location.
After the initial 2-min contrast adaptation, observers viewed 0.3 s of the physically undistorted face (which appeared distorted because of the contrast-adaptation aftereffect), followed by 6-s top-up contrast adaptation (Fig. 2). This checkerboard/face top-up cycle repeated 15 times before the first test-face presentation in order to attempt to produce strong initial meta-adaptation to the distorted face. To ensure that observers were attending to the face presented, we programmed one eye of the face occasionally wink (movement in the eyelid). Observers were instructed to press a button when they detected this movement.

Procedures of Experiment 1. Observers initially viewed 2 min of a contrast-reversing checkerboard pattern. Patches to the left and right of fixation alternated, each appearing for 1 s. After 2 min, a normal face was shown at the center of the screen for 300 ms, followed by 6 s of top-up adaptation to the checkerboards. This top-up checkerboard/face cycle was repeated six times (over 38 s); then a test face was shown in the upper visual field, where its eye positions were relatively unaffected by adaptation to the checkerboards. Observers judged whether the intereye distance in the test face was wider or narrower than in a normal face. After a 2-s response period, the experiment continued with six additional top-up checkerboard/face cycles (another 38 s) before the next test face was presented. The experiment comprised 30 test-face presentations.
After the 15th checkerboard/face cycle, observers again judged a test face. The test face was presented above fixation to ensure that it was not affected by the low-level contrast adaptation, which should be tied to its retinotopic location. Each test presentation was followed by six additional repetitions of the checkerboard/face top-up cycle. This continued for 30 trials, and we again used a staircase procedure to estimate PSE. A pilot experiment showed that this procedure could measure strong norm shifts when observers viewed physically distorted faces (i.e., traditional face adaptation; see Experiment S2 in the Supplemental Material available online). The main session lasted approximately 45 min.
Analysis
To estimate PSEs, we fitted a psychometric function (a Weibull function) to observers’ responses, which has the following general form:
where y is the fraction of “too narrow” responses, x is the physical intereye distance, and α and β are free parameters estimated by the fitting procedure. We used a maximum-likelihood fitting, and the PSE was estimated as the physical intereye distance that produced 50% “too narrow” responses in the fit function.
Results
To produce adaptation at early stages of processing, we presented observers with contrast-reversing checkerboard patterns placed at visual-field locations just central to where the eyes of our test face would appear (Figs. 1 and 2). Adaptation to contrast patterns such as these is known to produce effects in early visual cortex (e.g., Movshon & Lennie, 1979). Perceptually, adaptation to the checkerboard produced a repulsive spatial aftereffect (Dhruv & Carandini, 2014), causing the eyes of a subsequently presented face to appear shifted outward. A pilot experiment verified that this paradigm produced perceptual distortions that were specific to the adapted retinal locations, a sign of their arising relatively early in visual processing (see Experiment S1 in the Supplemental Material).
Can higher levels of the visual system adapt to this face distorted by lower-level adaptation? We measured higher-level face adaptation using a standard method, asking observers to judge the appearance of a face placed at a different location in the visual field from a repeatedly viewed adapting face. The spatial transfer, as well as other past work, indicates that this paradigm depends on adaptation at later stages of processing (e.g., Leopold et al., 2001). We measured observers’ PSE for eye separation, which is the point at which the face appeared to have eyes neither too wide nor too close together. We report PSEs as percentage-change scores relative to the true normal intereye distance (computed by first subtracting and then dividing the raw scores by 1.5°, the distance in the undistorted image).
Face perception showed clear evidence of adaptation following exposure to the face distorted by low-level adaptation. Observers’ mean PSE shifted significantly following exposure to the distorted face, compared with a baseline condition measured prior to viewing, t(14) = 2.72, p = .008, one-tailed, 95% confidence interval (CI) for the mean difference = [3.91%, +∞%], d = 0.70. Figure 3 plots individual observers’ PSEs before and after exposure to the distorted face, as well as the group mean. The mean PSE increased by about 6%, and 12 of 15 observers showed trends in the predicted direction (i.e., toward a face with eyes physically farther apart).

Results from Experiment 1. Points of subjective equality (PSEs) of individual observers (circles) are plotted with the group mean (diamond), both before and after adaptation. Error bars represent standard errors of the mean difference score (as appropriate for a within-subjects design; Loftus & Masson, 1994). PSEs are given as the percentage-change score between each observer’s response and the face’s true normal intereye distance. The dotted line represents 0% change in PSE (i.e., the physical intereye distance of the undistorted face). Positive change corresponds to a shift of the PSE to a face with wider intereye distance, and negative change corresponds to a shift of the PSE to a face with narrower intereye distance.
The PSE measured the physical face whose eyes appeared to be the “correct” distance apart, which is one definition of a perceptual norm. This norm shifted toward faces with eyes farther apart, the same direction as the shift that the lower-level adaptation produced. We conclude that the norm shift made the faces distorted by lower-level adaptation appear more normal; that is, it effectively corrected some of the distortions in appearance produced by the low-level adaptation.
Confirming this interpretation, many observers verbally reported that the physically undistorted face they saw at fixation, after adapting to the contrast patterns, initially appeared quite wide-eyed and then later appeared more normal. Some believed that the face physically changed over successive trials.
Experiment 2
The goal of Experiment 2 was to measure this change in appearance over time. Experiment 1 was optimized to measure the basic meta-adaptation effect and so began with almost 2 min of exposure to the distorted face before the norm task was first performed. This was meant to strengthen the high-level adaptation effect but may have prevented observation of its buildup, because adaptation can sometimes reach asymptote after a few minutes. Experiment 2 eliminated the initial checkerboard/face cycles that were presented before data collection began. All other experimental procedures were identical to those in Experiment 1.
Method
Participants
Fifteen observers participated in Experiment 2; 2 of them also participated in Experiment 1. We intentionally matched the sample size to that of Experiment 1. Data collection stopped when the sample size reached 15. The experimental procedures were approved by the University of Minnesota Institutional Review Board (IRB 1006M84457) and conformed to the principles of the Declaration of Helsinki.
Procedure
Trials were identical to those in Experiment 1; observers again indicated whether a test face presented for 300 ms centered in the upper visual field (1.87° above center fixation) had narrower or wider intereye distance than a normal face. Eye distance of the test face was controlled by the same one-up, one-down staircase procedure. Observers completed 30 baseline trials and then began the adaptation protocol, which began with 2 min of initial contrast adaptation. Test trials then immediately began, and alternated with six repetitions of the checkerboard/face cycle. Observers again completed a total of 30 trials.
Analysis
In an initial analysis, data were pooled over the entire time course, identically to how the analysis of Experiment 1 was conducted. To analyze the time course of the meta-adaptation effect, we divided data into four nonoverlapping time bins. There were not enough trials in each bin to allow for fitting of psychometric functions, so we simply averaged the presented stimuli (the staircase levels) within the bin as a simple estimate of observers’ PSEs. We then tested for a linear trend in the PSEs across bins. To confirm that simple averaging of the staircase levels is a reasonable estimate of PSE, we performed a Monte Carlo simulation of our experiment with a model observer responding with PSEs that increased linearly over time. The PSEs obtained from the simple estimate fell on a line that lay close to the model observer’s PSEs and did not reliably differ from them.
Results
An initial analysis pooled data across the entire time course, as in Experiment 1, and showed a similar meta-adaptation effect. Following adaptation, observers’ mean PSE again shifted significantly toward a wider intereye distance than in the baseline condition, t(14) = 2.11, p = .027, one-tailed, 95% CI for the mean difference = [3.81%, +∞%], d = 0.55. Figure 4 plots PSEs before and after meta-adaptation, again as percentage change relative to the intereye distance of a physically normal face (1.5°). The mean PSE increased by about 6%, comparable with the change measured in Experiment 1.

Results from Experiment 2. Points of subjective equality (PSEs) of individual observers (circles) are plotted with the group mean (diamond), both before and after adaptation. Error bars represent standard errors of the mean difference score. PSEs are given as the percentage-change score between each observer’s response and the face’s true normal intereye distance. The dotted line represents 0% change in PSE (i.e., the physical intereye distance of the undistorted face). Positive change corresponds to a shift of the PSE to a face with wider intereye distance, and negative change corresponds to a shift of the PSE to a face with narrower intereye distance.
The meta-adaptation effect grew gradually over time. We divided the data into four bins based on testing time. Figure 5 plots estimates of PSE in successive bins (see Method). A linear-trend analysis revealed a significant upward slope, t(13) = 8.75, p < .001, one-tailed, 95% CI for the slope = [3.12, +∞]. We interpret this trend to indicate that repeatedly viewing the faces distorted by low-level adaptation caused PSEs to shift gradually toward faces with wider intereye distances.

Time-course results from Experiment 2. Points of subjective equality (PSEs) of individual observers (circles) are plotted with the group mean (diamond) in each of four time bins. Error bars represent ±1 SEM. PSEs are given as the percentage-change score between each observer’s response and the face’s true normal intereye distance. The dotted line represents 0% change in PSE (i.e., the physical intereye distance of the undistorted face). Positive change corresponds to a shift of the PSE to a face with wider intereye distance, and negative change corresponds to a shift of the PSE to a face with narrower intereye distance.
Discussion
Prolonged exposure to a normal face distorted by contrast adaptation produced robust changes in observers’ face perception. That is, their visual systems adapted to distortions produced by adaptation. This meta-adaptation effect was reasonably large (ds > 0.5 in two independent experiments) and grew gradually stronger over time. Changes in face perception were measured as changes in the face that appeared normal. These perceptual-norm shifts were in the same direction as the distortions produced by the lower-level adaptation, leading the distorted face to appear more normal.
Our mechanistic interpretation of these results is that contrast adaptation affected early stages of visual processing; when a face was presented, signals modified by this lower-level adaptation were transmitted to later processing stages responsible for face perception, initially resulting in the percept of a distorted face. Over time, the later stages adapted to this altered input, shifting perceptual norms so that the percept of the face became more normal (Fig. 1).
Our results and interpretation agree with previous reports showing that adaptation at early stages of visual processing can propagate downstream to later stages (Dhruv & Carandini, 2014; Patterson et al., 2014; Xu et al., 2008). Webster and Mollon (1995) suggested that later stages can themselves adapt to the altered input arriving from adapted earlier stages in a way that is beneficial for color constancy.
Our results build on this past work, defining the concept of meta-adaptation, extending it to face perception, and showing that it can reduce illusory aftereffects produced by low-level adaptation (i.e., that it can mitigate the coding catastrophe). It is presumably beneficial because it allows earlier stages to take advantage of the coding-efficiency gains that result from adaptation (Brenner, Bialek, & de Ruyter van Steveninck, 2000; Clifford et al., 2007; Wainwright, 1999; Wark, Lundstrom, & Fairhall, 2007) while reducing its potential negative consequences for later stages.
Note that the norm shifts we observed cannot be due to spread of the low-level adaptation to the location where we measured perceptual norms. The presence of low-level adaptation, which caused the eyes to appear farther apart, during norm measurement would cause faces with narrower intereye distances to appear normal. Our results were in the opposite direction. The meta-adaptation phenomenon we observed also may not hold for test faces with identities that differ from that of the adapting face. This is likely because past work has established that our visual system maintains distinct perceptual norms for faces of different identities (Jaquet, Rhodes, & Hayward, 2008; Little, DeBruine, Jones, & Waitt, 2008). Future work can test whether this is indeed the case.
Perceptual learning is another mechanism that may allow the visual system to overcome the coding catastrophe (Haak, Fast, Bao, Lee, & Engel, 2014; McGovern, Roach, & Webb, 2012). Past work has shown, for example, that training can allow observers to accurately judge moving stimuli despite the presence of an ongoing motion aftereffect (McGovern et al., 2012). Learning may also allow observers to switch rapidly back and forth between adaptive states, which can minimize negative aftereffects of adaptation (Engel, Wilkins, Mand, Helwig, & Allen, 2016; Yehezkel, Sagi, Sterkin, Belkin, & Polat, 2010). Perceptual learning, however, typically requires multiple days of experience and is often specific to the practiced task. Meta-adaptation may act more rapidly, generally, and automatically.
Probably the primary way the visual system copes with the coding catastrophe is to recover quickly from adaptation. Most demonstrations of adaptation’s negative consequences rely on short-lived aftereffects that are visible once the adapting stimulus is removed. We have used such a paradigm here—the distorted face appeared only after the checkerboard pattern was removed. Aftereffects are generally used to isolate effects of adaptation from potential spatial interactions between the adapter and the test stimulus—in the present case, perceptual distortions that could have arisen if the checkerboards and eyes had been presented simultaneously. Importantly, in natural viewing, the adapting stimulus may remain present continuously, requiring a solution such as meta-adaptation.
Meta-adaptation likely operates in domains other than faces. In principle, it could be used to aid perception of any feature that uses norm-based encoding; these include color, blur, and body perception (Elliott et al., 2011; Rhodes et al., 2013; Webster & Leonard, 2008). Consider, for example, viewing an object in the presence of a high-spatial-frequency adapter (e.g., a house in the rain, though spatial interactions will be present in addition to adaptation). Adaptation should reduce the response to high frequencies and make the object appear blurry. Meta-adaptation could, in turn, alter a high-level norm for blur, making the object appear more normal. Future work will determine how commonly vision and other senses use meta-adaptation to reduce effects of the coding catastrophe.
Supplemental Material
Liu_OpenPracticesDisclosure_rev – Supplemental material for Higher-Level Meta-Adaptation Mitigates Visual Distortions Produced by Lower-Level Adaptation
Supplemental material, Liu_OpenPracticesDisclosure_rev for Higher-Level Meta-Adaptation Mitigates Visual Distortions Produced by Lower-Level Adaptation by Xinyu Liu and Stephen A. Engel in Psychological Science
Supplemental Material
Liu_Supplemental_Material_rev – Supplemental material for Higher-Level Meta-Adaptation Mitigates Visual Distortions Produced by Lower-Level Adaptation
Supplemental material, Liu_Supplemental_Material_rev for Higher-Level Meta-Adaptation Mitigates Visual Distortions Produced by Lower-Level Adaptation by Xinyu Liu and Stephen A. Engel in Psychological Science
Footnotes
Acknowledgements
We thank two anonymous reviewers for suggesting Experiment 2.
Transparency
Action Editor: Alice J. O’Toole
Editor: D. Stephen Lindsay
Author Contributions
Both authors developed the study concept and designed the experiments. Testing and data collection were performed by X. Liu. X. Liu analyzed and interpreted the data under the supervision of S. A. Engel. Both authors drafted the manuscript and approved the final version for submission.
References
Supplementary Material
Please find the following supplemental material available below.
For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.
For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.
