Learned Predictiveness Speeds Visual Processing

Abstract

When humans learn that the presence of a cue predicts the likelihood of an outcome, they can exploit this learned predictiveness, such that formation of subsequent associations between that cue and new outcomes is facilitated. Could such enhanced selection for association arise early enough to facilitate low-level visual processing? In a test of this possibility, adult volunteers first engaged in a value-learning task involving faces that were differentially predictive of monetary wins or losses. Later, in a simple recognition task, these faces were briefly presented for a variable duration and then masked. The critical presentation duration needed to produce criterion-level recognition was measured to index the visual processing speed for each learned face. Critical duration was significantly shorter for stimuli with high learned predictiveness than for stimuli with low learned predictiveness, regardless of whether they were associated with wins or losses. These results show that neural mechanisms involved in predicting future outcomes are able to modulate visual processing efficiency, probably via cortical feedback processes.

Keywords

value learning reward faces visual perception motivation

A primary function of the human brain is to accurately predict important future events. To do this, the brain needs to be able to associate a consequential event (e.g., new e-mail) with the sensory information, or cues (e.g., a computer “ping”), available just previous to the event’s occurrence. Such an association enables the prediction of that event should that cue be encountered again. After successive encounters with a cue, event predictions can be compared with actual events to assess whether the prediction afforded by the cue is accurate and, if so, how accurate it is (Sutton & Barto, 1990). During learning, error monitoring allows stimuli that are informative about an outcome to become strongly associated with it, and stimuli that are uninformative to lose associative strength (Schultz & Dickinson, 2000). Such monitoring can thus be used to prune out associations involving weakly predictive cues.

A problem for this type of learning is that the number of potential cues available just prior to a new event is typically vast, yet the mechanism needed to associate cues with subsequent events appears to be highly limited in capacity (Pearce & Hall, 1980; Wagner, 1978). If new learning is to proceed efficiently, these capacity limitations should be accommodated by a selection mechanism that can prioritize potential cues for association learning (Mackintosh, 1975). Studies of learned predictiveness support this possibility. These studies demonstrate that when some sensory cues are experienced as good predictors of outcomes and others are experienced as poor predictors of outcomes in one context, the good predictors are subsequently more readily associated with new outcomes or contingencies than the poor predictors are (Le Pelley & McLaren, 2003; Livesey & McLaren, 2007). A possible explanation for this effect is that learning the predictiveness of a cue (i.e., the difference in likelihood of an outcome when the cue is present vs. not present) enables the acquisition of a durable predictiveness code that becomes active when the cue is encountered again, even if the context is new. If sufficiently strong, this code can boost the cue’s ability to compete for association with a new outcome.

An unexplored question is how early in cue processing learned predictiveness can exert its influence. This influence could occur late, after cues are fully encoded (postperceptual processing), or early, as soon as the cues’ presence is detected (pre- perceptual processing). Dense interconnectivity between learning networks and visual processing networks (see O’Doherty, 2004, for a review) and perceptual learning effects (Roelfsema, van Ooyen, & Watanabe, 2010) support the latter possibility. One way to more directly test whether learned predictiveness can exert an early influence is to vary cues’ exposure duration to determine the minimum duration needed for recognition. If learned predictiveness influences encoding, then this minimum duration should decrease as learned predictiveness increases. If it does not, then minimum duration should be uninfluenced by prior learned predictiveness. Previously, we found that predictiveness associated with stimuli during a learning task facilitated their subsequent selection and recognition when viewed in rapid serial visual presentation displays (Raymond & O’Brien, 2009). However, that study did not vary the cues’ exposure duration, so it remains unknown whether the observed effects reflected modulation of visual encoding or postperceptual (e.g., long-term memory) processes.

To test whether learned predictiveness modulates visual encoding, we devised a two-phase experiment. In an initial conditioning phase, we imbued different face stimuli with high or low predictiveness of a win or a loss, in a fully crossed design. On each trial, participants selected one face from a pair, in an effort to maximize their monetary winnings. For some face pairs (win pairs), choice resulted in either a win or no outcome; for others (loss pairs), choice led to either a loss or no outcome. In each of these pairs, one face produced a win or a loss with a probability of .8, and the other face produced the same outcome with a probability of .2. For a third, control pair type, choice never produced an outcome. In the second phase of the experiment, participants performed a simple visual recognition task in which target faces (previously learned or novel) were presented for a variable but brief interval and then immediately backward masked. The task was to indicate whether each target face was “old” (presented in the learning task) or “new.”

It is widely held that target onset initiates a rapid feed-forward signal that carries information from V1 to high-level perceptual processing areas, which then feed back to early visual areas in an iterative fashion to selectively boost processing of relevant information (e.g., Friston, 2005). Masking disrupts this feedback (Di Lollo, Enns, & Rensink, 2000; Fahrenfort, Scholte, & Lamme, 2007; Lamme, Zipser, & Spekreijse, 2002) to an extent dependent on the stimulus onset asynchrony (SOA) between the target and mask. Thus, the shortest SOA (referred to here as the critical duration) yielding a criterion level of perceptual performance is thought to reflect the minimum time needed for cortical processing of the stimulus (Loffler, Gordon, Wilkinson, Goren, & Wilson, 2005). If neural codes for stimulus predictiveness modulate visual encoding processes, then the critical duration should be briefer for faces associated with high predictiveness than for faces associated with low or no predictiveness. If these codes act postperceptually, then critical duration should be invariant with respect to learned predictiveness.

Method

Participants

Seventeen experimentally naive members of the Bangor University community participated (12 females, 5 males; mean age = 22 years; normal or corrected-to-normal vision); all achieved criterion performance on the learning task (i.e., > 65% correct on at least one win and one loss pair). Data from 4 participants were excluded because of their excessively poor performance on the face recognition task (mean d′ < 1.0 and more than 2 SD below the mean) when faces were presented for the maximum duration (94 ms). Informed consent was obtained prior to participation, and participants were compensated with money and course credit.

Apparatus

A Pentium 4 computer, running E-Prime 1.0 (Schneider, Eshman, & Zuccolotto, 2002), recorded data and presented stimuli on a 51-cm monitor (85-Hz refresh, resolution of 1024 × 768 pixels). The viewing distance was approximately 70 cm.

Stimuli

The face stimuli (Fig. 1a) were static computer-generated (GenHead 1.2; Genemation, Inc., Manchester, England), gray-scale faces of young adult males (hair, teeth, and neck not visible). Each face subtended approximately 2.9° × 3.6°, and all faces were highly similar in contrast and brightness. Twenty different masks were created by dividing a digital face image (not otherwise used) into a 5 × 4 grid and then rearranging the pieces to create a non-facelike image while preserving the outer shape of the face.

Fig. 1.

Examples of stimulus pairings in the learning task (a) and trial sequence for the recognition task (b). From left to right, the face pairs in (a) illustrate win, loss, and control (no-outcome) pairs. The probability of a monetary outcome for each choice is shown; asterisks (not seen by participants) indicate the optimal choices. In the recognition task (b), participants judged whether the target face had been presented in the learning task or was new.

Procedure

Value learning

On each trial, two faces were presented, one above and one below a central fixation cross (Fig. 1a). After the participant chose a face (by pressing one of two designated keys), the screen immediately displayed the word “WIN” in green (accompanied by a “bing” sound), the word “LOSS” in red (accompanied by a “bong” sound), or the word “NOTHING” in black (no sound), depending on the face pair just presented and the probability governing the outcome. A running total of earnings also appeared. Each face always appeared with its mate, but their position (top vs. bottom) was randomized from trial to trial. On win and loss trials, the monetary outcome (win or loss of 5 pence) occurred with a probability of .8 or .2; no outcome was the default. Each pair (two win, two loss, and two control pairs) was presented 100 times in a self-paced, random order. Assignment of each face pair to outcome type (win, loss, or control) was counterbalanced across participants to eliminate image effects. Participants were instructed merely to choose the face in each trial that would maximize their payoff; they kept their winnings at the end of the experiment.

Recognition task

A few minutes after completing the value-learning task, participants began the face recognition task, after a short practice session (24 trials). Each experimental trial (Fig. 1b) began with a 1,000-ms central fixation cross, and then a face was presented in the center of the screen for 24, 35, 47, 71, or 94 ms. The face was immediately replaced with a scrambled face mask for 212 ms, and then a blank screen appeared until response. On half of the 480 trials, the target face was randomly selected from the 12 faces used in the value-learning task; on the other half, it was randomly selected from a set of 24 novel faces. Participants were instructed to report whether the target face was “old” (previously seen in the learning task) or “new” (not seen in the learning task) as accurately as possible. Responses were unspeeded, and trials were self-paced. Each face from the value-learning task was presented eight times for each duration.

Data analysis

Successful learning of a face pair was defined as choosing the optimal face on more than 65% of the last 30 trials for that pair. Recognition data for unsuccessfully learned pairs were excluded from subsequent analysis. Recognition performance was quantified using d ′ (calculated for each participant and condition), calculated as the difference in the Z-transformed probability of a hit (reporting “old” when the stimulus was old) and the Z-transformed probability of a false alarm (reporting “old” when the stimulus was new). (Probability values of .0 and 1.0 were converted to .01 and .99, respectively.) These d′ values were analyzed using a repeated measures analysis of variance with stimulus duration, prior valence (win, loss), and prior predictiveness (high, low) as within-subjects factors. Planned paired-samples, two-tailed t tests were used to compare means. Alpha levels were set at .05. To estimate the critical duration needed for criterion face recognition, we fit least squares lines to plots relating group mean d ′ value to stimulus duration for each combination of prior valence and prior predictiveness. The duration needed to yield a d′ of 1.5 (about 90% correct given the mean false alarm rate of .41) was then interpolated. Standard errors of estimate were calculated for each line and used to compare interpolated critical-duration values.

Results

Learning

At the end of the learning phase, performance approached asymptote, and learning approximated the outcome contingencies similarly for win and loss pairs, p > .20. For win pairs, the face with the high probability of a win was chosen on average on 75% (SE = 4%) of the last 30 relevant trials; for loss pairs, the face with the low probability of a loss was chosen on 70% (SE = 3%) of the last 30 relevant trials.

Face recognition

As expected, recognition performance improved as stimulus duration increased, F(4, 48) = 44.23, p < .001, η _p ² = .787. Stimuli with high prior predictiveness were recognized better than stimuli with low predictiveness, F(1, 12) = 11.37, p < .01, η _p ² = .487, and, critically, this benefit depended significantly on stimulus duration, F(4, 48) = 3.45, p < .02, η _p ² = .224 (Fig. 2). Stimuli associated with low predictiveness were not recognized at better than chance levels until stimulus duration was longer than 47 ms, whereas stimuli associated with high predictiveness were recognized at better than chance levels with only a 35-ms exposure. Learned valence associations (prior valence: win vs. loss) did not have a significant main effect on performance (F < 1) and did not interact with either prior predictiveness or stimulus duration (both Fs < 1). Note that if recognition had been better for faces that had previously been optimal choices (i.e., the high-predictiveness faces in win pairs and the low-predictiveness faces in loss pairs), an interaction of valence and predictiveness should have been found.

Fig. 2.

Group mean d′ values for face targets as a function of stimulus duration and prior learning. Results are shown for faces that had previously predicted a win (heavy black lines) and that had previously predicted a loss (gray lines) with high (solid lines) and low (dashed lines) probabilities, as well as for faces from the no-outcome, control condition. Error bars indicate ±1 SE. (SEs were similar in size for all conditions and are shown here for one condition only for clarity.)

The critical durations interpolated from the data are shown in Figure 3. For stimuli associated with high predictiveness, the critical duration was 63 ms (equivalent for win- and loss-associated targets), 23 ms shorter than the critical duration for stimuli associated with no outcome (86 ms, p < .001). Critical durations for stimuli associated with high predictiveness were also significantly shorter than those for both win- and loss-associated low-predictiveness stimuli (both ps < .001; the advantage was 27 ms for win-associated stimuli and 19 ms for loss-associated stimuli).

Fig. 3.

Critical duration (i.e., minimum stimulus duration needed to yield a d′ recognition score of 1.5) for target faces as a function of their previous predictiveness (highly predictive of a win or a loss, weakly predictive of a win or a loss, or predictive of no monetary outcome). Error bars indicate ±1 standard error of estimate.

Discussion

We have shown that learned predictiveness can determine the stimulus duration required to produce a criterion level of recognition in a simple backward-masking paradigm. Stimuli experienced previously as highly predictive of monetary outcomes could be recognized with 23 ms less exposure than other, similar stimuli previously experienced as nonpredictive of valuable outcomes, regardless of the valence (win or loss) of the associated outcome. This finding indicates that the neural mechanisms sensitive to the probability of outcomes in a previous context are able to selectively enhance fundamental perceptual processing of associated stimuli when they arise in a different context. Furthermore, these results support the idea that such effects occur early in visual processing.

Two possible explanations for how these early effects could occur are (a) that learning alters mechanisms analyzing incoming visual information (during a feed-forward sweep) or (b) that learning modulates the cortical feedback used to selectively boost perceptual representations. The former view, derived from studies of perceptual learning, posits that during value learning, rewarding outcomes induce neural plasticity (Schultz, 2002) that enables feedback from frontal cortical areas to fine-tune sensitivity of recently active neurons in visual cortex (Roelfsema et al., 2010). Thus, stimuli associated with rewards become more efficiently processed during the feed-forward sweep. This view could account for our findings if plasticity were to depend on outcome probability, not outcome valence. Alternatively, as suggested by the second possibility, neural codes for learned predictiveness could modulate reentrant feedback from prefrontal areas acting on visual cortex, boosting processing efficiency on the fly. Supporting this possibility is substantial evidence that areas of prefrontal cortex are specifically activated when cues that predict outcome probability are presented (Knutson, Taylor, Kaufman, Peterson, & Glover, 2005; Yacubian et al., 2006). A plausible means by which such neural activity could assert influence over visual efficiency, as reported here, could be via feedback to visual cortex (Haynes, Driver, & Rees, 2005). Although our simple behavioral study does not preferentially support either possibility (or indeed a combination of both; Vecera & O’Reilly, 2000), we have shown clearly that prior value learning can have a sustained effect on the efficiency of visual processing.

The primary finding of interest in our study is that learned predictiveness, not valence or prior optimality, modulates the speed of visual processing in a masked recognition task. In studies of human and animal learning, predictiveness has been shown to enhance subsequent stimulus associability (e.g., Livesey & McLaren, 2007), presumably by facilitating selection of stimuli for access to a limited-capacity associator mechanism. Although we did not measure subsequent associability of the previously learned stimuli in our experiment, our findings of enhanced visual processing efficiency raise the possibility that selection for association begins very early in visual processing.

Footnotes

The authors declared that they had no conflicts of interest with respect to their authorship or the publication of this article.

This project was supported by the Biotechnology and Biological Sciences Research Council (United Kingdom), Grant BBS/B/16178.

References

Di Lollo

Enns

Rensink

. (2000). Competition for consciousness among visual events: The psychophysics of reentrant visual processes. Journal of Experimental Psychology: General, 129, 481–507.

Fahrenfort

Scholte

Lamme

(2007). Masking disrupts reentrant processing in human visual cortex. Journal of Cognitive Neuroscience, 19, 1488–1497.

Friston

(2005). A theory of cortical responses. Philosophical Transactions of the Royal Society B: Biological Sciences, 360, 815–836.

Haynes

J. D.

Driver

Rees

(2005). Visibility reflects dynamic changes of effective connectivity between V1 and fusiform cortex. Neuron, 46, 811–821.

Knutson

Taylor

Kaufman

Peterson

Glover

(2005). Distributed neural representations of expected value. Journal of Neuroscience, 25, 4806–4812.

Lamme

Zipser

Spekreijse

(2002). Masking interrupts figure-ground signals in V1. Journal of Cognitive Neuroscience, 14, 1044–1053.

Le Pelley

M. E.

McLaren

I. P. L.

(2003). Learned associability and associative change in human casual learning. The Quarterly Journal of Experimental Psychology B, 56, 68–79.

Livesey

E. J.

McLaren

I. P. L.

(2007). Elemental associability changes in human discrimination learning. Journal of Experimental Psychology: Learning, Memory, and Cognition, 33, 148–159.

Loffler

Gordon

Wilkinson

Goren

Wilson

(2005). Configural masking of faces: Evidence for high-level interactions in face perception. Vision Research, 45, 2287–2297.

10.

Mackintosh

N. J.

(1975). A theory of attention: Variations in the associability of stimuli with reinforcement. Psychological Review, 82, 276–298.

11.

O’Doherty

(2004). Reward representations and reward-related learning in the human brain: Insights from neuroimaging. Current Opinion in Neurobiology, 14, 769–776.

12.

Pearce

J. M.

Hall

(1980). A model for Pavlovian learning: Variations in the effectiveness of conditioned but not of unconditioned stimuli. Psychological Review, 87, 532–552.

13.

Raymond

J. E.

O’Brien

J. L.

(2009). Selective visual attention and motivation: The consequences of value learning in an attentional blink task. Psychological Science, 20, 981–988.

14.

Roelfsema

P. R.

van Ooyen

Watanabe

(2010). Perceptual learning rules based on reinforcers and attention. Trends in Cognitive Sciences, 14, 64–71.

15.

Schneider

Eshman

Zuccolotto

(2002). E-Prime user’s guide. Pittsburgh, PA: Psychological Software Tools.

16.

Schultz

(2002). Getting formal with dopamine and reward. Neuron, 36, 241–263.

17.

Schultz

Dickinson

(2000). Neuronal coding of prediction errors. Annual Review of Neuroscience, 23, 473–500.

18.

Sutton

R. S.

Barto

R. G.

(1990). Time derivative models of Pavlovian reinforcement. In Gabriel

Moore

(Eds.), Learning and computational neuroscience: Foundations of adaptive networks (pp. 497–537). Cambridge, MA: MIT Press.

19.

Vecera

S. P.

O’Reilly

R. C.

(2000). Graded effects in hierarchical figure-ground organization: Reply to Peterson (1999). Journal of Experimental Psychology: Human Perception and Performance, 26, 1221–1231.

20.

Wagner

A. R.

(1978). Expectancies and the priming of STM. In Hulse

S. H.

Fowler

Honig

W. K.

(Eds.), Cognitive processes in animal behavior (pp. 177–209). Hillsdale, NJ: Erlbaum.

21.

Yacubian

Gläscher

Schroeder

Sommer

Braus

D. F.

Büchel

(2006). Dissociable systems for gain- and loss-related value predictions and errors of prediction in the human brain. Journal of Neuroscience, 26, 9530–9537.