Infants Are Not Sensitive to Synesthetic Cross-Modality Correspondences

Werner (1940) believed that people are born with a syncretic unity of the senses, and Spector and Maurer (2009) proposed that synesthesia is characteristic of the neonatal brain because of exuberant anatomical connectivity. The neonatal synesthesia hypothesis is reasonable, given that young infants can perceive various types of nonsynesthetic audiovisual relations (Bremner, Lewkowicz, & Spence, 2012; Lewkowicz & Lickliter, 1994), including intensity (Lewkowicz & Turkewitz, 1980), duration (Lewkowicz, 1986), and temporal synchrony (Lewkowicz, 1992, 2010). Consequently, Walker et al. (2010) put the neonatal synesthesia hypothesis to empirical test by investigating whether preverbal infants can perceive pitch-height and pitch-shape cross-modal correspondences. They found that 4-month-old infants looked longer at an animation of a ball moving up and down when they heard a sliding whistle sound correspondingly increasing and decreasing in pitch (congruent condition) than when they heard the whistle sound correspondingly decreasing and increasing in pitch (incongruent condition). Similarly, they found that 4-month-old infants looked longer at an animated object changing its corners from rounded to pointy and back to rounded when pitch correspondingly increased and then decreased than they did when pitch correspondingly decreased and then increased. These results prompted Walker et al. to conclude that synesthesia is characteristic of the neonatal brain and that it is an unlearned aspect of human perception.

Here, we report a failure to replicate the findings of Walker et al. and show that they reflect infants’ response to pitch-loudness interactions rather than synesthesia. As Figure 1a shows, the pitch of the sliding whistle sound presented by Walker et al. during the first phase of the animation increased between 300 and 1,700 Hz, while its loudness increased and then decreased between 54 and 80 dB, peaking when pitch was midrange. During the second phase of the animation, the pitch of the sound decreased from 1,700 to 300 Hz, and its loudness again increased and then decreased between 54 and 80 dB, peaking when pitch was midrange.

OPEN IN VIEWER

Fig. 1.

Pitch and loudness characteristics of the sliding whistle sound in the congruent trials of (a) the pitch-height experiment in the Walker et al. (2010) study and (b) Experiment 1 in the current study. In the Walker et al. study, an animated ball (represented here by a white circle) moved up and down continuously while the pitch of a sliding whistle sound rose and fell continuously. In addition, during each motion of the ball from one end of the display to the other, pitch changes were accompanied by loudness changes. Loudness first rose until pitch was midrange and then fell back to its original level. Similar to the procedure used by Walker et al., in the current study, a moving ball was accompanied by pitch changes. In contrast, however, loudness increased as the ball rose and until pitch reached 1,000 Hz. Loudness then remained constant while pitch continued to increase and the ball moved to the edge of the display. Pitch then decreased as the ball descended, but loudness remained constant until pitch dropped back down to 1,000 Hz, at which point loudness began to decrease until it reached its initial level.

The initial increasing pitch and increasing loudness profile of the sound during the congruent animations was perceptually more salient than the initial decreasing pitch and increasing loudness profile of the sound during the incongruent animations. Consequently, the former animation probably recruited greater initial attention than did the latter animation. In our view, the differential attentional salience of the sound profiles in the congruent and incongruent animations produced the preference for the congruent event in the Walker et al. study. Our conclusion is supported by findings indicating that differential recruitment of initial attention during a test of intersensory perception can affect infant responsiveness to audiovisual relations (Lewkowicz, 1992).

We conducted five experiments designed to replicate and expand on the findings of Walker et al., as well as to test our differential-attentional-salience hypothesis. In Experiment 1, we investigated pitch-height matching across three interleaved congruent and incongruent test trials in separate groups of 4-, 6-, and 8-month-old infants (ns = 23, 23, and 22, respectively). We used the same basic testing method as Walker et al. did and also presented an auditory stimulus that varied in pitch. In our experiment, however, we reduced the accompanying loudness cues in the manner illustrated in Figure 1b. Specifically, loudness increased only from 55.5 to 74.5 dB and only when pitch increased between 300 and 1,000 Hz. After that, as pitch increased to 1,700 Hz and then decreased back down to 1,000 Hz, loudness remained constant at 74.5 dB. At this point, as pitch continued to decrease, loudness decreased back to 55.5 dB. The visual stimulus was a red ball moving up and down at 10 cm per second in front of a 20 × 20 grid of small white dots against a black background on a 19-in. screen. The ball paused for 50 ms at each trajectory endpoint. Results indicated that infants did not look longer at the congruent event than at the incongruent event at any age. ¹ In Experiment 2, we tested pitch-height matching again, except that we varied pitch and visual motion at twice the rate and tested groups of 4-, 6-, 8-, 10-, and 12-month-old infants (ns = 21, 25, 21, 25, and 24, respectively). Again, results showed that infants did not prefer the congruent event at any age and that at 6 months, infants actually showed a significant preference for the incongruent event (p < .05).

In Experiments 3 and 4, we investigated responsiveness to pitch-sharpness correspondences with identical testing procedures as Walker et al. used and with similar stimuli (i.e., our changing-pitch sound and a gray star whose corners constantly morphed between rounded and sharp). Results from Experiment 3, in which separate groups of 4- and 8-month-old infants (ns = 16 and 10, respectively) participated, indicated no preference for the congruent event. Because the infants in Experiment 3 had participated in a prior but unrelated experiment, we conducted Experiment 4 to determine whether this might have somehow affected performance. Thus, we repeated Experiment 3 with a new cohort of 4-month-old infants (N = 19), for whom this was their first experiment. Once again, results yielded no preference for the congruent event.

Finally, in Experiment 5, we attempted a systematic replication of the Walker et al. study by using an intersensory paired-preference procedure to investigate pitch-height matching. Infants saw two identical red balls, moving at the faster rate used in Experiment 2, on each of two side-by-side 19-in. monitors separated by a 5-in. space. Each 30-s trial started with one ball at the bottom of one monitor and with the other ball at the top of the other monitor. During the first two trials, the balls moved in silence, and during the second two trials, the balls’ motion was accompanied by our changing-pitch sound. The results from separate groups of 4-, 6-, 8-, 10-, 12-, and 14-month-old infants (ns = 32, 28, 21, 18, 24, and 24, respectively) indicated that infants did not look longer at the congruent moving ball in the presence of the changing-pitch sound than in its absence at any age.

In sum, using both the same and different methods and employing stimuli nearly identical to those used by Walker et al., we found no evidence that infants perceived the link between auditory pitch and visuospatial height or visual sharpness. Critically, our failure to replicate the findings of Walker et al. cannot be due to slight procedural differences (see note 1), to a lack of statistical power (we tested many more infants and at multiple ages), or to repeated testing of some infants at different ages. ² Thus, our failure to replicate must be due to the reduction of the initial attentional salience of the congruent event. If so, the Walker et al. findings do not reflect infants’ perception of pitch-height and pitch-shape relations but, rather, interacting pitch and loudness cues. This, in turn, suggests that, contrary to Walker et al.’s claim, synesthesia is not an inherent characteristic of the neonatal brain nor an innate aspect of perception.