Body Perception From Connected Speech: Speaker Height Discrimination from Natural Sentences and Sine-Wave Replicas with and without Pitch

Introduction

The human voice, which serves as the raw material for spoken language, not only carries language information but also provides considerable information about the speaker’s physical appearance, personality, and social characteristics. Extensive research has addressed the relationship between the voice and human listeners’ perceptual accuracy of the physical characteristics of the speaker (particularly the body size) (Bricker & Pruzansky, 1976; for a review see Kreiman & Sidtis, 2011). Considering people of different ages and sexes, investigators have observed significant correlations between some acoustic parameters of voice, such as the average fundamental frequency (F0) and formant frequencies, and the speaker’s body size (Fitch & Giedd, 1999). However, among same sexed adults, correlations have been much weaker (González, 2006).

The voice is produced by vocal folds within the larynx, and it is subsequently filtered by the supralaryngeal vocal tract, giving rise to different formants or resonant frequencies (Fant, 1960). The vocal folds vibrate to produce the fundamental frequency (F0), perceived as voice pitch. As an individual grows, the vocal folds also grow, lowering F0. For this reason, adult voices are deeper than children’s voices; and, as women are, on average, shorter than men, their vocal folds are also shorter, making the pitch of a woman’s voice higher on average than that of a man’s voice. However, within the same sex/age group, the length of the vocal cords bears little relation to height. That is, a tall man does not necessarily have longer vocal cords than a short man, and the same goes for women.

However, beyond pitch alone, the formant frequencies seem to be more informative than F0 about the speaker’s body when distinguishing individuals within the same age/sex group. Formants are the resonant frequencies of the vocal tract, and, as the vocal tract length increases, the formant frequencies decrease, and vice versa (Fitch, 2000). The vocal tract is more constrained than the vocal folds by anatomical structures that are closely related to aspects of body size, including the skull or the skeleton and body length. Consequently, moderate negative correlations have usually been observed between formant frequencies and the speaker’s body size (Bruckert et al., 2006; González, 2004; 2006; Pisanski et al., 2014b). On the other hand, correlations between F0 or voice pitch and body size among people of the same sex/age have been found to be very weak or virtually nil, because vocal folds are made of soft tissue (Bruckert et al., 2006; González, 2006; Künzel, 1989; Pisanski et al., 2014b; Van Dommelen & Moxness, 1995).

Researchers have demonstrated that human listeners can roughly infer a speaker’s body height, based on the formant frequencies of the voice (Kreiman & Sidtis, 2011). Unexpectedly, researchers have also revealed that listeners mistakenly infer the speaker’s height based on voice pitch (F0) in that they have exhibited a consistent misattribution bias by tending to associate low F0 with tall people and high F0 with short people (Feinberg, et al., 2005; Pisanski & Rendall, 2011; Pisanski et al., 2014b; Rendall, et al., 2007; Van Dommelen & Moxness, 1995). Likely, this bias results from an overgeneralization of the physical principle that large or long objects produce low-frequency vibrations, whereas small or short objects produce high-frequency vibrations.

Because voice pitch barely correlates with body size, one would expect that adding pitch information to formant information would not increase accuracy in body size perception. However, Pisanski et al. (2014a) found that although men’s voice pitch and physical height were unrelated, the accuracy of the listeners’ size assessments from Canadian vowels increased in the presence versus absence of pitch information.

In the present study, we used sine-wave (SW) replicas of Spanish sentences to provide new unique acoustic information from formant frequencies, and we included a new condition in which we added pitch information. The sine-wave (SW) replicas or sinusoidal speech were first used as experimental stimuli and published in Science by Remez, et al. (1981). Despite the apparently unnatural speech quality in these replicas, researchers have demonstrated that this type of signal is perfectly intelligible for people with normal hearing, and it allows talker identification (Remez et al., 1981; Remez, et al., 1997; Remez, et al., 1994; for a recent review see Remez, 2021). The SW speech consists of time-varying sinusoidal patterns that follow the trajectories of formant frequencies. As they are “pure” sinusoids that follow the paths of the formants, they only carry information about the frequency pattern of the formants (and the amplitude envelope), omitting any other type of information provided by the human speech. To our knowledge, we are the first to use this type of stimulus in an experiment on the perception of the speaker’s body size (height) from listening to the speaker’s voice.

Method

Results

Figure 2 shows the proportions of accurate judgments of speaker height for each stimulus condition (Natural sentences, N; Sine-wave replicas, SW; Sine-wave replicas + Pitch, SWP), separated by the speaker’s sex. Keeping in mind that the chance level for this binary discrimination level was 0.500, every value was significantly higher than chance, except for the SW-males condition (.516; p = .404). Total values indicated that Natural sentences reached the highest level of accuracy, .643, 95% CI [.617, .668], followed by the Sine wave + Pitch condition, .564 [.534, .593], and the Sine wave condition, .559 [.530, .587].OPEN IN VIEWER

A 3 (Stimulus condition: N, SW, SWP) by 2 (Sex of speakers: male, female) ANOVA found a significant main effect of the Stimulus condition, F (2, 174) = 18.19, MS_e = .022, p < .001, η²_p = .173, and a significant main effect of the Sex of speakers, F (1, 87) = 22.05, MS_e = .025, p < .001, η²_p = .202, because the female speakers received higher rates of accurate judgments (.620, 95% CI [0.598, 0.642]) than the male speakers (.556, 95% CI [.535, .577]). The Stimulus condition by Sex of Speaker interaction did not reach statistical significance, F (2, 174) < 1. Within-subject contrasts showed, as expected, that accuracy from the Natural sentences (N) was significantly higher than accuracy from the other conditions, F (1, 87) = 23.66, MS_e = .026, p < .0001, η²_p = .214. However, the within-subject contrast did not find a significant difference between the scores from SW versus SWP conditions, F (1, 87) < 1. Similarly, this difference (SW vs. SWP) did not reach significance for only male speakers, F (1, 87) = 1.267, MS_e = .052, I = .263, η²_p = .014, or only female speakers, F (1, 87) < 1.

Discussion

Our data clearly suggested that the formant information contained in the sine-wave replicas (SW) of natural sentences were useful for listeners who attempted to judge the speakers’ body size at an above chance level in a binary discrimination task. Nevertheless, the level of accuracy listeners reached did not equal the score obtained from listening to natural sentences, indicating that, beyond the formants, natural speech contains additional information useful for discriminating the speaker’s height. González and Oliver (2004) found that actual height was the strongest predictor of perceptual judgments from a spoken sentence in both male and female speakers, suggesting that there may be other acoustic predictors of body size beyond the pitch and the formants. In the González and Oliver (2004) study, the partial correlation between actual height and judgments of “taller” in a binary discrimination task reached 0.51 for male stimuli after removing the influence of F0 and F1–F4. On the other hand, adding information about the voice pitch of each sentence in our experiment did not help increase the level of accuracy.

Our findings also revealed that stimuli recorded from the female speakers were more informative for judging the speakers’ body size than the stimuli recorded from male speakers. As mentioned above, we acoustically analyzed the stimuli used in our experiment, attending particularly to the natural sentences that served as the basis for creating the other two experimental conditions. Data in the Appendix, separated by speakers’ sex, includes the speakers’ heights and the acoustic parameters corresponding to the means of F0 (fundamental frequency), F1 (first formant), F2 (second formant), and F3 (third formant). We also, included the average frequency of the three formants, and the dispersion between formants [((F2 − F1) + (F3 − F2))/2]. The correlations within the same sex group between speakers’ heights and speech parameters were generally null or moderate, consistent with what has been found by other investigators (Barsties, et al., 2016; González, 2004; 2006; Owren, 2011; Pawelec, et al., 2020; for a review see Pisanski et al., 2014b). The highest correlations with speakers’ heights corresponded to the second formant (F2, r = −.306, accounting for 9.4% of the variance or r²) for male speakers, the second formant (F2, r = −.472, accounting for 22.3% of the variance), and the formant average (r = −.355; accounting for 12.6% of the variance) for female speakers. The fact that the correlations between speakers’ heights and formant frequencies were somewhat higher for female speakers, especially for the formant average, might explain why the participant assessments were more accurate for female stimuli. Note that part of the superiority of the height-formant correlation for perceiving female speakers could be because female heights were more variable than male heights. Pisanski et al. (2014b) conducted a large meta-analysis of vocal indicators of body size in adult men and women, including 295 correlation coefficients derived from 39 independent samples across participants in five different continents. They concluded that (a) formant-based estimates of vocal tract length explained about 10% of height within sexes and (b) voice pitch (F0) explained only 2% of the variance in predicting men’s height and 0.5% of the variance in predicting women’s height. Our stimuli yielded a correlation of r = −0.209 (accounting for 4.4% of the variance) between height and voice pitch (F0) for male samples, and a null correlation (r = −0.003; with no variance predicted) for female samples. This difference could explain why, in the case of the male stimuli, the SW replicas + pitch gave a slightly higher accuracy (.543) than the SW replicas alone (.516), although this small increase was nominal.

Conclusion

We employed sine-wave replicas of sentences to study the perception of body height from human voices for the first time. This type of stimuli allows a rigorous experimental control since it provides information solely and exclusively on the formants (resonances of the vocal tract) without any other information from the speech signal spectrum. Our results show that human listeners can perceive body height from this information alone and that adding vocal pitch does not significantly increase accuracy. Most previous studies used intact samples of vowels or syllables (usually in English), but we used connected speech in Spanish. Connected speech is more than the sum of a series of phonemes since it comprises a dynamic and coarticulated succession of speech sounds forming syllables as units of articulation, particularly in Spanish (González-Alvarez & Sos-Peña, 2022). The time-varying trajectories of the vocal formants might operate as sources of inference about the articulator apparatus’s biomechanical properties—mass, mobility, and inertial characteristics.