Language and Speech

An unavoidable problem in speech technology, particularly in the development of robust automatic speech recognition systems, is the extreme variability in the acoustic attributes of segments. Segments are highly sensitive to context and bear little resemblance to their intrinsic characteristics manifested when they are uttered in isolation. However, the problem can become tractable if we model the linguistic and physiological aspects of coarticulatory processes, the main source of systematic variability at the segmental level.

Many sound patterns in languages are cases of fossilized coarticulation, that is, synchronic or phonetic contextual variation became diachronic or phonological variation via sound change. An examination of languages' phonologies can therefore yield insights into the mechanisms of coarticulation. In this paper I discuss (a) the need to differentiate between phonological processes that are and are not due to coarticulation, (b) the need to differentiate between ‘on-line’ synchronic variation and comparable fossilized diachronic variation, (c) how to determine some of the constraints on coarticulation – especially the higher priority of maintaining acoustic-auditory, rather than articulatory, norms for the shape of speech elements, and (d) how coarticulation presents a “parsing” problem to the listener and, of course, to systems for automatic speech recognition.

In this article, we consider the concepts of coordination and coarticulation in speech production in the context of a task-dynamic model. Coordination reflects the transient establishment of constrained relationships among articulators that jointly produce linguistically significant actions of the vocal tract – that is phonetic gestures – in a flexible, context-sensitive manner. We ascribe the need for these constraints in part to the requirement of coarticulatory overlap in speech production. Coarticulation reflects temporally staggered activation of coordinative constraints for different phonetic gestures. We suggest that the anticipatory coarticulatory field for a gesture is more limited than look-ahead models have suggested, consistent with the idea that anticipatory coarticulation is the onset of activation of coordinative constraints for a forthcoming gesture. Finally, we ascribe much of the context-sensitivity in the anticipatory or carryover fields of a gesture (variation due to “coarticulation resistance”) to low-level (below the speech plan) interactions among the coordinative constraints for temporally overlapping gestures.

In this paper we draw on a linguistic model of prosodic structure and a task-dynamic model of speech gestures to account for the interplay of coarticulation and stress in English. We reinterpret results from two experiments in which articulator movements were recorded for utterances varying in pitch accent placement. In the first experiment, jaw kinematics were studied in post-nuclear unaccented and nuclear accented [pap] syllables. The kinematic patterns suggested that gestures in syllables with greater stress (nuclear accented) show less coarticulatory overlap. By contrast, the vowel's low jaw target is undershot in unaccented syllables. Two hypotheses are possible. Either the jaw is lower in stressed syllables so more energy can radiate from the mouth (“sonority expansion”) or the jaw is lower to help distinguish the low vowel from other vowels (“hyperarticulation”). Another experiment differentiates the two hypotheses by examining tongue point positions in [put] preceding a [Ö]. In the more stressed syllables, the tongue dorsum retracts more, likely to make a more distinct back vowel. Also, the amount of assimilation of the alveolar stop to the following dental is reduced. Both results suggest hyperarticulation rather than sonority expansion. Thus, it seems that coarticulation is reduced in stressed syllables, because stressed syllables are hyperarticulated.

Electropalatographic data for Catalan and Italian reported in this paper reveal the existence of two categories of palatal consonants, namely, alveolopalatals ([n], [λ]) and palatals proper ([j]). All these consonants are produced with a single place of articulation and thus are not good candidates for complex segments involving a tongue front articulator and a tongue dorsum articulator. A higher degree of coupling between the primary articulator and other tongue regions for alveolopalatals and palatals than for alveolar [n] accounts for a reduced sensitivity to coarticulatory effects for the former vs. the latter. Alveolarpalatal correlations reported in this study support the notion of relative independence between different tongue articulators for non-dorsal vs. dorsal consonants. Differences in articulation and coarticulation were found for Italian vs. Catalan. In comparison with their Catalan counterparts, Italian shows the following properties: Consonants are more anterior, [n] allows less coarticulation at the alveolar zone (in line with the laminal nature of the consonant), and long alveolopalatals exhibit more contact and less coarticulation at the front palatal zone.

The principal aim of this investigation was to compare coarticulatory effects at different levels of the speech production system, in order to gain insight into the relations between the different levels. To this end, the relative magnitudes of carryover and anticipatory coarticulation with adjacent vowels were measured at the midpoints of the two lingual fricatives /s/ and /∫/ in two speakers each of English, French, and German. Linguopalatal contact patterns derived from electropalatographic recordings were compared with an analysis of the acoustic output. The results indicated, firstly, that mismatches between articulatory and acoustic results are not uncommon. Secondly, and more surprisingly, while there was no difference in the overall magnitude of coarticulatory effects for /s/ and /∫/, not all speakers showed a predominance of the same coarticulatory direction on both fricatives; this complicated the observed tendency for the predominance of carryover coarticulation to be greater in German and English than in French. Two speakers were retested using comparative analyses of electropalatography and electromagnetic articulography. These two procedures gave a closely parallel picture of lingual coarticulatory regularities (while complementing each other in terms of characterizing articulation). The implications of these results for identifying language-specific coarticulatory regularities are discussed.

Temporal and spatial aspects of lingual coarticulation in /kl/ clusters in intervocalic position (VklV) were investigated in six European languages: Catalan, English, French, German, Italian, and Swedish. Three speakers of each language repeated a set of real words and nonsense items five times. Temporal overlap of /k/ and /l/ gestures, as represented by a numerical “overlap index”, was interpreted as tongue-tip/tongue-body coarticulation, and was measured from electropalatographic and airflow records. Results revealed several language-specific features. For example, the tendency to overlap was different for the six languages. All the languages except Swedish showed some instances of overlap, and Catalan showed the strongest tendency. The vowel environment also had a significant effect: The presence of an open back vowel in V₁ and V₂ positions increased the tendency for overlap to occur. However, there was no statistically significant difference in overlap between nonsense and real words. Spatial information from EPG data showed that articulatory placement of the tongue-body gesture was more retracted in a /kl/ cluster compared to a singleton consonant. All the languages under investigation showed this phenomenon. These findings are discussed in terms of possible constraints operating between the tip/blade and tongue-body systems.

The study of the spatio-temporal interactions between contiguous segments in speech is a means for a better understanding of how segments are serially organized. This research explores the relation between a vowel and the following consonant by studying the anticipatory C-to-V coarticulatory effects by means of electropalatography. The materials were ‘VCV utterances produced in isolation and in connected speech by three Italian speakers, with /a/ and /i/ as vowels and the coronals /t, d, l, z, ∫/ as intervocalic consonants. The results show that the consonants affect both the vocal tract configuration of the preceding vowel and its acoustic duration. The spatial effects increase from laterals to stops to fricatives. The tongue body position is raised during /a/ and lowered during /i/. The effects are much larger for /a/ than for /i/ and larger in connected speech than in isolated words. As for temporal coarticulatory effects, the data indicate that vowels tend to be shorter before /∫/ than before /z/, and shorter before /t/ than before /d/ than before /l/. Spatial and temporal measurements of change in tongue body contact from vowel to the consonantal closure/constriction suggest that the consonants differ among each other in the dynamics as well as in the timing of their gestures, with ampler/longer movements (e.g., for /∫/) starting earlier than smaller/shorter movements (e.g., for /d/ or /l/). These patterns result in smaller differences between the durations of the total VC sequences than between the individual durations of V or C segments, and suggest that intersegmental organization between vowels and following consonants may have the rhythmic function of reducing the variability of vowel-to-vowel temporal intervals.

The contextual effects of voiced/voiceless stops on the voice source of an adjacent vowel were examined for the first vowel in ‘CVCV utterances in German, English, Swedish, French, and Italian. The principal analysis technique involved interactive inverse filtering and parameterisation of the glottal waveform in terms of a four-parameter voice source model (the LF-model). This analysis procedure was supplemented by measures from narrowband spectral sections of the speech output and by oral airflow recordings which allow inferences about the relative timing of glottal and supraglottal gestures. Results indicated that the voiced/voiceless nature of the consonant does yield differences in the voice source of the vowel. The most striking effects were found in the context of voiceless consonants, and cross-language differences did emerge in terms of directionality and degree. Extensive anticipatory effects were found for Swedish and for some speakers of English. Preceding the voiceless stop the vowel becomes increasingly breathy-voiced, and it would appear that the glottal abduction gesture is anticipated very early in the course of the vowel. Italian exhibited a similar tendency, though to a considerably lesser degree. The German data, on the other hand, showed certain strong carryover effects: Following the voiceless aspirated stop there was extensive breathy voicing. French showed little contextual variation in either direction. Rather surprisingly, the observed effects were not directly correlated with, or predictable from, the phonetic categories involved (voiced, voiceless unaspirated, and voiceless postaspirated). These results yield insights into the control parameters which may be involved in regulating voicing oppositions in these languages. Whereas the anticipatory effects observed might be consistent with a “timing” model of glottal control, the carryover effects cannot be explained in terms of timing alone and suggest that differences in tension settings of the laryngeal musculature may also be implicated.

In this paper we demonstrate the feasibility and usefulness of articulation-based approaches in two major areas of speech technology: speech recognition and speech synthesis. Our articulatory recognition model estimates probabilities of categories of manner and place of articulation, which establish the articulatory feature vector. The transformation from the articulatory level to the symbolic level is performed by hidden Markov models or multi-layer perceptrons. Evaluations show that the articulatory approach is a good basis for speaker-independent and speaker-adaptive speech recognition. We are now working on a more realistic articulatory model for speech recognition. An algorithm based on an analysis by synthesis model maps the acoustic signal to 10 articulatory parameters which describe the position of the articulators. EMA (electro-magnetic articulograph) measurements recorded at the University of Munich provide good initial estimates of tongue coordinates. In order to improve articulatory speech synthesis we investigated an accurate physical model for the generation of the glottal source with the aid of a numerical simulation. This model takes into account nonlinear vortical flow and its interaction with soundwaves. The simulation results can be used to improve the articulatory synthesis model developed by Ishizaka and Flanagan (1972).