0 Acoustic Phonetics1. Jonathan Harrington To appear in the J. Laver & W. Hardcastle (Eds.), The Handbook of Phonetic Sciences. Blackwell. 1 X. Introduction In the production of speech, an acoustic signal is formed when the vocal organs move resulting in a pattern of disturbance to the air-molecules that is propagated outwards in all directions eventually reaching the ear of the listener. Acoustic phonetics is concerned with describing the different kinds of acoustic signal that the movement of the vocal organs gives rise to in the production of speech by male and female speakers across all age groups and in all languages, and under different speaking conditions and varieties of speaking style. Just about every field that is covered in this book needs to make use of some aspect of acoustic phonetics and with the ubiquity of PCs and the freely available software for making spectrograms, for processing speech signals, and for labelling speech data, it is also an area of experimental phonetics that is very readily accessible. Our knowledge of acoustic phonetics is derived from various different kinds of enquiry that can be grouped loosely into three areas that (even more loosely) derive primarily from the contact of phonetics with the disciplines of engineering/electronics, linguistics/phonology, and psychology/cognitive science respectively. 1. The acoustic theory of speech production. These studies make use of an idealized model of the vocal tract in order to predict how different vocal tract shapes and actions contribute to the acoustic signal (Stevens & House, 1955; Fant, 1960). Acoustic theory leads to the prediction that the source signal can be modelled as independent from the filter characteristics of the vocal tract, an idea that is fundamental to acoustic phonetics, to formant-based speech synthesis, and to linear predictive coding which allows formants to be tracked 2 digitally. The discovery that vowel formants can be accurately predicted by reducing the complexities of the vocal tract to a three-parameter, four-tube model (Fant, 1960) was one of the most important scientific breakthroughs in phonetics of the last century. The idea that the relationship between speech production and acoustics is non-linear and that, as predicted by the quantal theory of speech production (Stevens, 1972, 1989), such discontinuities are exploited by languages in building up their sound systems, is founded upon models that relate idealized vocal tracts to the acoustic signal. 2. Linguistic phonetics draws upon articulatory and acoustic phonetics in order to explain why the sounds of languages are shaped the way that they are. The contact with acoustic phonetics is in various forms, one of which (quantal theory) has already been mentioned. Developing models of the distribution of the possible sounds in the world's languages based on acoustic principles, as in the ground-breaking theory of adapative dispersion in Liljencrants & Lindblom (1972) is another. Using the relationship between speech production and acoustics as to explain sound change as misperception and misparsing of the speech signal (Ohala, 1993, Chapter x Ohala, this book) could also be grouped in this area. 3. Variability. The acoustic speech signal carries not only the linguistic structure of the utterance, but also a wealth of information about the speaker (physiology, language and language variety, attitude and emotional state). These are entwined in the acoustic signal in a complex way acoustically both with each other and with background noise that occurs in almost every natural dialogue. Moreover, speech is highly context-dependent. A time slice of an acoustic signal can contain information about context, both segmental (e.g., 3 whether a vowel is surrounded by nasal or oral sounds) and prosodic (e.g., whether the vowel is in a stressed-syllable, in an accented word at the beginning or near the end of a prosodic phrase). Obviously, listeners cope for the most part effortlessly with all these multiple strands of variability. Understanding how they do so (and how they fail to do so in situations of communication difficulty) is one of the main goals of speech perception and its relationship to speech production and the acoustic signal. As in any science, the advances in acoustic phonetics can be linked to technological development. Present day acoustic phonetics more or less began with the invention of the sound spectrograph in the 1940s (Koenig, Dunn, and Lacy, 1946). In the 1950s, the advances in vocal tract modelling and speech synthesis (Dunn, 1950; Lawrence, 1953; Fant, 1953, 1959) and a range of innovative experiments at the Haskins Laboratories (Cooper, Liberman, & Borst, 1951) using synthesis from hand- painted spectrograms provided the technology for carrying out many types of investigation in speech perception. The advances in speech signal processing in the 1960s and 1970s resulted in techniques like cepstral analysis and the linear prediction of speech (Atal & Hanauer, 1971) for source-filter separation and formant tracking. As a result of the further development in computer technology in the last 20-30 years and above all with the need to provide extensive training and testing material for speech technology systems, there are now large-scale acoustic databases, many of them phonetically labelled as well as tools for their analysis (Bird & Harrington, 2001). 4 A recording of the production of speech with a pressure-sensitive microphone shows that there are broadly a few basic kinds of acoustic speech signal that it will be convenient to consider in separate sections in this chapter. • Vowels and vowel-like sounds. Included here are sounds that are produced with periodic vocal fold vibration and a raised velum so that the airstream exits only from the mouth cavity. In these sounds, the waveform is periodic, energy is concentrated in the lower half of the spectrum and formants, due to the resonances of the vocal tract, are prominent. • Fricatives and fricated sounds. These will include e.g., fricatives and the release of oral stops that are produced with a turbulent airstream. If there is no vocal fold vibration, then the waveform is aperiodic; otherwise there is combined aperiodicity and periodicity that stem respectively from two sources at or near the constriction and due to the vibrating vocal folds. I will also include the silence that is clearly visible in oral stop production in this section. • Nasals and nasalized vowels. These are produced with a lowered velum and in most cases with periodic vocal fold vibration. The resulting waveform is, as for vowels, periodic but the lowered velum and excitation of a side- branching cavity causes a set of anti-resonances to be introduced into the signal. These are amongst the most complex sounds in acoustic phonetics. My emphasis will be on describing the acoustic phonetic characteristics of speech sounds drawing upon studies that fall into the three categories described earlier. Since prosody is covered elsewhere in two chapters in this book, my focus will be predominantly on the segmental aspects of speech. 5 X.1. Vowels, vowel-like sounds, and formants X.1.1 The F1 x F2 plane The acoustic theory of speech production has shown how vowels can be modelled as a straight-sided tube closed at one end (to model the closure phase of vocal fold vibration) and open at the lip end. Vowels also have point of greatest narrowing known as a constriction location (Stevens & House, 1955; Ladefoged, 1985) that is analogous to place of articulation in consonants and that divides the tube into a back cavity and a front cavity. As Fant's (1960) nomograms show, varying the constriction location from the front to the back of the tube causes changes predominantly to the first two resonance frequencies. The changes are non-linear which means that there are regions where large changes in the place of articulation, or constriction location, have a negligible effect on the formants (e.g., in the region of the soft-palate) and other regions such as between the hard and soft-palate where a small articulatory change can have dramatic acoustic consequences. Since there are no side-branching resonators – that is since there is only one exit at the mouth as the air is expelled from the lungs – the acoustic structure of a vowel is determined by resonances that, when combined (convolved) with the source signal give rise to formants. The formants are clearly visible in a spectrographic display and they occur on average at intervals of c/2L, where c is the speech of sound and L the length of the vocal tract (Fant, 1973) – that is, at about 1000 Hz intervals for an adult male vocal tract of length 17.5 cm (and with the speed of sound at 35000 cm/s). As far as the relationship between vocal tract shape and formants are concerned, some of the main findings are: 6 • All parts of the vocal cavities have some influence on all formants and each formant is dependent on the entire shape of the complete system (see e.g., Fant, 1973). • A maximally high F2 is associated with a tongue constriction in the palatal region. More forward constrictions produce an increase in F3 and F4 that is due to the shortening of the front tube (Ladefoged, 1985) so that there is a progressive increase first in F2, then in F3, then in F4 as the constriction location shifts forward of the palatal zone. F2 is maximally low when the tongue constriction is in the upper part of the pharynx • A maximally high F1 requires the main constriction to be located just above the larynx and the mouth cavity to be wide open. An increasing constriction in the mouth cavity results in a drop in F1 (see also Lindblom & Sundberg, 1971). • Either a decrease of lip-opening area or an increase of the length of the lip passage produce formant lowering. Lip-protrusion has a marked effect on F3 in front vowels and on F2 in back vowels – see e.g., Lindblom & Sundberg (1971) and Ladefoged & Bladon (1982). The acoustic theory of speech production shows that there is a relationship between phonetic height and F1 and phonetic backness and F2 from which it follows that if vowels are plotted in the plane of the first two formant frequencies with decreasing F1 on the x-axis and decreasing F2 on the y-axis, a shape resembling the vowel quadrilateral emerges. This was first demonstrated by Essner (1947) and Joos (1948) and since then, the F1 x F2 plane has become one of the standard ways of comparing 7 vowel quality in a whole range of studies in linguistic phonetics (Ladefoged, 1971), sociophonetics (Labov, 2001), and in many other fields. Experiments with hand-painted spectrograms using the Pattern Playback system at the Haskins Laboratories showed that vowels of different quality could be accurately identified from synthetic speech that included only the first two, or only the first three formant frequencies (Delattre, Liberman, Cooper & Gerstman, 1955). In the 1970s and 1980s, experimental evidence of a different kind, involving an analysis of the pattern of listeners' confusions between vowels (e.g., Fox, 1985; Kewley-Port & Atal, 1989; Klein et al., 1970; Rakerd & Verbrugge, 1985; Shepard, 1972) showed that perceived judgements of vowel quality depend in some way on the F1 x F2 space. The nature of these experiments varied: in some, listeners were presented with a sequence of three vowels and asked to judge whether the third is more similar to the first or to the second; or listeners might be asked to judge vowel quality in background noise. The pattern of resulting listener vowel confusions can be transformed into a spatial representation using a technique known as multidimensional scaling (Shephard, 1972). Studies have shown that up to six dimensions may be necessary to explain adequately the listeners' pattern of confusion between vowels (e.g., Terbeek, 1977), but also that the two most important dimensions for explaining these confusions are closely correlated with the first two formant frequencies (see also Johnson, 2004 for a discussion of Terbeek's data). These studies are important in showing that the F1 x F2 space, or some auditorily transformed version of it, are the principal dimensions in which listeners judge vowel quality. Moreover, if listener judgements of vowel quality are primarily dependent on the F1 x F2 space, then languages should maximise the distribution between vowels in this space in order that 8 they will be perceptually distinctive and just this has been shown in the computer simulation studies of vowel distributions in Liljencrants & Lindblom (1972). Even in citation-form speech, the formants of a vowel are not horizontal or 'steady-state' but change as a function of time. As discussed in X.1.6, much of this change comes about because preceding and following segments cause deviations away from a so-called vowel target (Lindblom, 1963, Stevens & House, 1963). The vowel target can be thought of as a single time point that in monophthongs typically occurs nearest the vowel's temporal midpoint, or a section of the vowel (again near the temporal midpoint) that shows the smallest degree of spectral change and which is the part of the vowel least influenced by these contextual effects. In speech research, there is no standard method for identifying where the vowel target occurs partly because, as various authors have noted (e.g., Benguerel & McFadden, 1989; Gay, 1978: Nearey & Assmann, 1986) many monophthongal vowels often have no clearly identifiable steady-state or else the steady-state, or interval that changes the least may be different for different formants (e.g., Di Benedetto, 1989). Some researchers (e.g., Broad & Wakita, 1977; Schouton & Pols, 1979a, b) apply a Euclidean-distance metric to the vowel formants to find the least changing section of the vowel while others estimate targets from the time at which the formants reach their maximum or minimum values (Fig. X.1). For example, since a greater mouth opening causes F1 to rise, then when a non-high vowel is surrounded by consonants, F1 generally rises to a maximum near the midpoint (since there is greater vocal tract constriction at the vowel margins) and so the F1-maximum can be taken to be the vowel target (e.g., Stevens, House & Paul, 1966; di Benedetto, 1989; see Son & Pols, 1990 for a detailed comparison of some of the different ways of finding a vowel target). 9 Fig.1 about here X.1.2 F3 and f0 When listeners labelled front vowels from two-formant stimuli in the Pattern Playback experiments at the Haskins Laboratories, Delattre et al. (1952) found that they preferred F2 to be higher than the F2 typically found in the corresponding natural vowels and they reasoned that this was due to the effects of F3. This preferred upwards shift in F2 in synthesising vowels with only two-formants was subsequently quantified in a further set of synthesis and labelling experiments (e.g., Carlson et al., 1970) in which listeners heard the same vowel (a) synthesised with two-formants and (b) synthesised with four-formants and were asked to adjust F2 until (a) was perceptually as close to (b) as possible. The adjusted F2 is sometimes referred to as an effective upper formant or F2-prime (see e.g., Bladon & Fant, 1978; Bladon, 1983; Carlson, Fant & Grantström, 1975; Paliwal, Lindsay & Ainsworth, 1983 for formulae). As discussed in Strange (1999), the influence of F3 on the perception of vowels can be related to studies by Chistovich (1985) and Chistovich & Lublinskaya (1979) showing that listeners integrate auditorily two spectral peaks if their frequencies are within 3-3.5 Bark. Thus in front vowels, listeners tend to integrate F2 and F3 because they are within 3.5 Bark of each other, and this is why in two-formant synthesis, an effective upper formant is preferred which is close to the F2 and F3 average. Based on the experiments by Chistovich referred to above, Syrdal (1985) and Syrdal & Gopal (1986) proposed F3 – F2 in Bark as alternative to F2 as the principal