Foundations of Voice Studies KKrreeiimmaann__ffffiirrss..iinndddd ii 11//1144//22001111 1122::3377::4488 PPMM Foundations of Voice Studies An Interdisciplinary Approach to Voice Production and Perception Jody Kreiman and Diana Sidtis A John Wiley & Sons, Ltd., Publication KKrreeiimmaann__ffffiirrss..iinndddd iiiiii 11//1144//22001111 1122::3377::4488 PPMM This edition fi rst published 2011 © 2011 Jody Kreiman and Diana Sidtis Blackwell Publishing was acquired by John Wiley & Sons in February 2007. Blackwell’s publishing program has been merged with Wiley’s global Scientifi c, Technical, and Medical business to form Wiley-Blackwell. Registered Offi ce John Wiley & Sons Ltd, The Atrium, Southern Gate, Chichester, West Sussex, PO19 8SQ, United Kingdom Editorial Offi ces 350 Main Street, Malden, MA 02148-5020, USA 9600 Garsington Road, Oxford, OX4 2DQ, UK The Atrium, Southern Gate, Chichester, West Sussex, PO19 8SQ, UK For details of our global editorial offi ces, for customer services, and for information about how to apply for permission to reuse the copyright material in this book please see our website at www.wiley.com/wiley-blackwell. The right of Jody Kreiman and Diana Sidtis to be identifi ed as the authors of this work has been asserted in accordance with the UK Copyright, Designs and Patents Act 1988. All rights reserved. No part of this publication may be reproduced, stored in a retrieval system, or transmitted, in any form or by any means, electronic, mechanical, photocopying, recording or otherwise, except as permitted by the UK Copyright, Designs and Patents Act 1988, without the prior permission of the publisher. Wiley also publishes its books in a variety of electronic formats. Some content that appears in print may not be available in electronic books. Designations used by companies to distinguish their products are often claimed as trademarks. All brand names and product names used in this book are trade names, service marks, trademarks or registered trademarks of their respective owners. The publisher is not associated with any product or vendor mentioned in this book. This publication is designed to provide accurate and authoritative information in regard to the subject matter covered. It is sold on the understanding that the publisher is not engaged in rendering professional services. If professional advice or other expert assistance is required, the services of a competent professional should be sought. Library of Congress Cataloging-in-Publication Data Kreiman, Jody. Foundations of voice studies : an interdisciplinary approach to voice production and perception / Jody Kreiman, Diana Sidtis. p. cm. Includes bibliographical references and index. ISBN 978-0-631-22297-2 (hardback) 1. Grammar, Comparative and general–Phonology. 2. Voice–Social aspects. 3. Sociolinguistics. I. Sidtis, Diana. II. Title. P217.3.K66 2011 414–dc22 2010049389 A catalogue record for this book is available from the British Library. This book is published in the following electronic formats: ePDFs [ISBN 9781444395044]; Wiley Online Library [ISBN 9781444395068]; ePub [ISBN 9781444395051] Set in 10/12.5pt ITC Galliard by SPi Publisher Services, Pondicherry, India 1 2011 KKrreeiimmaann__ffffiirrss..iinndddd iivv 11//1144//22001111 1122::3377::4499 PPMM To the memory of Peter Ladefoged, who always encouraged us KKrreeiimmaann__ffffiirrss..iinndddd vv 11//1144//22001111 1122::3377::4499 PPMM Contents 1 Introduction 1 2 Producing a Voice and Controlling Its Sound 25 3 Neurological Foundations of Voice Production and Perception 72 4 Physical Characteristics and the Voice: Can We Hear What a Speaker Looks Like? 110 5 Recognizing Speaker Identity From Voice: Theoretical and Ethological Perspectives and a Psychological Model 156 6 The Brain Behind the Voice: Cerebral Models of Voice Production and Perception 189 7 Identifying Unfamiliar Voices in Forensic Contexts 237 8 Linguistic Uses of Voice Quality: How Voice Signals Linguistic and Pragmatic Aspects of Communication 260 9 Perception of Emotion and Personality from Voice 302 10 Miscellany: Voice in Law Enforcement, Media and Singing 361 References 398 Author Index 470 Subject Index 495 KKrreeiimmaann__ffttoocc..iinndddd vviiii 11//1122//22001111 99::1100::2244 AAMM 1 Introduction 1.1 Why Should We Care About Voice Quality? Whenever we speak, our voices convey information about us as individuals. Speakers may sound young, or tired, or elated, or distracted. They may sound as if they are drunk, or lying, or ill, or bearing secret, exciting news. By their voices, adult speakers usually reveal whether they are male or female, and in addition, they may signal that they come from Texas, or Wisconsin, or France. Over the telephone or radio we may recognize the speaker as someone we know, or we may form a distinct impression of the physical appearance of someone we have never seen. The impressions listeners gain from voices are not necessarily accurate; for example, everyone has known the surprise of meeting a telephone acquaintance who does not match the mental picture we have previously formed of them. Despite such occasional mismatches, voice q uality is one of the primary means by which speakers project their identity – their “physical, psychological, and social characteristics” (Laver, 1980, p. 2) or their “auditory face” (Belin, Fecteau, and Bedard, 2004) – to the world. Table 1.1 non-exhaustively summarizes some of the kinds of judgments that listeners make when listening to voices. These human abilities arise from a long evolutionary process, and many animal species, including primates (Cheney and Seyfarth, 1980), wolves (Goldman, Phillips, and Fentress, 1995), penguins (Jouventin and Aubin, 2002), frogs (Bee, 2004), and bats (Balcombe and McCracken, 1992) use vocal quality to signal or perceive size, threat, and kin relationships. Human infants’ ability to recognize their mothers’ voices is in place at birth (DeCasper and Fifer, 1980), and responses to maternal voices can be measured in utero, suggesting such abilities develop even before birth (Hepper, Scott, and Shahidullah, 1993; Kisilevsky et al., 2003). Voice conveys much of the emotion and attitude communicated by speech (Williams and Stevens, 1972; Banse and Scherer, 1996; Ellgring and Scherer, 1996; Van Lancker and Pachana, 1998; Breitenstein, Van Lancker, and Daum, 2001). Alterations in voice quality relative to the speaker’s normal vocal delivery may signal Foundations of Voice Studies: An Interdisciplinary Approach to Voice Production and Perception, First Edition. Jody Kreiman and Diana Sidtis. © 2011 Jody Kreiman and Diana Sidtis. Published 2011 by Blackwell Publishing Ltd. KKrreeiimmaann__cc0011..iinndddd 11 11//1122//22001111 88::4499::3333 AAMM 2 Introduction Table 1.1 Some kinds of judgments listeners make from voice. Spoken message Physical characteristics of the speaker Age Appearance (height, weight, attractiveness) Dental/oral/nasal status Health status, fatigue Identity Intoxication Race, ethnicity Sex Sexual orientation Smoker/non-smoker Psychological characteristics of the speaker Arousal (relaxed, hurried) Competence Emotional status/mood Intelligence Personality Psychiatric status Stress Truthfulness Social characteristics of the speaker Education Occupation Regional origin Role in conversational setting Social status irony or sarcasm (Van Lancker, Canter, and Terbeek, 1981). Changes in rate and fundamental frequency affect the perceived “competence” (Brown, Strong, and Rencher, 1974) or credibility (Geiselman and Bellezza, 1977) of a speaker. Voice quality provides cues that indicate order of turn-taking in conversation (Schegloff, 1998; Wells and Macfarlane, 1998) and helps resolve sentential ambiguities (Kjelgaard, Titone, and Wingfield, 1999; Schafer, Speer, Warren, and White, 2000). Listeners may also judge the speaker’s sexual preference (Linville, 1998; Munson and Babel, 2007), status as native or nonnative speaker (Piske, MacKay, and Flege, 2001), and a myriad of personality factors (Scherer, 1979) based on voice quality cues. This book describes the manner in which these kinds of information are conveyed to listeners, and how listeners draw conclusions – correctly or incorrectly – about speakers from their voices. Many of the points described are illustrated by recorded examples provided on the accompanying web site. For example, consider the voice in audio sample 1.1. As you listen to this brief speech sample, you will probably automatically gather information about the speaker. Listeners agree that the speaker is female. Although opinions differ, listen- ers are likely to think that the speaker is adult but not elderly, cheerful, confident, KKrreeiimmaann__cc0011..iinndddd 22 11//1122//22001111 88::4499::3333 AAMM Introduction 3 alert, and in good health. She is American, but does not have a pronounced regional, social, or ethnic accent. She sounds average or slightly above average in height and weight. She seems educated and is speaking carefully. She does not sound like a smoker. You probably do not recognize the voice, but it may remind you of someone you know. Compare this talker to the voice in audio sample 1.2. This speaker is also female, but the voice sounds like a much older person. She has a New England accent, and the rhythm of her speech is unusual, making her sound rather upper-class or snobby (or merely self-conscious) to some listeners. She is not tired, depressed, or angry, but she is not obviously happy, either, and may be bored. Her voice is somewhat hoarse, suggesting that she is or has been a smoker, but she does not seem ill. Listeners disa- gree somewhat about her height and weight, but generally estimate that she is average or slightly below average in height, and slightly above average in weight. The voice of a speaker with a vocal pathology is presented in audio sample 1.3. Even this short sample may produce complex impressions of old age, illness, and unattractiveness, along with a sense of the speaker’s emotions or mood, intelligence, and competence. Patients who develop a voice disorder often complain that the dis- ordered voice is not really their voice, and does not convey who they are. In some cases, patients dislike the image they portray so much that they avoid speaking, result- ing in significant social and work-related difficulties. Severe voice quality problems may also interfere with speech intelligibility, creating a handicap in the communica- tion of verbal information (Kempler and Van Lancker, 2002). The strong impressions conveyed by voice quality are often manipulated by the media for multiple purposes. For example, in the classic film Singin’ in the Rain (Freed, Kelly, and Donen, 1952), the shrill, loud voice of the character Lina Lamont (played by actress Jean Hagen) surprises and amuses because it does not fit her appear- ance (a beautiful, smiling blonde) or the elegant, poised, sophisticated personality she visually projects. This contrast – a prototypically silly voice in an elegant physique – forms a running joke throughout the film, playing off such lines as, “What do you think I am, dumb or something?” spoken in the abrasive voice stereotypically associ- ated with a vulgar, uneducated, shrewish female. More often, voices are selected to fit the intended message. Documentary films enhance credibility through the use of a male narrator whose voice carries the stereotype of an authoritative figure who is solid, mature, calm, highly intelligent, and dignified. In the field of advertising, impressions conveyed by voice quality are integral to establishing a product image. Consider the characteristics projected by the voices typically used in advertisements for luxury automobiles. Low pitch, breathy quality, and a fairly rapid speaking rate produce the image of an intimate message from a mature but energetic male who pos- sesses authority, sex appeal, social status, and “coolness.” These vocal attributes are appropriate to the economic niche for the product and imply that its owners are pow- erful, sexy, and affluent. Given the wide range of information listeners derive from voices, it is not surprising that scholars from many different disciplines have studied the production and percep- tion of voice. Table 1.2 lists some of these disciplines, along with a sampling of typical research questions. These research questions encompass much of human existence, and indicate how central voice quality is to human life. KKrreeiimmaann__cc0011..iinndddd 33 11//1122//22001111 88::4499::3333 AAMM 4 Introduction Table 1.2 Disciplines incorporating the study of voice and voice quality. Discipline Some typical research questions Acoustics Deriving reliable and meaningful acoustic measures of voices Animal behavior Vocal recognition of kin and social information by nonhuman animals Biology Biological and evolutionary significance of vocalization Computer science, signal Transmission, measurement, and synthesis of voice processing, information Forensic science, law enforcement Reliability and verification of “earwitness” testimony; assessment of truthfulness from voice Linguistics, phonetics Meanings of vocal quality in speech Medicine: Developmental biology Infant voice recognition Gerontology Voice quality changes in aging Neurology Brain function underlying vocal behaviors Obstetrics Prenatal voice perception Otolaryngology Voice disorders Pediatrics Childrens’ processing of vocal information Physiology Control of phonation Respiration Role of breathing in vocalization Surgery Effects of surgical interventions in the vocal tract on voice; cosmetic changes for transgendered voices Music: Singing The singing voice: many questions Vocal coaching The effects of training on the voice Physics Vibrating laryngeal tissues; relation of vibration to sound; patterns of airflow through the glottis Psychology: Cognitive psychology Speaker recognition and its causes; interaction between speech recognition and voice quality Clinical psychology Detecting depression, psychopathology, and personality in the human voice Social psychology Voices as signals of social relationships including conversational turn taking, sarcasm, and successful con-artistry Neuropsychology Brain mechanisms underlying the perception and production of voice cuing personal identity as well as mood and motivation Psychophysics Relevant acoustic voice features for perception Psycholinguistics Voice information in meaning comprehension for grammatical structure and nonliteral meanings Sociology Voice types associated with social groups and their development Speech science Normal voice and speech production Speech pathology Effects of vocal pathologies on voice quality Theater arts Voice as artistic instrument KKrreeiimmaann__cc0011..iinndddd 44 11//1122//22001111 88::4499::3333 AAMM Introduction 5 1.2 What is Voice? What is Voice Quality? The Definitional Dilemma The terms “voice” and “voice quality” are variously used, sometimes apparently inter- changeably, and deriving consistent definitions has not proven easy. Adding to the confusion, authors also discuss a range of specific voice qualities (a creaky voice, a breathy voice), qualities associated with a speaker’s internal or physical state (a sad voice, a tired voice; a sexy voice), and so on, without benefit of a theoretical frame- work linking all these usages. We attempt to distinguish these meanings usefully by discussing the terms here. Although a clear definition of voice is a prerequisite to its study, the broad range of functions subserved by voice has made it difficult to provide a single, all-purpose definition that is valid and useful across disciplines, scholarly traditions, and research applications. As voice scientist Johann Sundberg has noted (1987), everyone knows what voice is until they try to pin it down, and several senses of the term are in common use. In scientific usage (and throughout this book), the term “voice” has a physical and physiological base that refers to the acoustic signal (as generated by the voice produc- tion system), while “voice quality” refers to the perceptual impression that occurs as a result of that signal, analogous to the distinction between “frequency” (a physical property of vibration) and “pitch” (a listener’s sensation). Definitions of voice fall into two general classes. In the first, voice can be defined very narrowly in physiological terms as “sound produced by vibration of the vocal folds.” Were this definition applied, voice would include only those aspects of the signal that are attributable to the action of the vocal folds, and would exclude the acoustic effects of vocal tract resonances, vocal tract excitation from turbulent noise, or anything else that occurs during speech production other than the action of the vocal folds. (Chapter 2 describes the voice production process in detail.) This definition corresponds approximately to the linguis- tic voicing feature that phonetically distinguishes voiced from voiceless sounds (for example, /s/ from /z/) in many languages. Authors who use the term “voice” in this sense (for example, Brackett, 1971) typically distinguish voice from speech. Voice in this sense is also synonymous with the term “laryngeal source,” which emphasizes the fact that vocal fold vibrations are the acoustic energy source for much of speech. Anatomical constraints make it difficult to study voice as narrowly defined. The larynx is located fairly low in the neck (see Chapter 2), and vocal fold function is dif- ficult to observe directly for long periods of time. Short sequences of open vowel phonation can be inspected through the use of a laryngeal mirror (see Sidebar in Chapter 2). Direct views of some aspects of laryngeal vibrations are available using endoscopic imaging technology and either stroboscopy1 (for example, Hertegard and Gauffin, 1995) or high-speed imaging (for example, Koike and Hirano, 1973; Berry, Montequin, and Tayama, 2001; Deliyski et al., 2008). Laryngeal vibrations can also be studied experimentally using excised larynx preparations (for example, van den Berg, 1968; Berry, 2001). Some authors have used the output of devices like the laryngograph 1 A technique by which rapid vocal fold vibrations are apparently “slowed” through use of a strobe light so that they can be easily viewed. KKrreeiimmaann__cc0011..iinndddd 55 11//1122//22001111 88::4499::3333 AAMM