Fang Liu
Department of Speech, Hearing and Phonetic Sciences
University College London
2 Wakefield Street
London WC1N 1PF

Research Associate
Office: Room 309, Chandler House
Phone: +44 (0)20 7679 4093
Fax: +44 (0)20 7679 4238

Speech prosody in tone versus non-tonal languages

In collaboration with Yi Xu, I have been investigating the phonetics and phonology of speech prosody in tone (e.g., Mandarin) versus non-tonal languages (e.g., English) using speech production and perception experiments and computational modeling techniques. Our main findings include:

1. Linguistic functions such as lexical tone (in Mandarin), word stress (in English), focus (sentence emphasis), and sentence type (statement vs. question) are encoded in parallel via language-specific schemes by specifying the pitch target, pitch range, articulatory strength, and duration of each syllable in spoken utterances (Liu, 2009; Liu and Xu, 2005; Liu et al., 2013b).

2. For both tone and non-tonal languages, underlying pitch targets (e.g., high, mid, low, rising, falling) are prosodic primitives in speech, realized through a process of syllable-synchronised target approximation (Xu et al., in press).

3. The neutral tone in Mandarin is not targetless. Rather, it is likely to have a [mid] pitch target plus a weak articulatory strength. The same is true of unstressed syllables in English. Such a weak strength can be simulated for both languages through computational modeling (Liu et al., 2013b; Prom-on et al., 2012).

4. Focus in Mandarin consists of both on-focus pitch range expansion and post-focus pitch range compression, which is the same as in English. While post-focus pitch range is lowered in both statements and yes-no questions in Mandarin, it is lowered in statements but raised in yes-no questions in English (Liu and Xu, 2005; Liu et al., 2013b).

5. The syllable is the basic temporal organization structure that assigns time intervals to both segmental and laryngeal units (Xu and Liu, 2006, 2007). This explains the nature of coarticulation, time intervals of segments, and temporal alignments of segmental and tonal events. It also unites segmental and suprasegmental aspects of speech, treating them as following the same basic articulatory dynamics (Xu and Liu, 2012).

Pitch processing in typical and atypical populations

Since 2008, I have been investigating pitch processing in speech and music in individuals with congenital amusia, a neurodevelopmental disorder of musical processing, in collaboration with Cunmei Jiang (Shanghai Normal University, China), Aniruddh D. Patel (Tufts University, USA), Peter Pfordresher (University at Buffalo, USA), Lauren Stewart (Goldsmiths, University of London, UK), Bill Thompson (Macquarie University, Australia), and Yufang Yang (Chinese Academy of Sciences, China). Given that individuals with congenital amusia can neither sing in tune nor detect an out-of-tune note in a melody, but rarely report language problems, it has been hypothesized that musical pitch processing involves different mechanisms than linguistic pitch processing in congenital amusia. However, our recent studies indicate that individuals with congenital amusia are also impaired in speech intonation processing (Liu et al., 2010), lexical tone perception (Liu et al., 2012a), speech comprehension in both quiet and noise (Liu et al., 2015a), and speech and song imitation abilities relative to normal controls in laboratory conditions (Liu et al., 2013a). In addition, pitch-processing deficits in congenital amusia appear to be associated with insensitivity to the direction of pitch movement in language and music (Liu et al., 2012b), with pitch thresholds of individuals with congenital amusia significantly dependent on task demands (Williamson et al., 2012). Together, our findings suggest that congenital amusia is a domain-general pitch-processing deficit, and that pitch processing in language and music involves shared mechanisms.

In collaboration with Cunmei Jiang, I have also started examining pitch processing in speech and music in individuals with Autism Spectrum Disorders (ASD). We recently conducted the first study to investigate processing of speech intonation and melodic contour in Mandarin-speaking individuals with high-functioning autism (HFA). Results indicate that Mandarin-speaking individuals with ASD demonstrated inferior prosodic processing but superior musical processing compared to matched neurotypical controls (Jiang et al., 2015), which is in direct contrast to the findings of linguistic (relatively intact) and musical (severely impaired) processing in congenital amusia.

Pitch production and perception in music and language

My postdoctoral work supported by the Economic and Social Research Council in the UK also examined the mechanisms of pitch production and perception in music and language. In our study on intonation processing in congenital amusia (Liu et al., 2010), both the amusic and control groups performed better on imitation than identification of statements and questions in English, suggesting unconscious pitch processing during imitation. In our speech and song imitation study (Liu et al., 2013a), both the amusic and control groups imitate pitches more accurately in singing than in speaking, suggesting shared pitch-production mechanisms but distinct requirements for pitch-matching accuracy in language and music processing. This may be due to the different nature of speech and music: speech is function-driven while music is form-driven. In particular, pitch patterns in speech are used for representing functional contrasts (e.g., lexical tone/stress, focus, sentence modality, etc.), and as such their execution only needs to satisfy contrastive adequacy. For music, musical understanding or communication relies on pitch accuracy and aesthetics, which are obvious aspects to be perfected in performances. In other words the “form” taken by pitch patterns acts as a means for communication in speech, but is the intended end product for music.

Cognitive and neural bases of pitch processing

At the Chinese University of Hong Kong, I worked with Patrick Wong on the cognitive and neural bases of pitch processing in congenital amusia using auditory training, EEG, and fMRI experiments. In one study, we investigated whether musical processing deficits in congenital amusia can be treated through auditory training in pitch direction identification via a scaffolding, incremental learning approach (Liu et al., under revision). We designed and implemented an adaptive-tracking training paradigm to help amusics consciously label the direction of fine-grained pitch movement in both speech syllables and piano tones. Compared with those untrained, trained amusics showed not only improved pitch direction identification thresholds for speech and music, but also increased melodic contour processing abilities. In another study, we investigated brainstem representation and behavioral identification of lexical tones in Cantonese-speaking individuals with congenital amusia (Liu et al., 2015b). Measurements of the FFR (frequency-following response) waveforms revealed no evidence of abnormal brainstem representation of lexical tones for amusics relative to controls, in terms of timing, frequency, and amplitude. However, amusics performed significantly worse than controls on identification of these tones. These findings suggest that amusics’ subcortical neural responses simply represent acoustic/sensory properties of the tone stimuli, rather than reflecting their higher-level pitch-processing deficits.

ESRC Postdoctoral Fellowship, "How do musically tone-deaf individuals produce and perceive pitch targets in speech?", 2009-2010, Economic and Social Research Council, UK, 92,772. Role: PI. 

Between December 2009 and November 2010, I was awarded a one-year Postdoctoral Fellowship by the Economic and Social Research Council (ESRC) in the UK. For my Fellowship project, I conducted a set of experiments to investigate whether, and to what extent, congenital amusia affects Mandarin-speaking amusics' tone and intonation processing, speech versus song imitation, and song imitation with lyrics versus 'la'.

My research outputs can be seen from the following links:

Liu, F. (2009). Intonation systems of Mandarin and English: A functional approach. Ph.D. Dissertation. University of Chicago.

My Ph.D. dissertation examined how communicative functions, such as lexical tone (in Chinese), word stress (in English), focus (sentence emphasis), and sentence type (statement vs. question), interactively affect surface pitch contours of spoken utterances in English and Chinese. Using a comparative approach, I found that in both languages surface prosodic forms result from articulatory implementation of underlying pitch targets (ideal production goals) that are associated with syllables and exhibit different trajectories depending on the communicative functions conveyed. Specifically, while tonal targets in Chinese remain unchanged regardless of focus or sentence type, pitch targets of English stressed syllables vary with the stress pattern of the word, focus and sentence type conditions of the utterances. For example, in English statements, 'blackboard' has a [high] pitch target on the stressed syllable followed by a [low] target on the unstressed syllable, and 'black board' (a board that is black) has a [low] pitch target on the unstressed syllable and a [fall] target on the stressed syllable. In questions, pitch targets of English stressed syllables change (from [high] or [fall] in statements) to [rise]. In both languages pitch range of the focused word is expanded in statements as well as in questions. In Chinese, post-focus pitch range is compressed and lowered in both statements and questions, although the latter is smaller in magnitude. In English, post-focus pitch range is compressed and lowered in statements but compressed and raised in questions. Overall, my PhD research demonstrates that pitch modulations in Chinese and English are complicated yet regulated so that the function-carrying pitch targets are maintained and modified in time, place, and occasion.

                                                         Page last updated by Fang Liu on March 7, 2015.