Auditory Meaning-Making in Multimodal Documentary Discourse: Functions, Features, and Synergies

Wenhan Pan

Abstract


This study explores how sound helps shape cultural meaning in the English-language documentary Festive China. While most multimodal research has focused on images and text, sound has received less attention. This paper aims to fill that gap by examining the use of narration, background music, and ambient sound in the film. The analysis focuses on five key features of sound: loudness, pitch, rhythm, timbre, and brightness. The study uses a qualitative method based on Systemic Functional Linguistics and multimodal discourse analysis. It looks at how sound works with visuals and language to build meaning in three ways: what is shown, how the audience is engaged, and how the content is organized. The results show that narration explains ideas and builds emotional tone. Background music creates mood and supports the story’s rhythm. Ambient sounds add realism and connect viewers to the setting. These elements often work together to support themes like celebration, memory, and tradition. The sound design also reflects cultural values such as balance and harmony. By focusing on audio modality, this study adds to current multimodal research and shows the importance of sound in meaning-making. It also offers insights for educators who want to help students better understand how documentaries use sound to share cultural ideas.


Keywords


multimodal discourse analysis, documentary studies, Systemic Functional Linguistics, cultural meaning-making

Full Text:

PDF

References


Bateman, J. A. (2023). Multimodal discourse analysis: Theories and methods. Cambridge University Press.

Chion, M. (2021). Audio-vision: Sound on screen (Updated ed.). Columbia University Press.

Couper-Kuhlen, E. (2021). Prosody as dialogic action: Evidence from speech. Frontiers in Psychology, 12, 640741. https://doi.org/10.3389/fpsyg.2021.640741

Dong, M., & Yuan, X. L. (2021). On constructing a multimodal aesthetic critical discourse analysis framework. Foreign Language Education, 42(1), 77–82. https://doi.org/10.16362/j.cnki.cn61-1023/h.2021.01.014

Eerola, T., Ferrer, R., & Alluri, V. (2012). Timbre and affect dimensions: Evidence from affect and similarity ratings and acoustic correlates of isolated instrument sounds. Music Perception, 30(1), 49–70. https://doi.org/10.1525/mp.2012.30.1.49

Feng, W. D. (2017). Metonymy and visual representation: Towards a social semiotic framework of visual metonymy. Visual Communication, 16(4), 441–466. https://doi.org/10.1177/1470357217717142

Groves, K., Farbood, M. M., Carone, B., Ripollés, P., & Zuanazzi, A. (2025). Acoustic features of instrumental movie soundtracks elicit distinct and mostly non-overlapping extra-musical meanings in the mind of the listener. Scientific Reports, 15, Article 2327. https://doi.org/10.1038/s41598-025-86089-6

Halliday, M. A. K., & Hasan, R. (1989). Language, context, and text: Aspects of language in a social-semiotic perspective (2nd ed.). Oxford University Press.

Jackson, B. (2022). Sound anchors: A cognitive and multimodal approach to sound and narrative structure in film. Projections, 16(3), 61–82. https://doi.org/10.3167/proj.2022.160303

Jewitt, C. (2009). The Routledge handbook of multimodal analysis. Routledge.

Kassabian, A. (2001). Hearing film: Tracking identifications in contemporary Hollywood film music. Routledge. https://doi.org/10.4324/9780203427460

Kress, G., & van Leeuwen, T. (2006). Reading images: The grammar of visual design (2nd ed.). Routledge.

Kress, G., & van Leeuwen, T. (2021). Reading images: The grammar of visual design (3rd ed.). Routledge. https://doi.org/10.4324/9781003099857

Li, M., Yasin, M. A. I., Hashim, N. B., & Ang, L. H. (2020). A historical review of the Chinese documentary programs. Journal of Critical Reviews, 7(2), 723–732. https://doi.org/10.31838/jcr.07.02.133

Machin, D. (2010). Analysing popular music: Image, sound, text. SAGE. https://doi.org/10.4135/9781446280027

Machin, D. (2013). What is multimodal critical discourse studies? Critical Discourse Studies, 10(4), 347–355. https://doi.org/10.1080/17405904.2013.813770

Martinec, R., & Salway, A. (2005). A system for image-text relations in new (and old) media. Visual Communication, 7(3), 337–371. https://doi.org/10.1177/1470357205055928

Nai, R., & Hassan, H. (2022). A multimodal discourse analysis of BBC documentary Chinese New Year: The Biggest Celebration on Earth. Hong Kong Journal of Social Sciences, (60), 402–411. https://doi.org/10.55463/hkjss.issn.1021-3619.60.40

Nan, N. (2024). Recording China in sound: Audio-visual landscape and cultural expression in “China in the Sound.” Education Reform and Development, 6(3), 263–268. https://doi.org/10.26689/erd.v6i3.6586

Neville, M. (Director). (2018). Won’t you be my neighbor? [Film]. Focus Features.

Pan, Y. Y., & Zheng, Z. H. (2018). On the criticality of multimodal discourse analysis. Journal of Tianjin Foreign Studies University, 25(2), 78–87.

Tian, H. L., & Pan, Y. Y. (2018). From meaning to intention: New development of multimodal discourse analysis to multimodal critical discourse analysis. Shandong Foreign Language Teaching, 39(1), 23–33. https://doi.org/10.16482/j.sdwy37-1026.2018-01-003

van Leeuwen, T. (1999). Speech, music, sound. Palgrave Macmillan.

van Leeuwen, T. (2004). Introducing social semiotics. Routledge.

van Leeuwen, T. (2021). Multimodality and identity. Routledge. https://doi.org/10.4324/9781003186625

Wang, R. (2023). Multimodal analysis of image-text relations in the translation of news cartoons in China Daily. Frontiers in Humanities and Social Sciences, 3(8), 27–37. https://doi.org/10.54691/fhss.v3i8.5534

Wei, X. (2024). Multimodal metaphors in a Sino-British co-produced documentary. Humanities and Social Sciences Communications, 11, Article 286. https://doi.org/10.1057/s41599-024-02739-6

Zhao, J. (2020). Interpreting the cultural symbols in Chinese documentaries: Taking A Bite of China and Beautiful China as examples. Frontiers in Educational Research, 3(11), 41–46. https://doi.org/10.25236/FER.2020.031107




DOI: http://dx.doi.org/10.21462/jeltl.v10i2.1696

Refbacks

  • There are currently no refbacks.






JELTL (Journal of English Language Teaching and Linguistics); Web: www.jeltl.org; Email: journal.eltl@gmail.com


Creative Commons License
JELTL by http://www.jeltl.org is licensed under a Creative Commons Attribution-ShareAlike 4.0 International License


Indexed and Abstracted BY: