Abstract
Contemporary first language curricula such as the Australian Curriculum: English strongly advocates the teaching of multimodal digital literature in the form of film in which understanding how characterisation is developed is a key task in comprehending and composing literature. In printed literary work, characterisation relies on the only semiotic resource-language to directly or indirectly depict a character’s personality. However, in digital multimodal literature such as film, language is only one among many resources including actions, body language, sound, cinematography, and lighting that all negotiate their shared roles to contribute to the characterisation process. This paper discusses the pivotal role of gesture in the characterisation of Coraline, the central character in a stop-motion animation adaptation of the novel with this name. The paper proposes a framework for examining attitudinal meanings in gesture from the perspective of systemic functional linguistics, which was applied to analyse the deployment of Coraline’s gesture during points of tension in the movie. The analysis indicates that gesture and co-occuring speech by the character never realise the same kinds of attitudinal meanings. Even when the two semiotics converge, gesture always has more attitudinal meaning commitment (i.e., Graduation) than language. These findings call for a conceptual renovation in first language education curricula. The pivotal role of body language should be given the status it deserves if digital multimodal literature education is to be taken seriously as indicated in the curriculum.
Article PDF
Similar content being viewed by others
Avoid common mistakes on your manuscript.
References
Australian Curriculum, Assessment and Reporting Authority [ACARA. (2014). Australian Curriculum: English. Retrieved from http://www.australiancurriculum.edu.au/english/Curriculum/F-10?layout=1
Buisine, S. & Martin, J.-C. (2007). The effects of speech- gesture cooperation in animated agents’ behavior in multimedia presentations. Interacting with Computers, 19 (4), 484–493.
Carter, J. (1999). Talking books. London: Routledge.
Chan, E. (2011). Integrating visual and verbal meaning in multimodal text comprehension: Towards a model of intermodal relations. In S. Dreyfus, S. Hood & M. Stenglin (Eds.), Semiotic margins. Meanings in multimodalities (pp. 144–167). London: Continuum.
Cleirigh, C. (2010). Gesture and posture semiosis: A systemicfunctional linguistic approach to ‘body language’. Linguistics Department. University of Sydney.
Focus-Features. (2008). ‘Coraline’ - the biggest smallest movie. Retrieved from https://www.youtube.com/watch?v=cayVMHSRHZI&list=PLIOvQw23ql2tbKIbBstoxZm OAkiUFRW7J
Gaiman, N. (2002). Coraline. UK: Bloomsbury.
Halliday, M.A.K. (1978). Language as Social Semiotic: The Social Interpretation of Language and Meaning. London: Edward Armold.
Halliday, M.A.K. & Matthiessen, C.M.I.M. (2004). An Introduction to Functional Grammar (3rd ed.). London: Hodder Education.
Hirsch, R. (1995). Gesture Universals: Material and Formal Constraints. In T.A. Sebeok & J. Umiker-Sebeok (Eds.), Advances in Visual Semiotics: The Semiotic Web 1992-93 (Vol. 118, pp. 475–492). Berlin: Mouton de Gruyter.
Hood, S. (2008). Summary writing in academic contexts: Implicating meaning in processes of change. Linguistics and Education, 19 (4), 351–365.
Hood, S. (2010). Appraising research: Evaluation in academic writing. New York: Palgrave.
Hood, S. (2011). Body language in face-to-face teaching: A focus on textual and interpersonal meaning. Semiotic margins: Reclaiming meaning, 31–52.
Hood, S. & Forey, G. (2005). Introducing a conference paper: Getting interpersonal with your audience. Journal of English for Academic Purposes, 4 (4), 291–306.
Kendon, A. (2004). Gesture: Visible action as utterance: Cambridge University Press.
Macken-Horarik, M. (2003). Appraisal and the special instructiveness of narrative. Text: an interdisciplinary journal for the study of discourse: Negotiating Heterglossia: Social Perspectives on Evaluation, 23 (2), 285–312.
Macken-Horarik, M. (2006). Hierarchies in diversities: What students’ examined responses tell us about literacy practices in contemporary school English. Australian Journal of Language and Literacy, 29 (1), 52–78.
Martin, J.R. & White, P.R. (2005). The Language of Evaluation: Appraisal in English. London/New York: Palgrave/Macmillan.
Martin, J.R., Zappavigna, M., Dwyer, P. & Cléirigh, C. (2013). Users in uses of language: embodied identity in Youth Justice Conferencing. Text & Talk, 33(4-5), 467–496.
Martinec, R. (2000). Types of process in action. Semiotica, 130(3-4), 243–268.
Martinec, R. (2001). Interpersonal resources in action. Semiotica, 135(1/4), 117–146.
Martinec, R. (2004). Gestures that co-occur with speech as a systematic resource: the realization of experiential meanings in indexes. Social Semiotics, 14 (2), 193–213.
Martinec, R. & Salway, A. (2005). A system for image-text relations in new (and old) media. Visual communication, 4 (3), 337–371.
McNeill, D. (1992). Hand and Mind: What Gestures Reveal About Thought. Chicago: University of Chicago Press.
McNeill, D. (2012). How language began: Gesture and speech in human evolution. UK: Cambridge university press.
Mills, K.A. (2016). Literacy theories for the digital age: Social, critical, multimodal, spatial, material and sensory lenses (Vol. 45). Bristol: UK: Multilingual Matters.
Mohamed, F.N. & Nor, N.L.M. (2015). Puppet Animation Films and Gesture Aesthetics. Animation, 10 (2), 102–118.
Nelmes, J. (Ed.) (2012). Introduction to film studies (5th ed.). London and New York: Routledge.
NESA. (2012). Suggested texts for the English K-10 Syllabus. Retrieved from http://syllabus.nesa.nsw.edu.au/assets/global/files/english-k10-suggested-texts.pdf
NESA. (2017). Respect and Responsibility values in Education: Curriculum Mapping_Stage 4 English. Retrieved from http://www.boardofstudies.nsw.edu.au/respect/curriculum/stage4_english.html
Ngo, T. (2013). The deployment of the language of evaluation in English and Vietnamese spoken discourse. (Ph.D Unpublished), University of New England.
Ngo, T. & Unsworth, L. (2015). Reworking the appraisal framework in ESL research: refining attitude resources. Functional Linguistics, 2 (1), 1–24.
O’Brien, A. (2014). Using Focalisation Choices to Manipulate Audience Viewpoint in 3-D Animation Narratives: What Do Student Authors Need to Know? In L. Unsworth & A. Thomas (Eds.), English Teaching and New Literacies Pedagogy: Interpretive and Authoring Digital Multimedia Narratives (pp. 123–150). New York: Peter Lang.
Painter, C., Martin, J.R. & Unsworth, L. (2012). Reading Visual Narratives. London: Equi-nox.
Phillips, W.H. (2005). Film: An introduction (3rd ed.). Boston, New York: Bedford/ST Martin’s.
Selick, H. (Writer). (2009). Coraline. In H. Selick, B. Mechanic, J. Jennifer & M. Sandell (Producer). Australia.
Showalter, E. (2003). Teaching literature. UK: Wiley-Blackwell.
Unsworth, L. (2013). Re-configuring image-language relations and interpretive possibilities in picture books as animated movies: A site for developing multimodal literacy pedagogy. Ilha do Desterro A Journal of English Language, Literatures in English and Cultural Studies (64), 015–048.
Unsworth, L. (2014a). The Image/Language Interface in Picture Books as Animated Films: A Focus for New Narrative Interpretation and Composition Pedagogies. In L. Unsworth & A. Thomas (Eds.), English Teaching and New Literacies Pedagogy: Interpretive and Authoring Digital Multimedia Narratives. New York: Peter Lang.
Unsworth, L. (2014b). Investigating point of view in picture books and animated movie adaptations. In K. Mallan (Ed.), Picture books and beyond (pp. 92–107). Australia: PETAA.
Unsworth, L. (2014c). Persuasive narratives: Evaluative images in picture books and animated movies. Visual communication.
Unsworth, L. & Cléirigh, C. (2009). Multimodality and reading: The construction of meaning through image-text interaction. In C. Jewitt (Ed.), Handbook of Multimodal Analysis (pp. 151–164). London: Routledge.
van Leeuwen, T. (2005). Introducing social semiotics: Psychology Press.
Wells, P. (1998). Understanding animation. New York: Routledge.
Author information
Authors and Affiliations
Rights and permissions
About this article
Cite this article
Ngo, T. Gesture as transduction of characterisation in children’s literature animation adaptation. AJLL 41, 30–43 (2018). https://doi.org/10.1007/BF03652004
Published:
Issue Date:
DOI: https://doi.org/10.1007/BF03652004