Abstract
Multimodality approaches representation and communication as something more than language. It attends to the complex repertoire of semiotic resources and organizational means through which people make meaning – image, speech, gesture, writing, three-dimensional forms, and so on. A social semiotic approach to multimodality sets out to reveal how processes of meaning making (i.e., signification and interpretation or what is called semiosis) shape individuals and societies. In this chapter, we use multimodality to refer to “multimodal social semiotics” (Kress, Multimodality: A social semiotic approach to contemporary communication. London: Routledge, 2010). Its basic assumption is that meanings derive from social action and interaction using semiotic resources as tools. A variety of disciplines and theoretical approaches can be used to explore different aspects of the multimodal landscape (Jewitt, The Routledge handbook of multimodal analysis. London: Routledge, 2014). Psychological theories can be applied to look at how people perceive different modes or to understand the impact of one mode over another on memory, for example. Sociological and anthropological theories and interests can be applied to examine how communities use multimodal conventions to mark and maintain identities. The term multimodality is, however, most strongly linked with theories rooted in linguistics, notably systemic functional linguistics, social semiotic theory, and conversation analysis (Jewitt et al., Introducing multimodality. London: Routledge, 2016a). Examining multimodal discourses across the classroom makes more visible the relationship between the use of semiotic resources by teachers and students and the production of curriculum knowledge, student subjectivity, and pedagogy.
References
Adami, E. (2009). We/YouTube: Exploring sign-making in video-interaction. Visual Communication, 8(4), 379–399.
Adami, E. (2015). What’s in a click? A social semiotic multimodal framework for the analysis of website interactivity. Visual Communication, 14(2), 133–153.
Archer, A. (2014). Power, social justice and multimodal pedagogies. In C. Jewitt (Ed.), The Routledge handbook of multimodal analysis. London: Routledge.
Barthes, R. (1993). Mythologies. London: Vintage.
Bateman, J. (2014). Using multimodal corpora for multimodal research. In C. Jewitt (Ed.), The Routledge handbook of multimodal analysis (pp. 238–252). London: Routledge.
Bearne, E., & Kress, G. (2001). Special issue on multimodality. Reading, Literacy and Language, 35(3), 106–110.
Bezemer, J., & Kress, G. (2016). Multimodality, learning and communication. London: Routledge.
Bezemer, J., & Mavers, D. (2011). Multimodal transcription as academic practice: A social semiotic perspective. International Journal of Social Research Methodology, 14, 191–206.
Burn, A., & Parker, D. (2001). Making your mark: Digital inscription, animation, and a new visual semiotic. Education, Communication and Information, 1(2), 149–154.
Carrington, V., & Marsh, J. (Eds.) (2005). Digital childhood and youth: New texts, new literacies special issue. Discourse: Studies in the Cultural Politics of Education, 26(3), 1–8.
Crescenzi, L., Jewitt, C., & Price, S. (2014). The role of touch in preschool children’s play and learning using iPad versus paper interaction. The Australian Journal of Language and Literacy, 37(2), 86–95.
Flewitt, R. (2011). Bringing ethnography to a multimodal investigation of early literacy in a digital age. Qualitative Research, 11(3), 293–310. doi:10.1177/1468794111399838.
Flewitt, R. (2012). Multimodal perspectives on early childhood literacies. In J. Larson & J. Marsh (Eds.), The Sage handbook of early childhood literacy (pp. 295–309). London: Sage.
Halliday, M. (1978). Language as a social semiotic. London: Edward Arnold.
Hodge, R., & Kress, G. (1988). Social semiotics. Cambridge: Polity Press.
Iedema, R. (2003). Multimodality, resemiotisation. Visual Communication, 2(1), 29–57.
Jewitt, C. (2002). The move from page to screen: The multimodal reshaping of school English. Journal of Visual Communication, 1(2), 171–196.
Jewitt, C. (2008). Technology, literacy and learning: A multimodal approach. London: Routledge.
Jewitt, C. (2013). Multimodal methods for researching digital technologies. In S. Price, C. Jewitt, & B. Brown (Eds.), Handbook of researching digital technology (pp. 250–265). London: Sage.
Jewitt, C. (2014). The Routledge handbook of multimodal analysis. London: Routledge.
Jewitt, C., & Henriksen, B. (2016). Social semiotic multimodality. In H. Stockl (Ed.), Language in multimodal contexts. London: De Gruyter.
Jewitt, C., & Kress, G. (2003). Multimodal literacy. New York: Peter Lang.
Jewitt, C., Bezemer, J., & O’Halloran, K. (2016a). Introducing multimodality. London: Routledge.
Jewitt, C., Bezemer, J., Price, S., & Kress, G. (2016b). Researching digital technologies: A multimodal approach. London: Routledge.
Jewitt, C., Xambo, A., & Price, S. (2017). Exploring methodological innovation in the social aciences: The body in digital environments and the arts. International Journal of Social Research Methodology, 20(1), 105–120.
Kenner, C. (2004). Becoming biliterate: Young children learning different writing systems. Stoke on Trent: Trentham Books.
Kress, G. (2003). Literacy in the new media age. London: Routledge.
Kress, G. (2010). Multimodality: A social semiotic approach to contemporary communication. London: Routledge.
Kress, G., & van Leeuwen, T. (1996). Reading images: The grammar of visual design. London: Routledge.
Kress, G., & van Leeuwen, T. (2001). Multimodal discourse: The modes and media of contemporary communication. London: Arnold.
Kress, G., Jewitt, C., Ogborn, J., & Tsatsarelis, C. (2001). Multimodal teaching and learning: The rhetorics of the science classroom. London: Continuum.
Lemke, J. (1998). Multimodality special issue. Linguistics and Education, 10(3), 359–386.
Lemke, J. (2002). Travels in hypermodality. Visual Communication, 1(3), 299–325.
New London Group. (1996). A pedagogy of multiliteracies: Designing social futures. Harvard Educational Review, 66, 60–92.
Newfield, D. (2014). Transformation, transduction and the transmodal moment. In C. Jewitt (Ed.), The Routledge handbook of multimodal analysis (pp. 100–114). London: Routledge.
Norris, S. (2004). Analyzing multimodal interaction: A methodological framework. London: Routledge.
Norris, S. (2014). Modal density and modal configurations: Multimodal actions. In C. Jewitt (Ed.), The Routledge handbook of multimodal analysis (pp. 86–99). London: Routledge.
O’Halloran, K. (2004). Multimodal discourse analysis: Systemic functional perspectives. London: Continuum.
O’Toole, M. (1994). The language of displayed art. Leicester: Leicester University Press.
Pahl, K. (1999). Transformations: Children’s meaning making in nursery education. Stoke on Trent: Trentham Books.
Price, S., & Jewitt, C. (2013, February). A multimodal approach to examining ‘embodiment’ in tangible learning environments. Proceedings of the Seventh International Conference on Tangible, Embedded and Embodied Interaction, Barcelona.
Price, S., Sakr, M., & Jewitt, C. (2015). Exploring whole-body interaction and design for museums. Interacting with Computers Advance. doi:10.1093/iwc/iwv032.
Sakr, M., Jewitt, C., & Price, S. (2015). Mobile experiences of historical place: A multimodal analysis of emotional engagement. Journal of the Learning Sciences. doi:10.1080/10508406.2015.1115761.
Scollon, R., & Wong-Scollon, S. (2004). Discourses in place: Language in the material world. London: Routledge.
Stein, P. (2003). The Olifantsvlei fresh stories project. In C. Jewitt & G. Kress (Eds.), Multimodal literacy (pp. 123–138). New York: Peter Lang.
Street, B., Pahl, K., & Rowsell, J. (2014). Multimodality and new literacy studies. In C. Jewitt (Ed.), The Routledge handbook of multimodal analysis (pp. 181–200). London: Routledge.
van Leeuwen, T. (1999). Speech, sound and music. London: Macmillan.
van Leeuwen, T. (2005). Introducing social semiotics. London: Routledge.
Van Leeuwen, T. (2010). The language of colour. London: Routledge.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2016 Springer International Publishing AG
About this entry
Cite this entry
Jewitt, C. (2016). Multimodal Discourses Across the Curriculum. In: Thorne, S., May, S. (eds) Language, Education and Technology. Encyclopedia of Language and Education. Springer, Cham. https://doi.org/10.1007/978-3-319-02328-1_4-1
Download citation
DOI: https://doi.org/10.1007/978-3-319-02328-1_4-1
Received:
Accepted:
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-02328-1
Online ISBN: 978-3-319-02328-1
eBook Packages: Springer Reference EducationReference Module Humanities and Social SciencesReference Module Education