Abstract
Establishing which neural systems support processing of sign languages informs a number of important neuroscience and linguistic questions. In this chapter, the linguistic structure of sign languages is introduced with a discussion of common myths about sign languages. This is followed by a more detailed discussion of the linguistics of British Sign Language, with special reference to features which resemble or contrast with spoken languages. The final section describes language and the brain by describing a number of neuroimaging studies with signers and research on signers who have aphasia or other language deficits following strokes. The neuroimaging and aphasia data are used to explore the ‘core language system’ - the regions of the brains used for language regardless of modality.
Access provided by Autonomous University of Puebla. Download to read the full chapter text
Chapter PDF
Similar content being viewed by others
References
Meier, R.P.: Why Different, Why the Same? Explaining Effects and Non-effects of Modality upon Linguistic Structure in Sign and Speech. In: Meier, R.P., Cormier, K., Quinto-Pozos, D. (eds.) Modality and Structure in Signed and Spoken Languages, pp. 1–25. Cambridge University Press, Cambridge (2002)
Pinker, S., Bloom, P.: Natural Language and Natural Selection. Behav. Br Sci. 13/4, 707–784 (1990)
Stokoe, W.C.: Sign language structure. An Outline of the visual communication system of the American deaf. Studies in Linguistics. Occasional Papers 8. University of Buffalo, Buffalo, NY (1960); revised edition. Linstok Press, Silver Spring (1978)
Sutton-Spence, R.L., Woll, B.: The linguistics of BSL: An introduction. Cambridge University Press, Cambridge (1999)
Pizzuto, E., Ardito, B., Caselli, M.C., Volterra, V.: Cognition and Language in Italian Deaf Preschoolers of Deaf and Hearing Families. In: Clark, M.D., Marschark, M., Karchmer, M.A. (eds.) Context, Cognition, and Deafness: An Introduction, pp. 49–70. Gallaudet University Press, Washington (2001)
Morgan, G., Herman, R., Woll, B.: The Development of Complex Verb Constructions in BSL. J. Ch. Lang 29, 655–675 (2002)
Tolar, T.D., Lederberg, A.R., Gokhale, S., Tomasello, M.: The Development of the Ability to Recognize the Meaning of Iconic Signs. J. Deaf. Stud. Deaf Ed. 13(2), 225–240 (2008)
Campbell, R., Martin, P., White, T.: Forced Choice Recognition of Sign in Novice Learners of British Sign Language. App. Ling 13(2), 185–201 (1992)
Liddell, S.K.: American Sign Language Syntax. The Hague, Mouton (1980)
Zeshan, U.: Interrogative constructions in signed languages: crosslinguistic perspectives. Language 80(1), 7–39 (2004)
Jackendoff, R.: The Architecture of the Linguistic– Spatial Interface. In: Bloom, P., Peterson, M.A., Nadal, L., Garrett, M.F. (eds.) Language and Space, pp. 1–30. MIT Press, Cambridge (1996)
De Vega, M., Cocude, M., Denis, M., Rodrigo, M.J., Zimmer, H.D.: The interface between language and visuo-spatial representations. In: Denis, M., Logie, R.H., Cornoldi, C., De Vega, M., Engelkamp, J. (eds.) Imagery, Language, and Visuo-Spatial Thinking, pp. 109–136. Psychology Press, Hove (2001)
Kemmerer, D.: ‘Near’ and ‘Far’ in Language and Perception. Cognition 73, 35–63 (1999)
Emmorey, K.: The Effects of Modality on Spatial Language: How Signers and Speakers Talk about Space. In: Meier, R.P., Cormier, K., Quinto-Pozos, D. (eds.) Modality and Structure in Signed and Spoken Language, pp. 405–421. Cambridge University Press, Cambridge (2002)
Agrafiotis, D., Canagarajah, N., Bull, D.R., Dye, M.: Perceptually optimised sign language video coding based on eye tracking analysis. Electron. Lett. 39(24), 1703–1705 (2003)
Emmorey, K., Corina, D., Bellugi, U.: Differential processing of topographic and referential functions of space. In: Emmorey, K., Reilly, J. (eds.) Language, Gesture and Space, pp. 43–62. Lawrence Erlbaum Associates, Hillsdale (1995)
Emmorey, K.: Language, cognition, and the brain: Insights from sign language research. Lawrence Erlbaum Associates, Hillsdale (2001)
Engberg-Pedersen, E.: Space in Danish Sign Language: the Semantics and Morphosyntax of the Use of Space in a Visual Language. Signum Press, Hamburg (1993)
Supalla, T.: The Classifier System in ASL. In: Craig, C. (ed.) Noun Classification and Categorization, pp. 181–214. John Benjamins, Amsterdam (1986)
Campbell, R., MacSweeney, M., Waters, D.: Sign Language and the Brain: a Review. J. Deaf. Stud. Deaf Ed. 13(1), 3–20 (2008)
MacSweeney, M., Woll, B., Campbell, R., McGuire, P.K., David, A.S., Williams, S.C.R., Suckling, J., Calvert, G.A., Brammer, M.J.: Neural Systems Underlying British Sign Language and Audiovisual English Processing in Native Users. Brain 125, 1583–1593 (2002)
Calvert, G.A., Bullmore, E.T., Brammer, M.J., Campbell, R., Williams, S.C., McGuire, P.K., Woodruff, P.W., Iversen, S.D., David, A.S.: Activation of Auditory Cortex During Silent Lipreading. Science 25:276 (5312), 593–596 (1997)
MacSweeney, M., Amaro, E., Calvert, G., Campbell, R., David, A.S., McGuire, P.K., Williams, S.C., Woll, B., Brammer, M.J.: Silent Speechreading in the Absence of Scanner Noise: An Event-related fMRI Study. Neurorep. 11(8), 1729–1733 (2000)
MacSweeney, M., Campbell, R., Calvert, G.A., McGuire, P.K., David, A.S., Suckling, J., Andrew, C., Woll, B., Brammer, M.J.: Dispersed Activation in the Left Temporal Cortex for Speech-reading in Congenitally Deaf People. Proc. Roy. Soc. B 268, 451–457 (2001)
Haist, F., Song, A.W., Wild, K., Faber, T.L., Popp, C.A., Morris, R.D.: Linking Sight and Sound: fMRI Evidence of Primary Auditory Cortex Activation during Visual Word Recognition. Br Lang. 763, 340–350 (2001)
Levänen, S., Jousmäki, V., Hari, R.: Vibration-induced Auditory-Cortex Activation in a Congenitally Deaf Adult. Curr. Biol. 8, 869–872 (1998)
Nishimura, H., Hashikawa, K., Doi, K., Iwaki, T., Watanabe, Y., Kusuoka, H., Nishimura, T., Kubo, T.: Sign Language ’Heard’ in The Auditory Cortex. Nature 3976715, 116 (1999)
Petitto, L.A., Zatorre, R.J., Gauna, K., Nikelski, E.J., Dostie, D., Evans, A.C.: Speech-like Cerebral Activity in Profoundly Deaf People Processing Signed Languages: Implications for the Neural Basis of Human Language. Proc. Nat. Acad. Sci. USA 97, 13961–13966 (2000)
MacSweeney, M., Woll, B., Campbell, R., Calvert, G.A., McGuire, P.K., David, A.S., Simmons, A., Brammer, M.J.: Neural Correlates of British Sign Language Comprehension: Spatial Processing Demands of Topographic Language. J. Cog. Neurosci. 14, 1064–1075 (2002)
Poizner, H., Klima, E., Bellugi, U.: What the Hands Reveal about the Brain. MIT Press, Cambridge (1987)
Hickok, G., Say, K., Bellugi, U., Klima, E.: The Basis of Hemispheric Asymmetries for Language and Spatial Cognition: Clues from Focal Brain Damage in 2 Deaf Native Signers. Aphasiol. 10, 577–591 (1996)
Hickok, G., Bellugi, U., Klima, E.: The Neural Organization of Sign Language: Evidence from Sign Language Aphasia. Trans. Cog. Sci. 2, 129–136 (1998)
Corina, D.P.: Aphasia in Users of Signed Languages. In: Coppens, P., Lebrun, Y., Basso, A. (eds.) Aphasia in Atypical Populations, pp. 261–309. Lawrence Erlbaum Associates, Mahwah (1998)
Corina, D.P.: The Processing of Sign Language: Evidence from Aphasia. In: Stemmer, B., Whitaker, H.A. (eds.) Handbook of Neurolinguistics, pp. 313–329. Academic Press, NY (1998)
Atkinson, J.R., Campbell, R., Marshall, J., Thacker, A., Woll, B.: Understanding ‘not’: Neuropsychological Dissociations between Hand and Head Markers of Negation in BSL. Neuropsychologia 42, 214–229 (2004)
Atkinson, J.R., Marshall, J., Woll, B., Thacker, A.: Testing Comprehension Abilities in Users of British Sign Language following CVA. Br Lang. 94(2), 233–248 (2005)
Marshall, J., Atkinson, J.R., Smulovitch, E., Thacker, A., Woll, B.: Aphasia in a user of British Sign Language: Dissociation between sign and gesture. Cog. Neuropsychol. 21(5), 537–554 (2004)
Marshall, J., Atkinson, J.R., Woll, B., Thacker, A.: Aphasia in a Bilingual User of British Sign Language and English: Effects of Cross Linguistic Cues. J. Cog. Neuropsychol. 22(6), 719–736 (2005)
Corina, D.P., Poizner, H., Bellugi, U., Feinberg, T., Dowd, D., O’Grady-Batch, L.: Dissociation between Linguistic and Nonlinguistic Gestural Systems: a Case for Compositionality. Br Lang. 43, 414–447 (1992)
Pizzuto, E., Volterra, V.: Iconicity and Transparency in Sign Languages: A Cross-linguistic Cross-cultural view. In: Emmorey, K., Harlan, L. (eds.) The Signs of Language Revisited: An Anthology in Honor of Ursula Bellugi and Edward Klima, pp. 261–286. Lawrence Erlbaum Associates, Hillsdale (2000)
Klima, E.S., Bellugi, U.: The Signs of Language. MIT Press, Cambridge (1979)
Emmorey, K., Grabowski, T., McCullough, S., Damasio, H., Ponto, L., Hichwa, R., Bellugi, U.: Motor-iconicity of Sign Language Does Not Alter the Neural Systems Underlying Tool and Action Naming. Br Lang. 89, 27–37 (2004)
Neville, H., Bavelier, D., Corina, D., Rauschecker, J., Karni, A., Lalwani, A., Braun, A., Clark, V., Jezzard, P., Turner, R.: Cerebral Organization for Language in Deaf and Hearing Subjects: Biological Constraints and Effects of Experience. Proc. Nat. Acad. Sci. USA 95, 922–929 (1998)
Newman, A.J., Bavelier, D., Corina, D., Jezzard, P., Neville, H.J.: A Critical Period for Right Hemisphere Recruitment in American Sign Language Processing. Nat. Neurosci. 5, 76–80 (2002)
Paulesu, E., Mehler, J.: Right on in Sign Language. Nature 392, 233–234 (1998)
Emmorey, K., Damasio, H., McCullough, S., Grabowski, T., Ponto, L., Hichwa, R., et al.: Neural Systems Underlying Spatial Language in American Sign Language. Neuroimage 17, 812–824 (2002)
Kegl, J., Poizner, H.: Crosslinguistic/crossmodal Syntactic Consequences of Left-Hemisphere Damage: Evidence from an Aphasic Signer and his Identical Twin. Aphasiol. 11, 1–37 (1997)
Loew, R.C., Kegl, J.A., Poizner, H.: Fractionation of the Components of Role Play in a Right-Hemisphere Lesioned Signer. Aphasiol. 11, 263–281 (1997)
Hickok, G., Wilson, M., Clark, K., Klima, E.S., Kritchevsky, M., Bellugi, U.: Discourse Deficits Following Right Hemisphere Damage in Deaf Signers. Br Lang. 66, 233–248 (1999)
Wapner, W., Hamby, S., Gardner, H.: The Role of the Right Hemisphere in the Apprehension of Complex Linguistic Materials. Br Lang. 14, 15–33 (1981)
Kaplan, J.A., Brownell, H.R., Jacobs, J.R., Gardner, H.: The Effects of Right Hemisphere Damage on the Pragmatic Interpretation of Conversational Remarks. Br Lang. 38(2), 315–333 (1990)
Rönnberg, J., Söderfeldt, B., Risberg, J.: The Cognitive Neuroscience of Signed Language. Acta. Psychol. 105(2-3), 237–254 (2000)
Sutton-Spence, R., Day, L.: Mouthings and Mouth Gestures in British Sign Language. In: Boyes-Braem, P., Sutton-Spence, R. (eds.) The Hands are the Head of the Mouth, pp. 69–86. Signum Press, Hamburg (2001)
Boyes-Braem, P., Sutton-Spence, R. (eds.): The Hands are the Head of the Mouth. Signum Press, Hamburg (2002)
Waters, D., Campbell, R., Capek, C.M., Woll, B., David, A.S., McGuire, P.K., Brammer, M.J., MacSweeney, M.: Fingerspelling, Signed Language, Text and Picture Processing in Deaf Native Signers: The Role of the Mid-fusiform Gyrus. Neuroimage 35(3), 1287–1302 (2007)
Capek, C., Woll, B., MacSweeney, M., Waters, D., David, A.S., McGuire, P.K., Brammer, M.J., Campbell, R.: Hand and Mouth: Cortical Correlates of Lexical Processing in BSL and Speechreading. J. Cog. Neurosci. 20, 1220–1234 (2008)
McCullough, S., Emmorey, K., Sereno, M.: Neural Organization for Recognition of Grammatical Facial Expressions in Deaf ASL Signers and Hearing Nonsigners. Cog. Br. Res. 22, 192–203 (2005)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2009 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Woll, B. (2009). How the Brain Processes Language in Different Modalities. In: Esposito, A., Hussain, A., Marinaro, M., Martone, R. (eds) Multimodal Signals: Cognitive and Algorithmic Issues. Lecture Notes in Computer Science(), vol 5398. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-00525-1_15
Download citation
DOI: https://doi.org/10.1007/978-3-642-00525-1_15
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-642-00524-4
Online ISBN: 978-3-642-00525-1
eBook Packages: Computer ScienceComputer Science (R0)