The development of technology has led to significant advances in applied research methods in the cognitive sciences. Eye tracking (oculography) is among the methods available for studying human behavior and its underlying cerebral mechanisms and cognitive processes and is a method for recording and analyzing oculomotor activity in real time. This review addresses the use of eye tracking in cognitive research both separately and in combination with electroencephalography, i.e., analysis of event-related potentials (ERP). Eye tracking will also be discussed in terms of its use in language research, from the study of comprehension and sentence construction to second language studies and bilingualism. Finally, the review will consider the parallel recording of eye movements and ERP. The review will draw attention not only to the strengths of the eye tracking technique, but also to studies which we believe can be addressed by using parallel recording of oculomotor activity and the EEG.
Article PDF
Similar content being viewed by others
Avoid common mistakes on your manuscript.
References
Allopenna, P. D., Magnuson, J. S., and Tanenhaus, M. K., “Tracking the time course of spoken word recognition using eye movements: Evidence for continuous mapping models, J. Mem. Lang., 38, No. 4, 419–439 (1998).
Altarriba, J., Kroll, J. F., Sholl, A., and Rayner, K., “The influence of lexical and conceptual constraints on reading mixed-language sentences: Evidence from eye fixations and naming times,” Mem. Cogn., 24, No. 4, 477–492 (1996).
Altmann, G. T. and Kamide, Y., “Incremental interpretation at verbs: Restricting the domain of subsequent reference,” Cognition, 73, No. 3, 247–264 (1999).
Anisimov, V. N., Kovaleva, A. V., Boiko, L. A., et al., “EEG and autonomic and oculomotor indicators of consumer interest using tourist excursions as an example,” Vestn. Psikhofiziol., 1, 45–54 (2019a).
Anisimov, V. N., Zinchenko, E. M., Boiko, L. A., et al., “Methods for evaluated commercial texts: a neuromarketing approach,” Praktich. Market., 3, No. 265, 3–14 (2019b).
Baccino, T. and Manunta, Y., “Eye-fixation-related potentials: Insight into parafoveal processing,” J. Psychophysiol., 19, No. 3, 204–215 (2005).
Balling, L. W., “Does good writing mean good reading?: An eye tracking investigation of the effect of writing advice on reading,” Fachsprache, 35, No. 1–2, 2–23 (2013).
Bastiaansen, M. and Hagoort, P., “Oscillatory neuronal dynamics during language comprehension,” Progr. Brain Res., 159, 179–196 (2006).
Bentin, S., Mouchetant-Rostaing, Y., Giard, M. H., et al., “ERP manifestations of processing printed words at different psycholinguistic levels: time course and scalp distribution,” J. Cogn. Neurosci., 11, No. 3, 235–260 (1999).
Berg, P. and Scherg, M., “Dipole modelling of eye activity and its application to the removal of eye artefacts from the EEG and MEG,” Clin. Physics Physiol. Meas., 12A, 49–54 (1991).
Blythe, H. I., Liversedge, S. P., Joseph, H. S., et al., “The binocular coordination of eye movements during reading in children and adults,” Vision Res., 46, No. 22, 3898–3908 (2006).
Bock, K., Irwin, D. E., Davidson, D. J., and Levelt, W. J., “Minding the clock,” J. Mem. Lang., 48, No. 4, 653–685 (2003).
Bolger, D. J., Balass, M., Landen, E., and Perfetti, C. A., “Context variation and definitions in learning the meanings of words: An instance-based learning approach,” Discourse Proc., 45, No. 2, 122–159 (2008).
Bressler, S. L. and Richter, C. G., “Interareal oscillatory synchronization in top-down neocortical processing,” Curr. Opin. Neurobiol., 31, 62–66 (2015).
Brysbaert, M., Drieghe, D., and Vitu, F., “Word skipping: Implications for theories of eye movement control in reading,” in: Cognitive Processes in Eye Guidance (2005).
Buswell, G. T., “An experimental study of the eye-voice span in reading,” Supplementary Educational Monographs, Iss. 17, University of Chicago (1920).
Chace, K. H., Rayner, K., and Well, A. D., “Eye movements and phonological parafoveal preview: effects of reading skill,” Can. J. Exp. Psychol., 59, No. 3, 209 (2005).
Chaffin, R., Morris, R. K., and Seely, R. E., “Learning new word meanings from context: A study of eye movements,” J. Exp. Psychol. Learn. Mem. Cogn., 27, No. 1, 225 (2001).
Chambers, C. G. and Cooke, H., “Lexical competition during second-language listening: Sentence context, but not proficiency, constrains interference from the native lexicon,” J. Exp. Psychol. Learn. Mem. Cogn., 35, No. 4, 1029 (2009).
Clifton, C. Jr., Staub, A., and Clifton, C., “Syntactic influences on eye movements during reading,” Eye, 3, No. 2 (2011).
Clifton, C., “Eye movements of highly skilled and average readers: Differential effects of frequency and predictability,” Q. J. Exp. Psychol. A, 58, No. 6, 1065–1086 (2005).
Clifton. C. Jr., Ferreira, F., Henderson, J. M., et al., “Eye movements in reading and information processing: Keith Rayner’s 40 year legacy,” J. Mem. Lang., 86, 1–19 (2016).
Conklin, K. and Pellicer-Sánchez, A., “Using eye tracking in applied linguistics and second language research,” Second Lang. Res., 32, No. 3, 453–467 (2016).
Conklin, K., Pellicer-Sánchez, A., and Carrol, G., Eye Tracking: A Guide for Applied Linguistics Research, Cambridge University Press (2018).
Cooper, R. M., “The control of eye fixation by the meaning of spoken language: A new methodology for the real-time investigation of speech perception, memory, and language processing,” Cogn. Psychol., 6, No. 1, 84–107 (1974).
Cop, U., Keuleers, E., Drieghe, D., and Duyck, W., “Frequency effects in monolingual and bilingual natural reading,” Psychonom. Bull. Rev, 22, No. 5, 1216–1234 (2015).
Coulson, S., “Electrifying results: ERP data and cognitive linguistics,” Meth. Cogn. Linguistics, 18 (2007).
Croft, R. J. and Barry, R. J., “Removal of ocular artifact from the EEG: a review,” Neurophysiol. Clin., 30, No. 1, 5–19 (2000).
Delorme, A., Sejnowski, T., and Makeig, S., “Enhanced detection of artifacts in EEG data using higher-order statistics and independent component analysis,” NeuroImage, 34, No. 4, 1443–1449 (2007).
Dimigen, O., Kliegl, R., and Sommer, W., “Trans-saccadic parafoveal preview benefits in fluent reading: A study with fixation-related brain potentials,” NeuroImage, 62, 381–393 (2012).
Dimigen, O., Sommer, W., Hohlfeld, A., et al., “Coregistration of eye movements and EEG in natural reading: Analyses and review,” J. Exp. Psychol. Gen., 140, 552–572 (2011).
Duchowski, A. T., “Diversity and types of eye tracking applications,” in: Eye Tracking Methodology (2017), pp. 247–248.
Eberhard, K. M., Spivey-Knowlton, M. J., Sedivy, J. C., and Tanenhaus, M. K., “Eye movements as a window into real-time spoken language comprehension in natural contexts,” J. Psycholinguist. Res., 24, No. 6, 409–436 (1995).
Ehrlich, K. and Rayner, K., “Pronoun assignment and semantic integration during reading: Eye movements and immediacy of processing,” J. Verbal Learn. Verbal Behav., 22, No. 1, 75–87 (1983).
Elgort, I., Brysbaert, M., Stevens, M., and Van Assche, E., “Contextual word learning during reading in a second language: An eye-movement study,” Stud. Second. Lang Acquis., 40, No. 2, 341–366 (2018).
Fairbanks, G., “The relation between eye-movements and voice in the oral reading of good and poor silent readers,” Psychol. Monogr., 48, No. 3, 78 (1937).
Feng, G., Miller, K., Shu, H., and Zhang, H., “Orthography and the development of reading processes: An eye-movement study of Chinese and English,” Child Dev., 80, No. 3, 720–735 (2009).
Fischer, T., Graupner, S. T., Velichkovsky, B. M., and Pannasch, S., “Attentional dynamics during free picture viewing: evidence from oculomotor behavior and electrocortical activity,” Front. Syst. Neurosci., 7, 17 (2013).
Friederici, A. D. and Weissenborn, J., “Mapping sentence form onto meaning: The syntax– semantic interface,” Brain Res., 1146, 50–58 (2007).
Gleitman, L. R., January, D., Nappa, R., and Trueswell, J. C., “On the give and take between event apprehension and utterance formulation,” J. Mem. Lang., 57, No. 4, 544–569 (2007).
Godfroid, A., Ahn, J., Choi, I., et al., “Incidental vocabulary learning in a natural reading context: an eye tracking study,” Bilingualism Lang. Cogn., 21, No. 3, 563–584 (2018).
Godfroid, A., Boers, F., and Housen, A., “An eye for words: Gauging the role of attention in incidental L2 vocabulary acquisition by means of eye tracking,” Stud. Second. Lang Acquis., 35, No. 3, 483–517 (2013).
Griffin, Z. M. and Bock, K., “What the eyes say about speaking,” Psychol. Sci., 11, No. 4, 274–279 (2000).
Griffin, Z. M. and Weinstein-Tull, J., “Conceptual structure modulates structural priming in the production of complex sentences,” J. Mem. Lang., 49, No. 4, 537–555 (2003).
Griffin, Z. M., “Gaze durations during speech reflect word selection and phonological encoding,” Cognition, 82, No. 1, B1–B14 (2001).
Henderson, J. M., Luke, S. G., Schmidt, J., and Richards, J. E., “Co-registration of eye movements and event-related potentials in connected-text paragraph reading,” Front. Syst. Neurosci., 7, 28 (2013).
Himmelstoss, N. A., Schuster, S., Hutzler, F., et al., “Co-registration of eye movements and neuroimaging for studying contextual predictions in natural reading,” Lang. Cogn. Neurosci., 1–18 (2019).
Holmqvist, K., Nyström, M., Andersson, R., et al., Eye Tracking: A Comprehensive Guide to Methods and Measures, Oxford University Press, Oxford (2011).
Huestegge, L., Radach, R., Corbic, D., and Huestegge, S. M., “Oculomotor and linguistic determinants of reading development: A longitudinal study,” Vision Res., 49, No. 24, 2948–2959 (2009).
Huettig, F. and McQueen, J. M., “The tug of war between phonological, semantic and shape information in language-mediated visual search,” J. Mem. Lang., 57, No. 4, 460– 482 (2007).
Huettig, F., Rommers, J., and Meyer, A. S., “Using the visual world paradigm to study language processing: A review and critical evaluation,” Acta Psychol., 137, No. 2), 151–171 (2011).
Huey, E. B., The Psychology and Pedagogy of Reading, The Macmillan Company (1908).
Hutzler, F., Braun, M., Võ, M. L. H., et al., “Welcome to the real world: validating fixation-related brain potentials for ecologically valid settings,” Brain Res., 1172, 124–129 (2007).
Ille, N., Berg, P., and Scherg, M., “Artifact correction of the ongoing EEG using spatial filters based on artifact and brain signal topographies,” J. Clin. Neurophysiol., 82, 242–253 (2002).
Javal, E., “Essai sur la physiologie de la lecture,” Annaels d’Ocilistique, 82, 242–253 (1879).
Joseph, H. S., Wonnacott, E., Forbes, P., and Nation, K., “Becoming a written word: Eye movements reveal order of acquisition effects following incidental exposure to new words during silent reading,” Cognition, 133, No. 1, 238–248 (2014).
Ju, M. and Luce, P. A., “Falling on sensitive ears: Constraints on bilingual lexical activation,” Psychol. Sci., 15, No. 5, 314–318 (2004).
Juhasz, B. J., White, S. J., Liversedge, S. P., and Rayner, K., “Eye movements and the use of parafoveal word length information in reading,” J. Experim. Psychol. Hum. Percept. Perform., 34, No. 6, 1560 (2008).
Just, M. A. and Carpenter, P. A., “A theory of reading: From eye fixations to comprehension,” Psychol. Rev., 87, No. 4, 329 (1980).
Kennedy, A., “Parafoveal processing in word recognition,” Q. J. Exp. Psychol. A, 53, No. 2, 429–455 (2000).
Knoeferle, P. and Crocker, M. W., “Incremental effects of mismatch during picture-sentence integration: Evidence from eye tracking,” in: Proc. 26th Ann. Conf. of the Cognitive Science Society (2005), pp. 1166–1171.
Knoeferle, P., Crocker, M. W., Scheepers, C., and Pickering, M. J., “The influence of the immediate visual context on incremental thematic role-assignment: Evidence from eye-movements in depicted events,” Cognition, 95, No. 1, 95–127 (2005).
Kornrumpf, B., Dimigen, O., and Sommer, W., “Lateralization of posterior alpha EEG reflects the distribution of spatial attention during saccadic reading,” Psychophysiology, 54, No. 6, 809–823 (2017).
Koval, N. G., “Testing the deficient processing account of the spacing effect in second language vocabulary learning: Evidence from eye tracking,” Appl. Psycholinguist., 40, No. 5, 1103–1139 (2019).
Kretzschmar, F., Bornkessel-Schlesewsky, I., and Schlesewsky, M., “Parafoveal versus foveal N400s dissociate spreading activation from contextual fit,” NeuroReport, 20, 1613–1618 (2009).
Kretzschmar, F., Schlesewsky, M., and Staub, A., “Dissociating word frequency and predictability effects in reading: Evidence from coregistration of eye movements and EEG,” J. Exp. Psychol. Learn. Mem. Cogn., 41, No. 6, 1648 (2015).
Kutas, M. and Federmeier, K. D., “Thirty years and counting: finding meaning in the N400 component of the event-related brain potential (ERP),” Ann. Rev. Psychol., 62, 621–647 (2011).
Kutas, M., Van Petten, C. K., and Kluender, R., “Psycholinguistics electrified II (1994–2005),” in: Handbook of Psycholinguistics. Academic Press (2006), pp. 659–724.
Laeng, B., Sirois, S., and Gredebäck, G., “Pupillometry: A window to the preconscious?” Perspect. Psychol. Sci., 7, No. 1, 18–27 (2012).
Lewis, A. G., Wang, L., and Bastiaansen, M., “Fast oscillatory dynamics during language comprehension: Unification versus maintenance and prediction?” Brain Lang., 148, 51–63
Li, L., Marinus, E., Castles, A., et al., “Eye tracking the effect of semantic decoding on orthographic learning in Chinese,” PsyArchiv Preprints (2019).
Li, N., Niefind, F., Wang, S., et al., “Parafoveal processing in reading Chinese sentences: Evidence from event-related brain potentials,” Psychophysiology, 52, No. 10, 1361–1374 (2015).
Libben, M. R. and Titone, D. A., “Bilingual lexical access in context: evidence from eye movements during reading,” J. Exp. Psychol. Learn. Mem. Cogn., 35, No. 2, 381 (2009).
López-Peréz, P. J., Dampuré, J., Hernández-Cabrera, J. A., and Barber, H. A., “Semantic parafoveal-on-foveal effects and preview benefits in reading: Evidence from Fixation Related Potentials,” Brain Lang., 162, 29–34 (2016).
Lowell, R. and Morris, R. K., “Word length effects on novel words: Evidence from eye movements,” Percept. Psychophys., 76, No. 1, 179–189 (2014).
MacGregor, L. J., Pulvermüller, F., Van Casteren, M., and Shtyrov, Y., “Ultrarapid access to words in the brain,” Nat. Commun., 3, 711 (2012).
Marian, V. and Spivey, M., “Competing activation in bilingual language processing: Within– and between-language competition,” Bilingualism Lang. Cogn., 6, No. 2, 97–115 (2003).
Marian, V., Spivey, M., and Hirsch, J., “Shared and separate systems in bilingual language processing: Converging evidence from eyetracking and brain imaging,” Brain Lang., 86, No. 1, 70–82 (2003).
Mathôt, S., “Pupillometry: Psychology, physiology, and function,” J. Cogn., 1, No. 1 (2018).
McConkie, G. W., Zola, D., Grimes, J., et al., “Children’s eye movements during reading,” Vis. Visual Dyslexia, 13, 251–262 (1991).
Metzner, P., von der Malsburg, T., Vasishth, S., and Rösler, F., “Brain responses to world knowledge violations: A comparison of stimulusand fixation-triggered event-related potentials and neural oscillations,” J. Cogn. Neurosci., 27, No. 5, 1017–1028 (2015).
Meyer, A. S., Sleiderink, A. M., and W. J. Levelt, “Viewing and naming objects: Eye movements during noun phrase production,” Cognition, 66, No. 2, B25–B33 (1988).
Miles, W. R., “Ocular dominance in human adults,” J. Gen. Psychol., 3, No. 3, 412–430 (1930).
Mohamed, A. A., “Exposure frequency in L2 reading: An eye-movement perspective of incidental vocabulary learning,” Stud. Second. Lang Acquis., 40, No. 2, 269–293 (2018).
Nikolaev, A. R., Jurica, P., Nakatani, C., et al., “Visual encoding and fixation target selection in free viewing: presaccadic brain potentials,” Front. Syst. Neurosci., 7, 26 (2013).
Nikolaev, A. R., Meghanathan, R. N., and van Leeuwen, C., “Combining EEG and eye movement recording in free viewing: Pitfalls and possibilities,” Brain Cogn., 107, 55–83 (2016).
Nikolaev, A. R., Nakatani, C., Plomp, G., et al., “Eye fixation-related potentials in free viewing identify encoding failures in change detection,” NeuroImage, 56, 1598–1607 (2011).
Nyström, M., Andersson, R., Holmqvist, K., and Van De Weijer, J., “The influence of calibration method and eye physiology on eyetracking data quality,” Behav. Res. Meth., 45, No. 1, 272–288 (2013).
Pellicer-Sánchez, A., “Incidental L2 vocabulary acquisition from and while reading: An eye tracking study,” Stud. Second. Lang Acquis., 38, No. 1, 97–130 (2016).
Pickering, M. J., Frisson, S., McElree, B., and Traxler, M. J., “Eye movements and semantic compositionm,” The On-Line Study of Sentence Comprehension: Eyetracking, ERP, and Beyond (2004), pp. 33–50.
Picton, T. W., van Roon, P., Armilio, M. L., et al., “The correction of ocular artifacts: a topographic perspective,” Clin. Neurophysiol., 111, No. 1, 53–65 (2000).
Raney, G. E., Campbell, S. J., and Bovee, J. C., “Using eye movements to evaluate the cognitive processes involved in text comprehension,” J. Vis. Exp., 83, (2014).
Rayner, K. and Duffy, S. A., “Lexical complexity and fixation times in reading: Effects of word frequency, verb complexity, and lexical ambiguity,” Mem. Cogn., 14, No. 3, 191–201 (1986).
Rayner, K., “Eye movements and the perceptual span in beginning and skilled readers,” J. Exp. Child Psychol., 41, No. 2, 211–236 (1986).
Rayner, K., “Eye movements in reading and information processing: 20 years of research,” Psychol. Bull., 124, No. 3, 372 (1998).
Rayner, K., Pollatsek, A., and Reisberg, D., “Basic processes in reading,” in: The Oxford Handbook of Cognitive Psychology, Oxford University Press, New York (2013), pp. 442–461.
Rayner, K., Sereno, S. C., and Raney, G. E., “Eye movement control in reading: a comparison of two types of models,” J. Experim. Psychol. Hum. Percept. Perform., 22, No. 5, 1188 (1996).
Rayner, K., Slattery, T. J., Drieghe, D., and Liversedge, S. P., “Eye movements and word skipping during reading: effects of word length and predictability,” J. Experim. Psychol. Hum. Percept. Perform., 37, No. 2, 514 (2011).
Reichle, E. D. and Perfetti, C. A., “Morphology in word identification: A word-experience model that accounts for morpheme frequency effects,” Sci. Stud. Read., 7, No. 3, 219–237 (2003).
Sereno, S. C. and Rayner, K., “Measuring word recognition in reading: eye movements and event-related potentials,” Trends Cogn. Sci., 7, 489–493 (2003).
Sereno, S. C., Rayner, K., and Posner, M. I., “Establishing a time-line of word recognition: evidence from eye movements and event-related potentials,” Neuroreport, 9, 2195–2200 (1998).
Serov, I. E., Kovaleva, A. V., Kaz’mina, V. G., and Galkina, N. V., “Prediction of the probability of remembering the logotypes of unknown brands assessed using objective electrophysiological indicators,” Praktich. Market., 5, No. 267, 3–10 (2019).
Shtyrov, Y. and Lenzen, M., “First-pass neocortical processing of spoken language takes only 30 msec: Electrophysiological evidence,” Cogn. Neurosci., 8, No. 1, 24–38 (2017).
Siegel, M., Donner, T. H., and Engel, A. K., “Spectral fingerprints of largescale neuronal interactions,” Nat. Rev. Neurosci., 13, No. 2, 121 (2012).
Simola, J., Holmqvist, K., and Lindgren, M., “Right visual field advantage in parafoveal processing: Evidence from eye-fixation-related potentials,” Brain Lang., 111, No. 2, 101–113 (2009).
Simola, J., Le Fevre, K., Torniainen, J., and Baccino, T., “Affective processing in natural scene viewing: Valence and arousal interactions in eye-fixation-related potentials,” NeuroImage, 106, 21–33 (2015).
Simola, J., Torniainen, J., Moisala, M., et al., “Eye movement related brain responses to emotional scenes during free viewing,” Front. Syst. Neurosci., 7, 41 (2013).
Singer, W., “Dynamic formation of functional networks by synchronization,” Neuron, 69, No. 2, 191–193 (2011).
Starr, M. and Inhoff, A., “Attention allocation to the right and left of a fixated word: Use of orthographic information from multiple words during reading,” Eur. J. Cogn. Psychol., 16, No. 1–2, 203–225 (2004).
Staub, A. and Rayner, K., “Eye movements and on-line comprehension processes,” The Oxford Handbook of Psycholinguistics, Oxford University Press (2007), pp. 327–342.
Takeda, Y., Sugai, M., and Yagi, A., “Eye fixation related potentials in a proof reading task,” Int. J. Psychophysiol., 40, No. 3, 181–186 (2001).
Tanenhaus, M. K. and Trueswell, J. C., “Eye movements and spoken language comprehension,” in: Handbook of Psycholinguistics, Academic Press (2006), pp. 863–900.
Tanenhaus, M. K., Spivey-Knowlton, M. J., Eberhard, K. M., and Sedivy, J. C., “Integration of visual and linguistic information in spoken language comprehension,” Science, 268, No. 5217), 1632–1634 (1995).
Tanenhaus, M. K., Spivey-Knowlton, M. J., Eberhard, K. M., and Sedivy, J. C., “Using eye movements to study spoken language comprehension: Evidence for visually mediated incremental interpretation,” in: Attention and Performance XVI: Information Integration in Perception and Communication, MIT Press (1996).
Vignali, L., Himmelstoss, N. A., Hawelka, S., et al., “Oscillatory brain dynamics during sentence reading: a fixation-related spectral perturbation analysis,” Front. Hum. Neurosci., 10, 191 (2016).
Von Stein, A., Chiang, C., and König, P., “Top-down processing mediated by interareal synchronization,” Proc. Natl. Acad. Sci. USA, 97, No. 26, 14,748–14,753 (2000).
Weber, A., Melinger, A., and Tapia, L., “The mapping of phonetic information to lexical presentations in Spanish: evidence from eye movements,” in: 16th Int. Congr. of Phonetic Sciences (ICPhS 2007) (2007), pp. 1941–1944.
Wochna, K. L. and Juhasz, B. J., “Context length and reading novel words: An eye-movement investigation,” Br. J. Psychol., 104, No. 3, 347–363 (2013).
Yarbus, A. L., Role of Eye Movements in the Visual Process (1965a).
Yarbus, A. L., The Role of Eye Movements in the Process of Vision, Nauka, Moscow (1965b).
Author information
Authors and Affiliations
Corresponding author
Additional information
Translated from Zhurnal Vysshei Nervnoi Deyatel’nosti imeni I. P. Pavlova, Vol. 72, No. 5, pp. 609–622, September–October, 2022.
Rights and permissions
Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.
About this article
Cite this article
Pokhoday, M.Y., Bermudes-Margaretto, B., Shtyrov, Y.Y. et al. Eye Tracking Methods in Psycholinguistics and Parallel EEG Recording. Neurosci Behav Physi 53, 220–229 (2023). https://doi.org/10.1007/s11055-023-01411-x
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11055-023-01411-x