Introduction

Amyotrophic lateral sclerosis (ALS) is a multi-system neurodegenerative disorder characterized by a progressive decline of physical mobility and respiratory functioning, which may lead to loss of both verbal and written communication capacities in some patients [1]. Recent studies suggest a distinct pattern of stages in which ALS pathology progresses in the brain [2] and reveal a substantial clinical, pathological, and genetic overlap with frontotemporal dementia [3]. About 30 % of patients with ALS exhibit cognitive impairments most prominently characterized by deficits in executive functioning, language abilities and verbal fluency [4–6]. The relevance of these symptoms for clinical practice in the domains of compliance with medical interventions [7], survival [8] and carer burden [9, 10] highlights the importance of reliable neuropsychological assessment.

Progressing motor impairments are a major obstacle in neuropsychological assessments of patients in an advanced state of the disease [11]. Although some aspects of oculomotor control might be impaired in patients with ALS [12], eye tracking is still a very promising way to study executive functioning in ALS, which has already been used in healthy subjects: One study has implemented an oculomotor version of the trail-making test and found a strong correlation between the paper–pencil and the eye-tracking-based performance of healthy subjects in one subtest [13] whereas another has demonstrated the usability of an eye-tracking-based verbal fluency task [14].

The aim of the current study was therefore to establish a method by which cognitive functions of ALS patients can reliably be determined in a motor-free test, based on eye movements only.

For proof of principle, we utilized two well-validated measures of executive functioning, easy to implement in a hand and speech motor-free version: the Raven’s coloured progressive matrices (CPM) [15] and the D2-test [16], both applicable in ALS [17, 18] and other neurological conditions [19, 20]. The CPM is a non-verbal test of fluid intelligence, visuospatial reasoning and executive functioning whose paper–pencil version has shown to be effective in revealing cognitive deficits among non-demented ALS patients [21, 22]. The D2-test is a non-verbal test for executive dysfunction in the domains of selective and sustained attention and visual processing speed.

We hypothesized that differences in cognitive abilities can be reproduced between and within patients and healthy controls and that performance scores in both settings show an intimate correlation.

Materials and methods

Participants

Forty-eight patients (n = 20 females), all diagnosed with sporadic ALS by a board-certified neurologist according to the Revised El Escorial criteria, [23] were recruited from the Department of Neurology at the Universitätsklinikum Ulm, Germany. Additionally, thirty-two healthy, age-, sex-, and education-matched controls (HC; n = 17 females) were recruited. None of the participants had any signs of neurological or psychiatric illness (other than ALS), major cognitive impairments, substantial behavioral alterations or visual impairments which might alter task performance.

The study was approved by the Ethics Committee of the University of Ulm (Statement No. 19/12) and was performed in accordance with the ethical standards laid down in the 1964 Declaration of Helsinki and its later amendments. All participants gave informed consent to the study.

Design

The ALS Functional Rating Scale revised version (ALS-FRS) [24] was used to get a measure of patients’ physical impairments. For general cognitive screening, the German version of the Edinburgh cognitive and behavioral ALS screen (ECAS) [25], assessing memory, visuospatial perception, language, verbal fluency and executive functioning, was administered. Subsequently, all participants completed both, the original and the motor-free version of the CPM and D2-test. To avoid sequence effects, controls and patients were subdivided into two groups matched for age, sex, education and if applicable disease duration. One group completed the oculomotor before the paper–pencil version of the neuropsychological tests, the other did the paper–pencil versions first (Fig. 1).

Fig. 1
figure 1

Pseudo-randomized study design. Both cohorts, ALS patients (upper left panel) and healthy controls (upper right panel) were pseudo-randomly separated into two matched subgroups to control for possible learning effects. Subgroup 1 performed eye-tracking testing followed by the paper–pencil version and subgroup 2 vice versa. Data from eye-tracking testing and the paper–pencil version for the two subgroups were pooled for ALS patients and controls for statistical data analysis (lower panel)

Full size image

Motor-dependent classical versions

Paper–pencil CPM

The CPM was administered according to standard protocol [15], using only set A and set B. In each set, participants had to choose 12 times between 6 possibilities to logically complete a given “matrix” in which a part was missing. We recorded each choice and subsequently computed the percentage of correct answers.

Paper–pencil D2-test

We used the 8th revised version of the D2-test according to standard procedures [16]. The participant had 20 s to tick as many targets (a “d” with two dashes) as possible in each line (47 stimuli of “d” or “p” with 1–4 dashes) before being instructed to move to the next line. Lines 2–6 were considered for analysis only (corresponding to oculomotor version, see below). “KL-value” as the number of correctly identified targets minus the number of incorrect ticks per line was used to get an estimate of performance accuracy ranging from 0 to 100 %.

Hands- and speech-free oculomotor versions

Oculomotor testing took place in an acoustically shielded, darkened room with participants comfortably seated in the center of a hemi-cylindrical screen (for details see [26, 27]). The distance between subjects’ eyes and screen was approximately 150 cm. A chin rest was used to stabilize head position and to avoid artificial movements. Stimuli were presented with a TOSHIBA® TDP-EX20 projector mounted above the subjects head and with a lens-to-screen distance of 150 cm. Eye movements were recorded using the portable video-oculography EyeSeeCam® device (EyeSeeTec GmbH, Fürstenfeldbruck, Germany) that measures binocular eye positions synchronously with 0.02° spatial resolution at a temporal sampling rate of 220 Hz [28]. An interactive MATLAB® (The Mathworks Inc., Natick, MA, USA)-based in-house software package OculoMotor Analysis [26, 27, 29] was used for analysis of eye movement recordings. The calibration procedures require the subject to track a ‘slow’ sinusoidal single-spot target oscillation (horizontal range ±20°; vertical range ±15°, f = 0.125 Hz) to map the non-calibrated orthogonalized ‘raw’ data from the EyeSeeCam® device with respect to the ‘true’ orthogonalized eye position. Neither the patient group nor the control group exhibited systematic differences between the right and the left eye, hence, the binocular recording was merged into a cyclopean signal [30]. After calibration procedure, the CPM followed by the D2-test were performed. Since all tasks were subsequently conducted with the subject and video-oculography device being in the same position, no re-calibration was required.

Oculomotor CPM

Corresponding to the paper–pencil version, set A and B was used (each 12 stimuli). For training purpose, participants completed trials 1 through 4 from set AB to get used to the procedure. All stimuli depicted a 22° long/15° high “matrix” of which a piece of approximately 6° in width and 5° in height was cut-out as well as the six possible alternatives which all had the same dimension as the cut-out piece of the matrix (Fig. 2). In-house developed real-time software recorded the participants’ choices and stored them in a separate text file to allow for computation of the percentage of correct answers.

Fig. 2
figure 2

Illustration of eye-tracking-based CPM. Example of the CPM selection procedure as displayed on the screen (upper panel) with its corresponding traces of horizontal (upper line) and vertical (lower line) eye positions (lower panel). Subjects had infinite time to look at the matrix and choose the missing part of each one (a). When they had internally decided for a choice, start off of a green frame was triggered by subjects closing their eyes for at least 250 ms (b). The green frame consecutively outlined each of the six possible patterns for 1500 ms. When a subject thought that it was around the correct pattern, he/she closed his/her eyes for minimum 250 ms to choose it (c). The selection was presented separately and if the subject agreed on the selection, he/she had to close their eyes again for at least 250 ms (d). Then, the next stimulus was presented

Full size image

Oculomotor D2-test

Technical setup was identical to the oculomotor CPM testing with an additional red laser spot (0.3° in diameter) at +10° vertical, being constantly present. Subjects were presented 5 blocks of 47 stimuli, each corresponding to line 2–6 of the paper–pencil version. The first line of the paper–pencil version was used as a training-block before the recording started. Each stimulus was presented for 2000 ms in the center of the screen and measured 11° in height and 2.5° in width. Participants were instructed to direct their gaze to the red spot each time they saw one of the target stimuli (a “d” with two dashes), whilst remaining their gaze focused on the middle of the screen during any other stimulus. The recorded eye movements were offline analyzed using an empiric threshold for vertical eye movements exceeding of 5° indicating that the subject recognized a target stimulus. All records were visually inspected by a trained person to minimize false detection. Eye movement parameters were manually assigned to each stimulus presentation to determine performance accuracy.

Quality control

The learning sessions for each subject performed before both the CPM and D2-test were carefully inspected by a trained oculomotor specialist (M.G.) to control for possible confounding factors comprising corrupted eye movement recordings, misunderstanding, or any considerable ‘genuine’ oculomotor deficits such as markedly slowed saccades or prolonged reaction times. Oculomotor data of one patient in the CPM and two patients in the D2-test were discarded from further analyses due to technical artifacts. Also, data of one control subject in the paper–pencil D2-test was discarded due to misunderstanding of the task instructions.

Statistical analysis

All analyses were performed using IBM® SPSS Version 21.0. Non-parametric Mann–Whitney U tests were used to detect group differences in performance accuracy between patients and HC. This procedure was also performed for comparisons in the CPM and D2-test of more and less cognitively impaired patients who were below and above the ECAS-median of all patients. Within patients and controls, Spearman-Rho correlation analyses were conducted to determine correlation coefficients between the paper–pencil and the oculomotor version of the CPM and D2. All analyses were two-sided and the significance level was set at p < 0.050.

Results

General cognitive screening

When compared to healthy controls, patients scored significantly worse in the language (p = 0.006), the executive function (p = 0.001) and the visuospatial (p = 0.049) domain of the ECAS and with regards to the overall ECAS-score (p = 0.012). Scores of the other domains (memory and verbal fluency) demonstrated no significant differences between groups. For detailed results and sample characteristics see Table 1.

Table 1 Demographic and clinical information
Full size table

Congruence of oculomotor and paper–pencil CPM

To determine whether performance accuracy of the oculomotor CPM could predict performance accuracy of the written version, a Spearman-Rho correlation analysis was performed, showing a significant correlation between the percentage of correct answers in the motor and motor-free version of patients (Spearman-Rho-R 2 = 0.712; p < 0.001), and HC (Spearman-Rho-R 2 = 0.610; p < 0.001) (Fig. 3), as well as for both groups together (Spearman-Rho-R 2 = 0.680; p < 0.001).

Fig. 3
figure 3

Correlation between results from paper–pencil and oculomotor condition of the CPM. Shown is the percentage of correct answers in the oculomotor (x-axis) and the paper–pencil (y-axis) condition of the CPM in ALS patients (a) and healthy controls (b)

Full size image

Congruence of oculomotor and paper–pencil D2-test

The Spearman-Rho correlation analysis showed a significant association between performance accuracy in the paper–pencil and oculomotor condition for HC (Spearman-Rho-R 2 = 0.128; p = 0.048) and both groups together (Spearman-Rho-R 2 = 0.078; p = 0.014). For ALS patients there was a trend which did not reach the threshold for significance (Spearman-Rho-R 2 = 0.062; p = 0.096).

Between group analyses CPM

In the paper–pencil condition, the median of correct answers of the control group was at 91.7 % as compared to 87.5 % in the oculomotor condition. The median of ALS patients’ performance accuracy was 83.3 % in the paper–pencil as well as in the oculomotor condition. A Mann–Whitney U test revealed significant differences between HC and patients in the paper–pencil (p = 0.042) and a trend bordering the level of statistical significance in the oculomotor condition (p = 0.053).

Between group analyses D2-test

The median of HC performance accuracy was 91.1 % in the motor and 98.1 % in the motor-free condition. ALS patients’ median was 87.8 % in the paper–pencil as compared to 98.1 % in the oculomotor condition of the D2-test. Mann–Whitney U test demonstrated no significant group differences in the oculomotor condition (p = 0.676), whereas a trend was observed in the paper–pencil condition (p = 0.082).

Patients ECAS-median-split

In the patient sample, a median-split on the ECAS-Score was performed to differentiate between more and less cognitively impaired patients. Using a Mann–Whitney U test, substantial differences in the percentage of correct answers in the paper–pencil (p < 0.001) and oculomotor (p < 0.001) version of the CPM between those patients with an ECAS-Score below the median and those with an ECAS-Score above it were discovered. When performing the same analysis in the D2-test, a significant difference in performance was found in the oculomotor condition (p = 0.024) but not in the paper–pencil condition (p = 0.322) (Fig. 4).

Fig. 4
figure 4

Comparison of results of patients below and above the ECAS-median in CPM and D2-test. Shown are box plots of performance accuracy among ALS patients above the ECAS-median in the oculomotor and the paper–pencil condition as well as of those below the ECAS-median, accordingly. Mann–Whitney U test revealed significant group differences in the CPM in both conditions and the oculomotor condition of the D2-test. All patients were within the whisker range

Full size image

Congruence of executive function as measured by the ECAS and oculomotor performance

Further analyses between the overall ECAS-score and the oculomotor performance accuracy of ALS patients revealed significant correlations between the ECAS total score and the CPM (Spearman-Rho-R 2 = 0.294; p < 0.001) as well as the D2-test (Spearman-Rho-R 2 = 0.128; p = 0.015). The executive function-subscore of the ECAS and oculomotor performance accuracy significantly correlated in both oculomotor tasks (CPM: Spearman-Rho-R 2 = 0.205; p = 0.001; D2-test: Spearman-Rho-R 2 = 0.097; p = 0.035).

Sensitivity and specificity of the oculomotor tasks

When using a cut-off score for cognitive impairment, defined as two standard deviations below the mean of our healthy control sample [6] in the oculomotor CPM and D2-test and patients’ ECAS total score as a reference, a specificity of 92 % was found for both tasks. The sensitivity was 44 % for the oculomotor CPM and 38 % for the oculomotor D2-test.

Influence of physical impairment on cognitive performance

Correlating patients’ ALS-FRS-Score as a measure of physical function decline with their performance in the CPM yielded non-significant results for the oculomotor (Spearman-Rho-R 2 = 0.028; p = 0.262) and for the paper–pencil condition (Spearman-Rho-R 2 = 0.017; p = 0.380) Similarly, no correlation between patients physical function decline and percentage of correct answers in the D2-test was observed (oculomotor: Spearman-Rho-R 2 = 0.007; p = 0.571 and paper–pencil: Spearman-Rho-R 2 = 0.028; p = 0.267).

Discussion

This study provides evidence that a ‘speech-free and motor-free’ eye-tracking version of the CPM and the D2-test can reliably assess cognitive functions in patients with ALS. By comparing classical paper–pencil and hand and speech motor-free oculomotor methods of neuropsychological tests, executive functions were determined using eye-tracking devices in ALS patients. Previous studies have shown the potential of oculomotor-based neuropsychological testing in healthy subjects [13, 14]. We successfully demonstrated the usability of such approaches in a clinical context. The strong link between paper–pencil and eye-tracking-based performance in neuropsychological tests, which has been reported by Hicks et al. [13], was also found in our sample and further supports previous statements that eye-tracking-based neuropsychological tests can substitute standard paper and pencil tests in highly physically impaired patients.

As expected, ALS patients performed significantly worse than HC on the overall ECAS-score, the language subscore and the executive function-subscore. There was, however, no significant group difference in the verbal fluency domain. Although verbal fluency deficits are usually common in ALS patients [4], it may vary between different subpopulations. Therefore, this finding is in accordance with recent studies in different ALS populations, where language abilities and attention were most prominently impaired, whereas verbal fluency was impaired in a smaller number of patients [25, 31].

The advantages of the CPM for motor-free cognitive screening are apparent: easy to understand, convenient to implement in an eye-tracking controlled setting, well-validated and widely used to assess executive abilities [32]. Even though some studies have used the CPM as an indicator for visuospatial dysfunctions and found impairment in only very few cases [22], ALS patients performed worse in the CPM than HC in this study. This is also in accordance with other studies using this test as a tool for non-verbal executive reasoning in ALS patients [21]. The main advantage of the CPM seems to be its robustness among different approaches of application, as a strong correlation between performance in the motor and motor-free variants could be observed in both groups. Most importantly, we also found significant group differences in the CPM between more and less cognitively impaired patients as determined by a median-split of their ECAS-score, implying the oculomotor version to be a good marker for general cognitive deficits in ALS. Overall, oculomotor CPM seems to be a very practical and reliable screening tool for executive deficits in physically impaired ALS patients.

Contrary to our hypothesis, there were no robust group differences between HC and ALS patients in the D2-test performance. This is in accordance with a previous study [18] and might be explained by the fact that the D2-test is relatively easy [33]. It might therefore not reliably distinguish between HC and our patient sample with no severe cognitive impairment on the group level. Also, a significant correlation between performance accuracy in the motor-dependent and motor-free variant was observed only for HC and both groups combined. The lack of a linear association between both versions in the patient group is most likely due to the fact that the oculomotor version of the D2-test is best suited to detect impairments in overall cognitive performance. In the patient group a subgroup of cognitively impaired patients accounted for a skewed distribution in this group which may have confounded the results of the linear correlation analysis. However, the eye-tracking version of the D2-test still proves to be very successful in what should be considered its main task, i.e., distinguishing between patients with more and less cognitive dysfunctions. Accordingly, the motor-free version of the D2-test requires additional cognitive resources apart from attention. For severely impaired patients, this oculomotor version of the D2-test might provide a simple and fast means for general cognitive screening, whereas the oculomotor version of the CPM is a dependable and fast method for detecting more specific cognitive changes in patients in an advanced stage of the disease with regards to executive functioning.

As an additional validation measure, a correlation analysis between the overall ECAS-score and the executive functioning-subscore of the ECAS and the performance accuracies in both oculomotor tasks revealed statistically significant correlations in our patient sample. Also, a very satisfying specificity of 92 % was found for both oculomotor tasks. This further supports our point that our eye-tracking paradigms are suited to detect executive deficits in severely impaired ALS patients.

Physical function decline was not associated with cognitive performance as measured with the oculomotor test versions and therefore, possible effects of physical impairment on test performance can be excluded. This is in accordance with our hypothesis and further supported by previous studies [34]. It also implies that physical function decline does not necessarily indicate impairment of cognitive functioning; however, for far advanced patients with long disease duration only sparse data on cognitive state is available, further highlighting the need for a reliable method of investigating cognitive profiles of severely impaired ALS patients.

Limitations of this study include the relatively homogenous sample of ALS patients lacking patients in a late stage of the disease. This is a method immanent limitation in a proof-of-principle study to secure reliable measures both in the paper–pencil and the eye-tracking domain. Yet, as Mioshi et al. have already described, neuropsychological abnormalities also occur in a very early stage of the disease and may even precede physical impairments [35]. Moreover, for completely locked-in state of ALS, the current study lacks the proof of applicability as correct eye movement control and the ability to blink in this state is not given by definition [14]. Then, additional Brain–Computer Interface technology is needed, which has recently been shown to be an effective means of communication in completely locked-in ALS patients [36]. A further limitation of our study is the possible impact of some oculomotor abnormalities in patients with ALS on our results that were not controlled for. Several studies [12, 37] reported impairment of eye movement control in patients with ALS. However, any deficits that might impact the performance of the study tasks were excluded by analyzing the data of the training session.

Moreover, we did not perform test–retest reliability analyses on the eye-tracking domain which should be addressed in future investigations to provide evidence for consistency. The usability of portable eye-tracking devices for severely impaired ALS patients, as has already been done in other neurological disorders [38] is another interesting aspect requiring further investigation. Additionally, more sophisticated eye-tracking controlled experimental procedures are warranted to investigate more subtle cognitive deficits in the physically impaired as well as individual distributions among patients.

As the detection of neuropsychological symptoms plays a pivotal role in clinical decisions and alleviating carer burden [39], the current approach provides the possibility for neuropsychological assessment in patients with major physical impairments—not necessarily limited to ALS patients—using a reliable, fast, easy to administer and very user-friendly method.