Introduction

According to the World Health Organization, stroke is the second leading cause of death worldwide, and the absolute number of individuals suffering from stroke is strongly increasing [1]. At the acute phase, patients may suffer from the so-called hyperacute cognitive stroke syndrome [2], which encompasses a broad spectrum of cognitive deficits depending on the dysfunction of complex and integrated brain networks. Cognitive impairments could be not directly caused by the lesion, rather than induced by the functional disconnection between anatomically distant preserved and damaged brain areas (diaschisis phenomenon). This clinical condition may result in a neuropsychological profile characterized by multiple deficits not necessarily associated with a single cognitive domain [3, 4].

Well-known cognitive syndromes typically associated with stroke are useful indicators of later disability. Linguistic symptoms (e.g., aphasia) and non-verbal impairments, such as unilateral spatial neglect, have been demonstrated to negatively impact the functional recovery of stroke patients [5, 6]. These disorders, in fact, reduce the level of patients’ collaboration during the clinical management and earlier rehabilitation treatments [7].

If on the one hand standardized neuropsychological tests require too much time and a level of cooperation lacking in the acute patients to be completed [8], on the other hand, available brief evaluations are generally borrowed from dementia screenings. The Mini-Mental State Examination (MMSE) [9], for example, is often used with acute stroke patients [10], as it is brief and easy to administer, although it may induce a bias in the evaluation towards cognitive domains typically dysfunctional in neurodegenerative disorders [5]. The very distinct clinical features of dementia and acute stroke require a more specific instrument when evaluating acute stroke patients involving a more comprehensive cognitive assessment providing both specific and global scores on the status of cognitive functioning.

Early cognitive diagnosis is fundamental to plan personalized rehabilitation programs [11]. Furthermore, it has been demonstrated that the domain-specific earlier identification of neuropsychological symptoms predicted problem-solving and social activities at 6–12 months post-stroke. More specifically, deficits of visuospatial perception, visual memory, and attention/executive functions were the stronger outcome predictors at 6–12 months post-stroke [12].

In the current study, we developed a brief and specific bedside screening test: the mental performance (in acute) stroke (MEPS). MEPS has been conceived as a brief bedside test to be also administered to patients with a low level of attention and collaboration, as frequently occurs in the very acute phase of stroke. MEPS assesses attention, language, memory, executive functions, praxis, and visuospatial functions providing single domains and global functioning scores. MEPS also provides useful information to the clinicians for the general care and treatments and indications for further detailed cognitive assessments. We here describe the validation of the screening with normative data, providing evidence on the sensitivity and specificity of the tool in a large sample of acute stroke patients and healthy controls.Footnote 1

Materials and methods

Participants

We firstly enrolled a sample of 27 stroke patients (16 males; age mean ± SD, 70.66 ± 11; education mean ± SD, 10.14 ± 4) in the acute phase (6 ± 3 days from the onset) to check for the MEPS validity.

Twelve patients presented with right and 11 with left brain damages. One patient showed a median cerebellar lesion. Patients were administered with (i) MEPS; (ii) the Montreal Cognitive Assessment (MOCA) [13]; (iii) Bedside Language Examination (ELLM) [14]; (iv) standard neuropsychological tests investigating the same cognitive domains as those included in MEPS (see Table 1). These tests were administered in a pseudo-randomized order across participants.

Table 1 MEPS subtests. Lists of all the subtests included in the MEPS. The third column represents both the minimum and the maximum score for each subtest. The fourth column indicates for each subtest if it was either taken from another test or created ex novo
Full size table

Furthermore, 204 patients were screened over a 5-year time period (from 2011 to 2016). Only patients who met the following inclusion criteria were included: (a) ischemic or hemorrhagic stroke documented by a TC or MRI scan; (b) acute phase (less than 30 days) of the neurological condition; (c) age > 18 years old; (d) absence of psychiatric history or comorbidity with other neurological disorders; (e) absence of drug abuse history; (f) absence of major organ impairments. The final sample was composed by 129 patients (89 women), with a mean age of 67.78 (SD = 13.93) and 8.68 years of education (SD = 4.26). On average, patients were 5.50 ± 3.71 days form the onset. The sample included 124 right-handed and 5 left-handed participants. All the patients were native Italian speakers with a normal or corrected-to-normal vision. Brain lesions involved the right hemisphere in 55 patients, while the left hemisphere was damaged in 53 patients. Eighteen patients showed bilateral brain lesions, while brain lesions of 3 patients were non-classifiable due to the negative CT.

A sample of 263 (148 women) healthy participants with a mean age of 52.61 (SD = 15.94), and 11.40 years of education (SD = 4.33), was also included. Seven healthy participants out of 263 were left-handed. All the participants were native Italian speakers, they had normal or corrected-to-normal vision, and they had no previous history of mental or neurological illness. The Mini-Mental State Examination (MMSE) [9] was administered to rule out any cognitive deficits.

This study was conducted in accordance with the Declaration of Helsinki. The experimental protocol received the ethical approval by the local Research Ethics Committee (study number 1549303). All the participants signed the informed consent prior to their participation in the study.

The mental performance in acute stroke

MEPS is composed by a set of 14 verbal and non-verbal subtests investigating spatial and temporal orientation, language comprehension, attention, and memory (see Table 1 for a list of items included in the MEPS). The items were created basing on the specific cognitive function (e.g., short-term memory). Some items were borrowed from existing standardized tests while others were created ex novo by a pilot study on both healthy subjects and patients (see Table 1).

To create the final version of MEPS, ten different parallel forms have been generated and administered on healthy subjects in a pilot study to select the items showing a ceiling effect in this population. As for the words repetition subtest, we firstly extracted from a lexical database of written Italian (CoLFIS; [20]) seven-letter words with a mean frequency equal to 25 (frequency range from 20 to 30). However, we decided that it would be more sensitive to explore the words by dividing them into two groups corresponding to high- and low-frequency words. We selected a total of 200 words balanced for living/non-living, male/female, and concrete/abstract, and we randomly extracted ten groups of 6 words each to create different parallel forms used in the pilot study. The participants were required to repeat, after the experimenter, one word a time.

As for the picture naming subtest, we modified the colored pictures of Snodgrass e Vanderwart [21] in a black-and-white fashion. The pictures, representing common objects with a high frequency of use, were divided into living and non-living items from which we randomly selected three items a time to obtain six balanced items to present to the participants. Therefore, we created ten groups of 6 pictures used in the pilot study. The participants were required to name each picture.

For the immediate visual memory subtest, we created three groups of matrices without edges composed of 4 × 4 squares (each square measures 1 × 1 cm). In each group, a different number (4, 5, or 6) of squares have been randomly blackened. Therefore, we administered to healthy participants all the stimuli to evaluate their difficulty level. The single matrix represented the target to be memorized and, after a brief presentation on an A4 sheet, it has to be recognized among two distractor matrices. The position of the target matrix and the distractors have been randomly chosen.

For the visual exploration and attention subtest, we created a subtest able to inform about the presence of any visual exploration impairment. Specifically, basing on the attentive matrices test [16] and the standard test for the neglect, each matrix is composed of eight rows and twenty columns with 28 target items among the distractors balanced across the quarters. In this way, the subject has to find the target in a larger horizontal space than the vertical one. During the subtest administration, the central point of the matrix has to coincide with the subject’s body midline and the total time of execution has to be collected. The matrix administration requires to search only one digit among the distractors, corresponding to number 5.

Basing on the Frontal Assessment Battery (FAB; [22]), the similarity subtest has been created by selecting pairs of words belonging to the same semantic category. From this sample of words, we extracted 10 groups of six pairs of words useful for the pilot study administration. The participants were required to indicate the semantic category for each presented pair of words.

The ideative apraxia subtest has been created by selecting objects with a high frequency of use, easy to transport, and easy to use at the bedside. We also controlled for different gestures’ execution to avoid any kind of interference between similar gestures. Participants are required to imitate the gesture associated with the target object. The order of administration has been evaluated by taking into account the complexity of the gestures.

Data analyses

Validity

Considering the sample of 27 stroke patients only, we performed Spearman’s correlation analyses between the MEPS (considering both the total score and the scores at each subtest) and all the other standard neuropsychological tests.

Difference based on demographic variables

By considering the entire experimental sample, we firstly tested whether MEPS total score and the individual subtests’ scores were able to differentiate between patients basing on the lesion side and gender. Then, we estimated the correlation between the MEPS subtests and age, time from lesion onset, and education. For gender and lesion side, we performed a series of Kruskal-Wallis tests with either gender or side as independent variable and each MEPS subtest, one at the time, as dependent variable.

Patient and healthy control discrimination

To test whether the MEPS subtests were able to discriminate between patients and healthy controls, we performed a logistic regression considering the age as a covariate.

Optimal cut-off and accuracy

A study of optimal cut-off has been conducted to establish which cut-off value of each MEPS subtest guaranteed the best accuracy in discriminating patients from healthy controls. Then, the sensitivity, specificity, and accuracy of each MEPS subtest parameter were computed for its optimal cut-off. An example of the optimal cut-off procedure is depicted in Fig. 1 for MEPS total score. The x-axis (bottom axis) represents all the possible cut-off proportions by which MEPS total score can be divided: For instance, 0.5 means that the cut-off corresponds to a median split (50% of cases above cut-off, 50 below) and 0.8 means that the cut-off divides the cases in 80% below and 20% above the cut-off. The top axis represents the actual value of MEPS total score corresponding to each possible cut-off. For instance, the value 70 in the MEPS total score corresponds to a cut-off which divides the sample in 20% below and 80% above the cut-off. The curve represents the accuracy of each cut-off value in discriminating the group (patients vs healthy controls). Based on this plot, the cut-off with the largest accuracy is chosen. In the example, a cut-off value of 79.9, corresponding to dividing the sample in 58% below and 42% above the cut-off, yields the largest accuracy. More intuitively, the optimal cut-off can be thought as the value of the scale which is able to discriminate between the distributions of the score of the two groups.

Fig. 1
figure 1

Optimal cut-off procedure. The x-axis (bottom) represents all the possible cut-off proportions by which MEPS total score can be divided. The x-axis (top) represents the MEPS total score. The y-axis represents the accuracy level of the cut-off in discriminating between patients and healthy controls (see the main text for a detailed description)

Full size image

Equivalent scores

Equivalent scores were computed based on the healthy controls’ distribution. They entail to estimate 4 cut-off values that divided the original scores into 5 groups: group 0, the worst performing group, whose scores were below the 5th percentile of the distribution; group 4, the group with normal performance, whose scores were above the median (50th percentile). The remaining 3 groups were formed by segmenting the scores between the 50th percentile and the 5th percentile in 3 segments of equal length. It should be noted that not all the subtests allowed for computing equivalent scores because there was not enough variability in the healthy control sample to identify different cut-offs for different equivalent scores. In that case, a score below the reported cut-off value has been considered as the cut-off of the lowest group. For example, for young people (age class 20–45; see Table 4), the subtest temporal orientation featured the same cut-off value (i.e., 5) for each equivalent score. This means that if a participant scores less than 5, it should be in the equivalent score group 0. The same reasoning has been applied to scales were there were only two distinct cut-off values. All the analyses were conducted considering the age classes separately.

Results

Validity

The results derived from the pilot study on 27 stroke patients indicated an adequate construct validity. Specifically, MOCA’s global score was significantly correlated with the MEPS’ total score (r = .833; p < .001). Furthermore, significant correlations emerged between MEPS subtests and MOCA subtests evaluating the same cognitive domains (see Table 2 for the detailed results). Both MEPS and MOCA showed a positive correlation with Barthel’s index (MOCA, r = .454, p = .017; MEPS, r = .691, p < .001), a measure of performance in activities of daily living [24]. Moreover, we found positive correlations between MEPS language subtests and ELLM subtests, as well as between MEPS and the other neuropsychological standard tests (see Table 2). However, we did not find significant correlations between MEPS ideative apraxia subtest and the ideative apraxia test (r = .215; p = .28), as well as between the MEPS visual discrimination subtest and the segment discrimination test (r = .323; p = 1).

Table 2 Validation results. The results from the correlation’ analyses performed between MEPS subtests (first column) and standard neuropsychological tests assessing the same cognitive domain (third column). The second and the fourth columns represent the score for each test (min/max). Spearman’s rho and p values are reported in the last two columns. The asterisk (*) indicates p < .05. Double asterisk (**) indicates p < .001
Full size table

Difference based on demographic variables

When considering the entire experimental sample, none of the scale showed an association with either the gender or the lesion side (see Tables S1 and S2 in the supplementary materials). However, MEPS subtests negatively correlated with age but not with education, except for the subtest of similarity judgments where a positive correlation was found (see Table 3).

Table 3 Correlation between MEPS, time from lesion onset, age, and education. Spearman’s rho is reported for each correlation. In italics are the highlighted results at p < .05
Full size table

Based on these results, patients seemed to be homogeneous in the tasks responses, allowing us to test their ability at the MEPS subtests to discriminate between patients and healthy subjects.

Patient and healthy control discrimination

All the MEPS subtests were able to discriminate patients from healthy controls (see Tables S4 and S5 in supplementary materials).

Optimal cut-off and accuracy

Basing on the optimal cut-off procedure described above (see “Data Analyses” section), we identify a value of 80 as a reasonable cut-off in separating the patient and healthy control distributions (see Fig. 2 and Table S6 in the supplementary material for cut-off, accuracy, sensitivity, and specificity).

Fig. 2
figure 2

Distribution of MEPS global score. Blue bars represent the patients’ global score distribution. Red bars represent the healthy controls’ global score distribution. The x-axis represents the actual MEPS global score. Y-axis represents the number of patients/healthy controls

Full size image

Equivalent scores

The following table reports the equivalent scores for the MEPS subtests divided by age classes (Table 4).

Table 4 Equivalent scores for the MEPS subtests divided by age classes
Full size table

Finally, we checked how the equivalent scores perform in classifying the patients. Since the cut-off values have been estimated on the healthy control group, one should expect the patients to aggregate more on the lower scores (i.e., groups 0, 1, and 2; see Tables S7, S8, and S9 in supplementary materials).

Discussion

MEPS can be considered a simple and highly valuable bedside battery, easy to administer, with values of sensitivity and specificity suitable to be proposed as a screening tool for patients with acute stroke. It specifically assesses all the cognitive domains, and we validated our tool in healthy subjects and patients, establishing a general and subtest specific cut-off. Earlier cognitive deficit identification can greatly contribute to optimizing the cost-effectiveness of early rehabilitation.

Interestingly, the clinical neuropsychologist could take into account both the total and single subtest MEPS scores. In fact, even if the total score may fall under the normality cut-off, single subtest’ scores could indicate spared cognitive domains, very relevant when considering a specific diagnostic investigation and the rehabilitation program.

MEPS also showed a significant correlation with other tests that evaluate the same cognitive domains, with the exception of the test for ideative apraxia and the visual discrimination task. One might hypothesize that such results could be ascribed to the patients’ performance, which was at the ceiling in these tests. Furthermore, in the case of the ideative apraxia, the two tests were performed with different execution modalities. In the case of the original ideative apraxia test, the patient is required to mime the tool usage, whereas in the MEPS subtest, the patient is required to actually use the tool.

Over the years, a number of assessment tools have been developed to explore the cognitive profile in the acute stage of the stroke [13, 25,26,27] along with MOCA-5 min [28]. Very recently, the Oxford Cognitive Screen (OCS) has been published [29]. This battery shares a number of similarities with MEPS although some of MEPS subtests are different. MEPS and OCS investigate both the main cognitive functions; however, there are differences in the stimuli and in the methods of task administration. For instance, MEPS was built by balancing the number of verbal and non-verbal tests by making homogeneous the scores of each subtest. The MEPS subtests were also created to evaluate specific deficits; for example, the visual-spatial exploration and attention test allow to evaluate both selective attention and neglect, even if not severe. The praxis tests evaluate both the ideative and the ideomotor apraxia. The executive functions are also evaluated with the verbal abstraction and categorization and by Clock Drawing test (visuospatial function) from which it is possible to obtain information also on other functions, such as constructive apraxia; the language tests include the assessment of verbal auditory comprehension and the words repetition. MEPS also provides a test for the evaluation of visuo-perceptual functions by the visual discrimination test, excluding the possible influence of visuospatial neglect. It is worth to note that the subtest choice grounded on the possible clinical conditions manifested by patients in the acute phase after stroke. As the OCS has been published while collecting data and preparing MEPS, additional researches are desirable to compare them, in order to provide specificity for each instrument in every single cognitive domain assessment.

From a qualitative point of view, MEPS is an instrument able to identify the pathological performance in a separated way basing on the age classes. Indeed, the greater the age class, the lesser the percentage of pathological scores (see for instance Tables S7, S8, and S9 in supplementary materials). In other words, by considering the young group (age class 20–45), MEPS seems to have high sensitivity and low specificity. Conversely, if one looks the age class 70–95, the percentage of pathology classification decreases compared with the other age classes. This could represent a strength if one takes into account the age at onset for the stroke.

Moreover, MEPS is able to identify both impaired and spared cognitive domains independently from their hemispheric lateralization. Indeed, no difference emerged when considering the side of the lesion. This could be due to the fact that a patient with a left-side brain lesion underperforms in verbal subtests, and a patient with a right-side brain lesion underperforms in visual-spatial subtests. The balance of the items included in each subtest leads to a no significant difference between patients with right- and left-brain lesions; however, it has to be considered that the two classes of patients underperform for different reasons.

Future studies are also needed to explore MEPS specificity in larger samples of patients, not only in the acute but also in the chronic stage: Studies of correlation between the behavioral impairments emerging by the different subtests and the brain lesions may contribute to better defining the predictive value of MEPS concerning the clinical outcome of patients and their response to the rehabilitative treatment. Furthermore, it could be useful to explore the accuracy of MEPS as a screening battery for other pathologies such as neurodegenerative disorders.