Telemedicine, Mobile, and Internet-Based Neurocognitive Assessment

Abstract

A new medium for delivering neuropsychological assessments has emerged as a result of the Internet called teleneuropsychology. This chapter reviews teleneuropsychology’s potential for extending the availability of neuropsychological assessments to remote and geographically isolated areas. Within the teleneuropsychology framework, neuropsychologists use video teleconferencing, smartphones, and Web-based assessment to administer, score, and aide interpretation of neuropsychological assessments. The chapter includes a discussion of studies comparing video teleconferencing to in-person assessments. Preliminary evaluation of patient acceptance of this methodology has revealed that teleneuropsychology appears to be well accepted by consumers. While teleneuropsychology offers a great deal of promise for remote assessment in general, there are strengths and weaknesses for various modalities and approaches that need to be discussed. For example, while teleconferencing allows neuropsychologists to connect with patients in remote areas, it falls far short of the stimulus presentation and logging found in computer-automated assessments. These limitations and the lack of ecological validity of assessments are discussed. While teleneuropsychology needs to move beyond video-teleconferenced administration of paper-and-pencil tasks, there are possibilities for enhancing data collection and database building for personalized medicine. Given the vast reaches of the Internet, future iterations of teleneuropsychological applications that embrace the technological advances of computerized administration (including simulated environments) have potential for enhanced stimulus presentation, data logging, and ecological validity.

Another possibility for increasing throughput is to adopt less labor-intensive procedures for example, it is possible to collect information of certain behavioral features through assessment tools that can be self-administered over the Internet by individuals throughout the world.

—Freimer and Sabatti (2003)

A new medium for delivering neuropsychological assessments has emerged as a result of the Internet. Recent surveys have revealed that over 3.1 billion people now have access to the Internet. The distribution of this number by country reveals the following: China = 642 million; USA = 280 million; India = 243 million; Japan = 109 million; Brazil = 108 million; Russia = 84 million, among others (Internet Live Stats 2015). In the USA, 86.75 % of the residents have access to the Internet. Telemedicine is an area that has developed for the use and exchange of medical information from one site to another via electronic communications, information technology, and telecommunications. When researchers are discussing “telemedicine,” they typically mean synchronous (interactive) technologies such as videoconferencing or telephony to deliver patient care. When the clinical services involve mental health or psychiatric services, the terms “telemental health” and “telepsychiatry” are generally used (Yellowlees et al. 2009).

Recently, the term “teleneuropsychology” has emerged for cognitive assessments that act as natural extensions of the movement in health care to expand the availability of specialty services (Cullum and Grosch 2012). Within the teleneuropsychology framework, neuropsychologists use video teleconferencing , smartphones, and Web-based assessment for remote assessment of patients in rural areas (Hilty et al. 2006). As a specialty service, teleneuropsychology allows for the assessment of patients potentially impacted by neurocognitive impairments that are geographically isolated. Evaluation of the patient is performed via a personal computer, digital tablet, smartphone, or other digital interface to administer, score, and aide interpretation of neuropsychological assessments (Cullum et al. 2014). Preliminary evaluation of patient acceptance of this methodology has revealed that teleneuropsychology appears to be well accepted by consumers. Parikh et al. (2013) found 98 percent satisfaction and approximately two-thirds of participants reported no preference between assessment via video teleconferencing and traditional in-person assessment.

While teleneuropsychology offers a great deal of promise for in general, there are strengths in weaknesses for various modalities and approaches that need to be discussed. For example, the most widely studied medium in teleneuropsychology is video teleconferencing, which represents an excellent advance in the incorporation of advanced technology for connecting neuropsychologists with patients in underserved areas. However, video teleconferencing falls far short of the stimulus presentation and logging found in Neuropsychological Assessment 2.0 and 3.0. Instead, it uses video teleconferencing to administer the sorts of paper-and-pencil measures found in Neuropsychological Assessment 1.0 . Further, it fails to add to the ecological validity of assessments and simply continues a construct-driven approach that falls short in the assessment of affect. A much more promising teleneuropsychology will also embrace smartphones (including ecological momentary assessment) and Web-based neuropsychological assessment . These approaches offer much of the promise found in Neuropsychological Assessment 2.0, and some of the enhanced ecological validity found in Neuropsychological Assessment 3.0.

The plan of this chapter is as follows: First, there will be a discussion of video teleconferencing and the studies comparing it to in-person assessments. Next, there will be a discussion of construct-driven and affective possibilities found in smartphones and Web-based approaches. The chapter will conclude with a discussion of the possibilities of enhanced data collection and database building for personalized medicine.

1 Video Teleconferencing for Teleneuropsychological Assessment

The use of video teleconferencing for teleneuropsychological assessment involves a set of videoconferencing platforms. One set of equipment is situated at the neuropsychologist’s location, and another set is located remotely at the client location. There are now a number of studies comparing the administration of neuropsychological measures via video teleconferencing to in-person administrations, and research in this area is being further developed and validated. In addition to video teleconferencing with healthy participants (Hildebrand et al. 2004; Jacobsen et al. 2003), a number of clinical studies have been performed: medical disorders (Ciemins et al. 2009; Menon et al. 2001), intellectual disability (Temple et al. 2010), alcohol abuse (Kirkwood et al. 2000), and dementia (Barton et al. 2011; Cullum et al. 2006; Grosch et al. 2015; Hanzevacki et al. 2011; Harrell et al. 2014; Loh et al. 2004, 2007; McEachern et al. 2008; Vestal et al. 2006).

1.1 Mini Mental State Examination via Remote Administration

Teleneuropsychology investigations have varied in their assessment measures and populations. Many have focused on cognitive screens such as the Mini Mental State Examination (MMSE; Ciemins et al. 2009; Loh et al. 2004, 2007; McEachern et al. 2008). For example, Ciemins et al. (2009) examined the reliability of MMSE administrations via remote administration. Their primary focus was to evaluate the auditory and visual components of the administration. The MMSE was administered only once to 72 participants with Type II diabetes. Responses from patients were recorded by both a remote administrator and an in-person examiner. Findings revealed that 80 % of individual items demonstrated remote to in-person agreement. In a related study administering the MMSE via video teleconferencing , McEachern et al. (2008) assessed elderly individuals referred to a memory clinic. Following an initial assessment, patients were seen at 6- and 12-week follow-ups. Findings revealed that MMSE scores did not differ significantly between remote and in-person assessments. In a similarly designed study, Loh et al. (2004) aimed to determine the interrater reliability of the Mini Mental State Examination through video teleconferencing as compared to in-person administration. Results revealed that the correlation between in-person and remote MMSE scores was 0.90. Findings suggest that remote assessments with the MMSE using video teleconferencing methods yielded similar results to direct assessments (see Table 1).

Table 1 Comparison of MMSE in video teleconference and face-to-face modalities

1.2 Neuropsychological Batteries via Video Teleconferencing

While the cognitive screening studies are helpful, many neuropsychologists are interested in the results of comparing larger neuropsychological batteries via video teleconferencing to in-person administrations (see Table 2). Jacobsen and colleagues (2003) evaluated the reliability of administering a broader neuropsychological assessment remotely using healthy volunteers. The battery was made up of twelve measures that covered eight cognitive domains: attention , information processing, visuomotor speed, nonverbal memory , visual perception, auditory attention , verbal memory, and verbal ability. Results revealed that for most of the measures, the in-person and remote scores were highly correlated (reliability coefficients ranging from 0.37 to 0.86; median value of 0.74).

Table 2 Neuropsychological battery and video teleconference and face-to-face modalities

In addition to some important reviews of the teleneuropsychology literature, Cullum , Grosch, and colleagues have completed a set of studies comparing video teleconference-based diagnostic interviewing with conventional in-person assessment (Cullum et al. 2006, 2012, 2014; Grosch et al. 2011, 2015). In an early study, Cullum et al. (2006) made this comparison with a sample of 33 participants, with 14 older persons having mild cognitive impairment and 19 participants with mild-to-moderate Alzheimer’s disease. The neuropsychology battery included the MMSE, Hopkins Verbal Learning Test-Revised, Boston Naming Test (short form), Digit Span, letter fluency, and category fluency. Robust correlations were found between video teleconference and in-person testing: MMSE (r = 0.89), Boston Naming (r = 0.88), digit span (r = 0.81), category fluency (r = 0.58), and letter fluency (r = 0.83), and showed excellent agreement. It is important to note that while there was a significant correlation between two conditions for the HVLT-R, verbal percentage retention score exhibited considerable variability in each test session. This suggests that this score may not be as reliable as the other memory indices. These results offer further support for the validity of video teleconferencing for conducting neuropsychological evaluations of older adults with cognitive impairment. Cullum and colleagues (2014) performed a follow-up study with a larger sample size. They examined the reliability of video teleconference-based neuropsychological assessment using the following: MMSE, Hopkins Verbal Learning Test-Revised, Boston Naming Test (short form), letter and category fluency, digit Span forward and backward, and clock drawing. The sample consisted of two hundred and two (cognitive impairment N = 83; healthy controls N = 119) adult participants. Highly similar results were found across video teleconferencing and in-person conditions regardless of whether participants had cognitive impairment. These findings suggest that video teleconferencing-based neuropsychological testing is a valid and reliable alternative to traditional in-person assessment. In a more recent study, this team aimed to validate remote video teleconferencing for geropsychiatry applications. Findings suggest that brief telecognitive screening is feasible in an outpatient geropsychiatry clinic and produces similar results for attention and visuospatial ability in older patients. The patients of neuropsychologists may benefit from a remote assessment and diagnosis because teleneuropsychology allows the neuropsychologist to overcome the barriers of displacing patients (and their caregivers) living in rural areas that are far from health institutions.

1.3 Gerontology Applications of Videoconference-Based Assessment

The role of teleneuropsychology is expected to expand, providing increased access to clinical care for geographically isolated geriatric patients in rural settings (Ramos-Ríos et al. 2012). Teleneuropsychology-based programs using video teleconferencing for gerontological care have been positively received by persons living in nursing homes, community dwelling patients, caregivers, and physicians. Video teleconferencing has also been shown to have adequate reliability with in-person assessment (Azad et al. 2012). To replicate best practices from traditional approaches found in memory clinic settings, interdisciplinary models for the care of elders with cognitive impairment have also been applied via video teleconferencing (Barton et al. 2011). Of specific interest to neuropsychologists are studies showing the feasibility and reliability of using video teleconferencing for administering objective cognitive testing with screening instruments (Cullum et al. 2006; Grosch et al. 2015; Hanzevacki et al. 2011; Loh et al. 2004, 2007; McEachern et al. 2008; Parikh et al. 2013; Turner et al. 2012; Vestal et al. 2006).

1.4 Language Assessments

In a study that emphasized the effectiveness of language assessment in mild Alzheimer’s patients, Vestal and colleagues (2006) compared video teleconferencing with in-person language assessments: Boston Naming Test, Picture Description (auditory response), Token Test, Aural Comprehension of Words and Phrases, and the Controlled Oral Word Association Test. Results from the Wilcoxon signed-rank test indicated no significant difference for performance on each of the five language tasks between the video teleconference and in-person conditions. It is important to note that the overall acceptance of the video teleconferencing evaluation in an elderly population was rated at a high level. Given these results, video teleconferencing appears to have promise for speech and language evaluation services in dementia.

1.5 Acceptability of Neuropsychological Screening Delivered via Telehealth

Although the above studies provide support for the feasibility, validity , and acceptability of neuropsychological screening delivered via telehealth, there have only been a couple studies that discuss the feasibility of neuropsychological assessment for a comprehensive dementia care program that services patients with limited accessibility (Barton et al. 2011; Harrell et al. 2014; Martin-Khan et al. 2012; Vestal et al. 2006). For example, Barton et al. (2011) employed a platform to administer multidisciplinary, state-of-the-art assessment of neurocognitive impairment by video teleconferencing. The participants were patients at a rural veteran’s community clinic that were referred by their local provider for evaluation of memory complaints. The neuropsychological evaluation was integrated into the typical clinical configuration and involved a neurological evaluation and neuropsychological testing via video teleconferencing. Results revealed that for each patient, the video teleconferencing format permitted the clinical team to arrive at a working diagnosis and relevant treatment recommendations were made. The evaluation results were discussed with providers who joined the post-clinic conference via video teleconferencing . These findings suggest that video teleconferencing may offer an effective way to provide consultation and care to rural residents.

Similar results were found in a study designed to determine the validity of the diagnosis of dementia via video teleconferencing. Martin-Khan and colleagues (2012) evaluated 205 patients using video teleconferencing and in-person administrations of neuropsychological tests: MMSE, Rowland Universal Dementia Assessment Scale (RUDAS), verbal fluency, animal naming, and the clock face test. Results revealed significant agreement between the video teleconference group and the in-person group. The summary kappa statistic indicated that video teleconferencing was similar to the in-person assessment. Findings suggested that incorporating video teleconferencing and in-person assessments has promise as a reliable process for differential diagnosis of dementia.

In summary, teleneuropsychology has emerged as a natural extension of the movement in health care to expand the availability of specialty services via telecommunication. Within the teleneuropsychology framework, neuropsychologists use video teleconferencing for remote assessment of patients in rural areas. A limitation of video teleconferencing is that it falls far short of the sorts of progress discussed in this manuscript. It continues a construct-driven approach and falls short in assessment of affect. Further, in terms of progress, video teleconferencing appears to be little more than Neuropsychological Assessment 1.0 using advanced technology. This is apparent in the low ecological validity of the tests that simply administer traditional construct-driven paper-and-pencil tests using video teleconferencing. Finally, video teleconferencing also fails to bring in any affective assessments beyond paper-and-pencil measures.

2 Smartphones for Telephone-Based Neuropsychological Assessment

Contemporary technological advances such as telemedicine and teleneuropsychology paradigms have been shown to have promise a multiplicity of settings (Cullum et al. 2006, 2011, 2014; Cullum and Grosch 2012; Rajan 2012; Clifford and Clifton 2012; Rogante et al. 2012). One approach to teleneuropsychology is the use of smartphones which include computing capabilities. Given their mobility and ubiquity in the general population, smartphones are increasingly part of medicine-related applications that have the potential for use in clinical settings. A number of papers have described the promise of smartphone-based applications for assisting in a broad range of clinical research areas and patient point-of-care services (Boulos et al. 2011; Doherty and oh 2012; Doherty et al. 2011; Fortney et al. 2011). Of particular interest for neuropsychologists is the potential of smartphones for research in cognitive science (Dufau et al. 2011). A handful of smartphone applications have emerged for cognitive assessment (Brouillette et al. 2013; Gentry et al. 2008; Kwan and Lai 2013; Lee et al. 2012; Svoboda et al. 2012; Thompson et al. 2012).

In one of these smartphone applications, Brouillette et al. (2013) developed a new application that utilizes touch screen technology to assess attention and processing speed. Initial validation was completed using an elderly nondemented population. Findings revealed that their color shape test was a reliable and valid tool for the assessment processing speed and attention in the elderly. These findings support the potential of smartphone-based assessment batteries for attentional processing in geriatric cohorts.

Smartphones move beyond the Neuropsychological Assessment 1.0 (paper and pencil) found in video teleconferencing. Given their use of advanced technologies (i.e., smartphones instead of paper-and-pencil), smartphone-based cognitive assessment represents Neuropsychological Assessment 2.0 . That said, smartphone-based cognitive assessment does not extend ecological validity because it incorporates construct-driven and veridical assessments that lack assessment of affective processing.

3 Ecological Momentary Assessments

Neuropsychologists are interested in the everyday real-world behavior of their patients because brain injury and its functional impairments are expressed in real-world contexts. An unfortunate limitation is that many neuropsychological assessments are construct-driven (e.g., working memory ) assessments that do little to tap into the affective aspects of their patient’s functioning. Instead, evaluation of activities of daily living , quality of life, affective processing, and life stressors is surveyed using global, summary, or retrospective self-reports. For example, a neuropsychologist may ask a patient how often they experience frustration, how many times they forgot their intentions during the past week or month, or how depressed their mood has been. The prominence of global questionnaires can keep neuropsychologists from observing and studying the dynamic fluctuations in behavior over time and across situations. Further, these questionnaires may obfuscate the ways in which a patient’s behavior varies and is governed by context. In reaction to the frequent reliance of neuropsychologists on global, retrospective reports (and the serious limits they place on accurately characterizing, understanding, and changing behavior in real-world settings), some neuropsychologists are turning to Ecological Momentary Assessment (EMA; Cain et al. 2009; Schuster et al. 2015; Waters and Li 2008; Waters et al. 2014). EMA is characterized by a series of (often computer-based) repeated assessments of then current cognitive, affective (including physiological), and contextual experiences of participants as they take part in everyday activities (Jones and Johnston 2011; Shiffman et al. 2008).

EMA moves beyond the Neuropsychological Assessment 1.0 (paper-and-pencil) found in video teleconferencing and to some extent the Neuropsychological Assessment 2.0 found in smartphone-based cognitive assessments. The EMA approach also moves beyond the frequent reliance of neuropsychologists on retrospective reports offers a series of computer-based repeated assessments of cognitive, affective (including physiological), and contextual experiences of participants as they take part in everyday activities. While this is a new application in neuropsychological assessment , it does offer promise (once adequately validated) for enhancing the ecological validity of neuropsychological assessments.

4 Web-Based Computerized Assessments

Web-based computerized assessments have the capacity for adaptive testing strategies that are likely to multiply efficiency in construct measurement (Bilder 2011). In one study, Gibbons and colleagues (2008) found that use of a computerized adaptive test resulted in a 95 % average reduction in the number of items administered. The potential of adaptive Web-based neurocognitive assessment protocols can be seen in the capacity to large sample of participants in relatively short periods of time. While there may be concerns that the neuropsychologist cannot be sure of the identity of the test-takers or that they are performing tasks as instructed, validity indicators, online video surveillance, and anthropometric identifiers can be included to remove these concerns. For example, algorithms can be implemented that allow for item-level response monitoring and automated consistency checks. Further, neuroinformatics algorithms are available that will allow for detection of outlying response patterns of uncertain validity .

While some Web-based neuropsychological assessments are limited to a single domain (e.g., attention and processing speed; Bart et al. 2014; Erlanger et al. 2003; Raz et al. 2014), there is an emerging suite of online batteries. Some of these Web-based assessments consist primarily of informant reports of cognitive decline (Brandt et al. 2013), but there are an increasing number of cognitive screens (Medalia et al. 2005; Scharre et al. 2014) and some larger batteries (Elbin et al. 2011; Gur et al. 2001; Schatz and Sandel 2013; Silverstein et al. 2007; Troyer et al. 2014). A recent battery called BRAINScreen was developed by Zakzanis and Azarbehi (2014) to offer a Web-based and real-time examination of cognitive functioning. The Web-based screening battery includes a number of cognitive measures: visual attention and information-processing speed; list learning and recall; spatial orientation-type task; and forward and backward Digit Span. Initial psychometric validation revealed (when combined into a composite score) a correlation with age, ability to distinguish normal from clinical groups, and robust overall reliability. BRAINScreen offers a straightforward and manageable Web-based automated screen for neurocognitive impairment with real-time interpretive results.

In another project, Troyer and colleagues (2014) developed a Web-based cognitive assessment for use with middle-aged and older adults. The Web-based battery emphasizes measures of memory and executive attention processes: spatial working memory task; stroop interference task; face–name association task; and number–letter alternation task. Results from a normative study revealed adequate internal consistency, construct validity, test–retest reliability , and alternate version reliability. Each of the neurocognitive tasks loaded on the same principle component. Demographically corrected z-scores from the individual tasks were combined to create an overall score, which showed good reliability and classification consistency. Scores were correlated with age. These findings suggest that the Web-based neuropsychological screening measure may be useful for identifying middle-aged and older adults with lower than expected scores who may benefit from further evaluation by a clinical neuropsychologist.

One of the most widely used and validated of the new Web-based neuropsychology batteries is WebNeuro . The battery includes assessments of the following domains of neurocognitive function: sensorimotor, memory, executive planning, attention, and emotion perception (social cognition; see Table 3; Mathersul et al. 2009; Silverstein et al. 2007; Williams et al. 2009). In addition to cognitive assessments, the WebNeuro battery also includes affective assessments of emotion recognition and identification: immediate explicit identification followed by implicit recognition (within a priming protocol). The WebNeuro protocol was developed as a Web-based version of the IntegNeuro computerized battery . The IntegNeuro battery was developed by a consortium of scientists interested in establishing a standardized international called Brain Resource International Database (BRID; Gordon 2003; Gordon et al. 2005; Paul et al. 2005). IntegNeuro and WebNeuro are part of the BRID project’s aim to move beyond outdated approaches to aggregating neuropsychological knowledgebases and develop standardized testing approaches that facilitate the integration of normally independent sources of data (genetic, neuroimaging , psychophysiological, neuropsychological, and clinical). The WebNeuro platform allows for data to be acquired internationally with a centralized database infrastructure for storage and manipulation of these data.

Table 3 WebNeuro Cognitive and Affective Assessments

Psychometric evaluation of the WebNeuro platform has revealed robust correlations with IntegNeuro, indicating high convergent validation. The correlation between the WebNeuro and IntegNeuro factor scores exceeded 0.56 in all cases, reflecting a statistically significant degree of overlap between the two variables (Silverstein et al. 2007). In further validation studies with a large sample (n = 1000; 6–91 years, 53.5 % female), the WebNeuro tests of emotion identification and recognition and tests of general cognitive function revealed seven domains of general cognition: information-processing speed, executive function, sustained attention/vigilance, verbal memory, working memory capacity, inhibition/impulsivity, and sensorimotor function (Mathersul et al. 2009; Silverstein et al. 2007; Williams et al. 2009).

While the normative findings for a Web-based neuropsychological assessment are promising, the greatest potential appears to be WebNeuro’s relation to Brain Resource International Database . The data from WebNeuro are linked to insights and correlations in a standardized and integrative international database (see www.BrainResource.com; www.BRAINnet.com). The WebNeuro data is incorporated as additional (in addition to other markers: genetic, neuroimaging , psychophysiological, neuropsychological, and clinical) clinical markers that can be incorporated into databases as new marker discoveries emerge from personalized medicine.

5 Summary and Conclusions

Despite breakthroughs in the human neurosciences (cognitive, social, and affective), neuroimaging, psychometrics, human–machine interfaces, and neuroinformatics , the neuropsychological assessments in use today have barely changed over the past century. In addition to failing to advance with technological innovations, traditional print publishing like the WAIS-IV/WMS-IV revisions has fallen short in terms of back-compatibility that may invalidate clinical interpretations (Loring and Bauer 2010). This lack of back-compatibility is especially disappointing given the fact that even modest updates to paper-and-pencil batteries take years. Neuropsychologists are increasingly interested in the potential for advanced psychometrics, online computer-automated assessment methods, and neuroinformatics for large-sample implementation and the development of collaborative neuropsychological knowledge bases (Bilder 2011; Jagaroo 2009). The use of modern psychometric theory and neuroinformatics enables preservation of robust back-compatibility with prior test versions and enables the introduction of new content and new constructs. This will become increasingly important as evidence-based medicine is systematically adopted within the context of clinical neuropsychology (e.g., Chelune 2010). There is increasing interest among neuropsychologists in adopting consolidated standards for the reporting of clinical trials that include neuropsychological endpoints that will ultimately serve to strengthen the empirical evidence and scientific base of neuropsychology (Loring and Bowden 2014; Miller et al. 2014).

Some neuropsychologists are also calling for greater inclusion of computer-automated assessments (Bilder 2011; Jagaroo 2009). It is important to note that some neuropsychologists have concerns that computerized neuropsychological assessment devices will somehow replace human neuropsychologists and/or overlook significant clinical information. However, computerized neuropsychological assessments are just tools for enhanced presentation and logging of stimuli. When properly used, computer-automated assessments can greatly enhance the assessment in terms of precision and rapid implementation of adaptive algorithms. Further, as Bilder (2011) points out, a distinct improvement of computer timing precision is that it allows for enactment of procedures from the human neurosciences that rely on more refined task operations and trial-by-trial analyses that may be more sensitive and specific to individual differences in neural system function. The exponential increase in access to computers and the Internet across the life span allows for interactive Web-based cognitive assessments (Bart et al. 2014; Raz et al. 2014; Wagner et al. 2010). Access to and use of computers are becoming increasingly more common among older adults. Currently, 59 % of those persons that are 64 years or older go online, and this number is increasing at a rapid pace (Smith 2014). Online neurocognitive assessment may enhance dissemination of testing because tests could be administered in a variety of settings (e.g., office, home, school), at different times of the day, and by multiple persons at the same time.