Keywords

Clinical Case

An 18-year-old high school senior and his parents present in your office after being told that he needs clearance by a physician prior to returning to football practice. What single test can reliably make the diagnosis of concussion?

One of the challenges with concussion is finding objective and definitive testing to both aid the clinician in the diagnosis and help define the recovery. Multiple studies have identified blood biomarkers that rise and fall during the course of a concussion marking the metabolic changes following this injury [1,2,3,4,5]. However, no individual biomarker or combination of these biomarkers has yet proven to be a reliable and definitive test to rule in or rule out a concussion. Concussion research in the animal model has given details on the cellular level metabolic crisis associated with concussion and the timeline for restoration of normal metabolic activity [1, 6]. This timeline does not necessarily appear to directly correlate with the timeline of the clinical symptoms or testing abnormalities that we see in humans [7]. As noted in previous chapters, concussion can present with a wide variety of symptoms and clinical findings spread over several different physical, emotional, and cognitive domains. Therefore, a thorough evaluation for concussion should contain elements that provide information from each of these domains. Neuropsychological testing can provide important and useful information in the cognitive and emotional domains.

Neuropsychological testing has been used in various formats for many years to provide information for several different emotional and cognitive disease processes. These tests can assess cognition including intelligence, academic functioning, attention, working memory, processing speed, learning memory, visual spatial skills, fine and gross motor skills, and executive functioning. Current neurocognitive assessment tools utilized by providers experienced in concussion care and frontline concussion clinics are typically an abbreviated form of neuropsychological testing referred to as neurocognitive testing (NCT). These assessments are often brief enough to allow for the baseline screening of large numbers of athletes and yet still provide enough information to assist the clinician in the evaluation and management of concussions. Neurocognitive testing contains information from a limited number of the domains and should not be used in a stand-alone manner but rather part of a more comprehensive clinical evaluation [8].

Question: Is neuropsychological testing a new development?

Attempts at trying to localize cognitive functions in the brain started very early in the history of medicine. Herophilius is generally given credit for first attempting this evaluation process in 300 B.C. [9]. More recently, neuropsychological testing has been described as “the normatively informed application of performance-based assessments of various cognitive skills” [10]. This testing is usually accomplished through a lengthy battery of subtests covering a spectrum of cognitive ability areas. The areas evaluated may vary from test to test but typically include assessments of several cognitive domains including memory, attention, processing speed, reasoning, judgment, problem solving, spatial function, and language function (Table 6.1). This testing is currently used to assess a variety of neuropsychiatric conditions including dementia, mood disorders, stroke, and traumatic brain injury (TBI) [11]. These test batteries are most accurate when administered under controlled and standardized conditions. Current validated assessments typically provide databases with demographically matched normative data. The classic neuropsychological testing batteries are often very time-consuming and must be administered and interpreted by psychologists (usually neuropsychologists) and neuropsychiatrists with extensive training and certifications.

Table 6.1 Common neuropsychological tests used in sports concussion assessments
Full size table

Question: Are computer-based neurocognitive tests a viable option in concussion evaluation?

Neurocognitive testing in the sports medicine and concussion community has been considered a cornerstone of concussion assessments [12,13,14]. Traditional neuropsychological testing can provide important diagnostic information for neurological, cognitive, and emotional conditions, many of which are not typically followed in a frontline concussion clinic or sports medicine clinic. The unique requirements in sports medicine have given rise to more concise and focused neurocognitive assessments that center on cognitive domains most often affected by concussion (e.g., memory, attention, processing speed, and reaction time). These newer assessments, often utilizing computer or tablet administration, can be accomplished in as little time as 5 minutes for the abbreviated field-side assessments and 10–30 minutes for more comprehensive pre- and post-injury assessment. While these sideline assessments do not yield as much information for decision-making and treatment, they provide objective information to aid clinical decision-making for a suspected concussion.

The Berlin Concussion Consensus statement and the National Athletic Trainers’ Association (NATA) position statements have helped define the use of neurocognitive testing in sports medicine [13, 15]. These position statements clearly indicate that neurocognitive testing is a vital part of the overall concussion assessment but is not intended to be used as stand-alone testing for evaluation or management. Neurocognitive testing is best utilized when it is part of a larger multi-domain evaluation process.

Question: What is the difference between the testing administered by a neuropsychologist and computerized neurocognitive testing?

The term “neurocognitive testing” is often used interchangeably with neuropsychological testing. While both are used to evaluate the relationship between the brain and behavior, there are significant differences in the scope of use, information derived, cost, evaluation time, and usefulness in neurorehabilitation planning. A neuropsychological examination is the gold standard for assessing all areas of brain function and includes a comprehensive evaluation of sensory/motor function, auditory and visual attention, working memory, verbal and visual memory, language, executive function, speed of processing, intellectual ability, and emotional capacity. The examination includes a combination or battery of tests that can provide an average or composite score across multiple ability areas and provide an overall index of how well a person functions cognitively at the time of testing [10]. As a result, testing is time-consuming and requires multiple hours, which may even be spread out over several days depending on the referral question and information needed. The final results are referenced to demographic groups of age, sex, race, and education levels. There are also internal checks built in to determine an examinee’s effort and testing validity. The results can be interpreted by psychologists, neuropsychologists, or neuropsychiatrists and compared against known deficit profiles related to illness, disease, and injury for diagnosis. The results are detailed enough to be used for neurorehabilitation planning, special education placement, competency determination, forensic/legal purposes, drug or treatment research, and identifying functional impairments. Many of the various testing instruments were developed before the widespread use of computers and are administered using paper, pencil, and a stopwatch. In recent years there have been concerted efforts to computerize many of these tests, which require lengthy re-standardization processes. Traditional pencil-and-paper tests include those seen in Table 6.1. Most of these tests are copyright protected and require advance training and licensing to purchase, administer, and interpret. Most experts recommend a licensed psychologist, usually a board-certified neuropsychologist with clinical experience in evaluating sport-related concussion, and administer or at least supervise testing [10, 16].

Neurocognitive testing is aimed at addressing a subset of symptoms or cognitive functions related to a particular illness or injury. Most of these tests were created from the ground up for computer use to facilitate ease of administration, portability, and rapid scoring; some even include basic interpretation and provide limited age-related norms. The SCAT5, ImPACT, ANAM, Axon, C3 Logix, and other neurocognitive tests were designed and standardized to quickly assess the cognitive deficits seen with concussion, concentrating on attention, processing speed, and immediate memory. However, these devices could be inappropriate and even invalid in the evaluation of other cognitive impairments including learning disability, ADHD, brain tumor, stroke, traumatic brain injury, and other neurological conditions, due to limited domain assessment, differing comparison groups, and interpretation by those not qualified to provide a medical or psychological diagnosis. The administration of neurocognitive testing can be done by non-physician medical or athletic training staff. Some offer tablet-based administration for complete portability and provide cloud storage for universal wireless access. The health-care provider will want to become familiar with administration and interpretation documentation, especially if there are questions or concerns about effort that may impact returning to play. Lingering recovery due to comorbidities or worrisome cognitive deficits may require a more comprehensive assessment and can always be referred for a full neuropsychological examination.

Question: What are some of the benefits of computerized neurocognitive testing?

Computerized neurocognitive assessments have found an increasingly common role in the evaluation and assessment of sport-related concussion. Different forms of these tests have increased in use for many reasons including practicality, ease of interpretation, and portability. The companies producing the CNTs suggest and provide information on how to administer and interpret the tests, but no formal training or certification is required. Programs can be downloaded to a computer or laptop or web-based programs utilized to administer the tests. Newer tablet-based tests have even improved the portability making it possible to take the test in nearly any environment. The results can then be uploaded to a central server allowing for review from any computer with web access. For example, these tests can be performed by a certified athletic trainer (ATc) at a school or sports training room environment and be remotely reviewed by a physician.

Computer-based neurocognitive assessment tools (Table 6.2) are much less time-consuming, often taking 10–30 minutes to complete, when compared to a traditional 4-hour neuropsychological test battery. These tests also differ from traditional neuropsychological testing in that they do not need to be administered or interpreted by a certified testing specialist. Scoring for these computer-based tests is automated and often produces a summary sheet for statistical analysis or automatically compared to baseline and/or normative data.

Table 6.2 Computerized neurocognitive assessments
Full size table

These computer-based neurocognitive tests are often useful for large group baseline testing preseason (often performed at the time of sport pre-participation examinations) and in the post-injury setting. With the relative ease of baseline testing, post-injury evaluations can readily be compared to the baseline test for an individual athlete and performed serially to assess for recovery [17]. A baseline comparison is particularly useful in those cases where learning or testing difficulties (ADHD, dyslexia, etc.) and other confounding diagnosis, such as depression, or chronic migraine, can interfere with normative testing result data. Another unique benefit to these computerized test batteries is that these tests allow for very accurate and quantified measurements of reaction time [16,17,18,19]. This is more difficult to obtain at the same level of accuracy on paper-and-pencil assessments. One of the final benefits of computer-based tests is the number of controlled test variations [17]. It may be more difficult to maintain the number of variations in traditional paper and pencil tests to accomplish this same task. Multiple retest variations are important for the athlete that is retaking tests over a relatively short time frame in order to track recovery and aid return-to-play decisions in an active management program.

Question: What is the role for computerized neurocognitive testing on the field side for possible concussions, and how does the field-side assessment differ from the evaluation in the clinic?

CNTs have increased in use and are part of many formal professional, collegiate, and high school concussion protocols because they can be administered relatively quickly and be performed as part of a baseline assessment and post-injury assessment. However, the requirement of a field-side test is different from a more detailed clinical assessment. Even a 10–30-minute CNT doesn’t have a role in the immediate field-side evaluation. Field side, an assessment is used to evaluate an athlete and establish some validation of a concussion injury which by definition is a neurological or neurocognitive impairment from this biomechanical force applied to the head [13]. The decision to return an athlete to the field of play can be difficult and usually needs to be made rapidly. In some cases, the signs and symptoms of a concussion evolve over a number of minutes to hours [13]. Therefore, if the athlete shows enough evidence for a presumed or possible diagnosis of concussion during the assessment, then the athlete must be removed from play. A more complete neurocognitive assessment can then be performed at a later time. Neuropsychological testing and even typical CNTs do not have clinical utility for making the immediate, sideline decisions for a concussion injury. The need for immediate decision-making on the sideline has been the motivation for the development of several brief assessment tools. The functionality of these tests differs from both the traditional neuropsychological test batteries and the more recent CNTs. Brief field-side neurocognitive tests include the paper-and-pencil Standardized Assessment of Concussion (SAC) and the Sports Concussion Assessment Tool 5th Edition (SCAT5) which includes modified Maddocks questions (Table 6.3). More recently, some applications such as C3Logix and ImPACT Quick Test have incorporated similar components of these paper-and-pencil brief field-side assessments into a digital format on a computer tablet.

Table 6.3 Modified Maddocks questions from SCAT 5
Full size table

Question: What are some concerns about using computerized neurocognitive testing?

With the rise in popularity and use of CNTs, some definite limitations and disadvantages have presented. The gold standard in neuropsychological testing is still considered to be the more traditional model administered and interpreted one-on-one by a trained and credentialed neuropsychologist.

There are several concerns from the neuropsychology community regarding CNTs. The first concern is that many of these tests have not yet undergone the same level of validation as the more traditional tests [16, 20]. Overall, CNTs have been found to have variable test-retest reliability and moderate sensitivity [7]. Any injury assessment tool or test is only useful if the test will reliably detect or rule out a specific injury or a finding related to a specific injury. Reliability is an extremely important concept in concussion testing due to individual serial testing strategies [21]. CNTs work most accurately when an individual has a baseline assessment and then is able to repeat a test post-injury for comparison and evaluation of the differences. The assumption is that the difference in testing scores is due to the injury, but many factors can affect testing results (Fig. 6.1). If a test has a low reliability coefficient, it may be difficult to attribute the testing result changes to the injury. When there has been no concussion, a change in scores from test to test indicates measurement error or other factors affecting the results.

Fig. 6.1
figure 1

Things that influence cognitive performance

Full size image

A significant challenge in trying to validate CNTs has been the lack of standardization of what is being tested. Each group or company producing the CNTs uses different testing formats to test the cognitive domains deemed most important. Similar cognitive domains may be measured, but the specific subtests are different enough to make comparisons to other similar tests problematic [16]. Much of the research has been industry-sponsored, and there is little peer-reviewed work directly comparing the performance of the currently available CNTs [18]. Additionally, many of the computer-based testing systems do not even measure the same cognitive domains [7]. These differences make head-to-head comparison of results very difficult if not impossible (see Table 6.2). This can be a significant challenge if an athlete uses one CNT platform at their high school and another CNT platform for their off-campus club sport. Another example is that of an athlete who completed baseline testing on one CNT platform at school as a part of pre-participation exam, and the physician performing a post-injury examination uses a different CNT platform in their office. The tests are often not comparable.

Secondly, there is no consensus among concussion experts on the protocols for the use of the CNTs. Baseline testing and post-injury testing present some similar and some unique challenges. In post-injury evaluations, one thought is that CNTs are most accurate immediately after a concussion injury. The sensitivities for three CNTs were found to be best within 24 hours of the injury. And after 8 days, sensitivity decreased to near false-positive rate in non-injured controls [22]. Another thought is perhaps CNTs are not sensitive enough for the subtle differences seen in the later stages of concussion recovery [18]. For baseline testing, the overall importance and utility of baseline testing have been questioned. Baseline testing allows an individual athlete to serve as their own control rather than relying on normative data to determine when an athlete as returned to “normal” [23]. Some experts report that there is insufficient evidence to recommend the widespread routine use of baseline neuropsychological testing [20]. The concept of routine baseline testing for large populations or teams was near impossible with traditional neuropsychological testing. Newer CNTs have used the ability to perform mass baseline testing sessions as an attractive feature.

One challenge in both baseline testing and post-injury testing is the testing environment. Differences in testing environments can significantly affect the baseline testing results and post-injury results [15]. Large group baseline evaluation can contribute to poor estimation of premorbid conditions which can have a significant effect on testing results [24]. Neuropsychological conditions are complex, and mood or motivation can affect cognitive functioning and performance on testing [10]. It is well documented that premorbid conditions, such as depression, can significantly affect performance on neurocognitive testing [25]. Other conditions (see Fig. 6.1) such as pain from injury or surgery, poor sleep, medications, fatigue, and hearing or visual deficits requiring eyeglasses or hearing aids that may have been noticed with one-on-one testing may not be recognized with large group testing. For example, sleep deprivation has been shown to have a significant effect on working memory and attention [26]. External factors include anything that can influence sensory inputs such as noise, activity, or noxious smells at the testing location. Both intrinsic and extrinsic factors can have a significant effect on attention. In a large group testing, the distractions and effort can be called into question especially for individuals with underlying attention challenges. The testing environment must allow the test taker to focus and provide the best possible effort.

Question: What do you do if you don’t have a baseline test or access to baseline testing results?

In any concussion clinic environment, there will be many cases that present without baseline testing. In these cases, normative data comparison is used for scoring the testing results. Each of the commercially available CNTs have their own normative data base. However, some care should be taken using normative data comparisons. Traditional neurological tests have normative data sets typically grouped by age in 5-year blocks from age 18 to 89. Newer CNTs do not necessarily use the groupings. Some only have 3–4 groups between the ages of 10 and 24. There can be a significant difference in performance norms especially in the younger and older populations. Many factors can affect performance on baseline and post-injury testing. Despite these limitations, baseline testing is already part of many professional, collegiate, and high school concussion protocols. More rigorous scientific data is needed before we can reach a firm conclusion on its validity and use as a gold standard.

Question: Can an athlete purposefully give poor effort during baseline testing or “sandbag” in order to return to play more quickly after a concussion?

Even with an optimal pretest screening and testing environment, the test taker’s effort can significantly affect testing results. It has been suggested that athletes may provide suboptimal effort—sometimes called “sandbagging”—in order to return to their baseline cognitive scores and return to play more quickly [23]. Traditional lengthy neuropsychological test batteries contain built-in measures to assess performance validity. These measures can be useful in understanding performance inconsistencies from comorbid conditions but are also useful in picking up effort-related inconsistencies. The more commonly used CNTs do not employ all of the same measures as classical neuropsychological testing to evaluate performance, but several tests use internal validity indicators in a similar fashion. These measures are designed to identify results that may have been affected by many factors including suboptimal effort, but much of the research on these measures has been industry-sponsored. One study showed 11% of ImPACT savvy college athletes were able to successfully “sandbag” a baseline ImPACT test without activating the test internal validity indicators [27]. Another study showed 30% of ImPACT-naïve nonathlete college students were able to “sandbag” without being caught by the ImPACT validity indicators [28].

Traditional neuropsychological tests were administered and interpreted one-on-one. Even though CNTs may be administered to a large group of people at one time, it is very important that baseline testing be reviewed and examined one test at a time for valid results. In a survey of athletic trainers in 2009 reviewing the use if ImPACT testing, only 55% examined baselines for valid results [29]. If the baseline test is invalid, it cannot reliably be used for comparison as part of the return-to-play decision-making process.

Evolution of Computerized Neurocognitive Testing

Question: Why do I have to use a wired mouse with some tests when I normally use a wireless mouse with my computer?

Rapid evolution of computer systems continues to play a role in the usability and portability of neurocognitive tests. Computer-based testing has evolved as technology has advanced. Testing just a few years ago on desktop systems with wired components evolved to testing on portable laptop systems with Bluetooth or wireless components. Some neurocognitive assessments now utilize portable tablets and cell phones for testing. One of the advantages of the computer-based neurocognitive testing batteries has been the ability of a computer to assess subtests such as reaction time to a very small and sensitive level. Establishing a valid normative database for subtests such as reaction time requires very specific and standardized computer hardware configurations. The power of large normative databases comes from the number of the same tests with the same hardware configurations stored and available for reference. Changing even one component, such as a wired mouse for a Bluetooth wireless mouse, can affect the sensitive results and ultimately challenge the integrity of the normative database. It is difficult for the neurocognitive testing systems to keep up with the rapid technology changes. Newer tests are coming to market on current technology, but even these may be outdated as technology advances. For example, the traditional ImPACT testing system requires a desktop or laptop computer with a wired mouse (not wireless or Bluetooth mouse) to assure correlation with their normative database. C3Logix and two newer ImPACT products, ImPACT quick test and ImPACT pediatric, utilize tablet-based hardware.

Key Points

  • Neurocognitive testing is a well-established and important tool and can be useful to assess for dysfunction in a unique domain not evaluated with other tests.

  • Computerized neurocognitive tests are evolving and have their own strengths and limitations.

  • It is important to recognize the strengths and weaknesses of the testing system you are using.

  • Neurocognitive testing is meant to be one part of a comprehensive concussion evaluation protocol but not as the sole determinant for diagnosis or return-to-play decisions.