Introduction

There is increasing recognition of the importance of neurocognitive outcome measures in clinical trials of brain tumor therapies. Since the US Food and Drug Administration designated improved neurocognitive function or delay in neurocognitive decline as acceptable end points in clinical trials in 1998, cognition has been demonstrated to be an independent predictor of survival in patients with central nervous system (CNS) tumors [1–4]. Cognitive deterioration has also been shown to be an early indicator of tumor progression, sometimes before it is detectable on imaging studies [5]. In addition to its relationship with survivorship, Meyers and Brown [6] predict that neurocognitive assessment will also likely impact future treatment decisions by providing critical risk-versus-benefit assessments of different treatment regimens. Tumor control, short- and long-term neurotoxicity, and rates of neurocognitive decline are all outcomes that may vary among different treatment regimens and subsequently inform treatment decisions [6].

Neurocognitive outcomes also provide objective, quantifiable data regarding how patients are functioning beyond their physical status and ability to engage in activities of daily living. Cognitive status plays a vital role in the overall well-being and quality of life of patients. Cognitive impairments, more often than physical disability, prevent individuals from continuing to work, which has implications for quality of life [7–9]. In a needs-assessment survey, spouses and caregivers of brain tumor patients also reported neurobehavioral changes were among the most prominent problems encountered [8].

Selecting test batteries to assess neurocognitive status in clinical trials is challenging due to competing demands for brevity versus sensitivity. Brevity is important to limit the demand on patients’ stamina and allow time for measurement of other outcomes. Brief batteries are also more economical (to keep research costs reasonable) whereas longer batteries offer greater sensitivity in detecting cognitive impairment and change. In selecting batteries for clinical trials, researchers generally try to sample from multiple cognitive domains (e.g., processing and motor speed, attention, visuospatial, language, memory, executive function), and also try to select tests that are sensitive to generalized dysfunction. This is an attempt to detect potential focal changes due to tumor effect, and also to detect more general dysfunction that might be due to medication or other factors. The selection of appropriate tests for clinical trials is complicated further by the range of tumor locations and associated cognitive symptoms possible in brain tumor populations.

The cognitive tasks used in clinical trials are typically selected from more comprehensive neuropsychological batteries that are commonly used in clinical settings. In a comprehensive clinical neuropsychological evaluation, multiple cognitive domains are assessed, and typically multiple tests within a domain are obtained. This approach enhances the reliability and sensitivity of the evaluation. Furthermore, a single domain, such as memory, can be divided into different sub-processes such as learning, immediate versus delayed recall, and free recall versus recognition. This evaluation of distinct sub-processes provides information on the integrity of discrete anatomical regions and processing systems; and patterns can indicate certain clinical syndromes. For clinical trials, a subset of these tasks and sub-processes must be sampled. Often, a research battery must be trimmed to under an hour, twenty minutes, or even less. This is especially true for complex multi-center protocols where multiple outcome measures are being administered, patients have limited time, and training demands on research staff must be kept to a minimum.

Psychomotor processing speed, attention, visual tracking, learning, memory, mental flexibility, problem solving, and verbal fluency have been implicated as cognitive domains demonstrating impairment in samples of patients with brain tumors [6, 10–17]. However, variability in the neurocognitive tasks selected and the length of the batteries, which have ranged from approximately 17 to 60 min, hinders comparisons between trials [2, 6, 13, 15, 17, 18]. Fortunately, the feasibility of including neurocognitive outcomes in clinical trials has been established by multiple studies [14–16] and therefore it is now appropriate to focus on refining neurocognitive assessment for this type of research.

In conclusion, selection of specific cognitive tasks for use in clinical trials can be challenging, pitting the sensitivity of sampling multiple domains against the need for brevity. This study aids the development of economical neurocognitive batteries for clinical trials by isolating tests and domains that are most likely to detect effects on a group level in a brain tumor population.

Methods

This is a retrospective study of 68 brain tumor patients who were receiving radiation therapy at Mayo Clinic Rochester and were referred for neuropsychological evaluation as part of standard of care. All patients completed the Repeatable Battery for the Assessment of Neuropsychological Status (RBANS) as part of the neuropsychological consultation [19]. The RBANS is a well standardized battery which includes twelve subtests with widely-used task formats and takes approximately 20–30 min to administer. These subtests converge on five cognitive domains titled, Immediate Memory, Visuospatial/Constructional, Language, Attention, and Delayed Memory. The twelve subtests are List Learning, Story Memory, Figure Copy, Line Orientation, Picture Naming, Semantic Fluency, Digit Span, Coding, List Recall, List Recognition, Story Recall, and Figure Recall. Age-corrected normative data were used to evaluate performance on individual subtests [20].

Medical records were reviewed by a neuropsychologist (SKL) to collect demographic and treatment related information, including gender, age at time of neuropsychological evaluation, education, race, initial or recurrent incidence of brain tumor, type of tumor, location of tumor, treatment (biopsy, surgical resection, radiation therapy, chemotherapy), and prior treatment if applicable. Ambiguous cases of tumor grade and location were reviewed by a radiation oncologist (PDB) to ensure accuracy. The data was entered into a Microsoft ExcelTM spreadsheet and analyzed using the software R [21].

This study was approved by the Mayo Clinic Institutional Review Board and all participants gave written consent for their medical records to be reviewed for research purposes.

Statistics

The primary parameters measured in this study were recorded using standard statistical descriptors of central tendency (mean and median) and dispersion (range and standard deviation (SD)). A conservative threshold of two SDs below published age means was used to indicate statistically significant impairment on the RBANS subtest scores. The total number of patients impaired on at least one subtest was 40. All analyses regarding sample impairment were conducted on the subgroup of 40 patients who performed within the impaired range on the RBANS.

To identify subsets of RBANS subtests that would capture most of these 40 patients that were classified as impaired on at least one subtest, all combinations of subtests were analyzed and the number of patients classified as impaired were recorded. For example, there are 66 unique combinations of 2 subtests. All unique combinations of 2 subtests through 11 subtests were created and analyzed for the percentage of the 40 patients classified as impaired.

Results

Demographics

A total of 68 patients with brain tumors were evaluated. The demographic data of the participants are presented in Table 1. This was a predominantly Caucasian sample of equal numbers of males and females, who were an average of 52 years old with an average of 14 years of education. The majority of patients had a first-time diagnosis of a primary brain tumor, had a glioma (60% high-grade glioma and 18% low-grade glioma), had undergone biopsy or surgical resection of the tumor, and were receiving focal radiation therapy at the time of the neuropsychological evaluation. At the time of neuropsychological evaluation, the majority of patients were in the beginning of radiation therapy. Sixty percent of the patients were also receiving concurrent chemotherapy (temozolomide). Patients completed neuropsychological evaluations a median of 36 days post-biopsy and a median of 40 days post-surgical resection.

Table 1 Demographic and tumor data
Full size table

Impaired RBANS subtests

Forty patients (58.82%) were impaired on at least one subtest, performing two or more SDs below published age means. There were no statistically significant differences in tumor type (χ = 1.8997, df = 1, p = 0.1681), laterality (χ = 0.7931, df = 3, p = 0.8511), and localization (χ = 4.678, df = 3, p = 0.1969) between the impaired and nonimpaired subgroups.

Individually, the Coding, Story Recall, and List Recognition subtests identified the highest number of patients as impaired at 52.5, 52.5, and 45% of the impaired subgroup respectively. In contrast, the Line Orientation and Picture Naming subtests individually identified the lowest number of patients as impaired, both at 12.5%.

Table 2 summarizes the number of patients identified by the best combinations of subtests. The best two-subtest combination (Figure Copy and Story Recall) identified 70% (28/40), while the best three-subtest combination (Figure Copy, Coding, and Story Recall) identified 82.5% (33/40). A combination of four subtests (Figure Copy, Coding, List Recognition, and Story Recall) captured 90% (36/40) of the patients who performed within the impaired range on the RBANS. This four-subtest combination maximizes the number of correctly identified impaired patients while remaining brief. Combinations for 5 and 6 subtests were only marginally better than the combination of 4 subtests with 92.5 and 95% of the impaired sample identified respectively.

Table 2 RBANS subtests combinations
Full size table

Discussion

In clinical trials, where demands for brevity and economy and competing demands for sensitivity co-exist, inclusion of cognitive tasks sensitive to cognitive impairment in the specific group under study is critical. These results suggest that verbal memory, psychomotor speed, and visuoconstruction measures may be the most sensitive domains to assess when evaluating cognitive impairments in patients with primary brain tumors.

Our results are consistent with previous reports of cognitive impairments in patients with primary brain tumors, which have identified psychomotor processing speed, attention, visuospatial processing, visual perception, learning, memory, mental flexibility, problem solving, and verbal fluency impairments [6, 10–17]. The consistency of processing speed, visuoconstruction, and memory deficits in past studies as well as the present study, which employed a conservative threshold of two SDs to indicate impairment, supports the inclusion of these cognitive domains in clinical trials. An advantage of the current study over previous reports is that the RBANS incorporates the use of a 540-adult standardization sample across its subtests, allowing for clean comparison across tasks, while using widely-used neurocognitive task formats [19]. Batteries used in previous work generally used different normative samples for each task, complicating comparisons.

Other batteries that have been proposed for use in brain-tumor clinical trials include a short, repeatable battery of motor, speech, and short-term memory function proposed by Grant et al. [22, 23]. Meyers and colleagues have also successfully used a neurocognitive battery, which assesses verbal learning and memory, verbal fluency, visuo-motor speed, and executive function, in multinational drug trials [4, 6]. Neither of these proposed batteries include visuoconstruction tasks, which have revealed impairment in multiple samples of patients with brain tumors, including the present study. A hierarchical approach for individualized neuropsychological assessment of patients with brain tumors was proposed, but required a total completion time of approximately one hour, which may not be suitable for all clinical trials [2]. When faced with the demand for sensitive but brief batteries, Weitzner and Meyers [23, p. 171] assert that tests should attempt to assess “the basic skills most required for functional independence”, including speed of information processing and right hemisphere function.

The RBANS presents many desirable features for investigators. It is convenient in length, uses proven formats, and has two forms, facilitating serial testing. Its strong standardization (N = 540) using a single sample across all tasks is also a significant asset. At the same time, the RBANS has some weaknesses important to consider, particularly with regards to its potential use in clinical trials. The abbreviated format of the RBANS subtests can result in a ceiling effect which might make it more difficult to detect subtle changes in brain tumor patients. Similarly, the abbreviated format of the RBANS limits its sensitivity, which may result in missed information about more subtle cognitive impairments that could occur in brain tumor patients. Executive functions are also minimally assessed by the RBANS and inclusion of this domain in research on the cognitive impairments observed in brain tumor patients is imperative, as executive functions (i.e., problem solving, mental flexibility, etc.) have great implications for patient safety and ability to engage in important activities of daily living [23].

While the RBANS itself may be less than ideal for use in some clinical trials, analysis of performance on RBANS subtests is well suited to directly inform selection of neurocognitive tests. The intent here was not to evaluate the use of the RBANS itself, but rather to take advantage of its uniform standardization sample and use of common test formats. For example, since the Coding subtest was found to be useful in identifying impaired patients, researchers designing a brain-tumor trial might consider including the Symbol Digit Modalities Test or the Digit Symbol-Coding subtest from the Wechsler Adult Intelligence Scale—Third Edition to tap the same underlying ability using a longer, and thus probably more reliable measure.

Similarly, the finding that Figure Copy was useful in this study might suggest the use of the Rey Osterrieth Complex Figure Test, which is widely used, probably has a higher ceiling than the RBANS copy task, and might avoid some scoring concerns with the RBANS figure [24]. However, the administration and scoring procedures of the Rey Osterrieth Complex Figure Test are complex and may be impractical to include in multi-site clinical trials. While other more basic visuoconstruction tasks may not be sensitive to subtle cognitive impairment in brain tumor patients, evaluation of visuoconstruction and visuospatial tests with planning and organizational demands in brain tumor patients are warranted to further identification of cognitive impairments in this patient population. Careful selection of sensitive tasks in relevant domains could lead to a focused battery that could be conducted within reasonable time constraints [6].

As this study included a heterogeneous sample of patients with a first-time diagnosis of primary brain tumor, primarily consisting of high- and low-grade gliomas and meningiomas, it is important to examine how the sample characteristics of this study influence the results obtained. Our sample included more left than right-hemisphere tumors and therefore expectedly, significant verbal memory impairments were observed. In addition, almost all of the patients in our sample were receiving radiation therapy at the time of the neuropsychological assessment and 60% were also receiving concurrent chemotherapy (temozolomide). Concurrent treatments in clinical trials are common and so we believe that our sample represents a typical group of study patients. While it is not possible to infer causation when multiple factors exist that may be contributing to cognitive impairment, future research is needed to identify the separate and synergistic effects of various factors on cognitive function including direct tumor effects, medication side effects (particularly antiepileptic medications), or effects of radiation and chemotherapy treatments.

Additional sample characteristics, including surgical treatments and radiation dosage, were also recorded. In this study, neuropsychological evaluations were conducted well after the immediate post-surgery recovery period, minimizing the influence of acute effects from biopsy or craniotomy and standard post-surgical medications. Moreover, the majority of patients were in the beginning of radiation therapy, limiting the potential impact of radiation side effects, such as fatigue, on neuropsychological test performance. Future larger studies could also evaluate any impact of demographic variables including age, gender, and race.

This study provides preliminary data to support the inclusion of verbal memory, psychomotor speed, and visuoconstruction measures in clinical trials with brain tumor patients. The results require replication, preferably using non-RBANS measures of the same cognitive constructs to ensure that the results were domain-specific and not just task-specific. Fortunately, most of the RBANS subtests are based on commonly-used measures that are longer and might provide more range to measure subtle changes. Future research could establish whether using these longer, commonly-used measures improves detection of cognitive impairments in patients with brain tumors.

More research on optimal neurocognitive tasks to assess executive functions (i.e., problem solving, mental flexibility, etc.) is needed, as this domain has great implications for patient safety and ability to engage in important activities of daily living [23] and, as previously noted, is minimally assessed by the RBANS. Another important next step is to identify neurocognitive tests sensitive to change in cognition over time, whereas this study looked at a single timepoint. This will enable longitudinal evaluation of patient responses to treatment agents in clinical trials.

This study identifies neuropsychological tests sensitive to cognitive impairment in patients with brain tumors and furthers the development of brief and economical neurocognitive batteries for clinical trials that are most likely to detect effects on a group level. It provides empirical support for the inclusion of psychomotor processing speed, visuoconstruction, and verbal memory domains in clinical trials of brain tumor therapies.