Keywords

These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

Introduction

In spite of the risks to development, children and adults with metabolic disorders may not receive neuropsychological evaluations as part of routine care in the metabolic clinic. This situation occurs because of limited access to a psychologist who specializes in metabolic disorders and because of challenges to obtaining insurance approval for assessments. Consequently, individuals with these disorders often go years without a neurodevelopmental or neuropsychological evaluation and thus receive needed interventions only after symptoms become severe and are far more difficult to treat. Moreover, research into the neuropsychological effects of metabolic disorders is hindered by the lack of uniform follow-up assessments.

Recognition of the need for a method for assessing psychological functioning and for gathering a standard data set in metabolic disorders led a team of 10 psychologists and a psychiatrist to select instruments to serve as a Uniform Assessment Method. The goal was to develop validated, short, and inexpensive standardized protocols that would be relevant throughout the life span, could be completed by parents or the affected adults, could be available in Spanish as well as English, and could be administered, scored, and interpreted by non-psychologists. In addition, instruments were chosen for their power to identify children or adults in need of further evaluation, treatment modifications, or closer monitoring (Waisbren and White 2010). Among the instruments selected were the Adaptive Behavior Assessment System, Second Edition (ABAS-II) (Harrison and Oakland 2003), and the Behavior Rating Inventory of Executive Function (BRIEF) (Gioia et al. 2000). Other instruments included were the Behavior Assessment System for Children – Second Edition (BASC-2) (Reynolds and Kamphaus 2004) and, for adults, the Beck Depression Inventory, Second Edition (Beck et al. 1996), and the Beck Anxiety Inventory (Beck and Steer 1993). While the questionnaires selected were already well validated in the general population, their psychometric properties had not been described for individuals with metabolic disorders. The purpose of this Uniform Assessment Method is not to find a substitute for intelligence testing or comprehensive neuropsychological evaluations but rather to establish a quick, valid, and feasible means to assess functioning for routine clinical follow-up, registries, and research studies.

In this report, results from the ABAS-II and BRIEF obtained on individuals with urea cycle disorders (UCDs), phenylketonuria (PKU), galactosemia, and fatty acid oxidation disorders (FAODs) were compared to results obtained from neuropsychological testing performed on the same day. PKU, galactosemia, and FAODs were selected for comparison to UCDs because they represent metabolic disorders with neuropsychological impact for which comparable test results were available from other research studies or medical records. We did not include the BASC-2 or the Beck Depression or Anxiety Inventories because they were not routinely available in the research databases or medical records.

Metabolic Disorders

Urea Cycle Disorders

UCDs interfere with the hepatic ammonia detoxification pathway, leading to hyperammonemia and other biochemical abnormalities. Absence or deficiency of the first four enzymes in the urea cycle (carbamoyl phosphate synthetase I (CPSI), ornithine transcarbamylase (OTC), argininosuccinate synthetase (ASS), argininosuccinate lyase (ASL)) leads to hyperammonemia within the first few days of life. Deficiency or absence of arginase 1 (ARG1) leads to other neurological and developmental problems, although hyperammonemia is not as common (Summar et al. 2008). Ornithine transcarbamylase (OTC) deficiency, the most common of the urea cycle disorders (OMIM 311250), is an X-linked inherited disorder, in which males are more severely affected. Many females remain asymptomatic throughout their lives, although subtle neuropsychological deficits arise that can interfere with functioning in day-to-day life (Gyato et al. 2004; Gropman et al. 2010). The clinical symptoms related to these disorders are variable, ranging from neonatal death due to complications of hyperammonemia to normal cognitive and developmental outcomes throughout life (Krivitzky et al. 2009). Newborn screening has recently become available for several of the urea cycle disorders (Beck et al. 2011), and while early diagnosis and treatment may reduce the frequency and severity of hyperammonemic episodes (Summar 2001), they do not eliminate risks of reduced cognitive abilities, maladaptive behaviors, and poor executive functioning (Krivitzky et al. 2009; Ah Mew et al. 2013).

Phenylketonuria (PKU)

PKU (OMIM 261600) is an autosomal recessive disorder that affects the body’s ability to metabolize phenylalanine into tyrosine, the precursor of the neurotransmitter, dopamine. Untreated PKU is associated with severe intellectual disabilities, seizures, eczema, motor difficulties, and significant behavioral problems. Fortunately, universal, mandatory newborn screening in the United States and in most developed countries around the world permits the identification and early treatment of this inborn error of metabolism. Treatment consists of a low-protein diet and a supplemental formula that provides all the necessary parts of protein without the “offending” amino acid (Scriver and Kaufman 2001). With early treatment, most children with PKU develop well, with intellectual abilities within the average range (Waisbren et al. 2007). However, many individuals experience executive functioning deficits (Christ et al. 2010), attention deficit disorder (Antshel and Waisbren 2003), and psychiatric problems, including anxiety and depression (Brumm et al. 2010).

Galactosemia

Galactosemia (OMIM 230400) is a rare, inherited metabolic disorder in which the metabolism of galactose is impaired, due to insufficiency or absence of the enzyme, galactose-1-phosphate (Fridovich-Keil and Walter 2008). If left untreated, classic galactosemia can cause severe neonatal sepsis, cataracts, and death. Those who survive the newborn period often experience intellectual disability, speech and language delay, and motor deficits. With early identification through newborn screening and treatment with a galactose-restricted diet, the more severe consequences of this disorder are prevented. However, almost every child with galactosemia has speech/language delay, and many have minor to moderate motor deficits (Bosch 2006). Adults also exhibit motor deficits, depression, and anxiety (Waisbren et al. 2012).

Fatty Acid Oxidation Disorders

Mitochondrial fatty acid oxidation is a complex process involving transport of activated acyl-coenzyme A (CoA) moieties into the mitochondria and sequential removal of two carbon acetyl-CoA units. This process in the mitochondria provides energy for many tissues including heart and skeletal muscle and is critically important during times of fasting or physiologic stress (Vockley and Whiteman 2002). Disorders of fatty acid oxidation, the most common of which is medium-chain acyl-CoA dehydrogenase deficiency (OMIM 607008), interrupt this cycle and lead to a deficit in the conversion of fat into energy. Most patients with fatty acid oxidation defects are now identified through newborn screening, and as a result, mortality and morbidity rates have vastly improved (Wilcken 2010). However, developmental delays are not uncommon, particularly in the area of speech and language (Iafolla et al. 1994; Waisbren et al. 2013).

This summary of metabolic disorders exposes the continued neuropsychological and behavioral risks confronting patients with these conditions. Without adequate developmental and neuropsychological screening and follow-up, their needs often remain unrecognized.

Instruments

The Adaptive Behavior Assessment System, Second Edition (ABAS-II) (Harrison and Oakland 2003), spans the entire life span, from early infancy to adulthood. The ABAS-II is a checklist of a broad range of skill areas related to development, behavior, and cognitive abilities. Parents or other informants can complete the age-appropriate form (0–5 years, 6–21 years). For capable adults there is a self-report form, as well as a parent/informant version. The ABAS-II includes subscales for communication, community use, functional academics, home living, health and safety, leisure, self-care, self-direction, social, and work. A scaled score of 10 ± 3 represents the mean. Four composite scores are derived from the sum of the scaled scores: general adaptive composite (GAC), conceptual, social, and practical. These composite scores have a mean of 100 and a standard deviation of 15. The ABAS-II standardization sample included 1,670 respondents. The ABAS-II GAC has a correlation of 0.54 with the Wechsler Preschool and Primary Scale of Intelligence (WPPSI-III) and 0.41 with the Wechsler Intelligence Scale for Children, Fourth Edition (WISC-IV). Computerized scoring and interpretation programs are now available. This questionnaire takes 10–15 min to complete.

The Behavior Rating Inventory of Executive Function (BRIEF) (Gioia et al. 2000) provides theoretically and empirically derived clinical scales that measure aspects of executive function. Executive functioning can be thought of as the ability to keep information in mind for problem solving. It involves such cognitive processes as memory, attention, planning, organization, and the ability to shift attention from one thought to another. The clinical scales form broad indices of behavior and cognition and an overall score, the global executive composite (GEC). For this study, parent response forms for school-aged children were analyzed, although there is also a preschool version. A self-report form and an informant response form are available for adults, permitting a uniform measure across all ages. All forms were standardized on normative samples representing a broad variety of race/ethnicity, age, and geographical population density. A T score of 50 ± 10 represents the mean of the T-score distribution, and a score of 65 represents 1.5 standard deviations above the mean, which is the recommended cut point for an “abnormally elevated” score and is considered “clinically significant.” The questionnaire is completed within 10–15 min.

Methods

This study incorporated information from research data sets and medical records. Parents of children with urea cycle disorders or adults with urea cycle disorders participating in a longitudinal study completed the ABAS-II and BRIEF as part of comprehensive neuropsychological evaluations (Seminara et al. 2010; Krivitzky et al. 2009; Ah Mew et al. 2013). Additional reports on results from this study are forthcoming. Data on children with PKU, galactosemia, and fatty acid oxidation disorders were obtained from a review of medical records from Boston Children’s Hospital, where the questionnaires from the Uniform Assessment Method have been used for the past 5 years at the time of routine psychological evaluation. (See Waisbren et al. 2013, for a review of results on children with FAODs.) Data on adult women with PKU were obtained from a study of maternal PKU (Waisbren et al. in press) that included administration of the ABAS-II, BRIEF, and intelligence testing. Data on adults with galactosemia were obtained from a study that also included the ABAS-II, BRIEF, and neuropsychological assessment (Waisbren et al. 2012). Approval for these various studies was obtained by the Boston Children’s Hospital Committee on Clinical Investigations (IRB) or other metabolic centers where the research was being conducted. Approval was also received to conduct a medical record review of Boston Children’s Hospital patients.

Mean and standard deviations were calculated to describe the scores on the ABAS-II, BRIEF, developmental tests, intelligence tests, and scales that measured aspects of executive functioning. T tests and analysis of variance were used to evaluate differences in scores between males and females and across the different disease groups. We categorized results according to the following age groups: infants (<3 years), preschool children (3–5 years), school-aged children (6–17 years), and adults (18+ years).

Results from subjects with PKU, galactosemia, and fatty acid oxidation disorders were combined into an “other metabolic disorders” group, due to relatively few cases with these disorders. This “other metabolic disorders” group was compared to results from subjects with UCDs. For 41 cases in the urea cycle group, a parent or informant completed the ABAS-II for adult subjects. Analyses were performed with and without these cases, and occasionally a difference in results was noted. Occasionally, subjects were rated more than once on the ABAS-II or BRIEF at the time of a neuropsychological evaluation, and their data were included in the analyses. Results from the BRIEF were derived only from school-aged children and adults.

Pearson correlation coefficients were computed to assess strength of relationships between the ABAS-II and measures of cognitive functioning. For infants, the GAC from the ABAS-II was compared to the cognitive composite score from the Bayley Scales of Infant and Toddler Development, Third Edition (Bayley 2005). For older children and adults, the ABAS-II GAC was compared to the full-scale IQ obtained from the Wechsler Preschool and Primary Scales of Intelligence, Third Edition (WPPSI-III) (Wechsler 2002), the Wechsler Intelligence Scale for Children, Fourth Edition (WISC-IV) (Wechsler 2003), the Wechsler Adult Intelligence Scale, Fourth Edition (WAIS-IV) (Wechsler 2008), or the Wechsler Abbreviated Scale of Intelligence (WASI) (Wechsler 1999), depending on the age of the subject and instrument used for clinical or research purposes. Pearson correlations were also calculated to assess the strength of associations between the BRIEF Global Executive Composite (GEC) and two scales from instruments that measure aspects of executive functioning, specifically working memory as measured by the California Verbal Learning Test (CVLT), Trial 5 (Delis et al. 2000; Tremont et al. 2000), and planning as measured by the block design subtest from the Wechsler intelligence tests (Weiss et al. 2006).

Concordance (agreement) between the ABAS-II and developmental or intelligence tests provided a measure of validity of the ABAS-II to identify children at risk and in need of further evaluation. A score of 85 (1 standard deviation below the published mean) represented the cut point for determining a risk for problems in adaptive functioning on the ABAS-II General Adaptive Composite (GAC) and for risk of developmental delay or intellectual deficits on the Bayley Cognitive Composite and full-scale IQ on the WPPSI-III, WISC-IV, WAIS-IV, or WASI.

Concordance between the BRIEF GEC and the two scales measuring aspects of executive functioning was also determined. For the BRIEF GEC, the cut point for executive functioning deficits is ≥65, as specified by the BRIEF manual. On the CVLT trial 5, a score one standard deviation below the mean (≤ −1.0) represented risk for executive functioning deficits. For the block design subtest from the Wechsler scales of intelligence, the cut point for risk of deficits was set at 7 or below (or ≤40 on the WASI). In addition to measuring agreement, percents for receiver operating characteristic (ROC) area under the curve (AUC) were calculated.

Results

As noted in Table 1, 516 individuals were rated on the ABAS-II and 265 individuals were rated on the BRIEF. More females than males were included in the study. The mean age of the sample was 13.8 years but ranged from infancy to middle age. Differences in scores on the ABAS-II or BRIEF between males and females were noted only when parent ratings of adults with UCDs on the ABAS-II were included with the self-ratings of adults with UCDs (p < 0.04).

Table 1 Total number and mean age of individuals with each disorder rated on the ABAS-II and BRIEF and number of males and females
Full size table

The mean score on the ABAS-II GAC was lower for the combined UCD cases (86 ± 23) than for the combined other metabolic disorders (mean = 99 ± 16, p < 0.0001). Similarly, scores on the BRIEF were higher (indicating greater difficulties in executive functioning) for the UCD cases (mean = 57 ± 12) compared to the cases with other metabolic disorders (mean = 53 ± 13, p < 0.01). Significant differences were not noted on the ABAS-II or BRIEF among individuals with PKU, galactosemia, or FAODs.

As noted in Table 2, mean scores on the ABAS-II GAC were within the low-average to high-average range for all study groups as were mean scores for the Bayley Cognitive Composite and full-scale IQ. The correlations between the ABAS-II GAC and the Bayley Cognitive Composite in infants or full-scale IQ in children and adults ranged from 0.20 to 0.70, with the correlations consistently high among school-aged children 6–17 years of age. For children and adults with urea cycle disorders, mean scores on the two measures were within 1–4 points. For children with other metabolic disorders, mean scores derived from parent ratings were 9 to 18 points lower than scores on tests administered directly (Bayley Scales or Wechsler intelligence tests), despite generally high correlations. For adults with the other metabolic disorders, the opposite was true, with mean self-rated ABAS-II GAC 10 points higher than full-scale IQ.

Table 2 Mean (standard deviation) scores on the ABAS-II general adaptive composite (GAC), Bayley cognitive composite, correlations with IQ tests, and concordance with scores below the cut point for the ABAS-II GAC
Full size table

For the UCD cases, the communication subscale from the ABAS-II correlated with the Bayley Language Composite score (r = 0.60, p < 0.000001). The motor subscale from the ABAS-II correlated with the Bayley Motor Composite score (r = 0.57, p < 0.00001). These data were not available for subjects with the other metabolic disorders.

Table 3 presents comparisons between the BRIEF global executive composite (GEC) and scores on two tests that measure aspects of executive functioning (working memory and planning). Among adults with UCDs, confidence intervals for the correlations indicate that the GEC was associated with working memory as measured by the CVLT. For children and adults with UCDs, the BRIEF GEC also correlated with scores on block design, measuring planning and organization. Although correlations were high between the GEC and aspects of executive functioning for subjects with other metabolic disorders, these associations did not reach significance. In general, individuals with UCDs exhibited deficits on both the CVLT and block design. For individuals with the other metabolic disorders, scores on block design remained relatively intact, while particular vulnerabilities were noted in working memory, as measured by the CVLT.

Table 3 Pearson correlations between BRIEF questionnaires global executive composite (GEC) and tests of executive functioning by study group and age and concordance with scores above the cut point for the BRIEF and below the cut point on the block design or CVLT
Full size table

For the ABAS-II GAC, concordance refers to agreement with the Bayley Cognitive Composite or full-scale IQ in terms of a score above or below the cut point indicative of risk for developmental delay or cognitive deficit. As noted in Table 2, overall concordance ranged from 69 to 90%. As noted in Table 3, concordance rates between the BRIEF GEC and either the CVLT or block design ranged from 52 to 80%, with agreement generally higher for the CVLT, measuring working memory, than for block design, measuring planning as well as visual spatial abilities.

For the ABAS-II GAC, ROC AUC summarizing overall sensitivity and specificity is 69% for infants with UCDs and 24% for infants with other metabolic disorders. The ROC AUC is 80% or higher for school-aged children and above 70% for adults with UCDs and other metabolic disorders. For the BRIEF GEC, the ROC AUC is 60% for school-aged children with UCDs and 84% for school-aged children with other metabolic disorders. For adults, it is above 60% for both individuals with UCDs and individuals with other metabolic disorders.

Discussion

In this study, 515 individuals with metabolic disorders contributed data from the ABAS-II and 265 contributed data from the BRIEF. Data from these questionnaires were compared to scores obtained through neurodevelopmental or neuropsychological testing. This study focused on UCDs, galactosemia, PKU, and FAODs because data on these populations were available. The assessment method proposed here could easily be applied to the many other metabolic disorders that also present at various ages and are associated with a broad range of psychological outcomes. Mean scores on the ABAS-II GAC and the Bayley Cognitive Composite or IQ tests from individuals with urea cycle disorders were within 4 points. For the other metabolic disorders, the ABAS-II GAC mean scores tended to be lower than the Bayley Cognitive Composite or full-scale IQ for children, but higher than full-scale IQ for adults.

The correlations between the ABAS-II GAC and full-scale IQ ranged from 0.62 to 0.68 in school-aged children, considerably higher than the correlation between the GAC and the WISC-IV full-scale IQ (r = 0.41) reported in the ABAS-II manual. When discrepancies occurred in this study, parent ratings usually indicated problems, whereas the cognitive composite or full-scale IQ was within the average range. A review of medical records and scores on other neuropsychological tests suggested that nearly all these children had attention deficits, hyperactivity, or other behavioral problems. These children received higher scores on an IQ test than on the ABAS-II, which identifies potential behavioral as well as cognitive risk factors.

Strong correlations were noted among scores from UCD individuals between the BRIEF GEC and block design for school-aged children and adults. The GEC was also highly correlated with the CVLT for adults with UCDs. Scores on the BRIEF from individuals with the other metabolic disorders were not significantly correlated with scores on the CVLT or block design. With test scores from only 9 to 35 individuals, this aspect of the study may have been underpowered.

The analyses for this study were limited to the summary scores on the ABAS-II and BRIEF. The subscales for both these instruments may prove to be valid as additional outcome measures. However, assessment of these subscales was beyond the scope of this study, which relied on existing data sets.

It is important to remember that adaptive functioning, as assessed by the ABAS-II, is not equivalent to intellectual functioning, since it describes a much broader range of behaviors, including social relationships, self-help skills, and ability to get along in the community. Similarly, executive functioning as measured by the BRIEF is not equivalent to isolated skills of memory and planning but includes attention, inhibition, organization, and other abilities. Given that the potential bias is in the direction of overidentification of at-risk children, assessments based on the ABAS-II and BRIEF will be unlikely to miss a child who has a developmental delay or deficit in executive functioning.

Ideally, the ABAS-II and BRIEF will be incorporated in every metabolic disorder study that includes measures of functioning so that results can be compared across studies and meta-analyses can be easily conducted. Uniform assessments with the ABAS-II and BRIEF can overcome the problems of small sample sizes and insufficient resources when examining functioning in newborn screening or when conducting long-term follow-up studies.

Use of the ABAS-II and BRIEF for assessing every child identified with a metabolic disorder will increase the likelihood of insurance coverage for further evaluations when needed and permit early identification of those requiring early interventions or treatment modifications. Moreover, inclusion of anonymous results from these measures in longitudinal registries will provide critical data for evaluating the efficacy of newborn screening, which has expanded greatly in the past decade and is likely to increase further as new screening technologies become available.

These instruments are available as paper forms or online and can be administered on a tablet or laptop computer. They can be administered by non-psychologists. They extend across all age groups and exist in Spanish and other languages as well as English. Reports can be generated electronically, and scores can be entered into databases for research purposes or tailored for feedback to patients, families, and health-care providers.

Future studies are needed to evaluate the sensitivity of the ABAS-II and BRIEF as pre- and posttests in medication or other treatment trials. Additional studies are needed to validate the other tests recommended in the Uniform Assessment Method described by Waisbren and White (2010), including the BASC, Beck Anxiety Inventory, and Beck Depression Inventory to assess psychiatric and emotional well-being.

In conclusion, the results of this study indicate that the ABAS-II and BRIEF together can confidently be used as an adjunct or supplementary method for clinical follow-up and for research on functional status in infants, children, and adults with metabolic disorders. This method can serve as an “early warning” system to detect neuropsychological deficits for which a full evaluation would be important. Ideally, the ABAS-II and BRIEF can be administered as part of regular follow-up for metabolic disorders to monitor treatment effectiveness and disease progression. While comprehensive neuropsychological evaluations provide a more differentiated picture of the types and severity of deficits associated with these disorders, the expense, time commitment, and limited access to such testing has prohibited and probably always will prohibit its use as a method for routine follow-up in clinical settings or for measures of functional status in registries or large research studies. The ABAS-II and BRIEF can fill this void.