Introduction

The Diagnostic and Statistical Manual, 5th edition of the American Psychiatric Association [1] included in the Conditions for Further Study section, a disorder intended to capture the range of mental health and developmental problems associated with prenatal alcohol exposure (PAE), referred to as Neurobehavioral Disorder Associated with Prenatal Alcohol Exposure (ND-PAE). The proposed diagnostic criteria for this disorder were developed by surveying the large body of evidence from experimental animal research, longitudinal prospective human studies, and clinical research documenting the teratogenic effects of PAE throughout the lifespan [2]. ND-PAE can be diagnosed either in the presence or absence of the physical effects of PAE (i.e., a diagnosis of Fetal Alcohol Syndrome (FAS) or partial FAS (pFAS)). The development of ND-PAE as a mental health disorder marks an important step in the appropriate identification and treatment of those individuals with a lifetime of behavioral and mental health problems associated with PAE [3].

For purposes of making this diagnosis, neurobehavioral outcomes were clustered into three domains, each of which needs to be endorsed to receive the ND-PAE diagnosis [2]. These domains are as follows: (1) neurocognitive, (2) self-regulation and (3) adaptive functioning. The neurocognitive impairment (NI) domain includes five potential symptoms (impairment in global intellectual functioning, executive functioning, learning, memory, or visual-spatial reasoning) of which only one has to be endorsed for the domain to be endorsed. The self-regulation (SR) domain includes three potential symptoms (impairment in mood or behavioral regulation, attention deficits, or impulse control) of which only one has to be endorsed. Finally, the adaptive functioning (AF) domain has four symptoms of which a minimum of two areas are needed for endorsement of the domain. These include impairments in communication, social, daily living skills and motor skills with at least one of the two symptoms being either communication or social impairments (AF 2 of 4 criteria). Although the symptoms of ND-PAE have many shared features with other neurobehavioral disorders, the uniqueness of the proposed disorder is derived from the convergence of the three proposed domains and subtleties of the symptoms. For example, children with a history of PAE have social deficits resulting from being socially disinhibited as compared to children with autism who are characterized as being “asocial” or socially inhibited [4, 5].

For the condition to be accepted as a unique psychiatric disorder, additional diagnostic and taxometric research is still needed. Establishing the diagnostic coverage and the homogeneity of symptoms are part of basic criteria for evaluating psychiatric classification [6]. To establish a latent trait of disorder severity, the symptoms should have a high level of internal consistency and the rate of symptom endorsement should vary as a function of endorsement of the overall disorder. Establishing that each symptom provides a unique contribution to understanding disorder severity and that symptoms are not too highly correlated [7] is also part of the process of assessing the internal validity of the disorder as symptom endorsement should aid in differentiating those meeting criteria for the overall disorder.

To examine the internal validity of the proposed disorder, we identified a group of individuals who were diagnosed with FAS or pFAS and had enrolled in a math intervention study [8, 9] where an extensive assessment of their neurobehavioral characteristics were obtained. Evidence of each of the symptoms included in the diagnostic formulation of ND-PAE was collected on these individuals. Individuals with alcohol-related neurodevelopmental impairment often have different neurobehavioral expression of the impact of PAE. Among the participants selected for this study who were diagnosed with FAS or pFAS, we anticipated that there would be differences in the expression of each of the ND-PAE symptoms that would allow a comparison of the cohesiveness of all of the symptoms and the necessity of each symptom in the identification of the latent trait of disorder severity. As there was no clear threshold for symptom severity specified in the DSM-5 proposed diagnostic criteria other than for global cognitive impairment, two possible criterion levels, 1.5 and 1.0 standard deviation units (SD) from the mean on standardized measures of neurobehavioral functioning, were evaluated. Setting a level of 2 SD was considered too restrictive and not representative of the characteristics of the population. In addition, the impact of using one symptom from the adaptive functioning domain (AF 1 of 4) or the recommended AF 2 of 4 criteria was also evaluated. Rate of symptom endorsement, shared variance with other symptoms and overall domain endorsement as well as endorsement of the disorder, internal consistency, and utility in differentiating those meeting criteria for the disorder from those who do not within this sample were used as evaluation criteria to assess the internal validity of the proposed ND-PAE disorder.

Methods

Data from 56 participants between the ages of 3 and 10 years of age who enrolled in a math intervention study [8, 9] were used. Participants were identified from medical records under a HIPAA partial waiver and recruited from a multidisciplinary diagnostic clinic in the Atlanta metropolitan area and were required to have been diagnosed with FAS or pFAS. A physical examination was conducted on all participants by a pediatric geneticist using a standard pediatric dysmorphology checklist [10], where characteristics associated with the disorder are listed and weighted based on their saliency for the FAS diagnosis (e.g., hypoplastic philtrum, small palpebral fissures, and thin vermillion receive 3 points and clindactyly receives 1 point). Scores ≥ 10 are assumed to indicate significant alcohol-related dysmorphology. Additional details on the diagnostic clinics methods are available [11]. Participants were not required to have a math disability to enroll in the intervention as the intervention was designed to enrich children in the area of math development in order to prevent math disabilities.

To participate, children were also required to have a stable home placement for the 6 months before enrollment and throughout the study. Participants were excluded if cognitive functioning was in the Moderate or Severely Intellectually Deficient range or if they had other mental health problems that may have interfered with their ability to benefit from academic instruction (i.e., autism). Children who had a dual diagnosis of attention deficit/hyperactivity disorder (ADHD) were not excluded. After an initial screening, qualifying guardians completed a consent procedure approved by the Human Subjects Committee of the Emory University School of Medicine.

Children received comprehensive neurodevelopmental assessments over the course of the intervention. Assessments were done during two pretest assessment sessions, again at post-test 1 conducted approximately 30 days after completing the intervention, and finally, at post-test 2 conducted 6 months after completing the intervention. Measures of behavioral and academic functioning were obtained at each assessment but only those collected at baseline were used for this analysis. Other measures collected at one of the post-test assessments were used for this analysis as indicated below. Additional details regarding the participants, recruitment and data collection are available [8, 9, 12].

Symptom Mapping

A symptom map was created from the available data and coded relative to the presence or absence of each of the ND-PAE symptoms using the ≥ 1.0 and ≥ 1.5 SD cut-off values relative to the reference samples of a given standardized test. For some symptoms, evidence was obtained from clear documentation of a problem in a given area or from tester ratings of problems with the child’s behavior during the assessment process. Details regarding the measures selected to provide information for each of the symptoms are discussed below. Endorsement of a symptom required only one positive endorsement on the measures collected related to each symptom. The specific measures used and the percentage of endorsements among participants for each measure are provided by cut-off level in Supplementary Table 1.

Neurocognitive Domain Measures

To assess global intellectual impairment, the Differential Ability Scales (DAS) [13], which was administered to all participants, and the Kaufman Brief Intelligence Test (K-BIT) [14], which was administered to participants who were 5 years of age and older, were used. The DAS is a nationally standardized test of children’s cognitive status. Performances on specific processing tasks are aggregated into cluster scores of the child’s verbal, nonverbal, and spatial cognitive skills. Finally, an index of the child’s overall cognitive functioning (the General Conceptual Ability score, GCA) is generated. The K-BIT is a screener test of intellectual functioning skills consisting of 2 subtests, which yield Vocabulary, Matrices, and Composite IQ scores. The GCA from the DAS and the K-BIT IQ Composite scores were used to assess impairments in global intellectual functioning. As per the criteria specified in the DSM-5 [1], scores of less than 70 were considered positive for endorsing impairment in global intellectual functioning.

Assessment of executive functioning skills was limited to the Metacognitive Index of the Behavior Rating Inventory of Executive Functions Parent Report Form (BRIEF) [15]. The BRIEF is a standardized rating scale designed to assess executive function skills in everyday life. The test consists of eight subscales (Inhibit, Shift, Emotional Control, Initiate, Working Memory, Plan/Organize, Organization of Materials, and Monitor) and three summary index scores (Global Executive Function, Behavioral Regulation, and Metacognition). The Metacognitive Index (MI) was used to assess executive functioning skills as this assesses the child’s cognitive inhibitory control and ability to plan, organize, and sequence. The overall executive functioning summary score from this measure was not used as it combines information from the MI and the Behavioral Regulation Index (BRI), which assesses behavioral control or self-regulation.

All children were administered the Test of Early Mathematical Ability [16] and the Test of Early Reading Ability, 2nd edition [17]. These are standardized tests for children that assess mathematical or reading concepts, respectively. For children who were 5 and older, the KeyMath-R/NU [18] was also administered to assess mathematical skills. In addition, endorsement on a pretest structured parent interview of receiving special education services, having an individualized educational plan (IEP), or repeating a grade was also a positive indicator.

To assess impairment in working memory, the Auditory Working Memory (AWM) subtest of the Woodcock-Johnson Tests of Cognitive Abilities [19] was administered. This test gives the child a list of numbers and words and requires the child to repeat them back listing the words first in alphabetical order and the numbers in numerical order. From the Wechsler Intelligence Scale for Children, 3rd edition, Processing Instrument [20], Letter Span and Spatial Span were administered to children who were 6 years of age and older. The Letter Span subtest consists of strings of letters that the child is asked to repeat back and is recognized as an index of short-term working memory. The Spatial Span subtest has two conditions that are then aggregated for a total score. The Forward condition is also recognized as measure of short-term memory skills in that it requires the participants to replicate a series of taps on blocks randomly distributed on a board. The Backward condition requires the participant to reverse the series of taps and is recognized as a measure of working memory skills.

Impairment in visual-spatial reasoning was assessed using the DAS Nonverbal and Spatial Cluster and the K-BIT Matrices scores as described above. In addition, the Spatial Relations and Visual Matching subtests from the Woodcock-Johnson, III Tests of Cognitive Functioning [19] were also used. The Spatial Relations task requires a child to identify two or three pieces from a designated target shape and is considered a measure of visual-spatial reasoning. The Visual Matching subtest requires the child to make visual symbol discriminations as quickly as possible and is considered a measure of cognitive efficiency within the visual-spatial domain. Finally, the Visual Perception component of the Developmental Test of Visual-Motor Integration, 4th edition [21] was administered to all participants. This test asks participants to identify which of a series of pictures most closely resembles a target picture.

Self-Regulation Domain Measures

To assess impairment of mood and behavioral regulation symptoms, the Behavioral Regulation Index (BRI) from the BRIEF was used. The BRI captures the child’s ability to shift cognitive set, modulate emotions, and exert appropriate inhibitory control over their behavior. It is comprised of the Inhibit, Shift, and Emotional Control clinical scales. The Total Problems scale from the Child Behavior Checklist [22, 23] was also used to assess behavioral regulation problems. In addition to the Total Problems score, summary scores are reported for Internalizing and Externalizing Problem Behaviors. Finally, tester ratings of anxiety and overall inappropriate behavior on the DAS Informal Behavior Scale were also used to assess mood and behavioral regulation symptoms. The tester was required to delineate whether or not the child was more like one of two behavioral traits with five boxes between the two traits. Positive endorsement of anxiety (calm vs. anxious) or problems with behavioral regulation (behaves appropriately vs. behaves inappropriately) symptoms during testing were considered endorsements of symptoms of impairment of mood and behavioral regulation.

To assess impairment in attentional regulation, the Visual Attention subtest from the NEPSY [24] was used. This test is a cancellation task that requires the child to mark out a designated picture or sequence of pictures from a model. The Attention and DSM ADHD scales from the CBCL were also used. In addition, a positive report of a medical diagnosis of ADHD or evidence of ADHD medications was considered positive endorsement of this symptom. On the DAS Informal Behavior Scale, tester ratings of distractibility (attentive vs. distractible) were considered endorsements of problems in attentional regulation skills. Previous research has suggested that tester reports of ADHD symptomatology are valid and correlate with parent and teacher assessments [25].

Finally, to assess impairment in impulse control, the Rule Breaking and Aggression subscales from the CBCL were used. Also, on the DAS Informal Behavior Scale, tester ratings of being fidgety (lethargic vs. fidgety), impulsive (deliberate vs. impulsive), and being haphazard (methodical vs. haphazard) were considered endorsements of problems in impairments in impulse control.

Adaptive Functioning

The Vineland Adaptive Behavior Scale, 2nd edition (VABS: [26]), a widely used standardized measure of adaptive functioning that is well normed, was used. The Parent/Caregiver Rating Form, allows parents to rate adaptive functioning in the following areas: communication, daily living skills, socialization, and motor skills. An overall composite score of adaptive skill functioning is also generated. To assess deficits in adaptive communication, the VABS Communication score was used. In addition, the Figurative Language subtest of the Test of Language Competence-Expanded Edition [27], which was given at the 6-month post-test, was used for children 5 years and older. This subtest assesses the pragmatic use of language by asking the child to explain figurative phrases.

To assess social impairment, the VABS Socialization score was used as well as the CBCL Social Competence scale. In addition, two summary scores (Social Skills and Problems) from the Social Skills Rating System (SSRS) [28], a parent report measure of the child’s social functioning, was used. To assess impairment in independent living skills, the VABS Daily Living Skills and the CBCL Total Competence score were used. Finally, to assess impairment in adaptive motor functioning, the VABS Motor Domain score and the Developmental Test of Visual Motor Integration (VMI) [21], which assesses graphomotor skills, were used.

Each of the measures used as described above serves to operationalize the ND-PAE symptoms for purposes of this study but is not the same as conducting a comprehensive clinical assessment where clinical judgement is used to evaluate a complex array of information, including formalized assessments, client history and interviews, and reports from other’s in the client’s life. Each operationalization of the ND-PAE symptoms does serve to approximate this clinical judgement by incorporating information available from this pre-existing dataset.

Statistical Analysis

The final result of the symptom mapping was a set of 12 binary coded symptoms indicating endorsement of the symptom or not. Comparisons of the endorsement rates for each measure, symptom, domain, and ND-PAE disorder were computed using the four proposed methods of classification. Inter-correlations of symptoms were then computed and examined for negative relationships, which would suggest the symptom may not belong in the diagnostic formulation, and relationships that were too highly correlated [7], suggesting redundant information. Each of the 12 proposed symptoms were then related to the three domains and the overall endorsement of the ND-PAE disorder. Cronbach’s alpha, a measure of internal consistency, was computed for symptoms. A discriminative validity analysis of all symptoms relative to endorsement of the ND-PAE disorder or not was then done. Finally, receiver operating characteristic curves (ROC) were constructed for the 12 symptoms to assess the contribution of each item in accurately discriminating those positive for the ND-PAE disorder from those not meeting criteria. Although we anticipated a high rate of endorsement of ND-PAE symptoms and endorsement of the overall diagnosis in the sample given that all participants included in this study were alcohol-affected and had met criteria for a FAS or pFAS clinical diagnosis, we anticipated some differences in the expression of each of the ND-PAE symptoms and that not all individuals would meet criteria for the disorder. Frequency of endorsed symptoms was also analyzed for differences in gender, ethnicity, age, and family characteristics.

Results

Sample Characteristics

The mean age of the participants was 6.4 years (SD = 2.0) with 26.8% of the sample being less than 5 years, 32.1% falling between 5 and <7 years, and 41.1% being 7 years or older. The sample was 55.4% male. Thirty-nine percent were Caucasian and 57.1% percent were African American. Children were placed predominantly in adoptive homes 66.0% with a non-relative. The average number of adults in the home was 1.8 (SD = 0.8) and the average number of children was 1.7 (SD = 1.7) with incomes ranging from 35,000 to 49,999 US$. Participants had an average birth weight of 2,402.5 (SD = 863) grams and a DAS GCA of 81.6 (SD = 13.7). Participants also had an average dysmorphia checklist score of 15.5 (SD = 6.8) on a scale where scores of 10 or higher are indicative of significant levels of alcohol-related physical features. Additional details regarding the sample are available [8, 9, 12].

Endorsement Rates for Symptoms, Domains, and ND-PAE Disorder

The rate of endorsement for each symptom, domain, and ND-PAE disorder were computed and are displayed in Table 1 for both cut-off values and methods of endorsing the AF symptoms. Using AF 1 of 4 criteria, 82.1% received an endorsement for the ND-PAE disorder using cut-off value of 1.5 SD and 89.3% using a cut-off value of 1.0 SD. Using the AF 2 of 4 criteria, 60.7% received an endorsement for the disorder using cut-off value of 1.5 SD and 83.9% using a cut-off value of 1.0 SD.

Table 1 ND-PAE symptom and domain endorsement by cut-off values used on standardized measures
Full size table

Symptom Characteristics

Table 2 displays the phi coefficients, a measure of association for binary measures, of the proposed ND-PAE symptoms. Coefficients in the top half of the matrix reflect the relationships using the 1.5 cut-off value and those in the lower half reflect the relationships using the 1.0 cut-off value. For the symptoms endorsed based on both the 1.0 and 1.5 SD cut-off value, low to moderate associations or non-significant associations were found among symptoms, suggesting little symptom redundancy.

Overall internal consistency of symptom endorsement resulted in a Cronbach’s alpha of 0 0.77 (F (11,605) = 13.4, p < .000) for the cut-off value of 1.5 SD and 0.74 (F (11,605) = 14.8, p < .000) for the cut-off value of 1.0 SD, suggesting that the differential cut-off value had little impact on shared variance among the symptoms. Cronbach’s alpha level was also examined across three age categories (3 to <5 years, 5 to <7 years, and >7 years) and the results indicated no significant difference in internal consistency by age group.

Table 2 Inter-correlations or phi coefficients of the ND-PAE symptoms by cut-off values used on standardized measures
Full size table

Diagnostic Formulations of Symptoms

Table 3 contains correlations of the proposed ND-PAE symptoms with domains of impairment, relative to AF 1 of 4 and AF 2 of 4 criteria, and endorsement of the ND-PAE disorder for both cut-off values. Relationships between symptoms in a specific domain were generally stronger than relationships from another domain for SR and AF symptoms, regardless of criteria used for categorization. NI symptoms had more variability in correlations with other NI symptoms and other domain symptoms across the two cut-off values evaluated. Table 4 displays the inter-correlations using phi coefficients of domains of impairment and endorsement of the ND-PAE disorder by the four methods of classification. Positive endorsement of adaptive functioning deficits was often highly confounded with meeting criteria for the disorder (AF 1 of 4, 1.5 SD Cut-off: r = 0.94; AF 2 of 4, 1.5 SD Cut-off: r = 1.0 and AF 2 of 4, 1.0 SD Cut-off: r = 0.93).

Table 3 Phi coefficients of specific ND-PAE symptoms with domains of impairment and ND-PAE status by cut-off values used on standardized measures
Full size table
Table 4 Inter-correlations or Phi coefficients of domains of impairment and ND-PAE status by cut-off values used on standardized measures
Full size table

Symptom Discriminative Power by Diagnostic Formulation Classification Method

The discriminative power of each symptom was evaluated by assessing to what extent each symptom contributed to making the overall diagnosis. If a symptom was not needed or irrelevant to making the diagnosis, then endorsement of the symptom would not be related to whether or not an individual had the diagnosis. Although this type of analysis is often done in evaluating each symptoms capacity to differentiate those with a specific disorder of interest (ND-PAE in our case) from other disorders or typically developing controls, in this study it was done within a pool of children who were all prenatal alcohol-affected of which some met the criteria for the disorder, as defined in this study, or not. Table 5 displays the group means of each of the 12 symptoms from discriminant function analyses by the four methods of classification relative to those who received the endorsement of the disorder or not. For the 1.5 cut-off level using the AF 1 of 4 criteria, group differences were found on all of the neurocognitive measures, one of the self-regulation symptoms (attention), and three of the adaptive functioning symptoms (communication, social, and adaptive living). There was a trend for group differences on adaptive motor functioning. A canonical correlation of 0.72 was found (χ (12) = 35.0, p < .000) among symptoms and 91.3% (n = 42) of those endorsed with the disorder were correctly classified and 90.0% (n = 9) of those without the disorder were correctly classified in a discriminate function analysis combining all of the 12 symptoms.

Table 5 Equality of group means between those endorsed with ND-PAE or not by classification method
Full size table

For the 1.5 cut-off level using the AF 2 of 4 criteria, neurocognitive symptoms differed in all but the learning impairment symptom between those meeting criteria for ND-PAE and those who did not. For the self-regulation symptoms, significant group differences were found in symptoms of mood/behavioral regulation and attentional impairment by diagnostic status. All adaptive functioning symptoms differed by diagnostic status. Discriminate function analysis resulted in a canonical correlation of 0.91 (χ (12) = 86.2, p < .000) among symptoms and correct classification of 97.1% (n = 33) of those with and 100% (n = 22) of those without the disorder.

In the 1.0 cut-off level using the AF 1 of 4 criteria, neurocognitive impairment was less differentiated between those with and without an endorsement for ND-PAE. Group differences were found only on the visual-spatial impairment for the neurocognitive symptoms, two of the self-regulation symptoms (mood/behavioral regulation and attention), and three of the adaptive functioning symptoms (communication, social, and adaptive living). A trend for a group difference was also found on adaptive motor functioning. A canonical correlation of 0.79 was found (χ (12) = 46.4, p < .000) for the symptoms. Correct classification from the discriminant function analysis combining all 12 symptoms was obtained on 94% (n = 47) of those with and 100% (n = 6) of those without the disorder.

In the 1.0 cut-off level using the AF 2 OF 4 criteria, comparison of those who received endorsement of the ND-PAE diagnosis to those who did not resulted in group differences on all of the neurocognitive measures, one of the self-regulation symptoms (attention), and all of the adaptive functioning symptoms (communication, social, adaptive living, and motor). A canonical correlation of 0.87 was found (χ (12) = 68.6, p < .000) for the symptoms and correct classification was obtained for 97.9% (n = 46) of those with and 100% (n = 9) of those without the disorder.

Receiver Operating Characteristic Curve Analysis

ROC analysis was also used to aid in analysis of the symptoms by plotting the true positive rate (sensitivity) against the false positive rate (false alarm rate or 1-specificity), which generates a visual representation of the ability of a binary measure to differentiate groups. By determining the area under the curve (AUC) associated with each of the 12 symptom curves, the accuracy in distinguishing between two diagnostic groups of each symptom can be assessed by generating a numerical value representing the discriminatory power with values ranging between 0 and 1 with chance level of prediction associated with an AUC value of 0.5. Table 6 displays the AUC values for each symptom by method of classification. The NI symptoms of impairment in executive functioning, memory, and visual-spatial impairment contributed more than chance levels for both of the 1.5 cut-off value methods of classification but global impairment was only significant for the AF 2 of 4 criteria. For the 1.0 cut-off value, NI symptoms were non-significant with the exception of executive functioning impairment in the AF 2 of 4 method of classification. SR symptoms were all non-significant with the exception of attention impairment in the 1.5 cut-off value requiring only one AF symptom. All AF symptoms were significant in the 1.0 cut-off value method of classification but in the 1.5 method of classification, AF symptoms were significant only for social, communication, and independent living skills impairment.

Table 6 Receiver operating characteristic curve analysis by cut-off values used on standardized measures
Full size table

Relationship of Sum of Symptoms to Participant Characteristics

The sum of symptoms was not significantly different between males (1.5 SD: 6.6 (2.8); 1.0 SD: 8.3 (2.3)) and females (1.5 SD: 7.4 (3.1); 1.0 SD: 8.8 (2.8)) using both cut-off values but did differ by race (Caucasians 1.5 SD: 6.3 (3.5); 1.0 SD: 7.5 (3.1)) and African Americans (1.5 SD: 7.5 (2.5); 1.0 SD: 9.1 (2.0)) with more symptoms reported for African American children when using the 1.0 SD cut-off value (F (1,53) = 5.2, p < .026). The number of symptoms was not related to the child’s level of dysmorphia, number of custody placements, child protective services involvement, years of education, and household income. The number of symptoms was positively related to the child’s age for both cut-off values (1.5 SD: r = .37, p < .006; 1.0 SD: r = .42, p < .001).

Discussion

The original formulation of the symptoms of ND-PAE was done by theoretical consensus among a group of FASD professionals with clinical experience but psychometric data is still needed to determine if the proposed symptoms and diagnostic formulation result in a valid psychiatric diagnosis. To validate the ND-PAE diagnosis, assessment of the internal validity of the disorder and consistency of its symptoms is required. This study used alcohol-affected children between the ages of 3 and 10 years of age who were enrolled in a randomized clinical trial of a math intervention and underwent comprehensive neurobehavioral assessments as part of the study protocol [8, 9, 12]. Two criterion levels for symptoms, 1.5 and 1.0 SD, were evaluated and the impact of using one symptom from the adaptive functioning domain or the recommended AF 2 of 4 criteria was also evaluated.

Symptom endorsement among the sample indicated that there was appropriate dispersion among the symptoms with none of the measures or symptoms being endorsed by everyone or no one. In addition, correlations among symptoms suggested that there were no completely overlapping or confounding symptoms. The internal consistency of the symptoms was high and did not significantly differ by cut-off value used. Furthermore, the internal consistency did not significantly differ by age level of the child.

Although the impact of environmental stressors, including poverty, exposure to trauma, and disruptions in placements, often found in children with FASD is frequently expressed as a potential confounder in understanding the impact of PAE on the child’s functioning level [1] and has been associated with increasing the severity of neurobehavioral symptoms associated with PAE [29], the endorsement of ND-PAE symptoms did not differ by gender, history of displacements or child protective service involvement, or family income or educational status. The number of symptoms did differ by race for the 1.0 cut-off value but was not significant at the 1.5 cut-off value, suggesting that increasing the threshold for symptom detection may reduce racial differences in symptom endorsement. The number of symptoms did differ by age level with more symptoms reported in older children using both cut-off values. The latter finding is, in part, an artifact of the data available for the sample as older children were given measures not appropriate for younger children but also reflects a real-world problem in that many of the symptoms are not adequately assessed in available standardized clinical measures for younger children. Difficulties with making the ND-PAE diagnosis in infants and young children is a problem that was acknowledged in the discussion of the disorder as proposed in the DSM-5 [1, 30].

The sample used for this study was small and may have been restricted in the range of symptom severity in that the participants were selected based on initial assessments of their growth, facial dysmorphia, and neurodevelopmental functioning. Those who were less affected were not included in this analysis as all participants were required to have the presence of alcohol-related dysmorphia. On the opposite end of prenatal alcohol-related impairment, those with extreme levels of intellectual impairment were excluded as there was concern that such individuals may not benefit from the math intervention conducted in the original study. The restriction in range [31] in symptoms, resulting from truncating both tails of the continuum of prenatal alcohol-related effects, may result in under-estimates of the true relationships, or internal consistency, of the symptoms. The classification data are also limited by the high base rate of the ND-PAE symptoms in the sample, which often results in over-estimates of the true diagnostic accuracy of the symptoms [32]. An analysis of a larger cohort with a broader range of symptoms is needed to address the limitations of this study. A sample of children with known exposure to prenatal alcohol who are recruited outside of a clinical context may be helpful with this in that the range of ND-PAE severity should be the greatest as one would anticipate some children having no effects at all and others who may be severely affected.

Symptoms that were significant for discriminating ND-PAE for all of the methods of classification evaluated included impairments in visual-spatial, adaptive communication, adaptive social skills, and adaptive independent living skills, suggesting that these symptoms were important in characterizing the latent trait of ND-PAE severity in this analysis. In contrast, the impairment in impulse control (SR-3) was not related to endorsement of the disorder within this sample. Endorsement of this domain occurred in 69.6% of the participants using the 1.5 cut-off value and 76.1% using the 1.0 cut-off value, suggesting that there was an appropriate level of dispersion but that the variability was not related to the latent trait of ND-PAE severity. It is possible that high preponderance of endorsement of the attentional impairments may have overshadowed the need for impairments in the impulse control given only one endorsement within the behavioral domain is needed for the diagnosis. Additional research is needed with a broader continuum of ND-PAE symptomatology and with other clinical groups to determine if the symptom is contributory towards characterizing ND-PAE disease severity in lesser affected individuals.

The learning impairment symptom was also problematic in that it was only significant for the 1.5 cut-off value in the AF 1 of 4 condition. In addition to the considerations mentioned for the impairment in impulsivity symptom, it is important to remember that the sample selected were individuals who enrolled in a math intervention study and 80.4% of those in the sample using the 1.5 SD cut-off value met criteria and 87.5% using the 1.0 SD cut-off criteria. Although the participants were not required to have a math disability to enroll in the original study, self-selection biases may have skewed the participants’ characteristics. It is possible that the relationship may be different in a random sample of alcohol-affected children. Finally, the motor impairment symptom was only significant in the AF 2 of 4 methods of classifications and not the AF 1 of 4 methods, where the results were only a trend. This suggests that the relationship between motor impairment and ND-PAE diagnosis was more tenuous within the sample and primarily useful only when two symptoms of AF were needed. Operationalization of the symptoms for learning impairment were quite extensive in the data available for this study but this was less true of the motor impairment and the impairment in impulsivity symptoms, suggesting that the results may be different in future research if more targeted measures were used to assess these symptoms.

This study was obviously limited to the existing data that were available in the original MILE study [9] and as a result, operationalization of some of the symptoms was easier than others. In addition to the limited data available for operationalizing the impulsivity and motor symptoms, the EF symptoms only had one measure, which was parent report, rather than direct assessments of the child’s EF skills using standardized testing. Previous research has indicated that EF measures from parent report and standardized measures often are not assessing the same constructs [33]. The relationships between standardized measures of EF skills and other ND-PAE symptoms is not yet known. There is some conceptual overlap between the assessment of EF and memory skills in this study as often indices of working memory skills are identified as being one component of a complex model of EF skills [2]. The assessment of memory functioning in this study also did not sample long-term or narrative memory skills that may be important in the diagnostic formulation. Although other studies using existing cohorts are underway [34], these studies also have similar limitations resulting from using pre-existing datasets as well. A study that was specifically designed to assess the internal consistency and discriminative validity relative to other clinical populations may be needed to obtain optimal operationalization of ND-PAE symptoms.

The ROC analysis was used to assess the sensitivity of the symptoms used in formulating the ND-PAE diagnosis. Although sensitivity and specificity are often used in the context of predicting two different groups: those with different disorders or one group with a disorder and one group without a disorder. In this case, all of the children were impacted by prenatal alcohol exposure and differentiation is done for each symptom relative to whether or not the alcohol-affected child met criteria for the diagnosis of ND-PAE. This analysis serves as an evaluation of each symptom’s consistency with the collection of other symptoms and can be used to determine which symptoms are not important in making the diagnosis. ROC provides a cost/benefit analysis of symptoms and is simply a method of graphically displaying the discrimination threshold of the symptoms of ND-PAE relative to a binary classification (yes or no) of the ND-PAE disorder. The results of the ROC analysis indicated that NI symptoms of executive functioning, memory and visual-spatial impairment contributed more than chance to differentiating ND-PAE disease severity regardless of the AF method of classification criteria in the 1.5 SD symptom cut-off value. Global intellectual impairment was also important but only in the method of classification using AF 2 of 4 criteria at the 1.5 SD symptom cut-off value. Using the 1.0 SD cut-off value, neurocognitive impairment symptoms were less effective with only visual-spatial impairment being significant in both AF methods of classification and executive functioning impairment being significant in only the AF 2 of 4 condition. For the self-regulation symptoms, only the attentional impairment in the 1.5 SD cut-off value using the AF 1 of 4 method of classification was significant. In contrast, the adaptive functioning impairments in social, communication, and independent living skills were significant for all of the methods of classification. Adaptive motor impairment was also significant for both AF methods of classification using the 1.0 SD threshold but not the 1.5 SD threshold.

The current study is not a true clinical study of the endorsement of the symptoms needed to formulate a diagnosis of ND-PAE and only serves to approximate the internal consistency and validity of the disorder in a clinical context. Making the clinical diagnosis of ND-PAE differs from what was done in this study in that the endorsement of symptoms in a clinical context does not require a given threshold (1.0 or 1.5) on a standardized test but rather identifiable impairment in “real world” functioning derived from a compilation of record reviews, interviews and standardized measures. It is possible that the “richness” of the clinical context may identify impairments not readily assessed by the instruments used in this study. Future research should evaluate the use of clinical judgement in making decisions regarding symptom endorsement and the relative cohesiveness of symptoms within this context. A previous study established a high inter-rater reliability (98%) in making the diagnosis within a clinical context but the internal consistency of the symptoms of the disorder were not assessed [35]. Our study also did not do differential diagnostic assessment of other mental health conditions in the sample and the relationships between endorsement of the ND-PAE disorder and other mental health conditions is not yet understood.

Since all of the children were diagnosed with FAS and pFAS, a diagnostic formulation that encompasses a greater percentage of this sample would be preferred. In general, those who did not meet criteria had impairments in one or two of the super domains but not impairments in all three. Within the models sampled, there was considerable variability with a range of 60.7–89.3. Although recommending the AF 1 of 4 model at the 1.0 cut-off value would encompass most of the alcohol-affected children in this sample, additional research is needed to evaluate the predictive validity of the symptoms by evaluating each symptom’s capacity to differentiate those affected by PAE from typically developing children and individuals with other disorders. Although the neurobehavioral measures used in this study to evaluate the symptoms of ND-PAE may be adequate for assessing the internal validity of the disorder, they may not serve as well in efforts to differentiate children with ND-PAE from children with other neurobehavioral disorders with significant overlapping features. Further, recommendations for the threshold (1.0 or 1.5) needed for symptom endorsement should be evaluated within the context of surveying multiple studies of the internal consistency of the symptoms of the disorder to reduce measurement error associated with any one study. Given that the criteria for the proposed disorder may be used by various mental health professionals, threshold levels established from such studies will also probably serve as only guidelines rather than rigid criteria as many mental health professionals may not have access to psychometric data to use in formulating a ND-PAE diagnosis.

Summary

Validation of the ND-PAE diagnosis can have a vital impact in improving access to mental health care and services and the future development of specific treatment strategies to target the needs of alcohol-affected individuals. The lack of a specific psychiatric disorder characterizing the neurobehavioral impairments associated with PAE has contributed to the difficulties many of these individuals and their families experience in accessing mental health services. Previous studies have reported that alcohol exposed individuals who did not meet full criteria for FAS, are at higher risk than those who are diagnosed for a number of adverse life outcomes, including delinquency, school failure, and substance abuse problems [36]. Although ND-PAE was included in the DSM-5 in the section for disorders that need further investigation, it is not yet recognized as its own unique neurobehavioral disorder. The potential inclusion of a diagnosis of ND-PAE in the DSM-5 as its own unique disorder may facilitate recognition of the treatment needs of individuals negatively impacted by PAE, irrespective of whether or not they have the associated dysmorphic facial features and growth impairment. Establishing the homogeneity of the symptoms of a disorder is a prerequisite step to exploring the disorder’s capacity to differentiate affected individuals from other populations. Although additional studies are needed with other populations and across the lifespan, this study is an initial step in a process needed to validate ND-PAE as a psychiatric disorder.