The last half century has changed our psychological focus from student behavior in isolation—to a view of students who are considered in tandem within an educational-neurodevelopmental paradigm (D’Amato and Wang 2015; Witsken et al. 2008). A comprehensive understanding of executive brain functioning can benefit the intervention development of all students from various levels and settings. For many years, adolescent psychiatric facilities who serve students with emotional disabilities (EDs) have had difficulty demonstrating their psychological intervention effectiveness. It would seem that executive functioning may be one of the factors that may impede the development of such students.

Adolescent females in residential treatment centers (RTCs) provide unique challenges for those providing psychological services. They display higher degrees of psychopathology (Hussey and Guo 2002) and have more difficulty transitioning to less restrictive settings (Frensch and Cameron 2002) than boys from similar placements. While children and adolescents with severe emotional and behavioral disorders commonly display maladaptive behaviors and affect, many also seem to have neurodevelopmental-related difficulties in processing information, understanding language, and accurately predicting consequences of their behavior (Connor et al. 2004; D’Amato et al. 2005). The relationship between neuropsychological functioning and emotional abilities has long interested researchers and practitioners (D’Amato and Hartlage 2008; Hartlage and D'Amato 2008; Power and D’Amato 2018). To better understand the physical manifestations of this relationship, investigators have investigated the frontal lobe of the brain and the corresponding executive functions as mediators between cognition and emotions (D’Amato and Wang 2015; Semrud-Clikeman et al. 2005).

Executive functioning (EF) represents those cognitive abilities that allow an individual to deliberately and willfully control his or her thoughts and environment (Pennington and Ozonoff 1996). Individuals with compromised EF skills have difficulty evaluating and adjusting their behavior to meet the individual demands of a situation (Damasio and Anderson 2003; Minassian et al. 2003). This inability to adjust thoughts and behaviors relates to a variety of emotional and behavioral problems. Individuals with attention deficit hyperactivity disorder (e.g., Coolidge et al. 2000), conduct disorder (e.g., Giancola et al. 1998), bipolar disorder (Dickstein et al. 2004), schizophrenia (e.g., Schenkel et al. 2005), sleep disorder (Mietchen et al. 2016), and borderline personality disorder (e.g., Coolidge et al. 1999) have all been shown to have corresponding executive deficits. Similarly, individuals with deficits in EF are also likely to have corresponding academic difficulties in verbal and math skills (Condro et al. 1995; D’Amico and Guarnera 2005; Hale et al. 2003; Reiter et al. 2004; Sikora et al. 2002; van der Sluis et al. 2004). As a result, many practicing psychologists seek to understand EF when developing treatment and educational plans for children and adolescents with emotional and behavioral disorders (Walker and D’Amato 2006).

The Behavior Rating Inventory of Executive Function

One measure that is frequently used to assess EF is The Behavior Rating Inventory of Executive Function (BRIEF-PRS; Gioia, Isquith, Guy, and Kenworthy 2000). The self-rating form of the BRIEF has been found to be a useful tool for clinical and educational settings (Walker and D’Amato 2006). While many traditional office-based tests of executive functioning have questionable ecological validity (Trauber and D’Amato 2005), the BRIEF-PRS relies on a caregiver’s observations of a child or adolescent in everyday settings to understand the behavioral consequences of executive deficits. The BRIEF test is purported to have at least two separate factors for a normative population, including (1) a Metacognitive Index and (2) a Behavioral Regulation Index (Gioia et al., 2000). With a clinical population of individuals receiving outpatient psychological and psychiatric services, Gioia et al. (2002b) conducted a confirmatory factor analysis finding that the parent rating scale of the BRIEF (BRIEF-PRS) measures three constructs: (1) a Metacognition Index, (2) a Behavioral Index, and (3) an Emotional Regulation Index. The BRIEF has also been the subject of validity studies for other populations including children and adolescents with attention deficit hyperactivity disorder (Jarrat et al. 2005), traumatic brain injury (Mangeot et al. 2002), and severe epilepsy (Slick et al. 2006). These studies all supported a two-factor structure for the BRIEF-PRS. To date, no study has specifically investigated the validity evidence of the BRIEF-PRS for a population of adolescent females in a RTC.

The present study investigated the BRIEF with adolescent females receiving inpatient psychological services for behavioral and emotional difficulties. Adolescent females in RTC represented a population for whom the BRIEF-PRS has not been studied. Thus, an investigation was conducted to understand the construct of EF for this group as measured by the BRIEF-PRS. Similarly, no previous studies have described the relationship between EF and academic achievement in a population of adolescent females in RTC.

Method

Participants

The Behavioral Rating Inventory of Executive Functioning—Parent Rating Scale (BRIEF-PRS; Gioia et al., 2000) was administered to the case managers of 93 adolescent females in an RTC in the western USA. The average age of the adolescent females was 16.12 with a standard deviation of 1.30 and a range of from 12 years to 19 years. The case managers provided both therapy and coordination of services for the clients and included both bachelors and masters-level individuals. Clients had been placed at the facility from approximately ten states for an average of 13 months. The reasons for placement varied but included individuals who had run away, abused drugs, demonstrated attachment problems, or displayed other dangerous behaviors. All clients were placed by government agencies including county social or human services, probation offices, school districts, and mental health agencies. Only clients who resided at the facility for less than 1 month were excluded from the study. The present sample excluded participants with known histories of autism, traumatic brain injuries, or intellectual disability.

Procedures

The case managers completed the BRIEF-PRS and returned them to be scored by a graduate student who had received training in the administration and scoring of neuropsychological measures. Every 6 months, each client in the facility was administered the Basic Achievement Skills Inventory (BASI) verbal and math test to measure their academic abilities. The most recent BASI verbal and math scores were collected from the client’s files for consideration in the analysis. All procedures performed in studies involving human participants were in accordance with the ethical standards of the institutional review board.

Instrumentation

The Behavior Rating Inventory of Executive Function (BRIEF)

The Parent Form of the Behavioral Rating Inventory of Executive Functioning (BRIEF; Gioia et al., 2000) is an 86-item checklist that purported to cover 8 different aspects of EF. To develop this measure, items were selected from the interviews of a sample of clinical neuropsychologists from around the country. The respondent answered never, sometimes, or often to each of the 86 items. The responses are organized into eight individual subscales (see Table 1 for a list of the subscales) and two validity scales: an inconsistency scale to detect random responding and a negativity scale to identify excessively negative responses. All of the respondent’s validity scales fell within the acceptable range. Data from each of these eight subscales were recorded and reported as T-scores (mean = 50, standard deviation = 10). All of the subscales are included within the Behavioral Regulation Index (BRI) or the Metacognition index (MI), and all eight subscales make up the Global Executive Composite (GEC).

Table 1 Subscales of the BRIEF-PRS as described by the authors (Gioia et al., 2000)
Full size table

Of the scores from the 93 case managers who completed the BRIEF-PRS, 52.7% (49 individuals) earned a T-score that was considered to be clinically elevated, or above a 65, while the other 44 of the sample earned scores that fell below the clinically significant level of 65. For the normative population of adolescent females between the ages of 11 to 13 and 14 to 18, the average global executive composite score (65) and the median score (66) was higher than 90% of the scores for the BRIEF normative sample.

The Basic Achievement Scales Inventory—Screener

The BASI is an instrument that was used to evaluate each adolescent’s verbal and math achievement. There are two forms—an in-depth, diagnostic form and a screener. The screener takes 50 min and includes two sections: a verbal component and a math component. Previously, the BASI screener has shown to correlate strongly with an individually administered measure of academic functioning for adolescent females in RTC (Bardos, 2004) and was given as an academic screener to gain an understanding or a student’s academic skills upon entrance and the development of those skills over the course of a year. The BASI was normed and standardized on a group of 2518 subjects between the ages of 8 and 18 from across the USA (Bardos, 2004).

Results

The correlations between each subscale ranged from a low of 0.46 between the initiate and inhibit subscales to a high of 0.83 between the planning/organization and working memory (see Table 2). This correlation chart served as the basis for the exploratory factor analysis (EFA).

Table 2 Intercorrelation matrix for BRIEF-PRS subscales
Full size table

The EFA was completed to discover the number of underlying factors in the dataset. Three criteria were used to determine the number of factors in the solution. First, factors with eigenvalues greater than one were included as suggested by Kaiser (1970). Only one factor met this criterion. This single factor described 64% of the variance. Next, a simple structure analysis required the researcher to determine if each factor loading was salient or strong enough to load on a particular factor. Any subscale with a loading over 0.71 was considered to be an excellent candidate for inclusion in each factor (Comrey and Lee 1992). A review of Table 3 demonstrates that each component loading was 0.74 or higher for the sample of adolescent females (see Table 3). (3) Finally, a scree-plot was examined (see Fig. 1). The scree-plot showed that an elbow bend occurred on the second factor indicating that only a single factor should be retained. To calculate the broad EF score for a participant, the mean of the subscale T-scores was calculated.

Table 3 Component matrix of BRIEF factor
Full size table
Fig. 1
figure 1

Scree-plot showing that an elbow bend occurred on the second factor indicating that only a single factor should be retained

Full size image

Academic Achievement

To determine the relationship between the BRIEF factor and the BASI, a correlation analysis was performed that compared the performance of the sample on the BRIEF subscales with the composite scores of the verbal and math skills portion of the BASI survey form. A visual inspection of the correlation chart was undertaken using Silverman’s (1993) criteria, and the statistical significance of each correlation was calculated. Correlations were calculated between 75 of the participants’ broad EF scores and their verbal scores from the BASI. A Pearson correlation showed a significant moderately negative correlation (r = − .32) between the BASI verbal score and the BRIEF broad EF score. The mean verbal score on the BASI (mean = 100, standard deviation = 15) was 87.4 with a standard deviation of 20.1. A similar relationship was demonstrated with 73 participants who also demonstrated a significantly negative correlation (r = − .40) between the BRIEF-PRS and the math scores from the BASI. The mean math score on the BASI (mean = 100, standard deviation = 15) was 79.7 with a standard deviation of 14.2.

Discussion

The present study demonstrated two important findings. The BRIEF-PRS, when completed by case managers of adolescent females in RTC, only measures one factor of EF. This finding is in contrast to the two-factor structure presented in the BRIEF-PRS normative manual (Gioia et al., 2000) and a later three-factor structure described from a clinical population (Gioia et al. 2002a, b). So too, adolescent females in an RTC would seem to benefit from services related to frontal lobe dysfunction. The present research was the first to explore the unique population of adolescent females who were receiving residential treatment services. The scale specificity of the BRIEF-PRS failed to adequately differentiate between multiple executive subtypes in this population. This suggested that only one construct is measured by the BRIEF-PRS for adolescent females who have been diagnosed as emotionally disturbed. When performing a clinical evaluation using the BRIEF with an adolescent female in residential treatment, these findings suggest that the general executive composite should be used when interpreting EF.

Additional results demonstrated that as scores increased on the BRIEF-PRS, verbal and math scores decreased. Because previous research has indicated a moderate correlation between executive and academic functioning (Condro et al. 1995; D’Amico and Guarnera 2005; Reiter et al. 2004; Sikora et al. 2002; van der Sluis et al. 2004), this finding provides support for the validity of the BRIEF-PRS as a measure of EF. This relationship may suggest that as interventions are designed to improve EF, they may also improve academic functioning in individuals with behavioral disorders (e.g., Lam et al. 1994).

To make these findings more generalizable, future research should examine the relationship among adolescent females from around the country. A larger sample size may allow an item level analysis to determine how well each item in the BRIEF differentiates between types of EF. To continue to support the evidence on construct validity, the relationship between the BRIEF-PRS and other measures of EF and emotional/behavior functioning should be evaluated. Future research should determine if the BRIEF’s factor structure changes based on nine subscales as suggested by Gioia and Isquith (2002) rather than the eight that are published in the present manual. Because individuals receiving residential care may undergo dramatic behavior improvements, research should be undertaken to understand changes in EF as measured by the BRIEF. These findings may provide support for the BRIEF to be used as a progress monitoring measure to determine a child’s ability to respond to intervention. To improve generalizability, the present study would need to be replicated with adolescent females from treatment facilities around the country to better determine the instrument’s factor structure and its relationship to academic achievement.

In clinical practice, using an instrument that does not recognize individual subscales for a large group makes it difficult to interpret subcomposites and subscales. For adolescent females in a residential treatment, reviewing only the general executive composite should be used, as use of more scales would be over-interpreting the findings. When a single factor is used, the instrument becomes less helpful in tailoring intervention to an individual. However, EF is a critical variable and is part of a neuropsychological profile which must be considered if we are to appropriately program for females with ED who are placed in an RTC (Power and D’Amato 2018).

The BRIEF-PRS provides scores for eight subscales with eight different names, as well as two separate index scores. Indeed, common practice may entice an evaluator to assume that the scores of the subscales represent measurements of unique and valid constructs. This study suggests that results of the BRIEF-PRS should be interpreted conservatively. A neuropsychologist should keep in mind that the BRIEF subscales all relate to one EF construct or idea. Thus, the different names of the subscales do not represent different ideas for the present population. While this analysis does provide evidence to suggest that the BRIEF-PRS measures EF, the instrument does not parsimoniously discriminate between multiple EF constructs.

Recent research which has maintained a focus on EF would seem fitting. Moreover, the advent of numerous manuals and checklists—which help individual students cope with EF difficulties—would appear to help the many impaired students manage this difficult area (e.g., D’Amato and Wang 2015; Witsken et al. 2008). Since the test measured a single factor, it would not seem suitable to use the test to specifically align interventions with scale scores. Instead, good interventions that focus on executive functions should include all of the different subtypes of executive deficit rather than individually tailored interventions that seek to isolate a specific executive deficit.