Emotional and behavioral symptoms in adolescence are clearly linked to poorer adaptive functioning across multiple domains such as academic and vocational settings, legal involvement, and interpersonal relationships (Capaldi and Stoolmiller 1999; Reinke et al. 2012). Clinical assessment and treatment research examining adaptive functioning has traditionally been limited to work with individuals with developmental or intellectual disabilities (Goldstein and Naglieri 2016). However, impairment in adaptive functioning is connected to a broad range of mental health problems and is a requirement of most diagnoses using the current diagnostic system (American Psychiatric Association 2013). Thus, it is crucial to extend research and measurement of adaptive functioning to all mental health problems using broad clinical samples. Such research will aid in screening for risk of psychopathology and evaluating treatment outcomes in mental health settings.

The current diagnostic system, the Diagnostic and Statistical Manual of Mental Disorders, Fifth Edition (DSM-5; American Psychiatric Association 2013), defines mental disorder as “a syndrome characterized by clinically significant disturbance in an individual’s cognition, emotion regulation, or behavior that reflects a dysfunction in the psychological, biological, or developmental processes underlying mental functioning” (American Psychiatric Association 2013, p. 20). The DSM-5 also indicates that mental disorders are associated with “distress or disability in social, occupational, or other important activities” (American Psychiatric Association 2013, p. 20), and throughout the manual, the diagnostic criteria for many of the disorders include either distress or significant impairment of the individual’s adaptive functioning. Given that youth often do not access services until their parents or other adults identify a need, impairment of adaptive functioning is especially relevant for youth, as others may more readily identify their problems in functioning than their internal distress. Thus, youth psychopathology is often determined by two critical components: the presence of mental health symptoms and the impairment of adaptive functioning.

Although youth psychopathology is determined by both symptoms and impairment to adaptive functioning, the majority of the literature on diagnostic assessment and treatment outcomes has focused only on symptoms (Goldstein and Naglieri 2016). Symptoms constitute the cognitive, emotional, or behavioral problems that distinguish mental disorders. According to the DSM-5, they represent a “dysfunction in the individual” (American Psychiatric Association 2013, p. 20). Therefore, they are expected to occur across multiple contexts, as they should be more readily explained by individual rather than environmental influences. The second component of youth psychopathology is impairment of adaptive functioning. Adaptive functioning is the ability to fulfill expected roles and responsibilities or adapt successfully across important domains in one’s life; conversely, functional impairment is the extent to which cognitive, emotional, or behavioral problems diminish one’s potential to fulfill expected roles and responsibilities or adapt in various situations. In the DSM-IV, either distress or functional impairment—particularly in the social and occupational realms—became necessary to deem symptoms as “clinically significant” and warrant a formal diagnosis, with impairment as a relatively newer diagnostic component compared to symptoms (American Psychiatric Association 2000). Consequently, there is no clear consensus within the emerging body of literature on functional impairment with regards to defining or measuring the construct (Goldstein and Naglieri 2016; Rapee et al. 2012). The term “impairment” is often used interchangeably with “disability” or “interference,” and the general definition of functional impairment implies a conceptually causal relationship between mental health symptoms and functional impairment—symptoms impair or interfere with one’s ability to function in different aspects of life (Rapee et al. 2012; Üstun and Kennedy 2009). From this definition, it appears that symptoms and adaptive functioning are intertwined constructs. In fact, several researchers have pointed out that the Global Assessment of Functioning Scale, which was introduced in the DSM-III-R to determine an individual’s level of functioning and later dropped from the DSM-5, confounds adaptive functioning with symptoms and is heavily influenced by diagnoses and symptom severity rather than social and occupational functioning (e.g., Smith et al. 2011; Üstun and Kennedy 2009).

However, true impairment in adaptive functioning is both conceptually and empirically distinct from symptom severity. While both constructs for the most part comprise behaviors, functional impairment differs from symptoms in that it represents a decrease in adaptive or desirable behaviors, whereas symptoms are most often characterized by increased maladaptive or problematic behaviors. For example, increased feelings of hopelessness and increased thoughts about death are categorized as depressive symptoms, whereas decreased social engagement and declining academic performance are examples of common impairments to depressed youths’ functioning. Although this distinction seems to suggest that they are two ends of a spectrum, research has demonstrated that symptoms and functioning are not perfectly correlated, and the presence of mental health symptoms does not always lead to impairment in adaptive functioning as would be expected if they represented ends of a single continuum (Bird et al. 2000; Pickles 2001; see Rapee et al. 2012 for a review). Along the same lines, if symptoms and functioning were different poles of the same construct, then including an assessment of adaptive functioning to the diagnostic process should not add any value or predictive power over and above an assessment of symptom severity and therefore would not affect diagnostic decisions. However, a recent meta-analysis demonstrated that when functional impairment is included as a diagnostic criterion, prevalence rates of mental disorders in youth are significantly lower compared to prevalence rates of diagnoses derived from symptoms alone (Polanczyk et al. 2015). This supports the argument that the presence of mental health symptoms does not guarantee functional impairment, and both are separate and valuable indicators of psychopathology. Further, individuals can exhibit impairment in functioning in the absence of symptoms. Keyes (2005, 2006) found that both symptoms of mental disorders and adaptive functioning are associated with psychological well-being in youth and adults, and individuals with lower psychological well-being experienced functional impairment even in the absence of symptoms of mental illness.

Another important distinction between mental health symptoms and adaptive functioning is the role of context in each construct. Unlike symptoms, which can occur regardless of the context, adaptive functioning is context-bound, and an individual’s level of adaptive functioning is based on the extent to which the individual is able to adapt to an environment and fulfill expected roles in a particular setting. Though in the DSM-5 functional impairment is confined to social or occupational areas of life, youth adaptive functioning is usually assessed in three separate but related domains: school/work, home/family, and social-emotional or interpersonal settings (Ezpeleta et al. 2001; Singer et al. 2011). These three domains encompass the primary contexts for the majority of youths’ daily activities. Consequently, clinicians assessing a youth’s “social and occupational functioning” to make a DSM-based diagnosis need to consider their functioning across family, peer, and school contexts.

Despite the importance of functional impairment in the diagnostic process, researchers and clinicians rarely focus on functioning in treatment outcome research and monitoring client progress; instead, symptom reduction is often the target outcome variable (Becker et al. 2011; Weisz et al. 2006). In fact, a review of 313 randomized control trials (RCT) of treatments for various youth disorders found that empirical support for any evidence-based intervention is based primarily on indicators of symptom reduction, and only about a quarter (26.3%) of the interventions were tested in RCTs that included any indicators of functioning—even though including functioning measures changed the strength of evidence for particular treatments (Becker et al. 2011). However, when surveyed about their preferences for outcome data, clinicians have indicated that they prefer to collect and use information about youths’ functioning—particularly their family functioning—to gauge progress and make treatment decisions (Bickman et al. 2000). Thus, there appears to be a mismatch between researchers’ focus on symptom reduction to develop and evaluate treatments and clinicians’ focus on adaptive functioning to make treatment decisions for their clients. If clinicians are expected to make diagnostic and treatment decisions based on adaptive functioning but use treatments designed for symptom reduction, then there is clearly a need for psychometrically sound measures that are practical for use in clinical settings and assess both symptomatic and adaptive behaviors.

Although the body of literature on assessing functional impairment has been expanding recently, the field of clinical psychology has yet to agree on a “gold standard” tool or protocol to measure functional impairment in clinic-referred youth (Canino 2016). Many traditional assessments of adaptive functioning have focused almost exclusively on the performance of physical or cognitive tasks. Such assessments—for example, the Vineland Adaptive Behavior Scales (Sparrow et al. 2005) or the Adaptive Behavior Assessment System (Oakland and Harrison 2008)—neglect functional impairment that results from more transient psychosocial or environmental problems, as they are designed to tap into extreme or persistent problems in adaptive functioning relative to a developmentally typical population (often resulting from pervasive intellectual or developmental disabilities) rather than changes in functioning that occur in typical individuals in response to stress or mental health problems (Canino 2016; Goldstein and Naglieri 2016). Of those assessments that do capture functional impairment beyond basic adaptive skills or capability, many measures rely heavily on clinical judgment or time-intensive clinical interviews, such as the Children’s Global Assessment Scale (CGAS; Setterberg et al. 1992), the Child and Adolescent Functional Assessment Scale (CAFAS; Hodges et al. 1999), or the Child and Adolescent Psychiatric Assessment (CAPA; Ezpeleta et al. 2001). Other measures focus exclusively on impairment specific to a single disorder or type of symptom rather than general functional impairment (e.g., Child Obsessive-Compulsive Impact Scale; Piacentini et al. 2007). Given that disorder-specific impairment does not always correspond with global measures of functional impairment (Bird et al. 2000), this necessitates using a battery of measures to capture the full extent of functional impairment in youth seeking services in a community mental health setting, where clients tend to show high rates of comorbidity (Copeland et al. 2013; Costello et al. 2014). The field of community mental health is often fast-paced and low on resources to invest in the time and training necessary to implement lengthy or complex assessment batteries, so youth-report and informant-report rating scales that are brief and easy to score are the most practical for use in outpatient clinical settings.

A review of the literature on self- and parent-report measures of functional impairment revealed additional limitations in the clinical utility of currently available measures. For instance, the parent-reported Impairment Rating Scale (IRS; Fabiano et al. 2006) and the youth and parent versions of the Strengths and Difficulties Questionnaire (SDQ; Goodman 1997) assess impairment in multiple domains of functioning using only one item for each domain of functional impairment. This limits a clinician’s ability to identify which specific behaviors or skills in a particular domain are impaired, and consequently, it limits the measures’ utility in formulating appropriate treatment goals. In addition, few impairment measures have both parent and youth versions, and items often differ between parent and youth versions of behavioral scales, which precludes empirical and meaningful comparisons between parent and youth perceptions of functioning (Canino 2016; Carlston and Ogles 2009). Research has shown that clinicians tend to favor information provided by parents over information provided by youth when making judgments about youths’ functioning (De Los Reyes et al. 2011, 2015). Therefore, impairment measures that obtain parent and youth ratings on the same behaviors could help clinicians move beyond a reliance on parents as a single source and inform a more integrative conceptualization of a youth’s functioning.

One assessment system that has shown promise as a practical, efficient, and valid tool suited for use with both youth and parents in outpatient or community mental health settings is the Ohio Youth Problems, Functioning, and Satisfaction Scales (Ohio Scales). The Ohio Scales (Ogles et al. 2001) were developed in order to provide clinicians with a briefer (48 items) and more affordable measure that is easy to score and interpret, while maintaining psychometric rigor. The measure includes parent-, youth-, and worker-report versions of four scales—Problem Severity, Functioning, Hopefulness, and Satisfaction (with services)—allowing clinicians to use a comprehensive assessment system that captures information important to community mental health rather than a battery of independently produced measures. The specific behavioral items were rationally derived using DSM criteria for diagnoses, the existing literature on youth treatment outcomes, a survey of professionals affiliated with child mental health services, and established child assessments. A team of child mental health service providers and parents were involved in evaluating and revising items before the scales were finalized (Ogles et al. 2001). Items make up four scales: Problem Severity, Functioning, Satisfaction, and Hopefulness. The initial validation studies found that scores on the Ohio Scales were significantly different between community and clinical samples, supporting that the Ohio Scales can be used to distinguish youth with clinically significant problems from the general population (Ogles et al. 2001). The Ohio Scales have been widely used in community mental health settings across several states, including Ohio, Texas, Hawaii, California, and Oklahoma (California Department of Health Care Services 2013; Hawaii Department of Health 2014; Texas Department of State Health Services 2007; Turchik et al. 2010; University of Oklahoma E-TEAM 2015).

Notably, the Functioning scale of the Ohio Scales is one of the few measures of youths’ functional impairment that has identical items between youth- and parent-report versions which cover a range of adaptive functioning indicators, including interpersonal, emotional, and structured and unstructured activities. In the original psychometric testing of the scale, the researchers conducted a principal components analysis with a varimax rotation to analyze the factor structure of the parent-reported Functioning scale. Their analysis resulted in two factors, one overarching factor for adaptive functioning and one smaller (three-item) factor that encompassed items that are more applicable to youth transitioning into adulthood (i.e., romantic relationships, financial responsibility, and learning job skills). Based on this analysis, they concluded that the Functioning scale is a unitary scale (Ogles et al. 2000). However, the authors did not include a factor analysis of the youth-reported scale, and their analysis sample combined several non-clinical (school-based) and clinical samples. Research conducted with community samples cannot inform the properties of measures intended for clinical use, as measures may demonstrate different properties when used with clinical samples compared to non-clinical samples; for instance, Price et al. (2013) found differences in the factor structure of a measure of youth problems between a clinic-referred sample and community-based sample of youth. Following publication of the Ohio Scales, a state department of mental health conducted a validation of the Ohio Scales that included factor analyses of parent- and youth-reported Functioning scales collected during intake interviews in three separate clinical samples. These researchers found inconsistent patterns of factor loadings across the three samples and as a result concluded that the Functioning scale is best considered a unitary measure of functioning (Texas Department of Mental Health and Mental Retardation 2004). However, no direct comparisons were made between the youth and parent data, so it is unclear whether inconsistencies in the factor structures could be attributed to differences between youth- and parent-report. Further, the use of an orthogonal rotation in both papers may have hindered the factor analyses because this rotation does not allow for correlation between factors; yet, research indicates that different areas of youths’ functioning are highly related (Chen et al. 1997; Ezpeleta et al. 2001).

In the current study, we address the methodological limitations in previous examinations of the Functioning scale by conducting a rigorous series of exploratory then confirmatory factor analyses of the scale using both youth- and parent-reported data collected during routine clinical intakes. By focusing on the Functioning scale within an exclusively clinical sample, this study strengthens the literature base for the Ohio Scales to inform the growing number of clinicians and mental health agencies that are employing the Ohio Scales in their routine clinical activities. Further, this study includes separate analyses of youth- and parent-reported data, allowing for comparisons between youth and parent conceptualizations of adaptive functioning.

Method

Participants

This study used an archival database of 1260 adolescents aged 11–19 years receiving services in a community mental health clinic in Northwest Ohio, and for whom youth- and parent-report measures were completed. The sample was predominantly Caucasian (87%) and half female (50%). In addition, it is estimated that the majority of adolescents were from low-income families, as approximately 60% of youth served at the clinic receive Medicaid. For the majority of the sample (79%), parent-report measures were completed by mothers, but for roughly a fifth (21%) of the sample for whom mother-report measures were unavailable, parent-report measures were completed by fathers (13%), grandparents (<1%), or other guardians (8%). For clarity, all guardians who completed the parent-report measures are referred to as parents.

Procedures

The current study was conducted within the context of a larger study involving secondary data analysis to explore treatment outcomes of youth in community mental health, which was approved by the Institutional Review Board at the University with which the first author is affiliated. The complete database contained self-report and parent-report measures that clinicians and mental health workers collected from clients and parents at intake, 3-month, 6-month, and annual visits as part of their routine practices at the clinic. This study limited analyses to the data collected from youth and parents at intake to minimize the influence of service involvement.

Measures

Impairment of adaptive functioning was measured using youth- and parent-report versions of the Functioning scale from the Ohio Scales (Ogles et al. 2001). The Functioning scale comprises 20 items assessing the degree to which youth experience trouble with everyday activities on a 5-point scale ranging from 0 (“Extreme Troubles”) to 4 (“Doing Very Well”), and the items include several indicators of positive functioning (e.g., interpersonal relationships, recreation, self-direction, and motivation) across different domains (e.g., academic, cognitive, and social-emotional; Ogles et al. 2001). The scale is scored by summing all raw items, with lower scores indicating more impairment. Raw scores are used for interpretation, although the Ohio Scales user’s manual also provides guidelines for using clinical cut-offs that were established during scale development (Ogles et al. 1999). The scale has been shown to have good internal consistency (Cronbach’s alpha coefficients above .90 for parent and youth reports) across both clinical and community samples, as well as adequate test-retest reliability in clinical samples (Ogles et al. 2001). The parent and youth ratings were found to have convergent validity with the Child Behavior Checklist Total Problems score (r = .77) and Youth Self Report Total Problems score (r = .46), respectively. The parent ratings on the Functioning scale have also demonstrated convergent validity with the Vanderbilt Functioning Index (r = .54; Ogles et al. 2000).

Data Analyses

First, siblings were identified in the dataset, and one youth was randomly selected from each family to be included in the final dataset such that there was an approximately equal distribution of oldest, middle, and youngest siblings in the dataset. Youth were also excluded from analyses if either the self-report or parent-report forms were missing more than 20% (i.e., 4 items) of the Problem Severity or Functioning scales. For the remaining 1080 cases, missing values were imputed using a multiple imputation model. Little’s test for Missing Completely at Random (MCAR; Little 1988) was performed to determine if there were patterns to the missing data or if the data could be considered MCAR. Results of Little’s MCAR test (χ2 = 223.54, df = 230, p = .608) indicated that the data were MCAR, and thus missing values were imputed using expectation maximization. Next, the total sample of youth was randomly split into two halves, and exploratory factor analyses (EFA) were conducted on half of the sample to identify a preliminary factor structure for the youth-reported and parent-reported Functioning scales separately. Principal axis factoring was performed, as this extraction method accounts for item-specific variance and measurement error. Factors with Eigenvalues above 1.0 were retained in the model. Additionally, an oblique rotation (promax) was used, which allows factors to be correlated. If separate domains of functioning emerged, it was expected that the domains would be correlated with one another, so this rotation method was preferable to an orthogonal rotation, which assumes factors are uncorrelated. Since an oblique rotation was used, both the pattern and structure matrices were examined to interpret the factors. The interpretation of the factors was consistent across both matrices, and for the sake of parsimony only the factor loadings from the rotated pattern matrices are reported in this paper. Factors were interpreted with items that had loadings of at least .30 (Floyd and Widaman 1995), and items that cross-loaded onto multiple factors were initially assigned to the factor for which the item loading was highest.

Confirmatory factor analyses (CFA) were performed with the other half of the sample to validate the findings from the EFAs. Adjustments were made to the model if modification indices signaled that adjustments might improve the fit, and if moving an item to a different factor improved the interpretability of the factors. Fit indices from adjusted models were compared to other factor solutions to determine the best-fitting model for the Functioning scale. Acceptable fit was determined by examining multiple fit indices, looking for a non-significant Chi-Square, a Root Mean Square Error of Approximation (RMSEA) below .08, Comparative Fit Index (CFI) above .90, and Standardized Root Mean Square Residual (SRMR) below .05 (Hu and Bentler 1999). Model comparisons were conducted using Chi-square difference tests. All factor analyses were run separately for youth and parent reports. The EFA were performed in SPSS Version 21 and CFA were performed in MPlus Version 7 (Muthén and Muthén 2012).

Results

Prior to running any analyses, the sample was randomly split into two subsamples, an EFA subsample and CFA subsample. Chi-square tests revealed that the two subsamples had similar distributions of age, gender, racial/ethnic diversity, and total functioning scores (Table 1). Independent samples t-tests indicated that the two subsamples were not significantly different on parent- or youth-reported scores on the total Functioning scale.

Table 1 Demographics and mean functioning scores
Full size table

Youth-Report Functioning Subscales

In the initial EFA, five factors were extracted with initial eigenvalues above 1.0, accounting for a total of 46.22% of the variance. All items loaded onto at least one factor above .30, and items were grouped into the factors onto which their loadings were highest. Factor loadings are reported in Table 2. Three items had cross-loadings above .30 on two factors (“Thinking clearly and making good decisions”; “Controlling emotions and staying out of trouble”; and “Caring for health needs and keeping good health habits”). Upon examining the content of the factors, it was determined that the factors represented the following domains of functioning: Responsibility (Factor 1; 33.21% variance explained), Social-Emotional (2; 3.96% variance explained), School (3; 3.61% variance explained), Recreational (4; 3.35% variance explained), and Self-Care (5; 2.09% variance explained). However, the Responsibility factor (e.g., managing emotions, using money wisely, completing chores) was strongly correlated with the Social-Emotional and School factors, with coefficients of .75 and .70 respectively (Table 2). Given the strong correlation between the Responsibility and Social-Emotional factors in conjunction with the multiple cross-loaded items, a second EFA was run in which the factor extraction was limited to four factors.

Table 2 Rotated pattern matrices and inter-factor correlations from youth functioning scale EFA
Full size table

When the number of factors was constrained to four, the model accounted for 43.59% of the variance, less than the five-factor model (factor loadings reported in Table 2). The four factors generally captured Social-Emotional (Factor 1; 33.10% variance explained), School/Work (2; 3.79% variance explained), Recreational (3; 3.42% variance explained), and Self-Care (4; 3.28% variance explained) domains of youth functioning, and all between-factor correlations were below .68. Three items had cross-loadings above .30 on two factors (“Accepting responsibility for actions”; “Concentrating, paying attention, and completing tasks”; and “Dating or developing relationships with boyfriends or girlfriends”). All cross-loaded items were included with their primary factor initially to test the model fit using CFA. Correlations between factors are reported in Table 2.

Confirmatory factor analyses were performed on the second half of the sample to replicate the five-factor and four-factor models and to determine the best-fitting model for the youth-report data. Results indicated that the EFA five-factor model provided a good fit to the data: χ2 (160) = 461.25, p < .05, RMSEA = .059 (90% CI = 0.053–0.065), CFI = .908, SRMR = .048. After running the initial model, modification indices were examined to determine which items had cross-loadings that were most likely to impact the fit statistics. Examining the modification indices, one item (“Dating or developing relationships with boyfriends or girlfriends”) had a modification index of 33.66 that suggested a cross-loading on the Social-Emotional subscale, whereas no other items had modification indices above 25 on any other subscale. The item was moved from the Self-Care subscale to the Social-Emotional subscale given that this change was theoretically supported. This revised five-factor model indicated an improved fit to the data: χ2 (160) = 433.61, RMSEA = .056 (90% CI = 0.050–0.063), CFI = .917, SRMR = .044. A CFA was then performed based on the model identified in the EFA that was constrained to four factors. Fit indices indicated an adequate fit to the data: χ2(164) = 547.92, RMSEA = .066 (90% CI = 0.060–0.072), CFI = .883, SRMR = .052. A Chi-square difference test (χ2 (4) = 114.314, p < .001) revealed that the reduction in the Chi-square coefficient for the five-factor model is statistically significant, indicating that the five-factor model is a better fit for the data. Thus, the final model for the youth-reported Functioning scale encompasses the following domains: Responsibility, Social-Emotional, School, Recreation, and Self-Care. Figure 1 illustrates the model.

Fig. 1
figure 1

Factor structure and factor loadings for youth-reported (left) and parent-reported (right) five-factor models

Full size image

Parent-Report Functioning Subscales

In the initial unconstrained EFA using parent-reports, four factors emerged with initial eigenvalues above 1.0, accounting for a total of 55.00% of the variance. Factor loadings are reported in Table 3. By examining the content of the items, it was determined that the factors represented the following domains of functioning: Social-Emotional (Factor 1; 42.32% variance explained), School and Work (2; 4.93% variance explained), Recreational (3; 3.99% variance explained), and Self-Care (4; 3.76% variance explained). Although the four factors were similar in overall content to the youth-report four-factor model, several individual items loaded on to different factors in the youth-report scale compared to the EFA of the parent-report scale. All inter-factor correlations were below .70 with the exception of the Social-Emotional and School/Work factors, which were correlated at .70 (Table 3). Similar to the youth results, three items had cross-loadings above .30. Items were included with the factor onto which the loading was highest for the initial model in the CFA.

Table 3 Rotated pattern matrix and inter-factor correlations from parent functioning scale EFA
Full size table

A CFA was performed on the second half of the sample using the four factors identified in the EFA. Fit indices indicated that the four-factor model identified in the EFA was a poor fit to the data: χ2 (164) = 874.33, RMSEA = .089 (90% CI = 0.084–0.095), CFI = .868, SRMR = .069. Examining the modification indices, it appeared that moving one item (“Dating or developing relationships with boyfriends or girlfriends”) from the Self-Care subscale to the Social-Emotional subscale would improve model fit and would be theoretically supported. Although this revised four-factor model showed a moderate improvement in model, the fit indices still indicated a poor fit to the data: χ2 (164) = 833.51, RMSEA = .087 (90% CI = 0.081–0.093), CFI = .876, SRMR = .054. Given that youth-report Functioning scale supported a five-factor model, a five-factor model replicating the youth-report model was also tested using the parent-report data. This model produced the following fit indices: χ2 (160) = 765.57, RMSEA = .084 (90% CI = 0.078–0.090), CFI = .887, SRMR = .051. Due to several elevated modification indices, two sequential modifications were made that improved model fit and were theoretically supported. First, the highest modification index (34.43) indicated that one item (“Getting along with family”) cross-loaded on the Responsibility subscale, and moving this item from the Social-Emotional subscale to the Responsibility subscale resulted in slightly improved fit indices: χ2 (160) = 740.42, RMSEA = .082 (90% CI = 0.076–0.088), CFI = .892, SRMR = .050. Upon re-reviewing the modification indices for this revised model, another item (“Earning money and using money wisely”) had a modification index of 26.16 that indicated a cross-loading on the School subscale, and thus was moved from the Responsibility subscale to the School subscale, resulting in the optimally fitting five-factor model for the parent-report data: χ2 (160) = 717.36, RMSEA = .080 (90% CI = 0.074–0.086), CFI = .896, SRMR = .048. Although this model also resulted in high modification indices (i.e., above 25) and only marginal fit statistics, no further theoretically-supported modifications improved the model fit. A Chi-square difference test (χ2 (4) = 116.159, p < .001) revealed that this model fit the data significantly better than the four-factor model. Given the deviations from the youth-reported model, it is important to note that the resulting Responsibility factor for parents primarily represents youths’ responsible behavior at home, including youths’ behavior towards the family, whereas the Responsibility factor for youth represents responsible behavior across contexts (i.e., home and work). Thus, the final model for the parent-reported items supports that the Functioning scale encompasses the following five domains: Responsibility at Home, Social-Emotional Functioning, School/Work, Recreation, and Self-Care (see Fig. 1). The differences in the factor structures between parent and youth models prohibited further statistical comparisons between the models.

Discussion

Although adaptive functioning is a critical factor in clinical decision-making, there is a dearth of research on practical, multi-informant measures of functional impairment in clinic-referred youth. The current study expanded this area of research by investigating the factor structure of a brief, economical, and practical tool that measures functional impairment and includes identical parent and youth versions. Furthermore, the data were collected as part of routine intake procedures at a community mental health clinic, which maximizes the generalizability of findings to usual care in outpatient mental health settings. Notably, by incorporating both parent and youth versions of the scale, this study was able to compare youth and parent perceptions of adaptive functioning. Such comparisons have not been possible in previous studies due to limitations in the measures chosen (e.g., lack of identical items across different informant versions) or data collection methods (e.g., only collecting responses from either parents or youth, but not both).

Results of the factor analyses suggested that the youth- and parent-reported Functioning scales comprised five separate domains: Responsibility, Social-Emotional, School, Recreation, and Self-Care. These results could have direct implications for clinical practice and diagnostic assessment of youth. First, the findings suggest that a global measure of functioning may not sufficiently capture the full extent of a youth’s functional impairment. Youth could be significantly impaired in any single domain of adaptive functioning, and global functioning scores may not reflect their impairment if they are adapting or thriving in other domains. This is consistent with previous studies that demonstrated impairment specific to one diagnosis is not related to global impairment in functioning (Bird et al. 2000). As a result, clinicians who rely on global measures of adaptive functioning may not interpret these youths’ symptoms as reaching the “clinically significant” threshold to warrant the diagnosis of a mental disorder and subsequent treatment.

Although these results are not definitive until replicated using other samples, the findings from these factor analyses suggest that the Functioning scale from the Ohio Scales may have a more nuanced factor structure than other similar measures. Most common youth- and parent-report impairment measures capture only three of the five domains of adaptive functioning identified in this study. For instance, Singer et al. (2011) found that a three-factor solution was the best fit for the Columbia Impairment Scale (CIS; Bird et al. 1993), resulting in three subscales that compare to the Social-Emotional, School, and Responsibility domains found in this study. Similarly, the commonly-used Achenbach (1991) competency scales span three areas of competence: Social, School, and Activities. The Brief Impairment Scale (BIS; Bird et al. 2005) also comprises three subscales but includes a broader range of items compared to other measures; in addition to school/work and interpersonal domains, the BIS includes a “self-fulfillment” subscale, which combines youths’ activities in recreation and self-care. An exception among other common impairment scales is the Child and Adolescent Social and Adaptive Functioning Scale (CASAFS; Price et al. 2002), a youth-report scale which demonstrated a four-factor solution that assesses domains of peer relationships, family relationships, school performance, and home duties/self-care. Another notable assessment tool is the Behavior Assessment System for Children (BASC-3; Reynolds and Kamphaus 2015), which is designed to produce scores on multiple clinical (symptom-related) and adaptive (positive behavior) scales, as well as an overall functional impairment index. The parent-report version of the BASC-3 captures five adaptive scales including activities of daily living, adaptability to the environment, functional communication, leadership, and social skills, whereas the adolescent self-report version captures primarily social-emotional adaptive scales including interpersonal relations, relations with parents, self-esteem, and self-reliance. However, no other measures of youth functioning were found to produce separate self- and parent-report scores in all domains assessed by the Functioning scale of the Ohio Scales. This suggests that the Ohio Scales may tap into aspects of functioning that are overlooked by other assessments, and consequently could provide clinicians with a more complete evaluation of functional impairment in their young clients while minimizing the need for a long assessment battery. By obtaining a more complete picture of youths’ adaptive functioning in an efficient manner, clinicians would be better equipped to identify clients’ individual needs and tailor treatment plans accordingly.

Notably, this study used identical parent- and youth-reported measures to complete a rigorous factor analysis of functional impairment, allowing for comparisons between parent and youth models of adaptive functioning. The results of the study supported that youth and parents view overall adaptive functioning similarly, as both models resulted in a similar five-factor model. However, it appears that youth and parents may conceptualize some domains of adaptive functioning differently; there were differences between the respective best-fitting models in the items included within certain factors. The data suggested that parents tended to distinguish between behaviors that they directly observe at home and behaviors that occur in other contexts. As a result, in the parent model the item pertaining to getting along with family loaded with other items pertaining to responsibility that may be observed at home, such as completing chores, controlling emotions, and making good decisions. For youth, however, the family item loaded onto Social-Emotional functioning, along with items that assess their interpersonal functioning in the peer context. Additionally, parents did not group youth financial responsibility with other indicators of responsibility as youth did; instead, the item pertaining to earning and using money wisely loaded with youth academic achievement and school behavior in the parent model. It may be that these are aspects of their child’s functioning that parents have the least opportunity to observe directly, as they typically occur outside of the home environment.

Overall, these results suggest that youth and parents bring unique perspectives to the diagnostic process, and so these results imply that clinicians should take both perspectives into account. Parents may be well-positioned to report on their child’s functioning in the home environment, and their responses likely reflect their direct observations at home. Youth, on the other hand, likely offer their general perception of their functioning across all contexts, and particularly in the school and peer contexts in which they increasingly spend time as they move through adolescence. Thus, their report of their own sense of responsibility is likely influenced by their experiences across multiple environments outside of the home. Clinicians may need to incorporate both youth and parent perspectives, and other key perspectives such as teachers’, in order to fully understand a youth’s level of adaptive functioning in each domain.

While the confirmatory factor analysis of the youth-reported Functioning scale resulted in acceptable fit by all four fit statistics examined, the final measurement model for the parent-reported Functioning scale only marginally fit the data, with two of the four fit indices just within acceptable ranges. This indicates that the factor structure of the Functioning scale may not be as reliable for parent report. The relatively poor fit of the parent model was largely due to instances of covariance between items that were unexplained by the latent factors. It could be that there are other factors influencing parent responses on this measure. For instance, research has shown that mothers who experience emotional problems are more likely to report that their children also exhibit emotional or behavioral problems (Fergusson et al. 1993). Parental stress or depression has also been linked with greater parent-youth disagreement on measures of emotional and behavioral symptoms (Youngstrom et al. 2000). These parents could experience different thresholds for defining behavior as a problem compared to their children. It could also be that parents interpret the items differently or lack awareness of their child’s behavior, as these are common reasons for parent-youth disagreement on behavioral measures (Kramer et al. 2004).

Limitations and Future Research

It is important to note several methodological limitations that may affect the interpretability and generalizability of results. First, the sample used in the study was not racially or ethnically diverse; the study was conducted with predominantly Caucasian youth from a low- to middle-income background seeking mental health services at a clinic in a semi-rural area. Given that the entire sample was recruited from a single clinic, it is unknown whether these results would generalize to other clinics, particularly those with more racially-diverse or urban clientele. Although the clinic serves primarily low-income families, this study did not control for socio-economic status, as data on family income were unavailable. Therefore, it is unclear whether socio-economic status influenced parents’ or youths’ responses on the Functioning scale. Research has established that socio-economic status is associated with parental stress, family conflict, and negative parenting behaviors, which are negatively related to adaptive functioning in youth (Conger et al. 2002). Future studies should investigate whether socio-economic status or other sample characteristics affect the observed psychometric properties of the Functioning scale.

Additionally, this study relied solely on parent- and self-report measures and did not include other measures to verify the validity of these scales. As previously noted, parent and youth responses on the Functioning scale may be influenced by different factors. It is possible that some items may not be reliable indicators of functioning in certain individual or family contexts. For instance, youth from families with fewer resources may not have opportunities to exhibit financial responsibility or engage in hobbies or recreational activities, and therefore their responses on relevant items may not accurately represent their impairment. Similarly, some items, such as dating or earning money, may depend heavily on the youth’s developmental stage. Additionally, parents may not be aware of their child’s problems in areas where they do not have the opportunity to directly observe their child’s behavior, such as at school or with friends. Parents’ responses may also be influenced by their own mental health problems or the quality of their relationship with their child (Fergusson et al. 1993; Treutler and Epkins 2003; Youngstrom et al. 2000). This may account for the relatively poor fit indices obtained for parent-reported models in the current study. However, it is important to note that both youth and their parents have unique perspectives on their behavior, and future studies should aim to better understand and explain differences in their perspectives to aid clinicians’ interpretation of youth- and parent-report measures.

This study has several implications for clinical research and assessment with youth. First, the findings imply that the Ohio Scales can be used in clinical settings to capture youth reports of their impaired functioning across different domains, although future research with the Ohio Scales should confirm the stability of this factor structure across different samples. The parent version may also be used to assess youth functioning from a different perspective, although the domains may be less reliable and parent responses may be influenced by other factors. If exploring these domains across reporters, researchers should interpret results cautiously given the relatively poor fit of the parent model and the different items contained in parent and youth domains. Interpretations must take into consideration the different conceptualizations of the varying domains for each reporter. Additionally, these differences between youth- and parent-report constructs should be examined in future research prior to using them in clinical settings. Together with previous studies, the results of this study support that youth adaptive functioning is best encapsulated by multiple dimensions of youth daily life rather than a unitary global functioning scale. Current evaluations of adaptive functioning are often limited to the home, school, and peer contexts, which may not encompass all the ways in which youth are affected by their mental health symptoms. Also, treatment research and practices have emphasized symptoms, whereas markers of adaptive functioning are largely ignored or often not evaluated in all domains relevant to youth daily life. For example, Becker et al. (2011) review of RCTs found that few existing youth treatments are evidenced to improve functioning, in large part because so few studies included any measures of functioning. Further, of the few RCTs that included functioning measures, the majority (76.9%) relied on unidimensional measures to assess functioning outcomes. Given that clinicians tend to target improvement in functioning in treatment (Bickman et al. 2000; Love et al. 2014), future studies should investigate the utility of multidimensional functional impairment measures in clinical settings and their sensitivity to treatment effects, and treatments need to be developed and evaluated using youth- and parent-rated measures of functional impairment, such as the Ohio Scales, in conjunction with symptom checklists to help clinicians choose suitable interventions that address the behavioral changes they target in treatment.

Data Availability

This study was conducted using archival data obtained through a collaboration with a community mental health agency. Public release of these data were not specified during the initial memorandum of understanding, and thus they cannot be made publicly available. Other information, such as non-copyrighted materials and syntax, can be obtained by contacting the authors.