Introduction

The general acceptance of the importance of families’ concrete resources (e.g., food, shelter, clothing, money) to children’s services research is implied in the ubiquitous inclusion of economic variables in analyses. More often than not, economic factors are included as control variables, presumably because of their association with many child and family outcomes of interest. Research has clearly documented an association between economic disadvantage and child psychopathology (e.g., Costello, Compton, Keeler, & Angold, 2003; Luthar, 1994). In addition, it is generally accepted that inadequate resources can negatively affect parenting (Bronfenbrenner, 1979; Conger et al., 2002; Kotchick & Forehand, 2002; Pinderhughes et al., 2001). Beyond the vague recognition that having more resources is generally better for families and children, however, children’s mental health services research has not fully considered how family resources are likely to influence child service utilization patterns, treatment adherence, and ultimate outcomes.

As applied in children’s mental health services research, concrete resources have been shown to influence service use patterns, but findings have been inconsistent. Concrete resources, operationalized as income, do not appear to be related to the probability of using any mental health services (Brannan & Heflinger, 2005; Wu et al., 1999). Once in treatment, higher household income may increase the probability of children receiving outpatient treatment (Cook et al., 2004), but does not seem related to inpatient hospitalization (Bickman, Foster, & Lambert, 1996; Brannan & Heflinger, in press; Cook et al., 2004). Using other definitions of family resources, children living with economically disadvantaged families appear more likely than their more advantaged counterparts to receive mental health services (Burns et al., 1995). Several studies found no relationship between children’s premature termination from treatment and socioeconomic status (Beer, 1992; Dover, Leahy, & Foreman, 1994; Gilbert, Fine, & Haley, 1994; Gould, Schaffer, & Kaplan, 1985; Kaminer et al., 1992). However, one study found that children from families in the upper socioeconomic groups were more likely to complete outpatient treatment (Armbruster & Fallon, 1994), and another found that adolescents from “non-poor” families remained in services longer than adolescents from poor families (Bui & Takeuchi, 1992).

When operationalized in terms of perceived adequacy of resources, having fewer family resources increased the risk that children would have breaks in care of 30 days or longer in one study (Brannan, Heflinger, & Foster, 2003). As part of an evaluation designed to examine the effectiveness of a continuum of care, some analyses included both perceived adequacy of resources and actual family income. One study found that children of caregivers who perceived family resources to be more adequate were at lower risk of being re-admitted for inpatient psychiatric hospitalization; actual income did not uniquely predict readmission (Foster, 1999). In the second study conducted with the same sample, having a family annual income of less than $20,000 was associated with experiencing less time in treatment at the demonstration (i.e., continuum of care) site; perceived adequacy of resources had no unique impact on time in treatment at either the demonstration or the comparison site (Foster, 1998).

The extant literature does not provide a collective understanding of the role of family concrete resources in shaping parental help-seeking or child mental health service experiences. One reason for discrepant findings may be that concrete resources have been operationalized in a variety of ways including socio-economic status, poverty, household income, and social class. We further posit that the lack of consistent definitions of concrete resources may be due, in great part, to the absence of a cohesive theory on the role of concrete resources in children’s mental health services research.

The family-strength and family-focused perspectives embraced by current service delivery models demand new research approaches (Koroloff & Friesen, 1997). Some have responded to that call by applying family systems and stress and coping theories to children’s mental health services research (e.g., Heflinger, Northrup, Sonnichsen, & Brannan, 1998). Family concrete resources are explicitly included as a component of these family systems and stress and coping theories (e.g., Conger et al., 1992; McCubbin & Patterson, 1983). Simply put, these models depict family and child outcomes as shaped by the interplay between family stressors (e.g., negative life events, child disability), available concrete resources (e.g., income, insurance, adequate housing, economic hardship), family psychosocial resources (e.g., emotional well-being, family functioning, education, parental partner relationships) and other family, cultural and community variables (e.g., health beliefs, attitudes toward professionals, neighborhood characteristics, available services in the community). In addition to affecting personal well being and parenting, lack of family resources has long been theorized to influence how families interact with service systems as they pursue care for their children with special needs (e.g., Dunst, Leet, & Trivette, 1988; McGrew, Gilman, & Johnson, 1992; Modrcin & Robison, 1991). While few would argue that a lack of sufficient resources affects how families engage in help-seeking and treatment decision making, the impact of resource adequacy on these processes among relatively economically secure families remains largely unexplored.

From this perspective, family concrete resources are essential to any consideration of the role of families in children’s mental health services research. To include this component in routine research and practice, the field needs tools to assess concrete resources in a manner relevant to this area. The Family Resource Scale (FRS; Dunst & Leet, 1987) has been identified as appropriate for family assessment from a strengths-based perspective (Early, 2001), and has been used in comprehensive evaluations of children’s mental health services (e.g., Bickman, Foster, & Lambert, 1995; Holden et al., 2003). The purpose of this study is to examine the reliability and validity of the FRS among families caring for children who are receiving mental health services; to date, the FRS has not been examined with this population. In addition, we will compare the measurement quality of the FRS across two samples that differ considerably on economic status to examine whether this one tool is appropriate for use with disadvantaged families, as well as families with more economic resources.

Family Resource Scale

The FRS was developed to assess the adequacy of concrete resources in households with young children. The explicit purpose for developing the FRS was to provide early intervention practitioners with a tool they could use to assess “adequacy of resources and provide a concrete basis for deciding upon intervention targets and appropriate intervention strategies” (Dunst & Leet, 1987, p. 113). The conceptual framework borrows from theories in human ecology, social support, family systems, and help-seeking. The framework asserts that inadequacy of resources necessary to meet individual and family needs will negatively affect the personal well-being of family members and impede parental ability to carry out professionally prescribed regimens (Dunst & Leet, 1987). The scale includes 31 items, loosely ordered from most basic (e.g., food, shelter) to least basic (e.g., money for vacation and entertainment). The respondent rates the adequacy of each item on a 5-point scale ranging from “not at all adequate” (1) to “almost always adequate” (5).

Although developed for use in early intervention clinical practice, we believe that the FRS approach is appealing for use in children’s mental health services research for a variety of reasons. We see caregivers (e.g., biological, adoptive, foster and step-parents, other legal guardians) as the best informants of the concrete needs that are salient to their own families. Families assess their needs in light of the requirements of individual members, broader family demands, and the community context. Hence, they can appraise the relative adequacy of their resources more appropriately than can direct measures of economic status such as household income, employment, and occupation. In our view, caregivers’ perceptions of the adequacy of family resources will impact help-seeking and treatment decisions more directly than so-called “objective” measures, and therefore are more likely to influence child and family service experiences and associated outcomes. Some preliminary support for this position can be found in previous research findings that perceived adequacy of resources predicted hospital readmission when actual income did not (Foster, 1999). Moreover, we suspect that perceived adequacy of resources influences help-seeking and treatment decisions in virtually all families, not only families experiencing economic hardship. This theory is supported in previous research that found that, in a sample of middle class families with little variation in income, perceived adequacy of concrete resources predicted gaps in care, and sequencing of services (after controlling for child clinical status) (Brannan et al., 2003).

In addition, collecting information on family income is fraught with challenges such as poor respondent recall and resistance to disclose. This is especially true for families entering programs with means-based eligibility criteria. Researchers can expect that unofficial “off the books” income (e.g., income not reported to the Internal Revenue Service, proceeds from illegal activities) is seldom reported even to independent data collectors unaffiliated with government programs or service providers. Even if income data could be accurately collected, obtaining other relevant information such as household expenditures is more problematic. In light of these data collection problems and our theoretical position that families are the best informants of what they need, we find the FRS approach (i.e., assessment of perceived adequacy of resources) promising for research purposes.

However, the measurement quality of the FRS has not been fully examined, and efforts to study the FRS’ reliability and validity have yielded mix results. The original exploratory factor analysis (EFA) of the FRS was conducted on a small sample (N=45) raising questions about statistical power and stability of findings (McGrew et al. 1992). Despite considerable cross-loading of items across factors, the authors concluded that the FRS contained seven independent dimensions: food and shelter, financial resources, time for family, extra-family support, child care, specialized child resources, and luxuries (Dunst & Leet, 1987). The authors reported a Cronbach’s alpha coefficient of .92 for the total scale, but did not report the internal consistency of the subscales. Criterion validity of the FRS was reportedly supported by significant correlations between FRS scores and measures of maternal well-being and commitment to the intervention program. However, it is not clear how the authors calculated the subscale scores for the test of criterion validity.

A subsequent effort to assess the measurement quality of the FRS yielded somewhat different findings. Van Horn, Bellis, and Snyder (2001) tested the FRS with two large samples of families whose children were in kindergarten Head Start-like programs at baseline. The following ten items were dropped in sequential EFAs because they were deemed non-salient or overly complex: money to buy necessities, money to pay monthly bills, medical care, public assistance, dependable transportation, babysitting, child care, money for special equipment for children, dental care, and toys for children. While elimination of these items can be defended from a purely empirical standpoint, consideration of these items in light of the guiding theory may lead to a different conclusion. For example, if inadequate child care and transportation impede treatment completion, following through on treatment recommendations, or parent involvement, then dropping these items may compromise the FRS’ usefulness for identifying those barriers and understanding their impact. Van Horn et al. (2001) reportedly confirmed the EFA solution with a separate sample using the smaller pool of items (N=1,883). In their assessment of construct validity of their modified FRS factors, Van Horn and colleagues found that the FRS correlated with percent of poverty but relationships with other salient variables were not explored.

Although these findings support the use of the FRS in some disciplines, the removal of potentially important items compromises its usefulness in services research. In addition, examining the appropriateness of the FRS for use with families of children and adolescents in treatment for emotional and behavioral disorders offers a test of the stability of previously discovered factor structures. In light of evidence that the help-seeking decisions of families from diverse economic strata are influenced by perceived adequacy of resources, it is also important to test whether the FRS operates similarly for families of different economic levels.

The primary goals of this study were to examine the reliability and validity of the FRS when used with families of children with emotional and behavioral disorders, and to compare results across two samples from different economic brackets. The extent to which the findings replicate across samples provides an indication of the stability of the FRS across economically diverse families, especially its utility for assessing resources among families not facing economic hardship.

Method

Sample

Samples from two children’s mental health services evaluation projects were used in this study. The Fort Bragg Evaluation Project (FBEP) was a quasi-experiment designed to examine the effectiveness of a continuum of care demonstration project. The FBEP followed 984 children and adolescents who were receiving behavioral health services paid through CHAMPUS, the insurance program for military dependents. The children lived on or near Army bases in North Carolina, Georgia, and Tennessee (Bickman et al., 1995). Of the 984 families recruited into the FBEP, 964 (98%) had completed the FRS and were included in the study.

The second sample participated in the national evaluation of the Comprehensive Community Mental Health Services for Children and Their Families Program (Holden et al., 2003). Data are included from 20 sites across the United States that received funding in the second phase of the federal program (i.e., communities funded in 1997 and 1998) to develop systems of care for children and adolescents with emotional and behavioral disorders and their families. For simplicity, we refer to this project as the System of Care (SOC) Study. The current research used data from the first 1,026 families recruited into the SOC Study’s longitudinal evaluation component that had completed some portion of the FRS.

The samples differ in several important ways (see Table 1). First, the families in the FBEP sample were less racially and ethnically diverse than the SOC Study sample. The relatively larger proportion of Native American families in the SOC Study is due to the fact that that federal program funded several system-of-care development efforts in Native American communities (Holden et al., 2003). Caregiver respondents in the FBEP sample were more likely to be biological parents (91%) than caregivers in the SOC Study sample (81%). Children in the SOC Study sample were older, on average, than their FBEP counterparts. Incomes tended to be higher in the FBEP sample with the majority of families (69%) earning $20,000 or more compared to the SOC Study sample in which only 36% of families had incomes in that bracket.

Table 1 Comparison of Samples
Full size table

The differences across these samples provide a good opportunity to test the stability of the FRS’ factor structure, reliability and validity across diverse samples. In particular, the considerable disparity in income across samples allows us to test the relative utility of the FRS across families experiencing varying levels of economic hardship.

Data Collection Procedures and Instruments

Recruitment procedures were very similar across the two studies. In the FBEP and the SOC Study, families were recruited into the evaluations at the time of the child’s entry into mental health services. Informed consent procedures were in keeping with established ethical guidelines and approved by the appropriate Institutional Review Boards. Data used in this study were collected from the primary caregiver at baseline (i.e., within 4 weeks of the child’s entry into services). Data for the FBEP were collected from 1991 to 1995. For the SOC Study sample used in this study, data were collected between 1997 and 2000.

The FRS was used to assess perceived adequacy of concrete resources in the previous six months. Additional variables were included to assess criterion-related validity including economic variables, family functioning and caregiver strain.

Family Income

Caregivers reported family annual income by selecting one of several income bracket categories. The annual income brackets were divided similarly in both studies up to $19,999. Differences across the FBEP and SOC studies in the way the income question was bracketed required us to collapse incomes equal to or greater than $20,000 into one category. Five annual household income levels are used in the study: less than $5,000; $5,000–9,999; $10,000–$14,999; $15,000–$19,999; and $20,000 or more (see Table 1). Although this partitioning of income levels is less sensitive than would be ideal, it is sufficient to demonstrate that families in the SOC sample were indeed struggling with greater financial hardship.

Financial Demands

We included household size because it directly relates to the financial resources a family needs. This variable is simply a count of the number of individuals who lived with the child at the time of the interview. FBEP households had more members, on average, than SOC Study families (see Table 1). As an indicator of added drain on families’ financial resources, we also included whether the family paid out of pocket for any portion of the mental health services their child received in the previous six months; not having paid for services was the comparison level. Families in the FBEP were significantly more likely to have paid for a mental health service for their child.

Family Functioning

A family’s ability to work together to meet family needs has been demonstrated to correlate with concrete resources (Brannan & Heflinger, 2001; Heflinger et al., 1998). In this study, family functioning in the previous six months was assessed with the Family Assessment Device (FAD; Epstein, Baldwin, & Bishop, 1983). Construct validity of the FAD has been supported in several studies with FAD subscales correlating in the expected directions with other measures of family functioning, family cohesion, marital satisfaction, and other family variables (Byles, Byrne, Boyle, & Offord,1988; Epstein et al., 1983; Heflinger et al., 1998). FAD items are rated on a 4-point scale ranging from 1 = strongly disagree to 4 = strongly agree. Scores are calculated as the mean of the items in the subscale with higher scores indicating poorer family functioning (positively worded items are reversed). The two FAD subscales with the closest apparent relationship with concrete resources were included to assess criterion-related validity. The role subscale includes 11 items that assess how well family members perform responsibilities such as providing resources, nurturance, and support personal development (e.g., we sometimes run out of things we need, we have trouble meeting our bills, we don’t have reasonable transportation). The problem-solving subscale assesses the family’s ability to resolve problems effectively and includes 6 items (e.g., we resolve most everyday problems around the house, we try to think of different ways to solve problems). These two subscales have demonstrated adequate internal consistency in previous samples (i.e., Cronbach alpha = .72 for the role subscale and .74 for the problem-solving subscale) (Epstein et al., 1983). In this study, we found significant differences on the FAD role subscale across samples with SOC Study families experiencing greater difficulty in that area (see Table 1).

Caregiver strain has also been shown to be associated with concrete family resources (Brannan & Heflinger, 2001; Kang, Brannan, & Heflinger, 2005; Kelley, Whitley, Sipe, & Yorker, 2000). Caregiver strain was assessed with the Caregiver Strain Questionnaire (CGSQ; Brannan, Heflinger, & Bickman, 1997) with higher scores indicating more strain. The CGSQ has 21 items that assess the extent to which caregivers were affected by the special demands associated with caring for a child with emotional and behavioral problems in the past six months. Caregivers rate items on a 5-point scale with response options ranging from 1=not at all a problem to 5=very much a problem. The CGSQ and its subscales demonstrated adequate internal consistency with alpha coefficients ranging from .73 to .91 (Brannan et al., 1997). In multiple samples, the CGSQ was found to correlate with measures of child symptoms, family functioning, and caregiver psychological distress providing evidence of construct validity (Brannan & Heflinger, 2001; Brannan et al., 1997; Kang et al., 2005). The total score was used in these analyses (i.e., calculated as the sum of the three subscale scores) with possible scores ranging from 3 to 15.

Three additional child demographic variables were included as control variables because the samples differed significantly on them. These include age, Hispanic ethnicity, and race.

Analysis

This examination of measurement quality included four sets of analyses. First, we compared mean item responses across samples using t-tests. We then used SAS (SAS, 1999–2000) to perform exploratory factor analysis (EFA) on data from a randomly selected subset of families (i.e., roughly one-third of each sample). The remaining two-thirds of the samples were reserved to confirm EFA findings using CFA. The EFAs were conducted separately for the SOC (N=342) and FBEP (N=321) randomly selected samples. Results from EFAs were compared across samples and subscales were developed based on the resulting factor structures. Because previous research indicated that the FRS factors were correlated, we used principal factor analysis as the factor extraction method and oblique rotation.

Confirmatory factor analysis (CFA) was then performed using Mplus (Muthen & Muthen, 2001) to confirm the existence of the latent factors that emerged in the EFAs and to test the stability of those factors across different samples. The CFAs were conducted on the remaining subset of the samples not included in the EFAs (i.e., 643 families in the FBEP sample, and 684 in the SOC Study sample) to cross-validate the EFA findings. Observed indicators for the CFAs were calculated as means of the items (see Table 3). We constrained the first indicator of each factor to 1 to aid identification. The latent variables were allowed to correlate freely. As the FRS items and the composite indicators were not normally distributed (see Tables 2 and 3), we used maximum likelihood robust estimation (MLR) (Muthen & Muthen, 2001). In order to ascertain the most parsimonious model that fit the data well, we performed nested model comparisons using the mean-adjusted robust chi-square differences test required for MLR (Bollen, 1989; Muthen & Muthen, 2001). Comparative fit index (CFI) and Tucker–Lewis Index (TLI) values greater than .90 are considered a good fit, as are Standardized Root Mean Square Residual statistics (SRMR) less than .04.

Table 2 Comparison of FRS Items Across FBEP and CMHS Samples
Full size table
Table 3 Descriptive Statistics on Indicators Used in Confirmatory Factor Analyses
Full size table

Combining the subsets used in the EFAs and CFAs for each sample, we then tested and compared the reliability of FRS across the two samples. Cronbach’s coefficient alpha was used to assess the internal consistency of the items that comprise the FRS and its new subscales. Feldt’s coefficient (1969) was used to test whether differences in internal consistency across samples were statistically significant.

Criterion-related validity (Ghiselli, Campbell, & Zedeck, 1981) was tested using multiple regression analysis on the combined FBEP and SOC samples. This analysis indicates whether relevant economic and family variables were related to the FRS in expected ways. We conducted separate regression analyses for each of the FRS subscales and the total score. We entered all predictor variables simultaneously to estimate the unique contribution of each to the explanation of FRS scores, given the presence of the other variables in the analysis. As the samples reflect substantially different economic status, a sample dummy variable is included in which being in the SOC Study sample is the comparison condition. A sample by income interaction term was also included to test whether the relationship between income and FRS scores were similar across samples. Income, household size, global caregiver strain, family problem solving, family role performance, and child age were included as continuous variables. Hispanic ethnicity was entered as a dummy variable with not being Hispanic designated as the comparison condition. Similarly, a child race dummy variable was included; minority race was the comparison level. Parenting arrangement was included as a dummy variable that compared caregivers who were biological parents to those who were not.

In general, we expected that perceived adequacy of concrete resources would be positively related to being from the FBEP sample and to family income. We also expected family size, having paid for a service, caregiver strain, and family functioning to be negatively related to perceived adequacy of resources.

Results

Descriptive Analysis

For both samples (see Table 2), item means were higher for needs deemed to be more basic by the original authors (i.e., items toward the top of the list such as two meals a day, shelter, clothes) and lower for luxury items (i.e. those toward the bottom of the list such as money for vacation, to save, and for family entertainment). The FBEP sample consistently had significantly higher scores on FRS items than the SOC Study families after Bonferroni adjustment for multiple tests (p<.0002). The only means that were not significantly different across the samples were item 15 (i.e., time for the family to be together), and item 16 (i.e., time to be with children). The differences tended to be small for more basic needs (e.g., food, clothes, shelter) suggesting a ceiling effect. Differences increased considerably among items that assess “higher-order” needs (e.g., money to save, vacation, entertainment). These findings suggest that the FRS items assess a loose hierarchy of needs (i.e., as the authors intended) with families in both samples experiencing greater difficulty meeting needs that some might consider non-essential. Nonetheless, families in the FBEP sample perceived their concrete resources to be more adequately met than did families in the SOC Study.

Exploratory Factor Analysis

Unlike CFAs that can accommodate missing data, EFA in SAS requires listwise deletion. This eliminated 11 families from the FBEP sample, resulting in 310 families being included in the EFA analysis. Thirteen families were dropped from the SOC Study sample because of missing data, leaving 329 families for the EFA analysis. Using the eigenvalues greater than 1 criterion and examining the scree plots, we determined that 6 factors emerged in each sample.

Comparison of findings across samples indicated that the factor structures were very similar across samples. Twenty-eight of the 30 FRS items (93%) loaded with the same items across both samples. Details of factor loading comparisons are not included because of page limitations but are available from the first author. Based on these findings we tentatively concluded from the exploratory factor analyses that the FRS’s factor structure replicated reasonably well across the two samples. Six-factors emerged that captured family concrete resource needs along the following lines: Basic Needs (i.e., items 1, 3, 4, 7, 26), Housing & Utilities (i.e., items 2, 5, 6, 11, 13, 18), Benefits (i.e., items 8, 9, 10, 22), Social Needs/Self Care (i.e., items 12, 14, 15, 16, 17, 23, 24, 25); Child Care (i.e., items 19, 20), and Extra Resources (i.e., 21, 27, 28, 29, 30).

Confirmatory Factor Analysis

We conducted separate CFAs on the remaining two-thirds of each sample and created composite indicators (i.e., mean of the selected items) for each of the factors (i.e., latent constructs) that emerged from the EFAs. To ensure that the model was conceptually identified under the two-indicator rule (Bollen, 1989), we created at least two indicators for each latent construct. Table 3 shows the items that make up the indicators and provides the descriptive statistics for the composite indicators. As many of the indicators are not normally distributed, we used MLR estimation in the CFA. Mplus handles missing data in the analysis, making the best use of all the available data (Muthen & Muthen, 2001).

We built four models and compared them to find the most parsimonious model that fit the data well. First, we tested the six-factor solution that emerged from the EFAs. As can be seen on Table 4, the six-factor model fit the data very well for both samples with CFI values of .98, TLI values of .97, and SRMR values at or below .03. In pursuit of the most parsimonious model, we then tested a five-factor model that collapsed Basic Needs and Housing & Utilities into one factor. This was based on the EFA findings in which items 2 (house or apartment) and 13 (furniture) loaded together in both samples, but in the FBEP sample they loaded with Basic Needs items and in the SOC Study sample loaded with Housing & Utilities items. In the five-factor model, we joined these two factors to determine if combining these factors would improve model fit. Although the five-factor model fit well, the six-factor model fit the data better according to the mean-adjusted robust chi-square difference test value (see Table 4).

Table 4 Model Comparisons Across Samples
Full size table

We then tested a seven-factor model to determine if we could improve model fit by dealing differently with the two problematic items discussed above. To test whether the model would fit the data better if those two items were depicted as a separate factor, we devised a seven-factor model that split the Housing & Utilities factor into two factors, one with utilities items (i.e., 5, 6, 11, 18) and another with housing (item 2) and furniture (item 13). The seven-factor model fit the data reasonably well for both samples (see Table 4). However, the six-factor model fit the data better.

Finally, to rule out that the null hypothesis that the FRS items are tapping a single construct, we tested a one-factor model that depicts all indicators as related to a single latent factor. As can be seen in Table 4, the CFA findings indicate that the one-factor model fit the data poorly for both samples (i.e., CFI = .69 and 74, TLI = .63 and .69).

Having demonstrated that the six-factor model provided the best overall fit of the data in both samples, we compared structural components across samples. The strength of the relationship between the indicators and the latent variables (factors) are shown in Table 3 as standardized factor loadings. Those for the SOC Study sample appear in parentheses. Larger factor loadings indicate stronger relationships between the observed indicator and the latent variable (i.e., the better the indicator represented that factor). The factor loadings were generally similar across samples although some differences were present. The weakest indicator for the FBEP sample data, and the only one that was below .600 was Ind5 (i.e., house or apartment). Ind8 (i.e., medical and dental care) was the only indicator in the SOC Study sample that had a factor loading below .600. In general, the indicators were better measures of the latent variables within the FBEP sample as all but four of the factor loadings were weaker for the SOC Study sample. For the most part, however, the indicators appear to be adequately related to their latent variables in both samples.

Table 5 presents the inter-factor correlations for both samples. These relationships were estimated as part of the CFA measurement model that simultaneously controls for the relationships among the variables in the model (i.e., among latent variables and among latent variables and indicators). Correlations across factors for the SOC Study sample data appear below and left of the diagonal. Again, the relationships between factors were similar across samples. Across the board, the relationships tended to be small to moderate. In both samples, the strongest relationships were found between the Basic Needs subscale and Housing & Utilities, Benefits, and Extra Resources. Benefits, and Housing & Utilities also demonstrated strong relationships although it was somewhat stronger in the FBEP Sample. Weak relationships were found in both samples between Child Care and the other subscales.

Table 5 Comparison of Inter-factor Correlations from CFAs
Full size table

Internal Consistency

We compared the internal consistency of the total FRS and the subscales that emerged from the EFAs and were confirmed by the CFAs. Listwise deletion in the reliability analyses reduced each sample slightly for each subscale. Table 6 compares Cronbach’s alpha coefficients across samples. The relative internal consistency of each subscale is the same across samples (i.e., from most to least reliable). Although the coefficients are slightly smaller in the SOC Study sample, the differences did not reach statistical significance as measured with Feldt’s statistic. In both samples, five of the subscales demonstrated strong internal consistency, as did the total scale. The least reliable subscale in both samples was Benefits with alpha coefficients just under the customary threshold of .70 for both samples.

Table 6 Comparison of Internal Consistency
Full size table

Criterion-related Validity

Having tentatively embraced the six-factor structure we performed regression analyses to examine whether FRS scores relate to other variables as expected. Listwise deletion eliminated 44 FBEP families from the regression analyses (5%) and 280 SOC Study families (27%). For the most part, SOC Study families were dropped from the analyses because they were missing one or two of the predictor variables; only 6% of the sample was missing three or more variables. Families included in the regression analyses were similar to those excluded on all of the FRS subscale scores and 9 of the 12 predictor variables. Excluded families were statistically significantly larger on average (N=177, M=4.60, SE=1.93) than included families (N=736, M=4.26, SE=1.71). The children in treatment were significantly older (N=206, M=12.86, SE=4.43) in the excluded families than those included (N=736, M=11.95, SE=4.02), and the excluded sample had a larger proportion of families in the Native American and other race categories (N=209, 43.79%) compared to the included group (N=736, 22.14%).

For regression analyses, we calculated subscale scores as the mean of the items in that factor (see Table 3). Subscale scores were not calculated if more than 15% of FRS items were missing. The FRS total score was calculated as the sum of the subscale scores. This approach gives equal weight to each subscale. Findings from the regression analyses indicated that, for the most part, the FRS and its subscales relate to the economic predictor variables in the expected ways (see Table 7). Being in the FBEP sample was significantly related to Basic Needs and Housing & Utilities. However, having more income was positively associated with all FRS subscales except Social Needs/Self Care. The sample by income interaction term demonstrated a significant positive relationship with Basic Needs, Benefits, Social Needs/Self Care, Extra Resources and the Total score. This finding indicates that at a given increase in income, caregivers in the FBEP sample experienced a greater increase in their perception of the adequacy of their families’ concrete resources than did the SOC Study sample. As expected, family size related negatively with Child Care, Social Needs/Self Care, Extra Resources, and the Total score; the larger the family, the less adequate were perceptions of those resources. Having paid for mental health services for their child was related only to Benefits, and in the negative direction. That is, families who had paid for a service perceived their benefits to be less adequate than those who had not.

Table 7 FRS Subscales Regressed on Economic Variables Controlling for Family and Child Characteristics
Full size table

For the most part, the family variables were also associated with FRS scales as hypothesized. Caregiver strain was negatively related to the total score and every subscale score except Housing & Utilities; caregivers who perceived their resources to be more adequate reported less strain. As anticipated, caregivers who reported better role functioning in their families reported having more adequate resources on all subscales. However, the opposite relationship was found for the problem-solving subscale. Better family problem-solving was related to perceptions of less adequate resources for Basic Needs, Child Care, and Social Needs/Self Care, all other variables held constant. Being a biological parent was negatively related to Benefits and Extra Resources. This suggests that caregivers who care for children who are not their biological offspring tend to see their families as having more adequate resources in these areas.

Among the demographic variables, child age was positively related to perceptions of more adequate Child Care, Social Needs/Self Care, and Extra Resources. It stands to reason that as children reach adolescence, the need for child care declines, caregivers have more time to pursue social and self-care activities, and more resources are brought into the family as youth get part-time jobs. Being Hispanic was associated with a perception of having more adequate Benefits, and being White was related to greater perceived adequacy of Basic Needs and Housing & Utilities.

In sum, we conclude from these analyses that the FRS assesses perceived adequacy of concrete resources along six adequately reliable dimensions. Although there were differences in structural components, the six-factor measurement model fit the data from both samples well indicating the FRS is generally appropriate for assessing the concrete needs of families at different levels of economic advantage. Findings from the regression analyses provide construct validity for the FRS and its subscales as demonstrated by significant relationships in the expected directions for all but one predictor variable (i.e., problem solving). However, the findings suggest that the FRS could benefit from some refinements. We discuss those below.

Limitations

This study has limitations that warrant discussion. The two samples used in this study represent a military population and a population of families experiencing economic hardship. Both samples comprised families caring for children receiving mental health services. Hence, these findings may not generalize to relatively affluent non-military families or families who do not have children in treatment for emotional or behavioral disorders. Another weakness of the study is the large proportion of families in the SOC Study sample who were dropped from the regression analyses used to assess construct validity. Although it is encouraging that the included and excluded families appeared similar on all FRS scores and the majority of predictor variables, it is a concern that families included in those analyses were on average smaller, had younger children and were less likely to be in the Native-American or other race categories than families not included. This concern does not pertain to the exploratory and confirmatory analysis and the assessment of internal consistency, as all of the families in the SOC Study sample were included in those analyses. Difference in the original data collection across samples required the higher levels of family income to be collapsed compromising the sensitivity of the family income variable. However, even with this less sensitive variable, the relationship between income and perceived family resources was detected in the regression analyses.

Discussion and implications

One of the problems with using factor analysis for instrument development and refinement is that it is a purely empirical approach and findings are given meaning through theoretical interpretation. Moreover, there is no empirical way to determine whether one interpretation of findings is more accurate than a competing interpretation (Tabachnick & Fidell, 1989). It is incumbent on the investigator to find the meaning of the results given the theoretical underpinnings that guide the work and the purposes to which the tool will be applied. In this examination of the FRS we make every effort to minimize the risk of spurious findings. First, interpretation of findings was guided by family systems and stress and coping theories within the children’s services research context. Second, we cross-validated findings within samples by applying CFA to confirm EFA findings on randomly selected sub-samples. Third, we replicated findings across samples to examine the stability of the FRS structure across economically diverse populations.

Taken together, findings from this study suggest that the FRS demonstrates good validity and reliability using the factor structure that emerged in these analyses. In addition, the FRS worked well for both economically disadvantaged and more financially secure families. Results of the EFAs performed on each sample were remarkably similar and suggested that the FRS assessed resources across six conceptually cohesive dimensions including Basic Needs, Housing & Utilities, Benefits, Social Needs/Self Care, Child Care, and Extra Resources. This six-factor structure was further supported in the CFAs. Internal validity of the FRS subscales and the total scale was found to be adequate, and not significantly different across samples.

Results from this study are similar to those of Van Horn et al. (2001). Differences in the application of the analytic techniques and interpretation of results, however, led to different conclusions. The three factors that emerged in the previous Van Horn et al.’s study (2001) are similar to three of the six factors in the current study. The differences that exist may be explained by the decision, in the earlier study, to eliminate nine items the investigators deemed non-salient or overly complex. In our study, we did not find those items problematic from an empirical perspective and several of them were relevant from a theoretical standpoint. The fundamental premise of this study was that family resources should be assessed in order to (a) discover family needs that should be addressed by programs, and (b) identify barriers to participation in, adherence to, and completion of, recommended interventions. Several of the items dropped in the previous analysis certainly seem important for addressing those goals (e.g., money to buy necessities, money to pay bills, dependable transportation, public assistance, child care), and findings from the current study support the retention of some of those items in the scale. This is not to suggest that Van Horn et al. (2001) were mistaken in their assessment of the FRS, but that these findings offer an alternative factor structure that might be more suitable for the examination and reduction of family-related barriers to service adherence.

All other variables held constant, being in the FBEP sample was related only to Basic Needs and Housing. Some military families enjoy benefits such as subsidized housing and tax-free purchasing on the military post. Many military families also have access to subsidized childcare, but being in the FBEP sample was not related to the Child Care subscale. The SOC Study sample may also have benefited from public programs such as food stamps, and subsidized housing and day care. Because data were not available on these types of public and military benefits, it was not possible to examine the impact of these benefits on perceived adequacy of family resources. Future studies should consider this question.

Although the original 30 items related well to other items in the instrument, some refinements are warranted. Some items may be redundant and could be eliminated without compromising the tool’s measurement quality. Applying item response theory (IRT) would be helpful for identifying redundant items. Findings from the assessment of internal consistency suggested that the Benefits subscale should be revised. In addition, “public assistance” is likely interpreted differently by the two samples. Caregivers in the FBEP sample may see military housing and childcare subsidies as public assistance while the caregivers in the SOC Study may be more focused on food stamps, so-called welfare benefits (e.g., TANF, AFDC), and other income support programs. Similarly, caregivers in these samples likely interpreted item 21 (i.e., money to buy special equipment for children) differently than the original authors intended. Because the FRS was developed for use with families of children with developmental and physical disabilities, this item originally referred to medical equipment needed by children dependent on technology such as wheelchairs, specialized beds, and leg braces. In the two samples included in this research, the children were struggling with emotional and behavioral disorders. It has been reported anecdotally that some caregivers interpreted this item as referring to recreational or sports equipment. We recommend that cognitive interviews be conducted with caregivers to gain a better understanding of how the items are interpreted.

Overall, however, we found that the FRS holds promise as a reliable and valid tool to assess perceived adequacy of resources among economically diverse families of children with emotional and behavioral disorders. Future research should focus on refining the tool to reduce respondent burden and improve interpretation of items. A better formulated theoretical basis and use of an improved instrument for assessing family perceptions of their resources should lead to a more cogent understanding of the role of resources in shaping important service processes such as help-seeking decisions, treatment adherence, service utilization, and parent involvement in their children’s care.