Abstract
In order to offer client-centered services, it is important to measure children’s service satisfaction and reflect their needs to out-of-home care practices and policies. However, a reliable measure that assesses children’s satisfaction about out-of-home care is not found in Korea. This study aimed to develop a Korean out-of-home care satisfaction scale. The study sample consisted of 484 children from institutional care, group homes, and foster homes in Korea. Half of the sample was chosen randomly for exploratory factor analysis (EFA) based on 16 items from the Korean Foster Care Improvements Project. The other half of the sample was used for confirmatory factor analyses (CFA). EFA yielded two-factor structures that consist of eight items for each factor. Confirmatory factor analyses supported the two-factor structures with reasonable fit, and all items loaded significantly on the factors. The Korean out-of-home care satisfaction scale could be used as a tool to assess children’s satisfaction with out-of-home care services, which could allow social workers to reflect children’s needs immediately into practice and help policymakers make more informed decisions about out-of-home care services and programs.
Similar content being viewed by others
Avoid common mistakes on your manuscript.
In Korea, out-of-home care services are provided for children who are separated from their families of origin due to poverty, parents’ divorce, domestic violence, runaway, and other reasons. They are placed and served mostly in one of the three out-of-home care services: institutional care, group homes, and foster care. In 2018, approximately 28,000 children were served in the three out-of-home care services including 11,665 children in institutional care, 2811 children in group homes, and 11,983 children in 9575 foster homes (Korean Ministry of Health and Welfare 2018). For those children who had already been traumatized by negative experiences from their original families, the social stigma and isolation generated from group care prevent adaptation to out-of-home care and even cause negative developmental outcomes (Kang et al. 2009).
To help children adapt well to out-of-home care and ultimately promote their positive and healthy development, it is necessary to provide services that children are satisfied with. However, studies have reported that children are not satisfied with these services in Korea, presenting low satisfaction with living in groups, clothes, curfew, allowance (Noh and Jang 1998) and counseling programs (Kim 2002). Children have also complained about insufficient private rooms, lack of extracurricular activities, and lack of tutoring support (Kim 1995). This is partly because children’s voices have not been reflected during the service-providing process. From the perspectives of service providers, impacts of programs served in out-of-home care have been investigated (Kim 2005; Park and Kim 2010; Park and Lee 2016), although child welfare workers were mostly targeted as respondents for studies that attempted to support out-of-home care service improvement (Ha and Lim 2006; Kim and Seo 2002; Lee 1999; Yang 2003). Little investigation on out-of-home care service satisfaction from children who are the clients and consumers of out-of-home care has been done in Korea. Lack of instruments to measure children’s satisfaction at out-of-home care practices may have contributed to this lack.
Assessing children’s satisfaction with out-of-home care services can not only help practitioners provide practical improvements in services by detecting children’s feeling and attitudes, but it can also help policymakers reflect children’s needs during the policy-making process (Johnson et al. 1995). In addition, the self-esteem of children may be improved by making children aware of and in control of their lives (Gilligan 2000). Ultimately, assessing children’s satisfaction with out-of-home care services and responding immediately may lead to the promotion of out-of-home care services (Delfabbro et al. 2002). Although it is important to assess satisfaction with out-of-home care service from children’s viewpoint, this has been overlooked in Korea, and there does not currently exist a reliable and valid measurement scale. Therefore, this study aims to develop a Korean out-of-home care satisfaction scale based on questions from the Foster Care Improvements Project (Nho et al. 2007).
1 Literature Review
Service satisfaction of clients is one of the methods to assess the quality of services provided and a core indicator that shows the impact of services (McMurtry and Hudson 2000). Although the concept of service satisfaction came from the fields of economics and business (Shin 2014), measuring satisfaction with social services from the perspectives of clients parallels the value of social work practice that respects clients’ opinions and makes clients participate in the process of social work (Geron 1998). There have already been efforts to develop instruments to assess clients’ satisfaction with social services, and standardized measurements are available depending on the field and clients (Kim et al. 2007). In the field of out-of-home care, there have been efforts to develop standardized instruments to assess birth parents’ satisfaction and primary care providers’ satisfaction with out-of-home care services (Harris et al. 2000; Kapp and Vela 2004). However, direct measurements of children’s satisfaction have been relatively understudied, and little attention has been given to develop a valid and reliable standardized instrument to assess children’s satisfaction with out-of-home care.
Whereas there are no studies on the development of children’s satisfaction with out-of-home care instruments, a few studies have assessed children’s satisfaction with out-of-home care directly using different methods. For example, Wilson and Conroy (1999) investigated satisfaction of children in out-of-home care by administering an instrument that they developed to meet the state agency’s needs. The instruments consisted of 56 quantitative questions; questions asked information on children’s satisfaction with various aspects of their lives and services including happiness with where they lived, quality of life in out-of-home placements, and relationships with their caseworkers and caregivers (Wilson and Conroy 1999). The researchers examined test-retest reliability and reported acceptable reliability based on 87.5% to 100% agreements. The researchers also reported face and content validity of the instrument by involving an advisory committee consisting of experts in the field into the process of instrument development. Wilson and Conroy (1999) reported three dimensions of children’s satisfaction in out-of-home care based on their instrument: well-being (“Are you loved?” and “Are you safe?”), quality-of-life, and perceptions of child welfare/provider agency. Quality-of-life was calculated with a composite score of 15 items, including health, clothes, comfort, food, place, feeling loved, feeling safe, family relationships, and happiness, with an alpha value of .94. However, Wilson and Conroy gave no information on how the dimensions were identified.
To assess children’s satisfaction with out-of-home care in South Australia, Delfabbro et al. (2002) adopted the Youth Satisfaction Questionnaire that Stuntzner-Gibson et al. (1995) developed to assess youth satisfaction about general services or activities, which was not specific to children in out-of-home settings. Whereas the Youth Satisfaction Questionnaire included five quantitative items, Delfabbro et al. (2002) expanded it to 11 items; however, their process of item creation was not explained. The measure included nine items on living in foster care such as, “Do you like living with this family?” “Do you feel happy with this family?” and “Do you feel safe with this family?” The measure also included two items on their relationship with their foster caregiver: “Does your foster carer listen to you?” and “Do you think things could be better between you and your foster carer?” (Delfabbro et al. 2002, p.531). A three-point scale (yes, sort of, and no) was used for scoring; the score of each item, not a composite score, was used for the study results.
In the research of Chapman et al. (2004), multiple measures were used to assess children’s satisfaction with out-of-home placement. Questions from the University of California at Berkeley Foster Care Study (Berrick et al. 2000) were selected to assess changes experienced in care, how children liked their current placements, and birth family relations. To measure relationships with children’s current primary caregiver, 12 questions from the Rochester Assessment Package for Schools-Students Relatedness Scale (RAPS) (Wellborn and Connell 1987, as cited in Lynch & Cicchetti 1991), and nine questions taken from the National Longitudinal Study of Adolescents Health (Add Health; Carolina Population Center, University of North Carolina at Chapel Hill 2001) were used. The RAPS questions focused on children’s feelings about their caregiver, the quality of involvement with their caregiver, their autonomy in their relationship with the caregiver, and communication with caregiver in terms of clear expectations and consequences (Chapman et al. 2004). The internal consistency of the 12 questions was .89. The Add Health questions ask about activities that the child and caregiver couple participated in together within the last four weeks, for example, shopping, discussing dating, and working on school projects. The summed score of the nine questions indicated a child’s closeness to their caregiver. Although the Add Health questionnaire has been tested and widely used, Chapman et al. gave no information on the reliability or validity of the measure.
A recent mixed-method study measured children’s satisfaction in out-of-home care using two open-ended questions: “What’s been the hardest thing about being in out of home care?” and “What’s been the most helpful thing about being in out of home care?” and three rating questions: “If you stayed with your family of origin, would your life be better than it is now, worse than it is now, or the same as it is now?” “Overall, what do you think about being placed in out-of-home care; is it very difficult, okay, or very good/helpful?” and rating current caregiver/home as very difficult, okay, or very good (Dunn et al. 2010, p.4). The study also assessed attachment to caregiver with a short form of the People in My Life Measure (PML) (Gifford-Smith 2000) that has demonstrated high internal consistency, from .90 to .83 (Dunn et al. 2010). Items of 30 questions ask about what extent the caregivers, “help me with problems,” “care about me,” “listen to me,” “accept me,” and “are proud of me,” and the degree of “sharing thoughts and feelings with my caregiver” and “feeling angry with my caregiver” (Dunn et al. 2010, p.5). It should be noted that all of the studies that assessed children’s perception on life in out-of-home care looked at well-being, quality of life, and positive relationships with caregivers as important dimensions of children’s satisfaction with out-of-home care.
In Korea, it has only been a decade since attention to children’s satisfaction with out-of-home care has been paid. The first study on the topic, the Korean Foster Care Improvements Project (Nho et al. 2007), adopted items from the study of Kapp and Vela (2004) that developed a three-point scale instrument to assess satisfaction of parents of children in foster care and children’s satisfaction in foster care, changing it into five-point scale. Nho et al. (2007) selected 20 items that were applicable to Korean children in foster care and translated into the Korean language. Half of the items were related to life in foster care such as food, care, and safety, and the other ten items were related to relationship with caregivers, such as listening to me, understanding me, and being empowered by foster parents. Whereas Kapp and Vela (2004) reported good reliability (a = .94) and validity, no information was available for reliability or validity of the 20 items in the study of Nho et al. (2007).
Another study by Nho et al. (2009) also measured satisfaction of children in foster care utilizing the 11 items from Delfabbro et al. (2002). Whereas Delfabbro et al. (2002) used a three-point scale and scores of separate items were used for their study results, Nho et al. (2009) changed it into five-point Likert scale and used a total score for the 11 items for their study results, reporting a reliability of .92. For their part, Kang et al. (2014) measured children’s satisfaction in out-of-home care, foster care, and institutional care, using 17 items adopted from the Korean Foster Care Improvements Project (Nho et al. 2007) and 20 items adopted from the Parents with Children in Foster Care Satisfaction Scale (Kapp and Vela 2004). Internal consistency of the 17 items were reported as .95.
Because the measures in the above studies that assessed children’s satisfaction were adopted from foreign studies, studies done in Korea had similar items about life in out-of-home care and positive relationships with caregivers. However, both foreign and Korean studies used a variety of methods and instruments to assess children’s satisfaction with out-of-home care due to lack of standardized instruments. There was no consistency on scale, scoring, or number of items. More importantly, the instruments used lacked evidence of reliability and validity specific to children in out-of-home care.
Unlike adults who express their opinions actively, children’s voices on social services have been easily ignored because they are relatively less active about their rights as social service consumers (Nho et al. 2009). This means that, in out-of-home care services where the consumers are children, the role of service providers who assess and reflect children’s satisfaction with providing services are even more important (Nho et al. 2009). To do so, a reliable and valid instrument is in need.
Unlike foreign countries, where foster care is the major out-of-home care system, in Korea, institutional care and group homes are significant out-of-home care systems (Korean Ministry of Health and Welfare 2018). Institutional care is on a much larger scale compared to group homes and foster care; for example, the number of children admitted in one institutional care center ranged from 20 to 150. Group homes in Korea are categorized into out-of-home care institutions and supervised by the government; however, one group home can only have a maximum of seven children. Foster care is family-based care, like in other countries; however, in Korea, guardians are mostly grandparents or relatives. The number of children per care provider (including administrative staff) is 2.08 for institutional care, 1.79 for group homes, and 1.26 for foster care (Korean Ministry of Health and Welfare 2018). Therefore, children in institutional care are less likely to receive individualized service compared to those in group homes and foster care, whereas children in foster care are more likely to receive individualized service compared to the others. This may cause differences in care satisfaction that children can perceive.
A standardized instrument that is applicable to all children in foster care, institutional care, and group homes is necessary. Therefore, the purpose of this study is to develop a Korean out-of-home care satisfaction scale based on the items from the Korean Foster Care Improvements Project (Nho et al. 2007). The following steps were conducted for the study: (1) items were selected based on expert review; (2) latent constructs of the selected items were examined based on exploratory factor analysis (EFA); and (3) pre-specified constructs based on EFA were confirmed by confirmatory factor analysis (CFA). In addition, criterion validity was tested based on the previous findings of positive associations between out-of-home care satisfaction and self-esteem (Nho et al. 2009) and service outcomes (Lee and Im 2008).
2 Methods
2.1 Participants and Data Source
The Panel Study on Korean Children in Out-of-Home Care recruited study participants from institutional care, group homes, and foster care in Korea. The target participants of this study were children who were at least fifth to sixth grade in elementary schools so that they were able to provide self-responses. A voluntary written consent form was obtained after providing information on anonymity, confidentiality, and protection from possible discomfort and harm. After obtaining agreements for study participation, surveys were conducted. This study was reviewed by the Institutional Review Board at [blinded for review] every year before the survey.
Participant recruitment was conducted as follows. First, among 250 institutional care centers nationwide, 31 centers in 15 regions were selected using a randomized stratified sampling method based on administrative district. After having agreements of study participation from 137 children, the survey was mailed to the children. For group homes, 332 group home lists were obtained from the Korea Council of Grouphome for Children and Youth (KGCY), and 90 group homes were randomly chosen. One hundred nineteen children from the 90 group homes agreed to participate in this study. To recruit children in foster care, 17 regional foster care centers were contacted. After obtaining lists of grandparents, relatives, and general foster families, children were randomly selected from the lists. Two hundred ninety-seven children agreed to participate in the study, including 137 children in grandparent foster care, 109 children in relative foster care, and 51 children in general foster care. A total of 489 children who were fifth to sixth grade in elementary school participated in the first wave of the study in 2011, and follow-up surveys were conducted every year for the next four years.
This study utilized responses of 484 children who had no missing data on 16 items on out-of-home care satisfaction from the first wave. The average age of the total sample was 11.74 (SD = .70) years; roughly half of the children were boys (n = 245, 50.6%). Approximately 48% (n = 233) of the children were in foster care, 28% (n = 133) were in institutional care, and 24% (n = 118) were in group homes.
2.2 Measures
Korean out-of-Home Care Satisfaction Scale
As indicated earlier, the Korean Foster Care Improvements Project (Nho et al. 2007) adopted 20 items that are applicable to children in out-of-home care, out of 27 items that the Parent Satisfaction with Foster Care Services Scale included to assess parents’ satisfaction with out-of-home care (Kapp and Vela 2004). Nho et al. (2007) translated the items into Korean, changed wordings so that children could answer at a level of fifth-graders in elementary schools, and altered the three-point scale into a five-point Likert scale. The Panel Study on Korean Children in Out-of-Home Care adopted 16 items based on the review of experts in the field of child welfare such as practitioners and researchers; they excluded items that might confuse to children due to their similarity to other questions. In the survey, “care provider” was worded to “foster care parent” for children in foster care and into “institutional teacher” for institutional care and groups homes. A five-point scale was used to assess levels of satisfaction with all items (1 = not agree at all, 2 = a little agree, 3 = sometimes agree, 4 = mostly agree, 5 = always agree).
Self-Esteem
Children’s self-esteem was measured using the Rosenberg Self-esteem Scale (Rosenberg 1965) that consists of ten items about self-worth. All items are measured using a 4-point Likert scale ranging from strongly disagree (1 point) to strongly agree (4 points). Nho et al. (2009) reported internal consistency of .94, and internal consistency for this study was .82. A mean of total score was used to represent a child’s self-esteem; a higher score means higher self-esteem.
Service Outcomes
Self-rated service outcomes from children were measured using seven items. The items present positive changes after living in out-of-home care, for example, “I became healthy in out-of-home care” and “I became proud of myself in out-of-home care.” The scale is considered as a reliable instrument that can assess impact of out-of-home care from the perspective of children (Lee and Chung 2010), and it has been used studies on service outcomes of child welfare institutions in Korea (Chung et al. 2012). Chung et al. (2012) reported internal consistency of .91 for both group home and institutional care children groups. This study also had good reliability of the scale, .91.
2.3 Analytic Method
Several steps were conducted for data analysis. First, the study sample was randomly split into two subsamples, with descriptive statistics conducted in Statistical Software for the Social Sciences 18.0 (SPSS, 2010) for separate analyses, EFA and CFA. To ensure the two subsamples were on par with one another, the characteristics of the subsamples were compared. Each group included 242 children, and there were no significant differences found for demographic characteristics. The results are presented in Table 1.
Second, convergent validity was examined by examining correlations among the 16 items for each subsample. Third, to test the latent factor structure of the 16 items, EFA was conducted. EFA extracts the number and nature of common factors (Fabrigar et al. 1999) and provides item loading that can be used to assign items to the extracted number of factors (Costello and Osborne 2005). Mplus 6.1 (Muthén and Muthén 2010) was used for EFA using oblique rotation (geomin) and maximum likelihood (ML) estimator. Oblique rotation is preferred to orthogonal rotation in psychosocial research because most factors are correlated in this field (Bares et al. 2011; Costello and Osborne 2005; Shmitt 2011).
Fourth, CFA was conducted to verify the pre-specified factor structure by the EFA. CFA was also conducted in Mplus 6.1 using ML. Multiple fit indices were examined to compare fits of the tested CFA models because they assess different aspects of model fit (Byrne 2012). The comparative fit index (CFI; Bentler 1990) and Tucker-Lewis fit index (TLI; Tucker & Lewis, 1973) are the most commonly used incremental indices (Byrne 2012). The cut-off criterion of CFI was a value greater than .90 initially for a good fit in the past (Hu and Bentler 1999); however, at present, a value of .95 or greater for both CFI and TLI indicates a model fits the data well (Byrne 2012). The root mean square error of approximation (RMSEA) accounts for the error of approximation in the population (Byrne 2012). When a model has perfect fit to the data, (i.e., RMSEA = 0) (Browne and Cudeck 1993), values of .06 or less indicate good fit (Hu and Bentler 1999) and a value of .08 indicates reasonable errors of approximation (Browne and Cudeck 1993). However, it should be noted that the fit indices introduced are not golden rules for assessment of model fit but guidelines (Hooper et al. 2008; Marsh et al. 2004). Lastly, correlation tests in SPSS were conducted to test criterion validity of identified factors; demonstration of positive association of the 16 items with self-esteem and service outcomes was examined.
3 Results
3.1 Correlations between Items
Correlations among items were examined (see Table 2). For the subsample used for EFA, the 16 items were significantly associated with each other (p < .01). The lowest correlation coefficient was .20, between item 5 (“care giver, paying attention to me”) and item 7 (“adequate rules and punishment”), and the highest was .80, between item 10 (“receiving care that I hope for”) and item 11 (“receiving help with things that I need). However, most items showed moderate to large (Cohen 1988: small = .10, moderate = .30, large = .50) correlation with each other. For the subsample used for CFA (p < .01), the items were also significantly correlated with each other; all 16 items showed moderated to large positive association with each other. The results for both subsamples support the fact that the 16 items converge into the construct of children’s satisfaction with out-of-home care.
3.2 Exploratory Factor Analysis
The latent factor structure of the Korean out-of-home care satisfaction scale was examined by EFA using half (n = 242) of the randomly split sample. The results are presented in Table 3. The EFA resulted in two factors with eigenvalues greater than one. The scree plot also indicated a two-factor structure. Based on the suggestion from Salkind (2010) that loadings greater than .40 on a factor should only be used for interpretation, item loadings were examined. All items were loaded on each factor with loading that was greater than .40. The way that the items loaded on two factors also demonstrated interpretability suggesting a distinct latent feature for each factor. Factor 1, consisting of eight items, for example, “being empowered by care provider” and “getting praised often care provider,” characterized children’s satisfaction with their relationships with their care provider in out-of-home care. On the other hand, factor 2 characterized their satisfaction with daily living in out-of-home care. The eight items loaded on factor 2 generally featured physical and psychological satisfaction with living conditions in out-of-home care, for example, “adequate rules and punishment” and “receiving help with things that I need.” Mixed results of the fit indices of the two-factor structure were found. Whereas the value of RMSEA was .1, meaning a mediocre fit, SRMR was .04, indicating a good fit. The values of CFI (.92) and TLI (.89) were acceptable.
3.3 Confirmatory Factor Analysis
To confirm the two-factor structure of the model suggested by EFA, multiple CFA were conducted. According to the literature, previous theory or evidence must specify CFA models (Brown 2006). This study used the results of the EFA as evidence of the two-factor structure of the Korean out-of-home care satisfaction scale. Additionally, a unidimensional model was analyzed for comparison of the fit indices to test if the data better supported the two-factor model (Bares et al. 2011). Table 4 and Fig. 1 present the results of the two CFA models, including fit indices and factor loadings. For RMSEA, the two-factor model indicated a mediocre fit (.10), whereas the unidimensional model indicated a poor fit (.13). The values of CFI (.92) and TLI (.89) were acceptable for the two-factor model; however, both CFI and TLI were below .90 for the unidimensional model. However, the values of SRMR for both two-factor and unidimensional models demonstrated good fit reporting, .04 and .05, respectively.
In conclusion, the results of the fit indices suggested that a two-factor model better fit the data than a unidimensional model. Figure 1 demonstrates the model with a two-factor structure. All items for each of the two factors loaded significantly in the expected direction (p < .001). Two latent constructs, “relationship with care provider (factor 1)” and “overall well-being (factor 2),” were supported by the CFA. Factor 1 was significantly associated with factor 2 (r = .87; p < .001).
3.4 Tests of Criterion Validity
Before the tests of criterion validity, internal consistency of factor 1 and factor 2 were examined. Cronbach’s alpha levels of .93 and .90 were observed for the two factors. Relationships between the Korean out-of-home care satisfaction scale and criterion measures were examined based on the two-factor model (see Table 5). The total score of the Korean out-of-home care satisfaction scale showed moderate to large positive relationships with the criterion measures (self-esteem: r = .45, p < .01; service outcomes: r = .75, p < .01). The two subscales, relationship with care provider (factor 1) and overall well-being (factor 2) showed significant positive correlations with all the criterion measures. Relationship with care provider (factor 1) showed a moderate correlation with self-esteem (r = .42, p < .01) and a large association with service outcomes (r = .71, p < .01). The results supported criterion validity of the Korean out-of-home care satisfaction scale.
3.5 Comparison on out-of-Home Care Satisfaction
Care satisfaction among children in three different out-of-home care systems was compared conducting ANOVA. The results are presented in Table 6. Significant differences were found for the total score of the Korean out-of-home care satisfaction scale (F(2, 481) = 18.99, p < .001), and the scores of the two subscales, relationship with care provider (F(2, 481) = 14.45, p < .001) and overall well-being (F(2, 481) = 17.04, p < .001). A Tamhane analysis was conducted for post hoc test due to violation of homogeneity of variance and different sample sizes across the groups. For the total score and the scores of the subscales, significant differences were found between all three out-of-home care systems. Children in group homes demonstrated the highest level of care satisfaction, and those in institutional care presented the lowest level of care satisfaction.
4 Discussion
The purpose of this study was to develop a scale that assesses children’s satisfaction with out-of-home care by investigating the factor structure of 16 items from the Korean Foster Care Improvements Project (Nho et al. 2007). Despite a long history of out-of-home care systems worldwide, an instrument that assesses satisfaction of children who are the direct clients of out-of-home care has not been adequately developed. Especially in Korea, a scale supported by evidence of reliability and validity was not found. Therefore, this study’s development of a Korean out-of-home care satisfaction scale will be valuable for practitioners, policy-makers, and researchers who work for children in out-of-home care.
The two-phase analytic method, using EFA and CFA, was efficient to investigate an empirical factor structure of the 16 items. EFA supported a two-factor structure––relationship with primary caregivers and overall well-being in out-of-home care––with eight items loaded on each factor. Well-being in out-of-home care and positive relationship with primary caregiver were identified as important concepts in assessing children’s satisfaction with out-of-home care in previous studies (Chapman et al. 2004; Delfabbro et al. 2002; Dunn et al. 2010; Nho et al. 2007, 2009; Kang et al. 2014). The two-factor structure was supported by subsequent analyses that compared a two-factor structure model to a unidimensional model using CFA. The two-factor structure better fit the data than the one-factor model. Items for each factor clustered well, suggesting a strong interrelation among items, and positive correlations among the items were supported based on the same direction of all of indicators (Park et al. 2013).
The two factors, overall well-being in out-of-home care and relationship with caregiver, were highly correlated (r = .87), indicating 75% overlap in the variance between the two factors. This may be interpreted as showing that the two factors are not discriminant. However, the two factors clearly convey unique aspects of children’s satisfaction with out-of-home care. Whereas well-being in out-of-home care indicates general satisfaction about life in out-of-home care, relationship with caregiver conceptualizes the degree that children are attached to the primary caregiver in out-of-home care. Previous empirical studies also support the idea that discriminant factors in a scale can be highly correlated (Bares et al. 2011; Park et al. 2013).
Furthermore, the significant results of correlations among the 16 items supported convergent validity of the Korean out-of-home care satisfaction scale. Significant association was found between the Korean out-of-home care satisfaction scale and criterion measures, self-esteem and service outcomes. In addition, internal consistency of the Korean out-of-home care satisfaction scale and the two subscales were excellent. These findings suggest that the Korean out-of-home care satisfaction scale is a valid and reliable measure to assess children’s satisfaction with out-of-home care.
A recent study that applied this 16-item Korean out-of-home care satisfaction scale to analyze the Panel Study on Korean Children in Out-of-Home Care reported continued high levels of Cronbach’s alpha, .96–.97, for the last four years of the study (Lee et al. 2017). Furthermore, their study found that children’s satisfaction with out-of-home care gradually increased in each type of out-of-home care for five years; however, children in institutional care presented the lowest care satisfaction compared to those in group homes and foster care. They also found that as self-esteem and social support increased, children’s care satisfaction in out-of-home care increased and children in group home and foster care demonstrated lower changes in care satisfaction compared to those in institutional care. Lee et al. (2017)‘s study supported the use of the Korean out-of-home care satisfaction scale as a tool assessing children’s care satisfaction in research and practice areas.
However, some limitations of this study should be noted. The results of fit indices were mixed, which may suggest that the model did not fit the data well. However, well-known and widely used measures often do not always report well-fitting of identified factors (e.g., the Parent Stress Index-Short Form) (Deater-Deckard & Scarr 1996; Haskett et al. 2006; Lee et al. 2016; Reitman et al. 2002), and a model that is incorrectly specified still can fit well (Kline 2011). Another limitation in this study is that we tried to make children in three out-of-home care systems comparable using random selection of the study sample and random split of the collected data to conduct EFA and CFA. Due to the purpose of this study, the sample was not homogenous because children were recruited from institutional care, group homes, and foster care; the sample was along not homogeneous because it consisted of fifth and sixth grade children in elementary schools whose literacy levels vary. These inhomogeneities may have caused the mixed results of model fit. Finally, the findings of this study may not be generalizable to children outside Korea because the setting of this study was limited to out-of-home care children in Korea.
This research was an initial study on the Korean out-of-home care satisfaction scale. More validation studies are necessary to support wide use of the Korean out-of-home care satisfaction scale as a reliable and valid instrument. Studies with a larger sample size that recruit children via a rigorous study method may supplement the limitations of this study. In particular, for foreign countries that have different out-of-home care systems and different characteristics of children, validation of the scale to their children will be necessary before administering the scale.
Nonetheless, this study has some significance in the field of social work. First of all, the Korean out-of-home care satisfaction scale can benefit many children in out-of-home care. For example, they can become aware of their own lives in out-of-home care by responding to the scale (Gilligan 2000). Furthermore, the scale can be administered to children in a variety of out-of-home care settings, including institutional care, group homes, and foster cares. Also, this scale can be administered to most youth in third and higher grades, aged approximately 10 and older, because the scale was initially developed for the third-grade level (Nho et al. 2007) and the evidence of this study with fifth to sixth grade children supported validity and reliability of the scale. Lee et al. (2017) even supported the possible application of this scale to adolescents by reporting good reliability for the next four-year follow-ups, although further examination with different groups of older children is needed to confirm their results. In addition, researchers in social work can conduct more research based on this study; validations on the Korean out-of-home care satisfaction can be done, and another strong instrument can be developed based on this scale.
More importantly, use of the Korean out-of-home care satisfaction scale in practice can lead to immediate responses from practitioners and policymakers to children’s needs. Workers in out-of-home care and social workers can provide better out-of-home care service by detecting children’s level of satisfaction. Policymakers can make more informed decisions about out-of-home care services and programs by hearing the voices of the direct clients––children. Furthermore, the two subscales developed here––relationships with care provider and overall well-being in out-of-home care––can help social work professional discern specific areas that need to be ameliorated for better out-of-home care services. The Korean out-of-home care satisfaction scale will be a valuable tool for out-of-home care stakeholders and ultimately foster promotion of out-of-home care services.
References
Bares, C. B., Andrae, F., Delva, J., & Grogan-Kaylor, A. (2011). Examining the factor structure of anxiety and depression symptom items among adolescents in Santiago, Chile. Journal of the Society for Social Work and Research, 2(1), 23–38.
Bentler, P. M. (1990). Comparative fit indexes in structural models. Psychological Bulletin, 107(2), 238–246. https://doi.org/10.1037/0033-2909.107.2.238.
Berrick, J. D., Frasch, K., & Fox, A. (2000). Assessing children's experiences of out-of-home care: Methodological challenges and opportunities. Social Work Research, 24(2), 119.
Brown, T. A. (2006). Confirmatory factor analysis for applied research. New York: Guilford Press.
Browne, M. W., & Cudeck, R. (1993). Alternative ways of assessing model fit. In K. A. Bollen & J. S. Long (Eds.), Testing structural equation models (pp. 136–162). Newbury Park: Sage.
Byrne, B. M. (2012). Structural equation modeling with Mplus: Basic concepts, applications, and programming. New York: Routledge.
Carolina Population Center, University of North Carolina at Chapel Hill. (2001). The national longitudinal study of adolescent health. Author.
Chapman, M. V., Wall, A., & Barth, R. P. (2004). Children’s voices: The perceptions of children in foster care. American Journal of Orthopsychiatry, 74(3), 293–304.
Chung, I. J., Woo, S., Kang, H., Chun, J., & Lee, J. (2012). The factors influencing service outcomes of group homes and residential care centers: Focusing on blinder-Oaxaca decomposition. Korean Journal of Child Studies, 33(4), 107–127.
Cohen, J. (1988). Statistical power analysis for the behavioral sciences (2nd ed.). Hillsdale, NJ: Lawrence Earlbaum Associates.
Costello, A. B., & Osborne, J. W. (2005). Best practices in exploratory factor analysis: Four recommendations for getting the most from your analysis. Practical Assessment, Research & Evaluation, 10(7), 1–9.
Deater-Deckard, K., & Scarr, S. (1996). Parenting stress among dual-earner mothers and fathers: Are there gender differences?. Journal of Family Pschology, 10(1), 45.
Delfabbro, P. H., Barber, J. G., & Bentham, Y. (2002). Children's satisfaction with out-of-home care in South Australia. Journal of Adolescence, 25(5), 523–533.
Dunn, D. M., Culhane, S. E., & Taussig, H. N. (2010). Children’s appraisals of their experiences in out-of-home care. Child Youth Services Review, 32(10), 1324–1330.
Fabrigar, L., Wegener, D., MacCallum, R., & Strahan, E. (1999). Evaluating the use of exploratory factor analysis in psychological research. Psychological Methods, 4, 272–299.
Geron, S. M. (1998). Assessing the satisfaction of older adults with long-term care services: Measurement and design challenges for social work. Research on Social Work Practice, 8(1), 103–119.
Gifford-Smith, M. (2000). People in my life. Retrieved June, 10, 2009.
Gilligan, R. (2000). The importance of listening to the child in foster care. In G. Kelly & R. Gilligan (Eds.), Issues in foster care: Policy, practice, and research (pp. 40–58). London: Jessica Kingsley Publisher.
Ha, S. M., & Im, D. H. (2006). A research on the status of welfare for children and suggestions for their improvements. Journal of Child Welfare and Development, 4(1), 145–171.
Harris, G., Poertner, J., & Joe, S. (2000). The parents with children in foster care satisfaction scale. Administration in Social Work, 24(2), 15–27.
Haskett, M. E., Ahern, L. S., Ward, C. S., & Allaire, J. C. (2006). Factor structure and validity of the parenting stress index-short form. Journal of Clinical Child and Adolescent Psychology, 35, 302–312. https://doi.org/10.1207/s15374424jccp3502_14.
Hooper, D., Coughlan, J., & Mullen, M. (2008). Structural equation modelling: Guidelines for determining model fit. Articles, 2.
Hu, L. T., & Bentler, P. M. (1999). Cutoff criteria for fit indexes in covariance structure analysis: Conventional criteria versus new alternatives. Structural Equation Modeling: A Multidisciplinary Journal, 6(1), 1–55.
Johnson, P. R., Yoken, C., & Voss, R. (1995). Family foster care placement: The child's perspective. Child Welfare, 74(5), 959–974.
Kang, H., Chung, I. J., Chun, J., Nho, C. R., & Woo, S. (2014). The outcomes of foster care in South Korea ten years after its foundation: A comparison with institutional care. Children and Youth Services Review, 39, 135–143.
Kang, H., Shin, H. R., & Park, E. M. (2009). The comparisons of the status of independence and social adjustment of youths who aged out from child residential care. Journal of the Korean Society of Child Welfare, 30, 41–69.
Kapp, S. A., & Vela, R. H. (2004). The unheard client: Assessing the satisfaction of parents of children in foster care. Child and Family Social Work, 9, 197–206.
Kim, E. J. (1995). Reform measures and problems of the actual living state of institution-cared children. Master Thesis, Daegu University, Korea.
Kim, J., & Seo, Y. (2002). A study on the perception and satisfaction of foster caregiver about foster care. Sookmyung Women’s University Child Study Center, 15(2), 1–23.
Kim, S. H. (2002). A study on the current conditions of the institutionalized adolescents. Master Thesis, Daejeon University, Korea. Retrieved from http://www.riss.kr/search/detail/DetailView.do?p_mat_type=be54d9b8bc7cdb09&control_no=702a6f8d8cdf2554
Kim, S. M. (2005). Emotional effects of art program using clay to the orphanage teenagers. Master Thesis, Ewha Womans University, Korea. Retrieved from http://www.riss.kr/search/detail/DetailView.do?p_mat_type=be54d9b8bc7cdb09&control_no=2b6b2c86ed5f344fffe0bdc3ef48d419
Kim, Y. S., Lee, E. Y., Ko, K. E., & Min, E. H. (2007). Reliability and validity of the Korean version of the client satisfaction inventory. Korean Journal of Social Welfare, 59(4), 83–109.
Kline, R. B. (2011). Principles and practice of structural equation modeling (3rd ed.). New York: The Guilford Press.
Korean Ministry of Health and Welfare. (2018). Statistics of children in out-of-home care. Retrieved from http://www.mohw.go.kr/react/jb/sjb030301vw.jsp?PAR_MENU_ID=03&MENU_ID=0321&CONT_SEQ=345304&page=1.
Lee, K. R., & Chung, I. J. (2010). Service outcomes of night care program and children’s psychosocial adjustment in community child centers. Korean Journal of Community Welfare, 34, 183–206.
Lee, K. Y., & Im, H. (2008). Influence of service quality and client satisfaction on the psychiatric social work service outcomes. Mental Health and Social Work, 28, 175–203.
Lee, S. J., Gopalan, G., & Harrington, D. (2016). Validation of the parenting stress index-short form with minority caregivers. Research on Social Work Practice, 26(4), 429–440.
Lee, S. J., Kang, H., Nho, C., Woo, S., Chun, J., & Chung, I. (2017). Longitudinal study on care satisfaction of children in out-of-home: Comparison among residential care centers, group homes, and Foster homes. Korean Journal of Social Welfare, 69(3), 97–120.
Lee, S. Y. (1999). A study on the environment of the family foster care. Korean Journal of Family Welfare, 4(2), 43–62.
Lynch, M., & Cicchetti, D. (1991). Patterns of relatedness in maltreated and nonmaltreated children: Connections among multiple representational models. Development and Psychopathology, 3(2), 207–226.
Marsh, H. W., Hau, K. T., & Wen, Z. (2004). In search of golden rules: Comment on hypothesis-testing approaches to setting cutoff values for fit indexes and dangers in overgeneralizing Hu and Bentler's (1999) findings. Structural Equation Modeling, 11(3), 320–341.
McMurtry, S. L., & Hudson, W. W. (2000). The client satisfaction inventory: Results of an initial validation study. Research on Social Work Practice, 10(5), 644–663.
Muthén, L. K., & Muthén, B. O. (2010). Mplus user’s guide 6.1. Los Angeles, CA: Muthén & Muthén.
Nho, C., Park, E., Kang, H., & Shin, H. (2007). Foster care improvements project. Policy Report: Korean Ministry of Health and Welfare.
Nho, C. R., Park, K. H., & Seo, J. W. (2009). Study on foster care satisfaction among children in foster care: Focusing on individual, foster parents, and foster care service factors. Journal of the Korean Society of Child Welfare, 29, 155–185.
Noh, H. L., & Jang, J. S. (1998). A study on the life condition and satisfaction of children residing in institution. Journal of the Korean society of child welfare, 6, 65–92.
Park, H., Barth, R. P., & Harrington, D. (2013). Factor structure of adoptive parent-child relationship items form the national study of adoptive parents. Journal of Society for Social Work and Research, 4(1), 20–30. https://doi.org/10.5243/jsswr.2013.2.
Park, B. J., & Lee, S. M. (2016). The effect of group art therapy on sociality of children in institutional care: Focus on anger and empathy. Industry Promotion Research, 1(2), 71–78.
Park, Y. H., & Kim, H. R. (2010). Effect of horticultural therapy on the self-efficacy and emotional intelligence of children in the child welfare institutions. Journal of People Plants and Environment, 13(2), 7–11.
Reitman, D., Currier, R. O., & Stickle, T. R. (2002). A critical evaluation of the parenting stress index-short-form (PSI-SF) in a head start population. Journal of Clinical Child and Adolescent Psychology, 31, 384–392. https://doi.org/10.1207/S15374424JCCP3103_10.
Rosenberg, M. (1965). Rosenberg self-esteem scale (RSE). Acceptance and commitment therapy. Measures package, 61, 52.
Shin, Y. J. (2014). A study on performance evaluation on the delivery system of social welfare service for the unmarried mother: Focused on the user satisfaction. Master Thesis, Hanyang University, Korea. Retrieved from http://www.riss.kr/search/detail/DetailView.do?p_mat_type=be54d9b8bc7cdb09&control_no=6a1589c1143be102ffe0bdc3ef48d419
Shmitt, T. A. (2011). Current methodological considerations in exploratory and confirmatory factor analysis. Journal of Psychoeducational Assessment, 29, 304–321.
Stuntzner-Gibson, D., Koren, P. E., & DeChillo, N. (1995). The youth satisfaction questionnaire: What kids think of services. Families in Society, 76(10), 616.
Tucker, L. R., & Lewis, C.(1973). A reliability coefficient for maximum likelihood factor analysis. Psychometrika, 38(1), 1–10.
Wellborn, J. G., & Connell, J. P. (1987). Manual for the Rochester assessment package for schools.
Wilson, L., & Conroy, J. (1999). Satisfaction of children in out-of-home care. Child Welfare, 78(1), 53–69.
Yang, S. (2003). The characteristics of family foster care types and factors of the adjustability for the foster children. Family and Environment Research, 41(5), 131–148.
Funding
This work was supported by the Ministry of Education of the Republic of Korea and the National Research Foundation of Korea(NRF-2016S1A3A2924375).
Author information
Authors and Affiliations
Corresponding author
Additional information
Publisher’s Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
Open Access This article is distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made.
About this article
Cite this article
Lee, S.J., An, E.M. & Chung, IJ. Assessing Satisfaction of Children in out-of-Home Care: Development of Korean out-of-Home Care Satisfaction Scale. Child Ind Res 13, 1217–1233 (2020). https://doi.org/10.1007/s12187-019-09688-6
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s12187-019-09688-6