Introduction

Recent shifts in child welfare policies, increases in child welfare caseloads, declines in availability of traditional foster care homes, and case workers’ favorable view of kin as foster parents have made placement with relatives the most common type of foster care for children with relatives willing and able to assume care (Dolan et al. 2009; Park and Greenberg 2007). Grandparents are among the first to take on this responsibility, and an estimated 937,784 grandparent householders provide care to a grandchild (GC) under age 18 without birth parents present (2011 American Community Survey). Families of this composition are known as either “custodial” or “skipped generation” families in which custodial grandmothers (CGMs) almost always serve as the primary caregiver (Park and Greenberg 2007).

Because the need for surrogate parenting by CGMs is largely due to a crisis or tragedy in the birth parent generation (e.g., drug use, teen pregnancy, divorce, mental and physical illness, AIDS, death, crime, child abuse and neglect, incarceration), most CGMs end up in a long-term commitment with about 40 % having this role for 5 or more years (Simmons and Dye 2003). Even though parenting is now recognized as the major responsibility of CGMs (Dolbin-MacNab 2006; Dunifon 2013), little is known about their parenting practices and how to best assess them.

Numerous stressors are faced by CGMs in relation to their child care responsibilities (e.g., financial strain, insufficient social support, life style disruption, disregard by service providers, social stigma, role ambiguity, health adversity, conflict with birth parents) that may increase their psychological distress and diminish their ability to parent effectively (Dunifon 2013; Hayslip and Kaminski 2005; Kelley et al. 2011). This is salient because there is consensus in the parenting literature that caregiver distress is related to poor parenting, that poor parenting is related to child adjustment, and that parenting mediates the association between caregiver distress and child adjustment (Deater-Deckard 1998; Downey and Coyne 1990; Shelton and Harold 2008). Children of distressed caregivers are at risk for adjustment problems even when caregivers face daily hassles only (Elgar et al. 2007).

Examining the parenting practices of CGMs is also necessary because the GCs in their care are at higher risk for psychological difficulties than age peers in the general population (Smith and Palmieri 2007). This is due primarily to exposure to prenatal toxins in the womb and to such early adversities as abuse, abandonment, and neglect by birth parents. Because some parenting behaviors (e.g., nurturance and apt discipline) can lessen the mental health problems of children previously exposed to major stressors (Sandler et al. 2003), it is critical to identify adequate measures of these parenting practices for CGMs.

It is also possible that parenting by CGMs is different than it is for young parents, which raises the question of whether parenting measures developed for birth parents are applicable to CGMs. Kaminski et al. (2008) found that grandparent caregivers were less sensitive to GCs’ needs and were less clear about proper parent–child role responsibilities than birth parents. An intergenerational transmission of poor parenting may also exist given that many CGMs did poorly at raising their own dysfunctional children, and might thus be even less competent in raising a GC (Climo et al. 2002; Gibson 2005; Smith et al. 2000). In fact, CGMs often feel guilty about how their offspring have fared and question their own parenting competency (Baird 2003; Edwards 2003; Glass and Huneycutt 2002; Smith and Dannison 2001; Williamson et al. 2003).

These caregivers have also been found to express difficulty in disciplining and setting limits due to the conflicting nature of their roles as grandparents and caregivers (Bratton et al. 1998). Shifting from being a traditional grandparent to caregiver may also yield role confusion and internal conflict given that the authoritative parent role means letting go of traditional grandparenting which sanctions fun, indulgence, and unconditional love (Glass and Huneycutt 2002; Landry-Meyer and Newman 2004; Weber and Waldrop 2000). Age-related factors may further influence the parenting practices of CGMs. Many worry about their ability to parent a GC due to their advanced age and corresponding health problems (Berrick 1997; Landry-Meyer and Newman 2004; Park and Greenberg 2007). It is further claimed that CGMs have outdated knowledge of childrearing and would benefit from training and educational activities on effective parenting (Bratton et al. 1998; Glass and Huneycutt 2002; Williamson et al. 2003).

Only a handful of published studies have examined the parenting practices of CGM to date, and it is unknown how suitable the parenting measures used in those studies are for CGMs. Rodgers-Farmer (1999) hypothesized that CGMs experiencing stress related to their parenting role would report being depressed and use inconsistent discipline, punishment, and rejection-oriented techniques. She measured these parenting practices with the Management of Children’s Behavior Scale (MCBS; Rodgers 1998), a 20-item questionnaire rated on a five-point scale. Alpha coefficients for these subscales were found to be acceptable (punishment = 0.77; inconsistent = 0.71; rejection-oriented behaviors = 0.77). However, a factor analysis was not conducted to examine the purported dimensionality of these three subscales. Nevertheless, hierarchical multiple regression analyses revealed that parenting stress had a significant effect on depression, and depression had a significant effect on inconsistent parenting practices (but not on the use of either punishment or rejection-oriented behaviors).

Smith and Richardson (2008) examined the psychometric properties of an adaptation of the Parenting Practice Interview (PPI; Webster-Stratton et al. 2001) in a study with 733 CGMs. The PPI was selected over other potential measures for two reasons: (a) Unlike the MCBS, the PPI measures both “effective” (Appropriate Discipline, Monitoring and Positive Parenting) and “ineffective” (Inconsistent and Harsh Discipline) parenting practices; and (b) these specific practices are related to behavioral outcomes of at-risk children (Perepletchikova and Kazdin 2004; Rubin and Burgess 2002; Webster-Stratton et al. 2001). Factor analytic methods revealed that each PPI item assessed uniquely the respective parenting practice it was originally intended to measure. Consistent with the general parenting literature, zero order correlations revealed that the ineffective practice subscales were associated with greater CGM psychological distress and more behavioral problems in CGs while the opposite was true for the effective subscales. A major shortcoming, however, is that each subscale was assessed by only three items which comprised the internal consistencies of these scales.

Smith et al. (2008) examined the relevance of the Family Stress Model (FSM), which considers causal links among family stressors, caregivers’ psychological distress, dysfunctional parenting practices, and children’s adjustment (Conger et al. 2002), to parenting by CGM. They modeled dysfunctional parenting as a higher-order factor encompassing two first-order factors (ineffective discipline and low nurturance). The indicators of ineffective discipline within their structural model were the three-item harsh and inconsistent disciple scales derived previously from the PPI by Smith and Richardson (2008). The indicators of low nurturance, both comprising subscales from the Parenting Stress Index (Abidin 1995), were the Reinforces Parent subscale (six items measuring how much a caregiver projects negative responses onto a child) and the Attachment subscale (seven items assessing emotional closeness and the caregiver’s real or perceived ability to observe and understand the child’s needs and feelings).

Smith et al. (2008) found both the measurement and structural components of their model to fit the data well for their sample. Findings regarding the structural model supported the basic FSM tenet that the effects of caregivers’ psychological distress on children’s adjustment are mediated through poor parenting. In turn, both the measurement and structural components of their model were largely invariant by CGM race (White and Black), CGM age (<55 and >55), CG age (4–7; 8–11; and 12–17), and CG gender. A key limitation, however, was that several of the parenting measures had low alphas (Attachment = 0.60; Reinforces Parent = 0.69; Harsh Discipline = 0.66; and Inconsistent Discipline = 0.54). Thus, uncertainty remained regarding the measurement properties of these instruments.

The present study encompassed three specific aims designed to assess the overall reliability and validity of several extant measures of parental nurturance and discipline, that were originally with birth parents, for their use with CGM. The first aim was to examine the underlying factor structure of these measures by using a confirmatory factor analytic (CFA) approach to test the four nested models, shown in Fig. 1. After identifying the best fitting of these models, the second aim was to examine if that model would be invariant by GC age. Parenting differs according to the age of the child which has implications for the applicability of measures of parenting (Smith 2011). The third aim was to investigate the construct validity of the final set of parenting scales identified under Aim 1. From the theoretical perspective of the FSM (Conger et al. 2000), which postulates the impact of parental psychological distress on children’s outcomes is mediated by parenting practices, we hypothesized that those parenting practices indicative of higher nurturance and effective discipline would be associated with lower psychological distress for CGMs as well as fewer psychological difficulties among CGs. In contrast, ineffective discipline (i.e., in the form of both punitive and inconsistent) was hypothesized to be associated with greater CGM distress and more difficulties for CGs.

Fig. 1
figure 1

Model 1: Two second order—Nurturance and Discipline. Model 2: Two second order—Nurturance and Discipline, w/Effective Discipline. Model 3: One second order—Discipline, w/PAI and SEB. Model 4: Five factor, first order, * Effective, Punitive, and Consistency are from original PPI factors; PAI (Positive Affect Index); SEB (Supportive Engagement Behaviors)

Full size image

Method

Participants

The participants were 343 CGMs enrolled in a RCT designed to compare two evidenced- based interventions (behavioral parent training and cognitive behavioral skills training) to each other and to a theoretically inert control condition. Inclusion criteria were that CGMs had provided care to a CG between ages 4–12 for at least 3 months at her home in complete absence of the CG’s birth parents; were fluent in the English language; were willing and able to attend 10 2-h long group sessions at a community site; and self-identified as being of either White, Black, or Hispanic origin. Recruitment occurred across four states (California, Maryland, Ohio, and Texas) and included diverse methods (e.g., mass media announcements; contacts through schools, social service and health agencies, courts, libraries, faith communities, and support groups; appearances at community events; brochures; and letters mailed to randomly selected households). The RCT was described to potential participants as providing “information that can help grandmothers get through the difficult job of caring for grandchildren in changing times”. If a CGM was caring for multiple CG who met study eligibility criteria then a target CG was selected based on asking the CGM which child was the most difficult to provide care to. The target CG was then used as the reference for all measures reported on here.

Key sociodemographic and background characteristics for these 343 CGMs and their target CGs are presented in Table 1. The mean age of the grandmothers was 58.45 (SD = 8.22, and 7.8 (SD = 2.6) for the GCs. The age range for CGMs was 40–89 years. Most CGMs were either Caucasian (44 %) or African American (43 %), followed by Hispanic/Latino (11 %), and then “other” (1 %). Only 38 % of CGMs were married, with 51 % being divorced or widowed. Most (44 %) of the CGMs had completed some college, 19 % earned their GED or high school diploma, 13 % did not complete high school, 13 % received bachelor’s degrees, and 6 % had graduate or professional degrees. At the time of this survey, 44 % of CGMs were working part-time, 19 % were retired, 13 % were unemployed/looking for work, and 13 % were working full-time.

Table 1 Demographic Information (N = 343)
Full size table

Procedure

At RCT baseline each CGM completed a telephone interview during which all of the measures for this study were obtained via self-report. Both verbal and written consent were obtained.

Measures and Data Analyses

Only measures of the broader constructs of discipline and nurturance were considered because these two constructs comprise the most influential parenting mechanisms known to affect adjustment problems in children (Cummings et al. 1994; Locke and Prinz 2002; Lovejoy et al. 2000; Rubin and Burgess 2002). We also selected measures that are brief (to reduce respondent burden), have shown good psychometric properties with birth parents, and include items with content that appears relevant to caregivers of children ages 4–12. We also favored scales that have been used in parenting intervention research. The specific items associated with each of these scales and their response alternatives are presented in Table 2.

Table 2 Manifest content
Full size table

Nurturance

Measures of this construct were selected in line with the view that parental nurturance is concerned with providing a positive atmosphere for the parent–child relationship and the child’s emotional development through both emotional (e.g., communication of acceptance) expressions and instrumental (e.g., playing a game together) acts (Locke and Prinz 2002).

The more instrumental side of nurturance was assessed by the 10-item SEB subscale of the PBI (Lovejoy et al. 1999), which corresponds closely to the construct of parental warmth and involves behaviors demonstrating acceptance of the child through affection, shared activities, and emotional and instrumental support (Lovejoy et al. 1999). With a sample of parents of preschool age and young school age children, the developers found support for the test–retest, internal consistency, inter-observer reliability, and construct validity of the SEB scale. In the present study, CGMs were asked to rate each of the 10 SEB items (Table 1) on a scale ranging from 0 “not at all” to 5 “very true”. Items were then summed to yield a total SEB score with a potential range of 0–50 with higher scores indicating more supportive and engaged behavior reported by the CGM (α = 0.88).

The more emotional form of nurturance was assessed with the 10-item PAI which measures the degree to which caretakers (e.g., parents and grandparents) report trust, fairness, respect, affection, and understanding between themselves and their child, as well as their perception of how the child feels about them in terms of these same core relationship dimensions (Bengtson and Schrader 1982). The items on the PAI were found by its developers to load highly on a single factor, with relatively uniform loadings. (Bengtson and Schrader 1982). In the present study, CGMs rated each item (Table 1) along a scale from 0 “none” to 4 “a great amount”. Total scores were computed by summing all items, with a potential range of 0–40 (α = 0.87). We chose this measure because it was designed to measure relationship quality irrespective of the target child’s age, and thereby averts the concern that specific acts of emotional expressiveness vary considerably by child age (Locke and Prinz 2002).

Discipline

Three specific types of discipline (Effective, Inconsistent, and Punitive) were measured by means of the PPI, a 17-item likert style instrument designed to assess these particular disciplinary styles that are known to be related to children’s adjustment outcomes (Lochman and Conduct Problems Prevention Research Group 2010). These three PPI sub scales, as established via exploratory factor analysis (Lochman and Conduct Problems Prevention Research Group 2010), reflect the view that discipline includes both parenting techniques deemed to be more effective and less effective (Locke and Prinz 2002). In the present study, CGMs rated each PPI item (Table 1) on a four-point scale ranging from 1 “never” to 4 “often”. The 6 items tapping Effective discipline were summed to yield a total core with a potential range of 6–24 (α = 0.83). The 6 items tapping Consistent discipline were summed to yield a total score with a potential range of 6–24 (α = 0.79). The 5 items tapping Punitive discipline were summed to yield a total score with a potential range of 5–20 (α = 0.73). Higher scores for each scale reflect greater frequency of the parenting behavior being assessed. An important caveat, however, is that the PPI Consistent scale was actually constructed to reflect parenting behaviors that are inconsistent in nature (See Table 1). Thus, higher scores on this scale are indicative of more frequent use of inconsistent disciplinary approaches.

TG Adjustment

The broadband internalizing and externalizing subscales from the parent-informant version of the Strengths and Difficulties Questionnaire (SDQ; Goodman 2001) were used. The SDQ has shown good psychometric properties in use with custodial grandfamilies (Palmieri and Smith 2007) and generally correlates highly with other indices of childhood maladjustment (Goodman 2001). Externalizing problems were assessed by summing the Hyperactivity-Inattention and Conduct Problems scales (potential range = 0 to 20; α = 0.75), while Internalizing problems were assessed by summing the Emotional Symptoms and Peer Problems scales (potential range = 0–20, α = 0.74) Each scale contained five items that were rated by grandmothers regarding the target GC behavior on a 3-point scale from 0 (not true) to 2 (certainly true). Higher scores indicate greater levels of the measured behavior.

GCM Psychological Distress

This included self-report measures of both depression and anxiety. Depressive symptoms were measured with the 20-item Center for Epidemiological Studies-Depression scale (CES-D; Radloff 1977). For each item, participants endorsed the response that best described how often they had felt a particular way in the past week on a 4-point Likert-type scale from 0 (rarely or none of the time-less than 1 day) to 3 (most or all of the time—5 to 7 days). Potential CES-D scores ranged from 0 to 60 (α = 0.91).

Anxiety was assessed using the 5-item Overall Anxiety Severity and Intensity Scale (OASIS; Norman et al. 2006) where items (e.g., “In the past week, how often have you felt anxious?” are asked on a five point Likert scale ranging from 0 “never” to 4 “all the time”. Potential scores range from 0 and 20 (α = 0.86), with higher scores representing greater anxiety.

From the 343 CGMs who qualified for placement into the RCT, there were 14 (4.1 %) who did not complete all measures. For these cases, multiple imputations was performed using Mplus 7.1 (Muthén and Muthén 2014). Multiple imputation is a process by which missing values within a variable are estimated and added to the data set based on patterns and relationships existing between other values.

The four measurement models depicted in Fig. 1 were tested respectively via CFA using EQS. 6.0. Bentler 1995). Model 1 is a five-factor, higher order solution, which corresponds to claims in the parenting literature that discipline and nurturance comprise the most important parenting mechanisms to affect adjustment problems in children (Cummings et al. 1994; Locke and Prinz 2002; Lovejoy et al. 2000; Rubin and Burgess 2002). It also represents the most parsimonious model. Model 2 differs only by excluding the effective discipline first-order factor from loading onto the second order discipline factor as proposed in Model 2. The rationale for this exclusion is that item content on the effective discipline factor emphasizes how a child behaves as a results of disciplinary tactics, whereas the other two discipline factors focus on what a parent actually does to discourage unwanted behavior (see items in Table 1). Moreover, measuring discipline effectiveness is generally thought to be much less clear than assessing ineffective discipline (Locke and Prinz 2002). Model 3 differs from Model 1 by excluding the second-order nurturance factor. The rationale for this difference is that items on the PAI emphasize perceptions of the parent–child relationship, whereas the SEB scale measures nurturing behaviors (see Table 1). Moreover, as Locke and Prinz (2002) have noted, “it is an open question whether nurturance is more useful as a unitary global construct or as a set of related subconstructs” (p. 922). Model 4 is a lower order version of Models 1–3, in which the relations among the five parenting measures are explained by their own unstructured covariation rather than by any overarching second-order factors. This slightly less parsimonious model emphasizes the importance of viewing each parenting practice in its own right independently from other measures of similar constructs. It also recognizes the measurement particularities considered within Models 2 and 3.

Because initial screening of items revealed skewed distributions the Maximum Likelihood Robust (MLR) estimation was chosen, which has the ability to more accurately estimate data outside a normal distribution (Savalei 2010). The model exhibiting the best overall fit was chosen for further analysis and further model building with young and old GC subgroups as described below. Four indices were used to evaluate model fit (Schumacker and Lomax 2010): Satorra–Bentler Chi-square, which functions as an adjusted Chi-square estimation for use with MLR (Muthén 2007); the root mean square error of approximation (RMSEA, with values equal to or lower than 0.05 considered to be an indication of good model fit; the standardized root squared residual (SRMR), a measure of absolute model fit, with values less than 0.08 indicating good fit.; These four fit indices provide the variety of model examination sufficient to support or refute the structure of the data as similar enough to the proposed models (Schumacker and Lomax 2010).

After the best fitting of the four proposed models was identified it was then examined for invariance by CG age groups of 4–<7 versus >7–12 years old. This age group distinction was made in light of the shift from Piaget’s preoperational stage of cognitive development to the concrete stage at about age seven, when children begin to reorganize mental images and symbols to create logically formed thoughts which are then modified and reinforced by parental behavior (Shaffer 2008; Slavin 1988).

First, the best fitting theoretical model was applied to each of the two GC age groups separately and evaluated by CFA to generate model fit statistics. Next, modifications to model structure in regards to cross-loadings and item error covariances were examined and incorporated sequentially within each age group. After no more theoretically meaningful modifications could be applied to each sample model, the age-specific modifications were then combined into a single model that was applied to both groups during the test of invariance. This was accomplished by constraining the factor loadings of each groups’ measurement model to be equal, and then comparing its Satorra–Bentler Chi-square value to that of a model in which factor loadings we estimated freely across groups. A non-significant Satorra–Bentler Chi-square difference would indicate that the hypothesis of loading appeared reasonable.

Lastly, zero order correlational analyses were performed to demonstrate the concurrent validity of the parenting scales. We specifically examined their hypothesized relations with indices of CGM psychological distress (symptoms of depression and anxiety) and GC behavioral and emotional difficulties (internalizing and externalizing symptoms). The FSM served as the conceptual framework for this correlational analysis.

Results

Estimates of the fit statistics for each the four proposed models are summarized in Table 3, where it is shown that Model 4 yielded the best overall fit to the observed data. Model 4 also yielded statistically significant loadings for each item on its respective factors and no modifications were indicated to the specification of the items outside of the suggestions of the scales’ original authors. Although Model 3 did show similar indices of fit to the observed data in comparison to Model 4, the better values on all fit statistics suggests that Model 4 was the superior of the two. Additionally a difference test was conducted finding both models to be significantly different. With model 4 also being more parsimonious, it was preferred. In contrast, neither Model 1 nor 2 were able to converge within the EQS. 6.0 analysis. Further investigation indicated that this was due to estimation within the PAI first order factor disrupting its second order variance.

Table 3 Summary of the Model Fit Indices for All Parenting CFA Models
Full size table

Given the superior performance of Model 4 found above, it was then examined separately for each GC age group with unacceptable model fit indices resulting for both the younger (χ2 = 933.39, df = 619; RMSEA = 0.055; CFI = 0.817; SRMR = 0.089) and older (χ2 = 1003.07, df = 619; RMSEA = 0.059; CFI = 0.801; SRMR = 0.083) age groups. To improve model fit, a series of appropriate modifications suggested by the Lagrange Multiplier tests were made separately to Model 4 for both age groups as depicted in Table 3. Lagrange Multiplier modification indices were used to identify items whose errors covaried with other items’ errors, or that cross-loaded on secondary factors. Regarding the former, suggested error covariances were allowed as supported by theory. As for the latter, as the goal of the measurement model was to obtain a relatively simple factor structure, severely cross-loading items were removed.

Specific changes to Model 4 identified for the younger group were the removal of two cross loading items and the addition of one error covariance parameter. Changes identified for the older group included the removal of one cross loading item and the addition of six error covariance parameters. The specifics of these modifications can be viewed in Table 3. After making these changes, model fit improved for the samples of both younger (χ2 = 769.02, df = 582, p < 0.01; RMSEA = 0.049; CFI = 0.86; SRMR = 0.08) and older GC (χ2 = 948.99, df = 578, p < 0.01; RMSEA = 0.051; CFI = 0.87; SRMR = 0.08). According to Hu and Bentler (1998), relying on RMSEA and SRMR as evidence of fit is adequate when the CFI is low due not to poor absolute (SRMR) and parsimonious (RMSEA) fit, but because of relatively low relations within the data.

After the establishment of the best fitting models for young and old age groups, modifications from each procedure were combined to create an overall model that was then tested for invariance across the two CG age groups. The invariance test compared the null model, in which no constraints were applied, with the alternative model in which the factor loadings of the two age groups were constrained to be equal. This test provided a non-significant scaled Chi-square (χ 2 = 39.35, df = 27, p < 0.05), supporting measurement invariance across these two groups. Additionally, all item factor loadings remained statistically significant and no additional modifications were suggested by the Lagrange.

As shown in Table 4, significant correlations were observed between the majority of the parenting practice measures from the five factor model that was used in the previous invariance test and the constructs representative of the Family Stress Model. Three of the five scales (Punitive, Effective, and PAI) correlated exactly as hypothesized with all indices of GC adjustment and CGM psychological distress at statically significant levels. The Consistency scale correlated significantly as expected with both indices of CGM distress, but only correlated with one indicator of CG adjustment (i.e., Externalizing Symptoms). Although the SEB scale correlated significantly with both indices of CG Adjustment, it did not correlate with either indicator of CGM distress. In sum, 16 (80 %) of the 20 hypothesized correlations were statistically significant.

Table 4 Correlations
Full size table

Discussion

The overall goal of this study was to examine whether or not five scales measuring nurturance and discipline originally developed for use with birth parents demonstrate adequate psychometric properties among a non-traditional parenting sample of CGM. We focused on five scales measuring particular facets of parental nurturance (Positive Affect and Supportive/Engaged Behavior, and discipline (Effective, Punitive, and Consistent discipline) because these two constructs are linked to children’s adjustment problems and are common targets of clinical intervention (Cummings et al. 1994; Locke and Prinz 2002; Lovejoy et al. 2000; Rubin and Burgess 2002).

Our first aim was to compare four nested CFA models in order to confirm the proposed factor structures for these five scales and to determine if their interrelations encompass the potential higher order factors depicted in Fig. 1. Of the four models tested, Model 4 yielded the best fit to the observed data. This finding, along with the acceptable internal consistency values observed for each scale, suggests that they are measuring distinct interrelated first-order constructs. Thus, it appears that each scale could be used by itself to assess the underlying construct it purports to measure. This is also suggested by the fact that shared variance between any two of the five subscales did not exceed 46 % as revealed by the squared zero-order correlations found across these five scales (Table 4).

It is important to consider, however, that the observed difference in fit between Models 3 and 4 is small albeit statistically significant. Unlike Model 4, Model 3 includes a second-order discipline factor along with supportive engagement (SEB) and positive affect (PAI) within the CGM-GC relationship as covarying first order factors. In contrast, Models 1 and 2 failed to converge, which may be due to the fact that both contain a second order nurturance factor. Put differently, only those models without a second order nurturance factor converged properly and demonstrated acceptable fit to the observed data.

The apparent inability of the SEB and PAI to form a second-order nurturance factor may be due to differences in the manifest content of the items in these respective scales. Specifically, all of the items in the SEB refer to actual parenting behaviors whereas the PAI contains a mix of behavioral and cognitive items. Another key difference is that the PAI also requires the respondent to infer the feelings of the other person within the dyadic relationship. It is reasonable to conclude then that Models 1 and 2 may have shown better fit if different scales indicative of parental nurturance had been used in the present study. In turn, this illustrates the need for future studies where parenting measures beyond those investigated here are examined for potential use with CGM. Because parenting is a multi-dimensional construct (Smith 2011), it is important to examine measures of other parenting dimensions (e.g., attitudes, styles, satisfaction, stress, competence/self-efficacy) and how they relate to one another in future studies with CGM.

It is also noteworthy that, unlike nurturance, the proposed second-order discipline factor appeared to be more stable within Models 1, 2, and 3. Although it is tempting to conclude from this that a broader discipline higher order construct might actually exist, it is important to note that all three of the discipline scales were taken from the PPI (Lochman and Conduct Problems Prevention Research Group 2010) which share the same response alternative format (Table 1). In contrast, the two nurturance scales not only had response alternatives that differed from the PPI but their response formats also differed from one another. Thus, the CFA outcomes observed for Models 1–4 may partially reflect these similarities and differences in response alternatives across the five scales examined here.

Our second aim was to determine if the best fitting model of the four examined via CFA would show measurement invariance by GC age. This was particularly important because the scales that we examined were from different sources and developed for use with parents of children from varying age groups. For example, the PAI has been used mostly with adult children and adolescents (Bengtson and Schrader 1982; Orsmond et al. 2006), whereas the PPI subscales (Punitive, Consistency, Effective) were initially tested on preschool age children (Strayhorn and Weidman 1988) and the SEB with students in a Head Start program (Webster-Stratton et al. 2001). Moreover, Locke and Prinz (2002) have asserted that “across all discipline and nurturance measures, better developmental mapping is needed” (p. 922).

Before performing the invariance test, we first examined Model 4 separately for the younger and older GC age groups. These analyses revealed only a handful of modest changes to Model 4 were required within each age group to yield acceptable fit. For younger GC, these changes involved the removal of two cross loading items and the addition of an error covariance term shared by the older child group. For the older group modifications included the removal of one cross loading item and adding six error covariances. In retrospect, these changes are conceptually sensible given that the suggested error covariance modifications all shared manifest content related to the same material (affectionate physical contact). The items identified as cross-loaders contained material that was related to both punitive and effective disciplinary measures thereby confounding these two constructs. In addition, the error covariances that added to improve model fit for both age groups do not alter the overall meaningfulness or interpretation of the involved subscales.

These changes were then combined into a final version of Model 4 in which measurement invariance was examined between the two GC age groups by constraining the loadings for each factor to be equal, and this model showed complete invariance by age. Not only were all factor loadings high and statistically significant for both age groups across all factors within the best fitting version of model (Model 4), there were also no differences in either the magnitude of these loadings or in the patterns of covariance observed between the five first order factors. These findings indicate that each of the five parenting measures examined in the present study can be used acceptably with CGM providing care to CG of ages 4–12 after making the modest changes described above. Specifically, item 7 from the consistency scale should be removed for use with CGM of both young GC, and item 5 from the punitive scale removed for use with CGM of younger GC. Future research should be conducted, however, to determine if these five scales (as well other parent measures) are appropriate for use with the CGM of very young children and adolescents.

Our final aim was to examine the concurrent validity of the five parenting scales identified by our final best fitting model. Specifically, from the conceptual framework of the FSM (Conger et al. 2002), we hypothesized that those scales measuring parental nurturance (SEB and PAI) and use of effective discipline would be significantly and inversely related to internal and externalizing symptoms of the CG as well as to indices of CGM psychological distress. In contrast, also based upon the FSM, we hypothesized that our measures of ineffective discipline (Punitive and Consistency) would show the exact opposite patterns. These hypotheses are also consistent with prior findings in the general parenting literature (Downey and Coyne1990; Shelton and Harold 2008) and with the rare parenting studies on CGM to date (Rodgers-Farmer 1999; Smith et al. 2008).

Almost without exception, our hypotheses were supported and a few noteworthy trends emerged as well. For example, among all five scales, the Effective scale demonstrated correlations of the highest magnitude with indices of GC adjustment and CGM psychological distress. As noted earlier, the item content of this scale is unique in focusing upon how well CG respond to parenting practices (e.g., “When you ask NAME to stop doing something, how often will s(he) stop?”), rather than upon parenting practices themselves. This is in contrast other measures of effective discipline that assess discipline practices deemed more effective such as use of clear rules and requests, direct reinforcement of appropriate behavior, application of reasoning and induction (Locke and Prinz 2002). Thus, it is sensible that correlations will be greater when CG outcomes are directly stated rather than being inferred as is true of the other four scales (e.g., Consistency, Punitive, SEB, PAI). In general, however, the measurement of effective discipline is thought to be knotty given that that there is no agreed-upon standard for discipline practices in terms of lesser or greater effectiveness; the concept itself embodies social desirability, and it is difficult to separate the effects of discipline from a larger constellation of parenting practices (Locke and Prinz 2002).

Another interesting trend in our validity findings is that the Punitive scale demonstrated across the board higher correlations with CG outcomes and CGM psychological distress than did Consistency scale. In fact, the correlation between Consistency and CG internalizing symptoms failed to reach statistical significance. Likewise, the PAI demonstrated across the board higher correlations with CG outcomes and CGM psychological distress than did the SEB with the latter not correlating significantly with CGM anxiety or depression scores. As a whole, this differential pattern of correlations reinforces our CFA findings which suggest that each of these scales is best viewed as a distinct factor in its own right.

Our concurrent validity findings are also in line with the results of earlier studies with CGM which found significant relations among their parenting practices, their personal distress, and CG adjustment (Rodgers-Farmer 1999; Smith et al. 2008). The fact that these prior studies used different measures of parenting practices than the ones investigated here lends credence to the collective generalizability of these studies. The present study, however, is unique in regards to its much fuller and more rigorous psychometric focus.

The present study is not without limitations. One drawback is that all of the measures used here, including our indices of concurrent validity, were self-reported by CGMs. In turn, our findings could have been much different if the five parenting practices had been assessed by other methods such as observations, interviews, rating scales or reports by other informants (e.g., GC; spouse). A chief advantage of caregiver self-reports of parenting is that they are more likely than others to have a comprehensive and wide-ranging knowledge of their parenting practices across differing contexts, whereas methods such as observations or querying independent informants do not provide this same breadth (Smith 2011). On the other hand, self-reports may be subject to biases such as social desirability and are necessarily retrospective in nature. Nevertheless, given that self-reports are generally less costly, easier to obtain; and can be sensitive to intervention change (Perepletchikova and Kazdin 2004), the present study is important in demonstrating the potential usefulness of the self-report scales we examined with CGMs.

Another limitation is that we did not consider other types of validity apart from concurrent validity. Because the data reported on here are entirely cross-sectional we are unable to comment on the predictive validity of these five scales in terms of whether they are associated with child outcomes over time. We also did not examine the incremental validity of these scales in terms of whether or not they contribute to the prediction of CG outcomes above and beyond other factors (e.g., socioeconomic disadvantage, family adversity, child abuse) associated with children’s adjustment (Perepletchikova and Kazdin 2004). Although these five scales are being used in our RCT we have yet to determine their sensitivity as measures of treatment change.

Another limitation worth noting is that our sample was restricted to CGMs who volunteered to enroll in a RCT and were recruited primarily by convenience. Thus, our sample may not be representative of CGMs on the whole. We also did not include grandfathers in the present study, which is regrettably typical of parenting research in general (Locke and Prinz 2002). However, our sample was from diverse regions across the US and included CGMs from the three ethnic/racial categories (Latinas, African-Americans, and Whites) that comprise the vast majority of the overall CGM population (2011 American Community Survey).

Despite the limitations of this study, it is the first attempt to comprehensively examine the psychometric properties of existing scales that assess the key parenting practices of nurturance and discipline which are known to be associated with child adjustment outcomes within a sample of CGM. Both our CFA and validity analyses provide preliminary evidence that each of the measures examined here can be used meaningfully with this target population either separately or in combination with only a handful of modest changes from their original design.