Introduction

Youth contribute to their individual development, their families’ well-being, and their communities’ civic fabrics. When young people are actively engaged in improving the well-being of their communities and their country, their own development is enhanced and civil society benefits (Lerner 2004; Levine and Youniss 2006; Zaff and Michelsen 2001). The question is whether the act of participation, that is, civic behavior, is sufficient to attain these individual and societal benefits, or whether a deeper, more substantive engagement is necessary. Are we as a society satisfied with volunteers who show-up for a community service activity because their friends attend but who are not otherwise connected to their community? Or, do we as a society aspire to have citizens who feel a sense of duty to give back to their community, feel a deeper connection to their community, and who believe they are competent to effect change in their community? In this study, we argue that a deeper engagement should be the aspiration for practice as well as the frame scholarly study should incorporate into research on civic engagement.

Larson (2000) proposed that initiative is one form of this deeper form of participation, what we would consider engagement, and consists of intrinsic motivation coupled with the drive to pursue a challenging goal. Consistent with Larson’s concept of initiative and consistent with youth development organizations, such as 4-H and the Boys and Girls Clubs, and universities, such as those classified by the Carnegie community engagement classification system, we hypothesized that active and engaged citizenship (AEC) is paramount and thus will maximize positive individual and societal outcomes. We consider AEC to be comprised of an integration of civic action, civic skills, social connection, and duty. If, as discussed below, AEC plays its theoretically expected role in youth development, it should be linked within and across time, to indicators of such outcomes. However, tests of such links require a valid measurement model of AEC. As such, a key focus of the present research is to provide such a test and, more specifically, ascertain if there is measurement invariance across grades and sex for the AEC model. To test the hypothesized link between AEC and positive outcomes, we must first develop valid measurements. The current study seeks to identify a measurement model of AEC and test the measurement invariance of this model.

Defining AEC

To define AEC, we reflect on the definition used by the Jonathan M. Tisch College of Citizenship and Public Service at Tufts University, which focuses on those who share a commitment to actively engage in their communities to build stronger, healthier, and safer communities. Thus, building from this definition, we operationalize an active and engaged citizen as someone who has a sense of civic duty, feeling of social connection to their community, confidence in their abilities to effect change, as well as someone who engages in civic behaviors. We derived this operationalization by drawing from the civic engagement literature, the facets that comprise civic engagement, and then theorizing that these factors, together, are potentially requisite for being an active and engaged citizen (see, for example, Flanagan and Faison 2001; Sherrod 2007; Sherrod and Lauckhardt 2009 who consider multiple, although not necessarily integrated, components of civic engagement, and Youniss 2006, for a discussion about integrating cognitive and behavioral components into developmental theories of political-civic development). In addition, this conceptualization is similar to those proposed by researchers of school engagement, who have used tripartite models that move past a sole reliance on behavioral measures and include cognitive and emotional factors (Fredericks et al. 2004).

This integrated civic construct is rooted in Erikson’s ego identity theory (involving a self-identity meeting both individual and societal needs; Erikson 1963; Marcia 1980) and German action theories (Baltes et al. 2006) that posit that cognitive and emotional processes and overt (and implicit) behaviors are inherently interconnected. For instance, German action theories express that adaptive development involves mutually beneficial relationships between the actions of the individual on the context and the actions of the context on the individual (e.g., incentives for or constraints on individual behavior; Brandtstädter 1998, 2006). Instead of assuming that a cognitive (e.g., sense of duty to give back to one’s community) or socioemotional factor (e.g., social connection) predicts a behavior (e.g., helping to fix a problem in the community), these constructs are part of the intra-individual developmental system, pieces that are inseparable in the developmental process. Action theorists, then, might hypothesize that civic engagement is expressed as a connection to one’s community, a commitment to improving that community, and the act of helping one’s community, consistent with the ideas of connection, duty, and behavior, respectively. Actions that enhance the community in this way will likely feed back to the individual, providing a context with the resources to support his/her positive development (Lerner 2004; Sherrod 2007).

In this article, we argue that there is a need for the development and use of an integrated civic construct that exemplifies that idea of active and engaged citizenship; that is, someone who has a sense of civic self-efficacy, responsibility and social connection to community, possesses appropriate skills, and engages in civic behaviors. Note that our conception of “active and engaged” does not necessarily refer to subjective or objective success in a civic task. For instance, youth who raise awareness about and seek to improve living conditions for homeless children might not be successful in changing or enacting policies or programs. The absence of productive outcomes of action does not mean that the individual is an ineffective citizen. In fact, Theiss-Morse and Hibbing (2004) suggest that failure in civic actions can be a productive element of citizenship development.

In this article, we build off of an initial study we undertook to identify an integrated construct of civic engagement (Bobek et al. 2009). In that study, we identified six first-order factors (civic duty, civic skills, civic participation, adult social connection, peer social connection, and neighborhood social connection) that loaded on a second-order factor of an integrated construct of civic engagement. Our goal for that initial study was to start an empirically based dialogue about whether there is a need to have integrated civic engagement measures. The goal of the current study is to refine the initial measurement model and test the factor structure and validity of the refined model across three years of adolescence and for boys and girls.

We began our conception of AEC with a critical lens on our initial theoretical measurement model (Bobek et al. 2009). We assessed whether social connections to peers and social connections to adults are necessary components of an integrated civic engagement construct or whether these are important predictors, but not components, of AEC. Developmentally, we would expect that connections with adults would be a core predictor of later connection to the broader community. For instance, as one’s sense of the world becomes more elaborated, the concept of social connection would move from adults and peers, and possibly the school, to a connection with the broader community (Ford and Lerner 1992). A sense of connection to the family has been considered a requisite to contributing to the family in positive ways (Grusec et al. 1996), which is consistent with community connection being related to engaging in and contributing to the broader community (Chavis and Wandersman 1990). Community is admittedly an ambiguous term, with the meaning often defined by the individual or by collective social norms (Burton and Jarrett 2000; Chavis and Wandersman 1990). The community could be as proximal as an individual’s neighborhood or as distal as one’s ethnic/racial or religious group, an entire nation, or even the entire world. Using data from the 4-H Study of Positive Youth Development, we are constrained somewhat by the civic questions relating to an understanding of community that is more proximal. Thus, for the testing of our theorized measurement model, we consider community social connection to be related to an individual’s surrounding neighborhood, thus retaining, based on our theory, the neighborhood social connection factor as an integral part of the AEC construct.

In addition to the conceptualization of community, we recognized that measures of civic behaviors can vary within and across individuals and contexts, such that behaviors that are considered civic in one context might not be considered civic in a different context. For example, youth confronting their school principal about a controversial issue in some districts might be considered a conduct problem, whereas it might be considered an appropriate form of civic expression in others (see Ginwright 2007, for a discussion of culturally relative forms of civic engagement). Considering the measures that are available in the data set used in this study, we do not purport to estimate AEC for all individuals living in all contexts. Instead, we seek to demonstrate the utility of our theoretical model to capture the multiple components of AEC. We also tested for measurement invariance across the two sex groups to ensure that the observed assessment of AEC was not biased by sex and therefore could be used to assess both sexes. Although we recognize the need to test for measurement invariance across other sub-groups, such as by race/ethnicity and income, we were not able to make such comparisons in this study, due to sample size and measurement issues. In addition, to ensure that the assessment was not biased by different ages, we assessed longitudinal measurement invariance.

Hypotheses

We tested two hypotheses for this study using data from the 8th, 9th, and 10th grade waves of the 4-H Study of PYD, a national, longitudinal study assessing the links between youth development and individual and ecological resources for positive, healthy development (Gestsdottir and Lerner 2007; Jelicic et al. 2007; Lerner et al. 2005; Phelps et al. 2009). Our hypotheses were as follows. First, a valid measurement model for AEC for a cohort of youth across Grades 8, 9, and 10 could be identified. Second, there would be measurement invariance for boys and girls and longitudinally. We note that we did not compare the previous model (Bobek et al. 2009) to AEC, because, as described above, we do not believe that the previous theoretical construct is refined and, therefore, we do not believe that a comparison is an empirical issue useful to elucidate.

Method

The current study is part of the larger 4-H Study of Positive Youth Development (4-H Study of PYD), a longitudinal investigation of youth in the United States, beginning when participants were Grade 5 and, at this writing, assessing youth in Grade 12 (Lerner et al. 2005; Phelps et al. 2009). The 4-H Study tests a theoretical model regarding the promotion of positive youth development (PYD) as defined by the Five Cs (competence, confidence, character, connection, and caring) and the emergence of a “sixth C” of contribution. More information about the research can be found in Lerner et al. (2005), Gestsdóttir and Lerner (2007), Jelicic et al. (2007), and Phelps et al. (2009). In this section, we present the methodology pertinent to the current investigation.

Design

The 4-H Study uses a form of longitudinal sequential design (Baltes et al. 1977), which involves following an initial sample of participants over time while also adding “retest control” cohorts at each time of testing. Fifth graders were initially sampled during the 2002–2003 school year and, at subsequent times of testing, a retest control group of youth was added at the initial cohort’s current grade level. Participants in 4-H programs were also added at each time of testing. For a full description of the study design, refer to Jelicic et al. (2007).

Participants

The sample for the current study consists of the Grade 8, 9, and 10 waves of data. During Grade 8, data were collected from a total of 1,990 youth. Youth were from 35 schools and 12 4-H sites located in a total of 17 states. Data were collected from 1,203 youth from 40 schools and 24 4-H sites in a total of 18 states during the 9th grade. During the 10th grade, data were collected from a total of 2,371 youth from 18 schools and 82 4-H sites in a total of 32 states.

The longitudinal sample, comprised of individuals who participated in at least two waves of data collection for Grades 8 through 10, consisted of 909 individuals. Within the sample used in this report, 27.5% participated in all waves, 34.0% participated in Grades 8 and 9, 19.9% participated in Grades 9 and 10, and 18.6% participated in Grades 8 and 10. Overall, the sample was 62.4% female (mean age at Grade 8 = 14.17 years, SD = 1.05; mean age at Grade 9 = 14.97 years, SD = 1.06; and mean age at Grade 10 = 16.03 years, SD = .97). The mean age for males was 14.22 years (SD = .87) at Grade 8, 15.16 years (SD = .99) at Grade 9, and 16.08 years (SD = .92) at Grade 10. More than two-thirds of the youth reported that they were European American, with fewer than 10% reporting that they were African American and a similar percentage saying they were Hispanic/Latino. Small numbers of youth reported either being Native American or Asian American. Table 1 summarizes the demographic characteristics of the sample.

Table 1 Demographic characteristics of participants (n = 909)
Full size table

Attrition

Attrition in the 4-H sample is not randomly distributed across schools. In Grades 6, 7, 8, and 9, some principals withdrew consent for their schools to participate and, thus, these students were “lost” to the study without their being asked whether they wanted to remain in the study. Overall, 561 participants were lost in Grade 6 because of the absence of principal or superintendent permission to continue. However, attrition from Grade 5 to Grade 6 for students who were given the opportunity to remain in the study was only 10%. Out of 1,954 participants tested in Grade 6, 337 participants (17.5%) were lost because of school/site attrition in Grade 7, but there was also 21.5% individual attrition. During Grades 8 and 9, many of the same schools did not allow for on-site data collection. Youth in these schools were contacted through mail or phone and were asked to complete the survey and mail it back to the investigators or to complete the survey online.

At each grade, the research protocol is to contact all adolescents who have ever participated in the study, such that many youth who were not surveyed in earlier waves return to the study in later waves. For Grade 9, 66.5% (n = 801) of the 1,203 youth assessed in Grade 9 participated in at least one prior wave. Approximately 34.0% (n = 797) of adolescents tested in Grade 10 (n = 2371) had participated in at least one prior wave. As previously mentioned, at each wave of data collection an “attrition replacement” cohort was added to the sample to maintain sample size and to control for the effects of repeated testing. For example, for Grade 6, the original cohort of 5th graders (who were now in the 6th grade) was retested, and a new cohort of 6th graders was added to the sample. During Grades 8 and 9, the attrition replacement cohorts included expanded recruitment of youth from 4-H clubs around the country. The ethnic distributions of the attrition replacement cohorts were equivalent to the ethnic distribution of the first cohort. Because data for the attrition replacement participants were missing by design (i.e., the data had not been included in the study at those time points prior to their entry into the study), these data could be treated as “missing by design” (Farrington 1991). Making assumptions about missing data mechanisms for individual variables or wave nonresponse as well as designed missingness is challenging in multivariate longitudinal studies. The current literature does not provide guidelines for handling complex patterns of missingness. In the current study, we assume that data were missing at random (MAR).

Measures

To estimate a measurement model of AEC across three waves of data (Grades 8–10), we began with the set of items that comprised the previously identified measure and then conducted confirmatory factor analysis. A complete description of how the items were initially chosen is available in Bobek et al. (2009). In brief, graduate student and “civic expert” panels rated sets of measures from the 4-H Study for their relevance to civic engagement. Those items that elicited high rates of engagement by the panels were supplemented by several other items that extant research suggested were also relevant to civic engagement. An exploratory factor analysis was then conducted to determine which items should be retained, resulting in 42 items and six factors. These forty-two items loaded onto six factors, which all showed acceptable reliability (civic participation, civic duty, civic skills, neighborhood social connection, peer social connection, and adult social connection) and these six first-order latent factors loaded on a second-order latent factor depicting an integrated civic engagement construct.

As discussed in the introduction, we refined the measurement model based on theory and, subsequently, removed the adult social connection and peer social connection factors, as well as an item related to participation in school government because it did not load well onto the civic participation factor. These steps resulted in a total of 32 items across four factors that we included in this study’s analysis. The response options for the items were measured on a variety of scales. Thus, for consistency, all of the items were rescaled so that their values ranged from 0 to 25. It is useful to describe the measures that were included in each of the first-order factors.

Civic Duty

The civic duty factor is comprised of 12 items drawn from the social responsibility scale (SRS; Greenberger and Bond, 1984) of the teen assessment project (TAP) survey question bank (Small and Rodgers, 1995); the political efficacy and participatory citizen constructs of the student voices measure (Flanagan et al. 2007a, b; and the Search Institute’s Profiles of Student Life-Attitudes and Beliefs (PSL-AB) questionnaire (Leffert et al. 1998). A sample item states, “How much do you agree or disagree with the following? I often think about doing things so that people in the future can have things better,” with response options ranging on a 5-point Likert-type scale (strongly disagree to strongly agree). A second sample item has respondents indicate the extent to which they agree or disagree to the following statement, “I believe I can make a difference in my community,” with response options ranging from strongly disagree to strongly agree on a 5-point Likert-type scale. Bobek et al. (2009) reported a Cronbach’s alpha of .80 for this factor, a value that is consistent with the alphas for the present study sample, which are shown in Table 2.

Table 2 Means, standard deviations, and cronbach’s alpha for the first-order factors and second-order factors by grade
Full size table

Civic Skills

The civic skills factor is comprised of six items adapted from the political voice and competence for civic action constructs of the student voices measure (Flanagan et al. 2007a, b). Sample items ask respondents to indicate the extent to which they can “write an opinion letter to a local newspaper” and “sign an e-mail or written petition,” with response options ranging from I definitely can’t to I definitely can on a 5-point Likert-type scale. Bobek et al. (2009) reported an alpha value of .91 for this factor. The Cronbach’s alphas for the present study sample are shown in Table 2. (It should be noted that civic skills represent abilities to enact behaviors. However, motivation and opportunity are also needed to enact these behaviors.)

Neighborhood Social Connection

The neighborhood social connection factor is comprised of 6 items taken from the Search Institute’s PSL-AB questionnaire (Leffert et al. 1998). Representative example items have respondents indicate the extent to which they agree or disagree to the following statements: “In my town or city, I feel like I matter to people” and “Adults in my town or city listen to what I have to say,” with response options ranging from strongly disagree to strongly agree on a 5-point Likert-type scale. Bobek et al. (2009) reported an alpha value of .86 for this factor. The Cronbach’s alphas for the present study sample are shown in Table 2.

Civic Participation

The civic participation factor is comprised of eight items created specifically for the 4-H Study and are drawn from the Search Institute’s PSL-AB questionnaire (Leffert et al. 1998). A representative item asks youth how often they “help make your city or town a better place for people to live,” with response options ranging from never to very often on a 5-point Likert-type scale. Other representative items ask participants to indicate how often they participate in specific service activities, such as volunteering and mentoring/peer advising, with responses ranging from never to every day on a 6-point Likert-type scale. An additional representative item asks, “During the last 12 months, how many times have you been a leader in a group or organization” with response options ranging from never to 5 or more times. The Cronbach’s alphas for the present study sample are shown in Table 2. Table 3 presents a list of the 32 items involved in the four factors associated with active and engaged citizenship.

Table 3 Items used in the four-factor active and engaged citizenship model
Full size table

In addition to the above-noted factors, we used demographic information from the student and parent questionnaires. We gathered information regarding participants’ sex, race/ethnicity, and grade level from the student questionnaire. Household income and maternal education were gathered from the parent questionnaire.

Procedure

For the first three waves of data collection (Grades 5–7), teachers or program staff gave each child an envelope to take home to his/her parent or guardian. The envelope contained a letter explaining the study, two consent forms (one that was returned to the school and one that could be kept for the records of the parent or guardian), a parent questionnaire, and a self-addressed, stamped manila envelope for returning the parent questionnaire and consent form. Trained study staff or assistants hired at more distant locations conducted data collection. Data collectors used a detailed protocol to ensure that data collection was administered uniformly and to ensure the return of all study materials. The procedure began with reading the instructions for the student questionnaire to the youth. We informed participants that they could skip any questions they did not wish to answer. There was a 2-h block of time allotted for data collection, which included one or two short rest periods. For Grades 6 and 7, students who were unable to be surveyed at their school or 4-H site, in that they were either absent during the day of testing or the school principal or superintendent did not allow testing to occur in the school, received a survey in the mail.

For Grades 8, 9, and 10 youth were surveyed in their schools or youth programs following the same procedure as in the first three 4-H Study waves (Grades 5–7). We contacted youth who were absent on the day of the survey or were from schools who did not allow on-site testing by e-mail, mail, or phone, and asked them to complete and return the survey to us. Beginning in Grade 9, youth could go online to complete the survey. Youth tested at 4-H clubs were either tested with the paper survey or used club computers to complete the survey online. During Grades 8, 9, and 10, we continued to contact all youth who were part of the first three waves (Grades 5–7) and, in addition, we increased the sample by expanding our recruitment of youth in 4-H clubs around the country. For new participants, parents were also asked to give consent and to complete the parent survey.

Results

The aim of this research was to test and refine a higher-order measurement model of active and engaged citizenship (AEC) among youth, and to examine the measurement invariance of this model. Specifically, the analyses were aimed at answering two key questions. First, does the proposed second-order model of AEC that encompasses four first-order latent constructs (i.e., Civic Duty, Civic Skills, Neighborhood Social Connection, and Civic Participation) fit the data? Second, if yes, can the proposed AEC model be measured equivalently for boys and girls and over time? We conducted a series of second-order confirmatory factor analyses to answer these two questions.

Missing Data and Preliminary Analysis

It was assumed that missing data were missing at random (Jelicic et al. 2009), and thus we performed multiple imputations using Expected Maximization (EM) procedures in LISREL 8.80 to replace missing data. The subsequent analyses were conducted with the complete data set.

Several preliminary analyses were performed using PRELIS, a companion package of LISREL, to examine and treat item-response missing data, to report the univariate statistics including the mean, standard deviation, skewness and kurtosis, and to test the extent to which the data met the assumption of multivariate normality. While the data are technically ordinal, they were treated as if they were continuous.

As expected, preliminary analyses revealed some evidence of nonnormality on most of the 32 items. In addition, multivariate kurtosis, as reflected by normalized Mardia coefficient of 51.49, indicated multivariate nonnormality. In light of evidence of nonnormality and kurtosis, assessment of statistical fit was based on the Satorra-Bentler scaled chi-square values (SB χ 2; Satorra and Bentler 1988), instead of on the uncorrected chi-square statistic. The SB χ 2 has been recognized as the most reliable test statistic for evaluating means and covariance models under nonnormal distributions and various sample sizes (Byrne and Stewart 2004; Curran et al. 1995).

Tests of the Hypothesized Model

The postulated model of the AEC factorial structure is presented in Fig. 1. As shown in Fig. 1, the postulated model of AEC is a higher-order model of civic engagement, consisting of one second-order factor (AEC), four first-order factors (civic duty, civic skills, neighborhood social connection, and civic participation), and 32 manifest indicators. Each indicator was constrained to load on the first-order factor that it was designed to measure, and not on any of the other three first-order factors. Covariance among the four first-order factors was hypothesized to be explained fully by their regression onto the second-order factor. In the first stage of confirmatory factor analysis, we tested the hypothesized second-order model of civic engagement for Grade 8. The model exhibited adequate fit: χ 2 = 4522.96, df = 460, p < .001, RMSEA = 0.066. Most modification indices were relatively low. All but one of the first-order factor loadings were above 0.3, with about 80% of the loadings above 0.5. All the second-order factor loadings were above 0.5 (see Fig. 1). The means and standard deviations of the first- and second-order factors are displayed in Table 2.

Fig. 1
figure 1

Measurement model for the second-order factor model of active and engaged citizenship (AEC) among Grade 8 youth

Full size image

Model Comparison and Fit Indices

Maximum likelihood estimation procedures were used with Satorra and Bentler’s (1994) scaling corrections, allowing for the calculation of the Satorra-Bentler scaled chi-square (SB χ 2) values. Overall and comparative fit indices were used in evaluating and comparing model fit. In addition to SB χ 2, comparative fit index (CFI) and root mean squared error of approximation (RMSEA) were also used as overall fit indices. Due to the complexity of our measurement model, it is not reasonable to expect a CFI to attain the 0.95 cutoff value conventionally sought. Thus, we regard values equal or greater than 0.90 for the CFI as indicative of a good fit (Hu and Bentler, 1999). For nonnormal conditions an adjusted RMSEA value of 0.08 is used as the threshold for determining whether or not to reject the model (Byrne 1998). In addition to overall fit indices, as recommended by Cheung and Rensvold (2000), we used change in CFI between two nested models (ΔCFI) to statistically evaluate the difference between two nested models. A value of ΔCFI smaller than or equal to 0.01 indicates a nonsignificant difference between the two models. Change in CFI was chosen over change in SB χ 2 because CFI is not as strongly affected by sample size and number of constraints.

Cross-Sex Measurement Invariance: Multi-Group Analysis

As a second step in the data analytic process, invariance within time was addressed in the Grade 8 cross-sectional examination of the measure. This procedure explicitly tested the acceptability of the measurement of the AEC construct within Grade 8 before proceeding to an evaluation of longitudinal factorial invariance (Feldt et al. 2000). We tested measurement invariance in multiple-group second-order confirmatory factor models using the full mean and covariance structures at five levels: configural, metric (factor loading), intercept, residual variance, and disturbance (Chen et al. 2005; Meredith 1993; Widaman and Reise 1997). Boys and girls were treated as independent samples.

We examined a series of hierarchically nested models that were designed to assess the invariance of our measure of AEC between boys and girls, using Grade 8 cross-sectional data. As recommended by Chen et al. (2005), configural invariance was tested first (Model 1), because it is a necessary condition for the evaluation of further aspects of measurement invariance. Invariance of first-order and second-order factor loadings (Models 2 and 3) was assessed after configural invariance was established. Similarly, tests for invariance of intercepts of observed variables (Model 4) and first-order latent factors should be conducted only if invariance at the metric level is obtained. If measurement invariance at the configural or metric level is not demonstrated, further tests, such as tests for equality of residuals of like items (i.e., item uniqueness) are unwarranted (Vandenberg and Lance, 2000).

Configural Invariance (Model 1)

In testing for this form of invariance, an unrestricted baseline model was specified in which boys and girls had the same structure of AEC. That is, the pattern of fixed and free factor loadings for the first- and second-order factor was specified to be the same for both groups, but different estimates were allowed for the corresponding parameters in the different groups. As shown in Table 4 the SB χ 2 statistic was 5393.39, df = 912, p < .001, RMSEA = 0.073, and CFI = 0.95. The results suggest an adequate fit of the model to the data.

Table 4 Summary of fit statistics for testing measurement invariance of second-order factor model of civic engagement between boys and girls
Full size table

Invariance of First-Order Factor Loadings (Model 2)

In testing for this level of measurement invariance, all of the first-order factor loadings for like items were constrained to be equal for boys and girls. From Model 1 to Model 2, the SB chi-square significantly decreased in value: SB χ 2 = 5343.04, df = 948, RMSEA = 0.071, CFI = 0.95. These results indicated that the first-order factor loadings were invariant across boys and girls.

Invariance of Second-Order Factor Loadings (Model 3)

In testing for this level of invariance, all first- and second-order factor loadings were constrained to be equal across groups. The SB chi-square was 5347.13, df = 951, RMSEA = 0.071, CFI = 0.95. The change in the SB chi-square and CFI from Model 2 were both non-significant, indicating that the second-order factor loadings were invariant between boys and girls.

Invariance of Intercepts of Observed Variables (Model 4)

Model 4 imposed additional constraints to determine whether the intercepts of the observed variables were invariant. In addition to constraining the first- and second-order factor loadings to be equal, we imposed more constraints on the intercepts of the 32 observed variables. The change in the SB χ 2 from Model 3 to Model 4 was significant. There was also a substantial difference in the CFI values (0.02). Thus, we concluded that there was appreciable difference between boys and girls on the intercepts of the observed variables. No further tests for invariance of intercepts of first-order latent factors, disturbance of first-order factors, and residuals of observed variables were pursued. These results indicated that while the second-order factor structure held both for boys and girls and the regression slopes linking the observed variables to the underlying construct of AEC are invariant between sexes, their means should not be compared.

Longitudinal Measurement Invariance

We then tested the AEC measurement model of the AEC construct to see if the second-order factor is invariant over time. The analysis was conducted by imputing a single augmented covariance matrix (Vandenberg and Lance 2000). The single augmented covariance matrix allowed us to merge Pearson’s covariance matrices for each time point into a single augmented variance–covariance matrix. Undertaking this computation allows us to take full advantage of the longitudinal nature of the data, address the interdependency among like variables measured at different time points, and treat as well as the within-time covariances (Vandenberg and Lance 2000).

A well-fitting configural invariance model suggests that a similar measurement model is plausible across Grades 8–10 and that additional measurement invariance tests may be ensued. We used the longitudinal sample described in the method section for this analysis. We first tested the postulated model at Grades 8, 9, and 10 to evaluate the goodness-of-fit of the hypothesized model to the data. First, the baseline model for configural invariance was acceptable because of its satisfactory fit indices values (CFI = 0.93, RMSEA = 0.071), although the χ 2 test is significant. We then estimated a longitudinal second-order factor model where only the variances of the errors of the observed variables were allowed to vary. This model resulted in an inadequate fit with the data, χ 2 (4449) = 36703.73, CFI = 0.91, RMSEA = 0.089).

Next, to account for method variance associated with each observed variable, 96 autocovariances among the errors of the same observed variables of the three measurement occasions were allowed to correlate, (Pitts et al. 1996). These estimates explicitly account for the unique aspect of each measured variable that is correlated over time, χ 2 (4353) = 24135.36, CFI = 0.93, RMSEA = 0.071. The improvement of the model with the estimated autocovariances (compared to its predecessor) was statistically significant (χ 2decreased by 12568.37 and CFI increased by 0.02), resulting in an adequate fit. Therefore, this model was chosen as the first model for testing measurement invariance (Model A, see Table 5).

Table 5 Summary of fit statistics for testing measurement invariance of second-order factor model of civic engagement over time
Full size table

In order to evaluate whether the structure of the second-order model is invariant over time, we set the factor loadings of the first-order factors across the three measurement occasions to be equal. This model (Model B) resulted in a significant decrease in chi square, and no change in CFI, suggesting that this model fit at least equally well, compared with Model A (see Table 5). We then next set the factor loadings of like second-order factors to be equal. This model (Model C) fit the data as well as the previous model, although producing a significant loss of fit in terms of chi square, χ 2 (4413) = 27662.00, CFI = 0.93, RMSEA = 0.076.

Subsequently, equality of intercepts across time is imposed on the model to test for strong invariance. The strong invariance model (Model D) also has satisfactory values on fit indices: χ 2 (4476) = 26501.69, CFI = 0.93, RMSEA = 0.074). Finally, equality of item uniqueness across time is further imposed to test strict invariance. The strict invariance model also has acceptable fit indices values (Model E). The residual variances of second-order factors were fixed in the next model, in addition to the fixed factor loadings, intercepts, and disturbances (Model F). Both models E and F exhibited a good fit to our data (see Table 5). The results provide evidence of the strictest form of factorial invariance of the AEC measure across the three measurement occasions. Thus, in the present study, the AEC measure demonstrated factorial invariance longitudinally (see Fig. 2).

Fig. 2
figure 2

Results of the second-order factor model for active and engaged citizenship (AEC) across measurement occasions: Standardized solution. Note: First-order factor loadings and correlated residuals of like items over time were not presented due to space constraints. They are available upon request

Full size image

Internal Consistency, Convergent, and Discriminant Validity

The composite reliability of each of the four factors and the overall higher-order factor underlying the AEC measure are presented in Table 2. The results suggest that the internal consistency of the four first-order factors and the second-order factor were all greater than .70. Convergent validity can be assessed by in-depth examination of the individual estimated parameters (Anderson and Gerbing 1988). The findings of the estimated second-order CFA model indicated that all factor loadings as well as structural regressions were all statistically significant with t-values ranging from 2.12 to 75.24. These results provide direct evidence of convergent validity of the AEC measure. The variance extracted for most of the factors was greater than the generally accepted value of .50, supporting the discriminant validity of the measure (Fornell and Larcker 1981).

Discussion

The study of youth civic engagement has important theoretical relevance and applied significance. Civic engagement is a core outcome of current theories of positive youth development (Lerner et al. 2009; Lerner 2004). In addition, understanding and enhancing youth engagement in civil societies is a critical facet in programs and policies aimed at maintaining and enhancing democracy (Sherrod and Lauckhardt 2009). Both tests of theory and intervention efforts rest on possession of an adequate means to measure the status of, and changes in, youth civic engagement, Accordingly, in this study, we pursued two primary purposes. First, we aimed to evaluate the goodness-of-fit between a hypothetical model of AEC and empirical data. Second, we aimed to test this model for measurement invariance across time and between boys and girls. In regard to the first goal, we used confirmatory factor analysis to identify four first-order factors of civic engagement (civic participation, civic duty, civic skills, and neighborhood social connection), which comprised a second-order factor of AEC. The Cronbach’s alphas were also sufficiently high for each of the individual factors and for the second-order factor.

Although we did not make an explicit comparison between the AEC model and the six-factor model that we identified in a previous study (Bobek et al. 2009), we believe that the current model provides a more parsimonious and more theoretically refined measurement model. That is, our previous model included social connection to adults, peers, and the broader community. However, we believe that focusing on the community is more theoretically sound, since feeling a connection to community appears to be a requisite for engaging in community affairs (Chavis and Wandersman 1990); whereas, connection to adults and peers have been found to be predictors of future engagement, not requisite components of current engagement (Zaff et al. 2008). In addition and similar to our previous study, our model of AEC provides a more nuanced understanding of civic engagement, going beyond the typical measures of civic behavior by including measures of emotional, psychological, and cognitive components.

For the second goal of this article, testing for measurement invariance, we found that the factor structure holds across time, thus enabling researchers to examine change, such as growth or trajectories of engagement. We also assessed measurement invariance for boys and girls. Given the civic gap that has been found based on sex and race/ethnicity (Flanagan et al. 2009; Marcelo et al. 2007) and the suggestion that some measures of civic engagement are not relevant to all racial/ethnic and income groups (Ginwright 2007), adequately and validly measuring across groups has important implications for understanding the intricacies of the civic gap and, subsequently, for ways to address the gap. We found that the structure of our measure of AEC holds across sex, thus allowing for comparisons across these groups on the first-order and second-order factors. Interestingly, invariance of the AEC measure at the scalar level was not obtained. In other words, although the measure is a valid representation of the construct for both boys and girls, across-sex comparison of the mean levels of the AEC should not be made. There may be many reasons for the lack of invariance, for example, noninvariance can be linked to problems of bias, which could be related to the intrinsic properties of the AEC measure, to some method bias such as instrument bias, or to the characteristics of the participants of different genders (Byrne and Watkins 2003). One possible instrument bias may be that boys may consistently select some scale points (high) while girls tend to select other scale points (low) and such selection is independent of the item content. We would recommend that future research on integrated constructs of civic engagement estimate the measurement models and test for measurement invariance across income groups and racial/ethnic groups.

Two other limitations, involving the need for more theory-predicated measures specifically designed to index youth civic engagement and the need for such measures to be developed in ways that are sensitive to racial, ethnic, religious, and cultural diversity, should be mentioned. First, then, we inevitably do not have all of the necessary measures to support our theoretical constructs across all contexts and for all types of civic actions. The 4-H Study was not originally designed for the purpose of constructing a measure of AEC. Thus, there are other measures that could more precisely assess the factors that comprise this second-order factor. For instance, the civic measures in the 4-H Study are focused primarily on community service (volunteering), but other forms of AEC, such as advocacy and working within the political system, are important aspects of being civically engaged. Such civic actions serve important functions in a democracy and therefore should be included in future tests of this measurement model. Including these additional measures in future studies, or using existing data sets that include such measures, would do much to expand our understanding of this integrated construct of civic engagement.

In turn, second, ensuring that such measures are culturally relevant and sensitive is also essential to accurately portraying the civic engagement of various groups, whether defined by race/ethnicity, income, or geographic residence, among others. The initial choice of the measures for the scales was based on the “expert ratings” of a non-representative group of raters; more specifically, a group of graduate students in child development at one university and a group of national civic engagement experts. If youth were asked to choose the most relevant measures or raters were more racially/ethnically diverse, different questions could have hypothetically been chosen. In sum, future research on a construct of AEC should include a broader range of measures to assess each of the four first-order factors, such as political engagement and the perspectives of a diverse group of researchers and youth should be authentically captured in choosing these measures.

Future studies on integrated civic engagement constructs should also examine the factors with older youth and young adults, as well as assess the utility of the measurement model with an even more diverse sample that includes sub-samples of racial/ethnic minorities, samples of which were too small to test for measurement invariance in this study. As well, diverse income groups should be studied. Future research projects could also use this integrated measure of civic engagement to assess change, examine the ecological and internal assets that promote the development of AEC, and examine the outcomes that result from being an active and engaged citizen. This research could include beginning to understand how various contexts, whether high-income versus low-income, socially disorganized versus socially calm, or civically promoting vs. civically inhibiting, among others, affect the development of both the four first-order factors and the second-order factor. Results from such projects would have important implications for practitioners and policymakers who want to understand how best to support the civic development of youth and who are concerned with ensuring that this country has an engaged citizenry. Moreover, because of the likely applied importance of this research, and thus its potentially substantive impact on youth and civil society, researchers should consider more nuanced ways to use this measure to enhance the understanding of youth civic engagement. For instance, cluster analysis could be used with the first order factors to illustrate how different patterns of AEC reside within individuals; For instance, some youth could score high on civic behaviors and civic skills, but score low on civic duty and social connection. This type of assessment would provide the person-centered insights that program staff could use to best support the development of civic behaviors, attitudes, and competencies of individual children. In turn, the present model of AEC provides evidence that practitioners should attend to cognitive, emotional, psychological, and behavioral aspects of civic engagement. By having a better understanding about how each of these factors function and how these factors aggregate together into a robust indicator of youth engagement, practitioners can improve the support they provide to encourage the development of AEC. In short, continuing integration of the interests of researchers and practitioners and policy makers around the measurement and meaning of youth civic engagement will be a valuable means to enhance both PYD theory and its use as a force for enhancing civil society.

Indeed, if such research-application collaborations involving youth civic engagement are a more ubiquitous feature of the developmental science of adolescence, then at least two broader benefits will occur. Science will be enhanced as the results of theory-based research are tested in the crucible of actual youth-serving programs. Second, practitioners and policy makers will be able to take evidence-based actions to enhance both youth civic engagement and the role of young people in furthering civil society and democracy.

In sum, the present research underscores the importance for both theory and application of conducting change-sensitive research to document the developmental course of civic engagement among adolescents and to derive measures that reflect the multivariate character of such development, its structure over the course of significant portions of the second decade of life, and its manifestation among boys and girls. If the use of such differentiated measures becomes a more prominent feature of research-application collaborations involving youth civic engagement, then at least two broader benefits will occur. Science will be enhanced as the results of theory-based research are tested in the crucible of actual youth-serving programs. Second, practitioners and policy makers will be able to take more nuanced, evidence-based actions to enhance both youth civic engagement and the role of young people in furthering civil society and democracy. Through such research and application outcomes, the study of the development of youth civic engagement may serve as an exemplar of the contributions of developmental science to both understanding and optimizing the lives of diverse young people.