Introduction

It is generally accepted by researchers that religious involvement has many dimensions that incorporate aspects such as belief in God or a transcendent being, membership in religious communities, distinctive religious practices or lifestyle, and religious values (Hall et al. 2008; Hill and Pargament 2003; Hoff et al. 2008). Religious involvement may be expressed by attendance at religious services, identification with a religious community, reading a sacred text (e.g., Torah, Koran, Bible), and commitment to distinctive beliefs and behaviors (Levin 2009). Spirituality is a related construct that tends to express the highly personalized and individualized engagement with transcendence or sacred matters affecting the spirit (Hall et al. 2008; Koenig 2008). Spirituality, “intrinsic” religion, “extrinsic” religion, and related terms have been defined in overlapping ways by different investigators over the years (Johnstone et al. 2009), and more research is needed to confirm the distinctiveness of these constructs, especially for minority populations.

Religious involvement is relatively high in the United States. Recent surveys indicate that more than 83% of adults in the US population self-identify with a religion (The Pew Forum on Religion and Public Life 2008) and that 42% of the US population reports attending religious services weekly or almost weekly (Gallup 2009). Religious involvement has been found to be particularly high among African Americans (Levin et al. 1994; Taylor et al. 1996), and the African American church has historically played a central role in the community beyond worship services and faith-based activities (Lincoln and Mamiya 1990).

Religion and Health Research

There has been a growing research interest in the role of religious involvement in preserving or improving one’s physical health. However, many early studies examined these relationships in an opportunistic fashion, assessing religious involvement using single items or indicators such as church attendance (Comstock and Partridge 1972; Levin and Vanderpool 1987; Strawbridge et al. 1997) or denominational affiliation (Benjamins et al. 2003; Hill et al. 2006; Oman et al. 2002). Associations have been found between religious attendance and longer life (McCullough et al. 2000) and lower disability (Idler and Kasl 1997), and more sophisticated measures of religious involvement beyond simple measures of church attendance might further distinguish protective effects of religious involvement versus the impact of poor health on one’s ability to attend religious services.

Assessment of Religious Involvement

As research has developed, measurement has become more sophisticated, with multidimensional measures and models of religious involvement now being the norm (Hill and Hood 1999). Among the notable multidimensional measures and models of religious involvement are the Multidimensional Measure of Religiousness and Spirituality (MMRS) and its short form the Brief MMRS (BMMRS) (Fetzer/NIA 1999; Johnstone and Yoon 2009; Johnstone et al. 2009), the Systems of Belief Inventory Revised (SBI-15R) (Holland, et al. 1998), and the Duke Religious Index (DRI) (Koenig et al. 1997). A strength of the MMRS is that it was validated using population-based data (Hall et al. 2008). The MMRS offers items that explore religiosity in terms of beliefs, values, religious preference, organizational religiousness, private religious practices, commitment, meaning, coping, history, forgiveness, daily spiritual experiences, and religious support (Fetzer/NIA 1999).

Many religious involvement instruments have been developed for use primarily with White populations and have not been subjected to psychometric examinations with minority samples. Fewer instruments have been developed specifically for use with minority populations. It can be risky to assume that the psychometric properties of an instrument (and its items) in one population remain intact when the instrument is applied to a different population (Stewart and Napoles-Springer 2003). Simply applying instruments across populations without re-examining psychometric properties may result in potential sacrifices in terms of understandability, cultural competence, as well as the reliability and validity of the summary scores.

For over a decade, we have been working with an instrument that assesses religious involvement in African Americans (originally reported: Lukwago et al. 2001). The instrument is based on a multidimensional model of religious involvement, assessing what we have termed religious beliefs and religious behaviors. The religious beliefs dimension involves feelings of having a personal relationship with God or a higher power and one’s personal/internal religious activities such as prayer. The religious behaviors dimension involves public and/or organized activities such as service attendance, participation in other organized religious activities (e.g., choir practice, scripture study), and talking to others about faith. This instrument was originally developed and utilized primarily with African American women (Lukwago et al. 2001; Holt et al. 2005). Further, it has mainly been utilized with local or regional population subgroups (e.g., Midwestern African American women [Holt et al. 2005; Steele et al. 2009]; African American women in North Carolina [Skinner et al. 2003]; African Americans in Alabama [Holt et al. 2009, 2010]).

Application to Health Disparities Research

Previous research has developed empirical evidence for the association between scores on our religious involvement instrument and dietary beliefs and behaviors (Holt et al. 2005), mammography utilization (Holt et al. 2003), stage progression for mammography (Steele et al. 2009), and prostate cancer screening (Holt et al. 2009) among African Americans. In addition, scores have been associated with how African American women react to health messages and health information (Kreuter et al. 2003, 2004) as well as tailored messages developed in part based on responses to items comprising the instrument (Kreuter et al. 2005). The instrument therefore has demonstrated applied value in health disparities research with selected samples of African Americans, but its psychometric properties have not been examined in a larger, population-based sample. If these psychometric properties could be verified in a broader sample, including both African American women and men, then this work could potentially have greater reach and generalizability.

The Present Study

The purpose of the present study was to evaluate the fit of the two-factor model embedded in this instrument with confirmatory factor analysis techniques using data from a national sample of African American adults. We also sought to examine the invariance of the factor loadings across men and women to determine whether the items tap the same underlying latent constructs in the same manner across gender groups. Though the instrument has performed adequately from an overall psychometric standpoint, the two-factor model has not been previously verified with confirmatory factor analysis techniques beyond convenience samples, and previous analyses have not examined possible gender difference in item meanings or sensitivities.

This study utilizes data from a larger project, the Religion and Health in African Americans (“RHIAA”) Study, assessing the nature of the religion–health connection in a national probability sample of African Americans. If the structure can be confirmed in such a sample, the instrument can have wide applicability similar to comparable instruments such as the Brief Multidimensional Measure of Religiousness/Spirituality for use in Health Research (NIA/Fetzer 1999). This is especially important because the present instrument is relatively brief and was developed specifically for use with African American samples, thus making it a potentially valuable tool for research designed to help understand and eliminate health disparities affecting this population. The brief nature of the instrument makes it particularly suitable for use in community-based and/or telephone survey research, where issues of participant burden can be a concern.

Because it is widely accepted that women, including African American women, tend to be higher in religious involvement than their male counterparts (Levin and Taylor 1993; Levin et al. 1994; Ferraro and Koch 1994; Taylor et al. 1996), it is particularly important to examine the performance of any widely utilized instrument assessing religious involvement across both male and female respondents. It is possible that particular items assessing religious involvement may be more or less sensitive indicators of the underlying construct for women compared with men. Comparisons on simple summary scores across demographic groups assume psychometric equivalence of the items across those groups under comparison, but such assumptions are rarely tested empirically in practice (Bingenheimer et al. 2005; Stewart and Napoles-Springer 2003). In this case, gender differences in religious involvement could reflect either real differences (at the latent construct level) or artifacts due to subtle changes in the meanings or sensitivities of the items. Therefore, the current study examined the invariance of the factor loadings and solutions across African American men and women.

Method

Participants

The participants in this study were African American adults age 21 and older who were living in a private residence with a telephone, in any of the 50 United States. A history of cancer was an exclusion criterion in the aforementioned larger study. Those without telephone access or housed group quarters, dormitories, jails/prisons, nursing homes, or hospitals were not eligible to participate. A list of potential participants was generated by a professional sampling firm (Genesys). This call list was derived from publicly available data aggregated from a wide variety of sources such as motor vehicle registrations, telephone directories, real estate listings, and driver’s license data. Households were randomly selected from data available to the sampling firm, based on a national representation of US census tracts. Due to the inevitable response bias, the sample was not considered to be an unbiased or “representative” sample of African Americans, but the study sample can be considered to be “probability-based” due to the sampling procedures used.

A total of 12,418 individuals were contacted for participation, and 2,370 individuals agreed to participate. The overall response rate therefore was 19% (accepted/[accepted + non-interviewed]). Of those contacted who did not participate (N = 10,048), 8,240 refused prior to a determination of eligibility and 1,658 were not eligible [81 were under age 21, 444 refused to provide an age for screening purposes, 878 were not African American, 224 reported a history of cancer, and 5 refused to respond to the question about cancer history]. Twenty-six were incapable of participating in the interview. Only 150 were determined to be eligible but then refused to participate, making for an upper bound response rate of 94% (2,370/2,520).

A total of 2,370 eligible participants completed the interview, and descriptive information for these participants is presented in Table 1. Missing data were observed on at least one of the nine religious involvement items for 23 participants (1%), and the present results are based on a final analysis sample of 2,347 participants (1,455 women and 892 men).

Table 1 Participant demographic characteristics
Full size table

Procedures

Professional interviewers recruited participants by calling phone numbers randomly selected from the study call list. Interviewers introduced themselves and when put into contact with an adult who lived at the address being dialed, they introduced the project. The first eligible individual who provided verbal consent was interviewed. Only one individual was interviewed per household. Interested individuals were then screened for eligibility criteria. If interested and eligible, interviewers read participants the study informed consent script and participants provided their assent. The entire interview took an average of 45 min. The data reported here consist of analyses of the religious involvement instrument only. A $25 gift card was mailed to each participant upon completing the full interview.

Religious Involvement

The religious involvement scale used is a 9-item instrument that assesses both religious beliefs (e.g., presence of God in one’s life, perceiving a personal relationship with God) and behaviors (e.g., church service attendance, involvement in other church activities). Seven of the items are assessed using a 5-point Likert-type format (1 = strongly disagree, 2 = disagree, 3 = neutral, 4 = agree, 5 = strongly agree), and 2 monthly service attendance items are assessed using a 3-point format (1 = 0 times per month, 2 = 1–3 times per month, 3 = 4 or more times per month). Items are shown in Table 2 along with descriptive data. Higher scores indicate higher levels of religious involvement. Previous analyses have reported excellent internal consistency for the instrument (α = .79 for religious behaviors; α = .85 for religious beliefs; Lukwago et al. 2001; Holt et al. 2003).

Table 2 Religious involvement items, means, and standard deviations
Full size table

Analyses

A confirmatory factor analysis (CFA) approach was used to examine the factor structure of the religious involvement instrument and to test for item invariance across gender groups. All analyses were conducted using Version 5.1 of Mplus analysis system (Muthén and Muthén 2008). Previous research using exploratory factor analysis methods had suggested that a two-factor model was appropriate for these nine items, with four items loading on a religious beliefs factor and five items loading on a religious behaviors factor (Holt et al. 2003, 2005, 2009, 2010). In the present CFA approach, we sought (1) to confirm the relative superiority of the two-factor model over a one-factor model with all nine items loading on a single factor, (2) to examine the modification indices of the better fitting model from the first step that might suggest further improvements in fit, and (3) to compare the factor loadings between male and female participants in the context of a multiple group CFA approach. Standard maximum likelihood was used as the estimation method. This method attempts to account for the observed variances and covariances among the items based on factor loadings, the factor correlation (for the two-factor model), and the item residual variances.

Overall model fit was evaluated using the chi-square goodness-of-fit test, the root mean square error of approximation (RMSEA), and the comparative fit index (CFI). Nested model comparisons (e.g., two-factor model vs. one-factor model, constrained versus unconstrained multiple group models) were conducted using chi-square difference tests, with the RMSEA and CFI also available to inform whether improvements in model fit were sufficient to offset increases in model complexity in these comparisons. An RMSEA of .05 or lower and a CFI of .95 or higher were considered to be indicative of excellent model fit.

Results

Factor Model Comparisons

A summary of the fit statistics for the models tested is provided in Table 3. For the entire sample, neither the one-factor model nor the two-factor model provided adequate fit to the observed data according to our criteria. The two-factor model was found to provide substantially better fit than the one-factor model (χ2 (1, N = 2,347) = 959.51, P < .0001), indicating significant distinctiveness of the religious beliefs and religious behaviors items. The modification indices from both solutions suggested that adding correlated residuals between two sets of items—items 1 and 2, and items 5 and 6—would substantially improve fit further. Because the first two items both loaded on the religious beliefs factor, reflected the concept of closeness with God, and were in consecutive order on the instrument, it was considered appropriate to allow this correlated residual. Likewise, items 5 and 6 both loaded on the religious behaviors factor, reflected attendance at religious activities, used the same phrasing and the same frequency-based response options, and also fall in consecutive order, justifying the estimation of a correlated residual to account for the methodological similarities between these two items. No other modification indices suggested theoretically defensible elaborations for further model respecification within these specific factor models. As indicated in Table 3, allowing these two correlated residuals as part of a modified two-factor model substantially improved model fit, with both the RMSEA and the CFI now indicating excellent fit for this modified two-factor model.

Table 3 Summary of confirmatory factor analysis fit statistics
Full size table

The standardized factor loadings and correlated item residuals for the modified two-factor model are displayed in Fig. 1. All estimates are significant statistically (P < .0001), indicating that each item does load highly on its designated factor. A moderate correlation of .67 was observed between the two factors, indicating that religious beliefs and behaviors were moderately correlated in this sample, but are also sufficiently distinct latent constructs.

Fig. 1
figure 1

Measurement model for religious involvement instrument

Full size image

Loading Invariance Findings

Two multiple group CFA models were estimated in the context of the modified two-factor model. First, a “constrained” model that forced the unstandardized factor loadings to be equal across gender groups was estimated, and then an “unconstrained” model that allowed men and women to have different factor loadings. In both of these models, factor means were also estimated, which allowed us to examine possible gender differences in construct endorsement. A nested model comparison indicated that the unconstrained model did not fit significantly better than the constrained model (χ2 (7, N = 2,347) = 5.01, ns). This indicates that allowing gender-specific factor loadings did not significantly improve model fit compared to a model that forced the factor loadings to be equal across the two genders. Therefore, the items were found to have equivalent sensitivities to their underlying latent constructs for both men and women. The factor mean comparisons indicated that women scored significantly higher than men on both latent constructs (P < .0001). Specifically, in the context of the constrained factor loading model, women had standardized factor means that were .44 and .50 standard deviation units higher than men on religious beliefs and religious behaviors, respectively.

Discussion

The present findings are consistent with previous research among African American women, but also provide important support for the use of this instrument to assess religious involvement in the overall African American population. The use of confirmatory factor analysis methods on data from a national sample is an important advance, as is the analysis of data from African American men. The overall group measurement model confirmed the factor structure from previous research conducted with US convenience samples, using the two-dimensional model of religious beliefs and religious behaviors. However, the two-factor model was not found to provide adequate fit to the observed item data until two correlated residuals were added. These residuals accounted for specific methodological commonalities shared between two sets of adjacent items and do not alter the general conceptual nature of the two-dimensional structure of this instrument. The present analysis constitutes the first test of these factor models for this instrument using national, population-based data from both genders.

The multiple group models addressed the question of whether the item loadings were statistically equivalent for African American men and women. Even with the large sample size that provided considerable power to detect relatively small differences in item loadings, the present findings indicated that men and women did have equivalent factor loadings for these items on these latent constructs. Our findings of gender differences on latent means are consistent with previous reports of differences in levels of religious involvement between African American men and women (Levin and Taylor 1993; Levin et al. 1994; Ferraro and Koch 1994; Taylor et al. 1996). The equivalence of the factor loadings across gender groups in our data is important because it confirms that our instrument includes items that are equally sensitive for assessing levels of religious involvement in African American men and women. The gender differences in rates of endorsement therefore can be confidently interpreted as true differences in religious involvement at the latent construct level, and not measurement artifacts or gender differences in the meanings or sensitivities of the individual items.

The multidimensional characterization of religious involvement is consistent with previous models. Several previous instruments include both a belief and a behavioral dimension of religious involvement. For example, the Cross-Cultural Dimensions of Religiosity Scale includes dimensions on beliefs, experience, practice, moral consequences, religious knowledge, and social consequences (DeJong et al. 1976). The instrument was validated in samples of United States and German college students. The Dimensions of Religious Commitment Scale involves Belief Orthodoxy, Particularism, Ethicalism, Practice Ritual Involvement, Devotionalism, Religious Experience, and Religious Knowledge and was validated with national US samples (Glock and Stark 1966). The aforementioned MMRS also contains subscales to assess both religious beliefs as well as practices, among a number of previously mentioned facets (Fetzer/NIA 1999). Though these instruments do contain similar subscales (e.g., beliefs), they are in many cases not directly overlapping from a conceptual standpoint. In addition, in many cases these instruments have not been subjected to confirmatory factor analysis procedures to verify their multiple dimensions or been validated across gender or race/ethnicity subsamples. The present instrument is also brief enough for use in community-based research and telephone surveys of multiple health-related variables, where issues of increased participant burden are often important.

Strengths and Limitations

Strengths of the present study include the use of a national US probability-based sample, which extended our evaluation of this instrument beyond previously used regional samples and samples of African American women only. In addition, due to the large sample size, the study was able to take advantage of sophisticated statistical techniques including multiple group factor loading invariance analyses. In terms of limitations, there was the inevitable response bias that is present in some degree with any survey method, telephone data collection notwithstanding. The brief refusal survey indicated that non-responders were in general older, more likely to be men, less educated, and less religiously active than responders.

Conclusion/Future Research

In summary, these findings support the use of the present relatively brief instrument for measuring the level of religious beliefs and religious behaviors in African Americans of both genders. Future work is needed to determine whether this instrument is suitable for other populations or whether additional items and constructs should be added. The present sample is largely Christian in religious affiliation, for example, and much less research has been conducted examining the measurement of religious involvement among non-Christian samples. When considering research with more diverse religious/spiritual groups, not only does item wording need to be considered (e.g., use of the term “God”) but also the constructs themselves need to be reconsidered and potentially altered. There may be additional and/or different subscales that are relevant for different groups that need to be considered. This is an area ripe for qualitative exploration, leading to subsequent item development. It is only through this in-depth, iterative, process that suitable instruments will be able to be developed for such diverse groups, and likely why research in this area has lagged behind the more mainstream.