Introduction

BlacksFootnote 1 are significantly less residentially segregated from whites than in the past. Overall, black-white residential segregation in U.S. metropolitan areas has declined slowly but steadily since the early 1970s (Logan et al. 2004; Timberlake and Iceland 2007), and the number of metropolitan areas in which blacks are highly segregated on multiple dimensions has fallen (Wilkes and Iceland 2004). After peaking in the 1960s, the average level of segregation among U.S. metropolitan areas declined by approximately 25 % (Logan and Stults 2011). The number of majority-white neighborhoods declined sharply, and the number of integrated and multiethnic neighborhoods increased significantly during this period (Farrell and Lee 2011; Logan and Zhang 2010; Wagmiller 2013). The traditional black-white axis of residential segregation is being replaced by a more complex pattern of residential mixing (Logan 2013; Massey et al. 2009).

Social scientists attribute both the high levels of residential segregation historically experienced by blacks as well as the more recent decreases in black-white residential segregation to two social processes. The spatial assimilation perspective maintains that the high levels of residential segregation blacks have experienced historically reflect their disadvantaged social and economic characteristics relative to whites (Duncan and Duncan 1955; Massey 1985; Park 1926; Timms 1971; White 1987). The place stratification perspective holds that the high levels of residential segregation experienced by blacks reflect social and politico-economic processes that impede their ability to access racially integrated neighborhoods, irrespective of their socioeconomic resources. Some place stratification theories emphasize the role of direct and indirect discrimination in the housing market in black-white residential segregation (Charles 2003; Logan and Alba 1993; Logan and Molotch 1987), whereas others have highlighted the roles of racial differences in residential preferences (Clark 2009; Krysan 2002; Krysan and Farley 2002), housing search strategies (Krysan 2008), and knowledge about different types of neighborhoods (Krysan and Bader 2009).

The spatial assimilation and place stratification perspectives highlight differing mechanisms that produce greater residential integration for blacks. Spatial assimilation theory emphasizes changes in the population composition of blacks over time as a principal force leading to the decline in black-white residential segregation. Rising levels of formal education for blacks and the growth of the black middle class over the last half-century are assumed to have led to greater residential integration with whites, while stagnant incomes, declining rates of marriage, and rising levels of incarceration are presumed to dampen progress. By contrast, place stratification theories suggest that greater integration reflects changes in the association between social and economic characteristics and residential outcomes over time. From the place stratification perspective, the central factors producing the decline in black-white residential segregation are lower barriers to integration.

Previous studies have highlighted the important role of social and economic resources in recent improvements in racial residential integration (Fischer et al. 2004; Freeman 2008; Iceland and Wilkes 2006; Logan et al. 2004; Pais et al. 2012; Reardon et al. 2015; Timberlake and Iceland 2007). However, current studies have not yet clearly differentiated these two mechanisms of change or quantified the relative strength of each. In this study, we propose an alternative strategy for assessing the relative strength of spatial assimilation and place stratification processes in fostering change in residential segregation. This analytic approach uses individual-level data on the socioeconomic and neighborhood attainment of individuals to partition the population-level decline in residential segregation into that which can be attributed to (1) changes in the social and economic characteristics of blacks (spatial assimilation processes) and (2) changes in the association between blacks’ social and economic characteristics and the level of residential segregation they experience (place stratification processes). To illustrate the utility of this approach, we compared the residential experiences of a series of black birth cohorts at middle age (45 years old) between 1968 and 2013 using data from the Panel Study of Income Dynamics (PSID) and Blinder-Oaxaca regression decomposition techniques.

Processes of Spatial Assimilation

Social and Economic Mobility and Spatial Assimilation

Spatial assimilation theory argues that one impediment to black neighborhood attainment is low socioeconomic status (SES) (Duncan and Duncan 1955; Massey 1985; Park 1926; Timms 1971; White 1987). Historically, blacks have been concentrated at or near the bottom of the SES hierarchy in the United States. Rates of poverty and unemployment for blacks have been higher than for native-born whites, and median household income and net wealth for blacks have been lower (e.g., Iceland 2013, 2014). As blacks experience upward socioeconomic mobility, spatial assimilation theory predicts that residential segregation will decrease.

Over the last half-century, the socioeconomic standing of blacks has improved in some notable ways (Harrison and Bennett 1995; Kao and Thompson 2003) but has stagnated or deteriorated in other ways. Today, many more blacks graduate from college, and many fewer fail to complete high school (U.S. Census Bureau 2014); in addition, a smaller share of the black population lives in poverty (DeNavas-Walt and Proctor 2014). However, black median household income (DeNavas-Walt et al. 2006), net wealth (Kochhar et al. 2011), and homeownership (Leigh and Huff 2007) have been stagnant since the early 1970s, and black marriage rates have declined (Raley et al. 2015), and their incarceration rates have risen (Western and Wlideman 2009). On the basis of rising educational achievement and declining poverty rates, spatial assimilation theory predicts that black-white residential segregation should have fallen since the 1970s, but that progress has likely been slowed by income stagnation, falling marriage rates, and rising rates of incarceration.

Barriers to Spatial Assimilation

Historically, blacks have experienced greater difficulty securing homes in predominantly white and racially integrated neighborhoods than members of other minority groups, regardless of their social and economic resources. One reason why even blacks of higher SES have faced barriers to integrating with whites is discrimination in the housing market. Real estate agents steer black home buyers to neighborhoods with more blacks and white home buyers away from such neighborhoods (e.g., Galster and Godfrey 2005; Ross and Turner 2005). Landlords—and to a lesser extent, realtors, banks, and mortgage companies—have used both exclusionary mechanisms (e.g., refusals to rent, unfair lease terms) and nonexclusionary mechanisms (harassment, intimidation, and coercion) to discourage blacks seeking homes from relocating to white neighborhoods (Roscigno et al. 2009). Mortgage lenders are less likely to approve applications from black applicants than those from white applicants, even when controlling for individual attributes such as credit scores and neighborhood socioeconomic characteristics (Bocian et al. 2006; Reibel 2000).

Moreover, federal mortgage lending policies have historically institutionalized racial discrimination in lending by channeling mortgage funds away from black and integrated neighborhoods (Massey and Denton 1993). Contemporary efforts to desegregate neighborhoods are slowed by local land-use policies, such as antidensity zoning (Galster and Cutsinger 2007; Rothwell and Massey 2009). Black homeowners and tenants still report being harassed and intimidated by white residential neighbors (Roscigno et al. 2009).

Social and economic resources are also more weakly correlated with neighborhood attainment for blacks because past experiences of discrimination coupled with expectations about prejudice and discrimination in white neighborhoods lessen blacks’ desire to relocate to these neighborhoods. Although blacks’ neighborhood racial preferences are more flexible than whites’ (Krysan and Bader 2007; Krysan and Farley 2002; Krysan et al. 2009), and many blacks consider moving to neighborhoods where they are in the minority (Krysan 2002), blacks overwhelmingly rate white neighborhoods as undesirable (Clark 2009; Krysan and Bader 2007; Krysan et al. 2009). Blacks’ avoidance of predominantly white neighborhoods reflects partly a neutral ethnocentrism (Clark 1986, 1991, 1992, 2009) and partly a fear of white hostility and discrimination (Krysan 2002; Krysan and Farley 2002; Krysan et al. 2009).

An important consequence of these place stratification processes is that even advantaged and socioeconomically mobile blacks face considerable obstacles to using their resources to attain greater integration. Although previous studies have found that blacks of higher SES tend to be less segregated from whites than blacks of lower SES, these studies have also consistently found that even more advantaged blacks have difficulty overcoming barriers to residence in racially integrated neighborhoods. For example, middle-class and affluent blacks tend to reside in less-advantaged neighborhoods that have fewer white residents than white families with the same resources (Alba and Logan 1991; Alba et al. 2000; Logan et al. 1996; Pattillo-McCoy 2000; Spivak and Monnat 2013). Moreover, although middle- and high-income blacks are more likely than low-income blacks to live in neighborhoods with whites, they are not more likely than lower-income blacks to live in neighborhoods with whites of comparable income (Adelman 2004; Adelman et al. 2001; Iceland et al. 2005; Massey and Fischer 1999).

Many of the social processes that place stratification theorists believe have historically prevented blacks from moving into integrated and predominantly white neighborhoods have weakened since the early 1970s. White attitudes toward racial equality have liberalized (Bobo 2001; Schuman et al. 1997). Over time, whites have become more likely to prefer racially integrated neighborhoods and less apt to prefer all-white neighborhoods, although few whites express a willingness to move to neighborhoods with significant black populations (Bobo et al. 1986; Farley and Frey 1994). Housing audit studies have found that discrimination by rental and real estate agents has declined (Choi et al. 2008), although racial steering has not (Pager and Shepherd 2008). Changes in government policy have largely ended the consideration of neighborhood racial composition in lending decisions (Benston and Horsky 1991; Munnell et al. 1996; Schafer and Ladd 1981). Loans to underserved populations and neighborhoods have risen significantly, although increases in subprime lending account for most of this growth (Williams et al. 2005). From the place stratification perspective, recent declines in black-white residential segregation reflect a weakening of the barriers that have historically restricted blacks’ ability to achieve integration with whites, regardless of their socioeconomic resources.

Evaluating the Sources of Increasing Black Spatial Assimilation

Previous studies evaluating the reasons for the decline in black-white residential segregation reveal the important role that social and economic resources play in progress toward racial residential integration. However, they do not differentiate clearly the two predominant explanations for the change, nor do they quantify the relative strength of each mechanism. One prominent analytic approach for evaluating the role of black socioeconomic mobility on black-white residential segregation has been to regress changes in metropolitan racial residential segregation levels on indicators of black SES and mobility (e.g., Iceland et al. 2005; Wilkes and Iceland 2004). Using this approach, Iceland and his colleagues found that the negative correlation between SES and black-white segregation has increased. A related approach is to compare changes in segregation in metropolitan areas in which blacks made socioeconomic gains relative to whites with areas in which they did not. Studies using these approaches have produced conflicting conclusions. For example, Logan et al. (2004) found that residential segregation between 1980 and 2000 did not decline more in areas where the black-white income ratio improved more, but Krivo and Kaufman (1999) and Timberlake and Iceland (2007) found that the narrowing of the black-white income ratio was associated with more rapid declines in segregation.

Other studies have explored how the residential segregation of blacks of different SES has changed over time (e.g., Adelman 2004; Adelman et al. 2001; Iceland and Wilkes 2006; Iceland et al. 2005; Sharkey 2014). Overall, these studies found that middle-class and affluent blacks have become less segregated from whites over time (Iceland and Wilkes 2006) and, to a more limited extent, that blacks have become less segregated from whites of similar SES (Adelman 2004; Iceland et al. 2005). Sharkey (2014) found that a growing segment of middle- and upper-income blacks have distanced themselves from disadvantaged neighborhoods by moving to neighborhoods in which they are not in the majority. Finally, some studies have used entropy-based measures that allow overall levels of residential segregation to be decomposed into the independent contributions of different dimensions of segregation, such as race, class, and geographic scale. In one such study, Fischer et al. (2004) found that the importance of income relative to race and ethnicity in shaping segregation has increased significantly since the 1970s.

Together, these approaches portray a shifting residential landscape in U.S. metropolitan areas since 1970. Blacks are less segregated from whites than they were in the 1970s, although segregation remains relatively high. Residential segregation declined partly because the gap between blacks and whites in resources diminished and partly because the barriers to blacks using their resources to secure residence in integrated and white neighborhoods lessened.

Nonetheless, fundamental questions about the reasons for the decline in black-white residential segregation remain unanswered. For example, what proportion of the decline in residential segregation is attributable to upward social and economic mobility among blacks (i.e., changes in the social and economic composition of the black population)? What proportion is attributable to improvements in the ability of blacks to use their resources to secure greater integration with whites (i.e., strengthening of the association between socioeconomic characteristics and residential attainment)? What forms of social mobility have been most central to the decline in segregation?

To answer these questions, we propose an alternative approach. Rather than using metropolitan-level data to examine the association between changes in blacks’ social and economic resources and changes in the level of segregation, we used individual-level data and Blinder-Oaxaca regression decomposition techniques to partition change in average segregation levels for cohorts of blacks reaching middle age between 1968 and 2013. This approach enabled us to partition the decline in residential segregation into that attributable to (1) changes in the social and economic composition of black cohorts and (2) intercohort changes in the associations between the black cohorts’ social and economic characteristics and the levels of segregation they experienced.

Methods

Data and Sample

We use data from the PSID to decompose change in black-white residential segregation for cohorts of blacks turning age 45 between 1968 and 2013. The PSID is a nationally representative panel study that has been conducted annually or bienniallyFootnote 2 since 1968 (Hofferth et al. 2001). Individuals from the original 5,000 families are reinterviewed at each wave regardless of whether they live in the same dwelling or with the same people. Adults are followed as they grow older, and children are observed as they advance into adulthood, forming family units of their own. This procedure produces an unbiased sample of families each year and a continuously representative sample of children born into families (Fitzgerald 2011; Fitzgerald et al. 1998). Attrition of the sample has been modest for an extended panel study and has not generally affected the representativeness of the sample (Fitzgerald et al. 1998) or patterns of association for socioeconomic outcomes (Fitzgerald 2011).

The PSID-Geocode Match File makes it possible to link information about respondents’ demographic, social, and economic characteristics at each interview to information about the racial and ethnic composition of their census tract of residence. Records for PSID respondents from 1968 to 2013Footnote 3 are linked to tract-level census data from the Longitudinal Tract Database (LTDB) (Logan et al. 2014, 2016), which contains 1970–2010 Census of Population and Housing data normalized to Census 2010 tract boundaries. The PSID-Geocode Match File matches respondents’ addresses at each wave since 1968 to the corresponding Census 2010 tract code. Because PSID geocode information is available for respondents at single-year intervals but tract-level data are available only at 10-year intervals, we used linear interpolation to impute the racial composition of people’s census tracts during years between censuses and linear extrapolation to impute characteristics for waves before 1970 and after 2010.

Our analysis examined the relationship between the social, economic, and demographic characteristics of black members of the PSID sample and the racial composition of these members’ neighborhoods the year they turned 45. By this age, most members of the sample had completed formal education, established themselves in jobs or careers, married and started families, bought homes, and made ties to neighborhoods and communities (Long 1988).

We imposed two other sample selection criteria for this analysis. First, we included only those people who are members of original PSID families or their offspring to avoid including multiple members of the same family (e.g., both husband and wife). Second, we included only those people residing within the boundaries of a core-based statistical area (CBSA), defined by the Office of Management and Budget 2010 statistical tabulations as having a population of at least 100,000 people in 1970. Imposing these restrictions yielded a sample size of 2,426.

Dependent Variables

The outcome measures in this study describe the level of black-white residential segregation experienced by black members of the PSID sample at age 45. Comparing levels of residential segregation for individuals living in different urban areas or historical periods is complicated by the fact that the racial and ethnic composition of an individual’s neighborhood at any given time depends not only on variation in the distribution of groups across neighborhoods in a metropolis but also on the overall racial and ethnic composition of the metropolis at that time. Moreover, different measures of segregation reveal different dimensions of the experience of residential segregation (Massey and Denton 1988). We used two measures of residential segregation. One measure, the Neighborhood Dissimilarity Score (D), reflects the extent to which the racial composition of the subject’s neighborhood differs from that of the CBSA. The other measure, the Neighborhood Isolation Score (I), gauges the extent to which the subject is exposed to members of the majority group based on residence in the neighborhood.

Neighborhood Dissimilarity Score

Index of Dissimilarity measures facilitate comparisons over time and across urban areas by comparing the distribution of racial groups across neighborhoods in an area at a given point to the distribution expected under the assumption that the area is perfectly integrated: that is, every neighborhood has a racial composition identical to the area as a whole (Reardon and Firebaugh 2002).

We can define segregation at the individual and neighborhood level similarly as the extent to which the racial composition of a person’s neighborhood reproduces that of the broader area. More specifically, the segregation individual i experiences in neighborhood j in CBSA k at time t reflects the deviation of the proportion of residents in the census tract that are non-Hispanic black from the proportion of CBSA residents that are non-Hispanic black. Formally, the subject’s neighborhood dissimilarity score is defined as

$$ {D}_{ijkt}=100\times \left(\frac{b_{jkt} - E\left({b}_{jkt}\right)}{p_{jkt}}\right), $$
(1)

where b jkt is the observed number of non-Hispanic blacks in tract j in CBSA k at time t, E(b jkt ) is the expected number of non-Hispanic blacks in tract j in CBSA k at time t (assuming perfect integration), and p jkt is the total number of non-Hispanic blacks and whites in tract j in CBSA k at time t.

The expected number of non-Hispanic blacks in tract j in CBSA k at time t assuming perfect integration is a function of the proportion of the CBSA population that is non-Hispanic black and the population size of the tract:

$$ \mathrm{E}\left({b}_{jkt}\right)=\left(\frac{B_{kt}}{p_{kt}}\right)\left({p}_{jkt}\right), $$
(2)

where B kt is the number of non-Hispanic blacks in CBSA k at time t, P kt is the total number of non-Hispanic blacks and whites in CBSA k at time t, and p jkt is the total number of non-Hispanic blacks and whites in tract j in CBSA k at time t.

D ijkt can range from –100 to 100. An individual’s neighborhood dissimilarity score is equal to 0 when the proportion of non-Hispanic blacks in the neighborhood is identical to the proportion in the CBSA. When non-Hispanic blacks are overrepresented in an individual’s neighborhood, D ijkt is positive; and when they are underrepresented, D ijkt is negative. D ijkt can be interpreted as the percentage of black residents in the neighborhood who would have to move out to equalize group proportions in the neighborhood and CBSA.

Counts of the number of non-Hispanic blacks and whites in each census tract are available from the LTDB for 1980–2010. However, counts of the number of whites and blacks by Hispanic ethnicity in 1970 are unavailable. Counts of the number of non-Hispanic blacks and whites for census tracts in 1970 are estimated based on counts of the number of blacks, whites, and Hispanics in the tract in 1970, and the distribution of Hispanics among racial groups in the tract in 1980. The algorithm used to allocate the 1970 Hispanic population count in a tract across racial groups assumes that the size of the Hispanic population in the tract may have changed between 1970 and 1980, but that the relative distribution of Hispanic blacks, Hispanic whites, and Hispanic members of other races in the tract did not.Footnote 4

Neighborhood Isolation Score

Exposure-type measures of segregation such as the Isolation Index describe the extent to which members of the minority group physically encounter members of the majority group by virtue of the neighborhood in which they live (Massey and Denton 1988). We measured exposure using the neighborhood isolation score, I ijkt , which is the percentage of the neighborhood population (blacks and whites) that is non-Hispanic black:

$$ {I}_{ijkt}=100\times \left(\frac{b_{jkt}}{p_{jkt}}\right). $$
(3)

Independent Variables

Primary independent variables of interest represent the social and economic characteristics of individuals and their families. Measures of SES included the subject’s educational degree and family income.Footnote 5 We did not include a measure of wealth because the PSID did not begin collecting information on family wealth until 1984. We also included measures capturing key social characteristics, such as whether a person was married, children were present in the home, the individual or family received welfare, or the individual had ever been imprisoned or served in the armed forces. We controlled for region of residence because blacks in the South and West regions of the country live in less-segregated neighborhoods than blacks in the Northeast and Midwest. We controlled for sex to account for intercohort differences in sex composition and potential sex differences in locational outcomes.Footnote 6 Unless otherwise noted, all measures represent subjects’ values when they were 45 years old. The independent variables used in the analysis are as follows:

  • Family income is the sum of all family members’ labor, assets, and public transfer income at age 45. Family income is classified into quintiles using the annual family income quintile cutoffs published by the U.S. Census Bureau (2015), with the bottom income quintile assigned to be the reference category.

  • Education is classified into four categories: high school dropout (reference category), high school graduate, some college, and college graduate or more.

  • Marital status is categorized as married; divorced, separated, or widowed; and never married (reference category).

  • Children is a dichotomous variable indicating whether children are present in the home.

  • Welfare receipt is a dummy variable coded as 1 if the family received public transfer income (Aid to Dependent Children, Aid to Families with Dependent Children, Temporary Assistance to Needy Families) during the previous year.

  • Imprisonment is a dichotomous variable indicating whether the individual was imprisoned prior to age 45.

  • Military service is a dichotomous variable coded as 1 if the family head ever served in the armed forces.

  • Female is coded 1 if the subject is female.

  • Region denotes the region in which the individual lived, with residence in the Northeast serving as the omitted reference category.

Choosing Cohorts for Comparison

To assess intercohort change in residential segregation using the Blinder-Oaxaca regression decomposition technique, it is necessary to place individuals from different birth cohorts together into a few groups for comparison purposes. Two goals guided our designation of cohort groups. First, we combined birth cohorts that shared similar historical experiences during late adolescence and early adulthood (ages 14 to 18). Second, we constructed cohort groups with enough individuals to permit statistically reliable comparisons between groups.

Generational models of social change emphasize the importance of late adolescence and early adulthood in forming values and attitudes. From this perspective, social change is a consequence of the unique social, economic, and historical conditions each generation experiences as they become politically and socially aware during late adolescence and early adulthood (Mannheim 1952; Ryder 1965). For this reason, we constructed cohort groups that shared sociohistorical experiences during these crucial stages of life.

The two earliest cohorts in our study reached adulthood prior to the passage of major civil rights legislation, such as the Voting Rights Act of 1965 and the Fair Housing Act of 1968. Members of the earliest cohort, which we refer to as the Great War cohort, were born between 1923 and 1934. These individuals were children and adolescents during the Great Depression and World War II and turned 18 between 1941 and 1952. They turned 45 during the civil rights and liberal post–civil rights eras (1968–1979). The second cohort, which we refer to as the pre–civil rights era cohort, was born between 1935 and 1945. This cohort’s formative years occurred just prior to the height of the civil rights era, with members turning 18 between 1953 and 1963. They turned 45 during the conservative post–civil rights era (1980–1990). Members of the third cohort, which we refer to as the civil rights era cohort, were born between 1946 and 1957. This cohort’s formative years occurred during and immediately after passage of major civil rights legislation. Members of this cohort turned 18 between 1964 and 1975, and turned 45 during the neoliberal era (1991–2002). Members of the final cohort, which we refer to as the post–civil rights era cohort, were born between 1958 and 1968. This cohort’s formative years occurred well after the passage of the major civil rights legislation of the 1960s. Members of this cohort turned 18 between 1976 and 1986, and turned 45 during the post–9/11 era (2003–2013).

Analytic Approach

We used the Blinder-Oaxaca decomposition method (Blinder 1973; Oaxaca 1973) to identify the sources of intercohort change in residential segregation. As a first step in partitioning change, we estimated separate regression equations for the cohorts being compared using ordinary least squares (OLS). We used OLS regression because our measures are distributed relatively normally, with few values near their theoretical minimums or maximums, and because studies that use dissimilarity or isolation indexes as their outcome measure have traditionally used linear models such as OLS (e.g., Iceland and Wilkes 2006; Spivak and Monnat 2013; Timberlake and Iceland 2007). We estimated the neighborhood segregation score for individual i in cohort C, \( {y}_i^c \) as follows:

$$ {y}_i^c={b}_{0 c}+{\sum}_{j = 1}^J{\sum}_{k = 1}^{K_j}{b}_{j kl}^C{x}_{j kl}^C+{e}_i^C, $$
(4)

where j refers to the jth factor, and k indexes the K levels of the jth factor. Each factor includes a complete set of dummy variables, with a reference group omitted.

In the second step, we used the estimated coefficients and intercept from each of the regression equations and the sample means for the covariates for each cohort to compute two counterfactuals. The first quantified how the neighborhood segregation score would change if the regression coefficients and intercept changed, as they did between cohorts, but the population composition did not change (i.e., the means of the covariates are the same for both cohorts). The second counterfactual quantified how segregation would change if the population composition changed, as it did between cohorts, but the regression coefficients did not change.

Adapting Kim’s (2013) notation, in the case of a single factor, the difference between mean neighborhood segregation scores for Cohorts 1 and 2, \( {y}_i^1 - {y}_i^2 \), can be partitioned into three components:

$$ {y}_i^1 - {y}_i^2=\underset{\mathrm{D}1\mathrm{A}}{\left({\widehat{b}}_0^1-{\widehat{b}}_0^2\right)}+\underset{\kern1.8em \mathrm{D}1\mathrm{B}}{\sum_{k = 1}^K\left({\widehat{b}}_k^1-{\widehat{b}}_k^2\right){\overline{x}}_k^2}+\underset{\kern1.8em \mathrm{D}2}{\sum_{k = 1}^K\left({\overline{x}}_k^1-{\overline{x}}_k^2\right){\widehat{b}}_k^1}, $$
(5)

where D2 represents the amount of total change in residential segregation across cohorts that is attributable to differences in the means for the covariates (i.e., total endowment effect), D1B denotes the amount of total change that is attributable to the slope coefficients (i.e., total coefficient effect), and D1A refers to the amount of total change that can be ascribed to differences in the intercepts (i.e., intercept component). D1A represents the unexplained portion of the difference that may be due to unobserved differences across cohorts (Jones and Kelley 1984).

Because estimates of the intercept in regression models vary with the choice of reference group, when the reference category is changed, the extent to which total change is ascribed to the intercept component (D1A) as opposed to the coefficient component (D1B) changes, as do estimates of the contributions of individual factors. To resolve this identification problem, Gardeazabal and Ugidos (2004) recommended imposing a restriction of Σb k = 0 for each factor. We followed Yun (2005) in using the averaging method to impose this normalization, although we reached similar conclusions using Kim’s (2013) grand-mean centering method. With this constraint, the intercept term can be interpreted as the generalized grand mean, and the estimated effects (i.e., the b k ) represent deviations from this mean. With this approach, estimated coefficients for the covariates and for the intercept do not depend on the choice of reference category.

We estimated all analyses using the PSID sampling weights that adjusted for different selection probabilities and attrition. Standard errors were estimated using complex sample survey variance estimators that account for the clustering of observations by CBSA. Because missing data rates were less than 1 % for all variables used in this analysis, we used listwise deletion.

Results

How Much Did Black-White Residential Segregation Decline?

Fig. 1 displays mean neighborhood dissimilarity and neighborhood isolation scores for each cohort. The civil rights era and post–civil rights era cohorts lived in more integrated neighborhoods at age 45 than did either of the cohorts that transitioned to adulthood prior to the civil rights era (i.e., Great War and pre–civil rights era cohorts), but the two pre–civil rights era cohorts (Δ D = 0.9, p = .86; Δ I = 5.4, p = .29) and the two civil rights era cohorts (Δ D = –1.3, p = .74; Δ I = –0.7, p = .87) did not experience statistically significant change in segregation. Blacks in the Great War and the pre–civil rights era cohorts resided in neighborhoods in which, on average, approximately 47 % of blacks needed to relocate to another neighborhood to integrate with whites, and 67 % of the neighborhood population was black. By comparison, blacks in the two civil rights era cohorts lived in neighborhoods in which roughly 38 % of blacks would have needed to relocate to another neighborhood to achieve racial equality, and 61 % of the population was black. The neighborhood dissimilarity index declined by 20 % between the pre–civil rights era cohort and the civil rights era cohort (Δ D = 9.3, p = .03), and the neighborhood isolation score fell by 12 % (Δ I = 8.1, p = .06), but D declined by only 3.4 % and I declined by only 1.1 % between the civil rights era and the post–civil rights era cohorts.

Fig. 1
figure 1

Black-white residential dissimilarity and isolation indices, by cohort: 1968–2013 PSID

Full size image

We combined the Great War and pre–civil rights era cohorts into a single cohort (born 1923–1945) and the two civil rights era cohorts into a single cohort (born 1946–1968) for all subsequent analyses because there was neither statistically significant nor meaningful change between cohorts in residential segregation to decompose.

How Has the Social and Economic Composition of Black Cohorts Changed?

Table 1 displays means and standard errors for the social, economic, and demographic characteristics of the pre– and post–civil rights era cohorts at age 45. Significance levels reported in the final column in the table represent tests for equality of the means for the cohorts.

Table 1 Means and standard errors for variables used in the analysis
Full size table

The social and economic standing of the civil rights era cohorts improved in some important ways relative to the pre–civil rights era cohorts and deteriorated in other ways. Blacks in the civil rights era cohorts were more likely to have attended (Δ = 0.20, p < .001) and graduated from college (Δ = 0.07, p = .02) and were less likely to have dropped out of high school (Δ = –0.27, p < .001). They were also less likely to receive welfare income (Δ = –0.07, p < .001) and have children (Δ = –0.20, p < .001). However, blacks in this cohort were also more likely to have been imprisoned (Δ = 0.09, p < .001) and less likely to have served in the armed forces (Δ = –0.13, p < .001). Fewer members of the civil rights era cohorts were married (Δ = –0.17, p < .001), and more were never married (Δ = 0.24, p < .001) and divorced, separated, or widowed (Δ = 0.06, p = .09). The distribution of family income was largely unchanged between the pre–civil rights era and civil rights era cohorts, although the share of the black population in the bottom quintile increased (Δ = 0.07, p = .10) and the share in the middle quintile fell (Δ = –0.08, p = .03).

How Has the Association Between Blacks’ Social and Economic Characteristics and Residential Segregation Changed?

The social and economic composition of the cohorts changed more than the association between cohort members’ characteristics and the level of residential segregation experienced. However, the pattern of association for the civil rights era cohorts differs in noteworthy ways from that for the pre–civil rights era cohorts. Table 2 presents results from OLS regression models predicting neighborhood dissimilarity and neighborhood isolation scores as a function of blacks’ social and economic characteristics for each cohort. The significance tests reported in the rightmost column for each segregation measure test for equality of the regression coefficients for the pre–civil rights era and post–civil rights era cohorts, and the significance tests reported in the coefficient columns test whether β k = 0.

Table 2 Estimated coefficients from an ordinary least squares regression predicting neighborhood dissimilarity and isolation scores
Full size table

Reflecting the high levels of discrimination that blacks have historically encountered in the housing market, many traditional predictors of neighborhood attainment (such as family income) are weakly related to levels of racial residential segregation in both cohorts. In both the pre–civil rights era and civil rights era cohorts, blacks in the upper quintiles of income distribution did not reside in more integrated neighborhoods than those in the poorest quintiles. For example, in the civil rights era cohorts, blacks in the top two quintiles are predicted to reside in neighborhoods with dissimilarity scores only 3.33 and 1.38 points lower, respectively, and isolation scores only 1.80 and 1.66 points lower than those in the bottom quintile. In the pre–civil rights era cohorts, blacks in the top quintile are predicted to reside in more segregated neighborhoods (D = 5.48; I = 9.98) than those in the bottom quintile, but those in the fourth quintile are estimated to live in less segregated neighborhoods (D = –10.10; I = –1.38). In neither cohort do any of the estimated coefficients for the family income variables approach conventional levels of statistical significance.

By contrast, the relationship between education and residential segregation shifted significantly across cohorts. In the pre–civil rights era cohorts, black college graduates lived in significantly more integrated neighborhoods that those with less education (D = –22.65; I = –23.95). However, everyone else—high school dropouts, high school graduates, and college attendees—resided in neighborhoods that were similarly segregated. In the civil rights cohorts, segregation levels were tied more closely to educational attainment. D scores for black high school graduates were 8.91 points and I scores were 6.23 points lower than for high school dropouts; D scores for black college attendees were 10.97 points and I scores were 7.34 points lower than for high school dropouts. Tests for equality of the regression coefficients for the pre–civil rights era and civil rights era cohorts indicate statistically significant change in the education coefficients but not in the other coefficients in the model.

Although the coefficients for other factors did not change significantly between the pre–civil rights era and post–civil rights era cohorts, the estimated coefficients for some characteristics did change in ways that could account for part of the decline in residential segregation between the two cohorts in the decomposition analysis. The residential advantages of married (Dpre-CR = –8.70, Dpost-CR = –3.75; Ipre-CR = –10.12, Ipost-CR = –3.08) and divorced, separated, and widowed families (Dpre-CR = –10.16, Dpost-CR = 0.08; Ipre-CR = –11.92, Ipost-CR = –0.01) relative to those who never married weakened between cohorts, as did the unexpected large and positive effect of previous incarceration on residential integration (Dpre-CR = –19.36, Dpost-CR = –0.70; Ipre-CR = –24.42, Ipost-CR = –2.80).Footnote 7 The relative residential disadvantage of welfare recipients declined modestly (Dpre-CR = 11.89, Dpost-CR = 7.92; Ipre-CR = 13.38, Ipost-CR = 9.47).

Regression Decomposition

Table 3 presents results from the Blinder-Oaxaca linear decomposition. The top panel of Table 3 displays total change in mean neighborhood dissimilarity and isolation scores, and partitions this decline into what can be explained by change in endowments and what resulted from change in the regression coefficients and intercept. The bottom panel of the table displays the contribution of changes in endowments and coefficients for individual predictors to overall change in residential segregation.

Table 3 Blinder-Oaxaca regression decomposition of the decline in neighborhood segregation
Full size table

If spatial assimilation processes are the key to recent improvements in residential segregation, changes in blacks’ socioeconomic endowments should explain most of the decline in neighborhood segregation between cohorts. For example, we would expect that the substantial educational gains blacks have made in the post–civil rights era are responsible for much of the decline in residential segregation. But if place stratification processes are largely responsible for residential segregation, we would expect most of the change in neighborhood segregation scores to be explained by changes in the coefficients for blacks’ social and economic characteristics that reflect an improved ability for blacks—especially better-off blacks—to achieve integration with whites.

Mean neighborhood dissimilarity for the civil rights era cohorts was 9.35 points lower than that for the pre–civil rights era cohorts, and mean neighborhood isolation was 5.30 points lower. Overall, change in the social and economic composition of the cohorts explained 4.94 points or 52.8 % of the total decline in dissimilarity and 3.14 points or 59.2 % of the total decline in isolation. The remainder of the overall decline in segregation is attributable to change in the coefficients for the covariates and unexplained change.

Improvements in black educational endowments and changes in the coefficients for the educational degree variables were the principal drivers of the decline in residential segregation between cohorts. Increases in black educational endowments between the pre–civil rights era and civil rights era cohorts explained 2.42 points or approximately one-quarter of the total decline in dissimilarity, and 1.59 points or 30 % of the decline in isolation. The dramatic decline in the share of the black population without a high school diploma (D: 1.80; I = 0.58) and the sharp increase in the share with at least a bachelor’s degree (D: 0.58; I = 0.48) accounted for most of the endowment-related change.

In addition, black integration was greater in the civil rights era cohorts because patterns of residential segregation by educational degree shifted (i.e., the coefficients for the educational degree variables changed). One-third of the total decline in dissimilarity (3.10 points) and nearly two-thirds of the decline in isolation (3.47 points) were attributable to greater integration for black high school graduates in the civil rights era cohorts. Increased integration for black college attendees was responsible for an additional 2.11 points (23 %) of the overall decline in dissimilarity and 1.89 points (36 %) of the decrease in isolation. Offsetting a portion of these gains was lower integration for blacks with the most and the least education. Lower integration for black college graduates in the civil rights era cohorts led to a 1.13-point drop in dissimilarity (–12 %) and a 1.38-point decline in isolation (–26 %), while higher segregation of black high school dropouts was responsible for a 1.96-point reduction in dissimilarity (–21 %) and 0.84-point decline in isolation (–16 %). Overall, approximately one-quarter (2.12 points) of the decrease in dissimilarity and nearly 60 % (3.15 points) of the decline in residential isolation was attributable to changes in the estimated coefficients for the educational degree categories between the pre–civil rights era and civil rights era cohorts.

Other traditional indicators of socioeconomic assimilation and mobility played small roles in the post-–civil rights era decline in residential segregation. The distribution of black income was largely unchanged across cohorts, and the change that did occur reflected an increase in the share of the black population in the bottom quintile. Thus it is unsurprising that intercohort changes in the distribution of income had a small and negative effect on black residential segregation in the later cohort (D = –0.29; I = –0.12). Changing patterns of segregation by income also had a limited impact on residential segregation in the civil rights era cohorts (D = –0.45; I = –0.76). None of the estimated effects of income in the decomposition analysis approached statistical significance. Lower welfare participation rates in the civil rights era cohort were tied to a statistically significant 0.77-point decline in dissimilarity and a 0.82-point decline in isolation.

Although no other factors had a statistically significant impact on the decline in black-white residential segregation, the results from the decomposition analysis suggest that the regional shift of the black population away from the Northeast and Midwest regions and toward the South region contributed modestly to the decline in segregation, and that declining marriage rates slowed progress toward racial residential integration.

Discussion

The two dominant frameworks for understanding racial residential segregation—the spatial assimilation and place stratification perspectives—emphasize differing mechanisms of neighborhood change. Spatial assimilation theory attributes the decline in black-white residential segregation over the last half-century to improvements in black social and economic attainments. By contrast, the place stratification perspective holds that improvements in residential segregation reflect a weakening of social and politico-economic processes that have historically constrained the ability of blacks to integrate with whites.

Methodologically, the mechanisms of change posited by these perspectives imply two different sources of change. The spatial assimilation perspective attributes change in residential segregation to changes in the population composition of blacks over time. By contrast, the place stratification perspective maintains that increases in integration are a result of changes in the association between socioeconomic characteristics and residential outcomes over time. Regression decomposition techniques offer a natural analytic solution for quantifying the relative prominence of these two sources of change because their purpose is to partition change into what can be attributable to compositional changes and what can be credited to changes in patterns of association.

We demonstrated the use of this analytic approach by decomposing change in black-white residential segregation at midlife for black cohorts transitioning to adulthood before and after the civil rights movement of the 1960s. One benefit of this approach is that it permits us to quantify how much of the total decline in residential segregation is attributable to spatial assimilation and place stratification processes. If we focus narrowly on the effects of education and income on residential segregation (because they are the most commonly used indicators of socioeconomic assimilation), we find that both processes played a prominent role in segregation trends. Approximately 25 % of the decline in black-white segregation reflected improvements in the educational and earnings endowments of blacks in the civil rights era cohort. As with European immigrants to the United States in the early twentieth century and Asian and Hispanic immigrants today, upward socioeconomic mobility for blacks coincided with greater spatial assimilation.

Regardless of their socioeconomic assimilation, blacks have confronted barriers to residential integration with whites that other racial and ethnic minority groups have not (Massey and Denton 1993). The Fair Housing Act of 1968 began the slow process of guaranteeing blacks equal opportunity to live where they chose. This process has been expanded and strengthened over time with efforts, such as the HOPE IV program to dismantle large public housing projects in segregated, high-poverty neighborhoods and, more recently, Executive Order 12892, which requires that policies of the Department of Housing and Urban Development are administered in a manner that affirmatively furthers the goal of fair housing. These policy changes, along with the liberalization of racial attitudes in the post–civil rights era, have weakened place stratification processes that led to a weaker association between SES and neighborhood attainment for blacks than for other groups. The decomposition approach quantifies the extent to which such changes have contributed to the decline in segregation. We found that 18 % of the decrease in dissimilarity across cohorts could be credited to the changing association between SES and residential segregation, compared with 45 % of the decrease in isolation.

The decomposition approach also uncovers which social and economic factors have been the principal drivers of change. The detailed regression decomposition reveals that improvement in residential integration for blacks in later cohorts is largely the result of the educational gains they made relative to earlier cohorts and the improved ability of blacks—especially high school graduates and college attendees—in these cohorts to relocate to more integrated neighborhoods. Overall, increases in the educational endowments of more recent black cohorts explain approximately one-quarter of the decline in black-white residential segregation, while changes in the coefficients for the educational degree variables account for an additional one-quarter of this decline.

Although our principal goal in this analysis was to determine the relative strength of spatial assimilation and place stratification processes in the overall decline in black-white residential segregation since the 1970s, our analysis revealed unexpected intercohort differences in residential segregation patterns. We found that black cohorts entering adulthood prior to the civil rights movement of the 1960s experienced consistently high levels of residential segregation at middle age. Mean residential segregation levels at middle age for blacks in the two pre–civil rights era cohorts, whose members turned 45 between 1968 and 1990, are nearly identical. Approximately 47 % of their black neighbors would have had to relocate to another neighborhood to achieve integration with whites, and roughly 67 % of their neighbors were black. By contrast, blacks in the two cohorts transitioning to adulthood during and well after the civil rights era, who turned 45 between 1991 and 2013, lived in neighborhoods in which 38 % of black residents would have had to relocate to another neighborhood to achieve racial equality, and 61 % of their neighbors were black.

These stark differences in residential segregation patterns between black cohorts transitioning to adulthood prior to and after the civil rights era suggest the need for greater attention in the residential segregation literature to the effect of historical conditions during this critical life stage on the formation of residential preferences, expectations, and decisions. The life course perspective (Elder 1974) and other generational models of social change (Mannheim 1952; Ryder 1965) highlight the importance of late adolescence and early adulthood in forming values and attitudes. Our findings suggest that the diverging social, economic, and historical conditions experienced by pre– and post–civil rights era black cohorts as they became politically and socially aware during late adolescence and early adulthood helped shape their residential attainment trajectories for the rest of their lives.

Our results also have important implications for understanding and interpreting long-term trends in metropolitan-level segregation indices. Studies of historical trends in black-white residential segregation using such indices consistently report a slow but steady decline in segregation (Logan and Stults 2011). Urban scholars have typically interpreted this gradual but steady decline in segregation indices as evidence of continued progress toward the goal of racial residential equality. Our findings suggest a competing explanation that may result in less optimism about future prospects for progress in black-white residential integration. We report a sharp increase in the integration of the black cohort transitioning to adulthood in the immediate aftermath of the civil rights era, relative to the pre–civil rights era cohorts. However, we find that progress toward racial residential integration stalled for the most recent cohorts. The neighborhoods of cohorts of blacks transitioning to adulthood decades after the civil rights movement were not significantly more integrated than the neighborhoods of the cohorts of blacks transitioning to adulthood during and immediately after the civil rights movement. This implies that the slow but steady decline in black-white residential segregation may not reflect continued racial progress so much as it reflects population turnover in which more highly segregated pre–civil rights era cohorts are replaced by less highly segregated post–civil rights era cohorts.