1 Introduction

For many reasons, the poor, elderly, and disabled are generally at much greater risk to natural disasters such as flooding and hurricanes than the general public.Footnote 1 The term social vulnerability is often used to describe this phenomenon. While there is no consensus as to the exact definition of social vulnerability, for the purposes of this paper we use Blaikie et al.’s (1994) definition that social vulnerability refers to the characteristics of an individual or group that influences their capacity to anticipate, cope with, resist and recover from a physical hazard. Since socially vulnerable groups are less resilient to physical hazards, they may face disproportionate losses from either a natural or man-made disaster. Our focus in this paper is on social vulnerability to natural hazards, particularly those that are likely to increase due to climate change and sea level rise. Following Hurricane Katrina, the Stafford Act, which directs how federal resources are used to respond to major disasters, was revised to require than any federal funds used to respond to a disaster must provide equitable treatment on the basis of race, sex, age, disability, English-language proficiency, and economic status.Footnote 2 Thus, as governments formulate or evaluate potential policy responses to impending sea level rise they will need to have an understanding of the social vulnerabilities in their communities to ensure such equitable treatment.

The literature on social vulnerability has its roots in the 1976 article in Science by O’Keefe et al., “Taking the Naturalness out of Natural Disasters.” In this article the authors argue that impact of disasters is as much, if not more, a consequence of socioeconomic factors, particularly what they term “underdevelopment,” than natural factors. This paper spurred a number of other papers and analyses examining socioeconomic vulnerability to physical hazards, primarily using a political ecology framework. Building on this initial literature, Blaikie et al. (1994) developed a “pressure and release model” of social vulnerability that examines the ways in which social, economic, and political processes influence how hazards affect different people. Their model focuses on the root causes and social processes that drive vulnerability and create the potential for unsafe conditions (“pressures”) and how to reduce the fundamental vulnerability of the community (“releases”). A separate strand of the literature focuses less on the root cause of social vulnerability and much more on the interaction between the exposure to physical hazards as a result of the geographic location of a community and the community’s social vulnerability. This “hazard of place” approach is outlined in Cutter (1996). More recently, authors like Turner et al. (2003) have used the framework of sustainability science to examine vulnerability, conceptualizing it as a coupled human–environment system and focusing attention on how to reduce vulnerability and build more adaptive and resilient communities. This approach considers the human and environmental conditions to be intertwined and interdependent and thus does not consider social vulnerability independent of physical conditions.

Even as the theoretical literature on social vulnerabilities continues to expand and evolve, there is concurrently increasing interest in measuring social vulnerability, however defined. One reason for this emphasis on measurement is that governments at all levels are increasing their attention to planning for and responding to natural hazards, particularly those associated with climate change. One of the key challenges in measuring social vulnerability is the wide range of approaches to social vulnerability and the lack of a consensus as to what exactly social vulnerability is. In this paper, we examine two different approaches to identifying and measuring social vulnerability using a case study of Hampton Roads, Virginia. The two methods examined are composite indexing using principal component analysis (PCA) and k-means clustering. The case study approach allows us to illustrate the strengths and weaknesses of these two methods using a concrete example, although the conclusions from our study should apply more globally.

We chose Hampton Roads as our case study because the area is one of the most threatened by sea level rise in the USA, and there are significant ongoing efforts to plan for and adapt to climate change. Hampton Roads stretches from Virginia’s Atlantic coast up the shore of the Chesapeake Bay and into the interior of the state. The area encompasses the cities of Norfolk, Virginia Beach, Hampton, Newport News, and Chesapeake. It is home to several US military installations, including the world’s largest naval base, and to one of the largest commercial ports on the East Coast. Relative to other coastal locations, Hampton Roads faces accelerated effects of sea level rise due to significant land subsidence and the prevalence of low-lying flood-prone terrain in the Chesapeake basin (Virginia Institute of Marine Sciences 2013). Several eastern Virginia communities have already seen a significant increase in the severity and frequency of tidal flooding and sea level rise is expected to bring even more tidal flooding and damaging storm surges to the area in coming decades.

The area includes both highly urbanized and very rural communities as well as some of the richest and the poorest neighborhoods in the state. In order to develop effective programs to adapt and respond to the impacts of climate change and sea level rise that meet the equity requirements of the Stafford Act, policymakers must consider the needs of all of the region’s residents and stakeholders, particularly those who are least able to provide for themselves in the event of disaster. It is this need which motivates this paper’s critical examination and refinement of methods for measuring social vulnerability to climate change and sea level rise. While we are primarily interested in developing a method to quantitatively estimate social vulnerability to help local and state planners determine how to best marshal limited resources to adapt to sea level rise while ensuring that resources are targeted effectively and equitably, our analysis has implications beyond sea level rise and climate change. In particular, our examination of methods for quantifying social vulnerability should also apply to other types of natural and man-made hazards that result in significant damage to physical infrastructure and disruption of communication, transportation, and utility networks.

The remainder of the paper is organized as follows. The next section provides a survey of existing quantitative methods for identifying socially vulnerable populations with a focus on composite indexing using principal component analysis (PCA), one of the most commonly used methods of measuring social vulnerability. We then present an analysis of the strengths and weaknesses of PCA-based indexing and identify some caveats to its use. The third section presents our case study of Hampton Roads which compares PCA-based indexing to an alternative method that uses k-means clustering to identify socially vulnerable communities, considering the merits and limitations of the two methods. The paper concludes with a discussion of our findings as well as recommendations for researchers and practitioners looking to measure social vulnerability to climate change and sea level rise.

2 Measuring social vulnerability: SoVI® and beyond

In the mid-2000 s, a group of researchers at the University of South Carolina (USC) led by Susan Cutter developed an approach to quantifying social vulnerability to environmental change and natural disasters (Cutter et al. 2003). This approach, known as the social vulnerability index, or SoVI®, uses principal component analysis to identify common factors underlying an assortment of potential vulnerability indicators. PCA is a statistical technique that takes as its input a matrix of interrelated socioeconomic variables—in this case those considered to measure various dimensions of social vulnerability—and creates a set of principal components that extract the important variation in the underlying input data while reducing the noise and redundancy in the data. After conducting the PCA, the researcher combines the newly created component variables into a composite index that provides a single value for each observation in the dataset, in this case a social vulnerability score. The utility of a PCA-based index is that it encapsulates a lot of information in an easily consumed form and individual observations can be ranked relative to each other.

More specifically, prior to conducting a PCA, variables in the input matrix are typically standardized to z scores with zero means and unit variances to avoid any confounding effects that might arise from using variables of different magnitudes in the analysis. The PCA is run on the standardized input matrix and generates a new set of orthogonal principal components. The researcher then decides which principal components to integrate into the final index and how to do so. In particular, the researcher must determine the directionality of each retained component, that is, whether higher values of the component increase the index value (positive directionality) or decrease it (negative directionality). The components are then added together either with or without weights. To ensure that the indices are easily comparable to each other, most final indices are standardized to z scores with zero means and unit variances.Footnote 3

A review of the existing academic literature on the construction of quantitative social vulnerability indicators reveals that since its creation, SoVI® in particular, and PCA-based composite indexing more generally, have become one of the most common paradigms for quantitatively identifying social vulnerability to environmental hazards, particularly for studies that focus on natural hazards associated with climate change such as hurricanes, storm surge, flooding, and coastal erosion. Consequently, a fairly robust academic literature using PCA-based indices has developed, as shown in Table 1.Footnote 4 PCA-based indices are also widely used in tools directed at policy practitioners and planners. For example, the state of California has incorporated a version of SoVI® into its State Hazard Mitigation Plan (California Emergency Management Agency 2013) and the Pacific Institute, a nonprofit research organization focusing on environmental issues uses a 19-variable census tract-level SoVI® replica for the state of California (Pacific Institute 2016). At the national level, the National Oceanic and Atmospheric Administration (NOAA) includes the county-level SoVI® as one of the available layers in its public sea level rise risk assessment mapping tool (NOAA 2016). Similarly, the nonprofit group Climate Central has included SoVI® in its Web-based application for identifying communities at risk from sea level rise (Climate Central 2016).

Table 1 Survey of SoVI® and PCA-based indices
Full size table

While SoVI® and related PCA-based indices have the advantage of replicability, transferability, and general acceptance in the academic literature, there are some caveats to using them to measure social vulnerability. First, the strength of the PCA method—that it reduces a complex matrix of information to a singe number—is also one of its weaknesses, as the index can be difficult to interpret and the methodology for computing the index is not transparent to most users of the index. More specifically, as we discuss below, users are unlikely to understand how the selection of input variables, the use of component selection and weighting criteria, and considerations of scale influence the resulting index.Footnote 5 Additionally, while the PCA process results in a unique set of components, the researcher must ultimately make subjective decisions about how to aggregate those components into the final index, which creates the potential for two researchers to develop different indices from the same underlying set of input data.

2.1 Complexity and interpretability of PCA-based indices

One of the key challenges facing practitioners who wish to capture the dimensions of social vulnerability quantitatively is that vulnerability depends upon many interacting factors and the specific context of a particular situation or hazard type. Attempting to account for a wide range of distinct factors by including them explicitly in a single composite index introduces complexity into the model, which can make the resulting index difficult to interpret and apply. As shown in Table 1, the number of variables included in PCA-based indices has ranged from a low of 8 to a high of 64 with most studies using between 25 and 40 variables. Technically, PCA can successfully capture subtle variations in the input data, which may encourage researchers to take an inclusive approach to collecting data that capture the many complex dimensions of social vulnerability. Existing analyses of both applied and simulated index construction, however, suggest that a PCA-based index constructed from many factors, most of which by themselves have weak explanatory power in predicting specific outcomes cannot necessarily distill meaningful value from fundamentally noisy data.Footnote 6 Additionally, the larger the number of input variables, the more likely the index will suffer from unexpected correlation between input variables, bidirectional vulnerability signals (where both high and low values of a variable indicate vulnerability), and differences in the true measures of vulnerability across heterogeneous populations. These types of interactions, which we refer to collectively as confounding variable effects, may bias the final index in many different ways.

For example, a variable measuring the proportion of mobile homes in a community may be an important indicator of vulnerability in rural areas, but including a principal component in the final index that is heavily influenced by that variable will result in a lower social vulnerability index for urban areas where mobile homes are not an important concern. Alternatively, including measures such as median age where vulnerability is a concern at both tails of the distribution can result in a noisy signal. While there are solutions to this, such as Fekete's (2012) recommendation to use binary indicators for extreme values of specific variables that signal potential vulnerability on both ends of the scale, many indices continue to use these types of variables. By limiting the input variables to only those that capture or are strongly correlated with the social vulnerability factors of greatest interest based on review of the relevant literature and engagement with local communities, researchers can minimize potential distortions from confounding variable effects. However, as shown by the large number of variables used in most of the indices listed in Table 1, the tendency seems to be to err on the side of inclusion rather than exclusion.

One argument for including a large set of variables in a PCA-based index is “to avoid both pre-principal components analysis data loading and ex post-data adjustment that could impart a bias in the results used for decision making” (Cutter et al. 2013, p. 342). However, subjective considerations of data loading are unavoidable in constructing a PCA-based index as such indices necessarily reflect the relative influence of the variables that were inputs into the PCA process. As Burton and Cutter (2008) note, composite indices “do not always capture the most important or ideal factors, but rather those that best explain the variation in input data” (p. 141). Thus the contribution of a particular dimension of vulnerability to the overall index depends in part on how many indicators of that dimension are included in the input matrix. For example, in Baum et al. 2008 study, only two out of 19 input variables measure income and thus income-related measures explain only a small share of the total variance in the input data and ultimately have a relatively small impact on the final index. On the other hand, when financial security is measured by the selection of many input variables such as income, home value, rent, and poverty level, financial security is likely to have a larger effect on the final index. Ultimately, choosing to include a variable in an index can alter the results in the same ways that excluding a variable can. Unfortunately, in many situations it can be difficult to come up with an observable measure of social vulnerability against which to externally validate a researcher’s choice of variables. For example, while there might be a well-defined outcome measure for some hazards such as the deaths associated with heat waves, in the case of recurrent flooding and storm surges from sea level rise one is likely interested in a range outcomes including deaths, changes in physical and mental health, property damage, increased cost of home maintenance, and lost wages, some of which might be quite difficult to measure.

In addition to potentially weakening the vulnerability measure, increasing the number of input variables to capture a wide range of vulnerability concerns also reduces the interpretability of the composite index as it becomes very difficult to trace how changes in a single factor influence the overall index. Thus, researchers and practitioners should be aware of all of the possible trade-offs associated with the inclusion of additional measures of social vulnerability and need to convey those tradeoffs to users of the index.

2.2 Component selection and weighting criteria

Another key consideration researchers must make in constructing a PCA-based composite index is which output components will be used in the index and how they will be weighted in the aggregation. As shown in Table 1, most indices we surveyed use the Kaiser selection criterion, in which components from the PCA are incorporated into the final matrix if they have an eigenvalue greater than 1.0 (i.e., the component explains more than 1/nth of the total variance of the input matrix, where n is the total number of input variables). Most indices then equally weight the retained components in the final index. Finch et al. (2010) defend equal factor weights as the most logical method for aggregating the retained components into the final index because “the purpose of the SoVI index generally…is to provide an aggregate measure of all the factors that contribute to the social vulnerability of the parish, not just one or two individual variables or components” (p. 184). We believe that weighting each factor according to its share of explained variance in the sample is preferable to equal weighting for two primary reasons. First, if one selects and combines the input variables in proportion to perceived overall contribution to vulnerability, variance weighting will preserve this choice while equal weighting will not. Second the Kaiser selection criterion used by most researchers is inconsistent with equal weighting. That is, if the researcher believes that the explained variance is relevant enough to merit its use as a component selection criterion, it should also be relevant enough to merit its consideration in weighting those components. Moreover, our case study, which is discussed in detail in Sect. 3, shows that variance-weighted indices are more consistent when there are changes in the geographic scope of analysis and in the input variables than equal-weighted indices.Footnote 7

2.3 Scale

Another issue that researchers and practitioners should consider when constructing a vulnerability measure is scale. Scale includes both the level of observation (e.g., census tract vs. county) as well as the overall geographic boundaries of the analysis. For some data sources such as the American Community Survey, data for the smaller geographies (census block groups), are more noisy than data at larger geographies (tracts and counties), due to the sampling method used to collect data. Moreover, some data sources may only have information for larger geographies. With respect to the geographic area spanned by the analysis, for the most part that is driven by the nature of the study. However, trying to capture the important dimensions of vulnerability when the factors that contribute the most strongly to vulnerability vary significantly across observations increases the likelihood of confounding variable effects, and larger geographic areas are more likely to face this problem than smaller and more homogeneous ones. Our case study examines the consistency of various PCA-based indices to variations in geographic boundaries, but not to variation in the granularity of the analysis.Footnote 8 In our case study presented in Sect. 3, we show that PCA-based indices are sufficiently sensitive to the drawing of a different set of boundaries to confound their utility even at relatively small changes in geographic boundaries, such as from Hampton Roads census tracts to Coastal Virginia census tracts.Footnote 9

2.4 Alternatives to PCA-based indexing

Several alternatives to SoVI® and PCA-based indices have been offered in the literature on social vulnerability to natural hazards. Generally these alternatives fall into three categories: separate consideration of individual vulnerability measures; simple aggregation of a small set of variables; and measurement of similarity to a reference district determined to be vulnerable. As an example of the first type, Wang and Yarnal (2012) consider individual factors that contribute to social vulnerability separately and map them to identify potential vulnerability hot spots. The method thereby avoids the problem of weighting factors, but at the cost of added complexity in the final measure and reduced transferability to other regions for comparison and validation. Additionally, this method does not allow one to rank locations relative to each other in terms of vulnerability.

As an example of the second category, Chakraborty et al. (2005) construct a composite Social Vulnerability for Evacuation Index (SVEAI), which is a simple additive aggregation of standardized variable values for a set of 10 social indicators thought to represent vulnerability dimensions for assessment of hurricane evacuation planning in Hillsborough County, Florida. As in other composite indices of social vulnerability, differential access to resources, as measured by both poverty and specific indicators of communication and transportation access, is an important consideration and serves as a proxy for a great deal of other correlated vulnerability signals. As is the case with PCA-based index, this method does provide a means for ranking observations relative to each other. As an example of the third approach, Chang et al. (2015) present a composite social vulnerability indicator meant to assess the similarity of administrative districts in British Columbia along a several different dimensions of vulnerability. Their Hazard Vulnerability Similarity Index (HVSI) measures the similarity of districts in their analysis to a chosen reference district in order to facilitate knowledge sharing among areas facing similar issues. The cluster analysis method we apply in our case study builds on this similarity approach, using a data mining technique to group similar communities within Hampton Roads.

3 Hampton Roads case study

In this section we present a case study for measuring social vulnerability in Hampton Roads, Virginia. As discussed in the introduction, Hampton Roads is under serious threat from sea level rise and climate change. As localities work on adaptation strategies and mitigation plans, there is pressure to ensure that these plans are informed not only by physical vulnerability, but also by social vulnerability to natural hazards. In this case study we compare a PCA-based approach to quantifying social vulnerability to an alternative method that uses k-means clustering to group similar communities. The resulting clusters are then evaluated with respect to predictors of social vulnerability to determine whether the communities in each cluster are socially vulnerable. In our comparison of the two methods, we focus primarily on the consistency of the methods under changes in the set of input variables and geographic scope. In addition we consider the consistency of the PCA-based indices under changes in the weighting method.Footnote 10

For the purposes of local planning for climate change and sea level rise, it is important to identify socially vulnerable communities at a reasonably small geographic scale. However, as is the case with most quantitative analyses of demographic data, the level of spatial aggregation often depends on the data available, rather than a methodological decision about the appropriate level at which to measure social vulnerability. This study is no different. In order to use a wide range of variables that have been identified by other studies as important indicators of social vulnerability, the smallest geographical unit that we feel comfortable using are census tracts.Footnote 11 For this analysis we consider the 1874 census tracts in Virginia that have at least 100 or more inhabitants. Including census tracts across all of Virginia allows us to explore the extent to which using more observations as well as a more diverse set of observations to construct the vulnerability measure affects the consistency of that measure.

3.1 Construction and evaluation of the PCA-based indices

For this case study we constructed eighteen different PCA-based composite social vulnerability indices that vary across three dimensions: the geographic boundaries of the analysis, the set of input variables, and the weighting of the components generated by the PCA. With respect to geographic boundaries, we consider three different sets of census tracts as shown in Fig. 1: all 1874 census tracts in Virginia with at least 100 inhabitants; the 1172 of those census tracts in Virginia’s coastal counties; and the 408 of those census tracts in the Hampton Roads area. We consider three different sets of input variables: The largest set includes 41 variables chosen to replicate the original Cutter et al. (2003) SoVI® index, measured at the census tract-level where available and at the county level otherwise; the next set includes only the 29 variables from the first set that can be measured at the census tract-level; and the smallest set includes 13 variables, all census tract-level, considered by us to be the most direct determinants of social vulnerability, including measures accounting for income, race, ethnicity, age, and gender.Footnote 12 Table 2 presents the variables used in each set.Footnote 13 Finally, we considered two different methods for aggregating the PCA-derived components: an equal-weighted index which gives the same weight to each of the PCA output components used in the analysis and a variance-weighted index that weights each component according to the total variance it captures from input variables. While the Cutter et al. (2003) set of input variables were purposefully constructed to measure social vulnerability to general environmental hazards and our selection of the limited set of 13 variables was made in the context of sea level rise and climate change, it is likely that similar sets of variables would be appropriate for quantifying social vulnerability to other types of natural and man-made hazards. Thus we expect the insights from our analysis to apply broadly to most PCA-based quantifications of social vulnerability.

Fig. 1
figure 1

Map of all Virginia, Coastal Virginia, and Hampton Roads census tracts

Full size image
Table 2 Variables used in the social vulnerability measures
Full size table

Before conducting the PCA, the variables were first standardized to z scores with zero means and unit variances, as is common in the literature, to avoid confounding effects that can occur from using variables of different scales.Footnote 14 For each combination of geographic area/input set we conducted a PCA, keeping those principal components with eigenvalues greater than 1 (the Kaiser selection criterion) as most of the studies cited in Table 1 do. The results of the PCA process are presented in Table 3. As expected, more complex input sets are reflected in more retained components. Since each component contributes to the final index, indices with more components are harder to interpret. Also note that while additional retained components do increase the total amount of variance explained, the difference in variance explained by the retained components does not decrease substantially even when the number of variables decreases substantially. The geographic area does not have a significant effect on the number of components, although the retained components do explain more variance when the geographic area is smaller and less heterogeneous.

Table 3 Results of the principal component analyses
Full size table

As a next step, we conducted a Varimax rotation of the components to facilitate interpretation of each component because—as is the case with all PCA-based indices—we must determine the directionality of each retained component, that is whether higher values of the component increase the level of social vulnerability (positive directionality) or decrease the level of social vulnerability (negative directionality). Where the directionality of the component was clearly negative, we scaled the component by a factor of −1 before including it in the composite index so that higher values of the scaled component would increase the overall vulnerability index. Following Cutter et al. (2003), in instances when the effect of the component on vulnerability is ambiguous (as is the case when the different variables that make up the component work in opposite ways or when a variable exhibits bidirectionality), we assume a positive directionality. For the equal-weighted indices, the components are added together without weights while for the variance-weighted indices the components are multiplied by the variance each component captures from the total input matrix before aggregation. To ensure that the indices are easily comparable to each other, we standardize the resulting aggregated values to z scores with zero means and unit variances.

Following Cutter et al. (2013) we compare the indices using the Pearson’s R correlation coefficient. Correlations above 0.90 indicate a strong positive correlation between the two indices while correlations below 0.70 indicate substantial inconsistencies.Footnote 15 As one might expect the various equal-weighted indices are highly correlated with the analogous variance-weighted indices. Seven of the nine pairs of indices have correlation coefficients above 0.90 while one pair has a coefficient on 0.82 (the 29-variable index for Hampton Roads) and one has a coefficient of 0.75. However, when we compare the indices based on geographic area or input set, overall the correlations are much lower. Table 4 shows the Pearson’s R coefficient for pairwise correlations among the various PCA-based indices that we constructed. The top half of the table shows the correlations across equal-weighted indices, while the bottom half shows the correlations across variance-weighted indices. For the equal-weighted indices, most pairwise comparisons result in a correlation coefficient below 0.70. There are two general exceptions—the coastal Virginia and Hampton Roads indices that use the smallest set of input variables (CV13 Eq and HR13 Eq)—are reasonably correlated with all of the overall Virginia indices and the HR13 Eq index is reasonably correlated with the CV41 Eq index and highly correlated with the CV13 Eq index. Overall the variance-weighted indices show higher correlations than the analogous pairs of equal-weighted indices, with the CV13Var and HR13Var indices showing the highest correlations. Thus the 13-variable indices show greater consistency under changes in geographic boundaries than the indices using larger numbers of variables. The 29- and 41-variable indices are much more inconsistent, and in one case the correlation between pairs of indices using 29 variables is actually negative, demonstrating the cumulative effects of confounding variables. In particular, we note that both the 29- and 41-variable sets contain potential bidirectional variables such as age, percent urban, and net internal migration while the 13-variable set does not. Additionally, we note that using variance-weighted principal components provides a more consistent measure across both different geographic scales and different input sets.

Table 4 Pearson’s R correlation for the PCA-based indices
Full size table

Looking at the correlations between indices is not the only way to compare them. In addition to using a PCA-based index to rank observations relative to each other in terms of vulnerability, many studies also use their index to identify a group of “highly vulnerable” observations. Various papers have implemented different decision rules for determining which observations are considered to be highly vulnerable, as shown in the last column in Table 1.Footnote 16 By far the most common method is to use a threshold value calculated using the mean and standard deviation of the index. Often, but not always, the threshold value is 1 standard deviation above the mean, which is the threshold we use for this analysis. Table 5 compares the number and percent of census tracts that are designated as highly vulnerable using this decision rule for the various indices in our case study. We also include a second category of “marginally vulnerable” tracts. These are defined as tracts with an index value between 1 and 0.5 standard deviations above the mean.

Table 5 Number and percent of tracts determined to be highly or marginally vulnerablea by PCA-based indices
Full size table

The percentage of tracts identified as highly vulnerable ranges from a low of 12 % to a high of 17 %. This relative consistency is expected given the nature of the decision rule and the fact that the index values are standardized with a mean of 0 and a standard deviation of 1. When we include marginally vulnerable tracts, the percentage ranges from 25 to 31 %. For most geography/input set combinations, the equal-weighted and variance-weighted indices identify roughly the same number of highly vulnerable or marginally vulnerable tracts. However, when one considers the overlap of tracts identified as highly or marginally vulnerable across the two weighting schemes (reported in columns three and six of Table 5), it is clear that while there is significant overlap, it is not perfect. Looking at the overlap of highly vulnerable tracts across indices that share the same geography and weighting (rows four, eight, and twelve), the number of consistently identified tracts is even lower. As was the case for the pairwise correlations, the variance-weighted indices appear to be somewhat more consistent under changes to the number of input variables than the equal-weighted indices.

A final comparison looks at the effect of geographic boundaries on the identification of highly vulnerable and marginally vulnerable tracts. For this comparison, we only consider the 401 counties that are in both the Hampton Roads and Coastal Virginia geographies.Footnote 17 Using the variance-weighted indices for the three different geographies, we determined how many census tracts are consistently identified as highly vulnerable for the various input sets. When the 41 variable input set is used, only 11 of the 401 tracts or 3 % are consistently identified as highly vulnerable across the three different geographic boundaries, even though typically between 10 and 15 % of tracts are identified as highly vulnerable for a given index. When we compare the indices for the 29 variable input set, 24 of the 401 (6 %) tracts are consistently identified as highly vulnerable and when we compare the indices for the 13 variable input set, 65 of the 401 (16 %) tracts are. Repeating the same exercise but counting tracts that are classified as either highly or marginally vulnerable, for the 41 variable input set, 34 of the 401 tracts or 8 % are consistently identified as highly or marginally vulnerable across the three different geographic boundaries compared to ranges of 25–31 % of tracts for a given index. For the 29 variable input set, 49 of the 401 (12 %) tracts are consistently identified as highly or marginally vulnerable and when we compare the indices for the 13 variable input set, 100 of the 401 (25 %) tracts are.

Overall, our findings indicate that the 13-variable indices are more consistent measures of social vulnerability. Additionally, using the 13-variable index eases the interpretation both of each PCA output component and of the overall index itself as each of the components map to specific input variables. In our case study, the PCA process for the 13-variable index results in an income/race component explaining about 30–40 % of total variance, as well as age, ethnicity, and gender components that each explain about 10–15 % of the total variance. Larger input matrices result in component scores that are difficult to interpret because of the larger number of inputs loading onto each component, particularly for components with lower eigenvalues where meaningful patterns in variables loading onto each component become more difficult to identify. Using smaller numbers of input variables reduces both statistical noise and subjective influence in interpreting components and hence in calculating a PCA-based index. Of course, reducing the number of variables included in the index does affect its ability to measure multi-dimensional social vulnerability. Thus the researcher needs to carefully weigh the trade-offs when deciding which variables to include or exclude from a particular index, as increasing the dimensions of the input matrix is not a costless decision.

3.2 Description and evaluation of the k-means clustering approach

Given the limitations associated with using a single index variable—however constructed—to capture such a multi-dimensional concept as social vulnerability, we examined the potential for using cluster analysis as alternative method for identifying socially vulnerable communities.Footnote 18 Like the PCA technique, cluster analysis also takes a large matrix of data and reduces it to a more manageable and arguably more meaningful measure.Footnote 19 However, rather than reducing the data matrix to a single ordinal value, cluster analysis groups “like” observations into clusters that share common characteristics. The researcher can then look at the characteristics of each cluster and determine whether tracts in that cluster are socially vulnerable. One of the advantages of this approach is that it identifies tracts that may be vulnerable in only one or two dimensions. It also allows factors to be considered holistically so that a factor that may not contribute to vulnerability in urban locations but would in rural locations can be included in the analysis. Similarly, if the concern is about the health implications of increased storm surge or recurrent flooding one may be interested in different factors than if one is concerned about financial implications of sea level rise. Of course, cluster analysis does have its own limitations. Cluster analysis certainly requires the researcher to make subjective judgments about the relative importance of different factors and how such factors might interact to increase or decrease social vulnerability to a particular hazard. Additionally the clustering process is not perfect—if a researcher uses too few clusters, the clusters may not be similar enough to result in accurate assessment. On the other hand, if the researcher uses too many clusters, the identification process becomes unwieldy. Finally the clustering process is more time intensive than construction of a PCA-based index, both in processing time and the time required for a researcher to evaluate each cluster. Thus for projects with large set of observations to evaluate, such as all counties in the USA, it may not be feasible to conduct.

At the most basic level, the underlying objective of cluster analysis is to group observations so that the observations in each cluster are more similar to other observations in that cluster than to objects in other clusters. Each cluster is represented by a prototype. The exact nature of the prototype depends on the particular clustering algorithm used. For our analysis, we use a k-means clustering algorithm where K is the number of clusters that are identified in the data and is a parameter in the analysis chosen by the researcher. In this algorithm, K observations are randomly selected as initial prototypes. Each observation is assigned to the cluster that has the “closest” prototype where proximity is measured as Euclidean distance between the observation and the prototype. Once observations have been assigned to an initial cluster, the algorithm computes the centroid, or mean of the observations, for that cluster and uses that as the cluster’s new prototype. The observations are reassigned to the closest prototype and the centroid is recomputed. This process is repeated until neither the centroids/prototypes nor the assignments change.

K-means clustering has the advantage that it is relatively straightforward on a conceptual level. Of course there are some drawbacks to using this method. First, randomly choosing the initial prototypes can result in selection of a local, rather than global, minimum sum of squared distances. Thus results are not always replicable when repeated with a different initial selection of prototypes. The typical solution to this, which we do in our analysis, is to perform multiple runs with different initial prototypes and select the one with the smallest sum of squared distances (SSD) (the squared distance between each observation and its assigned prototype distance). To determine how many runs to perform, we examined the improvement in the clustering process that resulted from additional runs and determined that after 500 runs, the likelihood of a significant improvement in the clustering process was quite small.Footnote 20

A second concern is the clustering might not be able to identify sufficiently homogeneous clusters either because too few clusters have been specified by the researcher or because the underlying observations are too heterogeneous to be clustered. If that is the case, clustering can result in mischaracterized observations. To determine the optimal number of clusters we used the η 2 measure to assess the improvement in the clustering process that resulted from additional clusters (Makles 2012). For each of the nine different geographic boundaries/inputs set combinations discussed in the prior section, we began with 20 clusters and computed η 2 for the run with the smallest SSD. We then increased the number of clusters by 5 to 25, completed 500 runs, calculated the η 2 for the best run where k = 25. As long as the increase in η 2 exceeded 2 % points we kept increasing the number of clusters by 5 until doing so increased η 2 by less than 2 % points. Thus the number of clusters used for each geography/input set is the highest multiple of 5 that increases η 2 by at least 2 % points. For VA41, VA29, CV41, HR41, and HR29 we use 30 clusters, for CV29 we use 35 clusters, and for VA13, CV13, and HR13 we use 25 clusters.Footnote 21

As was the case for the PCA-based indices, before conducting the cluster analyses we first standardized the variables relative to all of the Virginia census tracts. However, in evaluating a particular cluster’s profile to determine whether it was socially vulnerable or not, we examined both the absolute and standardized values of the variables. Thus we know not only the true level of the variable for each prototype but also how that relates to the average for Virginia. For example, consider the three prototypes presented in Table 6. What is shown are the absolute values for each variable. The prototype for Cluster A exhibits very low per capita income and very high poverty and unemployment rates. This area is a densely populated urban community and contains a large number of renters. Both housing values and rents are lower than average. There is low labor force participation overall relative to the mean for Virginia, although this is not a particularly elderly community, and over a third of adults do not have a high school degree. A very high percentage of the community is black and many families have female head-of-households. For all of these reasons, we classify this cluster as a vulnerable cluster. The prototype for Cluster B is a sparsely populated rural area although not many of the citizens in the community are farmers. The community has a moderately low per capita income, with just over 15 % of the community below the poverty line and almost 10 % unemployed. Education levels are low as over a quarter of adults do not have a high school degree. Labor force participation is relatively low, but this is consistent with the fact that almost one-fifth of the population is over 65 and almost 40 % receive social security benefits. While this prototype exhibits many factors that are indicative of social vulnerability, most of the levels are not extreme and thus we classify this cluster as marginally vulnerable. The prototype for Cluster C is a dense urban area with relatively high levels of internal migration and a significant increase in population between 2000 and 2010. The area is characterized by a large Hispanic population compared to the rest of Virginia. Although per capita income is lower than the state average and only around a fifth of the adults have a high school education, there is a high level of labor force participation and unemployment is lower than average. The area has a lot of new building permits and high levels of economic activity. A relatively small percentage of the population is dependent on social security income. This cluster is classified as not vulnerable.

Table 6 Sample clusters for all of Virginia using all 41 variables
Full size table

As shown by the examples above, a cluster approach clearly requires that the researcher use his or her judgment in deciding whether or not a prototype should be classified as socially vulnerable. While some might disagree whether or not a specific prototype represents a socially vulnerable community, the decision rules followed by the researcher can be laid out in the analysis and thus the rationale for classification and any potential biases and subjectivity will be transparent. For our analyses, we reviewed each cluster prototype and made a determination of vulnerability based on the prototype’s full characterization, that is, all of the variables used in the cluster analysis. As noted above, we looked at both the absolute value of each variable as well as how that value related to mean level for the rest of Virginia. We then looked at commonalities across the highly and marginally vulnerable clusters to identify the minimum common characteristics across each cluster that could describe why particular clusters were categorized as highly or marginally vulnerable to sea level rise and climate change. We believe these rules could be useful for others looking to classify communities as socially vulnerable but note that context is important in determining which clusters fall into which categories, as what might indicate lack of representation or resources in Virginia today may or may not be the same in other geographic areas or at other times.

Ultimately, a cluster was categorized as highly vulnerable if its prototype met any one of the following conditions:Footnote 22

  • Median income less than $20 K, percent living in poverty greater than 20 %, AND either percent unemployment greater than 10 % OR labor force participation less than 66 % OR female labor force participation less than 50 % (lack of resources).

  • Median income less than $20 K, percent living in poverty greater than 20 %, percent of the population without a HS degree is more than 25 %, AND percent black greater than 50 % (lack of resources, lack of representation).

  • Percent of the population over 65 is greater than 20 %, the percent of the population in nursing homes is greater than 2 %, the percent of the population receiving social security is greater than 25 % and median income is less than $25 K (lack of resources, high need).

Marginally Vulnerable clusters are identified as clusters that are not categorized as vulnerable but where the prototype meets one of the following conditions:

  • Percent of the population without a HS degree is more than 25 %, the percent of the population receiving social security is greater than 33 %, and the percent of housing units that are mobile homes is greater than 15 % (lack of resources, high need, vulnerable shelter).

  • Percent of the population without a HS degree is more than 25 % and there are high populations of under represented races (relative to Virginia as a whole), i.e., the percent black is greater than 50 % OR the percent Hispanic is greater than 15 % or the percent of Native Americans is greater than 1 % (lack of representation, lack of resources).

  • The percent of the population that is black is greater than 50 % and the percent of households that are headed by a female is greater than 20 % (lack of representation, high need).

  • The percent of the population in nursing homes is greater than 2 % (high need).

  • The percent of the population that is over 65 is greater than 20 % and the percent of the population receiving social security is greater than 33 % but median income is greater than $25 K (high need).

  • The percent of the population living in poverty is greater than 20 % (lack of resources).

  • Median income less than $20 K, less than 66 % labor force participation, and percent of the population without a HS degree is more than 25 % (lack of resources).

In the above decision rules, note that each highly or marginally vulnerable cluster represents populations facing one or more dimensions of social vulnerability that could make adapting and responding to climate change more difficult or which could exacerbate the impacts of sea level rise on health or financial outcomes: lack of resources, lack of political representation or access to government services, high levels of need, or vulnerable shelter. Each of these dimensions is represented by several different variables in our decision rules (and a larger set of variables in the overall dataset). For example, lack of resources is manifest not just in the income-oriented variables (percent in poverty, per capita income, percent unemployed) but also through limits on earning potential (percent without HS degree, percent female labor force participation). Lack of representation and access to government services is manifest in minority populations—for Virginia this means blacks, Hispanics, and Native Americans. High levels of need are measured by the percentage of the population over 65, in nursing homes and on social security. With respect to vulnerable shelter, we use both the percent of renters (as renters have little control over their shelter relative to homeowners) as well as the percent of mobile homes.

Table 7 presents the results of the clustering analysis for each of the geography/input set combinations. While the decisions rules for identifying clusters as vulnerable are based on the levels of poverty, unemployment, etc. in a tract, we note that determining whether a tract is vulnerable is not tied to the statistical properties of the input matrix. Thus, as shown in Table 7, the percentage of tracts that are identified as highly or marginally vulnerable changes for different geographies. For example, Coastal Virginia tends to have a lower percentage of tracts in the two vulnerable categories than Hampton Roads or Virginia overall, which reflects the fact that a high proportion of the Coastal Virginia tracts are in the wealthier and more developed Northern Virginia area.

Table 7 Number and percent of tracts determined to be highly or marginally vulnerable using the cluster method
Full size table

To assess the performance of the clustering method, we examine the ability of this method to consistently identify vulnerable tracts regardless of the level of geography chosen or the number of variables used in the analysis, just as we did for the PCA-based indexing. As shown in Table 7, for the Hampton Roads geographies, there is a reasonable level of consistency across the different input sets in terms of identifying highly vulnerable tracts. Well over half of the tracts identified as highly vulnerable in any one cluster analysis are also considered highly vulnerable in the other analyses for the same geographies. The Virginia and Coastal Virginia analyses are not as consistent. We see the same patterns with respect to the group of highly or marginally vulnerable tracts. In the Hampton Roads analyses, 105 tracts are consistently identified as either highly or marginally vulnerable, which represents well over three quarters of the tracts identified by any one cluster analysis. However, for Virginia overall only 317 tracts (12 %) are consistently identified as highly or marginally vulnerable which represents less than two-thirds of any one cluster analysis. Similarly, for Coastal Virginia 136 tracts are consistently identified as vulnerable, which is less than half of the tracts identified as vulnerable for the CV29 analysis. Adding together the percentage of tracts consistently identified as vulnerable (highly or marginally) and consistently identified as not vulnerable, for Virginia overall 73 % of tracts (17 % + 56 %) are classified consistently across all three variable sets.Footnote 23 For Coastal Virginia, 85 % (12 % + 73 %) are classified consistently and for Hampton Roads, 91 % (26 % + 65 %) are. These results are support our expectation that the clustering method is more consistent for smaller and more homogeneous geographies. Looking across the three input variable sets, no clear patterns emerge in Table 7. Obviously the number of variables in the input matrix does affect the determination of which tracts are clustered together as well as the number of clusters, which ultimately affects the identification of tracts as either highly or marginally vulnerable. However, from Table 7, it is not clear that using a particular input variable set is preferred to using one of the others.

Table 8 also examines the total percentage of tracts consistently identified as vulnerable (highly or marginally) and consistently identified as not vulnerable, but does so using pairwise comparison across all of the geographies/input sets. For most comparisons, the percentage of tracts consistently characterized is in the 80 s or low 90 s. Also note that the percentage of tracts that are consistently characterized is slightly higher for the 13-variable and 29-variable pairs than it is for the analogous 41-variable pairs. However, the differences are not dramatic, as one might expect given that the clustering method is not subject to confounding variable effects in the same way that PCA-based indexing is. Although one concern with clustering is that it might not be able to identify sufficiently homogeneous clusters because the underlying observations are too heterogeneous to be clustered, given the consistency of the clustering analysis to different geographies and sets of variables, we feel confident that the clustering method does significantly help a researcher to identify potentially vulnerable populations. Even though clustering is more consistent under changes in the input set than PCA-based indices, we continue to recommend that researchers carefully weigh the benefits of including additional variables. Finally we note that the clustering approach may be particularly useful in cross-jurisdictional analyses because it identifies groups that may be facing similar challenges and would facilitate sharing of ideas and resources across jurisdictions.

Table 8 Overlap of cluster analysis vulnerability classifications: percent of tracts consistently categorized as highly/marginally vulnerable or not vulnerable
Full size table

3.3 Comparison of composite indexing and clustering as identification strategies for social vulnerability

To compare the two methods to each other, we focus on the 401 census tracts that are in both the Hampton Roads and Coastal Virginia geographies and consider how consistently these tracts are characterized by the two methods across the various geographies and input sets. For the PCA-based indices, we consider only the variance-weighted indices as these were shown earlier to be more consistent than the equal-weighted indices. Table 9 shows the number of tracts that are identified as highly or marginally vulnerable by the two methods for each of the nine geography/input sets. Most importantly, it shows the number of tracts for each geography/input set that are consistently identified by the two methods. As was the case in Sect. 3.1, we used the greater than 1 standard deviation threshold for identifying highly vulnerable tracts and between 1 and 0.5 standard deviations as the thresholds for identifying marginally vulnerable tracts for the PCA-based index. In almost all cases, the 1 standard deviation threshold for highly vulnerable tracts results in more tracts identified as highly vulnerable than in the cluster analysis. Considering both highly and marginally vulnerable tracts, the PCA-based index identifies more tracts for Virginia for all three input sets and for two of the three Coastal Virginia input sets (CV29 and CV13). However, for Hampton Roads the PCA-based method identifies fewer tracts. We also provide data on the overlap of tracts identified as highly or marginally vulnerable by the two methods (columns 3 and 6 of Table 9). Overall, the overlaps are not particularly high, showing that the choice of method will affect which tracts are classified as vulnerable or not—or more specifically that the two methods are not interchangeable.

Table 9 Comparison of PCA-based indexing to cluster analysis for the 401 Hampton Roads/Coastal Virginia Tracts
Full size table

To better understand what may be happening across the two methods, we look more closely at the non-overlap tracts for the HR13 pair of analyses. Note that 40 tracts are identified as highly vulnerable by the PCA-based index and not by cluster analysis, while only 3 were identified as vulnerable by cluster analysis and not identified as vulnerable by the PCA-based index. Of the 3 tracts identified as vulnerable by cluster analysis and not by the PCA-based index, all are in clusters with very low median income, high percent living in poverty and either high unemployment or low labor force participation. However, all three of these tracts have factors that worked against them in the PCA-based index—that is they had very low levels of individuals over 65, very low nursing home populations, low percentage of individuals on social security, and low Hispanic and native populations, all of which decrease the vulnerability index without negating the vulnerability of the community.

Of the tracts 40 identified as highly vulnerable using the PCA-based index and not highly vulnerable by cluster analysis all were in clusters that were considered to be marginally vulnerable. One cluster has a high relative percentage of people over 65 and in nursing homes, but with above average income; the second has a has a high relative percentage of people over 65 and on social security, but only marginally low income; the third has a high percentage of people in nursing homes, but only moderate values for the other variables of interest; the fourth has a high percentage of people below the poverty rate, female heads of household, and blacks but only moderately low income and moderately high unemployment; and the fifth has a high percentage of female heads of household, blacks, and people on social security, but only moderately low income and moderately high unemployment.Footnote 24

Looking at all tracts identified as vulnerable (highly or marginally), only 8 tracts are identified as vulnerable by the PCA-based index and not by cluster analysis, while 25 were identified as vulnerable by cluster analysis and not by the PCA-based index. Of the 8 identified as vulnerable by the PCA alone, all were marginally vulnerable and all were in clusters whose prototypes did not exceed threshold values for any of the variables. Of the 25 identified as vulnerable by cluster analysis alone, only one was in a highly vulnerable cluster. The other 24 were all in marginally vulnerable clusters where only one or two variables at a high level were necessary for the cluster to be categorized as vulnerable such as a high percentage of individuals in nursing homes or a high black population and a high percentage of female head of households. As discussed before, the PCA-based method does not do a good job of identifying tracts that are vulnerable in only one or two dimensions.

4 Discussion and recommendations for policymakers

This paper presents an alternative to PCA-based indexing using cluster analysis. Based on our case study, we believe that cluster analysis should be considered as a complementary method to PCA-based indexing, that is the cluster analysis approach may be more appropriate in some situations and less appropriate in others. Cluster analysis appears to be best suited for analyses at the state or local level for a number of reasons: the method is more time intensive than PCA-based indexing and thus works best if there are a manageable number of clusters. Since larger and more diverse sets observations will require additional clusters, using cluster analysis may be unwieldy. One of the strengths of cluster analysis is that it appears quite consistent under changes in the set of input variables, so researchers do not need to worry excessively as to whether adding an additional variable will dramatically affect the results of the analysis. Another one of its strengths is that it is quite transparent and the results are easy for lay audiences to interpret, so that it is well suited to local planning efforts that involve stakeholders or efforts where practitioners need to use the vulnerability determinations in very context-specific situations. Finally cluster analysis groups like observations which allows a researcher to easily identify areas that may face similar types of social vulnerability and could potentially share adaptation and mitigation solutions.

PCA-based indices are likely better suited than cluster analyses to multi-state or national analyses where there are many diverse observations. However, they are best used by individuals who understand how the indices are constructed and how to interpret them, including any subsequent rankings. One of the strengths of the PCA-based indices is that they provide a way of ranking communities in terms of vulnerability which cluster analyses do not. In particular, PCA-based indices are more appropriate than cluster analyses for academic research where the ability to provide a continuous measure of vulnerability is important. However, based on the results of our case study, we strongly recommend that researchers who are developing PCA-based indices think carefully about which variables to include in their analysis, limiting the use of bidirectional variables and assessing the potential for other confounding effects, and that researchers using the Kaiser selection criterion use variance weighting in constructing the final index.

A caveat in using both a PCA-based index and the cluster method, or any quantitative method, is that doing so risks creating a false appearance of objectivity by reducing social vulnerability to a quantitative measure. Clearly, some subjective interpretation on the part of the researcher is still required—while this subjectivity is obvious for the clustering method, the subjectivity of the PCA-based index is less transparent. For both methods, the advantage of using readily available data from the US Census and other sources data to identify vulnerable communities is that it potentially provides a fair and consistent standard for evaluation and does not incur substantial cost in its implementation. However, quantitative analysis using data collected for purposes other than determining vulnerability to physical hazards cannot substitute for a more holistic and qualitative analysis of community need or for an inclusive stakeholder-oriented process. To be effective and equitable, adaptation and mitigation policies must include input from members of affected communities themselves as they experience environmental changes firsthand. For example, McNeeley (2014) provides a case study of drought in northwest Colorado using an analytical framework for the acquisition of ground-up, locally based knowledge useful in constructing and testing the validity of quantitative measures. This “toad’s eye” view of vulnerability emphasizes qualitative methods including interviews, participant observation, and analysis of critical documents informing stakeholder viewpoints, as well as traditional quantitative geophysical observation and analysis. Robust engagement with policy stakeholders builds trust between researchers and the community and allows the researcher to conduct an informed analysis of intersecting local social, cultural, economic, geophysical, and biophysical characteristics with strong internal validity.

Adaptation to climate change and sea level rise is a complex endeavor requiring sensitivity both to the needs of diverse communities and to the level of resources available to policymakers to respond to those needs. Assessing the social vulnerability of communities is an essential part of adaptation from the standpoint of both policy effectiveness and environmental justice. While quantitative approaches to identifying vulnerable communities have inherent limitations, they can provide an important guide for policymakers seeking to maximize the benefit of resources targeted at these efforts. With some modifications and caveats, both PCA-based indices and cluster analyses provide complementary, consistent, and transferable tools that policymakers can use to improve services in Hampton Roads, Virginia, in other communities in the USA, and around the world.