Enhanced Geospatial Validity for Meta-analysis and Environmental Benefit Transfer: An Application to Water Quality Improvements

Abstract

Meta-regression models are commonly used within benefit transfer to estimate willingness to pay (WTP) for environmental quality improvements. Theory suggests that these estimates should be sensitive to geospatial factors including resource scale, market extent, and the availability of substitutes and complements. Valuation meta-regression models addressing the quantity of non-market commodities sometimes incorporate spatial variables that proxy for a subset of these effects. However, meta-analyses of WTP for environmental quality generally omit geospatial factors such as these, leading to benefit transfers that are invariant to these factors. This paper reports on a meta-regression model for water quality benefit transfer that incorporates spatially explicit factors predicted by theory to influence WTP. The metadata are drawn from stated preference studies that estimate per household WTP for water quality changes in United States water bodies, and combine primary study information with extensive geospatial data from external sources. Results find that geospatial variables are associated with significant WTP variations as predicted by theory, and that inclusion of these variables reduces transfer errors.

1 Introduction

Meta-analyses in environmental economics are commonly used to evaluate systematic influences of study, economic, resource and population attributes on measures of nonmarket willingness to pay (WTP) for environmental quality improvements, and to generate parameterized functions for use in benefit transfer (Bergstrom and Taylor 2006; Boyle et al. 2013, 2015; Johnston and Rosenberger 2010; Moeltner et al. 2007; Nelson 2015; Nelson and Kennedy 2009; Smith and Pattanayak 2002). Within meta-regression models used for such purposes, the dependent variable is most often a comparable mean or median welfare measure (e.g., WTP) drawn from existing primary valuation studies.¹ Independent variables represent observable factors hypothesized to explain variation in this measure across observations. Meta-analyses of this type have been used to estimate benefit functions for changes in both the quantity and quality of non-market goods, including water quality, air quality, wetlands, fisheries, coral reefs, recreation sites, and others (Boyle et al. 2013; Johnston and Rosenberger 2010; Nelson and Kennedy 2009; Rolfe et al. 2015). Benefit transfers from these functions have been used to support multiple cost benefit analyses (CBAs) of environmental regulations (e.g., US EPA 2009, 2010, 2012, 2015).

With rare exceptions, theory suggests that these transferred welfare estimates should be sensitive to geospatial factors including resource scale (the geographical size of improved environmental resources or areas), market extent (the size of the market area over which WTP is estimated) and the availability of proximate substitutes and complements (Schaafsma 2015). Yet despite significant advances in benefit transfer over the past decade, no published meta-regression models of WTP for environmental quality changes enable simultaneous, continuous adjustments for geospatial factors such as these. The resulting benefit transfers do not exhibit sensitivity to spatial factors that should, according to theory, influence WTP. This implies that one of the primary tools used for benefit transfer (Johnston et al. 2015b), as commonly applied, is unable to account for spatial patterns expected in welfare estimates (Schaafsma 2015).

Here, we make a distinction between meta-analyses addressing WTP for environmental quality changes (e.g., water quality change) and similar models addressing values for quantity changes (e.g., per recreation day; per acre of a natural resource). The latter are often more amenable to spatial characterization, and primary studies often report at least some quantitative spatial information.² $^{, }$ ³ As a result, some past valuation meta-analyses addressing quantities of non-market commodities have included spatial variables that capture aspects of scale, substitutes and other factors (Brander et al. 2006, 2007, 2012a, b, 2015; Ghermandi 2015; Ghermandi et al. 2010; Ghermandi and Nunes 2013; Johnston and Duke 2009b; Londoño and Johnston 2012). In contrast, spatial variables such as these are generally absent from meta-regression models addressing WTP for environmental quality change. It is often these quality-related meta-regression models that are most applied for benefit transfer (Griffiths et al. 2012; US EPA 2009, 2010, 2012, 2015).

In the context of water quality valuation, for example, the geospatial scale of a water quality change typically reflects the size of affected water bodies or watersheds. The sampled market area would reflect the location of populations for which values were estimated by the primary study (Loomis 2000; Loomis and Rosenberger 2006). For example, were values measured for residents of a community, state or nation? As discussed below, the relevance of market area to average per household WTP is related to the expected correlation between market area and the average distance between households and improved resources, ceteris paribus (Johnston and Duke 2009a; Schaafsma 2015). Availability of substitutes and complements reflects the quantity/quality of substitute and complement resources in proximate areas (Brander et al. 2012b; Johnston et al. 2002b; Loomis and Rosenberger 2006; Schaafsma 2015; Schaafsma et al. 2012). For example, households’ WTP to improve water quality in a single lake might depend on the existence and size of other, nearby lakes.

All of these factors are potentially relevant to the welfare gain from environmental quality changes, and at least in principle should be incorporated within meta-analytic and other benefit transfers (Schaafsma 2015). Yet while there has been significant attention to approaches used to reconcile environmental quality measures across primary studies in valuation metadata [e.g., for water quality in Johnston et al. (2005) and Van Houtven et al. (2007)], there has been less attention to spatial context. Among the reasons for this lack of attention is the tendency of primary studies of environmental quality changes to omit information on geospatial aspects of resources, market areas and populations (Loomis and Rosenberger 2006). Inclusion of these data in meta-regression models hence requires these variables to be reconstructed, typically by combining information from primary studies with external spatial data from geographic information system (GIS) data layers. Although some recent meta-analyses and value maps have supplemented primary study metadata using external spatial data sources (e.g., Brander et al. 2006, 2007, 2012a, b, 2015; Ghermandi 2015; Ghermandi and Nunes 2013; Ghermandi et al. 2010; Lindhjem 2006; Londoño and Johnston 2012; Schägner et al. 2013), none of these enable transfers that account for the simultaneous effects of scale, market area and spatially proximate substitutes or complements, along with cardinal measures of environmental quality change.

This paper reports on a meta-analysis that incorporates core geospatial variables predicted by theory to influence WTP for environmental quality improvements. The model is designed to support benefit transfer for water quality improvements within US water bodies. The resulting benefit functions allow heretofore unavailable adjustments for these variables, enhancing the accuracy of benefit transfer. Results demonstrate that it is possible to develop meta-regression models that better accommodate spatial patterns suggested by theory.

2 Geospatial Variables in Meta-analysis and Benefit Transfer

Geospatial scale is among the principal spatial factors relevant to benefit transfer (Schaafsma 2015). For example, one might expect WTP for an improvement in lake water quality to be positively related to the size or number of improved lakes, ceteris paribus. Despite this expectation, meta-regression models used in environmental quality benefit transfer commonly omit variables characterizing spatial scale. Others include ordinal variables identifying features such as (1) whether quality changes affect single or multiple areas, (2) relative size categories such as large, medium and small, or (3) the type of geopolitical areas addressed by the analysis, e.g., improvements at a national, regional or local level (Brouwer et al. 1999; Johnston et al. 2003, 2005; Lindhjem 2006; Lindhjem and Navrud 2008; Rosenberger and Loomis 2000b; Santos 2007; Van Houtven et al. 2007). Some meta-analyses incorporate explicit measures of site area, for example as variables explaining value per day for recreation, per acre for ecosystem service provision, or per kilometer of river in good health (Brander et al. 2006, 2007, 2012a, b, 2015; Ghermandi and Nunes 2013; Ghermandi et al. 2010; Ghermandi 2015; Londoño and Johnston 2012; Rolfe et al. 2015). However, to the knowledge of the authors, no meta-analyses in the published valuation literature incorporate quantitative measures of both the magnitude (i.e., scope)⁴ of an environmental quality change and the size of the area (i.e., scale) over which the change occurs.

It is also established that mean WTP often declines, or decays, with distance to environmental improvements (Bateman et al. 2006; Schaafsma et al. 2012; Sutherland and Walsh 1985). Theoretical expectations for (and the empirical extent of) distance decay depend on the type of values under consideration (Hanley et al. 2003; Johnston and Ramachandran 2014; Johnston et al. 2015a; Rolfe and Windle 2012). When distance decay applies, accurate benefit transfers require one to account for the expected distance between populations (for whom WTP is being estimated) and improved resources (Schaafsma 2015). If quantitative measures of distance or market area are omitted from valuation meta-analysis, the resulting benefit transfers must use ad hoc assumptions to account for these expected welfare patterns.⁵

Although distances between households and affected resources are rarely reported by primary studies (and are hence unknown to meta-analysts), these distances are often correlated with the size of the market area over which values are estimated (Loomis 2000; Loomis and Rosenberger 2006). Larger market areas are often associated with larger average distances between individuals and improved resources, ceteris paribus, leading to lower mean per household WTP (Johnston and Duke 2009a).⁶ Because sampled markets are generally identified by primary valuation studies, the associated areas can be quantified using GIS data, thereby providing a proxy for average resource distance.

Finally, the expected economic value of environmental quality change is often related to the availability of proximate substitutes and complements (Ghermandi and Nunes 2013; Johnston et al. 2002b; Schaafsma 2015; Schaafsma et al. 2012). For example, WTP for quality improvements to particular water bodies may be influenced by the availability of substitute water bodies in the surrounding area (Jørgensen et al. 2013) or by the existence of complementary land uses that enhance the public’s ability to benefit from improved water quality (Johnston et al. 2002b). However, while some authors have speculated that meta-regression model results may be related to differential spatial availability of substitutes (e.g., Brander et al. 2006), and others have included quantitative proxies for substitute availability (e.g., Ghermandi et al. 2010), the authors are aware of no published valuation meta-analyses that incorporate quantitative measures of substitutes and complements for environmental quality changes.

3 Data and Empirical Model

The meta-regression model was designed to support benefit estimation for policies that improve water quality in US water bodies including rivers, lakes and estuaries. Model design was motivated by the need for benefit transfers to account for theoretically-anticipated WTP variations associated with differences in spatial scale, market area and the availability of substitutes and complements, along with changes in scope (i.e., water quality). The metadata are drawn from primary stated preference valuation studies that estimate per household (use and nonuse) WTP for water quality changes in US water bodies that affect ecosystem services including aquatic life support, recreational uses (such as fishing, boating, and swimming), and nonuse values. The metadata exclude studies focusing primarily on drinking water supplies.

To develop model data, we began with the metadata of Johnston et al. (2005). These original metadata were updated and expanded to enable the illustrated modeling. Primary changes included the deletion of studies conducted prior to 1980⁷ and others that did not meet updated screening criteria; the addition of 21 studies not included in Johnston et al. (2005), including 8 studies conducted since 2005⁸; and the development of new, spatially-explicit moderator variables. In addition, observations from two papers that were unpublished as of 2005 (Azevedo et al. 2001; Whitehead et al. 2002) were replaced with observations from subsequently published versions of the same studies (Corrigan et al. 2009; Whitehead 2006).

Observations were identified and added to the metadata following the guidelines of Stanley et al. (2013) for research identification and coding. This included documentation of protocols used to identify potential new studies, including (a) the databases and other sources searched, (b) the precise combination of keywords, and (c) the date completed.⁹ Following recommendations of Stanley et al. (2013), study review, identification and coding were completed and verified by multiple individuals, with all variables and coding documented. To ensure welfare consistency, observations were limited to US studies that estimate total (use and nonuse) value, use generally accepted stated preference methods and models, report theoretically comparable and quantifiable Hicksian welfare measures, and provide sufficient detail to verify methods applied (Boyle et al. 2013).

In addition to the required study characteristics identified above, studies were screened to ensure applicability. Studies omitting information needed to document key resource, context, and study attributes were excluded. Necessary data included information identifying improved water bodies, the extent of water quality change, and sampled market areas, along with core methodological attributes. Finally, studies were limited to those for which per household WTP estimates could be readily linked to water quality changes measured on the standard 100-point Water Quality Index (WQI); this index is a linear transformation of the 10-point water quality ladder used by Johnston et al. (2005). This screening led to the exclusion of studies for which WTP for water quality could not be disentangled from WTP for other ecosystem changes (e.g., riparian land restoration). Additional details on the reconciliation of water quality measures are provided below. Thirty-one studies were excluded due to this additional screening.

The resulting metadata include 140 observations from 51 stated preference studies published between 1985 and 2013. Multiple WTP estimates from a single study are available due to in-study variations in such factors as the extent of amenity change, elicitation methods applied, market area sampled, water body type and number, and uses affected. The inclusion of multiple observations per study is standard in valuation metadata (Nelson and Kennedy 2009). All monetary values are adjusted to 2007 US dollars. The dependent variable for all meta-regression models is the natural log of household WTP for water quality improvements measured on the 100-point WQI. Table 1 summarizes characteristics for studies included in the metadata.

Table 1 Primary studies in the metadata

Table 2 summarizes the set of independent variables included in the meta-analysis. Independent variables in the metadata characterize (1) study methodology and year, (2) surveyed populations, (3) study site, improved resources and market extent, and (4) water quality baseline and change. Study methodology variables characterize such features as the year in which a study was conducted, payment vehicle and elicitation formats, and WTP estimation methods. Region and surveyed populations variables characterize such features as the US region in which the study was conducted, average household income and the representation of users and nonusers within the survey sample. Study site, resource and market extent variables characterize geospatial factors discussed above, as well as hydrological features (i.e., water body type) and affected recreational uses. Finally, water quality baseline and change variables characterize baseline conditions and the extent of the water quality change. These variables were selected and specified based on guidance from theory and prior literature (Bergstrom and Taylor 2006).

When specifying the model, emphasis was given to core economic and resource variables directly relevant to benefit transfer. Methodological variables were included to capture related effects identified previously in the literature. Variables were also included to test for systematic value patterns associated with different publication types (Rosenberger and Johnston 2009). At the same time, care was taken to avoid over-parameterizing the model with methodological variables, as this can lead to models with seemingly good statistical fit that have poor benefit transfer performance (Bateman et al. 2011).

Table 2 Meta-analysis variables and descriptive statistics

3.1 Reconciling Measures of Water Quality Change

An important component of metadata development is the reconciliation of variables across observations (Johnston et al. 2005; Smith and Pattanayak 2002; Smith et al. 2002; Van Houtven et al. 2007). Although the calculation and reconciliation of most independent variables requires little explanation, there are some variables for which additional detail is warranted. These include variables characterizing surface water quality and its measurement. To reconcile measures of water quality across studies we adapt the prior approach of Johnston et al. (2005), mapping water quality changes to the 100-point WQI as noted above.¹⁰

A large number of the studies in the metadata (30 % of observations) include WQI or related 10-point water quality ladder measures as a native component. For these studies, no additional transformations were required. In most other cases the descriptions of water quality rendered mapping of water quality measures to the WQI straightforward. In cases where baseline and improved (or declined) water quality was not defined by suitability for recreational activities (e.g., boating, fishing, and swimming) or corresponding qualitative measures (e.g., poor, fair, good) that could be readily mapped to the WQI, we used descriptive information available from studies (e.g., amount/indication of the presence of specific pollutants; effects on sensitive aquatic species) to approximate the baseline level of water quality and the magnitude of the change. Preliminary meta-regression models failed to identify any systematic variation in results associated with studies for which the WQI was a native component, versus those for which quality changes were mapped to the WQI (see “Appendix”).

3.2 Geospatial Analysis and Variables

Attention was also given to the development and testing of variables characterizing geospatial scale, market extent and proximate substitutes/complements. Variable development was guided by theory and information available from primary studies and external databases. Required data were extracted from the National Hydrography Dataset (http://www.horizon-systems.com/NHDPlus/NHDPlusV2_home.php), Hydrologic Unit Code Watershed Boundary Dataset (http://water.usgs.gov/GIS/huc.html), National Land Cover Database (NLCD; http://www.mrlc.gov), NOAA Global Self-Consistent, Hierarchical, High-resolution Geography Database (GSHHD; http://www.ngdc.noaa.gov/mgg/shorelines/shorelines.html), and US Census (http://www.census.gov/geo/maps-data/data/tiger.html). None of these variables could be calculated using data reported in primary studies alone.

When considering water quality change, geospatial scale may be measured either using the scale of improved water bodies or land areas surrounding these water bodies; economic theory provides no guidance regarding which type of measure is preferred. Hence, we quantify scale using both approaches, with each included in a separate meta-regression model. The first approach measures geospatial scale using the shoreline length of each improved water body. Shoreline length (shoreline) is calculated in kilometers using GIS data layers, and accounts for the fact that improved river reaches have both a left and right bank. This provides a measure of geospatial scale that is quantifiable and comparable across all improved water bodies, regardless of type.¹¹ Measuring scale using shoreline length also improved model performance relative to alternative measures of water body scale that varied in measurability or interpretation across water body types (e.g., surface area).¹² The second approach measures geospatial scale using the total land area of all counties that intersect the improved water resource(s), in square kilometers. The resulting variable, land_area, is also directly measurable for all observations.¹³

Market area (sa_area), in contrast, is defined as the size of the geographic areas sampled by the stated preference survey, in square kilometers. It may also be interpreted as the geographic area over which sample-mean (or median) WTP was calculated. This area is calculated for each observation using external GIS data layers, based on sampled market areas identified by each primary study.

Multiple specifications including geospatial scale and market area were tested in preliminary models. However, economic intuition suggests that these two effects may be related. That is, the marginal effect of increasing water body size on WTP is expected to decline as size of the sampled market area increases, and vice versa. Preliminary models support this intuition; model performance is enhanced (e.g., in terms of model fit, variable significance, and correspondence of results with theoretical expectations) when the effect of geospatial scale (shoreline or land_area) is modeled as a function of market area (sa_area). This led to the development of two composite geospatial index variables that are included in the model as natural logs: ln_rel_size = ln(shoreline / sa_area) and ln_ar_ratio = ln(sa_area / land_area). The first of these includes sampled market area (sa_area) in the denominator, while the second includes this variable in the numerator. The former may be interpreted as an index of the size of the improved water body relative to the size of the sampled market area. The latter may be interpreted as an index of the size of sampled market area relative to the size of the affected land area. Hence, we expect a positive marginal effect of ln_rel_size on WTP, and a negative marginal effect of ln_ar_ratio. Table 2 provides additional details on both variables.

Finally, the model includes variables to characterize proportional effects on regional (potentially substitute) water bodies and the availability of complementary land uses. To characterize proportional effects, we calculate the proportion of water bodies (of the same hydrological type) improved by the water quality change, within affected state(s). For rivers, this is measured as the length of improved river reaches, as a proportion of the all reaches of the same order (prop_chg_reach).¹⁴ For non-river inland water bodies (e.g., lakes), the proportion is defined as the area of the improved water body as a proportion of all water bodies of the same National Hydrography Dataset classification (prop_chg_area). For bays and estuaries, the proportion is defined as the shoreline length of the water body as a proportion of all analogous (e.g., coastal) shoreline lengths (prop_chg_bay). These are combined into a composite variable, prop_chg, defined as max(prop_chg_reach, prop_chg_area, prop_chg_bay) for each observation.¹⁵ Model performance does not improve when including separate substitute variables for each water type (prop_chg_reach, prop_chg_area, prop_chg_bay); hence the final meta-regression model includes only the composite index variable prop_chg (Table 2).¹⁶

The expected influence of prop_chg on WTP may be interpreted from two perspectives. First, it may be viewed as proportional measure of affected resource scale, with an expected positive influence on WTP. Second, it may be viewed as inversely related to the existence of substitute water bodies. That is, as the water quality improvement valued by each primary study affects a larger proportion of regional waters, there are fewer remaining substitutes (i.e., water bodies that are not improved by the proposed policy). The quality of these potential substitutes is unknown; they could be of higher or lower quality than the water bodies valued within each primary study. Nonetheless, as the relative proportion of these potential substitute waters declines (i.e., as prop_chg increases), ceteris paribus, we expect WTP to increase. Hence, regardless of interpretation, the expected influence of prop_chg on WTP is positive.

Finally, potential land use complements to enhanced water quality (more specifically, the lack of complementary land uses) are characterized using the variable ln_ar_agr, representing the (natural log of the) proportion of the improved land_area with agricultural land use. The rationale for this variable is that non-agricultural land uses (e.g., forests, residential, open space) are often associated with recreational, residential and other human uses that potentially magnify the per household value of nearby water quality improvements (Johnston et al. 2002b; Leggett and Bockstael 2000), relative to parallel values for water quality improvements in heavily agricultural areas. Hence, the expected effect of this variable—reflecting the relative lack of these complementary land uses—is negative.¹⁷

It is also possible that the estimated effect of ln_ar_agr on WTP (along with estimated effects of other variables) could be potentially biased by research priority selection in the literature. Such issues are a potential concern for all meta-analyses (Rosenberger and Johnston 2009). For example, it is possible that heavily polluted agricultural watersheds might be systematically more (or less) likely to be targeted for valuation than more pristine watersheds elsewhere. If the propensity to target an area for a valuation study is systematically related to the extent of agricultural land use, then the resulting selection effects could potentially bias model results. Heckman correction methods (Heckman 1979) can sometimes be used to explore and ameliorate such biases, although results are often sensitive to assumptions regarding the form of the selection equation (Hoehn 2006; Rosenberger and Johnston 2009). We do not explore these issues formally here, but identify them as a relevant area for future research.

3.3 The Meta-regression Model

We estimate the meta-regression model as a multi-level model of the type common in the literature (Nelson and Kennedy 2009). The model allows for cross-sectional correlation among observations from the same study. If left unaddressed, such correlation can lead to heteroskedastic errors and inefficient, inconsistent parameter estimates (Rosenberger and Loomis 2000a). For each study in the metadata, a central tendency measure (mean or median) of WTP for the representative individual is given by $\bar{{y}}_{js}$, which is the measured effect size in the meta-regression model:

$$\begin{aligned} \bar{{y}}_{js} =\bar{{x}}_{js} \beta +\varepsilon _{js} . \end{aligned}$$

(1)

Here, $\bar{{y}}_{js}$ is the welfare measure for observation s in study j (here the natural log of WTP), and $\bar{{x}}_{js}$ is the vector of independent variables discussed above. The vector $\beta$ represents a conforming vector of parameters to be estimated.

To allow for potential effects of study-specific unobservable factors, we partition $\varepsilon _{js}$ into two components such that

$$\begin{aligned} \varepsilon _{js} =u_s +e_{js} . \end{aligned}$$

(2)

Here, $u_{s}$ represents a systematic, normally distributed, study-level random effect with $E(u_{s})= 0$ and $Var(u_{s}) = \sigma _{u}^{2}$, and $e_{js}$ is a standard iid estimation level error, distributed with a zero mean and constant variance $\sigma _{e}^{2}$ (Shrestha and Loomis 2001). Clustering by study to account for within-study correlation is standard practice. Other aspects of the econometric model follow standard conventions for valuation meta-regression models; we estimate the model using an unweighted GLS random-effects model with robust standard errors (Nelson and Kennedy 2009).¹⁸

Three model specifications are estimated. The first (model one) is an unrestricted model including variables that characterize scale, market area and non-improved substitutes using the two index variables ln_rel_size and prop_chg. The second (model two) is an unrestricted model characterizing these effects using ln_ar_ratio and prop_chg. As discussed above, the difference between these two specifications is that the former characterizes resource scale using the shoreline length of improved water bodies, while the latter characterizes scale using the area of intersecting counties. The third model is an otherwise identical restricted model that omits these variables; this model is akin to common meta-regression model specifications in the literature.

Table 3 Meta-regression model results (ln(WTP)): random effects model, robust standard errors

A trans-log specification is used for all meta-regression models. This specification incorporates the natural log of the dependent variable (WTP per household) on the left hand side and the natural logs of household income (lnincome), water quality change (lnquality_ch), relative geospatial scale (ln_rel_size or ln_ar_ratio), and the proportion of agricultural land in the improved area (ln_ar_agr) on the right hand side (Table 2). All other variables enter in linear form. Advantages of this functional form for meta-analysis are discussed by Johnston et al. (2005). These include an ability to capture curvature in the valuation function, a multiplicative rather than additive effect of independent variables on WTP, and the implied constraints that WTP approaches zero when water quality change, income, and the geospatial index variables (ln_rel_size and ln_ar_ratio) approach zero.¹⁹ Other tested functional forms included linear, semi-log, log-log and structural reduced forms, including those with alternative measures of the dependent variable.²⁰ None of these led to unambiguous improvements in model performance.

4 Results and Discussion

Table 3 shows results for both the unrestricted and restricted models. Wald ${\upchi }^{2}$ tests for both unrestricted models indicate that parameter estimates are jointly significant at $p<0.01\,({\upchi }^{2} = 658.64\hbox { and }729.61,\,\hbox {df}.\,24)$, with model $\hbox {R}^{2}$ statistics of 0.628 and 0.633. All measures of model fit decline for the restricted model $({\upchi }^{2}=415.47,\hbox { df. }21;\hbox { R}^{2}=0.54)$. Wald ${\upchi }^{2}$ tests also indicate that omission of the parameters on the spatial variables ln_rel_size or ln_ar_ratio, prop_chg, and ln_ar_agr is a statistically significant restriction $(p<0.01)$, compared to either unrestricted model $({\upchi }^{2} = 243.17\hbox { and }314.14,\hbox { df}.\,3)$. Of 23 non-intercept parameter estimates in the unrestricted models, 20 are statistically significant at $p<0.10$, with the majority significant at $p<0.01$. Breusch–Pagan Lagrangian multiplier tests for random effects reject the null hypothesis that $\sigma _{u}= 0$ in both unrestricted models ($p<0.04$ and $p<0.05$ for models one and two).²¹

Signs of statistically significant parameter estimates in all meta-regression models match those suggested by theory and intuition. For example, within both unrestricted models, WTP is positively related to the scope of water quality change (lnquality_ch), household income (lnincome), and one-time payments (lump_sum), among other factors. Nonuser samples (nonusers) are associated with systematically lower WTP estimates than user or general population samples. These and other non-spatial results are similar to those reported in prior meta-analyses. Given these findings, we move on to results associated with our geospatial variables of interest. Those interested in a broader discussion of systematic patterns in water quality values are referred to Johnston et al. (2003, 2005) and Van Houtven et al. (2007).

Coefficient estimates associated with the geospatial variables ln_rel_size, ln_ar_ratio, prop_chg and ln_ar_agr are statistically significant at $p<0.01$ (Table 3). Signs of coefficient estimates match expectations; with positive marginal effects associated with ln_rel_size and prop_chg, and a negative marginal effect associated with ln_ar_ratio and ln_ar_agr. For example, the coefficient estimate for ln_ar_ratio (model one) implies that per household WTP decreases with the size of the surveyed market area, relative to the size of counties that intersect improved water bodies. When viewed across and within different studies from the literature, studies over larger market areas are associated with lower per household WTP, ceteris paribus. This is intuitive, because larger sampled market areas imply greater mean distances between households and improved water bodies. Also as expected, there is a positive relationship between geospatial scale (i.e., the area of counties that intersect improved water bodies) and per household WTP. Combining these effects within the index ln_ar_ratio allows the marginal effect of market area to depend on geospatial scale, and vice versa.

Parallel intuition applies to ln_rel_size in model two, although the expected coefficient sign is reversed because geospatial scale is in the numerator of the index. The positive sign for this variable implies that per household WTP increases with the size of improved water bodies (measured by shoreline length in kilometers) relative to the size of the surveyed market area (measured in square kilometers). As above, combining these effects within the single index variable ln_rel_size allows the marginal effect of geospatial scale to depend on market area size.

Other results are robust across the unrestricted models. Measures of fit are similar, and parameter estimates are similar in sign, significance and magnitude. This finding suggests that model results are robust to the choice of variables used to characterize geospatial scale and market area (ln_rel_size vs. ln_ar_ratio). It is also expected given the high degree of (negative) correlation between ln_rel_size and ln_ar_ratio ($-0.92$). Similar robustness is found when watershed rather than county area is used as a measure of the affected land area (see “Appendix”).

This is the first published valuation meta-regression model to incorporate these joint effects (i.e., scale and market area) in quantitative, continuous form. As discussed above, these adjustments can be crucial to the content validity of benefit transfers. Consider, for example, a water quality improvement to a single lake in Massachusetts, USA. A meta-regression model without an adjustment for spatial scale would predict identical per household WTP for this improvement, regardless of lake size. A model without an adjustment for market area would predict identical mean per household WTP, regardless of whether one was forecasting WTP for residents in: (a) the community surrounding the lake, (b) the state of Massachusetts, or (c) the USA as a whole.²² In contrast, the present model predicts mean per household WTP to be successively smaller in (a), (b) and (c), respectively, as expected based on theoretical intuition.

The coefficient estimate for the variable prop_chg $(p<0.01)$ further implies that per household WTP increases when a larger proportion of regional water bodies (of the same type) are improved by the proposed policy, ceteris paribus. For example, the model predicts larger per household WTP for a water quality improvement over 30 % of a state’s river kilometers, compared to an otherwise identical policy that affects only 10 % of river kilometers (of equivalent river order; Table 2), ceteris paribus.²³ Finally, the negative coefficient on ln_ar_agr $(p<0.01)$ implies that heavily agricultural landscapes are associated with reduced WTP for water quality improvements, consistent with the expectation that many non-agricultural land uses serve as complements for water quality change.²⁴

The importance of these findings for benefit transfer depends not only on their statistical significance, but also on the magnitude of each effect. The partial elasticities associated with these effects appear to be relatively small, at least on an individual basis. For example, the partial elasticities of WTP with respect to the log variables ln_rel_size and ln_ar_ratio are 0.05 and $-0.07$, respectively. The partial elasticity of ln_ar_agr is somewhat larger ($-0.35$ in both unrestricted models). Parallel elasticities (at mean values) for the linear variable prop_chg vary from 0.13 to 0.09 in models one and two, respectively. These individual elasticities viewed in isolation, however, can provide a misleading perspective on the practical policy relevance of these value surfaces, particularly given that these variables vary in concert. Implications for benefit transfer accuracy are illustrated below.

The model also provides results that—when combined with findings for the geospatial variables discussed above—provide additional insight into WTP patterns. For example, the negative and statistical significant parameter estimate for mult_bod in both unrestricted models $(p<0.01)$ implies that for a given geospatial scale, WTP is lower when elicited for multiple rather than single water bodies. This implies that WTP is greater for improvements to one large water body compared to multiple smaller ones, ceteris paribus. This effect only emerges when controlling for other geospatial variables; the same parameter estimate is not statistically significant in the restricted model. Other core economic variables such as lnincome are similarly significant in the unrestricted models, but insignificant in the restricted model. Findings such as these suggest that the addition of theoretically-supported geospatial variables to meta-regression models can enable other statistically significant and theoretically intuitive patterns to emerge.

5 Implications for Benefit Transfer

Implications for benefit transfer are illustrated using two approaches. The first is an extreme case illustration that shows the extent to which transfer estimates vary when geospatial variables take on maximum and minimum values from the metadata, holding all else constant. The second analysis uses an iterative leave-one-out cross-validation convergent validity test (Brander et al. 2007; Londoño and Johnston 2012; Stapler and Johnston 2009), to characterize the average effect of these variables on transfer errors. Both analyses show that the inclusion of geospatial variables reduces transfer errors, in some cases by large percentages.

5.1 Effects of Geospatial Variables on Transfer Error: Extreme Case Illustrations

To illustrate implications of model results for benefit transfer, we project per household WTP for illustrative water quality improvements within policy sites that differ in geospatial scale, market extent and substitute/complement availability. This parallels the process that would be used to conduct benefit transfer using meta-regression model results. Results are forecast using coefficient estimates in Table 3. Other than differences in geospatial variables, the illustrative sites and scenarios are identical, and are designed to represent a typical scenario for which WTP might be forecast.

For illustration, we assume a water quality change equal to the mean over the metadata (lnquality_ch $=$ 2.907). This is equivalent to $18.301 = e^{2.907}$ on the 100-point WQI, beginning from a baseline of lnbase $=$ 3.589 (36.194 on the WQI). We assume annual mean WTP per household (lump_sum $=$ 0; wtp_median $=$ 0; volunt $=$ 0), and a general population sample (nonusers $=$ 0) in the US mid-Atlantic region (northeast $=$ central $=$ south $=$ 0), for a water quality improvement in a single river (river $=$ 1; mult_bod $=$ 0). These assignments ensure consistency of the resulting WTP estimates. Other variables are held at mean values from the metadata.²⁵ Forecasts incorporate the standard intercept adjustment $({\sigma _e^2 }/2)$ prior to the exponential transformation to obtain an estimate of mean WTP. Although we illustrate results for a single policy scenario, results are robust across alternative scenarios and illustrations (all of which show the relevance of spatial variables for benefit transfer).

Grounded in this policy setting, we forecast per household, annual WTP for five illustrative geospatial scenarios, chosen to span the range of in-sample values for prop_chg, ln_ar_agr, ln_ar_ratio and ln_rel_size. Scenario 1 assumes that these variables take on mean values from the metadata. Scenario 2 assumes that prop_chg and ln_rel_size take on their minimum values from the metadata, and ln_ar_ratio and ln_ar_agr take their maximum values; these together imply the smallest in-sample ratio of geospatial scale to market extent, the smallest proportional effect on regional water bodies, and the smallest availability of complementary land uses. Scenario 3 assumes that prop_chg and ln_rel_size take on their maximum values, and ln_ar_ratio and ln_ar_agr take minimum values. Scenarios 4 and 5 allow the combined indices of geospatial scale and market area (ln_ar_ratio and ln_rel_size) to take their corresponding maximum and minimum values, respectively, holding other variables at their means. These final scenarios illustrate the implications of varying only geospatial scale and market area, holding other variables constant.

Three WTP estimates are generated for each scenario. The first is the per household WTP estimate generated by model one, including ln_rel_size. The second is a WTP estimate generated by model two, including ln_ar_ratio. The third is the WTP estimate for each scenario from the restricted model. Because the restricted model omits all core geospatial variables, WTP estimates from this model are identical across all scenarios. Results are shown in Table 4.

Table 4 Illustrative scenarios: implications of geospatial scale, extent of the market and substitutes on benefit transfers

Table 5 Benefit transfer errors (iterative leave-one-out convergent validity test)

Results suggest that researchers should exercise caution when conducting benefit transfers that do not account for geospatial scale, market extent, and substitute/complement availability, particularly when these variables take on relatively high or low values (Table 4). The restricted model generates a WTP estimate of $58.15 for all scenarios, illustrating the invariance to geospatial factors common in published meta-regression models. As expected, the restricted model forecast ($58.15) is similar to the unrestricted model forecasts that assume mean values for all geospatial variables ($54.52 and $54.60). However, the unrestricted and restricted model WTP estimates diverge as geospatial variables depart from their mean values. For example, within Scenario 2, the restricted model leads to WTP estimates that are 240 and 216 % larger than parallel estimates from the unrestricted models ($58.15 vs. $17.12 or $18.38). Within Scenario 3, the restricted model leads to WTP estimates that are 87 and 84 % smaller than those from the unrestricted models ($58.15 vs. $433.58 or $360.54).

Scenarios 4 and 5 illustrate effects associated with only variations in geospatial scale and market area (ln_ar_ratio or ln_rel_size). In this case, the restricted model produces a WTP forecast ($58.15) that under/over-states values by as much as 37 and 87 % respectively, compared to unrestricted model forecasts ($92.56 and $31.14 from model one). Hence, omission of even a single geospatial variable from meta-regression models can, in some cases, have substantial implications for benefit transfer.

5.2 Effects of Geospatial Variables on Transfer Error: Mean Transfer Errors

The above analysis demonstrates implications for benefit transfer when spatial variables take on maximum or minimum values from the metadata. To provide a perspective on more typical or average effects, we conduct an iterative leave-one-out cross-validation convergent validity test that compares the out-of-sample performance of the restricted and unrestricted models (Brander et al. 2007; Londoño and Johnston 2012; Stapler and Johnston 2009). We begin with $n=1{\ldots }N$ observations from the metadata. The first step is the omission of the nth observation. The model is then estimated using the original model specification for the remaining $N-1$ observations. This is iterated for each n = 1...N observation, resulting in a vector of N unique parameter estimates, each corresponding to the omission of the nth observation. For each of these n = 1...N model runs, the nth observation represents an out-of-sample observation corresponding to the vector of parameter estimates resulting from that iteration. Parameter estimates for the nth iteration are then combined with independent variable values for the nth observation to generate a WTP forecast for the omitted observation. The result is N out-of-sample WTP forecasts, each drawn from a unique model estimation. Transfer error is assessed through comparisons of the predicted and actual WTP value for each of the N observations, and is reported as absolute value percentage error. Results of this test are compared for the unrestricted and restricted models.

Results of the test are illustrated by Table 5. Results verify that the inclusion of geospatial variables within meta-regression models reduces transfer error, but that the reduction in average error (as expected) is smaller than that found in the extreme cases shown above. The restricted model that omits core geospatial variables produces an average absolute value transfer error of 69.02 %, when used to predict out-of-sample values. This is equivalent to an average absolute value error of $45.50. In contrast, unrestricted models one and two produce transfer errors of 64.20 % ($42.32) and 63.31 % ($41.65), respectively. Hence, average transfer errors increase by approximately 5–6 % when geospatial variables are omitted.

These results, combined with those from the extreme value cases illustrated above, lead to three general conclusions. First, the inclusion of geospatial variables reduces mean transfer errors. Second, effects on benefit transfers can be substantial when geospatial variables take on large or small values. Third, the average effect of these variables on transfer accuracy, while positive, is smaller. We note that these effects will likely vary across WTP for different non-market commodities, and in some cases could be larger or smaller than those illustrated here.

6 Conclusion

The illustrated meta-regression models quantify systematic WTP variation associated with theoretically motivated geospatial factors. In some cases, the influence of these factors on WTP forecasts can be of greater magnitude than those associated with attributes traditionally included in benefit functions and associated meta-analyses. Comparison to a restricted meta-regression model that omits these variables illustrates the increased transfer errors that can occur when these effects are overlooked. Valuation meta-regression models in the published literature generally omit variables such as these, leading to concerns related to the accuracy of benefit transfers.

Results of this analysis must be interpreted within the context of our case study. The illustrated model specification was chosen after preliminary modeling to evaluate alternative means to account for these patterns. Nonetheless, other specifications are possible. Additional research is required to evaluate whether similar findings are applicable to other types of environmental changes and policy contexts. For example, it is unknown whether the present geospatial variables—while effective in the present meta-regression model—represent a widely applicable means to quantify similar patterns in other valuation metadata. Moreover, it is well known that the influence of spatial factors on WTP can vary across different types of resources and environmental improvements. Additional work is required to evaluate whether and how results such as these apply to other resource types and valuation contexts.

These and other caveats aside, results of the analysis demonstrate that meta-regression models and subsequent benefit transfers can accommodate geospatial factors suggested by economic theory. Beyond case study empirical findings, results suggest avenues for broader improvements in meta-analysis and benefit transfer. These include the capacity to use theoretical expectations as a basis for more extensive metadata supplementation, using externally-available GIS and other data to characterize potentially influential variables whose inclusion is supported by theory, but for which information is unreported by primary studies. In addition to supporting more accurate benefit transfers, such work can help evaluate and document whether empirical welfare estimates adhere to theoretically expected patterns.

Appendix

Table 6 illustrates results of an alternative specification of unrestricted model one, in which ln_ar_ratio is replaced with an alternative index of market area to geospatial scale, ln_ar_ratio2. Recall, ln_ar_ratio = ln(sa_area / land_area), where land_area is defined as the area of all counties that intersect with the improved water bodies. In contrast, ln_ar_ratio2 = ln(sa_area / watershed_area), where watershed_area is defined as the area of all watersheds (of hydrologic unit code 10, or HUC 10; https://water.usgs.gov/GIS/huc.html) that intersect with improved water bodies. Results parallel those shown in Table 3, with transfer error implications very similar to those shown in Tables 4 and 5.

Table 6 Alternative model results: improved area defined as intersecting watersheds (HUC 10)

Table 7 Alternative model results: includes dummy variable identifying studies using water quality indices as a native component (WQI_study)

Table 7 illustrates another preliminary version of model one estimated to test for systematic differences between studies for which the WQI was a native component, versus those for which descriptive information from each study was used to map water quality baselines and changes onto the standard WQI (see main text). The model includes a dummy variable (WQI_study) identifying studies that incorporate the WQI as a native component. This variable is included both as an individual variable and as an interaction with water quality change (WQI_study $\times$ lnquality_ch). As shown by Table 7, the associated coefficient estimates are not statistically significant ($p>0.67$ and $p>0.73$, respectively), and other model outcomes are unaffected (compare to Table 3). Nearly identical results are generated for model two. Because only 30 % of the sample (43 observations) includes the WQI as a native component, degrees of freedom constraints prevent estimation of the full models using these observations alone. However, no preliminary model was able to identify statistically significant, systematic differences between the two groups of studies.