1 Introduction

A substantial literature has emerged in the nonmarket valuation arena aimed at combining data from revealed preference (RP) and stated preference (SP) sources. The goal of such efforts vary. In some cases, the objective is to test the convergent validity of the RP and SP approaches (e.g., Azevedo et al. 2003; Huang et al. 1997, and Whitehead et al. 2010). In other instances, the two data sources are viewed as complementary, with RP data providing values grounded in individual behavior (rather than intentions), while SP data both expands on the range of variation in environmental amenities from what is observed in RP data and introduces experimental control over the impact of unobservable factors (e.g., von Haefen and Phaneuf 2008). To the extent that the RP and SP data are generated by the same underlying preferences, this approach argues that combining the two provides more accurate measures of value. Early examples along these lines include Cameron (1992) and Adamowicz et al. (1994), while more recent applications include Dosman and Adamowicz (2006) and Eom and Larson (2006). In either case, it is typically assumed that convergence between the RP and SP data sources is an “all or nothing” proposition; i.e., the RP and SP data are either consistent with each other or they are not. The purpose of this paper is to propose an alternative framework that allows for possible divergence among individuals in terms the consistency between their RP and SP responses. In particular, we suggest the use of a latent class approach to segment the population into two groups. The first group has RP and SP responses that are internally consistent, while the remaining group exhibits some form of inconsistent preferences. Examining differences between the preferences of the two groups provides additional insights into the wedge between RP and SP responses. The framework also opens up the possibility of modeling class membership, along the lines employed by Boxall and Adamowicz (2002), with the goal of mitigating the behavior of those in the “inconsistent” class in subsequent RP/SP exercises.

The remainder of the paper is organized as follows. Section 2 provides a brief overview of the literature combining stated and revealed preference data. We then describe the proposed latent class model in Sect. 3, along with a description of the EM algorithm used in estimation. Section 4 presents a generated data experiment to illustrate the performance and characteristics of the model under different parameterizations, with particular attention paid to the size of the “inconsistent” class as a share of the population. These Monte Carlo exercises illustrate the impact, both in terms of parameter and welfare estimates, of ignoring discrepancies between the underlying RP and SP data generating processes, particularly when the “consistent” class is only a small share of the target population. We illustrate our framework in Sect. 5 using two data sets. The first is the Alberta Moose Hunting data first introduced by Adamowicz et al. (1997) in their RP/SP exercise. The second is the similar Saskatchewan Moose Hunting data set used by Haener et al. (2001) to compare and contrast the predictive power of RP, SP and combined RP-SP models. Both data sets were subsequently employed by von Haefen and Phaneuf (2008). Our results indicate that somewhere between one-half and two-thirds of the samples provide responses that suggest different RP and SP data generating processes and that welfare predictions are sensitive to the choice of which subgroup is used in valuing changes to the environment. The paper wraps up in Sect. 6 with a summary and conclusions.

2 The Literature on Combining RP/SP Data Sources

The idea of combining information from revealed preference and stated preference sources is by no means a new one, with papers appearing in the marketing, transportation, health and environmental economics literatures. In their recent review, Whitehead et al. (2008) note that the earliest efforts along these lines appeared in the transportation and marketing literatures nearly twenty-five years ago, with papers by Ben-Akiva and Morikawa (1990) and Ben-Akiva et al. (1994). Comparisons between RP- and SP-based welfare measures have, of course, been around for years in the environmental arena, including the pioneering goose hunting permit study by Bishop and Heberlein (1979). However, the objective of such comparisons was typically a convergent validity test, with the usual, though not universal, presumption being that the RP results were more reliable as they were based on actual behavior.Footnote 1

The earliest efforts to explicitly combine the two sources in the environmental literature appeared somewhat later, with papers by Cameron (1992) and Adamowicz et al. (1994). These authors argued that RP and SP data should be viewed as complementary, rather than competing, sources of information. In particular, two key limitations of the revealed preference data are (a) insufficient variation in environmental amenities of interest and (b) the potential for the environmental amenities to be confounded with other observed or unobserved factors. Proposed environmental policy scenarios often involve changes that are outside of the range of historical environmental conditions, making extrapolation of preferences for such changes tenuous and dependent on strong assumptions regarding the form of individual preferences. More fundamentally, there may simply not be sufficient historical variation in the environmental attribute of interest to identify its impact on preferences. A related problem is that what variation is observed for an environmental amenity may be correlated with other observed or unobserved factors impacting consumer preferences, making it difficult to disentangle its causal effect on consumer behavior. Stated preference data, on the other hand, provides the researcher with greater control over the variation in environmental conditions presented to survey participants. In many cases, orthogonal treatments can be employed, though such treatments may be limited by the need to present realistic choice scenarios. von Haefen and Phaneuf (2008) highlight the fact that the experimental control associated with stated preference surveys can be used to isolate the causal impact of an environmental amenity on individual behavioral, avoiding problems of omitted variables bias encountered in stand-alone RP exercises. Eom and Larson (2006) illustrate the use of SP data, in combination with RP data, to identify non-use (or passive use) values that simply cannot be identified with RP data alone.

The major concern with stated preference data sources is that they might be susceptible to hypothetical bias. Revealed preference data can be used to “discipline” the stated preference responses with information on choices observed in the marketplace. One strategy is to rely primarily upon RP data to estimate the key preference parameters, such as the marginal utility of income, leaving SP with the role of “filling-out” the marginal impacts of environmental amenities on individual preferences (e.g. von Haefen and Phaneuf von Haefen 2003). Alternatively, if both sources are viewed as suspect, combining the two data sources may provide the best overall picture of consumer preferences.

The evidence regarding combining RP and SP data sources is mixed. Adamowicz et al. (1994) and Adamowicz et al. (1997), for example, find “...RP-SP parameter equality, once variance heterogeneity is accounted for, and ...that joint RP-SP models are superior to RP models alone.” In contrast, von Haefen and Phaneuf von Haefen and Phaneuf (2008), using the same data as Adamowicz et al. (1997), reject consistency between the RP and SP responses, as do Azevedo et al. (2003) in a different setting. Both Jeon and Herriges (2010) and Whitehead et al. (2008) reject consistency between RP and SP responses in their respective studies, though the differences between the welfare measures derived from the RP and SP sources are not substantial. In all these studies, the tests for consistency are for the sample as a whole. In the next section, we outline a latent class model that estimates the proportion of the sample that exhibits inconsistent RP and SP preferences.

3 Model

This section begins by describing a single class joint model of RP and SP data in a repeated discrete choice setting. The structure of the model is similar to the one employed by von Haefen and Phaneuf (2008). The model is then extended using a latent class framework, allowing for some portion of the sample (s) to exhibit consistent RP and SP preferences, while the RP and SP parameters diverge for the remainder of the sample. As is typical of the recent literature on latent class models (e.g., Breffle et al. 2011; Evans and Herriges 2010; Kuriyama et al. 2010), we propose estimating the parameters of the model using of an EM algorithm so as to avoid numerical difficulties often encounter with standard maximum likelihood estimation of latent class models (see, e.g., Train 2009).

3.1 Combining RP and SP Data

There are two common issues encountered when combining RP and SP recreation demand data. First, the relevant site attributes are generally different for the two data sources. Of particular concern in the context of the modeling RP choices is the fact that the analyst may observe only a subset of the choice attributes impacting an individual’s decision. To the extent that there are unobserved choice attributes that are correlated with the attributes available to the researcher, steps must be taken to control for potential omitted variables bias. In contrast, stated preference choices can be thought of as providing the analyst with complete information on the relevant choice attributes, assuming of course that the SP study is well-designed and the respondents fully understand the instructions. To the extent that there are unobservable individual or site attributes impacting an individual’s choices, the random assignment of observable treatment affects should avoid potential omitted variables bias. Second, given the differences in the decision making processes underlying the RP and SP data sources, there are likely to be differences in the unobservable factors impacting the corresponding decisions. These differences manifest themselves in differences between the scale parameters associated with the RP and SP portions of the model. Control for changes in the scale parameters of the two models is important in testing for consistency between the two data sources (see, e.g., Adamowicz et al. 1994 and Adamowicz et al. 1997).

Starting with the revealed preference portion on the model, the data provide information on the number of times (\(n_{ij}^{RP}\)) individual i chooses to visit each of j sites over the course of \(T_i\) trips.Footnote 2 The utility (\(U_{ijt}^{RP}\)) that individual i receives from choosing site j on trip t is assumed to be a linear function of observed (\(X_{j}^{RP}\)) and unobserved (\(\tilde{X}_j^{RP}\)) site specific attributes, travel costs to the site (\(p_{ij}\)), and an idiosyncratic error component (\(\mu ^{RP}\varepsilon _{ijt}\)), where \(\varepsilon _{ijt}\) is an iid Type I extreme value error term and \(\mu ^{RP}\) is the associated scale factor.Footnote 3 Formally,

$$\begin{aligned} U_{ijt}^{RP}&=X_{j}^{RP}\beta ^{RP}+\tilde{X}_{j}^{RP}\tilde{\beta }^{RP}+p_{ij}^{RP}\gamma ^{RP}+\mu ^{RP}\varepsilon _{ijt}\nonumber \\&=X_{j}^{RP}\beta ^{RP}+\xi _{j}^{RP}+p_{ij}^{RP}\gamma ^{RP}+\mu ^{RP}\varepsilon _{ijt} \end{aligned}$$
(1)
$$\begin{aligned}&=\alpha _{j}^{RP}+p_{ij}^{RP}\gamma ^{RP}+\mu ^{RP}\varepsilon _{ijt}\\&=V_{ij}^{RP}+\mu ^{RP}\varepsilon _{ijt}\nonumber \end{aligned}$$
(2)

where \(V_{ij}\equiv \alpha _{j}^{RP}+p_{ij}^{RP}\gamma ^{RP}\), \(\xi _{j}^{RP}\equiv \tilde{X}_{j}^{RP}\tilde{\beta }^{RP}\) and

$$\begin{aligned} \alpha _{j}^{RP}\equiv X_{j}^{RP}\beta ^{RP}+\xi _{j}^{RP}. \end{aligned}$$
(3)

Historically, analysts have often included observable factors in the conditional utility specification without controlling for possible unobservable sites attributes. As noted by Murdock (2006), there are two advantages of explicitly introducing unobservable sites attributes into the model. First, to the extent that unobserved and observed site attributes are correlated, including only the observed site attributes at this stage of the estimation process would lead to omitted variables bias. Segmenting off the site attributes into a second stage analysis insulates the parameters in Eq. (2) (particularly the key travel cost parameter) from this potential source of bias. Second, as Murdock (2006) demonstrated, ignoring the unobservable site attributes and including only the observable attributes in (1) inflates the precision with which the site attribute parameters are estimated. It is important to keep in mind that the observed attributes are not missing from the utility specification in Eq. (2). Rather, they are simply imbedded in the alternative specific constants. As suggested by Murdock (2006), a second stage regression can then be used to recover \(\beta ^{RP}\) by estimating Eq. (3) using fitted values for the alternative specific constants (i.e., the \(\alpha _j^{RP}\)’s) and, if necessary, properly instrumenting for the \(X_{j}^{RP}\).

Turning to the stated preference data, the individuals are presented with a series of H choice scenarios, with each choice scenario involving K alternatives (\(K=3\) in both Moose Hunting data sets). The utility \(U_{ikh}^{SP}\) that individual i associates with alternative k from choice scenario h is assumed to be a linear function of the designed characteristics for each of the choice alternatives (\(X_{ikh}^{SP}\)), the cost of the presented alternative (\(p_{ikh}\)), and an idiosyncratic error components (\(\mu ^{SP}\varepsilon _{ikh}\)), where \(\varepsilon _{ikh}\) is an iid Type I extreme value error term and \(\mu ^{SP}\) is the associated scale factor. Formally

$$\begin{aligned} U_{ikh}^{SP}&=X_{kh}^{SP}\beta ^{SP}+p_{ikh}^{SP}\gamma ^{SP}+\mu ^{SP}\varepsilon _{ikh}. \end{aligned}$$
(4)

There are several features of (4) worth noting. First, there are no unobservable factors associated with the SP choice utilities, except of course those imbedded in the idiosyncratic error term. The random assignment of choice characteristics breaks the potential correlation between the observable treatments and any unobserved factors influencing the individual’s decision. This is one of the key strengths of the stated preference approach. Second, while \(U_{ijt}^{RP}\) is constant over the choice alternatives (with, of course, the exception of the idiosyncratic error term), the utilities associated with the SP choices can vary substantially over the alternative choice occasions. This is a second key strength of the SP data.

Without further restrictions on the two sources of preference information, neither of the scale parameters \(\mu ^{RP}\) and \(\mu ^{SP}\) are identified and must be normalized to 1. The corresponding contribution of an individual to the likelihood function is then given by:

$$\begin{aligned} \mathcal {L}_{i}^{IC}\left( \theta ^{IC}\right)&= \left\{ \prod _{j=1}^{J} \left[ \frac{\exp \left( \alpha _{j}^{RP}+p_{ij}^{RP}\gamma ^{RP}\right) }{\sum _{m=1}^{J} \exp \left( \alpha _{m}^{RP}+p_{im}^{RP}\gamma ^{RP}\right) }\right] ^{n_{ij}^{RP}}\right\} \nonumber \\&\quad \cdot \prod _{h=1}^{H}\left[ \prod _{k=1}^{K} \left\{ \frac{\exp \left( X_{ikh}^{SP}\beta ^{SP}+p_{ikh}^{SP}\gamma ^{SP}\right) }{\sum _{r=1}^{K}\exp \left( X_{irh}^{SP}\beta ^{SP}+p_{irh}^{SP}\gamma ^{SP}\right) }\right\} ^{1_{ikh}^{SP}}\right] , \end{aligned}$$
(5)

where \(1_{ikh}^{SP}=1\) if individual i chose alternative k in SP choice scenario h and equals zero otherwise and \(\theta ^{IC}\equiv (\alpha _\bullet ^{RP},\gamma ^{RP},\beta ^{SP},\gamma ^{SP})\) denotes the parameter of the model, with \(\alpha _\bullet ^{RP}\equiv (\alpha _1^{RP},\ldots ,\alpha _{J-1}^{RP}\)) denoting the complete vector of ASC’s. The IC superscript (i.e., “inconsistent”) on the likelihood function is used to indicate that this specification does not impose consistency between preferences underlying the RP and SP responses.

The insight of von Haefen and Phaneuf (2008) is that, by combining the two data sources and imposing consistency in the underlying data generating processes, portions of the RP preferences parameters can now be identified. Specifically, assuming that \(\beta ^{RP}=\beta ^{SP}=\beta ^C\) and \(\gamma ^{RP}=\gamma ^{SP}=\gamma ^C\), the corresponding likelihood function becomes:

$$\begin{aligned} \mathcal {L}_{i}^{C}\left( \theta ^{C}\right)&=\left\{ \prod _{j=1}^{J}\left[ \frac{\exp \left( X_{j}^{RP}\beta ^{C} +\xi _{j}^{C}+p_{ij}^{RP}\gamma ^{C}\right) }{\sum _{m=1}^{J}\exp \left( X_{j}^{RP}\beta ^{C}+\xi _{j}^{C}+p_{im}^{RP} \gamma ^{C}\right) }\right] ^{n_{ij}^{RP}}\right\} \nonumber \\&\quad \cdot \prod _{h=1}^{H} \left[ \prod _{k=1}^{K}\left\{ \frac{\exp \left[ \omega \left( X_{ikh}^{SP}\beta ^{C}+p_{ikh}^{SP}\gamma ^{C}\right) \right] }{\sum _{r=1}^{K} \exp \left[ \omega \left( X_{irh}^{SP}\beta ^{C}+p_{irh}^{SP}\gamma ^{C}\right) \right] }\right\} ^{1_{ikh}^{SP}}\right] . \end{aligned}$$
(6)

where \(\omega \equiv \mu ^{RP}/\mu ^{SP}\) is the ratio of RP and SP scale parameters and \(\theta ^{C}\equiv (\xi _\bullet ^{C},\gamma ^{C},\beta ^{C},\omega )\) and \(\xi _\bullet ^{C}\equiv (\xi _1^{C},\ldots ,\xi _{J-1}^{C})\). Note that, unlike in the case when consistency was not imposed, we can now estimate the composite impact of the unobservable factors (i.e., the \(\xi _j^C\)’s). Also note that in imposing consistency we are not requiring that the scale parameter be the same across the two data sources, but we still need to normalize \(\mu ^{RP}=1\).

3.2 Latent Class Model

The standard approach in the literature is to estimate both the consistent and inconsistent models (i.e., using the likelihood functions in Eqs. (6) and (5), respectively) and to choose between the two specifications based standard tests. The model being proposed in this paper is to consider an in-between approach, allowing for the possibility that individuals differ in terms of the consistency of their RP and SP responses. In particular, we adopt latent class model with two distinct groups: Class C in which individual exhibit consistent preference parameters across their RP and SP data sources as in depicted in (6) and Class IC in which individuals have disparate RP and SP parameters as depicted in (5). Class membership is not known to the analysts. Therefore, the overall likelihood function (i.e., unconditional on class membership) for individual i can be formulated as

$$\begin{aligned} \mathcal {L}_{i}(\theta )=s\mathcal {L}_{i}^{C}\left( \theta ^{C}\right) +(1-s)\mathcal {L}_{i}^{IC}\left( \theta ^{IC}\right) \end{aligned}$$
(7)

where \(s\in [0,1]\) is the probability of being in the consistent class and \(\theta \equiv (\theta ^{C},\theta ^{IC},s)\) denotes the full set of parameters to be estimated. The class membership probability can be modeled as a function of individual characteristics, including the individual’s socio-demographic or attitudinal characteristics (see, e.g., Boxall and Adamowicz 2002). The advantage of this approach is that, by understanding the factors that influence membership in the inconsistent class, researchers may be able target corrective measures to avoid the inconsistencies themselves. For now, however, we focus on the simpler case is which the probability of class membership is a constant.Footnote 4

Equation (7) can be used directly to estimate all of the model’s parameters, including the class membership probability s, by standard maximum likelihood techniques. However, latent class models are notoriously difficult to estimate directly. Instead, following the current practice in the latent class literature (e.g., Morey et al. 2006; Evans and Herriges 2010), we employ an Expectation-Maximization (EM) algorithm. The next subsection briefly describes steps involved in the EM algorithm used in our applications.

3.3 EM Algorithm

EM algorithms can be useful for maximizing a likelihood function when standard optimization procedures can be numerically challenging, which is often the case in the presence of latent variables and particularly the case in latent class models. In our framework, the latent variable is class membership \(c_i\), which equals C if the individual belongs to the consistent class and equals IC if the individual belongs to the inconsistent class, with \(Pr(c_i=C)=s\).

The EM algorithm is an iterative procedure, alternating between two steps: (1) Calculating an expectation as a function of the current iteration’s parameter values and (2) maximizing that expectation with respect to the parameters of the model. Specifically, following the general notation in chapter 14 of Train (2009), let \(\theta _{t}\) denote the value of the parameters at iteration t. To maximize (7) using the EM algorithm, we define a new function evaluated at \(\theta _{t}\) that can be used to obtain the parameter vector’s next iteration; i.e., \(\theta _{t+1}\). Specifically, let

$$\begin{aligned} \mathcal {E}(\theta |\theta _{t})&\equiv \sum _{i=1}^{N}\left\{ h_{it}^{C}\ln \left[ s\mathcal {L}_{i}^{C}(\theta ^{C})\right] +h_{it}^{IC}\ln \left[ (1-s)\mathcal {L}_{i}^{IC}(\theta ^{IC})\right] \right\} \nonumber \\&=\sum _{i=1}^{N} \left[ h_{it}^{C}\ln (s)+h_{it}^{IC}\ln (1-s)\right] \!+\!\sum _{i=1}^{N}h_{it}^{C}\ln \left[ (\mathcal {L}_{i}^{C}(\theta ^{C})\right] \!+\!\sum _{i=1}^{N}h_{it}^{IC}\ln \left[ \mathcal {L}_{i}^{IC}(\theta ^{IC})\right] \end{aligned}$$
(8)

where s is the share of the population in class C and \(h_{it}^{c}\) denotes the probability of membership in class c (\(c=C,IC\)) conditional on the individual’s observed choices. Using Bayes rule:

$$\begin{aligned} h_{it}^{c}=h(c_{i}=c|y_{\bullet },s_{t})=\frac{s_{t}\mathcal {L}_{i}^{c}(\theta ^{c})}{s_{t}\mathcal {L}_{i}^{C}(\theta ^{C})+(1-s_{t})\mathcal {L}_{i}^{IC}(\theta ^{IC})} \end{aligned}$$
(9)

where \(y_\bullet \) denotes the full set of choices (i.e., the \(n_{ij}^{RP}\)’s and \(1_{ikh}^{SP}\)’s). Forming this expectation represents the first step in the EM algorithm.

The second step involves maximizing \(\mathcal {E}(\theta |\theta _{t})\) with respect to \(\theta \). Conveniently, as can be seen in Eq. (8), \(\mathcal {E}(\theta |\theta _{t})\) is separable into three distinct components that can be independently maximized. In particular, maximizing \(\mathcal {E}(\theta |\theta _{t})\) with respect to s corresponds to maximizing

$$\begin{aligned} \mathcal {E}(s|\theta _{t})=\sum _{i=1}^{N}\left[ h_{it}^{C}\ln (s)+h_{it}^{IC}\ln (1-s)\right] , \end{aligned}$$
(10)

yielding

$$\begin{aligned} s_{t+1}=\frac{\sum _{i=1}^{N}h_{it}^{C}}{\sum _{i=1}^{N}\left( h_{it}^{C}+h_{it}^{IC}\right) }. \end{aligned}$$
(11)

Maximizing \(\mathcal {E}(\theta |\theta _{t})\) with respect to \(\theta ^c\) (\(c=C,IC\)) corresponds to maximizing

$$\begin{aligned} \mathcal {E}(\theta ^c|\theta _{t})=\sum _{i=1}^{N}h_{it}^{c}\ln \left[ (\mathcal {L}_{i}^{c}(\theta ^{c})\right] , \end{aligned}$$
(12)

which is just class-specific maximum likelihood estimation using \(h_{it}^{c}\) as weights. The updated parameters (i.e., \(\theta _{t+1}^c\)) are the corresponding solutions to these maximizations; i.e.,

$$\begin{aligned} \theta _{t+1}^c=arg\max _{\theta ^c} \sum _{i=1}^{N}h_{it}^{c}\ln \left[ (\mathcal {L}_{i}^{c}(\theta ^{c})\right] . \end{aligned}$$
(13)

Thus, the steps for estimation of the latent class model using the EM algorithm are

  1. 1.

    Specify initial values for the share and coefficients in each class. We set \(s_{0}=0.5\) and obtain \(\theta ^c_0\) for class c using unweighted maximum likelihood for that class.

  2. 2.

    Calculate the probability of being in each class conditional on the observed choices using (9).

  3. 3.

    Update the share s of class C using (11).

  4. 4.

    Update the parameters of each class by estimating weighted MLE using (13)

  5. 5.

    Repeat steps 2–4 until convergence.

4 Generated Data Experiments

In this section, we describe a series of generated data experiments designed to illustrate the latent class model introduced in Sect. 3. Particular attention is paid to the performance of the model given different sample sizes and the proportion of the population belonging to the consistent class, as well as the impact of erroneously assuming that this class proportion is either zero or 1. Throughout, the pseudo-data sets were structured so as to mimic the general structure of the data sets used in the applications in Sect. 5.

As described in previous section, each individual is assumed to belong to either the consistent class (\(c_i=C\)) or inconsistent class (\(c_i=IC\)), with \(Pr(c_i=C)=s\). Using a slight generalization of the model from the previous section (i.e., incorporating interactions between site and individual characteristics), the RP and SP conditional utilities for individuals belonging to the consistent class are assumed to take the form:

$$\begin{aligned} U_{ijt}^{RP}&= X_{j}^{RP}\beta ^{C}+Z_{i}X_{j}^{RP}\rho ^{C}+p_{ij}^{RP}\gamma ^{C}+ \xi _j^{RP} +\mu ^{RP}\varepsilon _{ijt} \nonumber \\ U_{ikh}^{SP}&= X_{k}^{SP}\beta ^{C}+Z_{i}X_{k}^{SP}\rho ^{C}+p_{ik}^{SP}\gamma ^{C}+ \mu ^{SP}\varepsilon _{ikh} \end{aligned}$$
(14)

where \(Z_i\) denotes an individual characteristics such as age, gender, or education. On the other hand, for individuals belongs to the inconsistent class, these conditional utilities are assumed to take the form:

$$\begin{aligned} U_{ijt}^{RP}= & {} X_{j}^{RP}\beta ^{RP}+Z_{i}X_{j}^{RP}\rho ^{RP}+p_{ij}^{RP}\gamma ^{RP}+ \xi _j^{RP} +\mu ^{RP}\varepsilon _{ijt}\nonumber \\ U_{ikh}^{SP}= & {} X_{k}^{SP}\beta ^{SP}+Z_{i}X_{k}^{SP}\rho ^{SP}+p_{ik}^{SP}\gamma ^{SP}+ \mu ^{SP}\varepsilon _{ikh} \end{aligned}$$
(15)

In the generated data experiments, we consider a total of 15 scenarios varying the scenarios along two dimensions:

  1. 1.

    The probability of membership in the consistent class, with \(s\in \{0.1, 0.25, 0.5, 0.75, 0.9\}\); and

  2. 2.

    The number of observations, with \(N\in \{200, 500, 1000\}\).

In all of the scenarios, the number of alternatives available on each choice occasion is fixed in the RP and SP settings, with \(J = 20\) and \(K = 3\), respectively. The corresponding total number of choice occasions are likewise fixed for the RP and SP settings, with \(T=10\) and \(H=15\), respectively. Finally, for each scenario, 100 generated data sets were constructed.

The specific steps used to generate data sets are as follows:

  1. 1.

    The vector of individual characteristics (\(Z_{i}\)), site characteristics (\(X_{j}\)), and travel costs (\(p_{ij}\)) were drawn from the following distributions:

    $$\begin{aligned} Z_{i}\sim & {} N(0,1)\\ X_{j}^{RP}\sim & {} N(0,1)\\ X_{k}^{SP}\sim & {} N(0,2)\\ p_{ij}^{RP}\sim & {} \log N(0,1)\\ p_{ik}^{SP}\sim & {} \log N(0,2) \\ \xi _j^{RP}\sim & {} N(-2,0.05) \end{aligned}$$
  2. 2.

    Each individual in the sample was then randomly assigned to either the consistent class (i.e., \(c_i=C\)) or the inconsistent class (i.e., \(c_i=IC\)), with \(Pr(c_i=C)=s\).

  3. 3.

    Depending upon the class to which they were assigned, either Eqs. (14) or (15) were then used to generate the conditional utilities \(U_{ijt}^{RP}\) and \(U_{ikh}^{SP}\) for each choice occasion and alternative employing the following parameters:

    • \(\beta ^{C}=-2.0\);

    • \(\rho ^{C}=-3.0\);

    • \(\gamma ^{C}=-0.8\); and

    • \(\omega =0.4\)

    for the consistent class and

    • \(\beta ^{RP}=-1.2\);

    • \(\rho ^{RP}=-0.7\);

    • \(\gamma ^{RP}=-1.8\);

    • \(\beta ^{SP}=-0.6\);

    • \(\rho ^{SP}=-0.5\); and

    • \(\gamma ^{SP}=-0.4\).

    for the inconsistent class. For both classes, the error terms (i.e., \(\varepsilon _{ijt}\)’s and \(\varepsilon _{ikh}\)’s) were drawn from the Type I extreme value distribution.

  4. 4.

    Given the conditional utilities \(U_{ijt}^{RP}\) and \(U_{ikh}^{SP}\) for each choice occasion, the individual’s choices (i.e., \(1_{ijt}^{RP}\) and \(1_{ikh}^{SP}\)) were then determined by the alternative yielding the highest utility.

For each generated sample, we estimate three different models:

  • Model 1. The latent class model described in Section 4 and based on the likehood function in Eq. (7);

  • Model 2. The fully inconsistent model based on the likelihood function in Eq. (5); and

  • Model 3. The fully consistent model based on the likelihood function in Eq. (6).

We then compare and contrast the three models in terms of the implied welfare impact from closing the most popular site in the sample.

Table 1 Generated data experiments—Model 1 parameter estimates

Table 1 summarizes the resulting parameter estimates for Model 1.Footnote 5 In particular, for each scenario (i.e., combination of s and N), the table reports the mean parameter estimates across the 100 replications, as well as the corresponding 5th and 95th percentile values. Since Model 1 is consistent with the underlying data generating process, it is not surprising that the mean parameter estimates are generally quite close to the true parameters. However, the estimates are less stable when the share of individuals in the consistent class (i.e., s) is small. This is to be expected since the estimation then relies on relatively few individuals to identify the parameters for the consistent class. Somewhat unexpected is the fact that the parameter estimates are not as varied at the other extreme (i.e., when \(s=0.9\)).

Table 2 Generated data experiments: mean abs. percentage errors of welfare estimates

Parameter estimates using the other two models (i.e., Models 2 and 3), are provided in Appendix Tables 8 and 9, respectively. Since these models are not consistent with the underlying data generating process, it is not surprising that they tend to yield greater departures from the underlying parameters of the model. In general, Model 2 performs relatively well when most of the population is drawn from the inconsistent class (e.g. \(s=0.1\)), whereas Model 3 performs relatively well when most of the population is drawn from the consistent class (e.g., \(s=0.9\)).Footnote 6

Perhaps more important than the performance of a model in terms of individual parameter estimates is its performance in estimating the welfare impacts of a proposed policy scenario. Table 2 summarizes the performance of the three models in terms of estimating the average welfare impact of two policy scenarios:

  • Scenario A. Closure of site 1.

  • Scenario B. Improvement in site quality for site 1. This corresponds to a fifty percent reduction in \(X_1^{RP}\), where \(X_1^{RP}\) is a bad (i.e., \(\beta ^{RP}<0\)).

For the latent class model (i.e., Model 1), the appropriate welfare measure is a weighted average of the compensating variation from the consistent and inconsistent class models, with the weights being the corresponding class probabilities; i.e.,

$$\begin{aligned} CV= & {} s\times CV^{C}+(1-s)CV^{RP} \end{aligned}$$
(16)

where s is the probability of being in the consistent class, with \(CV^{C}\) and \(CV^{RP}\) denote the standard log-sum calculations based on the consistent class and inconsistent class RP parameter estimates, respectively.

In contrast, the standard approaches in the literature are to either not impose consistency across the RP and SP data source (as in Model 2), computing compensating variation based on the RP parameter estimates alone, or to impose consistency for all individuals (as in Model 3), computing compensating variation based on the constrained parameter estimates derived from the two data sources.

Table 2 summarizes the mean absolute percentage errors (MAPE) associated with these three approaches, i.e.

$$\begin{aligned} { MAPE}= & {} \frac{1}{N}\sum _{i=1}^{N}\left| \frac{\text{ True } \text{ welfare } \text{ loss }_{i}-\text{ Welfare } \text{ loss } \text{ estimates }_{i}}{\text{ True } \text{ welfare } \text{ loss }_{i}}\right| \end{aligned}$$
(17)

For all six experiments, the MAPEs are generally lowest for the latent class model (i.e., Model 1), which should be the case since it is in accord with the underlying data generating process. For Scenario A (the closure of site 1), the MAPE’s from the latent class model lie between 5 and 12 %, with the errors diminishing as the available sample size increases. The errors are larger for both single class specifications.

The MAPE’s are substantially larger for Scenario B, ranging from approximately 10 percent when \(N=1000\) and \(s=0.5\) to over 70 percent when \(N=200\) and \(s=0.10\). This pattern is not surprising. The larger errors for Scenario B are expected, since welfare calculation in this case depends crucially on estimates of \(\beta ^{RP}\), which are obtained from a second stage regression of only \(J=20\) site alternative specific constants on site attributes \(X_j^{RP}\). The MAPE’s are typically smallest for the latent class model when \(s=0.5\), with the population evenly divided between the inconsistent and consistent classes, effectively providing a more balanced bases for estimating the underlying class parameters. In contrast, when \(s=0.1\), only 10 percent of the sample is assumed to be from the consistent class, providing little information for gleaning the parameters of that class. As was the case for Scenario A, Scenario B generally yields higher MAPE’s for the single class specifications. The consistent class model performs best as the proportion of individuals in the consistent class is largest (i.e., \(s=0.9\)), whereas the inconsistent class model performs best as the portion of individuals in the inconsistent class is largest (i.e., \(s=0.1\)).

Table 3 Summary statistics

The generated experiments in this section illustrate the potential of our model to recover the underlying preference parameters in a setting where individuals differ in terms of the consistency between their RP and SP responses, as well as the biases induced by assuming that consistency is an all or nothing proposition. To the extent that households fall largely into the consistent or inconsistent categories (i.e., s is close to one or zero, respectively), the traditional approach provides a reasonable approximation to preferences. However, if the population is more evenly divided between the two possibilities, both the preference parameters and corresponding welfare estimates can be significantly biased.

5 Application

5.1 Data

To illustrate our proposed latent class model, we reconsider two Moose Hunting data sets used by Adamowicz et al. (1997), Haener et al. (2001), and von Haefen and Phaneuf (2008) to examine the potential for combining RP and SP data sources. The first of the data set (the Alberta Moose Hunting Study) was collected from a sample of 422 individuals drawn from moose hunting license holders living in the Canadian towns of Drayton Valley, Edson, Hinton, Edmonton, and Whitecourt. Individuals were initially contacted by mail, with a follow-up phone call inviting them to attend a meeting. Of the 422 hunters initially contacted, 312 individuals (74 %) agreed to attend the meeting. Of the 312 hunters who confirmed attendance, 271 (87 %) actually attended the meeting.Footnote 7

The Alberta study provides both revealed preference (RP) and stated preference (SP) data. The RP data consists of reported moose hunting trips to 14 wildlife management units (WMUs) during 1992, as well as respondent socio-demographic characteristics. SP data takes the form of a choice experiment in which each respondent was presented with a series of 16 choice scenarios (i.e., \(H=16\)) each including three alternatives (i.e., \(K=3\)), with two of the alternatives involving hypothetical sites while the third alternative was an opt-out (i.e., not hunting) option.Footnote 8

Table 3.a reports summary statistics from the Alberta Moose Hunting Study for both individual and site characteristics. The mean age of hunters in the sample was just under forty years, and they had an average of about 20 years of general hunting experience and about 16 years of experience hunting moose. More than half of hunters completed high school and most of them reported incomes in the ranges of $20,000–$60,000. For both real (RP) and hypothetical (SP) sites, the alternatives are defined in terms of six attributes: travel cost, moose population, level of congestion, access within hunting area (no trail, cutlines or seismic lines), quality of road and the presence of forest activity (logging).

The second data set (the Saskatchewan Moose Hunting Study) is similar in structure to its Alberta counterpart, extracting both RP and SP information from 532 moose hunting license holders. In the Saskatchewan database, respondents provide RP information on trips to 11 wildlife management zones (WMZ’s), together with SP data on response to up to 14 choice scenarios. Table 3.b provides the corresponding summary statistics for the Saskatchewan study.

5.2 Results

A total of four models were estimated using the two moose hunting data sets:

  1. 1.

    SC-C. A single class (SC) model imposing consistency across the RP and SP data sources;

  2. 2.

    SC-RP. A single class model of preferences based only on the RP data;

  3. 3.

    SC-SP. A single class model of preferences based only on the SP data;

  4. 4.

    LC. A latent class model with a portion s belonging to the consistent class (denoted LC-C) and a portion \((1-s)\) belong to the inconsistent class (denoted by LC-RP and LC-SP for the revealed and stated preference components, respectively).

Tables 4 through 6 provide the resulting parameter estimates.Footnote 9 Table 4 focuses on the core parameters for both data sets; i.e., the class share s in the case of the latent class model, the relative RP/SP scale parameter \(\omega \) identified only when consistency is imposed for a class, and the travel cost parameters (i.e., the \(\gamma \)’s). Tables 5 and 6 report the main effect of site characteristics (i.e, the \(\beta \)’s) and interactions between site characteristics and individual attributes (i.e., the \(\rho \)’s) for the Alberta and Saskatchewan data sets, respectively.Footnote 10

Table 4 Core parameter estimates
Table 5 Parameter estimates for site characteristics—Alberta study
Table 6 Parameter estimates for site characteristics—Saskatchewan study

Starting with Table 4a, the latent class model in the Alberta setting indicates that the population is roughly evenly divided between the consistent and inconsistent classes, with \(s=0.51\). Both the single and latent class models yield a significant difference in scales between the RP and SP responses, with \(\omega \) in the range of 0.19–0.22. This indicates that there is greater variability in the unobservable components of individual preferences in the case of SP data relative to RP data (i.e. \(\mu ^{RP}<\mu ^{SP}\)). Finally, while all of the specifications yield negative and statistically significant travel cost coefficient, the \(\gamma \)’s vary substantially. Cross-model comparisons of the estimated \(\gamma \)’s is difficult, since the scale parameter differences between the RP and SP models cannot be estimated when consistency is not imposed. However, it does appear as though the latent class structure highlights the gap between consistent and inconsistent preferences. As indicated by Table 4b, a similar pattern of results emerges in the Saskatchewan data set. The most obvious difference is that the consistent class is a significantly smaller proportion of the population, with s roughly equal to one-third (rather than one-half) of the population.

Turning to Tables 5 and 6, note that there are two sets of parameters being presented. In each table, the first column of parameters are the main effects associated with the site characteristics; i.e., the \(\beta \)’s in Eq. 1. For those models involving only the RP data, the \(\beta \)’s can generally only be recovered in a second stage regression using the estimated ASC’s (i.e., the \(\alpha _j\)’s) and Eq. (3).Footnote 11 However, in the Alberta study, with \(J=14\), the main effects for the eleven site characteristics used by von Haefen and Phaneuf (2008) cannot be reasonably estimated and are not reported here. A similar problem emerges for the Saskatchewan study, with \(J=11.\) In each table, the second set of parameters are the \(\rho \)’s in Eq. (14), reflecting interactions between individual and site characteristics. In general, these parameters vary substantially across the various RP and SP specifications, often changing signs and significance. The pattern of these parameters for the single class models are similar to those reported in von Haefen and Phaneuf (2008).

Interpreting the individual parameters in Tables 5 and 6 is difficult. In order to illustrate the differences across the various models, we consider their implications in terms of welfare estimates for the three scenarios considered for the Alberta (Saskatchewan) data set by von Haefen and Phaneuf (2008) :

  • Scenario A. The closure of site WMU #344 (WMZ #5).

  • Scenario B. A decrease moose population from more than 4 per day to 3-4 per day at WMU #348 (WMZ #7).

  • Scenario C. An increase moose population from less than 1 per day to 1-2 moose per day at WMU #344 (WMZ #5).

In order to see the alternative CV measures one can compute using the competing models, consider a generic model of the form:

$$\begin{aligned} U_{ijt}&=X_j\beta + Z_iX_j\rho + p_{ij}\gamma +\xi _j+\mu \epsilon _{ijt} = V_{ij}+\mu \epsilon _{ijt}, \end{aligned}$$
(18)

where \(V_{ij}=X_j\beta + Z_iX_j\rho + p_{ij}\gamma +\xi _j=\alpha _j+ Z_iX_j\rho + p_{ij}\gamma .\) The compensating variation (per choice occasion) associated with the loss of a single site (as in Scenario A), say site 1, would have the familiar form:

$$\begin{aligned} CV_{i1A}\!=\!\frac{1}{\gamma /\mu }\left\{ ln\left[ \sum _{j\ne 1} exp\left( \frac{V_{ij}}{\mu }\right) \right] \!-\! ln\left[ \sum _{j=1}^J exp\left( \frac{V_{ij}}{\mu }\right) \right] \right\} \!=\!\frac{1}{\gamma /\mu }ln\left( 1\!-\!P_{i1}\right) , \end{aligned}$$
(19)

where

$$\begin{aligned} P_{i1}=\frac{exp\left( \frac{V_{i1}}{\mu }\right) }{\sum _{j=1}^J exp\left( \frac{V_{ij}}{\mu }\right) } \end{aligned}$$
(20)

denotes the probability of choosing site 1 under baseline conditions. Note that we have explicitly carried through the scale parameter \(\mu \) to emphasize its role in the CV calculation. There are several implications of Eq. (19). First, because we are interested in valuing a specific site, parameters from models based on SP data alone (i.e., SC-SP or LC-SP) cannot be used to compute \(CV_{i1A}\). Equation (19) requires estimated of the \(\xi _j\)’s or \(\alpha _j\)’s in order to compute \(V_{ij}\)’s and the RP data in this case is the only source of information on these parameters. Second, the scale parameter used in the calculation of \(P_{i1}\) must be the same as the scale parameter used in computing the normalized marginal utility of income \(\gamma /\mu \). This leaves five possible measures of \(CV_{i1A}\) using the estimated models above:

  1. 1.

    Measure 1: Single Class Consistent Model. Using the model imposing consistency across the RP and SP data sources, we can compute

    $$\begin{aligned} CV_{1A}^{SC-C}&=\frac{1}{\gamma ^{SC-C}}ln\left( 1-P^{SC-C}_{i1}\right) , \end{aligned}$$
    (21)

    where \(\gamma ^{SC-C}\) denotes the estimate of \(\gamma ^C\) based on the model in Eq. (6). \(P^{SC-C}_{i1}\) is computed using the corresponding parameter estimates. This measure combines data from both the RP and SP data sources, but imposes consistency across the two sources.

  2. 2.

    Measure 2: Single Class RP Model. Using only the RP data source, we can compute

    $$\begin{aligned} CV_{1A}^{SC-RP}&=\frac{1}{\gamma ^{SC-RP}}ln\left( 1-P^{SC-RP}_{i1}\right) , \end{aligned}$$
    (22)

    where \(\gamma ^{SC-RP}\) denotes the estimate of \(\gamma ^{RP}\) based on the model in Eq. (5). The measure makes no use of the SP data sources.

  3. 3.

    Measure 3: Latent Class Model—Consistent Class Only. Using our latent class model results for the consistent class, we can compute

    $$\begin{aligned} CV_{1A}^{LC-C}&=\frac{1}{\gamma ^{LC-C}}ln\left( 1-P^{LC-C}_{i1}\right) , \end{aligned}$$
    (23)

    where \(\gamma ^{LC-C}\) denotes the estimate of \(\gamma ^C\) for the consistent class in the latent class model. While this measure draws on both the RP and SP data sources, it does so only for the consistent class, and thus may not be representative of the population as a whole.

  4. 4.

    Measure 4: Latent Class Model—Inconsistent Class RP Results Only. Using our latent class model results for the RP portion of the inconsistent class, we can compute

    $$\begin{aligned} CV_{1A}^{LC-RP}&=\frac{1}{\gamma ^{LC-RP}}ln\left( 1-P^{LC-RP}_{i1}\right) , \end{aligned}$$
    (24)

    where \(\gamma ^{LC-RP}\) denotes the estimate of \(\gamma ^{RP}\) for the inconsistent class in the latent class model. This measure ignores both SP data and the consistent class portion of the population.

  5. 5.

    Measure 5: Latent Class Model—Weighted Average. Using our latent class model results, we can compute weighted average of class-specific CV estimates following (16); i.e.,

    $$\begin{aligned} \overline{CV}_{1A}^{IC}&=s\cdot CV_{1A}^{LC-C} + (1-s)\cdot CV_{1A}^{LC-RP}, \end{aligned}$$
    (25)

    The advantage of this, our preferred measure, is that it incorporates information from both classes and, for the class indicating consistency between the RP and SP data sources, it draws on the SP data to improve the estimate of the associated welfare loss.

In the case of a quality changes to the characteristics of a single site (as in Scenarios B and C), the possibilities are more limited. The general compensating variation formula for a change in the observed attributes of site 1 from \(X_1\) to \(X_1 + \Delta X_1\) takes the generic form

$$\begin{aligned} CV_{i1B}&=\frac{1}{\gamma /\mu }\left\{ ln\left[ exp\left( \frac{V_{i1}+\Delta X_1\beta + Z_i\Delta X_1\rho }{\mu }\right) +\sum _{j\ne 1} exp\left( \frac{V_{ij}}{\mu }\right) \right] \right. \nonumber \\&\quad \left. -ln\left[ \sum _{j=1}^J exp\left( \frac{V_{ij}}{\mu }\right) \right] \right\} \nonumber \\&=\frac{1}{\gamma /\mu }ln\left[ P_{i1}\cdot exp\left( \frac{\Delta X_1\beta + Z_i\Delta X_1\rho }{\mu }\right) +\left( 1-P_{i1}\right) \right] . \end{aligned}$$
(26)

The available welfare measures are few in the current application. In particular, we can no longer construct the counterparts to Measures 2, 4 or 5 above, since for the Inconsistent class we are not able to isolate \(\beta ^{RP}\).Footnote 12 This leaves two possibilities:

  1. 1.

    Measure 1: Single (Consistent) Class Model. Using the model imposing consistency across the RP and SP data sources, we can compute

    $$\begin{aligned} CV_{1B}^{SC-C}&=\frac{1}{\gamma ^{SC-C}}ln\left[ P^{SC-C}_{i1}exp(\Delta X_1\beta ^{SC-C} + Z_i\Delta X_1\rho ^{SC-C})+\left( 1-P^{SC-C}_{i1}\right) \right] . \end{aligned}$$
    (27)
  2. 2.

    Measure 3: Latent Class Model—Consistent Class Only. Using our latent class model results for the consistent class, we can compute

    $$\begin{aligned} CV_{1B}^{LC-C}&=\frac{1}{\gamma ^{LC-C}}ln\left[ P^{LC-C}_{i1}exp(\Delta X_1\beta ^{LC-C} + Z_i\Delta X_1\rho ^{LC-C})+\left( 1-P^{LC-C}_{i1}\right) \right] . \end{aligned}$$
    (28)

Note that both of these measures are available only when consistency is imposed, either for the entire population (in the case of Measure 1) or for the consistent class (in the case of Measure 3).

Table 7 The results of welfare analysis

Table 7 provides the resulting welfare estimates for the three scenarios for each of the data sets. Starting with the Alberta study, the closure of site WMU #344 (Scenario A) yields welfare losses that vary substantially across the 5 measures. For example, the compensating variation is more than three times larger for the inconsistent class (Measure #4) relative to its consistent class counterpart (Measure #3). Our preferred measure in this case (Measure #5) provides a compromise, drawing on the SP to improve the welfare estimated for the consistent class (i.e., when consistency between the RP and SP sources is suggested), while drawing only on the RP source when consistency is not suggested. However, many of the welfare measures would not appear to be statistically significant from each other. Indeed, Measures #2, #4, and #5, all of which rely heavily on the RP data, are each imprecisely measured and not individually significantly different from zero.Footnote 13 On the other hand, Measures #1 and #3, which impose some form of consistency between the RP and SP data, are statistically significantly different from zero and would appear to be significantly different from each other. Unfortunately, a formal test in this case is not possible without an estimate of the correlation between the two measures, which we lack.

In the case of the Saskatchewan data set, we again see substantial differences among the competing welfare measures. However, in this study all of the welfare measures are individually significant at a one percent significance level and most would appear to be significantly different from each other. Indeed, because the parameters used in constructing Measures #3 through #5 are jointly estimated, we can explicitly test for pairwise differences among the measures. In all pairwise comparisons, the differences are statically significant.

Turning to the other two scenarios, there are substantial differences between the available measures for both studies and both scenarios. In the case of the Saskatchewan study, in particular, there would appear to be large and statistically significant differences between Measures #1 and #3. Unfortunately, there are also substantial limitations associated with both of the available welfare measures. Specifically, Measure #1 relies on the assumption that the RP and SP data are drawn from the same underlying preference, a restriction that is rejected by the data. Measure #3, on the other hand, only combines the RP and SP data for the consistent class, but the resulting welfare measure is applicable only for that class. There are two solutions to this problem in a more general setting. First, the latent class model could be generalized to make class membership a function of individual attributes. In this way, one could better characterize the subpopulation belonging to the consistent class and for whom Measure #3 would be applicable.Footnote 14 Second, in a setting with more sites, a second stage regression could be used to estimate marginal effect of individual site attributes, as suggested in Murdock (2006). In this case, five welfare measures analogous to the five welfare measures for site closure would be available, including Measure #5 that uses a class-weighted average of the welfare measures from the consistent and inconsistent classes.

6 Conclusion

Models of consumer preferences that draw on both stated and revealed preference data have the potential to improve upon models that rely on either data source alone. This potential, however, is predicated on the implicit or explicit assumption that the SP and RP data are truthful revelations of the same underlying preferences. To date, the literature has assumed that the consistency between RP and SP data is an “all or nothing” proposition. If consistency is not rejected by the data, then a combined RP/SP model is used to generate welfare estimates, whereas if consistency is rejected then the typical response is to rely upon revealed preference data alone. The purpose of this paper has been to suggest a middle ground, explicitly modeling the RP and SP data using a latent class framework in which the population is segmented into two classes; i.e., consistent and inconsistent classes. We see two distinct advantages to this approach. First, the resulting welfare calculations (our Measure #5 above) represents a compromise between the existing two extremes (Measures #1 and #2). For the consistent class, the welfare impact draws on both the RP and SP data sources, using the latter to increase the precision with which a given compensating variation is estimated, whereas for the inconsistent class the RP portion of the model alone is used. Second, the latent class model structure would allow class membership itself to be modeled as a function of individual characteristics or attitudinal metrics. In this case, one could potentially identify which respondents are more likely to fall into the inconsistent class, information that could potentially be used to mitigate such inconsistencies in future survey applications.