Food assistance programs and food insecurity: implications for Canada in light of the mixing problem

Abstract

In light of concerns about high rates of food insecurity, some have suggested that it might be time for Canada to implement national food assistance programs like those provided in the US, namely the Supplemental Nutrition Assistance Program (SNAP) and the National School Lunch Program (NSLP). In this paper, we assess how adopting these types of assistance programs would change the food insecurity rate in Canada among households with children. Using data from the Current Population Survey (CPS), we first evaluate the causal impact of these programs on food insecurity rates in the US using the Canadian definition of food security. Following other recent evaluations of food assistance programs, we use partial identification methods to address the selection problem that arises because the decision to take up the program is not random. We then combine these estimated impacts for the US with data from the Canadian Community Health Survey (CCHS) to predict how SNAP and NSLP would impact food insecurity rates in Canada. Partial identification methods are used to address the “mixing problem” that arises if some eligible Canadian households would participate in SNAP and others would not. The strength of the conclusions depends on the strength of the identifying assumptions. Under the weakest assumptions, we cannot determine whether food insecurity rates would rise or fall. Under our strongest nonparametric assumptions, we find that food insecurity would fall by at least 16% if SNAP were implemented and 11% if NSLP were implemented.

1 Introduction

In 2014, 8.2% of Canadian households were classified as moderately or severely food insecure (Tarasuk et al. 2016), a rate substantially higher than observed in 2007 despite the end of the Great Recession. These high rates, combined with recent evidence that food insecurity is associated with higher health care spending in Canada (Fitzpatrick et al. 2015; Tarasuk et al. 2015), have lead Canadians to become increasingly concerned about food insecurity.¹

Historically, Canada has addressed problems of economic hardship through a variety of income transfer programs, some of which have been found to reduce food insecurity.² At the same time, some have pondered whether national food assistance programs like those provided in the US, namely the Supplemental Nutrition Assistance Program (SNAP) and the National School Lunch Program (NSLP), might reduce food insecurity in Canada (see, e.g., Howard and Edge 2013; Power et al. 2015). After all, a growing body of research demonstrates that these programs reduce the incidence of food insecurity in the US (see, e.g., Kreider et al. 2012; Gundersen et al. 2012; references therein).³

In this paper, we provide the first formal evaluation of what would happen to the Canadian food insecurity rate in households with children if Canada introduced SNAP or NSLP. Since Canada does not provide these types of national assistance programs, data alone cannot reveal the outcome of interest. Instead, using the Canadian definition of food insecurity, we use recent data from the Current Population Survey (CPS) to estimate the causal impact of these programs on food insecurity rates in the US and then, based on these estimates, predict how the Canadian food insecurity rate would change if Canada were to adopt SNAP or NSLP. We focus on addressing two key methodological issues: the selection and mixing problems. First, as with other evaluations of food assistance programs, a selection problem arises because the decision to take up the program is not random. A nontrivial portion of eligible households do not participate in SNAP or NSLP, and this participation decision is likely to be endogenously related to factors associated with the underlying food insecurity outcomes. Second, when predicting prospective food insecurity rates in Canada under a new program, a “mixing problem” (Manski 1997a) arises if some eligible Canadian households would participate and others would not.

To evaluate these distinct identification problems, we build on earlier work using partial identification methods. After describing the data in Sect. 2, our analysis proceeds in two parts. In Sect. 3, we apply the partial identification methods developed in Kreider et al. (2012) to assess the impact of SNAP on food insecurity rates in the US. Using the Canadian definition of food insecurity and more recent data, we focus on addressing the selection problem that arises when a household’s decision to participate in SNAP or NSLP is not random.⁴ $^{,}$ ⁵ Much of the literature evaluating the impact of food assistance programs has addressed the selection problem by either maintaining the assumption that selection is exogenous or by applying a linear instrumental variable model. Yet, the exogenous selection assumption seems untenable, and researchers have struggled to find credible instrumental variables for programs with little cross-state or time variation.⁶ The methods developed in Kreider et al. (2012) allow us to evaluate the impact of food assistance programs under relatively weak assumptions that may possess greater credibility.⁷ The assumptions do not generally point-identify the average treatment effects, but they do partially identify them, yielding bounds rather than point estimates. Using the Canadian definition of food insecurity, the results in Sect. 3 suggest that SNAP and NSLP lead to meaningful reductions in food insecurity in the US. Under our strongest nonparametric assumptions, we find that SNAP reduces food insecurity rates between 16 and 45% and NSLP reduces rates between 11 and nearly 50%.

Given these estimated impacts of food assistance programs in the US, in Sect. 4 we turn to the problem of predicting potential food insecurity outcomes in Canada. Data from the Canadian Community Health Survey (CCHS) identify the food insecurity rates without SNAP or NSLP but contain no information about what would happen if eligible households were to receive benefits. To infer the food insecurity rates in this counterfactual scenario, we use the results from the Sect. 3 evaluation of the effects of SNAP and NSLP in the US to estimate the Canadian food insecurity rate if all eligible households were to participate in SNAP or NSLP.

Our main interest, however, is in learning what would happen if treatments are mixed across the Canadian population: some eligible households would participate, and others would not. The data alone cannot reveal the distribution of outcomes if the treatment selection process may be mixed. There is no unique resolution to this fundamental identification problem, and almost no attention has been given to resolving the ambiguity created by the mixing problem. In practice, the most common assumption is that all households would receive a single treatment. Parametric latent variable models describing how treatments are selected and outcomes determined may also identify the outcome probability (see, e.g., Dehejia 2005).

To address this mixing problem, we formalize the evaluation problem using the approach in Manski (1997a) and Pepper (2003). After considering what the data alone reveal, we then apply a constrained optimization model developed in Pepper (2003) and a monotone treatment response model. These nonparametric models, which are consistent with commonplace theories of the selection process, bound but do not identify the distribution of interest. We provide concluding remarks in Sect. 5.

2 Data

Our analysis uses data from the December Supplement of the 2011 CPS and from Statistics Canada’s 2009–2010 Canadian Community Health Survey (CCHS). The CPS is the official data source for poverty and unemployment rates in the US. Since 1995, the CPS has also been used as the official data source for food insecurity rates in the US. (see Coleman-Jensen et al. 2015). Like the CPS, the CCHS is used to calculate the official food insecurity rates in Canada (e.g., Tarasuk et al. 2014).

We restrict the samples from the CPS and the CCHS to households with children classified as income eligible for SNAP or NSLP.⁸ To be eligible for SNAP in the US, a household’s gross income cannot exceed 130% of the poverty line, net monthly income (gross income minus a standard deduction and expenses for care for disabled dependents, medical expenses, and excessive shelter costs) cannot exceed the poverty line, and assets must be less than $2000. Because the CPS does not provide sufficient information to measure net income and assets, we focus on gross income eligibility.⁹

To be eligible for NSLP, a household’s gross income must be less than 185% of the poverty line.¹⁰ Among those income eligible for NSLP, we evaluate two subsamples. The first comprises all households with children from ages 6 through 17. The second comprises households with elementary school-aged children (i.e., those between the ages of 6 and 13 in the CPS and 6 and 11 in the CCHS).¹¹ We consider both samples to reflect the possibility that Canada might implement a school lunch program only for certain ages.

All participants in SNAP are categorically eligible for NSLP and, conversely, a high proportion of NSLP participants are eligible for SNAP. Consequently, there is substantial overlap in participation in the two programs. While our samples of households with eligible children do overlap, we analyze the effects of SNAP and NSLP on food insecurity separately.¹²

Table 1 displays means of the three main indicator variables used in our analyses: SNAP participation, NSLP participation, and food insecurity among households with children eligible for SNAP or NSLP.¹³ The SNAP and NSLP participation indicators are based on self-reported measures of participation over the previous 12 months. Among income-eligible households, the self-reported SNAP participation rate is 57.4% and the NSLP participation rate is 67.7% among all households with children from ages 6 through 17.

Table 1 Food insecurity rates for the US and Canada and NSLP and SNAP participation rates for US

Food insecurity, which is notably lower in Canada than in the US, is measured through responses to a series of 18 survey questions used in both the CPS and the CCHS. Ten of the items pertain to all households, while eight focus on children; since we are examining household food insecurity, we use the full set of questions. The items include: “I worried whether our food would run out before we got money to buy more” (the least severe), “Did you or the other adults in your household ever cut the size of your meals or skip meals because there wasn’t enough money for food?,” “Did you ever cut the size of any of the children’s meals because there wasn’t enough money for food?,” and “Did any of the children ever not eat for a whole day because there wasn’t enough money for food?” (the most severe for households with children).

Based on the survey responses, households are classified into the several standard food insecurity categories. The most often used categorization in the US is food secure (two or fewer affirmative responses) versus food insecure (three or more affirmative responses). However, the threshold used to define food insecurity is arbitrary since the 18 questions function as a scale of severity, with even a single affirmative response denoting some level of vulnerability (Coleman-Jensen 2010). Canada applies a less conservative threshold, classifying a household as food insecure over the previous 12 months if it responds affirmatively to two or more child questions regarding food insecurity or to two or more adult questions (Health Canada 2007). Thus, if a household is deemed food insecure by the US measure, they would also be deemed food insecure in Canada but the converse does not hold.¹⁴ Given that the interest in this paper is in the potential impact of SNAP and NSLP in Canada, we use Health Canada’s definition of food insecurity.

3 The effects of SNAP and NSLP on food insecurity in the US

We begin by focusing on the problem of evaluating the impact of food assistance on food insecurity in the US, using the Canadian definition of food insecurity (see Sect. 2). For binary outcomes, this treatment effect can be expressed as

$$\begin{aligned} ATE =P[Y(1)=1]-P[Y(0)=1] \end{aligned}$$

(1)

where Y is the realized food insecurity rate, Y(1) denotes the potential rate if the household were to receive food assistance, and Y(0) denotes the analogous potential rate if the household were not to receive food assistance.¹⁵ $^{,}$ ¹⁶ The average treatment effect reveals the mean impact of food assistance participation, compared with nonparticipation, for a household chosen randomly from the underlying population of interest.

The mean response functions in Eq. (1) are not identified by the data alone. The potential outcome Y(1) is counterfactual for all households not receiving food assistance, while Y(0) is counterfactual for all households receiving food assistance. This is referred to as the selection problem. Using the Law of Total Probability, this selection problem can be highlighted by writing the first term of Eq. (1) as

$$\begin{aligned} P[Y(1)=1]=P[Y(1)=1|Z=1]P(Z=1)+P[Y(1)=1|Z=0]P(Z=0)\nonumber \\ \end{aligned}$$

(2)

where $Z=1$ denotes that a household received food assistance. The sampling process identifies the selection probability $P(Z=1)$, the censoring probability $P(Z=0)$, and the expectation of outcomes when the outcome is observed, $P[Y(1)=1|Z=1] \quad =P(Y=1|Z=1)$. However, the sampling process cannot reveal the mean outcome conditional on censoring, $P[Y(1)=1|Z=0]$. Given this censoring, $P[Y(1)=1]$ is not point-identified by the sampling process alone. Analogously, the second term in Eq. (1), $P[Y(0)=1]$ is also not identified.¹⁷

3.1 Nonparametric models to address the selection problem

To address the selection problem, we apply a number of different monotonicity restrictions. A natural starting point is to assume nothing about the counterfactual probabilities: what do the data alone reveal? Since the latent probability $P[Y(1)=1|Z=0]$ in Eq. (2) must lie within [0,1], it follows that

$$\begin{aligned} P[Y(1)=1|Z=1]P(Z=1)\le & {} P[Y(1)=1] \nonumber \\\le & {} P[Y(1)=1|Z=1]P(Z=1)+P(Z=0).\nonumber \\ \end{aligned}$$

(3)

Thus, the worst case lower bound on $P[Y(1)=1]$ in Eq. (2) is derived by assuming that nonrecipients would not have been food insecure had they received food assistance, and the upper bound is derived by assuming that nonrecipients would have been food insecure had they received food assistance. Analogous bounds apply to $P[Y(0)=1]$. A sharp (narrow as possible) upper bound on the ATE in Eq. (1) then follows by subtracting the lower bound on $P[Y(0)=1]$ from the upper bound on $P[Y(1)=1]$, and analogously for the lower bound.

Following Kreider et al. (2012), we use a number of middle ground assumptions that narrow the worst-case endogenous selection bounds in Eq. (3) by restricting relationships between food assistance participation, food insecurity outcomes, and observed covariates. In particular, we apply three common monotonicity assumptions.

First, the Monotone Treatment Selection (MTS) assumption (Manski and Pepper 2000) places structure on the selection mechanism through which households with children become food assistance recipients. A common theme in the literature is that unobserved factors associated with food insecurity are likely to be positively associated with a household’s decision to take up food assistance (Currie 2003). In this case, recipients have worse latent food security outcomes than nonrecipients on average.¹⁸ The MTS assumption is formalized as follows:

$$\begin{aligned} P[Y(j)=1|Z=0]\le P[Y(j)=1|Z=1], j=0,1. \end{aligned}$$

(4)

That is, conditional on either treatment, $j = 0$ or 1, eligible households that chose to participate in the program, $Z=1$, have a higher latent prevalence of food insecurity than eligible households that did not receive benefits.

Second, the Monotone Instrumental Variable (MIV) assumption (Manski and Pepper 2000) formalizes the notion that the latent probability of food insecurity, $P[Y(j)=1]$, varies monotonically with certain observed covariates. Let v be the observed monotone instrumental variable such that

$$\begin{aligned} u_1\le & {} u\le u_2 \Rightarrow P[Y(j)=1|v=u_2 ]\le P[Y(j)=1|v=u] \nonumber \\\le & {} P[Y(j)=1|v=u_1 ], j=0,1. \end{aligned}$$

(5)

These conditional probabilities are not identified, but they can be bounded using the methods above.

We apply two different MIV assumptions. First, following Kreider et al. (2012), we assume that the food insecurity rate weakly declines with the ratio of a family’s income to the poverty threshold.¹⁹ Second, as in Gundersen et al. (2012), we assume that the food insecurity rate is higher among income-eligible households than among ineligible households. For the ineligibles group, we use households between 130 and 150 percent of the poverty line for SNAP and between 185 and 200 percent for NSLP. In this case, the MIV bounds simplify. Assuming ineligible households do not receive SNAP, the sampling process point-identifies $P[Y(0)=1|v=\hbox {income ineligible]}$ as $P(Y=1|v=\hbox {income ineligible})$.²⁰ Thus, this MIV ineligibles restriction implies a lower bound on $P[Y(0)=1]$ in Eq. (1):

$$\begin{aligned} P[Y(0)=1]\ge P(Y=1|v=\hbox {income ineligible}) \end{aligned}$$

where the right-hand side is identified by the data. This restriction provides no information on $P[Y(1)=1]$.

Finally, despite the observed correlations in the data, the receipt of additional resources through participation in SNAP or NSLP is unlikely to increase a household’s probability of food insecurity. To formalize this idea, we use a generalized version of the Monotone Treatment Response (MTR) assumption (Manski 1997b and Pepper 2000):

$$\begin{aligned} P[Y(1)=1|Z]\le P[Y(0)=1|Z]. \end{aligned}$$

(6)

Given the realized treatment, this assumption implies that the food insecurity rate weakly decreases with participation in the food assistance program (see Kreider et al. 2016, working paper). While this MTR assumption precludes a strictly negative average treatment effect, it provides no information on the magnitude of the ATE and does not rule out a value of 0. Combining the MTR and MIV assumptions can further narrow the range of uncertainty about a program’s effect and may identify the sign of the effect.

3.2 Results

For each of the three subgroups, Table 2 displays the estimated bounds and confidence intervals under a variety of different models. In row (i), we present point estimates found under the exogenous selection assumption. Here, we replicate a primary finding in much of the literature that these programs are associated with higher rates of food insecurity. The food insecurity rate for SNAP recipients, for example, is 21.9 points higher than for eligible nonrecipients. Given that the decision to participate in SNAP or NSLP is not likely to be exogenous, this estimate should not be thought of as a credible estimate of the average treatment effect. Rather, this estimate indicates that there is a large gap in the food insecurity rates by participation status.

Table 2 Estimated bounds on the ATE of SNAP and NSLP on food insecurity under different models

Rather than assume selection is exogenous, it is important to ask what the data alone reveal. For the results in row (ii), we make no assumptions about how eligible households select into SNAP or NSLP. The width of these worst case selection bounds always equals 1, and the bounds always include zero. These bounds highlight a researcher’s inability to make strong inferences about the efficacy of government policies without making assumptions about unknown counterfactual outcomes. In the absence of restrictions that address the selection problem, we cannot rule out the possibility that these programs have a large positive or negative impact on the food insecurity rate.

The bounds are narrowed substantially under common monotonicity assumptions on the relationships between the latent outcome and observed instrumental variables (MIV) and treatment selection (MTS) (see rows iii–vi). Still, under these assumptions we cannot identify the sign of the ATE. For example, under the joint MTS+MIV model, we find the effect of SNAP lies within [-0.327, 0.112], and the effect of NSLP lies within [-0.389, 0.153] and [-0.364, 0.158], respectively, for households with children in the 6–13 and 6–17 age groups.

The last row in Table 2 adds the MTR assumption which, by construction, reduces the upper bound to zero. Combined with the MTS+MIV assumptions, the estimated upper bounds are all negative, suggesting that both SNAP and NSLP reduce food insecurity. In particular, SNAP is estimated to reduce the food insecurity rate by at least 9.2 percentage points. NSLP is estimated to reduce the food insecurity rate by at least 1.2 percentage points for the 6–13 age group and by at least 5.4 percentage points for the 6–17 age group, though these NSLP results are not significantly different from zero. The point estimates of these bounds suggest that SNAP and NSLP have substantial beneficial effects on food security in the US.²¹

4 Food insecurity rates in Canada: the mixing problem

A primary objective of this paper is to assess the food insecurity rates in Canada if SNAP or NSLP were available. Given that Canada does not have SNAP or NSLP, the data alone cannot answer this basic prospective question. The CCHS sampling process point-identifies the food insecurity rates in the absence of these national food assistance programs, but it does not provide any information about the rates that would be realized if all eligible households participated in SNAP or NSLP, or about which households would participate. To address this identification problem, we combine nonparametric assumptions with data from the CCHS and the CPS.

Our analysis proceeds in two parts. First, in Sect. 4.1 we assess prospective Canadian food insecurity rates under a uniform treatment regime in which either all households would participate in SNAP or NSLP, or all households would not participate. Using the notation from above, let $P_C [Y(1)=1]$ and $P_C [Y(0)=1]$, respectively, denote these two potential outcome probabilities for Canada. As noted above, data from the CCHS point-identify the food insecurity rate if no households were to participate, $P_C [Y(0)=1]$ (see Table 1), but they do not reveal the outcome if all eligible households were to participate. Instead, we partially identify the Canadian food insecurity rates that would occur if all eligible households were to participate in SNAP or NSLP using the MTR-MTS-MIV results from the evaluation of the ATE in the US (Sect. 3, Table 2).

Second, in Sect. 4.2 we estimate what the Canadian food insecurity rates would be if some eligible households would participate and others would not. In the US, for example, we estimate that 57.4% of eligible households participate in SNAP, while the other 43.6% do not participate (see Table 1).

4.1 The Canadian food insecurity rate with full take-up of SNAP or NSLP

Based on the results in Table 2, we first estimate food insecurity rates in Canada under homogenous treatment regimes in which either all households would participate or all households would not participate in SNAP or NSLP. As noted above, the CCHS sampling process point-identifies the food insecurity rate if no one would participate, $P_C [Y(0)=1]$. The food insecurity rates are estimated to be 32.9% for SNAP eligible households with children and 24.0% for NSLP eligible households with children age 6–17 (see Table 1). These data, however, do not provide any information about the Canadian food insecurity rate if all eligible households would participate in SNAP or NSLP, $P_C [Y(1)=1]$.

To draw inferences on this unknown probability, we apply an assumption that links the findings on the ATE in the US from Sect. 3 to what would happen if Canada adopted these national food assistance programs. Given the similarities between Canada and the US, we consider the possibility that SNAP and NSLP would have similar percentage effects on food insecurity.²² To formalize this idea, we assume the percent change in the potential outcome probabilities estimated for the US, $\frac{ATE}{P[Y(0)=1]} =\frac{P[Y(1)=1] -P[Y(0)=1]}{P[Y(0)=1]},$ equals the analogous percentage change in the potential outcome probabilities that would be realized in Canada, $\frac{P_C [Y(1)=1]-P_C [Y(0)=1]}{P_C [Y(0)=1]}$. Given this assumption, it follows that

$$\begin{aligned} P_C [Y(1)=1]=P_C [Y(0)=1]+\frac{P[Y(1)=1]-P[Y(0)=1]}{P[Y(0)=1]}P_C [Y(0)=1].\nonumber \\ \end{aligned}$$

(7)

On the right-hand side of this equation, notice that the CCHS data point-identify the Canadian food insecurity rate if all eligible households would not participate in SNAP or NSLP, $P_C [Y(0)=1]$, while the findings on the ATE of SNAP or NSLP in the US (see Sect. 3) provide bounds on the percentage change. In particular, we use the MTS-MIV-MTR model to bound this ratio. Thus, given this assumption, we can bound the Canadian food insecurity rate if all eligible households would participate in the program of interest.

Table 3 displays estimated bounds on the Canadian food insecurity rates when either all households participate or all households do not participate in SNAP or NSLP, along with bounds on the estimated relative (percentage change) effects of SNAP and NSLP on the US food insecurity rate. Panel A displays the results using upper bound estimates of $P_C [Y(1)=1]$, and Panel B displays results using lower bound estimates of this quantity. Focusing our attention on the identification problem arising from the mixing problem, we do not provide confidence intervals or other measures of statistical precision when presenting our empirical findings on the Canadian food insecurity rates.²³

Table 3 Canadian food insecurity rates with no participation or full participation in SNAP and NSLP

Recall that under the MTS-MIV-MTR assumption, we find that SNAP and NSLP reduce food insecurity (see Table 2)—that is, the upper bound is negative. Thus, given Eq. (7), we find that the Canadian food insecurity rate would fall if all eligible households participated in SNAP or NSLP. If all eligible households participated in SNAP, we estimate that the food insecurity rate would fall from 32.9% to somewhere between 17.8 and 27.5%. Likewise, if all eligible households participated in NSLP, we estimate that the food insecurity rate for households with children of ages of 6–11 (6–17) would drop from 24.3% (24.0%) to somewhere between 12.3 and 23.7% (12.5 and 21.4%). In other words, SNAP is estimated to reduce food insecurity rates for households with children by at least 16.3%, and possibly by as much as 45.9%. The NSLP is estimated to reduce food insecurity between 2.5 and 49.6% for households with children between the ages of 6–11, and between 10.7 and 47.8% for those with children between the ages of 6–17.

4.2 The Canadian food insecurity rate with partial take-up of SNAP or NSLP

If Canada were to adopt SNAP or NSLP, it is unlikely that all eligible households would participate. Rather, a selection process would lead some eligible households to take up the programs, whereas others would not. To address this mixing problem, we introduce a mixing process, m, whereby, for a given set of two mutually exclusive and exhaustive treatments, m determines which treatment each household receives (Pepper 2003). Each household then realizes a binary outcome of interest that may depend on the treatment. Given m, let $z_m$ be the realized treatment and $y_m$ be the realized outcome.

Given this mixing problem, the food insecurity rate that would occur under SNAP or NSLP with assignment policy m, $P_C (Y_m =1)$, is not observable. The estimated bounds in Table 3 cannot identify the probability a recipient receives a particular treatment nor the outcome probabilities among households that would receive that treatment. Rather, inferences will depend critically on the marginal distributions $P_C [Y(1)=1]$ and $P_C [Y(0)=1]$ along with prior information the evaluator can bring to bear on the mixing problem.

Here, using the estimates of $P_C [Y(1)=1]$ and $P_C [Y(0)=1]$ in Table 3, we consider what can be inferred about the prospective Canadian food insecurity rate under the mixing process, m. The problem we face is how to use the marginal distributions to bound the joint distribution. Starting with the basic setup in Manski (1997a) and Pepper (2003), we evaluate what can be learned about the outcome distribution under policy m given weak assumptions on the process determining treatment selection. The result is a bound on $P(Y_m =1)$.

To formalize the identification problem, it is useful to first explore how selection policies might affect outcomes. Treatment only affects some households. Recall that the latent outcomes equal 1 when a household with children would be food insecure. Thus, a fraction $P_C [Y(1)=1\cap Y(0)=0]$ of the population benefits from not receiving SNAP or NSLP, while a fraction $P_C [Y(1)=0\cap Y(0)=1]$ would benefit from receiving SNAP or NSLP. Regardless of the treatment assignment policy, at least the fraction $P_C [Y(1)=0\cap Y(0)=0]$ would be food secure and at least the fraction $P_C [Y(1)=1\cap Y(0)=1]$ would be food insecure. Thus, the joint distribution of outcome indicators, Y(1) and Y(0), implies that $P_C (Y_m =1)$ is bounded as follows:

$$\begin{aligned} P_C [Y(1)=1\cap Y(0)=1]\le P_C (Y_m =1)\le 1-P_C [Y(1)=0\cap Y(0)=0]. \end{aligned}$$

(8)

Notice that the width of the bound in Eq. (8) equals the fraction of households affected by the mixing process, $P_C [Y(1)=1\cap Y(0)=0]+P_C [Y(1)=0\cap Y(0)=1].$ In the absence of assumptions on the assignment policy, m, the joint distribution of Y(1) and Y(0) point-identifies the food insecurity rate only if treatments have no effect on outcomes. Otherwise, the precise location of the outcome probability depends on the mixing process among the affected populations.

Table 4 Prospective Canadian food insecurity rates among households with children under SNAP and $\hbox {NSLP}^{*}$

Since the data do not reveal the joint distribution of the food insecurity indicators, Y(1) and Y(0), the bounds in Eq. (8) are not identified. This section formalizes what knowledge of the marginal distributions on Y(1) and Y(0) combined with prior assumptions on the treatment selection policy reveal about the prospective food insecurity rate.

4.2.1 No-assumption bounds

A logical first step is to examine what these data reveal in the absence of assumptions. In fact, the observed marginal distributions imply informative restrictions on the joint distribution in Eq. (8). The “no-assumption” result is that knowledge of the outcome probabilities under homogenous treatment policies yields a one-sided bound on the outcome probability under policy m. Formally, using the Frechet (1951) bounds, Manski (1997a, Proposition 1) shows that

$$\begin{aligned}&\max \{0,P_C [Y(1)=1]+P_C [Y(0)=1]-1\} \nonumber \\&\quad \le P_C (Y_m =1) \nonumber \\&\quad \le \min \{1,P_C [Y(1)=1]+P_C [Y(0)=1]\}. \end{aligned}$$

(9)

In the absence of additional assumptions restricting the mixing problem, the food insecurity rate if Canada were to adopt SNAP is estimated to lie between 0 and about 0.604 based on the data alone (see Table 4). There are two sources of uncertainty reflected in these bounds (Pepper 2003). First, as in Eq. (8), the realized outcome probability depends on the unknown assignment rule, m. The lower bound, for example, is only realized if all who would benefit from SNAP or NSLP decide to participate. Second, additional uncertainty is introduced in that the data cannot reveal what fraction of the caseload is influenced by the treatment selection process.

In the absence of additional data, the only way to resolve ambiguous findings is to impose assumptions. In what follows, we examine the implications of three easily understood and commonly suggested restrictions. The first model assumes that households would take up SNAP or NSLP to minimize the probability of being food insecure. The second model restricts the fraction of households that would participate but makes no assumptions about the selection rule. In particular, we assume that between 90 and 95 percent of eligible households would take up SNAP or NSLP. In the third model, we impose the monotone treatment response assumption that SNAP and NSLP would not increase food insecurity rates.

4.2.2 Outcome optimization

We begin with the outcome optimization model that formalizes the assumption that a household would take up food assistance if doing so minimizes the chances of being food insecure. Suppose the household knows the response functions, Y(1) and Y(0), and minimizes food insecurity. Then, $Y_m =\min \{Y(1),Y(0)\}$, and the treatment selection policy would minimize the food insecure outcome probability. In this case, $P_C (Y_m =1)=P_C [Y(1)=1\cap Y(0)=1]$, the lower bound in Eq. (8). That is, the food insecurity rate can be no less than the fraction of households that would be food insecure regardless of whether they participate in SNAP or NSLP.

Formally, the Frechet (1951) bounds imply the following sharp restriction on $P(Y_m =1)$:

$$\begin{aligned}&\max \{0,P_C [Y(1)=1]+P_C [Y(0)=1]-1\} \nonumber \\&\quad \le P_C (Y_m =1) \nonumber \\&\quad \le \min \{P_C [Y(1)=1],P_C [Y(0)=1]\}. \end{aligned}$$

(10)

While the lower bound in Eq. (10) coincides with the no-assumption lower bound in Eq. (9), the upper bound is informative.

This model requires that each respondent knows the response functions and would decide to participate based solely on these functions. If other factors such as the costs of participating in the program play an important role, households with $Y(1)<Y(0)$ may still decide not to participate. In this case, the optimization model would not hold. While it seems likely that other factors play a role in the decision to participate, this outcome optimization model is a natural starting point for thinking about household decision-making. Below, we combine this optimization model with an assumption that some fraction of households would not participate in SNAP or NSLP.

4.2.3 Constrained participation

Next, we consider a model that places restrictions on the fraction of eligible households participating in the program. This model formalizes the idea that, perhaps because of the costs associated with taking up the program, only a fraction of eligible households participate. Formally, suppose that there is a known lower and upper bound on the participation rate such that $p_L \le P_C (z_m =1)\le p_U$. This restriction combined with the data implies sharp bounds on the outcome distribution. In particular, Pepper (2003) shows that

$$\begin{aligned}&\max \{0,P_C [Y(0)=1]+P_C [Y(1)=1]-1,P_C [Y(0)=1]-p_U, \nonumber \\&\qquad P_C [Y(1)=1]-(1-P_L )\} \nonumber \\&\quad \le P_C (Y_m =1) \nonumber \\&\quad \le \min \{1,P_C [Y(0)=1]+P_C [Y(1)=1],P_C [Y(0)=1]+p_U, \nonumber \\&\qquad P_C [Y(1)=1]+(1-P_L )\}. \end{aligned}$$

(11)

We anchor the values of these participation bounds using information on participation rates in the US. The CPS data reveal that 57.4% of eligible households report receiving SNAP and 70.2% NSLP (see Table 1), and administrative data indicate much higher participation rates—nearly 90% in NSLP (Gundersen et al. 2012). For this model, we assume participation rates would be at least as high if not higher in Canada. As noted above, we assume the participation rate in Canada would lie within [0.90, 0.95]. As the fraction that participates becomes unconstrained, the bounds converge to the no-assumption bounds in Eq. (9). In that case, the constraint is nonbinding. As the upper bound on the fraction participating approaches 0, the bounds center around the outcome that would be observed if all recipients receive treatment 0, $P_C [Y(0)=1]$. Likewise, as the lower bound approaches 1, the bounds center around $P_C [Y(1)=1]$.

Arguably, both the outcome optimization and constrained participation models apply. That is, households may act to minimize the food insecurity rate but some bounded fraction of households participate. Intuitively, under this constrained optimization model with rates of participation in excess of 90%, the prospective food insecurity rate cannot exceed the rate that would occur if all eligible households participate, and it cannot fall below the lower bound established under the constrained participation model in Eq. (10).²⁴

4.2.4 Monotone treatment response

Given the mixing problem, one might speculate that the food insecurity rate under the new regime will necessarily lie between the outcomes under mandated participation and the status quo without SNAP or NSLP, with the precise location depending on what fraction of households take up the program. This hypothesis is true if receiving benefits never reduces the likelihood of being food insecure. After all, under this monotone treatment response assumption, households can do no better in terms of minimizing the chances of food insecurity than participating in SNAP or NSLP and no worse than not receiving benefits.²⁵

If we combine the constrained participation models with the MTR assumption, we obtain the following new bounds on $P_C (Y_m =1)$:

$$\begin{aligned}&\max \{P_C [Y(1)=1],P_C [Y(0)=1]-p_U \} \nonumber \\&\quad \le P_C (Y_m =1)\nonumber \\&\quad \le \min \{P_C [Y(0)=1],P_C [Y(1)=1]+(1-P_L )\}. \end{aligned}$$

(12)

4.3 Results

Using a variety of models to address the mixing problem, Table 4 presents estimates of the prospective Canadian food insecurity rate among households that would be deemed eligible for food assistance. Panel A presents estimates based on the upper bound on $P_C [Y(1)=1]$, and Panel B presents estimates based on its lower bound. In general, we find that there is a great deal of uncertainty about the prospective food insecurity rate of interest, namely the rate that would be realized if Canada introduced SNAP or NSLP. This uncertainty reflects the inability of the data to identify the food insecurity rate if all eligible households were to participate, $P_C [Y(1)=1]$, the fraction of households that would participate, $P_C (Z_m =1)$, or the outcomes that would be realized under the resulting mixing process. Still, the estimated bounds are informative.

In the absence of data, all we know is that the prospective food insecurity probability lies between 0 and 1. In the absence of assumptions to address the mixing problem, the worst case bounds are informative on one side. In particular, the data narrow the upper bound, while the lower bound remains at zero. Thus, without additional assumptions, we learn in row (i) that if Canada adopted SNAP, food insecurity could be eliminated but the rate could also rise to as high as 0.604, substantially higher than the status quo rate of 0.329. While it seems implausible that food insecurity would appreciably rise under SNAP, this result highlights the limits of what evidence logically can be uncovered under minimal assumptions. To narrow these bounds, we consider three different models: the outcome optimization model, the constrained participation model, and the MTR model.

Suppose households make the decision to take up SNAP or NSLP in order to minimize the food insecurity rate: participate in SNAP or NSLP if $Y(1)=0$. Thus, under this outcome optimization model, the food insecurity rate can be no higher than the outcome realized if all eligible households would participate, $P_C [Y(1)=1]$, and no lower than zero. Continuing with Table 4, the estimates in row (ii) of Panel A suggest that no more than 27.5% of households with children would be food insecure under SNAP. The corresponding value under NSLP is 23.7% for households with children between the ages of 6 and 11, a number that declines slightly to 21.4% for the age range 6–17. Where the realized food insecurity rate would lie depends on the association between the latent food insecurity indicators, Y(1) and Y(0). If these outcomes have a strong positive association, the participation decision has little impact in general and the realized probability would lie closer to the upper bound. In contrast, if the association is strongly negative, the realized probability would approach the lower bound.

As an alternative model, suppose that nearly all eligible households decided to participate but some would not. Table 4 displays estimated bounds under the assumption that between 90 and 95 percent of households would participate in SNAP or NSLP. Given this model, the results in Panel A, row (iii) indicate that the food insecurity rate with SNAP would lie within [0.175, 0.375]. Notice that the estimated upper bound exceeds the status quo rate of 0.329, while the lower bound is lower than the 0.275 upper bound under the assumption that all households would participate. In Panel B, the upper bound in row (iii) falls to 0.278, which implies that SNAP would lead to a reduction in the food insecurity rate. Still, even in a model with nearly full participation in SNAP and NSLP, there remains much uncertainty about the realized outcomes and we cannot conclude that SNAP or NSLP would lead to a realized reduction in food insecurity. As noted above, this uncertainty reflects the fact that we do not know the joint distribution of outcomes or how households would decide whether to participate. If the potential outcomes have a strong negative association and households optimize over the outcome (i.e., minimize the food insecurity rate), then the food insecurity rate would lie near the lower bound estimate of 0.078. If instead there is a strong positive association in outcomes and households do not optimize, then the realized outcome would be closer to the upper bound.

Arguably, both the outcome optimization and constrained participation models apply. Under this constrained optimization model, we find that SNAP and NSLP would lead to notable reductions in the Canadian food insecurity rate. Given SNAP, for example, food insecurity rates would be no larger than 0.275 (see Panel A, row iv), the rate under full participation, and no less than 0.175. Thus, adopting SNAP would reduce food insecurity rates by at least 16%, from 0.329 to 0.275. Moreover, in Panel B, the upper bound falls to 0.178, notably (46%) lower than the status quo rate of 0.329. Likewise, in panel B, NSLP for 6–17-year-olds is estimated to result in food insecurity rates within [0.025, 0.125], 48–90% lower than the status quo rate of 0.240.

Finally, rather than modelling how households make decisions whether to participate, we consider the MTR model that links the two potential outcomes. If SNAP and NSLP are assumed to do no harm, then the food security outcome probability would lie between the status quo outcome and the outcome that would be realized if all households took up the program. So, for SNAP, this implies that the food insecurity rate lies between the lower bound of 0.275 (or 0.178 in Panel B) and the status quo rate of 0.329. If we add the constrained participation model that between 90 and 95% of eligible households would participate, the results in Panel A do not change, but in panel B the upper bound falls from the status quo rate of 0.329 to 0.278. Thus, under the MTR model, we find that SNAP and NSLP would (weakly) reduce the food insecurity rates, but the magnitude of this reduction is uncertain. Whether the change is small or substantial would depend on how households decide to participate and on the unknown correlation in the potential outcomes.

5 Conclusion

Predicting outcomes that would be realized under a new program is a central concern to researchers, policymakers, and program administrators. What would happen if Canada were to adopt national food assistance programs like SNAP or NSLP? Yet, this type of prospective prediction problem is an inherently difficult undertaking. As with many program evaluations, the observed data cannot reveal the counterfactual outcome if the new treatment were mandated and, in practice, some households may not take up the new program.

We use partial identification models to address these identification problems. In the first part of the analysis (Sect. 3), we use data from the CPS to infer the effects of SNAP and NSLP in the US. As in recent studies that use these methods, we find under the strongest nonparametric models that SNAP and NSLP lead to notable reductions in food insecurity among households with children.

Then, assuming that effects in Canada would be similar to the effects in the US, we assess what would happen if Canada were to introduce SNAP or NSLP. The question is relatively straightforward if all eligible households take up the program; if Canada were to adopt SNAP, food insecurity rates would decline by at least 16% and perhaps by as much as 46%.

The question is much more complicated if not all households would participate. To address the mixing problem, we apply the partial identification methods developed in Manski (1997a) and Pepper (2003). Two general findings emerge. First, very little can be said about potential food security rates without imposing moderately strong assumptions. Under the weakest models, food insecurity under SNAP, for example, could be eliminated or be as high as 60%. This extreme degree of uncertainty reflects a lack of information on how households would decide whether to participate—i.e., the mixing process—and the joint distribution of the potential outcome – i.e., what fraction would be affected by SNAP or NSLP.

Second, under the stronger models, our results suggest that SNAP and NSLP would both reduce food insecurity rates. Under the MTR models, we find that SNAP and NSLP might have no effect but could lead to large reductions in the food insecurity rate. Under a constrained optimization assumption, we find that SNAP would reduce food insecurity among households with children by at least 16%, from 0.329 to 0.275, and NSLP (for households with children between the ages of 6–17) by at least 11%, from 0.240 to 0.214.

For Canada, the implementation of large-scale, publicly-funded food assistance programs similar to SNAP or NSLP would represent a marked departure from the income support programs that have traditionally defined the country’s social safety net. As discussed by Power et al. (2015), such a shift raises a number of questions. Our findings provide evidence-based answers to one such question, namely the reductions in food insecurity that can be expected should Canada adopt such programs. While this type of information is critical to making informed decisions, there are many important questions that are not addressed in this paper. Most notably, we do not provide a cost–benefit analysis or account for the possibility that the introduction of food assistance programs might trigger changes to existing programs. In particular, funding SNAP or NSLP might lead to benefit cuts in the existing income support programs for working-aged adults and their families. To the extent that the introduction of SNAP or NSLP leads to other related changes to the social safety net, the net impact on food insecurity is uncertain.