1 Introduction

Globally, over the past three decades, the number of weather anomalies, including extreme heat, droughts, floods and storms, has doubled, reaching an yearly average of 213 such events during the 1990–2016 period (FAO et al. 2018). These extreme climate events lead to an increase in the number of disasters, which have severe impacts on people’s lives and livelihoods (Hallegatte et al. 2016). Rural households in developing countries are particularly vulnerable to weather shocks for several reasons. First, they depend on weather-sensitive income generating activities such as agriculture. Bad weather can increase the unit cost of production, widening the distance between observed production and the feasible production frontier. Second, poor rural households are more likely to live in high-risk geographical locations because they tend to be the most affordable, and thus have limited capacity to cope with climate hazards due to lack of savings, weaker social networks and low asset base (del Ninno and Lundberg 2005; Jakobsen 2012; Shehu and Sidique 2015; Lohmann and Lechtenfeld 2015). Third, developing countries have weaker institutional arrangements and the existing early warning/early action systems against extreme weather events are very often limited by severe financial constraints (Kellet and Caravani 2013; Hallegatte et al. 2017).

By exploiting exogenous variation in weather outcomes over time, a growing literature seeks to examine and identify a causal relationship on how climatic factors influence economically relevant outcomes (Dell et al. 2014). Unsurprisingly, because of the natural relationship between weather and agricultural production, agriculture has been the center of the existing research on the impacts of climate. One stream of research looks at the impacts of climate at a regional/national scale, using aggregate economic data (Deschênes and Greenstone 2007; Hsiang 2010; Dell et al. 2012). A second body of research analyzes the relationship between climatic factors and individual commodity production or productivity, such as crop or milk yields (Mukherjee et al. 2013; Key and Sneeringer 2014; Burke and Emerick 2016). Finally, a third stream of research analyzes farm-level adaptation to climate change, such as irrigation investments, crop switching and migration, finding generally that there are limits to the extent to which such adaptation can reduce climate-induced agricultural losses (Hornbeck 2012; Burke and Emerick 2016; Taraz 2017, 2018) and significant information asymmetries and financial constraints preventing adaptation (Deressa et al. 2009; Di Falco et al. 2011).

The limited efficacy of private adaptation suggests a potentially significant role for public policies promoting large-scale adaptation to climate change. Social protection programs can complement both formal risk management tools provided by markets and informal support mechanisms from communities and informal insurance. While the importance of integrating weather risks within the planning of new and existing social protection programs has been already recognized by international organizations such as the World Bank and the Food and Agriculture Organization (Kuriakose et al. 2013; FAO and Red Crescent Climate Centre 2022), very few national Government programs are explicitly tailored to protect households with low levels of adaptive capacity from weather-related shocks. One of the few considerable exceptions is represented by the Productive Safety Net Programme (PSNP) in Ethiopia, which was launched in 2005 by the Government and a consortium of donors as a joint response to chronic food insecurity in rural areas, going beyond the near-annual emergency appeals for food aid and other form of emergency assistance that characterized the previous decade.

Our paper contributes generally to the literature studying how social protection interventions affect livelihoods and agricultural production (Banerjee et al. 2015; Tirivayi et al. 2016; Hidrobo et al. 2018; Daidone et al. 2019), and more specifically to the limited but growing body evidence on the role of social protection in helping individuals cope with the consequences of weather shocks (de Janvry et al. 2006; Patnaik and Das 2017; Asfaw et al. 2017; Adhvaryu et al. 2018; Mueller et al. 2020). Focusing on Ethiopia, we study whether participating in the PSNP and receiving other in-kind social protection programs can mitigate the negative relationship between adverse rainfall shocks and agricultural production, thus acting as a way of coping with these adverse weather conditions. First, we test whether being a PSNP beneficiary of public works or a PSNP cash transfer beneficiary or receiving other in-kind assistance (mostly free food) influences the technical efficiency of smallholders and how these effects on agricultural production are shaped jointly with adverse rainfall shocks. Second, we analyze whether the effects are heterogeneous across outcome variables, considering production, sales revenue and profit functions.

2 Conceptual framework

In this section we discuss how social protection could both affect farm incomes, revenues and profits directly as well as their sensitivity to weather shocks. There are several mechanisms through which various typologies of social protection may influence these relationships. Two of the most important channels are the labor and the income channels. Public Works (PWs) bring about a labor reallocation by beneficiary household members with labor capacity from family farms to the temporary jobs provided by the program. This is expected to generate not only a reduction of overall farm income and sales, but also a reduction of productive efficiency if the work carried out under the PW scheme conflict with activities usually performed during the agricultural season. In the case of Ethiopia PSNP, this issue is particularly important, since the program targets food insecure subsistence farmers. The PW component of PSNP is expected to have an additional negative effect on productive efficiency in response to a covariate idiosyncratic shock too, because PSNP take-up will increase after the weather shock and this is likely to further reduce family labor on the farm for households with members engaged in PWs. With respect to farm profits, participation in a PW program has an ambiguous effect and depends on two factors: 1) whether households that cultivate land are net sellers or buyers of agricultural labor, and 2) the wage floor set by the Government for PW employment. If the PSNP wage is set above the market wage rate, this will lead to higher agricultural wages in the local rural economy (Imbert and Papp 2015; Muralidharan et al. 2017) and consequently to lower profit efficiency if households are net buyers of labor. If instead the wage floor is set below the equilibrium wage, this will depress wages in the private sector, and will increase profit efficiency if farmers are net buyers. The effects of adverse weather shocks on farm profits is ambiguous too, since equilibrium harvest-stage wage are always lower in bad than good weather for both households with and without PW beneficiaries (Rosenzweig and Udry 2014).

A labor reallocation from off-farm wage employment to family businesses on- or off-farm may occur also under a cash transfer. In fact, despite the classical prediction of a fall in working hours and earnings due to unexpected cash windfall, labor responses to cash transfer cannot be determined a priori, as they depend on multiple alternative mechanisms that can be broadly grouped as arising from missing markets, price effects from behavioral conditions attached to transfers, and dynamic and general equilibrium effects (Baird et al. 2018). Indeed Daidone et al. (2019) find that in the impact evaluation of seven cash transfers in sub-Saharan Africa, a reduction of wage labor in five countries is offset by an increase in family labor on- and off-farm in Zambia only, where the program targeted households with labor capacity. Since the pure cash transfer component of the PSNP explicitly targets labor constrained households, we do not expect the labor channel to affect significantly farm production. However, cash transfers are expected to have some influence on farming through an income channel. Higher incomes can potentially increase crop yields and efficiency by relaxing households’ liquidity constraints and allowing them to invest in modern inputs and assets. Further, they could contribute to a reduction of transaction costs and make households more engaged with markets (Prifti et al. 2020), hence increasing their revenues efficiency as well. Profit efficiency is also expected to be greater as a consequence of cash transfers if hired labor is perfectly substituted with household labor or if the cost of adopting more modern inputs is lower than the cost of hired labor. It is not clear however if higher incomes from cash transfers contribute to lower or greater sensitivity to climate change. On the one hand, cash transfers may increase risk-taking behavior by their beneficiaries (Hennessy 1998; Moro and Sckokai 2013; Prifti et al. 2019), hence increase sensitivity to weather shocks. On the other hand, cash transfers may be invested in modern inputs that can reduce yields volatility, thus contributing to a decrease in the sensitivity to adverse weather.

In-kind transfers such as free food distribution can also have productive impacts. In particular, a food transfer may have an insurance function similar to the role of food crop production, thus alleviating the risk associated with the production of cash or higher-values crops or inducing greater off-farm economic opportunities (Margolies and Hoddinott 2014; Schwab 2019).Footnote 1 Further, as long as the in-kind transfer is infra-marginal (smaller than what was consumed prior to the intervention), there should be no difference in how labor supply responds to a cash or an in-kind transfer. However, an infra-marginal food transfer contributes to lowering the price variance of the overall food budget, thus reducing also the exposure to sudden price spikes of staple commodities, which may occur because of weather shocks.

3 The context of Ethiopia

Despite substantial progress made in the last two decades in terms of economic growth and poverty reduction, Ethiopia remains a low-income country. Between 2000 and 2019, per capita Gross National Income rose from 130$ to 850$, yet two-thirds of the economically active population remains engaged in rain-fed agricultural activities.Footnote 2 In addition to this, the levels of food insecurity in Ethiopia are still high and the last decades have witnessed several droughts that have led to famines or protracted increases in food shortages. The geographical location and topography, combined with a low adaptive capacity, make the country highly vulnerable to climate change and weather shocks. Estimates suggest that projected reduction in agricultural productivity may lead to a 20 percent income reduction due to climate change (Gebreegziabher et al. 2016).

The Ethiopian government and development agencies have been increasingly concerned with tackling food insecurity and poverty in Ethiopia. Their commitment to this objective began with the establishment of the Productive Safety Net Programme (PSNP) in 2005, which aimed to respond not only to chronic food insecurity, but also to shorter-term shocks, especially droughts. During phase 1 and 2, spanning from 2005–2009 and 2009–2011, the PSNP provided cash or food to people with predictable food needs to enable them to improve their livelihoods and become more resilient to shocks in the future (van Domelen et al. 2010). In phase 3 (2011–2015), the PSNP expanded its coverage and improved the timeliness of cash transfers, increasing the shift from food to cash transfers. The fourth phase of PSNP (2015–2019), aimed at enhancing resilience to shocks and improving livelihoods, food security and nutrition for rural households vulnerable to chronic or recurrent food shocks. It reached about eight million beneficiaries nationwide and responded to the Social Protection Policy, validated in 2014, by including a series of new program elements, which aimed to provide a transition towards a system of integrated service delivery in social protection and disaster risk management (Schubert 2015).

Most PSNP beneficiary households are engaged in public works, which require a household to be poor, food insecure, and have able-bodied individuals in order to be eligible. PW focus on integrated community-based watershed development, covering activities such as soil and water conservation measures and the development of community assets such as roads, water infrastructure, schools and clinics. The objective of these works is to contribute to livelihoods, disaster risk management and climate resilience, and nutrition. Households without labor capacity are recipients of cash transfers.

Despite the success of the PSNP, humanitarian assistance persisted. In 2015, the Government of Ethiopia created the National Disaster Risk Management Commission (NDRMC). The NDRMC has the overall responsibility for the coordination of disaster management and direct implementation responsibility for the Humanitarian Food Aid (HFA) programme, which is provided to rural households that are food-insecure because of a shock, most often drought (World Bank 2017). The number of beneficiaries and length of support is determined through a needs assessment conducted twice a year, while households are selected through a community-based targeting process. Humanitarian food assistance is largely provided in the form of food, but can also be provided in cash. Transfers in food are provided through the Government’s food management systems and through the World Food Programme. The typical food basket is comprised of 15 kg of cereal, 1 and 12 kg of pulses and 0.45 litres of oil. Transfers in cash have largely been provided through the NDRMC (WFP 2019).

Both the PSNP and the HFA use a mix of geographic and community-based targeting to identify beneficiary households in chronically food insecure woredas,Footnote 3 which are typically the most drought-prone of the country (Knippenberg and Hoddinott 2017). However, while the PSNP has very specific targeting guidelines, HFA does not target any specific group. Overall, there is a considerable geographical overlap between the two programmes, with 90 percent of PSNP woredas receiving HFA too (Sabates-Wheeler et al. 2022).

4 Data and methodology

4.1 Data

We use the three rounds of the longitudinal Ethiopian Socioeconomic Survey (ESS) (Central Statistical Agency of Ethiopia 2012, 2014, 2016), covering three agricultural seasons (2011/12, 2013/14 and 2015/2016). The three rounds of data contain over 14,000 observations and more than 5000 households. However, since the value of agricultural production, sales and profits represent our outcomes of interest, we restrict the sample to households engaged in either crop production (including tree crops) or livestock herding and that report consistent information concerning cultivated land, household composition and labor. This leads to a final sample of 9339 observations.

The ESS provides several data on the PSNP. The surveys include both individual-level questions through a specific module on temporary PWs labor carried out by family members, and household-level questions about cash and in-kind assistance received from PSNP. In the latter section of the questionnaire, the survey reports also other assistance received by the households, which did not originate from PSNP, including free food. This latter item does not provide a reference to a specific intervention, though according to the survey data, 80 percent of food beneficiaries received this assistance from the Government, while the remaining 20 percent reported assistance from either local or international NGOs.

Table 1 describes the coverage of the three social protection tools described in the conceptual framework: participation in PSNP PWs, PSNP direct cash transfers and receipt of free food. In this table, we combined the three waves of data, while we show the dynamics in the coverage rate in Fig. 1. We find a lot of heterogeneity in the data, with regions like Tigray and Diredwa consistently reporting a larger share of PSNP PWs beneficiaries, while free food is the most used social protection tool in Gambela and Somali regions. The low coverage rate in Amhara, Oromia and SNNP reflects the wide within-region disparities, since half of the woredas in these regions are not served with neither PSNP nor HFA (Sabates-Wheeler et al. 2022).

Table 1 Social protection coverage, by region
Full size table
Fig. 1
figure 1

Social protection coverage by region and year

Full size image

Table 2 provides a description of the main variables used in the analysis, summarizing them by the status of participation in social protection programs. Column (1) shows the descriptive statistics for the full sample, while column (2) reports the group of households whose members do not have access to any social protection program. Columns (3), (4) and (5) show the summary statistics for, respectively, those households participating in the PSNP Public Works component, the PSNP cash transfer component, and free food recipients (column 5), which we assume mostly include HFA beneficiaries. Since the participation in these programs is not mutually exclusive, we include the summary statistics for those households participating in more than one social protection program in column (6).

Table 2 Summary statistics
Full size table

Table 2 highlights that groups differ in several ways in terms of their characteristics. First, the total value of farm production is higher for those households not participating in social protection programs. These mean differences are statistically significant and large, with the exception of the difference between the group of households benefiting from PWs and those without social protection. However, despite having a lower value of production, households with access to social protection programs, with the exception of those receiving free food, have higher monetary profits. These are defined as the difference between the revenues from sales and the monetary costs of production.Footnote 4 Part of this seems to be explained by the fact that sales are, on average, higher for households participating in social protection programs (with the exception of those receiving free food). Beyond the higher value of sales, however, households not benefiting from social protection programs also have higher production costs, purchase higher levels of inputs (especially fertilizer) and use more labor (including hired labor). This is not surprising given the average size of land owned is approximately 35% larger than that of households benefiting from social protection programs.Footnote 5

Rainfall anomalies were calculated using the Climate Hazards Group InfraRed Precipitation with Station data (CHIRPS) database estimated, and are defined as the woreda-specific deviation of the average monthly rainfall over a year from the long-term average monthly rainfall, divided by its long-run standard deviations.Footnote 6 Formally, the rainfall anomaly RA in woreda w in year t is given by:

$$RA_{wt} = \frac{{R_{wt} - \mu ^{LR}\left( {R_w} \right)}}{{\sigma ^{LR}\left( {R_w} \right)}}$$
(1)

μLR and σLR are woreda mean and standard deviation of the rainfall data over the long run from 2000 to the survey year. These anomalies measure the magnitude of the rainfall shock relative to the long-term mean of the rainfall variable. As shown in figure S1, woredas in Afar and Somali region have the lowest long-term average monthly rainfall, which is likely to denote lower overall agricultural potential.Footnote 7 In terms of their exposure to adverse rainfall conditions, Table 2 also shows that households benefiting from social protection programs, on average, live in areas with both long-term lower average rainfall and have been more exposed to negative rainfall shocks. The former is proxied by the lower value of the long-term average monthly rainfall, whereas negative rainfall shocks are proxied by the deviations from the long-term rainfall, which are consistently lower for households benefiting from social protection programs. This is not unexpected since, in general, households targeted by social protection programs tend to be more vulnerable and often live in areas more exposed to extreme weather events.

We also note that households benefiting from social protection programs tend to own fewer agricultural assets, represented by an index constructed with principal component analysis, and are more likely to be headed by a female. This is particularly striking in the case of those benefiting from free food, where this is the case for almost half of the households. Finally, households with no access to social protection programs are more distant to large cities than households in the PWs group and those receiving free food, but they are closer to large cities than cash transfer beneficiaries. This points out probably to the difficulty of logistics for free food delivery and PWs, while cash transfers are relatively easier to organize also in most remote rural areas.

4.2 Stochastic Frontier model

We adopt a Stochastic Frontier framework to estimate farmer’s technical efficiency and its determinants (Kumbhakar and Lovell 2000). Using the original model proposed by Aigner et al. (1977) and Meeusen and van Den Broeck (1977), the single output stochastic frontier production function is defined as:

$$y_i = f\left( {x_i,\beta } \right)\exp \left( {v_i - u_i} \right)$$
(2)

Where yi is the dependent variable of interest for farmer i (in our case farm income, sales and profits), xi is a vector of inputs for farmer i. These could include factors of production such as land, labor and different inputs used in the production process (e.g. fertilizers). β represents the vector of technology parameters associated to the inputs of production. vi is an independently and identically distributed (iid) random error distributed as a Ν(0, σ2). This term represents random factors that, not under the control of a farmer. Finally, the term ui is represents the inefficiency term and captures those factors that prevent farmer i from being efficient.

Following Eq. 2, and given that the frontier of farmer i is given by the expression \(y_i^ \ast = f\left( {x_i,\beta } \right)exp\left( {v_i} \right)\), the measure of technical efficiency for a given observation can be defined as:

$$TE_i = \frac{{y_i}}{{y_i^ \ast }} = \frac{{f\left( {x_i,\beta } \right)exp\left( {v_i - u_i} \right)}}{{f\left( {x_i,\beta } \right)exp\left( {v_i} \right)}} = exp\left( {u_i} \right)$$
(3)

Given the relationship described in Eq. 3, a given household can be described as efficient when the technical efficiency score is equal to 1. Any value below 1 indicates the presence of inefficiency, as a household is not producing the maximum achievable output. Values further away from 1 indicate higher levels of inefficiency. Assuming the simplest a Cobb-Douglas functional form for the deterministic part of the frontier,Footnote 8 we can re-write Eq. 2 as follows:

$$ln\left( {y_{it}} \right) = \beta _0 + \mathop {\sum}\limits_j^j {\beta _jln\left( {x_{jt}} \right) + v_{it} - u_{it}}$$
(4)

An important aspect in the stochastic frontier analysis literature relates to the variables that influence the inefficiency term ui. The most common approaches to including these variables in the inefficiency term are those suggested by (Kumbhakar et al. 1991; Huang and Liu 1994). The authors essentially propose to parameterize the mean of the pre-truncated inefficiency distribution, which can be expressed as follows:

$$u_{it} = \delta _0 + \mathop {\sum}\limits_k^K {\delta _kZ_{kit} + \eta _{it}}$$
(5)

One challenge that is particularly prevalent in agricultural economics when estimating Eq. 4 is the existence of zero and negative values for several key variables. Many farmers do not use fertilizer and monetary profits may well be negative for some farmers and therefore if we were to use the log transformation of these farmers, we would be forced to drop a large number of observations. We opt to follow the approach proposed in Bellemare and Wichman (2020) and use a different transformation, namely the hyperbolic sine transformation (IHS). This approach has two key advantages over alternative treatments of zero and negative values. First, it does not introduce a bias in the estimated coefficients, which occurs when a small number is added to 0, as first suggested by MaCurdy and Pencavel (1986). Second, it is able to handle negative values, which are not handled in the correction proposed by Battese (1997).Footnote 9 As such, throughout the paper, rather than estimating Eq. 4, we estimate the following benchmark equation:

$$\begin{array}{l}arcsinh\left( {y_i} \right) = \beta _0 + \mathop {\sum}\limits_j^J {\beta _jarcsinh\left( {x_{jt}} \right)} + \mathop {\sum}\limits_t^T {\beta _td_t}\\ +\, \mathop {\sum}\limits_r^R {\beta _rd_r + v_i - u_i} \end{array}$$
(6)

Where the term arcsinh stands for the hyperbolic sine transformation. The full list of variables used to estimate the frontier and the inefficiency are available in Table S1 in the Supplementary Material file. As far as the estimation of the frontier is concerned, in addition to the inputs to production, we also include year- and region-specific dummy variables to capture year-specific shocks that are common to the full sample as well as region-specific shifts to the frontier, which capture factors such as region-specific policies and/or production constraints.

To see whether the results are robust to a different estimation procedure, we also estimate Eq. 6 using the True Random Effects (TRE) model, which is able to disentangle time-variant inefficiency from time-invariant unobserved heterogeneity in the SFM model (Greene 2005a, 2005b).Footnote 10 In this case, the estimating equation changes to

$$\begin{array}{l}arcsinh\left( {y_it} \right) = \left( {\beta _0 + w_i} \right) + \mathop {\sum}\limits_j^J {\beta _jarcsinh\left( {x_{jt}} \right)}\\ +\, \mathop {\sum}\limits_t^T {\beta _td_t} + \mathop {\sum}\limits_r^R {\beta _rd_r + v_i - u_{it}} \end{array}$$
(7)

where wi is a time invariant, farmer specific random term meant to capture cross farmers heterogeneity.

Another important aspect relates to the existence of other social protection instruments (unobserved to the researcher) affecting the results. This is a legitimate concern in the case of Ethiopia, given the large number of social safety nets. There are three reasons that lead us to believe that the probability of this occurrence is negligible in our case. First, the PSNP accounts for a very large proportion of beneficiaries of social safety nets in rural areas, while we capture also food aid, and both were key social protection mechanisms during the period under analysis.Footnote 11 Second, the increase in internally displaced people largely occurred after the last survey round and we believe that social safety nets focusing on refugee populations are unlikely to affect our results.Footnote 12 Finally, while school-feeding programmes existed in the period under study, before their scale-up in a context of emergency, these programmes reached a relatively small proportion of the total rural population. Therefore, we believe that the probability that our findings are driven by other unobserved social safety net programmes is small.

A final important aspect related to our chosen estimation approach concerns the potential endogeneity of the social protection interventions considered in the analysis. The stochastic frontier models discussed until now implicitly assume exogeneity of both inputs of production in the frontier equation and of inefficiency determinants in the ancillary equation, which is a strong assumption. Over the last three decades, the industrial organization literature proposed several techniques to solve the endogeneity bias in the estimation of production function parameters (Olley and Pakes 1996; Levinsohn and Petrin 2003; Wooldridge 2009; Ackerberg et al. 2015). In the stochastic frontier context, however, only in recent years did the issue of endogeneity receive increasing attention (Kutlu 2010; Tran and Tsionas 2013; Karakaplan and Kutlu 2017; Amsler et al. 2017; Kutlu et al. 2019). In practical terms, the presence of endogeneity can lead to inconsistent parameter estimates. These may occur either because the determinants of the production frontier and the two-sided error term can be correlated or because the inefficiency term and two-sided error term can be correlated. In particular, the determinants of the inefficiency can cause this correlation, which highlights the importance of at least testing the sensitivity of the results to using methods that account for endogeneity.

In this analysis, we follow Karakaplan and Kutlu (2017), who provide a general maximum likelihood based framework to handle and test the endogeneity problem.Footnote 13 This approach is based on the use of an instrumental variable for each of the variables that are assumed to be endogenous. Given the focus on social protection instruments and their interaction with exogenous climatic risks, this requires that we find a valid set of instruments not only for the dummy variables indicating participation in PSNP labour, PSNP cash and free food, but also their interactions with rainfall anomalies, which we will discuss shortly. Our instrumenting strategy uses the interaction of key targeting mechanisms of the programs (Nunn and Qian 2014). To proxy the geographical targeting mechanism, for each social protection instrument, we use the share of beneficiaries in the kebele k in the year of the survey (ShareSPk). To reproduce the community targeting, we carry out three separate probit estimates and predict the related probabilities to be treated in the social protection instrument for household i, Pr(SPi), using the demographic and economic criteria defined to identify the beneficiary household.Footnote 14 With regard to the demographic criteria, our models include the following controls: adult equivalent household size, the number of male and female household members in working age, the number of children and a dummy indicator assuming value one if the household head is female. To proxy the economic conditions of the households, our models include the number of months in which the household experienced a food gap, the size of the land owned by the household, an indicator of agricultural assets wealth created with principal component analysis, the rainfall precipitation anomalies and self-reported measures of shocks for flood and drought. Finally, among the regressors we also include time dummies for each survey wave, a dummy for the household living in an urban cluster and a dummy for being a free food recipient in the two PSNP probits and a dummy for being a PSNP beneficiary in the free food probit. Thus, we estimate the following first-stage regression in order to address the endogeneity of the social protection variables:

$$SP_{ik} = \alpha _0 + \alpha _1\left( {\Pr \left( {\widehat {SP}_{ik}} \right) \times ShareSP_k} \right) + \mu _{ik}$$
(8)

Where \(\Pr ( {{\widehat {SP}}_{ik}} )\) and ShareSPk represent the community targeting and the geographical targeting mechanisms respectively. Our strategy is based on the assumption that the program beneficiary share has no impact on household productive inefficiency other than through its influence on the participation on the program. The idea of using the program beneficiary share as exogenous variable is inspired by Ravallion and Wodon (2000) who explain that while the likelihood of any individual (or household) receiving a transfer is higher when its cluster is selected not all individuals/households in the cluster benefit from the transfer program.

As we suggested earlier in this section, the potential endogeneity of the social protection instruments imply that their interactions with the exogenous rainfall anomalies can be endogenous too. However, if for each endogenous variable the instruments described in Eq. 8 are valid, then the interaction term between the instrumental variable for the given social protection instrument and the exogenous rainfall anomalies will be a valid instrument for the interaction term.

Importantly, we do not select the method proposed by Karakaplan and Kutlu (2017) as the main results because simultaneously instrumenting six variables (different social transfers and their interactions) is both extremely challenging from an econometric perspective, both in terms of computation (e.g. convergence), reliability and interpretability. However, to assess the sensitivity of our main results to issues related to endogeneity we carry out the following estimates: firstly, we instrument the three social protection interventions jointly, without including the weather anomalies in the estimating equation; then we instrument one social protection instrument at the time and its interaction term with the weather anomalies.

While it is impossible to guarantee that any given instrument is valid, we nevertheless carry out falsification tests similar to those implemented by Di Falco et al. (2011) and Fontes (2020), adapted to a stochastic frontier setting. The basic idea of this falsification test is that an instrument should be highly correlated with the participation in a social protection programme. However, for the instrument to be valid we would expect it to be correlated with the outcome only through participation, thus we would expect the instrument to be an insignificant determinant of the actual outcome (inefficiency) in our case. We thus proceed in two stages. As summarized in Table S3, we show that for each instrument and outcome, the instrument is highly correlated with participation, but unrelated with the inefficiency term for those not participating. While this does not prove the validity of the instrument, the fact that it passes the falsification test provides us re-assurance regarding the approach adopted.

Finally, we decided to not cluster standard errors for the main set of results. As highlighted in Abadie et al. (2023), standard errors should be clustered, not when it makes a material difference to the result, but rather when it is needed, and more specifically: a) when the sample design is based on cluster sampling and we want to say something about the population; b) when a treatment is experimentally assigned to a cluster of units rather than individual units. Since the second case does not apply to our setting and we only aim to keep the inference at the level of the sample, we argue that there is no need to cluster standard errors. However, we show that clustering at the woreda level does not invalidate results that do not fail other robustness checks.

5 Results

5.1 Farm production value

Table 3 shows the estimated coefficients for the variables included in the term using the value of farm production as a dependent variable. Since our aim is to analyze the relationship between social protection, weather shocks and inefficiency, we only include the inefficiency equation and omit the deterministic part of the frontier, which is included in the supplementary material (Table S10).

Table 3 Inefficiency term - Farm income
Full size table

As can be seen in Table 3, most variables display the expected sign.Footnote 15 Households with more agricultural assets, with larger shares of irrigated areas and those with more household members have higher levels of efficiency. The opposite holds true for those households who participate in off-farm activities, have a female head and that are farther away from a city (proxy for market access). Perhaps surprisingly, households with higher levels of education are more inefficient, though this result is insignificant in most cases. This is likely because more educated households have more off-farm opportunities, which acts as a disincentive to work on farm.

Turning to the main variables in our analysis (CHIRPS deviations), positive deviations with respect to the long-term mean, which proxy better rainfall, lead to higher levels of efficiency. This is both intuitive and consistent with findings in the broader stochastic frontier literature, which tend to find that increased heat stress and worsening climatic conditions tend to lead to higher inefficiencies (Key and Sneeringer 2014; Wang et al. 2017).

As far as social protection programmes are concerned, we consistently find a positive effect of participation in PW programs on inefficiency. This means that households that participate in PW programs have lower farm income relative to their potential. A potential explanation for this result is that PW programs are likely to act as a disincentive to produce on-farm, since they divert part of the household labor off-farm, thereby decreasing farm production. However, more interestingly, in column 2 we report the interaction with our rainfall shock variable, which displays a negative sign and is significant.Footnote 16 Together with the positive coefficient on participation in PW programs, this suggests that participation in PWs leads to higher inefficiencies when weather conditions are worse.Footnote 17 We argue that the mechanism explaining this relationship are likely to be similar to the ones found by Branco and Féres (2021) in the case of Brazil, who find that, owing to the lower returns to agriculture during droughts, households increasingly attempt to find off-farm opportunities and reduce their amount of on-farm work. In this case, PW programs are likely to amplify this effect as they create the jobs that allow people to move off-farm, leading to lower farm income.

As shown in Table S10, this result is robust to different functional specifications and choice of estimators and, as we show in Table S13, it is also robust to the use of alternative weather variables. As shown in Table S4, when accounting for endogeneity using the method proposed by Karakaplan and Kutlu 2017 we find that, while the direct effect of Public Works (i.e. column 4) becomes insignificant, the magnitude of its interaction with weather anomalies more than doubles. Therefore, while the results are slightly weaker the support for the hypothesis that weather shocks compound the inefficiency effect of PW programmes is even stronger, suggesting that a mechanism similar to that found in Branco and Feres (2021) may be at play. Finally, we also test the sensitivity of our results to clustering the standard errors at the Woreda level (summarized in Table S7) the results remain similar, although the PSNP labour variable becomes less significant.

With regards to the other social protection programs, broadly speaking, the results are not significant, although we note that cash transfers and free food seem to be associated with lower inefficiencies. In the case of free food, the results suggest that these effects could be amplified when there are adverse rainfall shocks. These findings are not sensitive to a method that accounts for potential endogeneity (Table S4) nor to the clustering of standard errors (Table S7).

5.2 Farm sales

Table 4, below, shows the estimates for the inefficiency equation when using sales as a dependent variable (the full table is available in the supplementary material, see Table S11). The coefficients on the exogenous determinants of inefficiency other than social protection remain very similar to the results discussed in Table 3 for farm income. However, the sign of the coefficients on the social protection variables are very different compared to the previous outcome indicator and point to heterogeneous effects of these programs. While the coefficient on public work programmes remains positive throughout (indicating higher efficiencies), the coefficients are no longer significant in most specifications nor are the interactions with rainfall shocks.

Table 4 Inefficiency term – Sales
Full size table

In contrast to participation in PW, households receiving food aid is associated with higher levels of efficiency. This implies that households that receive food aid sell amounts closer to their sales potential. We explain the result by the fact that, despite having no effect on farm production (insignificant effect in Table 3), by receiving free food households reduce their need for self-consumption. In turn, this allows households to sell larger quantities of their produce. This result is statistically significant in all eight specifications. However, the interaction terms with rainfall are insignificant throughout.

We test the robustness of these results to an alternative estimation method (proposed by Karakaplan and Kutlu 2017) in Table S5 and find no large difference in terms of sign and significance of the coefficients of the different social protection instruments. We also find very similar results when we cluster the standard errors (see Table S8).

5.3 Farm monetary profits

Table 5 shows the estimated coefficients for the inefficiency equation when using monetary profits as the outcome variable (full table is available in the supplementary material, see S12). We find that all social protection programs are associated with an increase in efficiency (i.e. higher profits, relative to their potential), as shown by the negative coefficient associated to the three social protection programs, though their interaction with the weather anomalies variables are very heterogeneous.

Table 5 Inefficiency term – Profits
Full size table

Starting with the results of PW programs, we find that they lead to higher efficiency in terms of profits, while we previously found no effect on sales and a negative effect on production. The reduction of farming because of the existence of greater off-farm opportunities may increase the proportion of household labor in the family farm, as hired labor becomes redundant.

Unlike the previous results on farm income and sales, we find that cash transfers also have a positive effect on profit efficiency and that this effect is greater in the presence of adverse rainfall shocks, although the interactions are insignificant throughout. There are two potential explanations for this pattern. First, cash transfer beneficiary households are able to invest in modern productive inputs, which makes the activity more profitable in the presence of weather shocks. Second, the effects operate through the reduction in hired labour. Specifically, cash transfers have been found to increase the time a household spends working on farm (Boone et al. 2013; Prifti et al. 2017) (Boone et al. 2013; Prifti et al. 2017) and this effect may be larger in years characterized by negative weather shocks. As such, this could lead to a shift away from hired labour to household labour, which would lower costs. The fact that households that participate in cash transfers have the lowest costs in terms of hired labour would provide some support to this hypothesis.

The result of profit efficiency on free food beneficiaries are consistent with expectations. The dummy variable displays a negative and significant coefficient for all specifications and the interaction with the rainfall variable is also negative and significant. In other words, households receiving free food are more profit-efficient, but that, in the presence of negative weather shocks, this effect is lower. We argue that this is because in good years, the additional food, together with own production, may satisfy the needs of the household who may even be able to sell some of its own output. However, in years of bad weather, the lower own production levels due to the weather shock may prompt households to use the free food for own consumption, rather than sales. This would explain why the effect on efficiency is higher when there are positive (or in the absence of) rainfall shocks. Finally, turning to the results of the PSNP on profits, despite the previous finding that efficiency of farm income is lower, we find that it increases profit efficiency. At first glance, this may seem contradictory since we may expect a reduction in income to be related to lower profits. However, we argue that in this case the explanation is likely to lie in the costs and there are two plausible candidates to explain this pattern. The first is the wage channel. While our data does not allow us to test the hypothesis, the low wages set in the framework of the PSNP could depress local wages. Since PW beneficiaries have high hired labour costs compared to beneficiaries of other social safety nets, the cost reduction for these beneficiaries may be higher. A second potential explanation could be that PWs push farmers off-farm, reducing the scale of operations which result in cost-savings, especially in terms of hired labour.

As a robustness check we carried out the same set of estimates using the NDVI and report the results in table S15. Overall, most of the results remain similar in terms of sign. In terms of significance, we note that all the interactions between the NDVI and the participation in cash transfers are significant throughout. Similarly, we also test the robustness of the profit results to an estimation method that allows for an endogenous regressor in the inefficiency term in Table S6 and find no difference in terms of sign and significance of the coefficients of the different social protection instruments, suggesting that the practical implications of endogeneity, at least for the outcome on profits, are likely to be relatively small. We also find that clustering the standard errors at the Woreda level does not change the interpretation results, although in some cases the level of significance decreases marginally (Table S9).

6 Conclusions and discussions

Higher temperatures and unpredictable rainfall patterns caused by climate change are expected to have adverse effects on crop yields during the coming decades. The agricultural sector in many African countries is particularly vulnerable to these weather shocks, as it remains largely based on rain-fed agriculture and characterized by low adaptive capacities of farmers to adopt management practices that reduce the exposure to such shocks. In Ethiopia, despite the robust growth of the services and industry sectors in the last decade, agriculture still employs around two thirds of the labor force, accounts for about one third of the gross domestic product and is heavily dependent on rainfalls. Climate change has therefore the potential to trigger food shortages, exacerbating food insecurity in many areas of the country. To address this, the Government of Ethiopia has moved away from ad hoc responses to a planned systematic approach, embodied originally in the Food Security Programme launched in 2005 and more recently in the National Social Protection Policy of 2014. Social protection is now at the center of Ethiopia’s development policy, with spending equivalent to 2.77% of gross domestic product on average between 2012 and 2016 (Endale et al. 2019). Although domestic financing has increased considerably in recent years, donors financed approximately 60% of social protection spending (ibidem).

Given the structure of the Ethiopian economy and the high exposure to weather shocks, it is essential to understand how the large investment made by the Government in social protection programs can contribute to increased resilience of households against chronic and transitory food-insecurity. Several impact evaluations of the Productive Safety Nets Programme were conducted since the program started, highlighting its positive impacts on food security and household well-being (Gilligan et al. 2009; Berhane et al. 2014; Hoddinott and Mekasha 2020). Studies of the PSNP impacts on production and productivity are scarce and point to a positive effect of the PSNP only when combined with other livelihood interventions, but not when provided alone (Hoddinott et al. 2012).

In this paper, we seek to understand the potential contribution of different social protection programs on poor farmers’ agricultural production in rural Ethiopia, and their interaction with rainfall anomalies. We find highly heterogeneous effects, where participation into a cash-for-work scheme crowds-out work on the family farm and reducing productive efficiency, especially for farmers exposed to adverse rainfall shocks. We also find evidence that unconditional cash transfers to households without labor capacity do not have any significant effect on farm income and sales, regardless of weather conditions, but improves the family farm profitability. Finally, food transfers do not lead to changes in farm production, but lead to higher sales, which translate into higher profits when farmers are exposed to positive rainfall shocks. These results are consistent with the labor and income channels hypothesized in the conceptual framework and contribute to the growing literature on poor and vulnerable households adaptation to weather shocks, pointing to alternative social protection coping mechanisms.

The sensitivity of farm production due to the PSNP is very relevant for the policymakers. Under climate change, farmers will be increasingly vulnerable to weather anomalies and therefore it is crucial to formulate social protection programs that are able to mitigate weather risk. The original objective of the PSNP was to provide transfers to chronically insecure households to smooth consumption and avoid distress sale of assets in times of crises. Households with labor capacity were engaged in labour-intensive projects designed to build community assets and happening between the months of January and June, so as not to interfere with farming activities, which, in most regions, occur in the second half of the year (Gilligan et al. 2009). However, according to our estimates, it seems that this objective has only partially been met. Previous impact evaluation studies suggested some programmatic implications for the PSNP such as: 1) making sure PW activities do not overlap with key phases of the agricultural cycle; 2) disbursing PSNP wage payments timely to allow investment in agricultural inputs and assets (Gilligan et al. 2009; Hirvonen and Hoddinott 2021). While not directly derived from the findings of this article, if adequately implemented, these recommendations may help reducing some of the negative production effects we observed in the econometric analysis.

The reduced productive efficiency of PWs during adverse weather shocks is clearly concerning from a food security perspective. It entails that either the public infrastructures created and/or rehabilitated under the PSNP do not contribute to total factor productivity growth or that farmers do not have easy access to modern inputs that can reduce yields volatility. Our findings therefore suggest actions on these directions. Finally, while setting a low daily wage rate for PW activities under the PSNP has been crucial for targeting and avoiding significant inclusion errors, it may have had as unintended consequence the depression of local wages, thus penalizing households that are net sellers of casual agricultural labor, who might now bear a greater share of the cost associated with weather shocks. Overall, we did not find any significant association between cash transfers and productive efficiency. The limited amount of the transfers does not seem to have induced significantly liquidity-constrained households to make investments in more modern and risk-reducing inputs or technologies. To maximize the impacts of these transfers, and the overall effectiveness of the PSNP, the Government of Ethiopia undertook a thorough revision of the program, which resulted in the fifth phase, officially launched in March 2021. PSNP5 aims to promote productive opportunities, by including a new sub-program component, which will tailor livelihood options for beneficiary households, mainly in the form of business training, agricultural extension and credit (Government of Ethiopia 2023). Strengthening delivery and accessibility of its beneficiaries to agricultural inputs, services and technologies, and credit facilities is crucial to improve program’s effectiveness in achieving better outcomes. The provision of complementary livelihood services has in fact the objective to enhance and diversify beneficiaries’ incomes, fostering a sustainable pathway out of poverty. Another area in which the PSNP5 seeks to make key enhancements from early phases of implementation is shock responsiveness, supporting the expansion of PSNP to additional drought-prone woredas. The shock-responsive component of the safety net is supposed to encompass improved early warning systems, standard operating procedures for scale-up, and a drought response plan (MoA 2020). Clearly more research will be needed in the forthcoming years to assess the adequacy of these policy changes.

One limitation of the study, which we view as a fruitful avenue for future research, is that we look at average effects across all the regions in our sample. Given the heterogeneity across regions in Ethiopia, it would have been very interesting to carry out region-specific analyses or understand whether the interactions between climate change and social protection differ across settings (such as moisture regimes or aridity). However, given the sample sizes we were not able to do so, but argue that disentangling the heterogeneity in this relationship is likely to be key to a better understanding of the relationship between social protection, agriculture and how these interact with climate.