The impact of unconditional cash transfers on enhancing household wellbeing in Pakistan: evidence from a quasi-experimental design

Abstract

Cash transfer programs have emerged as potent tools for alleviating poverty and enhancing the living conditions of the most deprived households in developing nations. Such initiatives have now become integral components of social protection systems in many developing countries, including Pakistan. Evaluating the efficacy of these safety nets is crucial to comprehend their value in terms of public expenditure. This research employed a well-being index to gauge the impact of unconditional cash transfers on the socioeconomics of recipients. By employing three rounds of the BISP impact assessment survey conducted in 2011, 2016, and 2019. The study measured the overall impact of cash transfers on well-being by utilizing Principal Component Analysis in conjunction with a Difference-in-Differences Quasi-Experimental design over the years. The findings of the study indicate that between 2011 and 2016, the socioeconomic status declined for both treatment and control groups, but the reduction was less pronounced among those who received treatment compared to the control group with insignificant. From 2016 to 2019, there was a marginal positive increase in socioeconomic status, although it was of insignificance. Overall, the well-being of both the control and treatment groups decreased from 2011 to 2019, with a slightly more pronounced improvement observed in the treatment group. This trend suggests that the impact of the BISP unconditional cash transfer program had a limited effect on altering the well-being of the beneficiaries. The lack of substantial impact from the BISP’s cash transfers on household well-being can be attributed to factors such as inflation, unemployment, economic slowdown, payment challenges, insufficient funding, and gaps between installments. The government of Pakistan should reconsider the substantial investment in BISP within the context of value for money. It is advisable for the government and policymakers to channel this substantial investment into income generation, capital asset development, microbusinesses, climate-resilient agriculture, and livestock in order to stimulate the real economy sectors.

Introduction

Cash transfers have emerged as a powerful tool in reducing poverty and improving the wellbeing of the poorest households in developing countries (Baird et al. 2014). These transfers provide direct monetary assistance to low-income families with the primary objective of improving their income and increasing their consumption of essential commodities (Handa and Davis 2006). According to recent estimates, cash transfers have already benefitted approximately 718 million people in developing countries, with the numbers expected to rise in the coming years (De Groot et al. 2017).

In the short term, these transfers can provide resources that families can use to meet immediate needs like food and shelter, while in the long term, they can enable families to invest in education, health, and other productive activities to break out of the poverty cycle (Ayoo 2022). Cash transfers can also empower women, improve school attendance, and contribute to better nutrition and health outcomes, particularly in patriarchal societies where women have limited access to resources and decision-making (Sugiyama and Hunter 2020).

Cash transfer schemes are now an integral part of the social protection systems of many developing countries due to poverty and social well-being being significant obstacles to sustainable development. Education, health, and women’s empowerment are particularly critical areas that need to be addressed (De Groot et al. 2017). Despite progress made over the past few decades, the challenges remain substantial. For instance, although more than one billion individuals were lifted out of extreme poverty between 1990 and 2011, 258 million children and youth worldwide are still out of school. Moreover, 828 million people in 2021 do not have sufficient food to eat (UNICEF 2021; WFP and UNICEF 2022).

Low and Middle-Income nations such as Pakistan face even more significant challenges. A substantial proportion of the population lives in extreme poverty and experiences food insecurity, with educational challenges and limited access to resources and decision-making opportunities, such as those relating to women’s empowerment. According to the 2018 National Nutrition Survey, 36.9% of households in Pakistan experience food insecurity. Additionally, the Pakistan Demographic and Health Survey (PDHS) conducted in 2017–18 indicated that 37.8% of children under the age of five suffer from stunting, with boys slightly more affected (38.2%) than girls (37.1%). Malnutrition remains a critical issue for children in the country, and 20.3% of the population continues to experience undernourishment. The Global Hunger Index scored Pakistan at 26.1 in 2019, indicating significant food insecurity in the country. Despite these challenges, poverty levels in Pakistan have decreased from 57.9% in 1998–99 to 21.9% in 2018–19 over the last two decades (NIPS and ICF 2019; von Grebmer et al. 2019).

Between 1998–99 and 2018–19, poverty rates in Pakistan decreased from 44.5% to 10.9% in urban areas and from 63.4% to 28.2% in rural regions. However, the rate of poverty reduction has been uneven over time. Poverty in Pakistan is a dynamic and transitional phenomenon, with individuals frequently moving in and out of poverty, as noted by Farooq and Ahmad (2020). A recent study indicates that 40.2% of households in Pakistan suffer from food insecurity, with varying degrees of severity observed across provinces. Specifically, the province of Balochistan has the highest prevalence of food insecurity at 53.3%, followed by KP at 49.5%, Sindh at 43.9%, and Punjab at 35.5% (Hameed et al. 2022).

Cash transfer programs, both conditional and unconditional, have the objective of enhancing the socioeconomic welfare of the poor and vulnerable groups in developing countries. The underlying rationale behind these interventions is to promote resilience, equity, and opportunities for individuals and households that are poor and vulnerable (Beegle et al. 2018; Iqbal et al. 2021). Although the primary focus of cash transfers is to reduce poverty and hunger through asset creation, the programme also seek to improve women’s empowerment, particularly their access to financial inclusion and livelihood opportunities. Therefore, the programme positioned as the third pillar of inclusive growth, in addition to sustained economic growth and social inclusion. The objective is to safeguard the poor and impoverished segments by managing uncertain risks, enhancing their resilience, and promoting greater societal equity (Jha et al. 2013).

The effectiveness of social safety nets (SSNs) varies across countries and is dependent on several factors such as targeting, coverage, beneficiary enrollment, and the sufficiency of financial assistance (Gentilini et al. 2014). While the welfare impacts of SSNs in reducing poverty remain debatable, they are becoming increasingly popular as an effective mechanism for poverty reduction in the developing world. Social protection aims to improve the standard of living and resilience of marginalized and vulnerable segments through public interventions and collective efforts (Ahmed et al. 2014).

Benazir Income Support Programme (BISP) in Pakistan

In 2001, the poverty reduction strategy paper (PRSP) was established, followed by the Social Protection Policy (NSPP) in 2007, which provided the foundation for the Benazir Income Support Program (BISP) in 2008. The BISP is widely recognized as one of the world’s best programs in terms of targeting and coverage, according to the World Bank (2018). The program provides cash assistance to 5.8 million families (ever-married women) with a quarterly stipend of Rs. 5000 (approximately US $35) in accordance to 2011 data. Additionally, the program aims to assist low-income families’ children in completing their primary-level education, with 3.5 million children enrolled so far, and their mothers receive an additional top-up of Rs. 750 per quarter for male children and Rs. 1000 per quarter for female children, on the condition that the child attends school and achieves a minimum attendance goal of 70% (Iqbal et al. 2020).

BISP as its flagship program in 2008, significantly increasing Pakistan’s spending on social safety nets from 0.1% to 2.3% of Gross Domestic product (GDP) in 2018. The BISP is committed to achieving its short-term and long-term objectives. In the short term, the program aims to cushion the poor from the negative effects of the food price crisis and inflation through the provision of unconditional cash transfers (UCT) to eligible families. In the long term, the program aims to provide eligible beneficiaries with access to a wide range of social and productive services, addressed through the provision of conditional cash transfers (CCTs) to eligible families, which gradually lifts them out of poverty. Initially, the program identified its 2.2 million beneficiaries through parliamentarians. However, to avoid political bias, a national targeting mechanism based on a Proxy Means Test (PMT) was adopted in 2010, which replaced the parliamentarian phase. In 2010/11, data from 27 million households were gathered through a door-to-door survey.

The Government of Pakistan has implemented two large-scale programs under the BISP initiative, namely the CCT and UCT program. As of now, the government has disbursed Rs. 45.91 billion under the CCT and Rs. 1280.36 billion under the UCT. The UCT program has been implemented to provide financial support to vulnerable households in rural and urban areas, who do not have access to social protection programs. The CCT program is specifically designed to support the education and health needs of children from poor households.

In addition to these programs, the government has disbursed a significant amount of money to several emergency relief programs in various parts of the country. These programs include those in AJ &K, Harnai, and Tirah valley, among others. The government has also invested in financial literacy and inclusion programs and hybrid social protection programs, which aim to provide support to households that are not covered under the existing social protection schemes. Under the CCT program, the government has disbursed Rs. 30 billion to Ehsase Taleemi Wazaif, which aims to provide financial assistance to children from poor households to continue their education. The government has also allocated Rs. 22.7 billion for scholarships to deserving students between 2020 to 2022. The BISP Board has given its endorsement for an unprecedented budget of Rs. 471 billion for the fiscal year 2023–24. This allocation comprises Rs. 361.5 billion designated for the Benazir Kafaalat Programme, catering to 9.3 million families. Additionally, Rs. 32.27 billion has been allocated for the Benazir Nashonuma Programme, targeting 1.5 million individuals, while Rs. 55.4 billion has been set aside for the Benazir Taleemi Wazaif initiative, expected to benefit 9.2 million children. Moreover, there is a provision of 6 billion for the Benazir Scholarships for Undergraduates (GoP 2023).

These efforts by the government show a commitment to addressing poverty and promoting education and social welfare in Pakistan. The main objective of this study is to create a socioeconomic well-being index that is used to measure the overall wellbeing and prosperity of individuals and households. This index utilized to evaluate the impact of cash transfers on improving household well-being in Pakistan. Furthermore, the index employed to determine the effects of unconditional cash transfers on various factors, including those related to women’s empowerment, financial stability, living standard and material resources. The study aims to provide a comprehensive understanding of the diverse impacts of financial transfers on multiple aspects of well-being by analyzing household and individual recipient characteristics such as household size, income levels, age, gender, and other relevant attributes.

A substantial body of evidence has been accumulated to evaluate the impact of cash transfers (CTs) on various socioeconomic indicators such as poverty, health, education, consumption, women’s empowerment, and food security. While there has been an increasing amount of research on CTs that includes measures of subjective indicators, there has been no attempt to integrate these indicators into socioeconomic wellbeing index along with the decomposition of these subjective indicators, which is a key aspect of this study. The aim of this study is to construct a socioeconomic wellbeing index and assess the impact of cash transfers on different domains of socioeconomic wellbeing, including poverty, education, health, financial hardship, and women’s empowerment.

Additionally, this research investigates the differential impact of socioeconomic and cash transfers on household and beneficiaries’ characteristics. The findings of this research are crucial for informing evidence-based policy analysis to scale up contributions to cash transfer programs. This is particularly relevant in the context of national government commitments to improving value for money, expanding coverage, ensuring financial sustainability, clarifying the level and type of impact, and adequately reflecting these in social wellbeing. Moreover, this research aims to take a strategic approach to providing technical assistance on national cash transfer systems, in line with the government’s commitment to reducing poverty, improving health, education, and empowering women.

Literature review

In the world, across 72 to 149 Social Safety Nets (SSN)/Social Assistance (SA) programs have been launched in various developing and developed nations to alleviate poverty, increase well-being, social development, and safeguard against economic shocks (Saleem 2019 and World Bank 2017). In developing nations, tailored interventions have long been used to reduce poverty and promote social well-being. Ivaschenko et al. (2018) investigate trends in SSN programme coverage, spending, and programme performance throughout the world and find evidence of an increasing commitment to SSN/SA programmes, emphasizing their substantial contribution to poverty reduction efforts. However, they also stress the continuous difficulty of low programme coverage and benefits, implying that while these programmes have made achievements, more effort is needed to achieve meaningful poverty reduction.

Governments with limited poverty-reduction funds have implemented targeted programmes that concentrate benefits on the poor while excluding the non-poor (Srivastava 2004). There are different types of targeting programmes have been lunched in the developed and developing nations. Targeting is viewed as a technique of efficiently allocating resources by focusing spending on those in greatest need, thereby saving money and improving programme efficiency (Kakwani et al. 2006). Analytical studies indicate that tailored transfers can remove inefficiencies associated with uniform transfers, maximizing coverage and benefit levels for the poor (Devereux et al. 2017). Yusuf (2010) looked at 30 community programs in poor countries. He found that only four of them gave more benefits to the rich than to the poor. The rest gave more help to the poor.

The poverty targeting in anti-poverty programmes is also heavily influenced by the project’s type and setting. Pradhan and Rawlings (2002) found that most, but not all, social fund investments were well-targeted in underprivileged neighborhoods and households. According to them, the self-targeted nature of certain investments made them more pro-poor than others. Dakyes and Mundi (2013) conduct a review of the government’s anti-poverty policy, the National Poverty Eradication Programme (NAPEP), with a focus on its implementation in rural regions within the Mangu local government council. The study reveals that, while a sizable section of the rural population in the study region is aware of NAPEP’s existence, they have not benefited from any of its programmes or packages. This lack of benefit is mostly due to extensive corruption and incompetence by the program’s organizers.

According to Quisumbing et al. (2011), both state and civil society actions have the potential to reduce poverty in Bangladesh, but with differing short and long-term effects on households and individuals. They point out disparities in the timing of net gains from interventions and spillover effects. Given limited government resources, better targeting might allow for bigger distributions to fewer recipients under the PES. Furthermore, when it comes to using NGOs and organizations for service delivery, it is important to avoid idealizing NGOs. There is significant variability within the NGO sector, with some organizations outperforming others in reaching underprivileged populations.

Other methods used to identify individuals for the SSN/SA include Individual assessment through Proxy Mean Testing (PMT) and self-targeting. In PMT, the Government gathers a comprehensive dataset on household assets and demographic characteristics to estimate household income and consumption (Alatas et al. 2012). However, this method has limitations concerning unobserved variables within households, rather than solely relying on household assets. Another method involves the government identifying beneficiaries or eligible individuals or households through the use of agents. This strategy presents issues in identifying poor and non-poor individuals without national threshold. The agent selects persons or households based on local interests, and rent-seeking is another big challenge.

The community-based programme, in which a committee of community members chooses who is eligible for the programme, even if they are not directly linked with it. The primary disadvantages of community-based anti-poverty programmes are limited record-keeping, potential for rent-seeking, and the use of local poverty lines rather than national standards same as the agent method. Despite these obstacles, Yusuf (2010) contends that community targeting is better suited to places where sociocultural conflicts and excessive inequality are not major problems. Otherwise, dominant societies may reject weaker and marginalized members of their group.

Since 2008, the federal government of Pakistan has been implementing the flagship cash transfer program, the BISP. It aims to provide support to the chronic and transient poor across the country, identified through a national poverty scoring system. The program targets millions of poor households, providing cash transfers on a quarterly basis, primarily to women within these households. These transfers are designed to mitigate the adverse impacts of socio-economic shocks and improve the overall living conditions of the poor (Saleem 2019).

The theory of change of BISP is to reduce poverty, improve well-being, social development and protect from the economic shocks. This program employed two distinct approaches. Initially, it relied on local political community members to evaluate potential beneficiaries based on predefined and self-criteria. Additionally, BISP identified elderly individuals, widows, orphans, and the unemployed as eligible for the program. Currently, BISP has implemented a National Socioeconomic Registry and utilizes PMT to target program beneficiaries.

This program is not without limitations concerning the exclusion and inclusion of beneficiaries, payment transfers, and program administration issues. However, despite these challenges, the program boasts the country’s broadest coverage of poor beneficiaries and provides various types of assistance, including food and nutrition support, income-generating activities, resilience against economic and climate-related shocks, as well as education and health services, to its beneficiaries.

Several studies have been carried out to assess the efficiency of the BISP programme. The World Bank (2009), a rapid review indicated that beneficiary identification by parliamentarian was pro-poor, with around 65% of total payments going to the poorest 40% of the population. However, the study revealed that the poverty scorecard technique may not be a particularly useful tool for identifying the poor, as it evaluates just a restricted set of traits and may neglect other qualities deemed relevant by parliamentarian. Farooq’s (2014) study on targeting efficiency revealed that BISP clients were mostly impoverished. Khan and Qutub (2010) stated that throughout the BISP’s targeting phase, benefits were mostly distributed to the poor, but with a high under-coverage rate. Jamal (2010) advises that policymakers employ PMT for beneficiary selection, however he does not back up this advice with robust data review from his own or other studies.

Hou and Ma (2011) argue that if BISP can successfully increase women’s decision-making power, it will most likely lead to gains in human development indicators like as health, education, and nutrition, since women tend to devote more resources to these areas than males. Ambler and De Brauw (2017) find statistically significant effects showing that BISP transfers have a substantial favorable influence on specific factors evaluating women’s decision-making capacity and empowerment. Hou’s (2016) empirical research showed no conclusive evidence associating increased women’s decision-making power to improved nutrition, but it did find a substantial association between women’s decision-making power and girls’ schooling in rural regions.

Tahir et al. (2018) found that, while BISP has facilitated beneficiaries in initiating or strengthening various enterprises under ‘individually-led’ or ‘female-male partnership’ models, it has not changed the patriarchal division of labor within families or contributed significantly to women’s economic or social empowerment. According to Naqvi et al. (2014), BISP distributions have provided structural stability to recipient households, reducing daily household spending on food, education, and health. According to Jalal’s (2017) results, the BISP has a 52.6% under-coverage (exclusion) rate and a 73.6% over-coverage rate (inclusion), with no discernible effects on household savings, indebtedness, food security, or child welfare.

According to the OPM (2016) research, BISP increased monthly food intake among treatment households, decreased deprivation in beneficiaries’ living conditions, and enhanced beneficiary women’s mobility. However, the OPM’s final assessment in 2020 did not found indication that BISP is effectively decreasing poverty among its users or boosting child nutrition. Despite this, BISP continues to have a major influence on women’s empowerment, increases school enrollment rates, and no signs of boosting productive investment.

The literature review mentioned above about BISP corresponds with worldwide research endeavors dedicated to comprehending and enhancing the efficiency of SSN/SA programs in reducing poverty and promoting wellness. There has been no comprehensive exploration of the overall improvements in well-being for BISP beneficiaries so far.

Different methods are available for choosing SSN beneficiaries, and Proxy Means Testing (PMT) has been utilized by BISP since 2010. This approach identifies recipients using various criteria such as household reliance, wealth, impairment, education level, and availability of safe water and sanitation facilities. Although BISP is the largest cash transfer program in Pakistan, it encounters the same difficulties as other programs around the world. The main objective of this research is to assess the impact of these unconditional cash transfer on the welfare of households in Pakistan.

Examining research on the effects of BISP, underscores the program’s varied results on reducing poverty, empowering women, and stabilizing households. This research aims to fill in the gaps found in prior studies by utilizing a holistic well-being index instead of only considering financial advantages. Various methodological approaches are available for evaluating the effectiveness of a program. This research utilizes a quasi-experimental design with Difference-in-Differences analysis, a simple method that requires less assumptions. The results of this research will offer practical suggestions for policy implementation.

Materials and methods

Data

This study utilized three rounds of the BISP impact evaluation survey to assess the effects of cash transfers on overall well-being. The initial baseline survey was conducted in 2011, followed by subsequent rounds in 2016 and 2019. These surveys were administered by the Oxford Policy Management for the purpose of evaluating the impact of cash transfers within the BISP framework.

The Government of Pakistan initiated this flagship program in 2008 and conducted a national-level poverty scorecard census. The poverty scorecard threshold of 16.17 was utilized for determining eligibility for the UCT assistance. In 2011, the baseline survey sample was randomly selected from eligible households as the treatment group, while households above the threshold, deemed ineligible, were chosen as the control group.

For the baseline survey, a total of 8675 households were covered across the country. The second follow-up round involved 9,139 households, and the third follow-up round expanded the dataset to encompass 12,557 households. The panel data, comprising the same respondents from baseline, round-1, 2, and 3, consisted of 5265 households.

The analysis of this study focused on 5265 panel households (Panel data means using the same groups of households in treatment and control at many times over a period), of which 1296 were taken as the control group, and 3,969 were beneficiary households receiving treatment, spanning the years from 2011 to 2019 (see Table 1). For the dependent and independent variables, this study utilized 15 different indicators (see Table 3) extracted from the data to construct socioeconomic status (SES) as the variable of interest or dependent variable. The time trend, baseline average, difference between two groups pre-intervention, and difference in changes over time were employed as independent variables, while household remittances and employment status served as covariates (see Table 2).

Table 1 Study sample.

Methods

This study used Principal Component Analysis (PCA) to create a socioeconomic score (SES) index. The index constructed using data from household and individual levels including material resources, education, health, financial hardship, women empowerment, and other factors. PCA used to combine these variables into a single composite index that reflects the socioeconomic status of the study participants. These are the proxies of household well-being. It is extensively used to estimate the socioeconomic status scores in the social sciences and reduces the number of variables in a dataset into fewer appropriate dimensions (Padda and Hameed, 2018). This technique is used in two ways: covariance and correlation matrix techniques. However, this study will use the correlation matrix technique to estimate the SES because the variables’ measurement units are different. After the PCA estimation, it yields the first principal component (PC1), with the largest weights among the positive and negative values. For the conversion into standardized value, the mean value of the score is subtracted from the actual value of the respective indicator and divided by the standard deviation of scores (Karim et al. 2021). Additionally, these standardized scores are multiplied by PC1 to obtain the scores of each dimension. The summation of each dimension score yields the final SES at the household level. The mathematical form of the PCA is given as follows:

$${{PC}}_{1}={\alpha }_{11}{Y}_{1}+{\alpha }_{12}{Y}_{2}+----+{\alpha }_{1n}{Y}_{n}$$

$${{PC}}_{2}={\alpha }_{21}{Y}_{1}+{\alpha }_{22}{Y}_{2}+----+{\alpha }_{2n}{Y}_{n}$$

$$-----------------$$

$$-----------------$$

$${{PC}}_{m}={\alpha }_{m1}{Y}_{1}+{\alpha }_{m2}{Y}_{2}+----+{\alpha }_{{mn}}{Y}_{n}$$

Where, ${{PC}}_{1}$, ${{PC}}_{2}$ and ${{PC}}_{m}$ are the principal components equations with ${Y}_{n}$ different variables and ${mn}$ equations weights. Formally, the linear combination of the SES for household $i$ is calculated based on the following equation:

$${Y}_{i}={\alpha }_{1}\left(\frac{{X}_{1}-{\bar{X}}_{1}}{{S}_{1}}\right)+{\alpha }_{2}\left(\frac{{X}_{2}-{\bar{X}}_{2}}{{S}_{2}}\right)+----+{\alpha }_{k}\left(\frac{{X}_{k}-{\bar{X}}_{k}}{{S}_{k}}\right)$$

Where, ${Y}_{i}$ is the SES, ${\bar{X}}_{k}$ is the mean of indicators, ${S}_{k}$ is the standard deviation and ${\alpha }_{k}$ are the weights.

Following the assessment of SES, this research employed a quasi-experimental design by using Difference-in-Differences (DID) to examine how a cash transfer program influenced the well-being of households. This approach utilized longitudinal data from both the treatment and control groups to establish a suitable counterfactual for estimating the causal effect (Branas et al. 2011). DID is commonly employed to determine the impact of a particular intervention or treatment by comparing changes in outcomes over time between a group that participated in the program (the intervention group) and a group that did not (the control group) (An et al. 2021; Galiani et al. 2005).

Assumptions:

• The intervention given was not related to the outcome measured at the beginning (allocation of intervention was not based on outcome).

• The treatment and control groups showed similar trends in outcomes over time.

• The intervention and comparison groups remained consistent in a repeated cross-sectional design.

• There were no unintended effects of the intervention that affected the control group or other groups (i.e., no spillover effects).

Regression Model

The mathematical form of DID design is given below and Table 2 and Fig. 1 provide the coefficient explanation and visualization.

$${\rm{Y}}={\rm{\beta }}0+{\rm{\beta }}1* [{\rm{Time}}]+{\rm{\beta }}2* [{\rm{Intervention}}]+{\rm{\beta }}3* [{\rm{Time}}* {\rm{Intervention}}]+{\rm{\beta }}4* [{\rm{Covariates}}]+{\rm{\varepsilon }}$$

Table 2 Explanation of regression model.

Fig. 1

Distribution of PCA scores in 2011.

Table 3 provides information about different domains and indicators that used to assess the level of deprivation or well-being of households. Each domain represents a specific aspect of household conditions or characteristics that are considered important for measuring deprivation. The indicators are used specific variables within each domain to assess the level of deprivation, and the deprivation cutoffs indicate the criteria used to determine whether a household is deprived or not. This study used four different domains with 15 indicators that were selected from the different literature to assess the household level well-being or deprivation (Padda and Hameed 2018; Hameed and Qaiser 2019; Karim et al. 2021; Vyas and Kumaranayake, 2006).

Table 3 Selection of domains and Indicators.

Material sources

This domain includes indicators related to ownership of agricultural land, livestock and other household assets like refrigerator, freezer, air conditioner, air cooler, fan, geyser, washing machine, etc. If a household owns these assets, it is assigned a value of 1, otherwise 0, indicating the presence or absence of material sources of well-being.

Living standard

This domain includes indicators related to access to basic amenities like electricity, clean drinking water, toilet facilities, and having a clean roof and adequate sanitation. If a household has access to these amenities, it is assigned a value of 1, otherwise 0, indicating the presence or absence of a satisfactory living standard.

Financial hardship

This domain includes indicators related to household financial status, including cash savings and loan-taking behavior. If a household has cash savings and no member has taken a loan in the last 12 months, it is assigned a value of 1, otherwise 0, indicating the presence or absence of financial hardship.

Women empowerment/mobility

This domain includes indicators related to women’s empowerment and mobility, including their ability to go to the local market, visit health facilities or doctors, go to the homes of friends in the neighborhood, visit shrines or mosques, and vote in previous elections without any restrictions. If women in the household have these freedoms, it is assigned a value of 1, otherwise 0, indicating the presence or absence of women’s empowerment and mobility. All these domains and indicators used in this study are critical for assessing social safety net programs at the local level. They provide a comprehensive, evidence-based, and context-specific measurement of household conditions, and have policy and programmatic implications for effective targeting and implementation of social safety net interventions.

Results

Figures 1 and 2 display the distribution of PCA scores based on histogram data from the years 2011 to 2019. However, there is no available recent impact evaluation data beyond 2019. In the initial years of the BISP program intervention (2011), the PCA scores ranged from −2 to 2, whereas by 2019, the range had expanded to −5 to 5. At the aggregate level, the SES score of some individuals in both the control and treatment groups has increased over time.

Fig. 2

Distribution of PCA scores in 2019.

Table 4 presented well-being or socioeconomic score (SES) for BISP beneficiaries and Non-beneficiaries across three different years of the monitoring data, including 2011, 2016, and 2019 with Standard Deviation (SD) and maximum and minimum. The SES serves as a measure to assess the overall well-being or welfare of household and individuals of the beneficiaries and non-beneficiaries based on various abovementioned indicators. In 2011, the average SES for beneficiaries was estimated at 57.1 and the average SES for the comparison or control group in the same year was 58.8. Analyzing the progress and impact of the BISP unconditional cash program over time, the study finds a reduction in the SES among BISP beneficiaries and control group as well. From 2011 to 2016, the SES decreased from 57.1 to 48.3, and further declined from 48.3 to 47.9 between 2016 and 2019. A similar trend is observed in the non-beneficiaries, where the SES decreased from 58.8 in 2011 to 49.7 in 2016, and then to 48.7 in 2019. Over the entire period from 2011 to 2019, there was a significant decrease of 9.2 points in the SES among beneficiaries and a 10.1-point decrease among non-beneficiaries.

Table 4 Socioeconomic score 2011–2019.

According to regional-level calculations, SES of BISP beneficiaries in urban households (HHs) was estimated at 60.6 in 2011, 57.7 in 2016, and 54.9 in 2019. For rural HHs, the SES figures were 56.2 in 2011, 46.0 in 2016, and 46.1 in 2019. This data indicates a reduction of 2.9 points in SES among urban BISP beneficiaries between 2011 and 2016, and a substantial decrease of 10.2 points in rural HHs during the same period. Notably, rural BISP beneficiaries experienced a larger decline in well-being compared to urban HHs from 2011 to 2016, while the reverse was observed from 2016 to 2019, where urban BISP beneficiaries faced a significant drop in well-being as compared to rural HHs. Overall, there was a decrease of 5.7 points in SES among urban HHs and 10.1 points in rural HHs. Contrasting with non-beneficiary data, urban non-beneficiary HHs experienced an average 9.1-point reduction in SES, whereas rural non-beneficiary HHs faced a larger decrease of 10.7 points from 2011 to 2019. Urban non-beneficiaries encountered a more substantial SES reduction compared to beneficiaries, while rural beneficiaries and non-beneficiaries had nearly equal reductions in SES from 2011 to 2019 (Fig. 3). Decomposition of SES among respective years of quintile shows the more depth details of SES reduction over the period among beneficiaries and non-beneficiaries in Figs. 3, 4. By examining SES for each quantile and each year, it provides a more precise understanding of how the well-being levels vary and how the BISP program’s impact may have influenced different segments of the population.

Fig. 3

Socioeconomic score 2011–2019 by urban and rural.

Fig. 4

BISP beneficiary SES by quintile.

In the year 2011, the well-being scores for BISP beneficiaries were higher in all quantiles of the HHs. Over the period of time, well-being has been reduced from 37.4 to 25.3 in Q1 HHs from 2011 to 2019, 57.2 to 44.4 in Q2, 63.5 to 54.0 in Q3 and 71.0 to 68.5 in Q4. The overall, higher quintile HHs of BISP beneficiaries have reduced less portion of well-being as compared to the low quintile. A similar trend was observed in the non-beneficiary data. The well-being index (SES) decreased by an average of 14.4 points in Q1 non-beneficiary HHs from 2011 to 2019, 12.7 points in Q2, 9.8 points in Q3, and 2.7 points in Q4. In contrast, BISP beneficiaries experienced relatively smaller reductions. The average decrease was 12.2 points in Q1, 12.8 points in Q2, 9.6 points in Q3, and 2.5 points in Q4. Overall, higher quintile groups faced less reduction in well-being compared to lower quintile groups. This indicates that the BISP beneficiaries in the lower quintiles experienced relatively better preservation of their well-being compared to their non-beneficiary counterparts in the same quintiles (Fig. 5).

Fig. 5

BISP non-beneficiary SES by quintile.

Domain wise SES of BISP beneficiary and non-beneficiaries

Material sources

In 2011, Non-beneficiaries had 0.20 SES of the material sources, while BISP beneficiaries had 0.19 at the baseline survey. Moving to 2016, there was a slightly increase in material resource score for both groups. Non-beneficiaries’ rose to 0.2 point, indicating a small improvement in their access to material sources. Similarly, BISP beneficiaries also experienced an increase in material source to same proportion of non-beneficiaries. However, by 2019, the access to material sources showed a different trend. Non-beneficiaries decreased 0.20 to 0.18 and BISP beneficiaries decrease 0.19 to 0.17 during 2011 to 2019. These figures suggest that both groups faced a reduction in their access to material resources by 2019 but BISP beneficiaries faced less reduction of well-being portion of material resources as compared to control group.

Living standard

In 2011, the living standard of Non-beneficiaries was measured at 0.53, slightly surpassing the living standard of BISP beneficiaries, which stood at 0.50. By 2016, both Non-beneficiaries and BISP beneficiaries witnessed an improvement in their living standards, as both groups reached a level of 0.60. However, in 2019, there was an increase in living standards for both Non-beneficiaries and BISP beneficiaries, with both groups experiencing increase to a standard of 0.63.

Financial hardship

In 2011, both Non-beneficiaries and BISP beneficiaries faced almost similar levels of financial hardship well-being, scoring 0.62 and 0.61 respectively. In 2016, the levels of financial hardship well-being decreased relatively for both groups, with Non-beneficiaries at 0.60 and BISP beneficiaries at 0.58. In 2019, the level of financial hardship well-being was decreased 5 points for non-benefices and 4 points for BISP beneficiaries. It shows that non-beneficiary faces more decrease in financial hardship as compared to BISP beneficiary but magnitude was low in BISP beneficiaries.

Women empowerment/mobility

In 2011, Non-beneficiaries had a relatively higher level of women empowerment/mobility with a score of 0.90, while BISP beneficiaries scored slightly lower at 0.89. In 2016, the levels of women empowerment/mobility remained consistent for Non-beneficiaries (0.90) but increased for BISP beneficiaries (0.91). In 2019, Non-beneficiaries experienced a decline in women empowerment/mobility, scoring 0.83, while BISP beneficiaries maintained a higher level at 0.89 (Fig. 6).

Fig. 6

Domain Wise SES of BISP beneficiary and non-beneficiaries.

Difference-in-difference analysis

The DiD analysis is used as the ideal method for analyses in a developing nation program when we cannot be in the pure experimental assumption, this can still be clever analytical tools to look at in longitudinal data. This refers to quasi-experimental design, as it relies on counterfactuals to assess the causal impact. It makes less assumptions about interchangeability. In this study, we investigated the SES during 2011–2016, 2016–2019 and 2011–2019, respectively. Table 5 shows the result of DID analysis and Model-1 of examining the UCT effects on SES during 2011–2016. In the intervening years, the average decline in SES is 9.38 points, which is statistically significant. The treatment variable is even more pronounced, decreasing 2.12 (p-value = 1%) from the benchmarks. In this model, the coefficient for the treatment variable has a significant coefficient in the DiD analysis (ie, indicating there are systematic differences in outcomes between treatment and control groups. This study did not employ propensity score matching due to a small control group. This raises a question about the BISP program’s design of the treatment and control groups data.

Table 5 Difference-in-difference analysis.

Despite these challenges, the research community regularly analysis studies of the impact of BISP on food security, poverty, and women’s empowerment. To comprehensively assess the current and likely impacts of the BISP Initiative, good quality data are urgently required. This was consistent in the other 2016–2019 models and also for the 2011–2019 models. The time variable indicates a decline of 0.26 points, which is not statistically significant, while the treatment variable alone shows a significant decline of 2.24 points at the 1% level for 2016–2019. Overall, Model-3, which covers the period from 2011 to 2019, reveals a similar trend-the time variable shows a decline of 9.61 points, and the treatment variable shows a decline of 2.23 points in SES, both are significant.

DiD variable (average treatment effect on the treated.) exhibited a similar trend (the intervention group slightly improved SES but none of the 3 models were significant). This indicates an increase of 0.16 points in 2011–2016, 0.68 in 2016–2019 and 0.43 in 2011–2019. In contrast, household size was only mildly associated with the outcome with a coefficient of 0.5; 0.79; and.0.69 for each period, all demonstrating a positive relationship.

The results of the three models showed that employment was a significant factor. The coefficients for 2011–2016, 2016–2019, and 2011–2019 showed increases in SES of 4.35 points, 5.64 points, and 4.93 points, respectively, all significant at the 1% level. This substantial evidence suggests that employment is a key factor influencing changes in SES and supports a strong relationship between employment and SES. On the other hand, the effect of household remittances was less noticeable as compared to employment but increases were significant at the 1% level. The coefficient shows that 1.67 points from 2011 to 2016, 3.97 points from 2016 to 2019, and 2.59 points from 2011 to 2019.

Discussion

The SES is crucial for understanding the multidimensionality of households or individuals, it combines various Sustainable Development Goals (SDGs) such as poverty, education, health, access to drinking water, sanitation, clean energy, and women’s empowerment (Padda and Hameed 2018; Hameed and Qaiser 2019; and Karim et al. 2021). It serves as the complete progress picture for any interventions at the household or individual level and is used to gauge the success of certain programs (Saif-Ur-Rahman et al. 2018). In order to comprehend the effects of the flagship program of BISP under the UCT intervention, this study used SES. The goals of this program are to reduce poverty and give low-income households social safety nets against inflation and other economic uncertainties (Nayyab and Farooq, 2014). Due to the lack of independent research on BISP program effect evaluation, this study is the unique study to analyze UCT influence on household well-being using SES analysis.

The study’s findings showed that the SES decreased from 2011 to 2016 in both groups. However, the decrease was less pronounced in the treated group compared to the control group. Between 2016 and 2019, the SES experienced a slightly positive increase, but the change was very small and not meaningful. Overall, the SES decreased from 2011 to 2019 in both the control and treatment groups, but there was a slightly higher increase observed in the treatment group. This trend suggests that the BISP UCT program’s impact didn’t significantly alter the SES status of the treatment groups. The lack of a visible effect of the BISP’s cash transfers on households’ socioeconomic well-being over time and among various categories can be attributed to a number of factors. These motives range widely. First, after accounting for inflation, the nominal value of the cash payments climbed by 67% from 2011 to 2019, while their real worth fell by 9%. Second, households who receive payments have issues when there are inconsistencies in the frequency of payments. Beneficiaries should ideally get payouts every three months; however, this does not always happen. Third, the transfer’s funding is inadequate; based on what recipients actually get (Cheema et al. 2020; PIDE 2020).

In different countries, similar or opposite patterns were seen. The Indonesian government made programs to help poor families. These programs aim to lower poverty and raise the income of those families. A study by Hengky et al. in 2018 found that these programs didn’t make a big difference. Another study by Abay et al. (2022) looked at a program in Ethiopia. It found that smaller payments didn’t help much. They didn’t improve families’ situations in the short or long term. This suggests that different things affect short-term and long-term outcomes. So, making changes to help short-term welfare might not always help families be more resilient, and vice versa. Asma et al. (2023) found that amount supported under social safety nets, duration and payment interval are the significant factors of poverty, food insecurity and standard of living. Berhane et al. (2014) studied a program in Ethiopia. It found that this program made hungry seasons shorter and increased the number of livestock families owned. A meta-analysis by Andrews et al. in 2018 showed that social safety nets help families by increasing their consumption, lowering poverty and making them more resilient.

The BISP program does bring some assistance to the poor, but prior research suggests that it is not a “magic bullet” capable of totally eliminating or considerably lowering poverty (Nayyab and Farooq, 2014). However, research financed by BISP itself indicates that the program continues to contribute to the reduction of poverty. According to Cheema et al. (2016), it is linked to food intake per adult equivalent, greater women’s empowerment, improvements in livelihoods, and improved educational results. However, a comparable research financed in 2020 did not discover strong proof that BISP’s recipients’ levels of poverty, child malnutrition, or productive investments are reduced in a meaningful way. Women’s empowerment was the only area where there was a discernible improvement (Cheema et al. 2020; Iqbal et al. 2021). Research has demonstrated a link between BISP cash transfers and poverty when looking at the total dataset, but the link is less when simply looking at people in the lowest consumption quintiles (Saeed and Hayat 2020). Cash transfers may also have good and substantial benefits on children’s nutrition, according to results of a Regression Discontinuity Design (RDD) analysis (Mustafa et al. 2019).

Other factors, such as employment and receiving financial support from family members residing either abroad or domestically, significantly influence the well-being of families from 2011 to 2019. According to Ahmed et al. (2010), families are less likely to fall into poverty if they receive financial assistance from relatives living elsewhere. Household economic well-being contributes significantly and positively to the proportion of remittances in total household income (Shams and Kadow, 2020). It is crucial for families to allocate this money wisely to improve their circumstances (Mahapatro et al. 2017). Receiving financial support from family members, whether abroad or domestic, contributes to greater happiness (Joarder et al. 2017). Employment status significantly impacts household expenditure and is positively associated with food and nutrition insecurity (Hameed et al. 2023).

In order to make sure the benefits are working, it’s critical to frequently evaluate their levels. Nearly a quarter of Pakistan’s rapidly expanding population continued to live in poverty. In addition, Pakistan has not performed well on the multidimensional poverty index (MPI) (Padda and Hameed, 2018). This indicates that a variety of deprivation-related factors, such as a lack of access to a good education, access to healthcare, to a healthy diet, to acceptable housing, and to a safe environment, can negatively affect a person’s well-being.

According to theory about the value of money, the GoP should review the BISP program. There are several ways to invest tax and donations money, including income generation, the development of capital assets, microbusinesses, and climatically adaptable agriculture and livestock. Employment at the household level are strongly correlated with SES, the household employment ratio shows that despite a decline in poverty in Pakistan from 2014–15 to 2019–20, there is still room for discussion on economic growth, agricultural productivity, income inequality, and small business prospects (Salam and Hameed 2022). It is essential to keep in mind that injustice or inequality may lead to conflict and dissatisfaction in society by dividing people and damaging social and economic cohesion. The interconnection of social, economic, and political inclusion must always be kept in mind. In order to provide everyone a fair chance to succeed, inclusive development is the best thing the government can do at this pivotal time. These measures will promote inclusive growth at the macro level and aid in raising SES at the household level by decreasing gender and employment inequities (Fleurbaey et al. 2018).

It is well recognized that worker remittances contribute significantly to a country’s development by promoting economic expansion and reducing poverty gap (Vargas-Silva et al. 2009). Both in terms of the macroeconomic picture and for recipient households, remittances are becoming increasingly significant (Khan et al. 2021; Kock and Sun 2011). According to the Author’s qualitative observations, climate resilience is a further outside issue of the development of SES model that is crucial to the ability to sustain Pakistan’s growth paradigm. The production of livestock and agricultural crops is significantly impacted by climate change. Animal insurance is not offered, even though Pakistan’s livestock industry is crucial to improving the SES. Crop insurance is supplied on a very limited scale through a public-private partnership. The growth of livestock in Pakistan should be regularly taken into consideration by the GoP development model.

Conclusions and recommendations

The SES acts as an all-encompassing indicator that considers several SDGs targets and reflects the multifaceted welfare of families and people. This study used SES analysis to look at how the BISP and its main UCT intervention affected household welfare. The results showed a very simple picture. Both the treatment and control groups’ SES decreased over the time but increase was significant within higher quintile and less increase was estimated on average. It appeared that the SES status of the treatment groups did not significantly change over the observation period as a result of the BISP UCT program.

This absence of a noticeable shift in SES may be caused by a number of variables. First, over economic uncertainty, inflation, the real worth of monetary payments, diminishing their effectiveness. Second, recipients had difficulties as a result of inconsistent payment frequency. Thirdly, the program’s budget was insufficient to produce significant improvements in household consumption. Although the BISP’s effectiveness in reducing poverty has been questioned, research suggests that it have a positive influence on a number of areas, including women’s empowerment and education. However, there is a dearth of clear proof that poverty and other socioeconomic indices have decreased. Although their total impact on the lowest consumption quintiles is yet unknown, cash transfers may be promising for improving children’s nutrition.

Several recommendations to improve the SES by investing BISP funds in line with the theory of the value of money are included:

• The GoP should conduct a thorough review of the BISP program. Consideration should be given to alternative investment strategies that focus on income generation, capital asset development, microbusinesses, and climate-resilient agriculture and livestock. Prioritizing strategies that foster employment and economic growth can have a substantial impact on raising SES levels.

• Inclusive growth is essential for raising SES and reducing inequalities. Economic, social, and political inclusion should be promoted to provide equal opportunities and foster social cohesion. Policies aimed at reducing gender and employment disparities will contribute to inclusive development and improved SES outcomes.

• Acknowledging the significance of worker remittances in economic development, GoP should further encourage remittance inflows to enhance economic expansion and bridge the poverty gap. Policies that harness the potential of remittances can significantly impact household well-being.

• In order to ensure effective program outcomes and target the right beneficiaries, rigorous and continuous evaluation of welfare programs, including BISP, is crucial. This will allow for timely adjustments and improvements in program design.

• A holistic approach to poverty reduction that addresses education, healthcare, housing, and access to basic necessities is vital. The multidimensional poverty index should be considered to better understand and address deprivation-related factors.

• Climate change poses a serious threat to livelihoods, particularly in agriculture and livestock sectors. Implementing climate-resilient practices and offering insurance schemes for livestock and crops can mitigate the adverse effects of climate change on SES.