FormalPara Key Points for Decision Makers

There is a relative dearth of economic evaluation evidence for health-care interventions in India.

The quality of economic evaluation studies for health care in India needs improvement, especially in addressing the uncertainties involved in the modelling estimates.

There is a need to generate capacity of researchers to undertake quality economic evaluations, as well as an orientation of the policy makers so that there is demand for such studies as well as a scope for its use in policy making.

1 Introduction

In high income countries (HIC) such as USA, UK, Canada and Western European countries, policies to promote the use of evidence on value for money have long been in place [1–3]. Countries in the South East Asia region (SEAR) such as Taiwan, Malaysia and Thailand have also institutionalised health-technology assessment processes (HTA) into certain areas of policy. These policy measures have strengthened the imperative in each setting on achieving efficacy, cost effectiveness and value for money in the health sector [4, 5].

Recent initiatives in India have indicated a growing recognition of the important role of economic evidence in setting health-sector priorities. For instance, the National Technical Advisory Group on Immunization (NTAGI) was set up to inform decision making for introduction of new vaccines and strengthening the Universal Immunisation Programme (UIP) [6]. The Department of Health Research in India has recently set up a Medical Technology Assessment Board (MTAB) for evaluation of appropriateness and cost effectiveness of the available and new health technologies in India [7]. The MTAB aims to encourage investment in cost-effective interventions that will reduce the cost and variations in patient care, expenditure on medical equipment in directly affecting the cost of patient care, overall cost of medical treatment, reduction in out-of-pocket expenditure of patients and streamline the medical reimbursement procedures. Also, recently, the Indian chapter of International Society for Pharmacoeconomics and Outcomes Research (ISPOR) produced guidelines on how to conduct high-quality economic evaluation studies [8]. ISPOR provides an environment for knowledge sharing among researchers, health-care practitioners and decision makers interested in pharmacoeconomics and outcomes research.

A recent systematic review of pharmacoeconomic studies from India found 29 articles published during the period from 1998 to 2012 [9]. However, this review covered only economic evaluation studies which focussed on drugs. Moreover, the review included both full economic evaluations (cost per outcome description with comparison of alternatives) as well as simple cost analyses. Another systematic review from India by Mishra et al. found 132 articles published between 1999 and 2012 [10]. However, this was not focussed on full economic evaluations alone, and included several other studies such as cost-only analysis and studies which measured changes in outcome (estimation of quality of life) alone. Following both these reviews the present review breaks new ground in assessing full economic evaluations on all health-care interventions and programmes reported from India.

Given the very low public investment in the health sector, it is critical that resources are used wisely on those interventions that are proven to yield best results. Sound investment decisions require technical evaluations, and it is important at this juncture of policy environment to assess progress that has been made in generating a body of evidence around cost effectiveness of health programmes in India. Hence we undertook this study to assess the extent and quality of evidence for economic evaluation of health-care interventions or programmes in India. Based on the results, we indicate how India might move ahead to optimise resource use in its various health programmes and interventions. While it may be possible that some evaluation results have gone directly into government policies without being published, we believe it is not very probable given that sound evaluation studies are highly publishable.

2 Methods

2.1 Search Strategy

A comprehensive computerised search was conducted in November 2014 to search for published health economic evaluations pertaining to India and addressing a health-related intervention or programme. PubMed, Scopus, Embase, ScienceDirect and York CRD (Centre for Reviews and Dissemination) database were identified to search for evaluations published from January 1980 to the middle of November 2014. Website search of some important UN agencies (World Health Organization, World Bank and Asian Development Bank) and an Indian economic research agency (National Council of Applied Economic Research) was also carried out. The search strategy and the keywords are presented in “Box 1” and Figure 1. The key words were checked for controlled vocabulary under Medical Subject Headings (MeSH) of PubMed, and ‘exploded’ in the database thesauri.

Fig. 1
figure 1

Flow diagram showing study selection

Full size image

Library staff of Post-Graduate Institute of Medical Education and Research (PGIMER), and the research staff from the Advanced Centre for Evidence-Based Child Health in the department of Paediatrics, Post-Graduate Institute of Medical Education and Research, Chandigarh were consulted to finalise the search strategy. The review included only peer-reviewed articles that were reported in the English language and excluded abstracts, reports, expert opinion, narrative reviews, etc. To our knowledge, all of the seven leading Indian journals on economics and health economics that are published in local languages are also published in English language. Hence, we did not include papers published in non-English language.

2.2 Study Selection and Inclusion Criteria

The studies were selected based on a two-stage screening process as shown in Fig. 1. The search started by screening the titles and abstracts of all articles found in the initial search from the databases and websites. Based on the screening of titles and abstracts for removing duplicates, potentially relevant studies were selected for further review, which involved examining the content of their full text. In the second-stage screening, only those studies were considered, which were full health economic evaluations, i.e. comparing both costs and outcomes of two or more interventions and excluding partial economic evaluations and cost-only analysis. Only those peer-reviewed papers which presented a full economic evaluation of a health-care intervention or programme pertaining to India, published in English during 1980 to November 2014 were considered eligible for full review. At this stage, a bibliographic search of the selected studies was carried out to identify additional relevant economic evaluations. The search was continued until no new article was found. Two authors (ASC and BA) had access to abstract and full text of the paper to decide on its inclusion. Discrepancies between the two investigators were solved by discussion with the lead author (SP). Efforts were made to remove any bias by following strict criteria for inclusion of studies in the review. Three authors for the present review are health economists, with two authors having significant experience of undertaking systematic reviews. Two of the authors have additional background as medical professionals.

2.3 Data Extraction and Quality Appraisal

A standardised data extraction form was developed to collect the general and methodological data from the selected studies. An electronic data collection form was used, which was designed by the same people who extracted the data. Section 7.5.3 of Cochrane Review Handbook was consulted while designing of this form [11]. For assessing the quality of studies, Drummond checklist developed by Drummond et al. was used [12]. For evaluating studies based on a decision model, decision-analytic modelling checklist was adapted from guidelines developed by Philips et al. [13]. A weighted version of Drummond checklist developed by La Torre et al. was used to give a composite score to each study based on its quality [14].

The general information section of the data extraction form included the following items: year of publication, lead and corresponding author, institutional affiliation, number of authors, country of residence of lead author, profession of the lead author, publishing journal, country of the journal (Indian or foreign), funding source, disease/subject area of the study and the type of intervention (pharmaceuticals, public-health programme, service delivery, etc.). The methodological section included the following: type of economic evaluation, study design, perspective, time period, discount rate, the primary outcomes, type of costs and sensitivity analysis. Drummond checklist, which assessed the quality, comprised 35 items divided into three sections: study design, data collection and analysis and interpretation of results. For each quality item, a response was recorded either as ‘yes,’ ‘no,’ ‘not clear’ or ‘not applicable.’ A weighted version of the Drummond checklist was used to compute a composite quality score (global score) for each study based on the weights assigned to each of the 35 items in the checklist. The maximum overall quality score was 119. Weighted scores of individual studies were converted into a percentage. Modelling checklist included 17 items for assessing the model characteristics, model assumptions, quality of secondary data and measures to address uncertainties.

Two researchers (ASC and BA) independently assessed the quality of the studies based on Drummond and modelling checklist. Discrepancies between the two investigators were solved by discussion with the lead author (SP). Kappa statistic was calculated to measure the agreement between the two reviewers.

2.4 Data Analysis

Descriptive statistical analysis, including frequency and percentages, was used to describe the characteristics of the studies. The studies published pre 2005 (including 2005) and post 2005 were compared based on certain general characteristics and quality attributes, using chi square and one-way ANOVA. The year 2005 was used as a cut-off for multiple reasons. Firstly, India’s flagship programme, National Rural Health Mission (NRHM), which led to beginning of decentralised planning process was introduced in 2005. Secondly, India produced its 2nd and more comprehensive National Health Accounts in 2005. At the global level, an impetus to the use of economic evidence for policy planning was provided by release of the report of 2nd edition of the Disease Control Priorities Project in April 2006 [15]. Finally, the planning for a number of publicly financed health insurance schemes in India also started around that period. The association between quality and various factors such as year of publication, lead author affiliation and speciality, focus and type of evaluation, study design, perspective, etc., was also examined. Mendeley software was used to manage the references. Microsoft excel was used for data entry.

Incremental cost effectiveness ratios (ICERs) of all those studies where the primary outcome was reported as disability-adjusted life-years (DALY) or quality-adjusted life-years (QALY) were compared to present a summary evidence for use in India’s policy making. ICERs of all these cost-utility studies were adjusted to 2013 values, based on the wholesale price inflation (WPI) index in India [16]. ICERs of all cost-utility studies, which were reported in US Dollar (US$) were adjusted for inflation and converted to their values in 2013. Studies which reported ICERs in Indian National Rupee (INR) were first converted to US$ using currency exchange rates in the year of research and subsequently were adjusted for inflation as done for other cost-utility studies. The WHO Guide to Cost-Effectiveness Analysis (WHO CHOICE) categorises interventions as ‘‘highly cost effective’’ when the ICER is less than the gross domestic product (GDP) per capita, ‘‘cost-effective’’ when the ICER is between one and three times GDP per capita, and ‘‘not cost-effective’’ when the ICER is more than three times higher than GDP per capita [17]. Following these guidelines, the inflation-adjusted ICERs of various interventions, were compared with GDP per capita of India in 2013, which was US$1500 [18].

3 Results

A total of 5494 articles were identified from databases (4509 from PubMed, 372 from York CRD database, 296 from Embase, 250 from Scopus, 56 from ScienceDirect), websites (n = 6) and bibliographic search (n = 3) as shown in Figure 1. After removing duplicates, the remaining 5013 articles were screened by applying inclusion criteria to the titles and abstracts. A total of 4856 articles were excluded in the first-stage screening and 157 studies were identified as eligible for 2nd screening. Full text papers of these 157 studies were reviewed in the second stage. Ultimately, 104 articles were found eligible for this systematic review [19–122].

3.1 General Characteristics of Included Studies

Of the selected studies, the majority (64 %) were cost-effectiveness analyses followed by cost-utility (30 %) and cost-benefit analyses (6 %) as reported in Table 1. Only 20 % studies were published in Indian journals while the remaining (80 %) were published in international journals. The lead author was affiliated to an Indian institution in 39 % of these studies. Among 70 % of studies, either the lead author or a co-author was affiliated to a foreign institution. In most of these economic evaluations, lead authors were clinicians (36 %) or a public-health professional (41 %) with health economists authoring in only 7 % studies. An average number of 6.22 researchers authored the studies. About half of the studies used a provider or payer perspective (48 %) followed by societal perspective in 38 % of the studies. The perspective of study was not clear in 11 % of studies while for about 39 % of studies, perspective of the study was not explicitly stated.

Table 1 General and methodological characteristics of economic evaluation studies for health-care interventions and programmes in India
Full size table

The most common study comparator scenario used in these evaluations was the routine programme or care (41 %) followed by a ‘do-nothing’ scenario for comparison in 37 % of the studies. Decision modelling was used in 63 (61 %) studies, with remaining 41 being the trial based evaluations (39 %). Among trial-based studies, randomised controlled design was reported in 54 % of the studies. Similarly, among model-based studies, Markov model was used in around 50 % of the studies. Secondary data on cost and effectiveness were used in the 46 % and 56 % of the studies, respectively. Utility based outcome measures were used in 29 % (DALY) and 9 % (QALY) of the studies. Remaining studies used clinical end points (20 %), life-years saved (14 %) and illness prevented (20 %) as measures to value consequences or benefits. Around 22.2 % (n = 14) of the model-based studies relied on the expert opinion on some of their parameters. Specifically, five studies had used expert opinion to come up with cost estimates, three studies for quality-of-life weight assessment and eight studies for deciding on transitional probabilities/model structure.

Sensitivity analysis was performed in 69 % of the studies, with univariate and multiway analysis being followed in 91.6 % of these studies. Around 16.3 % (n = 17) of the studies had done probabilistic sensitivity analysis, while 3.8 % (n = 4) of the studies reported having undertaken bootstrapping to estimate confidence interval for the ICER estimate. Discount rate was stated in 60 % of the studies, with about half (49 %) the studies using a 3 % rate to discount future costs and benefits. In terms of funding, 42 % of the studies were funded either by an international donor or UN/bilateral aid agency. Nearly 34 % studies did not list the funding source, while it was reported as nil in 8 % of the evaluations. Only 6 % economic evaluations in health were commissioned and funded by Indian national or state government. Cost was reported in USD in 67 % of the studies followed by INR in 29 % of the studies.

Around 30 % of these evaluations focussed on pharmaceuticals, 26 % on public-health programme, 19 % on vaccines and 12 % on screening programmes. The interventions that were evaluated, addressed communicable diseases in 58 % of cases, while the remaining focussed on a non-communicable disease or injury. The interventions were mostly (60.5 %) preventive in nature. In a majority of cases the intervention took place in a primary-care setting (60 %), followed by tertiary (27 %) and secondary care (13 %). Around 50 % of the interventions addressed in the evaluations were community based followed by facility-based intervention (45 %). State-wise distribution shows that around 14 % of the interventions were done in Southern India, followed by 7 % in Delhi, 6 % in Gujarat, 5 % in Maharashtra, 3 % each from Andhra Pradesh and Bihar, and 2 % from north eastern state of Sikkim. Around 52 % of the interventions were reported considering India as a whole instead of focussing on any state.

More than three-quarters (78 %) of these evaluations identified limitations of their analysis, while in 69 % studies the authors discussed their findings in light of what others had reported. However, only 36 % of studies had considered the fiscal implications of the intervention on budget, and only 40 % considered generalisability of their findings.

3.2 Characteristics of Studies Before and After 2005

A total of 25 studies were identified as published pre 2005 while 79 were published after 2005 (Fig. 2). The percentage of lead authors from an Indian institution has fallen from 60 % in pre 2005 to 33 % post 2005, which was statistically significant (p < 0.05). Other factors which registered a statistically significant increase include: clinician and public-health expert as the lead author, publication in an international journal, HIV/AIDS and tuberculosis as the disease investigated, application of multiway sensitivity analysis and use of 3 % as the discount rate. For some other characteristics such as DALY as the outcome indicator, application of modelling for assessing cost effectiveness, use of secondary cost and effectiveness data, adoption of a provider or payer perspective, international funding, and evaluation of preventive care—a statistically insignificant difference was noted.

Fig. 2
figure 2

Comparison of economic evaluation studies pre and post 2005 based on certain general and methodological characteristics. Asterisk shows statistically significant at a value of <0.05)

Full size image

3.3 Quality of Studies in India

There was a high level of agreement between the two assessors, with the kappa statistic of 87.7 %. An assessment of quality of the studies in India is presented in Table 2. The studies from India were of good quality in terms of specifying the counterfactual or comparator scenario (89 %), description of alternative scenarios (94 %), indicating sources for effectiveness data (95 %), and description of the primary outcome (93 %). However, the studies had some major limitations. The choice of the type of economic evaluation used was not well justified in a large majority of 82 % studies. There was no explanation of the perspective used in 39 % of studies. Quantity of resources were not reported separately from their unit costs in 51 % studies, while details on price adjustments for inflation or currency conversion was not reported in 35.6 % of the studies. The discounting was either not done (30 %) or lacking in justification (37 %). In about one-fifth of the studies, the conclusions were not accompanied by appropriate caveats (19 %), or incremental analysis was not reported (21 %).

Table 2 Quality of economic evaluations studies for health-care interventions and programmes in India
Full size table

A total of 63 studies used decision analytic methods to model costs and effects. These model-based evaluations were of good quality in terms of presenting the rationale for model structure (94 %), appropriateness of model (94 %) and its time horizon (81 %), biological plausibility of the model and its disease transition states (92 %) and the choice of assumptions for transition probabilities (86 %) as reported in Table 3. The major limitation of the model-based studies in India emanate from their ability to address uncertainties involved in modelling. Only about one-third of the studies involving modelling addressed the structural uncertainties (33 %), or ran sub-group analyses to account for heterogeneity (36.5 %) or analysed methodological uncertainty (32 %).

Table 3 Quality of decision-model based economic evaluation studies for health-care interventions and programmes in India
Full size table

3.4 Factors Influencing Quality of Studies

The overall quality score of economic evaluation studies in India was 65.1 % (Table 4). There was no statistically significant change in quality of studies undertaken before (61.3 %) or after (66.3 %) the year 2005. However, the quality of studies was significantly higher with the following characteristics: involvement of a foreign author as lead author (71 %) or any co-author (69.4 %); published in international journals (69.4 %); using a cost-utility design (74.1 %); and funded by an international agency (73.8 %). The quality was significantly lower for those studies where: a clinician was the lead author (60.2 %); involved evaluation of curative (57.8 %) or tertiary care (57.2 %); evaluated drugs, i.e. pharmacoeconomic studies (56 %); and those which used a patient perspective alone (55.3 %). Also, studies based on observational trials had a higher mean quality score, i.e. of 58.7 %, followed by 56.5 % among randomised trials and 51.8 % in non-randomised trials. Among model-based studies, economic evaluations based on Markov model had a higher mean quality score, i.e. of 75.5 % followed by a score of 69.9 % in decision-tree model based studies.

Table 4 Factors affecting quality of economic evaluation studies for health-care interventions and programs in India
Full size table

3.5 Cost Effectiveness of Health-Care Programmes and Interventions in India

Among HIV/AIDS-related studies, all the interventions were reported as highly cost effective (Table S1). Similarly, ICERs for immunisation and tuberculosis interventions were rated as highly cost effective. Some of the non-communicable disease interventions were reported to be highly cost effective, i.e. ECG for acute coronary syndrome, screening and delivery of hearing aids at secondary and tertiary level, school-based eye screening programme and universal gestational diabetes screening. Tele-ophthalmology for diabetic retinopathy, school-based smoking prevention programme and primary eye care screening programme were cost-effective interventions. Among life-style interventions, only food labelling was cost effective. Use of auto-disable syringes, vitamin A supplementation, genetically modified (GM) fortification for vitamin A and treatment strategies for Helicobacter pylori were labelled as highly cost effective. Table S1 (supplementary material) contains the characteristics of these cost-utility studies.

Out of a total of 104 studies, 77 (74 %) authors reported the intervention as cost effective. While only 11 (10.5 %) found that intervention was not cost effective. The results of the remaining 16 (15.5 %) studies were either unclear or no strong conclusions were made by the authors.

4 Discussion

To our knowledge this paper is the first comprehensive systematic review of the evidence on economic evaluation for health care in India. Our review yielded a total of 104 full economic evaluations published from 1980 to 2014. The majority of these papers were cost-effectiveness studies (64 %), led by a clinician or public-health professional (77 %), using decision analysis-based methods (59 %) and published in an international journal (80 %). In addition, 42 % were funded by an international funding agency or UN/ bilateral aid agency, and 30 % focussed on pharmaceuticals. The average quality score of these full economic evaluations was 65.1 %.

4.1 Extent of Economic Evaluations for Health Care in India

The absolute number of studies uncovered in the review indicates that economic evaluation in health in India is at an early stage of development. The 104 papers included in this review compares to 1249 papers on cost effectiveness published in the USA between 1979 and 1990, and 1167 published between 1991 and 1996 [123]. Nevertheless, this exceeds the number of economic evaluation studies we found when using the same inclusion criteria in relation to other developing countries such as South Africa (n = 45), Thailand (n = 39), Vietnam (n = 26), Bangladesh (n = 12), Nigeria (n = 44) and Zimbabwe (n = 26) [124–129]. However, the gap in current economic evidence in India is exacerbated by the need for region- or state-specific studies that account for variations in epidemiological transition, health-care costs, and health-care infrastructure across the country.

A number of factors could explain this relative lack of economic evidence for health-care interventions and programmes. Firstly, the specialty of health economics is nascent in India. Two associations for health economics—the Indian Health Economics and Policy Association (IHEPA) and the Health Economics Association of India (HEAI) have had their inception within the last 5 years [130, 131]. There are no specialty courses in the field of health economics for those who undertake mainstream economics courses. For example, the premier post-graduate economics department in the country—the Delhi School of Economics—does not offer a graduate course on health economics. As a result, not many mainstream economists work in the field of health. Among those in the medical and public-health stream, there have been no courses which sensitise the students on economic or more specifically—health-economics issues. Hence, there is a general lack of awareness in terms of its value or potential application in clinical or public-health research. More recently, with the creation of Schools of Public Health, a multi-disciplinary approach has been engrained in courses such as Masters of Public Health (MPH) which includes teaching on health economics. However, the limited teaching of health economics in the MPH curriculum can only be useful to generate interest and a sense of awareness for economic issues, but does not train health economists who can independently carry out full economic evaluations. More recently, a free online course on health economics has been started [132]. While these are good for introductory level, there is a limitation to which these courses can engrain the more substantive areas of micro- and macroeconomics seen through the prism of health concerns. This may explain, to a certain extent, why economic tools and analysis remain a non-integral part of social science research in the health sector.

At the same time, adapting evaluation techniques does not necessarily require an economics degree and can be easily picked up by competent scientists and researchers who are not economists. This may explain why most studies in India are led by non-economists. A wider perspective of the health sector and a deeper understanding of core economic issues would be ideal for economic evaluation studies. This is evidenced by the finding that quality of economic evaluations was significantly high when it was conducted by an economist in the lead position, or when there was association of an author from a foreign university. The former indicates the need to bring trained economists into health sector analysis, and the latter indicates that qualified and technically sound researchers in India are not sufficiently interested in economic evaluation studies. Only a few and not all health economists in India may be interested in economic evaluation research. The reason could be the low use of evidence-based policy making in the country. Ultimately, economic evaluation studies are publishable, but not necessarily in high demand from domestic policy makers. The incorporation of cost-effective interventions into policies in India has not necessarily been based on results from cost-effectiveness studies carried out domestically: these have mostly been driven by international evidence and best practices that are subsequently adopted in the country. This may explain the disinterest among Indian researchers to devote time to economic evaluation studies.

Despite introduction of decentralised planning process with the onset of National Rural Health Mission (NRHM), a review of the programme implementation plans (PIP) of various state governments shows no consideration of economic evidence in guiding the choice of interventions included in the plan document [133–136]. In order to improve the access to medicines, Government of India has drafted an essential list of medicines. A total of 348 drugs have been included in National List of Essential Medicines (NLEM) in India [137]. Taking a cue from the Central Government, several state governments have also drafted their own essential drug lists [138, 139]. Although the draft document on the formulation of NLEM refers to the use of criteria of cost effectiveness in determining the selection of a given drug, of the 87 experts who participated in the discussion for formulation of NLEM, none were health economists or subject experts in economic evaluations. To get more economists interested in health economics generally and evaluation studies specifically demand must be generated either through the policy window (generating demand for such studies) or through the academic window (health economics as a field subject).

Another important factor to consider is the lack of government funding for economic evaluation studies in India, which emanates from the lack of interest among policy makers for such research. Only 6 % of the total studies were funded by the national or state government. Almost 30 % of the economic evaluation research in India was funded by international agencies or the UN/bilateral aid agencies. This explains the relatively large number of studies within the communicable disease section, which were done to evaluate HIV-related interventions. This may not be commensurate with the disease burden in India, where HIV does not figure among the top 10 causes of mortality [140]. With lack of domestic funding forthcoming, it is not surprising that there is a preponderance of non-domestic funding as well as partners in such research. Also, large programme evaluations are costly to carry out, and given the tight research funding situation in the country combined with the lack of interest in the government in evaluating existing programmes, the results of this research are not surprising. Conversely, the lack of government funding suggests that more could be done to promote the use of such evidence in policy making.

Another potential audience which could commission and use economic evaluations in health care could be various non-governmental organisations (NGO) involved in delivering health-care services such as related HIV, maternal health, child health, etc. These NGOs may not be as interested in influencing national public policy or academic debate, but may want to generate evidence to recruit support from international funding agencies to support their expansion.

4.2 Recent Policy Developments in India

There have been recent efforts on the part of Indian government in creating political infrastructure, guidelines and policy initiatives to incorporate economic evaluations in the Indian public health sector. Firstly, a memorandum of understanding (MoU) has been signed between the Department of Health Research of India and UK National Institute of Health and Care Excellence. It would create an opportunity for the exchange of institutional expertise and experience on clinical practice guideline pathways and quality standards, application of health-technology assessment, and implementation of the decisions of the assessment into clinical policy and practice [141]. As part of this collaboration, a manual for determining the standard treatment guidelines is being developed, which also includes a chapter on the “reference case” for economic evaluations in India. Further, the Department of Health Research in India has recently set up a Medical Technology Assessment Board (MTAB) for evaluation of appropriateness and cost effectiveness of the available and new health technologies in India. At present, there is no specific and structured role of economic evaluation in the pricing and reimbursement process in Indian context, which is reflected in pricing and reimbursement system for various publicly financed health-insurance schemes, as well as drugs and diagnostics. Under various public-health insurance schemes from India such as Rashtriya Swasthya Bima Yojana (RSBY), Rajiv Aarogyasri Health Insurance Scheme or Rajiv Gandhi Jeevandayee Arogya Yojana, the reimbursement rates are based on expert opinions and not on any formal costing or cost-effectiveness analysis. Drug price control order (DPCO) 2013, which has been given the responsibility of regulating the prices of drugs under essential medicine list, also does not take into consideration any formal costing or cost-effectiveness studies. The Government of India has recently set up a separate expert group on costing, in order to guide on evidence-based reimbursement for various benefit packages under the largest publicly financed health insurance scheme in India—Rashtriya Swasthya Bima Yojana (RSBY).

The Disease Control Priorities Project 3rd edition (DCP3), which contains an up-to-date comprehensive review of the cost effectiveness of priority health interventions with the special focus on low- and middle-income countries, includes contributions from a significant number of Indian researchers. Also, some of the economic evaluations being included under this project are based purely on Indian perspective [142–144]. Thus, DCP3 has also stimulated further interest among researchers doing economic evaluation of health interventions in India, as well as advocacy for use of such evidence by policy makers.

While it is too soon to comment on the influence of some of new initiatives mentioned earlier, on the general trend in evidenced-based policy making, indirect evidence indicates that some of the expert groups set up do not have the requisite structures to carry out evaluation studies; there seems to be a gap between intention and mechanisms required by way of resources and time—to carry out such studies. This again confirms that while global influences have prompted the government to at least acknowledge the usefulness of economic evaluation, systems are yet to be set in place to generate the kind of data required for such studies, which would necessitate a re-think on funding for such research.

4.3 Quality of Economic Evaluation Studies in India

In terms of the characteristics of the studies in India, our findings are quite similar to what has been found by others from developing country settings. Cost-effectiveness design is most predominant among the full economic evaluations undertaken in India, which is very similar to what has been reported from South Africa, Vietnam and Thailand [124–126]. Many factors could possibly explain the relative dearth of cost-utility studies. First is the application of more complex analytical methods to compute such measures. Second, there is the lack of locally available evidence on disability or quality-of-life weights. The majority (84.5 %) of the cost-utility studies used evidence on utility weights borrowed from non-Indian settings.

A major factor, which is likely to influence application of this evidence, is visibility of research to policy makers. Our review shows that a large majority of Indian research on economic evaluation is published in international (80 %), rather than national journals. This is much more than what has been reported elsewhere. This could be influenced by several factors such as higher impact factors of these international journals, their more specialised nature, i.e. covering health and economics aspects, and wider international readership. Given the preceding discussion on the results, it stands to reason to assume that publication is going to remain an important positive incentive for researchers to undertake such studies in India and those interested in high-quality publishable research would, therefore, look for outside funding, collaboration as well as best international journals to disseminate their work. This may or may not be accompanied by high visibility domestic dissemination of the results, if, in fact, there is not much interest within the country for such research. This further deepens the disconnection between policy and academic research, with low visibility of the usefulness of evaluation studies from the perspective of policy makers, who, clearly do not have the time or interest in the academic press.

The findings of our review highlight the role of International collaboration and researchers from outside India, who have played an important role in conducting these economic evaluations. This shows that while on one hand it is important to develop local capacity for undertaking economic evaluations, it is also useful to harness such collaborations in the short term till there is national capacity built.

In terms of quality, significant areas of improvement for economic evaluation studies have been highlighted in our review. Some areas which need significant attention of the researchers are focus of the viewpoint of the evaluations being undertaken, justification on the type of economic evaluation being used, lack of use of discount rates, weak costing methodologies and the extent to which these studies address the uncertainties in methodologies of economic evaluation, especially for model-based evaluations. Overall, our findings on quality of evidence are again very similar to what others have reported in the developing countries [125–128]. However, the overall quality in India seems better than what is reported for most of other developing countries. Still, quality needs to improve to meet standards set from high-income countries and cost-effectiveness analysis (CEA) checklists to improve the usefulness of the Indian body of evidence to decision making. Nevertheless strengthening the quality of such studies through measures such as extensive training will increase the credibility of such evidence and promote uptake.

In Indian settings, the health care is financed primarily through out-of-pocket expenditure. Out-of-pocket expenditure reflects the full cost of care when patients seek care in the private sector, and partial cost when care is sought in the subsidised public sector. This cost of care, represented by out-of-pocket expenditure is captured in economic evaluations using “patient perspective”. The provider perspective is used synonymously with “payer perspective”. Payer perspective also includes insurance reimbursement or cashless provision of care for insured persons. Hence, the provider perspective includes both instances where government acts as a provider of subsidised care or instances where insurance is used to pay for the health care.

4.4 Limitations

This review has some limitations. This study included only published literature in peer-reviewed journals and excluded grey literature such as government reports, pharmaceutical company reports, academic theses and conference proceedings. The inclusion of only published literature might have introduced publication bias, since studies with positive results are more likely to be published than studies with negative findings [145–147]. Furthermore, as in any review study, it is difficult to rule out selection bias or disagreement between the criteria of the reviewers. To minimise this bias, we used pre-defined inclusion criteria and discussion of disagreement between the investigators throughout the review process. We would also like to acknowledge that some economic evaluation studies which did include India in their analysis could have been missed in case the disaggregated results were not presented for India. We also acknowledge that the method developed by La Torre et al. is one of the ways to assess the quality of economic evaluations [14]. However, there have been other attempts such as the use of CHEERS checklist for assessing quality [148]. Besides, others have argued to apply separate weights to individual quality parameters in the checklist. A comprehensive assessment on limitations of scales for assessing quality is beyond the scope of this systematic review, and is suggested as a potentially relevant area for research in future.

5 Conclusion

The study indicates that evaluation of programmes and interventions has been somewhat sparse in the country, and also not of a very high quality. The existing body of results has been inadequate to feed into sound policy making. There is an urgent need to generate awareness within the government of how economic evaluation can inform and benefit policy making, and at the same time build capacity of health-care professionals in understanding the economic principles of health-care delivery system. The lack of demand is the main reason for these findings, and it is our belief that once the policy makers understand and demand such studies, engagement of technical experts and quality studies would be forthcoming, even if supported by outside funding. In a parallel fashion, government will have to actively encourage economists to focus on the health sector, which would go beyond the Ministry of Health and would need dialogues with the education sector. Evaluation studies remain currently somewhere in-between the medical sciences and social sciences, with neither field owning it fully. With greater demand and interest articulated by the government, India can see many more effective economic evaluation studies, done by competent researchers from both fields. While so far economic evaluation has not been a major feature of government programmes, the recent steps taken by the government need to be watched, to see whether they change the course of evidenced-based policy making in the health sector.