FormalPara Key points for decision makers

• Patient heterogeneity not only impacts relative treatment effects: its impact on baseline risk, utility and resource utilization should be acknowledged in economic evaluations

• General sources of patient heterogeneity are demographics, preferences and clinical characteristics

• Different methods for acknowledging patient heterogeneity in an economic evaluation are available for the design, analysis and presentation phases

• Potential equity concerns and resulting opportunity losses associated with acknowledging heterogeneity in economic evaluations should be presented

1 Introduction

Heterogeneity can be defined as ‘diversity in character or content’ [1]. In medical research, heterogeneity is depicted as the part of variability (the natural variation between patients) that can be explained by certain patient characteristics, such as age or stage of the disease [2]. This type of heterogeneity, which is referred to as patient heterogeneity, may influence the cost effectiveness of a treatment. Patient heterogeneity should be distinguished from differences in the nature of the treatment (e.g. different drug doses or different techniques used) or in the geographical region (transferability) that may impact cost effectiveness. These factors relate more to the generalizability of cost-effectiveness results [3] and are beyond the scope of this paper. The present paper collects available knowledge on acknowledging patient heterogeneity in the economic evaluation of healthcare programmes.

In healthcare, it has been recognized that patients differ in their response to treatment. Therefore, treatments are becoming increasingly tailored to the individual patient [4, 5]. At the same time, average results of economic evaluations for patient groups are used to assist population reimbursement decisions upon new healthcare programmes [6, 7]. Average population-based economic evaluations, however, can mask important sources of patient heterogeneity within a patient population that may be important to improve population reimbursement decisions [7, 8]. For example, a programme that is deemed cost effective for a subpopulation may not be reimbursed because another subpopulation has a poorer outcome, and therefore on average it is not cost effective for all patients with that particular disease. This would result in health benefits forgone, as a subgroup of patients does not receive the optimal treatment. Or, inefficiency occurs because a programme is reimbursed based on its average cost effectiveness for the total population, while it may not be cost effective for a subpopulation of these patients. This would result in suboptimal use of scarce resources. Hence ignoring patient heterogeneity could be costly both in monetary terms and in health gain. Coyle et al. [9] for example showed that by acknowledging patient heterogeneity in the treatment of acute myocardial infarction a considerable monetary gain could be achieved. Basu and Meltzer [10] calculated that acknowledging patient heterogeneity in prostate cancer patients in the USA could even result in US$70 million gained annually. They argue that the value of reflecting patient heterogeneity in reimbursement decisions can be far greater than the value of improved decision making at the population level [10]. Population reimbursement decisions that acknowledge patient heterogeneity could save money and increase population health. Obviously, this is only possible if the patient heterogeneity is known at the moment a treatment decision is made. This paper therefore focuses on observed patient heterogeneity, on which population reimbursement decisions could be based.

To date, only limited attention is paid in economic evaluations to patient heterogeneity. In clinical studies patient heterogeneity is often examined in terms of treatment effect [6, 7]. It is then examined whether heterogeneity exists in the relative treatment effect, using measures like relative risks, odds ratios or hazard ratios. This information, however, is insufficient for economic evaluations, as homogeneity in treatment effects from a clinical point of view does not necessarily imply homogeneity in cost effectiveness [6]. However, there is no systematic overview of which sources of patient heterogeneity exist and in which of the input parameters of an economic evaluation they may occur. Additionally, pharmacoeconomic guidelines provide hardly any methodological guidance on acknowledging heterogeneity [11]. Moreover, reflecting patient heterogeneity in reimbursement decisions may have ethical implications if it leads to limiting financial access to certain groups of individuals. For example, it is questionable whether policy makers are willing to reimburse a treatment only for a subgroup of patients, when this subgroup is based on race [12].

The objective of the present paper is to provide a comprehensive overview of the current knowledge regarding patient heterogeneity within economic evaluations of healthcare programmes. More specifically, we aim to address the following research questions:

  1. 1.

    Which sources of patient heterogeneity can be relevant in economic evaluation and in which of the input parameters of an economic evaluation can they occur?

  2. 2.

    Which methods can be used for acknowledging patient heterogeneity in an economic evaluation?

  3. 3.

    What concerns exist with regard to acknowledging patient heterogeneity in economic evaluation and population reimbursement decisions?

2 Methods

2.1 Search Strategy and Selection Criteria

A systematic literature review was performed to identify methodological papers on the topic of patient heterogeneity in economic evaluation. We used the electronic database PubMed to search for peer-reviewed full papers, published before 2011. The electronic search term was as follows: (‘heterogeneity’) and (‘cost-effectiveness’ or ‘economic evaluation’). Additional references were searched through manual searches of the reference lists of retrieved papers and specialist journals, and through discussion with the authors. Studies were included only if they were written in English, had an explicit methodological objective (i.e. objective, keywords or publication type includes ‘methodology’ or ‘methods’; or objective is to ‘discuss’, ‘propose’, ‘develop’, or ‘argue for change in economic evaluation’), and considered patient heterogeneity that is potentially known at the time of treatment decision. Papers were excluded if the main goal was to report outcomes, if they concerned heterogeneity not related to patients (e.g. heterogeneity in the nature of the treatment or issues relating to transferability) or if they concerned other methodological issues. Two authors (MJ, JG) independently reviewed the titles, abstracts and, if necessary, full papers, to determine whether a paper met the inclusion criteria. Discrepancies were solved by all authors in a consensus meeting. Additionally, handbooks on economic evaluation were searched for recommendations on patient heterogeneity. Expert consensus by all authors was used to determine whether the most important papers and handbooks were included.

2.2 Data Extraction and Analysis

Relevant data were extracted from the publications by two authors (JG, MJ) independently. First, for all included publications we retrieved the sources of patient heterogeneity that were mentioned in their examples, and in which of the input parameters of an economic evaluation this heterogeneity occurred. With input parameters, we refer to the data inputs of an economic evaluation in general, be it a trial-based or a model-based economic evaluation. Subsequently, these sources and input parameters were combined in a framework, in order to allow for a systematic consideration of patient heterogeneity in economic evaluations. Second, from the publications that explicitly mentioned methods to acknowledge patient heterogeneity, we extracted these methods. Methods were categorized into methodology for the design (e.g. choosing trial outcomes, collecting model input parameters, designing model structure), analysis (e.g. analysing costs, effects, cost effectiveness, uncertainty) and presentation (e.g. presentation of costs, effects, cost effectiveness, uncertainty) phases of an economic evaluation [6]. Methods were extracted for both trial-based and model-based economic evaluations. If a method is only useful for one type of economic evaluation (model or trial), this is specified. Third, we extracted specific concerns that were mentioned with regard to acknowledging patient heterogeneity. Discrepancies were solved in a consensus meeting with the authors.

3 Results

3.1 Selected Sources and Characteristics

On the basis of the keywords, 231 articles were found (Fig. 1). A total of 220 articles were excluded because they were not deemed relevant, which left a total number of eleven studies included in the review [7–10, 13–19]. Most excluded references reported on heterogeneity between studies or differences in the nature of treatment instead of patient heterogeneity. Five additional articles were retrieved through manual searches of the reference lists and discussion with the authors [12, 20–23]. Four additional publications were included because they were responses to the included references [24–27]. The 20 included studies were published between 2001 [23] and 2010 [22]. Five handbooks on economic evaluation were included in the review [2, 6, 29, 48, 49]. All included publications are listed in Table 1.

Fig. 1
figure 1

Flow diagram of the search results

Full size image
Table 1 Overview of the included publications and the source of patient heterogeneity described
Full size table

3.2 Sources of Patient Heterogeneity

Table 1 summarizes the sources of patient heterogeneity that were used as examples in the included publications. The columns describe the input parameters of an economic evaluation where the patient heterogeneity occurred in the examples.

Based on Table 1 we developed a framework to systematically consider patient heterogeneity in economic evaluations. Economic evaluations consider both costs and effects. Regarding the costs, patient heterogeneity will not occur in unit costs, as these are likely to be independent of patient characteristics. Differences in unit costs are more likely a result of differences between jurisdictions, which is an issue of transferability [28]. Resource utilization, however, may well be affected by characteristics of the patient. Regarding the effects, from Table 1 it was clear that there may be patient heterogeneity in baseline risk (e.g. treatment-independent probability of a clinical event, mortality rate, incidence), treatment effect and health state utility. Heterogeneity in the treatment effect is the most recognized form of heterogeneity, as it is also examined in clinical studies. This mostly refers to heterogeneity in relative treatment effects such as relative risks, odds ratios or hazard ratios. Heterogeneity in baseline risk and health state utility refers to absolute effects, which are also of importance in economic evaluations.

Demographic characteristics of a patient, such as age, sex, intelligence and race, may be a source of patient heterogeneity. Preferences may also be a source of heterogeneity, for example because of differences in beliefs, attitude or risk tolerance. Finally, several clinical characteristics that relate to the disease were mentioned as a source of heterogeneity, such as severity, disease history and genetic profile. In the framework, we consider genetic factors such as pharmacogenetic heterogeneity to be clinical characteristics. The framework to consider patient heterogeneity in economic evaluations therefore consists of three general sources of patient heterogeneity (demographics, preferences and clinical characteristics) that may impact four input parameters of economic evaluations (baseline risk, treatment effect, health state utility and resource utilization) [Fig. 2].

Fig. 2
figure 2

Overview of the sources of patient heterogeneity that can be relevant in economic evaluation, and the input parameters where they may occur

Full size image

3.3 Methodology to Acknowledge Patient Heterogeneity

Sixteen publications describe methodology to acknowledge heterogeneity in the design, analysis or presentation phase of an economic evaluation [2, 6, 7, 9, 10, 12, 14–21, 23, 29]. In any economic evaluation, stratified analyses of cost effectiveness (in the most extreme case stratified to N = 1) are needed to inform whether patient heterogeneity should be reflected in the reimbursement decision. The identified methodologies can be used to allow for such stratified analyses. Six publications describe methodology for use in the design phase [2, 14, 15, 18, 19, 23]. The most comprehensive methodology to acknowledge patient heterogeneity in an economic evaluation in this phase is provided by Briggs et al. [2]. They provide detailed guidance as to how regression methods can be used in a decision model to relate input parameters (e.g. survival, event probability) to a patient characteristic. These methods can be used in models that simulate cohorts of patients as well as in models that simulate individual patients. The authors describe a number of regression models that can be used depending on the nature of the data [2]. In line with this, three publications propose to use different input values for different subgroups of patients [18, 19, 23]. Both Brennan et al. [14] and Heeg et al. [15] state that stratified analyses are possible to acknowledge heterogeneity in models that simulate cohorts of patients, but this may not be ideal depending on the size of the model and the number of subgroups. Brennan et al. [14] describe that different subgroups (based on, for example, baseline risk) can start out in different health states. This way one can take into account that a specific course of action may differ depending on the baseline risk, by explicitly modelling this conditional on the health state. In case this technique becomes too complex, patient-level simulation provides a useful alternative to a cohort model [14]. However, stratified analyses will still be needed to inform whether patient heterogeneity should be reflected in the reimbursement decision.

Nine publications propose specific methods for the analysis phase of an economic evaluation [6, 9, 10, 12, 16, 17, 20, 21, 29]. When individual patient data are available on costs, effects and patient characteristics, the existence of patient heterogeneity can be analysed through the use of regression techniques. Hoch et al. [12] were the first to propose that regression analysis could be used within the net benefit framework. Two years later, the same authors further developed the regression approach to cost-effectiveness analysis by proposing the use of a system of seemingly unrelated regression equations [20]. This approach has the advantage of allowing different co-variates and functional forms for the two equations (for costs and effects). Nixon and Thompson [21] further developed this regression approach by proposing a coherent set of Bayesian methods that consider costs and effects jointly, and that allow for the usual skewness of cost data. Drummond et al. [6], Hoch and Dewa [16] and Marshall and Hux [17] all propose regression methods to explore subgroups, referring to the above-mentioned studies. Glick et al. [29] also propose regression techniques, but only to examine whether there is heterogeneity in the costs. It should be noted that the use of regression techniques informs about the subgroups that are statistically significant, but these techniques do not pronounce on the clinical relevance and relevance with regard to costs of a subgroup for reimbursement decisions. Additionally, regression techniques can lead to false-positive results if many subgroups are investigated or if they are not specified in advance of looking at the data [21].

Coyle et al. [9] propose to quantify the efficiency gains resulting from subgroup policy, which relates more to the relevance of potential subgroups. They provide a framework to calculate the net monetary benefit (NMB) of different subgroup policies, to examine whether subgroup policy is valuable and which subgroup policy is the most valuable. When comparative data on costs and effects are available for each individual patient, the expected value of individualizing care (EVIC) can be calculated. In this technique, proposed by Basu and Meltzer [10], the value of optimal decision making based on population-based averages is compared with the value of incorporating individual-level values of heterogeneous parameters (in their example individual preferences) in the decision-making process. They calculate the monetary gain that can be obtained through giving patients the treatment that is cost effective for that particular individual patient, compared with giving patients the treatment that is on average cost effective.

In the presentation of an economic evaluation, Briggs et al. [2] propose to present full results, including multiple cost-effectiveness acceptability curves, for subgroups of patients. Additionally, it is advised to present possible equity constraints that may be relevant, and their resulting opportunity losses [7, 9]. For example, if there are potential equity concerns regarding subgroup policy based on age, this should be presented, as well as the opportunity losses associated with (not) reimbursing the treatment for all age groups [9].

All methods are listed in Table 2. They are illustrated by hypothetical examples concerning the economic evaluation of treatments in non-small-cell lung cancer.

Table 2 Overview of methods for acknowledging patient heterogeneity
Full size table

3.4 Concerns With Regard to Acknowledging Patient Heterogeneity

Nine studies specifically describe potential concerns associated with acknowledging patient heterogeneity [6–9, 13, 20, 21, 24, 25]. Four publications caution for post hoc stratified analyses and the potential of misleading results and provide some requirements to avoid such misleading results [6, 8, 20, 21]. Drummond et al. [6] for example state that there should always be a plausible clinical explanation for differences between subgroups of patients, rather than crude data mining. Nixon and Thompson [21] in their paper also note the danger of false-positive results if many subgroups are investigated or if they are not specified in advance of looking at the data. This danger of false-positive results is also briefly mentioned by Stevens and Normand [8], who further state that this is more a statistical problem than an economic one. That is, subgroups should not only be statistically significant, but also relevant for reimbursement decisions. In this light, Willan et al. [20] mention that the role of stratified analysis in economic evaluation should be explored in terms of its importance for policy purposes in comparison with the usual reluctance to undertake stratified analyses in clinical evaluation.

Additionally, in a comment to the paper by Bala and Zarkin [13], La Caze [25] states that pharmacogenomics provides a challenge to fundamental principles that underlie most systems for deciding which drugs should be publicly subsidized. The author provides no solution, but states that ‘discussion and debate are required to navigate a line between the extremes while giving due consideration to both equality and utility’ [25], which was subsequently agreed upon by Bala and Zarkin [24]. While this discussion is specific to pharmacogenomics, acknowledging heterogeneity in general may cause equity concerns. Sculpher [7] for example mentions that there may be equity constraints on which subgroups are used for reimbursement decisions. In particular, the use of sociodemographic characteristics to determine which subgroups should have access to a technology may be considered inequitable. However, this will depend on whether, for example, a subgroup with a higher age actually experiences a lower treatment effect, or they merely have less years of life left and consequently less QALYs to gain. Also, it may for example be more acceptable to use subgroup policy based on race when race for biological reasons impacts the effect of a treatment, than if race only impacts cost effectiveness through resource utilization as was found by Willan et al. [20]. This emphasizes the importance of clarity regarding which input parameter of an economic evaluation is impacted by the patient heterogeneity. As was shown by Coyle et al. [9], equity constraints can impose costs in terms of health benefits forgone. They propose a framework to quantify the costs of these possible equity constraints by calculating the expected opportunity loss (net benefit loss) when gains from stratification cannot be fully realized due to equity constraints or professional non-adherence. However, they do not seek to determine what are and what are not suitable grounds for the stratification of patients. Sculpher [7] also mentions that the use of preference subgroups would raise equity issues. This is for example the case when preferences differ because of the income of respondents [23]. This equity concern is emphasized by Robinson and Parkin [26] in a comment on Sculpher and Gafni’s paper [23]. They state that preference subgroups conflict with the idea that societal welfare is to be determined by aggregation of the preferences of that society. In a reply, Sculpher and Gafni [27] dispute this conflict because their method uses societal preferences. The critique is however relevant when individual preferences are used, as was proposed in two of the included papers [10, 13].

4 Discussion

To the best of our knowledge, our paper is the first to systematically review the current knowledge on acknowledging observed patient heterogeneity in economic evaluation. We found that patient heterogeneity may relate to patient demographics, their preferences or clinical characteristics, and that it can impact baseline risk, treatment effect, health state utility and resource utilization. We have developed a framework that allows researchers and policy makers to systematically explore which source of patient heterogeneity could be relevant in each of the input parameters of an economic evaluation. Additionally, we have listed the various methods that can be used to account for patient heterogeneity, such as linking inputs to patient characteristics, regression methods and quantifying the value of subgroup or individualized policy. Finally, we have summarized the statistical and ethical concerns that may be related to acknowledging patient heterogeneity in population reimbursement decisions. Our results can provide guidance for researchers to systematically acknowledge heterogeneity in their assessments. Also, they can assist policy makers in requesting relevant evidence and assessing the evidence they base their decisions on.

In the present study, we searched for methodological papers on the topic of patient heterogeneity. Although our search was systematic, it may not have been comprehensive. Our search terms were restricted to heterogeneity and economic evaluation, and did not include related terms like subgroups, individual or variability. Therefore, only 11 of the 20 included papers were obtained through the initial search. Five additional papers were found by examining the reference lists of the included papers and discussion with the authors, and four additional papers were comments on the included papers. The three papers that described regression techniques to identify or define subgroups [12, 20, 21] were not found in the initial search, but were key publications in the present paper. These papers were probably not found because they did not refer to heterogeneity, but to subgroup analyses. Including more search terms (i.e. subgroup analysis, variability) might have increased the sensitivity of the initial search, but also decreased the specificity. Despite the low sensitivity of the initial electronic search, we are convinced that through searching the reference lists of the included publications and extensive discussion with the authors our literature review was sufficiently thorough and that we have not missed any key publications.

While the present review provides a list of potential methods for use in economic evaluation, it does not result in one preferred method. Because most methods serve different purposes, the choice of method depends more on the data that are available and the purpose (e.g. in the design or analysis of an economic evaluation) than on its quality. However, three different methods were found that all use regression techniques to identify subgroups of patients. In one method, the NMB is the dependent variable [12], while in the method presented in a more recent paper by the same authors costs and effects are separated [20]. Information about whether a patient characteristic influences NMB through costs or effects can be very relevant for decision makers, especially with regard to potential equity concerns. The third method also regards costs and effects as separate dependent variables, but is more flexible than the second method [21]. The advantage of this method is that it does not assume Normal distributions, which may make it more appropriate for real data. However, the disadvantage is that the method is technically more difficult to implement. Other regression techniques that are used in the context of economic evaluation may also be used to explore heterogeneity, but only these three were explicitly suggested in the literature. No studies were found that directly compare different methods. A potential disadvantage of all regression methods is that they require individual patient data, which may not always be available. Regardless of the method that is used, the testing of heterogeneity should be undertaken with due care and diligence.

Two methods were identified that assess the value of acknowledging heterogeneity [9, 10]. The EVIC technique that was proposed by Basu and Meltzer [10] is relatively new and not yet frequently used. Similar to the expected value of perfect information (EVPI), the EVIC represents an upper bound of the monetary gain that can be achieved in theory. In practice, this upper bound may not be reached, as due to variability we may not always be able to predict the optimal treatment for an individual patient (EVIC), just like we will never reach complete certainty (EVPI). A recent application of EVIC indicates that it is a useful and feasible technique on both a clinical and a policy level, especially when considering the EVIC for specific patient characteristics (parameter-specific EVIC) [30]. More studies on either the application or further development of EVIC are needed to support a widespread use of EVIC to assess the value of acknowledging heterogeneity in population reimbursement decisions.

When reflecting patient heterogeneity in economic evaluations and population reimbursement decisions, it is important to be aware of potential equity concerns. As mentioned by Sculpher [7], whether a subgroup policy will be deemed acceptable may depend on whether the heterogeneity in cost effectiveness is caused by differences in treatment effect or in one of the other input parameters. Patient heterogeneity in treatment effect is expectedly a more acceptable ground for subgroup policy. Also, the source of heterogeneity may be relevant, since a subgroup policy based on differences in clinical characteristics is expectedly more acceptable than a subgroup policy based on patient demographics such as age or race. A special source of patient heterogeneity is individual preferences. While Sculpher and Gafni [23] propose to acknowledge differences in preferences while maintaining the use of utility values of the general public, both Bala and Zarkin [13] and Basu and Meltzer [10] suggest the use of individual preferences to value health state utility. Techniques to reflect heterogeneity in preferences when valuing health state utility are becoming increasingly available [31]. Whether individual preferences should have a role in population reimbursement decisions is however a topic of debate [32–35]. Because of the differences in ethical concerns regarding sources of heterogeneity and the input parameters, it is important that when proposing subgroup policy it is clear which source of heterogeneity is relevant and in which of the input parameters of an economic evaluation this patient heterogeneity occurs. Also, it is important to quantify the potential costs of equity constraints, as proposed by Coyle et al. [9]. With this information policy makers can make a trade-off between efficiency and equity.

From the results of our review, we found that many papers and handbooks recommend to reflect heterogeneity by performing economic evaluations for specific patient groups using stratified analyses. Extensive literature exists on how to define subgroups based on clinical evidence [36–43]. However, this clinical heterogeneity relates to differences on a relative scale (i.e. heterogeneity in measures such as relative risks, odds ratios and hazard ratios) [6, 7]. It was clear from the present review that, since economic evaluations concern absolute benefits, heterogeneity in absolute effectiveness, for example because of differences in baseline risk, health state utility or resource utilization, should also be acknowledged in economic evaluations. That is, homogeneity in treatment effects from a clinical point of view does not necessarily imply homogeneity in cost effectiveness. This is nicely illustrated in cost-effectiveness analyses on cardiovascular disease, where baseline risk was found to cause heterogeneity in the cost effectiveness of the intervention [44, 45]. The framework in Fig. 2 allows researchers to systematically explore whether patient heterogeneity could be relevant in each of the input parameters of an economic evaluation. This allows researchers to consider heterogeneity on an absolute scale. In addition to the difference between heterogeneity in relative versus absolute effects, two other challenges arise when using stratified analyses in economic evaluation. First, it is unclear whether policy makers wish to implement stratified policy if there is no heterogeneity in the relative treatment effect. By presenting potential subgroups and costs of equity constraints, policy makers can make an informed trade-off between equity and efficiency. Second, if we incorporate stratified analyses in economic evaluations, we are uncertain about the definition of our subgroups, especially when the characteristic is a continuous measure and cut-off points are needed. Ideally this uncertainty should be incorporated in the economic evaluation. For these gaps in current knowledge, future methodological research could generate new knowledge that can be used when conducting economic evaluations.

Pharmacoeconomic guidelines, which present country-specific guidelines for conducting economic evaluations, provide hardly any methodological guidance on acknowledging heterogeneity [11]. As these guidelines have a very different purpose than original methodological articles, and the focus of the current review was on original methodology, we decided not to include them. However, it would be an interesting topic for further research to assess to what extent these guidelines are attuned to the available methodology identified in the current review.

While the present paper contributes to current knowledge, it also highlighted where knowledge was lacking. Although Sculpher [7] made an important attempt in this direction, none of the studies provided guidance on the types of subgroups that should be explored, or how subgroups should be defined. By systematically reviewing the sources of heterogeneity and input parameters where they may occur, and summarizing them in a concise figure, we hope to partly provide such guidance. Also, we have summarized which methods can be used in each phase of an economic evaluation. This may help researchers to systematically consider whether patient heterogeneity should be acknowledged in their economic evaluation, and how this can be done. Our results may be used in the field of comparative effectiveness research, where patient heterogeneity is recognized, but still very much focused on heterogeneity in relative treatment effects [46]. Our results may also facilitate the collection of data within access with evidence development schemes to examine patient heterogeneity [47]. It could help policy makers and researchers to agree upon which evidence to collect, and could foster the acknowledgement of patient heterogeneity, even when an intervention is on average cost effective for the population. In the end, acknowledging patient heterogeneity may well improve the efficiency of healthcare.

5 Conclusion

By systematically reviewing current knowledge regarding patient heterogeneity within economic evaluations of healthcare programmes, we provide guidance for future economic evaluations. Guidance is provided on which sources of patient heterogeneity to consider, how to acknowledge them in economic evaluation and potential concerns. The improved acknowledgement of patient heterogeneity in future economic evaluations may well improve the efficiency of healthcare.