Principles and Practice for Trial-Based Health Economic Analysis

Abstract

In this chapter, key principles and practice for health economic analysis to undertake robust within-study cost effectiveness analysis are identified and illustrated. Principles are introduced considering the decision analytic basis for comparing alternate strategies in defined patient populations and their costs and effects along treatment pathways. Decision analytic principles for robust cost effectiveness analysis are shown to require joint coverage and compatibility of cost and effect evidence to allow unbiased estimation, the predominant consideration in informing societal decision making under the Arrow-Lind theorem (Arrow and Lind 1970).

These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

1 Overview

In this chapter, key principles and practice for health economic analysis to undertake robust within-study cost effectiveness analysis are identified and illustrated. Principles are introduced considering the decision analytic basis for comparing alternate strategies in defined patient populations and their costs and effects along treatment pathways. Decision analytic principles for robust cost effectiveness analysis are shown to require joint coverage and compatibility of cost and effect evidence to allow unbiased estimation, the predominant consideration in informing societal decision making under the Arrow-Lind theorem (Arrow and Lind 1970).

These principles are initially applied in this chapter to within-study cost effectiveness analysis for two-strategy comparisons, before being extended to more complex analysis in later chapters. In this simplest two-strategy within-study case, evidence of joint incremental cost and effects can be directly presented from trials on to the incremental cost effectiveness plane. Nevertheless, this only provides unbiased cost effectiveness analysis estimates to inform societal decision making where trial coverage and comparability of relevant incremental effects and costs along alternative treatment pathways are satisfied. Satisfying coverage and comparability conditions to inform unbiased cost effectiveness estimation and decision making more generally requires unbiased methods for evidence synthesis, translation and extrapolation relevant to the context of the jurisdiction to which decisions relate (see Chap. 3). The primary importance under the Arrow-Lind theorem of establishing unbiased cost-effectiveness estimates prior to considering societal decision making under uncertainty in informing joint reimbursement and research decisions (see Chaps. 5, 6 and 7) is nevertheless clarified.

Partialisation problems of the box method when attempting to present cost effectiveness evidence under uncertainty are shown as able to be overcome with non-parametric methods (bootstrapping) and parametric methods (Fieller’s method). Joint consideration of cost and effect uncertainty with these methods enables within-study cost effectiveness uncertainty to be appropriately considered with bivariate distributions on the incremental cost effectiveness plane where within-study analysis is directly applicable to societal decision making (does not require evidence synthesis, translation or extrapolation as per Chap. 3). Similarly, for two strategy comparisons bivariate distributions can in turn be simply summarised for societal decision making with cost effectiveness acceptability and net benefit curves. They respectively directly inform societal decision makers of the probability of, and incremental expected net benefit from adopting strategies, conditional on societal threshold values for effects. Principles and methods are illustrated with reference to the seminal ‘Thinking outside the box’ paper (Briggs et al. 2002) and the LIPID study of statin use (Glasziou et al. 2002).

The importance of net benefit as a robust metric to jointly allow for costs and effects in decision making under uncertainty while avoiding ordering problems inherent with incremental cost effectiveness ratios (ICERs) is highlighted, following Willan and Briggs (2006). Incremental net benefit metrics as the value relative to a comparator of incremental effects, less incremental costs also make explicit the need for economically meaningful threshold values for effects (Graham 1981, 1992) conditional on decision context for investment. Methods for determining economically meaningful threshold values that reflect opportunity costs (alternative best actions) conditional on local contexts (health system allocative and displacement inefficiency) are introduced with the health shadow price (Pekarsky 2012, 2015; Eckermann and Pekarsky 2014). However, this health shadow price and economically meaningful threshold values for effects are not fully considered, allowing for all relevant decision contexts, and emprically, until Chap. 11.

Nevertheless, summary measures for two-strategy comparisons of net benefit and cost-effectiveness acceptability curves introduced are shown to appropriately condition across potential threshold values in the absence of knowledge by analysts of the relevant empirical threshold value and related decision contexts in any given jurisdiction. That is, they present the probability of maximising net benefit (CEA curves) and expected incremental net benefit (INB curves) as a function across plausible ranges for threshold values. Similarly, robust summary measures for multiple strategy and effect comparisons (expected net loss curves and frontiers in Chap. 8 and planes and surfaces in Chap. 10) are presented as functions of plausible threshold values for effects in informing related reimbursement and research decisions.

Hence, jointly satisfying coverage and comparability principles and evaluating costs and effects together with net benefit analysis is illustrated not only as key for robust two-strategy within-study comparison but also as a foundation later for robust more complex analysis. Coverage and comparability principles with consistent and joint consideration of cost and effects along alternative pathways are later shown to also be critical building blocks for robust methods of:

• (i) Evidence synthesis, translation and extrapolation (O’Brien 1996; Eckermann et al. 2009, 2011) that are usually required to robustly inform societal decision making within any jurisdiction, as highlighted in Chap. 3;

• (ii) Joint research and reimbursement decisions when considering cost effectiveness evidence of promising strategies under uncertainty (Chaps. 5, 6 and 7); and

• (iii) Cost effectiveness analysis with more than two strategies (Eckermann et al. 2008; Eckermann and Willan 2011) in Chap. 8 and more than two outcomes (McCaffrey 2013; McCaffrey et al. 2015) in Chap. 10.

• (iv) Comparisons of providers, strategies and health systems in practice (Eckermann 2004, 2009; Eckermann and Coelli 2013) in Chaps. 10 and 11

.

2 Principles for Robust Health Technology Assessment

In undertaking economic evaluation, public health systems are responding to scarcity of resources in attempting to satisfy health needs across populations over time. Processes of health technology assessment attempt to inform choices between alternative strategies in treating defined patient populations based on comparing their relative costs and value of effects. Trade-offs arise in two strategy comparisons unless one strategy has lower costs and higher effects (dominates the other strategy) or equivalently the other strategy is dominated (has higher costs and lower effects). Where trade-offs arise, assessing value can be viewed as a set of scales (Fig. 2.1) weighing up the value of net incremental effects relative to net incremental costs.

Fig. 2.1

Cost effectiveness analysis – weighing up value of incremental impacts

Note, however, that calibration of such scales is required to represent value in trade-offs between incremental costs and effects. Hence, in making a decision about whether to invest in, or reimburse (adopt and finance) a strategy that has higher expected net costs, decision maker threshold values for effects need to reflect opportunity costs of adopting and financing actions to optimise health effects within any constrained budget (Pekarsky 2012, 2015; Eckermann and Pekarsky 2014). That is, threshold values for effects in reimbursement decisions should reflect highest value alternative adoption and financing actions. In Sect. 2.10, we start to consider how threshold values reflecting opportunity cost should be appropriately determined, a critical issue which we later return to in detail in Chap. 12 allowing for relevant decision contexts faced by jurisdictions in their health systems (allocative and displacement inefficiency particularly). Suffice to say from the beginning that one should always be mindful of the opportunity cost – the best alternative action(s) – that such threshold values should reflect to enable resource-constrained optimisation in decision making for any given health system or jurisdiction of interest.

3 Decision Analytic Approaches to Robust Analysis

A decision analytic approach (Fig. 2.2) provides a systematic and explicit way to estimate incremental effects, resource use and costs of alternative strategies and points to principles for undertaking robust analysis.

Fig. 2.2

Decision analytic principles – coverage and comparability in capturing costs and outcomes (Eckermann, 2nd April 2014)

Each patient in a target population travelling down care pathways (whether prevention, diagnosis, treatment, rehabilitation or palliative care, etc.) associated with alternative interventions or strategies has a cost and effect associated with that pathway. Principles of comparability and coverage are highlighted in such decision trees. For any given target patient population unbiased estimation of incremental effects, resource use and costs require that their comparable relative impacts are adequately captured along treatment pathways. To support comparability when estimating relative effects, resource use and costs, randomised control trial evidence for compared strategies compared is ideally available to avoid selection biases (both observed and non-observed factors) in allocating patients between arms. However, adequate coverage of the scope and duration of effects and associated costs along treatment pathways is also required to avoid selection biases arising in the coverage of effects, resource use and costs included in incremental cost effectiveness comparison.

Without randomised control trial evidence, the potential arises for systematic biases in relation to non-observed as well as observed factors associated with selection of patients by arm, in estimating net incremental cost and effects. However, not having adequate coverage of the scope and duration of effects and associated costs of treatment also leads to systematic bias in estimating net incremental costs and effects, for example, if the health impacts and cost of treatment associated with side effects are not included or study duration does not capture downstream cost and effect impacts of differences in rates of sequalae. Hence, decision analytic principles underlying health economics highlight the need for adequate and consistent coverage (scope and duration), as well as comparability of evidence, to robustly estimate relative and absolute incremental effects, resource use and costs for defined patient populations across alternate pathways.

For two strategy comparisons robust estimation of incremental costs, effects and their joint consideration, incremental cost effectiveness analysis (or equivalently incremental net benefit analysis as we later see in this chapter) requires:

• (i) Unbiased estimation of treatment effects on health affects resources relative to an appropriate comparator (compatibility)

• (ii) Sufficient length of follow-up and scope of resource use and health effects to capture incremental costs and effects (coverage)

Joint consideration and satisfaction of these coverage and comparability principles is key to preventing biases in cost-effectiveness analysis.

Importantly, the Arrow-Lind theorem (Arrow and Lind 1970) highlights the primary importance of avoiding biased cost effectiveness estimates before considering cost effectiveness uncertainty for societal decision making to be best informed in processes of health technology assessment. The Arrow-Lind theorem establishes that societal risk preferences asymptote towards risk neutrality with risk spreading across large populations and multiple decisions. Hence, minimising bias should predominate over increasing precision as the primary focus of cost effectiveness analysis in health technology assessment processes. Consequently, repeated decision making across large populations informed by bodies such as The National Institute for Health and Care Excellence (NICE) in the UK and the PBAC in Australia should primarily be interested in avoiding biases in estimating expected incremental cost, effect and their joint consideration, cost-effectiveness.

This is highlighted in Fig. 2.3, where unbiased estimation of incremental effects, costs and INB is the primary foundation to robustly informing optimal decision making cycles locally.

Fig. 2.3

Optimal decision making cycles for joint research, reimbursement and regulatory processes locally and globally

Note that this does not mean that uncertainty is not important. Considering uncertainty of INB is the key consideration in creating appropriate incentives for adequate research (Chaps. 5, 6 and 7) and more generally for optimal joint research, reimbursement and pricing decisions in evaluation, policy and practice (Chaps. 8, 9, 10, 11 and 12). However, meaningful consideration of such uncertainty and associated decision making requires unbiased estimation of incremental costs, effects and net benefit, as considered in this chapter for within-trial evidence and in Chap. 3 when translating trial evidence to jurisdictions of interest.

The alternative, modelling uncertainty with biased methods, is akin to looking with rose-coloured glasses at a light that you primarily need to identify the central colour of, because you might be able to see the edge shapes better.

Biased cost effectiveness analysis cannot be justified for reimbursement decisions given an underlying objective and decision context for HTA informed by the Arrow-Lind theorem. Given many reimbursement decisions made across large populations, the Arrow-Lind theorem makes clear the need for unbiased methods to maximise expected net benefit of such decisions. Further, for research decisions, location of the distribution of INB is also fundamental for any jurisdiction(s) to robustly compare the expected value and cost of further research locally (DT vs. AN) and globally (AT vs. AN) allowing for key decision contexts (Chaps. 5, 6 and 7). While rose-coloured glasses might make a shape marginally more discernible at the edges, they end up changing the whole colour (shifting the location of the whole distribution). Satisfying coverage and comparability principles for unbiased cost effectiveness analysis provides the key to robust reimbursement and research decisions and their joint optimisation.

Consequently, the starting points for a building block to consider any such more complex methods are principles and methods for unbiased cost effectiveness analysis. Minimising bias by jointly satisfying comparability and coverage principles for effects and costs along relevant pathways of care is paramount to robust within-trial analysis (this chapter), inform decision making in any jurisdiction of interest (Chap. 3) or any more complex forms of analysis.

Figure 2.4 highlights the decision analytic principles of coverage and compatibility in practice and points towards methods required to enable robust unbiased analysis satisfying these principles (this chapter), but more generally for bodies such as the PBAC in Australia to best inform cost effectiveness decisions for their relevant jurisdiction. In particular, the need to move beyond within-trial-based analysis developed in this chapter to methods and metrics for consistently synthesising and translating trial evidence to inform clinical and health economic policy decisions in any given jurisdiction of interest (Chap. 3).

Fig. 2.4

HTA processes informing decision making in a jurisdiction of interest – e.g. PBAC in Australia

Policy decisions from a community perspective need to consider net clinical benefit of strategies expected in a given patient population trading off expected harms and benefits. For two-strategy comparisons incremental net clinical benefit is in many settings ideally measured with incremental quality adjusted life years (QALYs) allowing for relative mortality, morbidity and side effect impacts over time. However, note that in areas such as palliative care, multiple additional key domains of effect not able to be integrated with survival such as finalising personal and financial affairs in process of death, family and carer distress and carer burden and preference for place of palliative care and death are primary concerns, as highlighted in Chap. 4 and multiple domain methods in Chap. 10. Incremental net benefit (Graham 1981, 1992; Claxton and Possnet 1996; Stinnett and Mullahy 1998) simply extends assessment of absolute incremental effect or net clinical benefit (ΔE) to additionally allow for impacts on resource use and net incremental cost (ΔC). Incremental net monetary benefit (INMB) considers the value of net incremental effects at a threshold value (λ) for effect, less net incremental cost:

$$ \mathrm{INMB}\kern0.5em =\kern0.5em \lambda \varDelta E\kern0.5em -\kern0.5em \varDelta C. $$

Incremental net benefit (INB) can also be expressed in terms of effects as incremental net effect benefit (INEB): INEB  =  ΔE – ΔC/λ.

Nevertheless, for health economics analysis and to avoid issues that arise with INEB where a 0 threshold value for effects is considered, we will stick to INMB in considering INB.

4 Why Use Incremental Net Benefit and Not Incremental Cost Effectiveness Ratios

During the late 1980s and 1990s, the incremental cost effectiveness ratio (ICER) was proposed and became a popular way of summarizing cost effectiveness evidence to inform health technology assessment. The ICER represents the incremental cost (including direct cost and downstream costs associated with effects) divided by incremental effect of a strategy relative to a comparator.

Formally, for intervention i (e.g. the treatment arm of a trial) and comparator c (e.g. control arm of a trial), an estimate of the ICER for intervention i relative to comparator c can be estimated from evidence for mean costs and effects as

$$ {\mathrm{ICER}}_{i, c}=\kern0.5em \frac{{\mathrm{Cost}}_i\kern0.5em -\kern0.5em {\mathrm{Cost}}_c}{{\mathrm{Effect}}_i\kern0.5em -\kern0.5em {\mathrm{Effect}}_c}\kern0.5em =\kern1em \frac{\varDelta {C}_{i, c}}{\varDelta {E}_{i, c}} $$

If the effect per patient were survival, then the ICER becomes incremental mean cost per survivor. If the effect were life years, then the ICER becomes incremental cost per life year. If the effect were QALYs, then the ICER estimate becomes incremental cost per QALY.

Following Willan and Briggs (2006), the ICER can alternatively be written as

$$ {\mathrm{ICER}}_{i, c}\kern0.5em =\kern0.5em NN{T}_{i, c}\kern0.5em \times \kern0.5em \varDelta {\mathrm{Cost}}_{i, c} $$

noting that

$$ NN{T}_{i, c}\kern0.5em =\kern0.5em \frac{1}{\varDelta {E}_{i, c}} $$

That is, the number needed to treat (NNT) to gain one unit of effect, an extra survivor, life year or QALY, is the inverse of change in effect per patient. Hence, it naturally follows that the incremental cost per unit effect is the average incremental cost per patient multiplied by the NNT (expected number of patients required to achieve one incremental unit of effect). Incremental costs, effects and the ICER for an intervention or strategy relative to a comparator are also simply and informatively presented on the incremental cost effectiveness plane (Fig. 2.5).

Fig. 2.5

The incremental cost effectiveness plane

The incremental cost effectiveness plane presents incremental effects and costs of the intervention relative to a fixed comparator at the origin. By convention, incremental effects are presented on the horizontal axis and incremental costs on the vertical axis. These axes divide the incremental cost effectiveness plane into four quadrants which can be described by quadrants as in a compass, as northeast (NE), southeast (SE), southwest (SW) and northwest (NW) quadrants.

If the new therapy has expected positive incremental net clinical effect and lower net cost (allowing for costs associated with effects as well as direct costs of the intervention and comparator strategies) and lies in the SE quadrant (ΔE > 0, ΔC < 0), then the existing strategy is said to dominate the comparator. Conversely, if the new therapy has negative incremental net effect and higher net cost relative to the comparator strategy, and lies in the NW quadrant (ΔE < 0, ΔC > 0), then the existing strategy is said to be dominated by the comparator. Note that in either of these cases there is not a trade-off between incremental cost and effects in distinguishing which intervention is preferred and a threshold value for effects is not required to discriminate what should be the preferred intervention (at least not until uncertainty is considered).

In the NE and SW quadrants, trade-offs between incremental cost and effects arise, and a threshold value for effects is required to distinguish which strategy is preferred. Presenting evidence on the incremental cost effectiveness plane relative to a fixed comparator, the ICER at any point is represented by the slope of a line from the origin. That is, the slope of a line from the origin to any point on the plane represents the ICER or incremental costs divided by incremental effects.

Given the slope of any line through the origin represents the ICER, if one considers the maximum threshold value of the ICER on the NE quadrant for a given jurisdiction at a point in time (and implicitly for given decision contexts, see Chap. 11) as a constant (i.e. not altered by size of budget impacts), then a line from the origin on the NE quadrant with that slope can represent the threshold acceptable ICER. Under this assumption, for two-strategy comparison, a line with slope equivalent to the threshold ICER can distinguish which intervention is preferred in the NE quadrant given evidence of incremental expected costs and effects.

However, note that such analysis is not able to delineate preferred strategies for more than two strategy comparisons, as with multiple strategies there is no longer one fixed comparator (Eckermann et al. 2008; Eckermann and Willan 2011; Eckermann 2004), and requires alternate methods and summary measures as identified in Chap. 8. Further, the direction of budget impacts, additional cost (NE quadrant) or cost reduction (SW quadrant) is also shown to alter the subjective nature of opportunity cost (alternative adoption and financing vs. funding generation) and appropriate threshold values in the SW and NE quadrant (Eckermann 2015), as considered at length in Chap. 11.

Of more obvious and immediate importance, problems arise with ICER metrics when change in effect is 0 or crosses the horizontal axis across 0 effect. When change in effect is 0, the ICER directly or as NNT (inverse of incremental effect) multiplied by incremental cost per patient is undefined. This is the first of a series of problems with the ICER, which in general is not well ordered. As Willan and Briggs (2006) highlight, the ICER has:

• (i) A discontinuity when ΔE changes sign. For example, with positive incremental cost, an ICER changes from approaching infinity when change in effect is small and positive in the NE quadrant to approaching negative infinity when change in effects is small and negative, in crossing to the NW quadrant.

• (ii) The same negative sign in the NW and SE quadrants, but diametrically opposite implications with an intervention or strategy dominating the comparator (having higher effect and lower cost) in the SE quadrant while being dominated by the comparator (having lower effect and higher cost) in the NW quadrant.

• (iii) The same value in moving along any given ray from origin, while in the SE and NW quadrants, respectively, representing increasing domination of (SE) and domination by (NW) the comparator strategy.

The ICER as a result of (i) and (ii) requires separate statements and consideration of which strategy is preferred when effects are positive or negative, while (iii) implies that even within such separate statements, ICER ordering makes no sense where the ICER is negative. These ordering problems make the ICER highly problematic as a summary measure of cost effectiveness in interpreting or comparing point estimates, let alone under uncertainty. Additional knowledge of which quadrant incremental cost and effect estimates are in is required to allow any meaningful interpretation for decision making. Further, these ordering problems mean the ICER usually becomes untenable as a summary measure once cost effectiveness uncertainty is considered.

Hence, in general the ICER as a ratio measure does not have good statistical properties, where any evidence lies outside the NE quadrant.

The inherent and largely intractable ordering problems of the ICER as a ratio measure in attempting to inform cost effectiveness decision making are, however, simply circumvented by use of incremental net benefit metrics. Incremental net monetary benefit (INMB) as the value of incremental effects (λΔE) less incremental costs (ΔC), INMB = λΔE − ΔC, provides a continuous metric that does not face the decision ordering problems of the ICER as a ratio, while representing the same decision rule. That is, INMB being greater than 0 represents the same decision rule as the ICER being acceptable relative to a decision threshold value for effects for two strategy comparisons.

Formally, the cost effectiveness decision rule of

$$ \varDelta C/\varDelta E\kern0.5em <\lambda, \kern1em \varDelta E > 0, $$

where λ is the threshold value per unit effect, or the less often considered

$$ \varDelta C/\varDelta E\kern0.5em >\kern0.5em \lambda \kern0.5em for\ \varDelta E\kern0.5em <\kern0.5em 0 $$

can both be rewritten as

$$ \mathrm{INMB}\kern0.5em =\kern0.5em \lambda \varDelta E\kern0.5em -\kern0.5em \varDelta C\kern0.5em >\kern0.5em 0 $$

As a linear combination of ΔE and ΔC, INMB is continuous with regard to both and has linear properties in relation to their mean and variance. INMB also does not require separate consideration of whether incremental effect is positive or negative while representing the same decision rule with respect to cost effectiveness. Together these advantages of INMB overcome the statistical and interpretability problems of the ICER.

In contrast to the ICER, the INMB statistic is well ordered within and across quadrants, and its direction and extent reflect appropriate decision making, with INMB:

• (i) Continuous when effect changes from being positive to negative or vice versa (the sign of ΔE changes around 0).

• (ii) Unambiguously negative in the NW quadrant where the comparator dominates the new intervention and positive in the SE quadrant where the new treatment dominates.

• (iii) Increasingly negative and positive, as appropriate in the NW and SE quadrants respectively, as one moves along a ray away from the origin. That is, INMB reflects increasingly being dominated or dominating in moving away from the origin along a ray in the NW and SE quadrants.

Hence, the direction and extent of gain or loss expected with decision making are reflected in INMB. Further, when we compare to multiple strategies in Chap. 8, INMB unlike the ICER has the property of being additively separable (Stinnett and Paltiel 1997). This implies that with comparison of multiple strategies, INMB ordering across strategies at a given threshold value does not change with choice of comparator, while such ordering can easily change with choice of comparator with the ICER.

5 Illustrating Principles Within Study: The LIPID Trial Case Study

The LIPID study represents a double blinded placebo-controlled randomised trial comparing pravastatin incremental to standard care undertaken in 9014 Australian patients with prior myocardial infarction (MI) or unstable angina pectoris (UAP). The health economic analysis undertaken on behalf of the LIPID study group (Glasziou et al. 2002; Eckermann and Kirby 2003) was motivated by concern about the long-term cost-effectiveness of statin use in Australia for these populations.

The LIPID trial design (Fig. 2.6) satisfies the key principles of coverage as well as comparability required for unbiased health economic analysis. Comparability is satisfied by the randomised double blinded nature of the placebo-controlled study. Coverage is addressed both in terms of duration of outcomes over the median 6-year follow-up and in terms of scope of outcomes with evaluation of mortality, hospital and medication use across all 9014 patients and sub-studies of ambulatory care use, medication dose and quality of life impacts on utility measures in more than 1100 patients.

Fig. 2.6

LIPID cost effectiveness study design

LIPID study results are summarized for all-cause mortality by arm (pravastatin vs. placebo) over the trial follow-up, the primary within-study effect in Fig. 2.7.

Fig. 2.7

Lipid study all-cause mortality over study follow-up for pravastatin versus placebo (Source: Eckermann and Kirby (2003) on Behalf of the LIPID Study Investigators)

Pravastatin reduced all-cause mortality by 3.01% in absolute terms over a 6-year follow-up, which reflects a 22% relative risk reduction on a baseline risk of 14.1% in the placebo-controlled arm (Table 2.1).

Table 2.1 LIPID within-study incremental cost per life saved

A mean cost of pravastatin of $4913 per patient over a 6-year follow-up was somewhat offset by reduced hospitalisation and other medication costs, leading to an incremental cost of $3246 per patient. Given this mean incremental cost and reduction in all-cause mortality rate, the incremental cost per additional survivor is estimated as $107,730 ($3246/0.0301) and presented on the incremental cost effectiveness plane as the slope of line from the origin (comparator) to the incremental effect and cost (ΔE, ΔC) point estimate (Fig. 2.8).

Fig. 2.8

LIPID evidence on the incremental cost effectiveness plane

This point represents the within-study estimate for incremental costs and effects, and their joint consideration in relation to cost effectiveness is reflected in the ICER estimate, meaningful here noting that it lies on the NE quadrant. The trial population and practice in the LIPID control arm also represented secondary prevention of CHD in Australia at the time of analysis. Hence, for societal decision making in Australia, this also represented the expected incremental costs, effects and their joint consideration in the Australian population at the time analysis was undertaken. More generally, as Chap. 3 highlights, robust estimation of absolute incremental cost and effect requires evidence translation to reflect the baseline risk of the population in practice for the jurisdiction of interest where the decision is being made.

6 Representing Cost Effectiveness Uncertainty

To allow for uncertainty around incremental cost effectiveness ratios, a box method was initially proposed in health economics literature (O’Brien et al. 1994; Wakker and Klaassen 1995). The ‘box method’ literally drew a box around the point estimate with the boxes corner points representing the various lower and upper 95% confidence interval (CI) for costs and effects (see Fig. 2.9 for the case of LIPID).

Fig. 2.9

The ‘box method’ with LIPID evidence (Source: Adapted from Eckermann and Kirby 2003)

The box method proposed that the 95% confidence interval around the point estimate for the ICER, for example, $107,730 per additional survivor in the case of the LIPID study, could be estimated from the ICER (slope) of lines from the origin to corners of the box representing:

• (i) The lower 95%CI for costs and upper 95% CI for effects

• (ii) The upper 95% CI for costs and lower 95% CI for effects

Hence, for the LIPID study, the box method would estimate the lower and upper 95% CIs around the point estimate of $107, 730 for the ICER as ranging from about $60,000 per life saved ($2637/0.0439) up to $235,000 per life saved ($3854/0.0164).

In their seminal paper ‘Thinking outside the box’, Briggs et al. (2002) show distinct problems arising with the box method approach in estimating such uncertainty around the ICER. They note the box method implicitly assumes that the upper and lower CI for cost and effects will occur together and contain 95% of the joint cost and effect distribution. In doing so, the box method fails to allow for the bivariate nature of the relationship (covariance) between incremental costs and effects in estimating their joint distribution. Hence, the box method effectively treats separately, or partialises, costs and effects and their distributions. This fails to appropriately reflect the joint nature of how costs and effects arise along treatment pathways and hence the joint distribution of incremental cost and effects under uncertainty.

In reality even if there were no covariance between incremental cost and effects, the box methods’ extreme 95% CI highest cost and lowest effect and lowest cost and highest effect points would not be expected to arise together or the box shape around this includes 95% of the distribution. As Briggs et al. (2002) show if there were no covariance between incremental cost and effects, then a distribution radially radiating out from the point estimate is expected. Hence, if there were no covariance between incremental cost and effects (covariance = 0), then the joint distribution of costs and effects would result in a radial shape with narrower band for ICER 95% CI than the box methods in Fig. 2.9 suggest. Rather it would reflect a narrower radial distribution such as that in Fig. 2.10.

Fig. 2.10

Box method versus bivariate distribution with no covariance

However, this does not imply the box method is necessarily conservative, as more generally the joint distribution of costs and effects is elliptical with the orientation and shape of the joint distribution determined by the sign and extent of covariance between incremental cost and effect.

Hence, while ICER uncertainty with the box method will be overestimated if there is no or a positive covariance between incremental costs and incremental effects, ICER uncertainty can be easily underestimated where there is a negative relationship (covariance) between incremental costs and effects. A negative relationship between incremental cost and effects (framed from a utility-bearing perspective on the CE plane, e.g. survival) causes radial joint distributions on the incremental CE plane, such as that in Fig. 2.10, to elliptically flatten out and orientate with a SE direction. Negative covariance relationships consequently increase cost effectiveness (INB or ICER) uncertainty, widening 95% CIs for the ICER (or INB). Negative covariance between incremental cost and effects in practice typically reflects where incremental effects mainly relate to morbidity, given reducing morbidity (increasing effect) reduces downstream treatment costs while conversely increasing morbidity (reducing effect) increases downstream treatment costs.

However, the box method can significantly overestimate cost effectiveness uncertainty (NB or ICER 95% CIs) if there is a positive relationship between incremental costs and effects. For example, where net effects mainly relate to survival, given increased survival is expected to increase incremental downstream treatment costs of survivors, or equivalently reducing survival is expected to reduce downstream treatment costs of survivors. Such positive relationships between incremental costs and effects cause the distribution in Fig. 2.10 to flatten out and orientate with an NE-positive slope, narrowing cost effectiveness uncertainty from that with no covariance.

In summary, problems of the box method in estimating 95% CI for cost effectiveness arise in inappropriately combining partially determined separate cost and effect inference in attempting to inform cost effectiveness inference. Consequently, the box method does not appropriately allow for the linked relationship (covariance) between costs and effects along treatment pathways and the impact this has on the joint cost and effect distribution on the CE plane or cost effectiveness uncertainty.

Importantly, Briggs et al. (2002) in addressing problems of the box method identify and illustrate how these partial problems can be overcome with methods that jointly consider costs and effects – think outside the box. That is, with robust estimation methods allowing for the joint relationship and covariance of the bivariate distribution between costs and effects, either non-parametrically with bootstrapping or parametrically using Feiller’s method.

Both bootstrapping and Fieller’s methods enable incremental costs and effects and their joint distribution to be jointly considered allowing for their joint relationship along alternate treatment pathways (covariance). We first consider non-parametric bootstrapping and then turn our attention to Fieller’s method.

7 Bootstrapping the CE Distribution

Bootstrapping is simply repeated resampling with replacement, a non-parametric method which can be used to build up a sampling distribution for joint incremental costs and effects and uncertainty around point estimates for related cost effectiveness summary measures (Briggs et al. 2002). In the case of a trial with Nt patients in the treatment arm and Nc patients in the control arm bootstrapping, the bivariate CE distribution can be summarized as a four-stage process where joint cost and effect patient level data are:

• (i) Randomly resampled with replacement for Nc patients and their associated cost and effects from the control group: calculate mean cost and effects for this control group resample.

• (ii) Randomly resampled with replacement for Nt patients and their associated cost and effects from the treatment group: calculate mean costs and effects in the treatment group.

• (iii) Form a replicate from (i) and (ii) where calculate mean incremental effects and cost for treatment relative to control (∆E, ∆C).

• (iv) Repeat many times (1000 or more) to build up a bootstrapped sampling distribution around the point estimate.

In undertaking these four steps if the seed for random number generation is recorded this allow such resampled bootstrapping of the ICER distribution to be repeatable in various software packages. Importantly, whatever package is used, there should be an equal chance of resampling any individual in any draw when bootstrapping patients with random resampling with replacement. In practice if there are Nc patients (e.g. 200) in the control arm, then a random number between 0 and 1 generated by Rand(), for example, would require random patient assignment using formulae of the general form

round(rand() × Nc + 0.5); Nc + 0.5 = nc.

That is, if there were 200 patients: round(rand() × 200 + 0.5); 200.5 = 200.

This allows an equal chance for each patient to be resampled with any random number, choosing patient 1 for random values from 0 up to 1/200 (0.005), patient 2 from 1/200 (0.005) up to 2/200 (0.01), etc., and patient 200 with values from 199/200 (0.995) up to 1.

A bootstrapped sampling distribution around the point estimate for incremental cost and survival in the LIPID study is shown in Fig. 2.11 for 1000 replicates.

Fig. 2.11

LIPID – bootstrapped distributions in bivariate space and 95% CI for cost per life saved (Source: Adapted from Eckermann and Kirby 2003)

When bootstrapping the bivariate CE distribution, covariance between cost and effects is implicitly maintained as resampling patients retains the relationship between costs and effects for each patient. For two-strategy comparisons considered in this chapter where the comparator is fixed, bootstrapping such distributions allows simple unbiased estimation of the bivariate distribution and summary measures such as the probability of being cost effective (having positive net benefit). The probability of being cost effective at any given threshold value can be simply calculated as the proportion of the distribution with positive INB or equivalently in the acceptance region below (south east of) a threshold line through the origin whose slope reflects the threshold value.

For example, in the case of LIPID 2.5% (25/1000) of the distribution lies at or below $68,626 per life saved and 97.5% at or below $209,881 per life saved (or equivalently 2.5% above). Hence, a 95% CI for the ICER distribution is estimated from the bootstrapped distribution as between $68,626 per life saved and $209,881 per life saved.

Note, however, that while bootstrapping is simple to understand and useful for within-trial and illustrative purposes in establishing the need to jointly consider cost and effects, it does face at least one potentially significant drawback. The method is not exact, with estimates varying depending upon resamples in building up a sampling distribution and subsequent estimating uncertainty. This has led to bodies such as the PBAC being suspicious of such methods when applied and presented by groups with vested interests such as manufacturers. This is particularly the case where bootstrap estimates are presented as a black box without associated replicates or the proportion of times each individual is chosen across replicates.

However, there is a parametric method, Fieller’s method, which addresses this concern, providing an exact closed from solution, under the central limit theorem (CLT) assumption of normality, for INMB, as a bivariate distribution.

8 Fieller’s Method

Fieller’s method fits a bivariate normal distribution to INMB from summary measures for mean incremental cost and effect, their variances and covariance. That is, Fieller’s method uses the fact that INMB is linear in ΔE, ΔC and λ and hence the mean and variance of INMB depends only on the mean and variance of ΔE, ΔC and their covariance. Formally

$$ \mathrm{INMB}\kern0.5em =\kern0.5em \lambda \varDelta E\kern0.5em -\kern0.5em \varDelta C $$

has a variance of

$$ {\lambda}^2\mathrm{var}\left(\varDelta E\right)+\mathrm{var}\left(\varDelta C\right)\kern0.5em -\kern0.5em 2\lambda cov\left(\varDelta E,\kern0.5em \varDelta C\right) $$

Hence, dividing the INB statistic at the threshold value through by its standard error (square root of the variance) results in a standard normal distribution. This can then be used to find the upper and lower 95% CI, but also the bivariate distribution more generally, dependant only on incremental costs and effects, their variance and covariance, following Willan and Briggs (2006). Further, Nixon et al. (2010) show that such CLT parametric methods outperform bootstrapping with small samples, as the asymptotic properties of the CLT kick in at smaller trial sample sizes than with bootstrapping.

In the case of LIPID, Fieller’s method is simply applied with the cost and effect estimates ($3246 and 0.03013 increased survival over the 6-year median study follow-up) and their variance ($100,651 and 0.0000487) and covariance −0.209. This results in a 95% CI for cost per life saved from $68,732 to $204,889. However, the 95% CI for the ICER is a very crude summary of uncertainty of the CE distribution. In simply representing two extreme points on the CE distribution, the ICER 95% CI:

• (i) Fails to capture implications across potential sets of decision maker threshold values;

• (ii) Is highly reductionist in picking two arbitrary points, the 2.5% and 97.5% point on the ICER distribution; and

• (iii) Lacks interpretability where either of these extreme points on the ICER distribution lies outside the NE quadrant.

Summary measures which inform societal decision makers of the whole distribution and across the range of potential decision making threshold values are more useful than throwing away evidence from all but two arbitrarily picked extreme points on the ICER distribution in informing cost effectiveness related decisions.

9 Useful Cost Effectiveness Summary Measures from Bivariate Distributions Conditioning on Threshold Values for Effect

Conditioning on threshold values per unit of effect, more useful and interpretable summary measures can be found across the full distribution at any threshold value. For two-strategy comparisons where there is only one comparator and one distribution to summarise on the CE plane, useful C-E summary measures informing decision makers across plausible threshold values include:

• (i) The cost effectiveness acceptability curve – the probability that a treatment is cost effective (has highest net benefit) across plausible threshold values for a unit of effect; and

• (ii) The incremental net benefit (INB) curve and 95% CI curves – the incremental net benefit expected and 95% CI for INMB across plausible threshold values for a unit of effect.

Figure 2.12 shows the LIPID CEA curve for the probability of pravastatin being cost effective in Australia at threshold values from A$0 to A$260,000 per life saved.

Fig. 2.12

LIPID cost effectiveness acceptability curve ($/life saved) – Fieller’s method

Similarly, an expected incremental net benefit curve conditional on potential threshold values and curves representing 95% confidence intervals around this expected NB line can also be presented. Figure 2.13 shows such INB curves for pravastatin relative to placebo from the LIPID study conditional on the same range of potential threshold values per life saved as Fig. 2.12.

Fig. 2.13

LIPID net benefit curve with 95% CI – Fieller’s method

Where threshold values for effect are 0, INMB (INMB = λΔE − ΔC) simplifies to − ∆C. Hence, the expected value and 95% CI for INMB on the vertical axis in Fig. 2.13 with a 0 threshold effect value is simply negative incremental cost. In the case of LIPID, the point estimate for INMB at a 0 threshold value as shown in Fig. 2.13 is −$3246, with 95% CI from −$2637 to −$3854. More generally, INMB depends on the threshold value for incremental effects as well as incremental cost, with expected INMB changing with λ at a rate of ∆E per unit of the threshold value. That is, the slope of the INB line as a function of λ is ∆E. In the case of LIPID, the expected INB line has a slope of 0.03013 (reflecting the absolute 3.013% mortality reduction), increasing at a rate of $30.13 (=0.03013 × 1000) for every $1000 increase in the value of a life saved. On the horizontal axis, expected INMB is 0 at the threshold value where λΔE − ΔC = 0, and hence at a threshold value where λ = ΔE/ΔC, the ICER estimate. In the case of LIPID, the expected INB curve crosses the horizontal axis at $107,730 per life saved.

In general, the expected incremental net monetary benefit line INMB = λΔE − ΔC passes through points of intersection on the vertical axis at INMB = − ΔC and horizontal axis (λ value with INMB = 0) at the expected ICER (λ = ∆C/∆E when INB = 0) and have slope ∆E. Hence, INMB lines will be upward sloping as a function of λ where there is a positive expected treatment effect and downward sloping where there is a negative treatment effect. Expected INMB lines start with an implicit threshold value for valued of 0 on the vertical axis. Hence, INMB on the vertical axis simplifies to minus incremental cost, with negative INMB where the strategy has net additional costs, while starting with positive INMB if the strategy is cost saving. This in general leads to four types of expected INMB line (Willian and Briggs 2006) reflecting different potential combinations of cost and effects on the four quadrants of the CE plane:

• (i) INMB lines which start negative and become positive corresponding to positive incremental cost and effects (NE quadrant on CE plane).

• (ii) INMB lines which start positive and increase corresponding to negative incremental cost and positive effect (SE quadrant on CE plane), and indicate a new therapy dominates existing care.

• (iii) INMB lines which start positive and become negative corresponding to negative incremental cost and negative effects (SW quadrant on CE plane).

• (iv) INMB lines which start negative and decrease corresponding to positive cost and negative effect (NW quadrant on CE plane), and indicate the comparator (e.g. existing care) dominates then new therapy.

Similarly, the lower and upper 95% CI curves for INMB start on the vertical axis at minus the 95% CIs for incremental cost. The lower and upper 95% CI curves for INMB will cross the horizontal axis at the lower and upper 95% CI for the ICER unless they don’t arise – the new strategy dominates or is dominated at these points. Where new strategies are expected to dominate, expected INMB is positive for all feasible positive threshold values for a positive effect, while where new strategies are expected to be dominated, expected INMB is negative for all plausible threshold values. These curves do not cross the horizontal axis (have INMB=0) over feasible ranges for threshold values.

10 How Should Economically Meaningful Threshold Values for Effects Be Estimated?

Historically, processes of health technology assessment in developed countries such as the PBAC in Australia or NICE in the UK have focused on the NE quadrant with requirements for new technology to demonstrate incremental effectiveness to justify a price premium over comparators, which profit-motivated manufacturers’ try to maximise. From a societal decision maker perspective on the NE quadrant if the estimated combination of incremental costs and effects lies to the SE of (below) the threshold line, then the value of incremental effects is greater than incremental costs, and the new strategy should be preferred. Conversely, if the estimated combination of incremental costs and effects on the NE quadrant lies above (to the NW of) such a threshold line, then the value of incremental effects is less than incremental cost of the new intervention and the comparator strategy should be preferred.

However, one should note that in presentations such as Fig. 2.5, the threshold line and its use in distinguishing what should be preferred assumes a decision-making threshold value which appropriately reflects opportunity cost of reimbursing the new technology. In general, appropriately determined threshold values should reflect opportunity costs in a jurisdiction of interest under local health system conditions, to enable optimisation given investment options and budget (resource) constraints. If there was a pool of new money available to spend on health care then the opportunity cost of spending that new budget on adopting a new technology is the best alternative adoption action, the most cost effective expansion of exisiting programs and technology. However, more generally budgets are fixed and hence reimbursing a new technology requires both adoption and financing actions (Pekarsky 2012, 2015). Given an underlying objective of maximising net benefit from a fixed budget, the decision maker threshold values per unit effect should reflect best alternative adoption and financing actions in relation to this goal (Eckermann and Pekarsky 2014). In reimbursing new technology with net incremental cost, the best alternative actions are most cost effective alternative expansion of current programs or technology funded by contraction of the least cost effective current program or technology, reflected in the health shadow price of Pekarsky (2012, 2015).

The health shadow price is derived by Pekarsky as

$$ {\beta}_{\mathrm{c}}\kern0.5em =\kern0.5em {\left(\frac{1}{n}\kern0.5em +\kern0.5em \frac{1}{d}\kern0.5em -\kern0.5em \frac{1}{m}\right)}^{-1} $$

where n is the ICER of the most cost effective expansion of current programs, m is the ICER of the least cost effective current program in contraction, and d is the ICER of services displaced.

This derivation arises from finding the threshold cost per unit effect (threshold ICER = β _c) for a new strategy or technology with net cost of investment (I) at which returns from adopting the new strategy or technology financed with displacement of services (ICER = d) given a fixed budget equate with the opportunity cost, that of best alternative actions. The best alternative actions are the most cost effective expansion of current programs (ICER = n) and technology financed by contraction of the least cost effective current programs or technologies (ICER = m). Hence, β _c is solved from equating investment returns as

$$ \frac{I}{\beta_{\mathrm{c}}}\kern0.5em -\kern0.5em \frac{I}{d}\kern0.5em =\kern0.5em \frac{I}{n}\kern0.5em -\kern0.5em \frac{I}{m} $$

which dividing through by I and rearranging simplifies to

$$ \frac{1}{\beta_{\mathrm{c}}}\kern0.5em =\kern0.5em \frac{1}{n}\kern0.5em +\kern0.5em \frac{1}{d}\kern0.5em -\kern0.5em \frac{1}{m} $$

and hence

$$ {\beta}_{\mathrm{c}}\kern0.5em =\kern0.5em {\left(\frac{1}{n}\kern0.5em +\kern0.5em \frac{1}{d}\kern0.5em -\kern0.5em \frac{1}{m}\right)}^{-1} $$

This represents the true opportunity cost (Pekarsky 2012, 2015) of reimbursing new strategies or technologies where they have a net incremental cost.

Importantly the health shadow price in comparing with best alternative adoption and financing actions encourages optimal displacement as well as optimal investment actions (Pekarsky 2012, 2015; Eckermann and Pekarsky 2014). If displacement is optimal (the least cost effective current program or technology is displaced, d = m), then the health shadow price equates to the most cost effective expansion of current programs and technology, with ICER n. That is, if d = m, then

$$ {\beta}_{\mathrm{c}}\kern0.5em =\kern0.5em {\left(\frac{1}{n}\kern0.5em +\kern0.5em \frac{1}{d}\kern0.5em -\kern0.5em \frac{1}{m}\right)}^{-1}\kern0.5em =\kern0.5em {\left(\frac{1}{n}\kern0.5em +\kern0.5em \frac{1}{m}\kern0.5em -\kern0.5em \frac{1}{m}\right)}^{-1}\kern0.5em =\kern0.5em n $$

However, where displacement is not optimal (d<m), then

$$ \frac{1}{d}\kern0.5em >\kern0.5em \frac{1}{m} $$

$$ \frac{1}{n}\kern0.5em +\kern0.5em \frac{1}{d}\kern0.5em -\kern0.5em \frac{1}{m}\kern0.5em >\kern0.5em \frac{1}{n} $$

$$ {\left(\frac{1}{n}\kern0.5em +\kern0.5em \frac{1}{d}\kern0.5em -\kern0.5em \frac{1}{m}\right)}^{-1}\kern0.5em <\kern0.5em n $$

$$ {\beta}_{\mathrm{c}}\kern0.5em =\kern0.5em {\left(\frac{1}{n}\kern0.5em +\kern0.5em \frac{1}{d}\kern0.5em -\kern0.5em \frac{1}{m}\right)}^{-1}\kern0.5em <\kern0.5em n $$

These results are key to appropriately interpreting the appropriate threshold value for effects in net benefit – that which reflects opportunity cost and allows budget-constrained optimisation. As Chap. 11 highlights, this should be the case whether net benefit for decision making relates to new technology reimbursement, research decisions or regulatory and policy making assessment in practice. Until Chap. 11, as with other health economic practitioners, we will condition analysis across potential decision making threshold values for effects in analysing and summarising cost effectiveness evidence.

Summary measures conditional on plausible threshold values for effects include:

• (i) Cost effectiveness acceptability and net benefit curves for two strategies comparisons with one effect introduced in this chapter;

• (ii) Expected net gain in optimising the expected value relative to cost of research designs and associated joint research and reimbursement decision locally (Chap. 5) and globally (Chaps. 6 and 7);

• (iii) Expected net loss curves and frontiers for multiple strategies (Chap. 8);

• (iv) Net benefit efficiency measures (Chap. 9); and

• (v) Expected net loss planes and contours for multiple outcomes (Chap. 10).

The implications of the health shadow price in expansion for net benefit maximisation and budget-constrained optimisation are considered at length in Chap. 11. The health shadow price in contraction is also considered following Eckermann (2015), with empirical estimates of the health shadow price of expansion and contraction in the UK presented based on program budgeting marginal analysis (PBMA) evidence. Alternative threshold values for health which have previously been proposed for comparing and pricing new technology with higher net costs against – willingness to pay or the ICER of displaced programs (d) are also critiqued in Chap. 11 following Eckermann and Pekarsky (2014). Unlike the health shadow price, these alternatives are shown to not reflect opportunity cost of best alternative actions or allow a pathway to budget-constrained optimisation from current allocative (n<m) or displacement (d<m) inefficiency.

Critically, the health shadow price (Pekarsky 2012, 2015; Eckermann and Pekarsky 2014) points to the need for research in relation to expansion and contraction of current technology and programs, that is, research to identify the most cost effective expansion of current technology and programs and where to contract or displace the least cost effective current programs and technology. Hence, the health shadow price is shown in Chap. 11 to establish the threshold values societal decision makers should be using in creating a pathway to optimisation across research, reimbursement and regulatory decisions (pricing and provider performance in practice).

11 Conclusion

Satisfying decision analytic principles of coverage, comparability and consistency are keys to obtaining unbiased estimates for relative comparison of absolute costs and effects and cost effectiveness analysis.

These represent the primary considerations to best inform decision making of bodies such as NICE and the PBAC about cost effectiveness in process of health technology assessment. To meaningfully model cost effectiveness uncertainty or summary measures such as CEA and NB curves requires costs, and effect uncertainty is also jointly considered (Briggs et al. 2002). However, first and foremost, these distributions need to be based around unbiased estimates of absolute incremental costs and effects, where coverage and comparability principles are jointly satisfied. Satisfying comparability, coverage and consistency principles in estimating joint costs and effects along alternative pathways prior to consideration of decision uncertainty lays the foundation stone for robust, unbiased cost effectiveness analysis. These principles are central to allowing robust analysis for within-study RCT evidence comparing two strategies illustrated in this chapter, but also any more complex forms of analysis. Coverage, comparability and consistency principles are also central to robust analysis throughout the text in:

• (i) Synthesizing, translating and extrapolating evidence (Chap. 3);

• (ii) Evaluating health promotion and prevention strategies (Chap. 4);

• (iii) Informing and optimizing joint research and reimbursement decisions locally and globally (Chaps. 5, 6 and 7);

• (iv) Multiple strategy and multiple outcome comparisons (Chaps. 8 and 10);

• (v) Evaluating efficiency in performance of health care providers, and health funding systems, in practice allowing for quality of care consistent with maximizing net benefit (Chap. 9 and Sect. 12.5), where the net benefit correspondence theorem underlying these methods makes explicit the need to satisfy coverage and comparability conditions to enable robust analysis and create appropriate incentives in practice (Eckermann 2004; Eckermann and Coelli 2013);

• (vi) Establishing economically meaningful opportunity costs and threshold values for effects in jurisdictions of interest given relevant decision contexts for health system allocative and displacement inefficiency (Chap. 11), following Pekarsky (2012, 2015) and Eckermann and Pekarsky (2014); and

• (vii) Policy analysis (Chap. 12).

12 Discussion – Satisfying Coverage, the Need for Robust Evidence Synthesis, Translation and Extrapolation

The LIPID study satisfies comparability and coverage principles in providing RCT evidence with adequate scope (mortality and quality of life) and duration of coverage (6-year median follow-up) for a trial-based analysis. This also doubled as an Australian analysis given study patients and their treatment were representative of practice at time of decision making. The inclusiveness of patient in the LIPID study supports trial analysis providing a robust estimate of baseline risk expected in practice in secondary prevention populations in Australian decisions related to adopting statin therapy, as well as relative treatment effect. However, synthesis, translation and potentially extrapolation of trial evidence are more generally needed to allow valid estimation in a jurisdiction of absolute incremental effects, cost and INB. Differences in population INB in practice in a jurisdiction of interest can differ from that in a trial where trial inclusion and exclusion criteria as well as geography and associated populations, practice, prices and preferences differ.

In this chapter, we have considered the simplest case of within-trial two-strategy comparisons, as often reported in CE literature alongside trials. Such estimates can provide meaningful analysis of expected effects, costs and cost effectiveness for decision making in the jurisdiction where the trial is undertaken, provided the comparator arm reflects usual practice and the trial population is the same as that expected in practice.

Robust, unbiased methods for trial evidence translation to reflect the baseline risk expected in practice in any given jurisdiction of interest are established in Chap. 3 (Eckermann et al. 2011).

Chapter 3 more generally makes clear that avoiding bias requires consistent methods for evidence synthesis, translation and extrapolation as well as coverage of the scope and duration of incremental cost and effects. In general, both coverage and comparability need to be satisfied to enable unbiased estimates of INB for any given jurisdiction, a precursor to any meaningful consideration of cost effectiveness (INB) decision uncertainty. Hence, principles of coverage and comparability form the basis for robust cost effectiveness analysis whether analysis is purely based on a RCT or is undertaken with model-based analysis. Bernie O’Brien’s seminal paper ‘Frankenstein’s Monster or the Vampire of Trials’ (O’Brien 1996) takes centre stage in Chap. 3 establishing the need to jointly satisfy coverage and comparability principles with model and trial-based analysis.