FormalPara Key Points for Decision Makers

While etelcalcetide appears to be more effective than standard therapy with phosphate binders and vitamin D (PBVD) in terms of short-term biochemical outcomes (e.g. parathyroid hormone), the impact on longer-term clinical outcomes such as cardiovascular events is highly uncertain.

Methods of synthesising clinical trial efficacy data used in the company submission (CS) compromised the randomised nature of the treatment comparisons, favouring etelcalcetide.

None of the methods used in the CS to tie short-term biomarker data to long-term outcomes were validated, and all should be considered highly uncertain.

While incremental cost-effectiveness ratios for etelcalcetide versus cinacalcet were under £30,000 per quality-adjusted life-year, the National Institute for Health and Care Excellence (NICE) Appraisal Committee decided that due to uncertainty in long-term efficacy, etelcalcetide should only be recommended in adults with chronic kidney disease and secondary hyperparathyroidism who are refractory to standard therapy (PBVD) and for whom cinacalcet is not appropriate.

1 Introduction

The National Institute for Health and Care Excellence (NICE) is a non-departmental public body that produces, among other things, health care guidance for the English National Health Service (NHS). NICE assesses the clinical and cost effectiveness of health technologies (medicines, diagnostic tests and medical devices) through its technology appraisal programme, in order to make recommendations about the use of technologies in the NHS.

NICE’s Single Technology Appraisal (STA) programme is designed to evaluate a single health technology for a single indication, as near to market launch as possible. The clinical- and cost-effectiveness information required by NICE advisory committees for STAs is supplied by the company or sponsor of the technology and critiqued by an independent research assessment team, the Evidence Review Group (ERG).

A range of stakeholders, including the company, the ERG, expert clinical representatives, patients, and patient group representatives, provide evidence for the appraisal. The NICE Appraisal Committee evaluates the evidence provided by stakeholders in order to reach conclusions on the clinical and cost effectiveness of the new technology, and make recommendations about its use in the NHS.

This article presents a summary of the ERG’s review and critique of the company submission (CS) to NICE, additional work conducted by the ERG, and the key issues that arose during the committee decision-making processes for the technology appraisal of etelcalcetide for secondary hyperparathyroidism (SHPT) in patients with chronic kidney disease (CKD). We summarise the evidence submitted by the company and then present our critique of this evidence. Full details of the appraisal documents, including the company’s evidence submission, the ERG report, and Appraisal Committee decision documents, are available on the NICE website [1].

2 The Decision Problem

Patients on haemodialysis may develop SHPT, a potentially serious complication of CKD that is characterised by elevated production and serum levels of parathyroid hormone (PTH), disturbances in calcium and phosphate levels, and abnormal bone and mineral metabolism [2]. Elevated PTH contributes to increased risk of bone fractures, cardiovascular disease due to calcification of blood vessels, and death.

According to the Dialysis Outcomes and Practice Patterns Study, 41.4% of the dialysis population in the UK in 2011 had SHPT [3]. Similarly, a systematic review reported a prevalence of 42.9%, which translates to approximately 9000 of 21,000 UK dialysis patients if applied to the current UK Renal Registry data [4, 5].

The aim of treatment of SHPT in patients with CKD receiving haemodialysis is to manage phosphate, calcium and PTH levels, to bring them within the normal ranges for dialysis patients. Phosphate binders and vitamin D (PBVD) analogues are used to try to normalise calcium and phosphate levels. Dietary modification is also used to reduce phosphate intake. In some patients, the oral calcimimetic drug cinacalcet may be used in addition to PBVD and dietary modification.

NICE published guidance on the use of cinacalcet in 2007—NICE TA117 [6]. In that STA, treatment with cinacalcet and PBVD was compared against standard care consisting of PBVD alone. Cinacalcet was only recommended for use in patients with very high levels of PTH in their blood that is refractory to standard treatment, and who are contraindicated to parathyroidectomy. The 2009 international Kidney Disease: Improving Global Outcomes (KDIGO) guideline for the diagnosis, evaluation, prevention and treatment of mineral and bone disorders in CKD, provides guidance on how SHPT should be managed [7].

Etelcalcetide (Parsabiv™, Amgen Europe B.V.) is an intravenous calcimimetic that binds to and activates calcium-sensing receptors to reduce production of PTH, which in turn reduces serum calcium and phosphate levels. In November 2016, etelcalcetide received marketing authorisation from the European Medicines Agency for the treatment of SHPT in adult patients with CKD on haemodialysis.

The scope of the appraisal issued by NICE was to appraise the clinical and cost effectiveness of etelcalcetide within its marketing authorisation for the treatment of SHPT in patients with CKD receiving haemodialysis (i.e. those with end-stage kidney disease). The comparators specified in the scope and the company’s decision problem were established clinical practice without calcimimetics (dietary modification and PBVD), and the calcimimetic cinacalcet, for use specifically in a population of patients with SHPT who are refractory to established clinical practice without calcimimetics (specified in accordance with NICE technology appraisal 117 [6]).

3 The Independent Evidence Review Group (ERG) Review

3.1 Clinical-Effectiveness Evidence

The CS included a systematic literature review that identified three relevant randomised controlled trials (RCTs) of etelcalcetide versus the comparators specified in the scope. The CS also included brief findings from three non-RCTs as supporting data. The company did not conduct a network meta-analysis or formal indirect comparison.

The systematic review identified and included the following evidence:

  • Two phase III, double-blind, multicentre RCTs of etelcalcetide (plus PBVD) versus placebo (plus PBVD), administered for 26 weeks in a broad population of patients with CKD and SHPT receiving haemodialysis (trials 20120229 and 20120230) [8]. The trials were of near-identical design and the company presented pooled analyses of results from the two trials in addition to separate results. The trials included a total of 1023 participants.

  • One phase III, double-blind, multicentre RCT of etelcalcetide (plus PBVD) versus cinacalcet (plus PBVD) administered for 26 weeks in a broad population of patients with CKD and SHPT receiving haemodialysis (trial 20120360, N = 515) [9]. All three etelcalcetide RCTs were sponsored by Amgen Limited.

  • Two phase III, single-arm extension studies to trials of etelcalcetide including trials 20120229, 20120230 and 20120360 (studies 20120231, N = 891 [10]; and 20130213, N = 902 [11]).

  • One phase III, single-arm study of the efficacy and safety of patients switching from cinacalcet to etelcalcetide (study 20120359, N = 158) [12]. The reasons for switching were not provided.

The phase III RCTs measured some of the outcomes specified in the NICE scope for the appraisal, including various measures of PTH, serum levels of calcium and phosphate, health-related quality of life (HRQoL) [in the cinacalcet-controlled trial only] and adverse events. The CS uses the pg/mL unit to describe PTH levels, but we note that in the UK, PTH is measured in pmol/L units. Therefore, where we discuss PTH in this article, we lead with the pg/mL units, but supply the equivalent pmol/L units in brackets. We note the trials did not report achievement of target PTH used in practice for patients receiving dialysis. The target used in practice is a PTH of two to nine times the upper limit of normal of the reference limit of the laboratory test used, which we note translates to a PTH range of approximately 130–600 pg/mL (13.8–63.6 pmol/L). Importantly, the trials also did not use the target ranges for phosphate and calcium used in UK clinical practice as outcomes. The trials did not measure the longer-term outcomes specified in the scope: survival and incidences of fractures, cardiovascular events and parathyroidectomy.

The results of the trials showed that participants treated with etelcalcetide (plus PBVD) were statistically significantly more likely to achieve a > 30% reduction in mean PTH from baseline during the efficacy assessment phase (weeks 20–27) than those treated with placebo (plus PBVD) [pooled analysis 74.7 vs. 8.9%; stratified odds ratio 30.80, 95% confidence interval (CI) 18.18–52.17, p < 0.001; data pooled from intention-to-treat (ITT) analyses]. Etelcalcetide (plus PBVD) was found to be both non-inferior and superior to treatment with cinacalcet (plus PBVD) on this outcome (superiority analysis: cinacalcet 57.7% vs. etelcalcetide 68.2%; odds ratio 1.59, 95% CI 1.16–2.17), p = 0.004; ITT analysis).

Proportionally more participants treated with etelcalcetide (plus PBVD) achieved a mean PTH of ≤ 300 pg/mL (31.8 pmol/L) than those treated with placebo (plus PBVD) in both placebo-controlled trials (pooled analysis 51.5 vs. 4.9%; stratified odds ratio 27.02, 95% CI 16.62–43.93, p < 0.001; data pooled from ITT analyses).

Proportionally more participants treated with etelcalcetide (plus PBVD) than those treated with cinacalcet (plus PBVD) also achieved this target in the cinacalcet-controlled trial [9] (odds ratio, 95% CI and p value not reported in the CS).

Participants treated with etelcalcetide (plus PBVD) had greater reductions in phosphate levels than those treated with placebo (plus PBVD) [not ITT analyses] in the placebo-controlled trials [8]. However, there was no difference between etelcalcetide and cinacalcet in the proportion of participants reaching the phosphate target used in the cinacalcet-controlled trial [9] (an ITT analysis; not a target used in practice). Participants treated with etelcalcetide (plus PBVD) experienced greater reductions in calcium than those treated with placebo (i.e. PBVD alone) [who experienced a slight increase] or cinacalcet. HRQoL in the cinacalcet-controlled trial [9] did not appear to change substantially over time in either the etelcalcetide or cinacalcet arms, although scores were slightly lower in the etelcalcetide arm by week 26 (lower scores indicated reduced HRQoL). Neither the calcium nor HRQoL outcomes appear to have been analysed in the ITT population.

The most common adverse event experienced by participants treated with etelcalcetide in all three trials was an asymptomatic decrease in blood calcium. This was experienced by a higher proportion of patients treated with etelcalcetide (plus PBVD) [68.9%] compared with cinacalcet (plus PBVD) [59.8%] in the cinacalcet-controlled trial [9], and by a higher proportion of patients treated with etelcalcetide than those treated with placebo (i.e. PBVD alone) in the placebo-controlled trials (etelcalcetide 63.8%, placebo 10.1%) [8]. Rates of symptomatic hypocalcaemia events and cardiac failure were also higher with etelcalcetide than placebo or cinacalcet.

3.1.1 ERG Critique of the Clinical-Effectiveness Evidence

The company’s literature searches included a wide range of electronic databases and other sources. We evaluated the search strategies as fit-for-purpose, and our update search did not identify any additional relevant RCTs. The clinical-effectiveness review followed standard systematic review procedures and data were appropriately synthesised (with the exception of the unadjusted method of pooling PTH outcome data from the phase III etelcalcetide trials, discussed in Sect. 3.2.1 below). We consider there is a low chance of systematic error in the review, based on the methods reported in the CS.

The CS review identified relevant international phase III RCTs that included a large number of patients (approximately 1500 in total). Clinical expert advice to the ERG suggested that the included patients were generally representative of those seen in practice in the UK. We judged the three trials to be of good quality, except it was unclear whether double-blinding had been adequately preserved and results for some secondary outcomes did not appear to be ITT analyses.

The single identified cinacalcet-controlled trial [9] included a broad population of patients with SHPT, rather than specifically those with refractory SHPT, for whom cinacalcet is the relevant comparator in the NICE scope. It is uncertain if, as the company argued in the CS, the subgroups of patients in this trial who had previously been treated with cinacalcet are representative of patients refractory to treatment with PBVD alone. The strength of this argument depends on how cinacalcet is used in the countries in which the trials took place, i.e. whether cinacalcet was used as an initial treatment in a broad population of patients or as a second-line treatment for patients specifically with refractory SHPT. In this respect, the CS did not fully meet the NICE scope. We attempted to adjust the clinical-effectiveness results to reflect the different risks of patients who are ‘refractory’ to standard treatment in the economic model (see additional ERG analysis below).

The trials included in the review did not measure the most clinically relevant outcomes, i.e. survival, incidence of cardiovascular events and bone fractures, and achievement of the PTH target currently used in UK clinical practice for patients receiving haemodialysis (two to nine times the upper limit of the normal reference range; approximately 130–600 pg/mL; 13.8–63.6 pmol/L). This means it is uncertain how etelcalcetide impacts on longer-term outcomes compared with cinacalcet, and standard of care without calcimimetics. The company presented different methods to estimate a relationship between the primary outcome of the etelcalcetide trials (30% reduction in PTH) and longer-term clinical outcomes (e.g. survival, cardiovascular events, fractures) in the economic model, but, as discussed in Sect. 3.2.1 of this article, no direct empirical evidence was presented to validate any hypothesised relationships.

The CS states that the safety profile of etelcalcetide is similar to cinacalcet, but we consider this not entirely justified; higher rates of asymptomatic decreased blood calcium (acknowledged in the company’s interpretation of the evidence), symptomatic hypocalcaemia and cardiac failure were observed with etelcalcetide than cinacalcet. Clinical expert advice to the ERG indicated that symptomatic hypocalcaemia or very low calcium would likely result in increased health care resource utilisation to manage these AEs. Information about the effect of etelcalcetide treatment and related adverse effects on HRQoL is also lacking. These factors were not included in the economic model.

It is also uncertain what proportion of patients would meet the PTH target used in practice when treated with etelcalcetide compared with either treatment with cinacalcet or standard of care without calcimimetics. Relatedly, drug doses in all three trials were titrated to a PTH target of < 300 pg/mL (31.8 pmol/L), but we suggest, based on clinical advice we received, that this is not reflective of clinical practice. Expert clinical advice to the ERG noted 300 pg/mL is in the middle of the two to nine times the upper limit of normal reference range, but that, in practice, clinicians would aim for a PTH range of 150–300 pg/mL (15.9–31.8 pmol/L), and would accept a PTH in the range of two to nine times the upper limit of the normal reference range in selected patients (approximately 130–600 pg/mL; 13.8–63.6 pmol/L) depending on the levels of other parameters such as calcium and phosphate. Therefore, the treatment protocols (i.e. PTH target and drug doses administered to reach this target) used in the trials are not necessarily reflective of current practice in the UK. Outcomes may be different to those found in the trials when using the less-stringent treatment target (i.e. in patients who are left with a higher PTH). This also means that longer-term outcomes in the economic model were not extrapolated from the most clinically relevant PTH endpoint (i.e. the less-stringent target used for some patients in practice), which could impact on the rates of longer-term outcomes estimated, and, hence, cost effectiveness.

3.2 Cost-Effectiveness Evidence

The company’s submission to NICE included a systematic review of published economic evaluations (cost-effectiveness, cost-utility and cost-benefit studies) and a de novo economic model.

Inclusion criteria in the company’s systematic review were in line with the NICE scope: treatments for SHPT in adult patients receiving haemodialysis for CKD. The search identified 16 economic evaluations, none of which evaluated etelcalcetide. Of the 16 studies identified, three studies in particular were used to inform the economic model.

  • A Health Technology Assessment (HTA) by Garside et al. provided assumptions and data sources [13].

  • An economic evaluation by Belozeroff et al., based on the EVOLVE RCT of cinacalcet (and PBVD) compared with placebo (and PBVD), informed the model structure and input parameters [14]. The EVOLVE trial was a large (n = 3883 patients) international trial, with a long follow-up (up to 5 years), comparing cinacalcet with placebo [15].

  • An economic evaluation by Eandi et al. provided a biomarker-based risk-prediction equation that was used to predict long-term outcomes of calcimimetic therapy in a scenario analysis [16].

The company submitted a de novo Markov state-transition model to estimate the cost effectiveness of etelcalcetide compared with cinacalcet, or compared with standard therapy alone (PBVD) for the treatment of SHPT in adult patients receiving haemodialysis for CKD. The model consists of health states representing the three principal adverse events related to SHPT: all-cause mortality; non-fatal clinical fractures; and non-fatal cardiovascular events (including myocardial infarction, hospitalisation for unstable angina, heart failure and peripheral arterial disease). Patients begin the model in the event-free state, and over time may experience one or more non-fatal cardiovascular events and/or bone fractures. After one non-fatal event, patients are at higher risk of recurrence of the same type of event. Parathyroidectomy was included in the model as an incident event, rather than as a health state or treatment. This means that the model cannot reflect long-term costs or health effects of parathryroidectomy.

Treatment effectiveness is modelled using hazard ratios for each of the principal events and parathyroidectomy. Background event rates were calculated from the placebo arm of the EVOLVE trial. Hazard ratios for cinacalcet compared with PBVD were derived from a covariate-adjusted, lag-censored analysis of the EVOLVE trial. The lag-censored approach attempts to account for high rates of treatment discontinuation and switching in the EVOLVE trial. The lag time for censoring of 6 months after discontinuation was prespecified and informed by expert opinion. Hazard ratios for etelcalcetide were extrapolated from those estimated for cinacalcet from the EVOLVE trial by assuming a linear relationship between the proportion of patients achieving a > 30% reduction in PTH and log-hazard ratios. The company estimated proportions of patients achieving a > 30% reduction in PTH from baseline for all interventions from a ‘naive’ (unadjusted) pooling of the pivotal phase III etelcalcetide RCTs [8, 9]. Discontinuation of cinacalcet treatment was modelled using a Weibull curve fitted to EVOLVE trial data, while etelcalcetide discontinuation was assumed to be equivalent to cinacalcet discontinuation. Adverse events were not modelled as the company argued that calcimimetics were well-tolerated, with an event profile consistent with pre-existing comorbid conditions associated with SHPT.

Estimation of HRQoL was informed by a systematic review conducted by the company that identified five studies, one of which was an analysis of EQ-5D data from the EVOLVE trial by Briggs et al. [17]. This was used as the source of utilities in the model, including a utility value for patients on dialysis but ‘event free’, and disutilities for the first 3 months after an event and subsequently. In the company’s base case, no utility gain was assigned for the use of calcimimetics; we explored this assumption in a scenario analysis.

The company also conducted a systematic review of resource use and costs, but only used one of the seven identified cost-of-illness studies from this review in the model, i.e. a study by Pockett et al. [18] that estimated the cost of parathyroidectomy. Other resource use was obtained from the pivotal etelcalcetide RCTs [8, 9]. Costs included drug costs, monitoring costs and acute event costs. Dialysis costs were not included in the base case, but were evaluated in a scenario analysis. Unit costs were derived from NHS sources (NHS Drug Tariff, British National Formulary, NHS Reference Costs) for the year 2015.

Subsequent to their initial submission to NICE, the company provided a confidential Patient Access Scheme (PAS) discount for the cost of etelcalcetide. All results are presented, with the price discounts from this PAS included.

For the base-case comparison to PBVD in refractory SHPT, the base-case incremental cost-effectiveness ratio (ICER) produced was £27,251 per quality-adjusted life-year (QALY) gained. Scenario analyses conducted by the company produced ICERs between £23,609 and £61,280 per QALY gained. The base-case comparison of etelcalcetide to cinacalcet in the broad licensed population produced an ICER of £14,777 per QALY gained, while scenario analyses conducted by the company produced ICERs of between £13,156 and £48,677 per QALY gained. We note that in all analyses, the etelcalcetide QALYs gained did not vary by treatment population (i.e. refractory and non-refractory patients) as they were based on the broad SHPT population in the EVOLVE trial.

3.2.1 Critique of the Cost-Effectiveness Evidence

The analysis presented by the company has several strengths. The model structure reflected the nature of SHPT and its impacts on patient outcomes. The model was also well implemented, and we did not identify any important coding errors. The choice of sources for the main input parameters—effectiveness, utility and resource use/costs—were informed by systematic literature reviews. The model and results were clearly described in the CS and in responses to clarification questions about the CS raised by the ERG and NICE, and justification was given for most important modelling decisions. The company also used a range of approaches to explore the impact of major structural uncertainties over the extrapolation of 6-month intermediate outcomes to estimate long-term risks and health outcomes. A number of key modelling assumptions and data sources were conservative, and did not unreasonably exaggerate the effects or cost effectiveness of etelcalcetide.

However, the extrapolation from the short-term biochemical outcomes (e.g. PTH) measured in the etelcalcetide trials to clinically relevant outcomes introduces considerable uncertainty in the economic results. The model relies particularly on the EVOLVE trial for this extrapolation, as well as for other parameters, including estimates of long-term risks, discontinuation rates, utilities and resource use. As stated, this was a large, long-term trial, however results were confounded by some imbalance in patient characteristics at baseline (primarily age), and by high rates of discontinuation; 71% of patients randomised to placebo and 67% of patients randomised to cinacalcet discontinued randomised treatment. Much of this discontinuation was due to treatment switching through obtaining commercially available cinacalcet or having a kidney transplant or parathyroidectomy, as detailed in Table 1. Furthermore, the company has presented several analyses that attempt to correct for baseline covariates and non-adherence, but it is not clear whether these successfully minimise bias.

Table 1 Discontinuation and treatment switching in the EVOLVE trial [15]
Full size table

The log-linear method used to extrapolate from the etelcalcetide primary outcome (≥ 30% reduction in PTH) is reasonable in the absence of any other data but entails a strong assumption that cannot be empirically validated or adequately justified with current published data. The EVOLVE RCT reported data on PTH levels over time but it was unclear if these data were available to translate into the primary outcome of the etelcalcetide trials, i.e. the percentage of patients achieving a 30% PTH-level reduction. If the raw data are still available, validation of the model assumptions, and potentially a network meta-analysis on the primary efficacy outcome of the phase III etelcalcetide trials with data from the EVOLVE trial, would be possible, which would greatly reduce uncertainty in the model. In its analysis, the company used a ‘naive’ unadjusted method of pooling PTH outcome data from the phase III etelcalcetide trials [8, 9]. We consider this inappropriate as the randomised nature of the treatment comparisons within the trials is broken. To examine the impact of this, we applied a simple method of indirect treatment comparison in the model, which gave different results.

The company presented another method of extrapolation that did not rely on the EVOLVE trial: using a published algorithm by Eandi et al. [16] to predict the risk of clinical events based on biomarker measurements (PTH, calcium and phosphate) for patients in the phase III etelcalcetide RCTs [8, 9]. While this method is, in principle, superior as it incorporates more clinically relevant variables, the evidence base that the algorithm was constructed from was poor as it was based on a heterogeneous selection of observational studies. The prediction algorithm was presented without validation [15, 16]. It should be noted that PTH, calcium and phosphate levels were all measured in the EVOLVE trial alongside clinical outcomes (survival, cardiovascular events, fractures). It was unclear if these data are still available to validate the prediction algorithms used in the Eandi et al. study or to create a new prediction algorithm [15]. On balance, we consider the EVOLVE trial-based methods preferable.

The economic model had a number of other limitations. It included acute-care costs and disutility for patients undergoing parathyroidectomy, but excluded any longer-term savings or health effects that might be associated with this procedure. This tends to favour etelcalcetide, because it was estimated (through the extrapolation method outlined above) to cause a large reduction in the use of this procedure. Without major restructuring of the model, it is not possible to explore the impact of the omission of the future costs and benefits of parathyroidectomy. Costs for cardiovascular events and fractures were limited to initial acute treatment. Re-admissions and ongoing outpatient, community and primary care costs were not included; thus, cost savings associated with better management of SHPT are likely to be underestimated. We also note that efficacy for etelcalcetide was identical, irrespective of whether patients were refractory or non-refractory, therefore the company analysis does not specifically reflect risks for the refractory group, for whom cinacalcet is the appropriate comparator. It is also uncertain whether some model parameters (mortality, cardiovascular, fracture and parathyroidectomy rates, and drug doses) are representative of a UK population as very few UK patients were included in the trials.

3.2.2 Additional Analyses Conducted by the ERG

We conducted a number of scenario analyses to further test the robustness of the company’s base-case economic analyses. These were using a simple chained method of indirect comparison to estimate PTH reduction achievement; using formal methods recommended by the NICE Decision Support Unit guidance [19] to adjust for treatment switching in the EVOLVE trial; using observed discontinuation hazard ratios from trial evidence; allowing utility gain for calcimimetic use; and evaluating calcimimetic treatment sequences and using different clinical-effectiveness outcomes for refractory and non-refractory patients. Finally, we report the ERG preferred base case, incorporating the key assumptions we consider most plausible. As previously stated, all analyses are presented with the PAS discount included.

We used a simple chained method of indirect comparison to estimate the proportion of patients achieving > 30% reduction in PTH for use in the extrapolation of EVOLVE trial risks. Our preferred approach only used the phase III etelcalcetide RCTs [8, 9]. Results differed from the company’s approach: 8.9% with PBVD alone, 66.1% with cinacalcet and PBVD, and 75.6% with etelcalcetide and PBVD (compared with 8.9, 57.1 and 72.1%, respectively, in the company’s analysis). This led to a small increase in the ICER for etelcalcetide versus PBVD (£29,730 per QALY gained), but a larger increase in the ICER for etelcalcetide versus cinacalcet (£27,701 per QALY gained). For comparison, we also conducted analyses using results from an ERG meta-analysis of cinacalcet (plus PBVD) versus placebo (plus PBVD) RCTs. This highlighted the heterogeneity of these data and the sensitivity of the etelcalcetide versus cinacalcet comparison to the method of pooling used. This point was further emphasised in a scenario analysis provided by the company in response to an ERG clarification question. This analysis used the secondary outcome of the proportion of patients reaching a PTH of ≤ 300 pg/mL, rather than the proportion of patients achieving a > 30% reduction in mean PTH from baseline, and led to an ICER of £11,490 per QALY gained compared with cinacalcet when added to our simple indirect treatment comparison analysis.

ICERs were also sensitive to the method used to adjust EVOLVE trial results for non-adherence. The company presented two methods in the CS: (1) lag-censoring, in which patients (in both arms) were censored from the analysis 6 months after discontinuing the study drug (the company’s preferred base case); and (2) a ‘disaggregation’ method in which ITT estimates were adjusted to account for time spent on and off treatment. In response to a clarification question, the company provided estimates of effects using two complex formal methods of adjustment: the Rank Preserving Structural Failure Time Model (RPSFTM) and Iterative Parameter Estimation (IPE) approaches, which we consider more appropriate than the lag-censored approach used in the base case [19]. These methods yielded lower ICERs; for example, the IPE method gave an ICER of £25,111 per QALY gained for etelcalcetide versus PBVD alone, and £14,292 per QALY gained for etelcalcetide versus cinacalcet.

The analysis of EQ-5D data from the EVOLVE trial, by Briggs and colleagues [17], estimated a statistically significant independent utility gain of 0.02 (95% CI 0.01–0.03) for patients receiving cinacalcet, after adjusting for clinical events. This suggests that there may be a symptomatic improvement in HRQoL with cinacalcet. In their base case, the company excluded this effect but conducted a scenario analysis in which they assumed that it applied equally to both calcimimetics. This produced an ICER of £23,843 per QALY gained versus PBVD, and £14,633 per QALY gained versus cinacalcet, thus very small ICER decreases for etelcalcetide. We tested the impact of a differential utility effect for the two drugs. When we applied the utility gain to cinacalcet only, the ICER rose to £42,761 per QALY gained for the etelcalcetide versus cinacalcet comparison. A plausible explanation for why there may be a differential utility effect is due to the increase of hypocalcaemia observed in etelcalcetide patients in the cinacalcet controlled trial [9]. Clinical experts we consulted indicated that hypocalcaemia adversely affects quality of life.

The company reported a post hoc subgroup analysis for patients who had discontinued cinacalcet due to lack of efficacy, adverse events or intolerability. The effectiveness of etelcalcetide was not significantly lower in this population, although we note that the statistical power for this analysis would have been low. Nevertheless, it does suggest that a sequenced approach to the use of calcimimetic drugs might be appropriate. We therefore adapted the model to conduct an incremental analysis, including two sequenced calcimimetic strategies. To avoid out-of-scope comparisons, we did not allow treatment starting with cinacalcet for patients not refractory to PBVD alone, or PBVD alone for refractory patients. In both groups, treatment with etelcalcetide (with PBVD) followed by PBVD alone was dominated by a sequenced strategy (i.e. it was less expensive and more effective).

A drawback with this analysis, as with the company’s base case, is that it assumes equivalent outcomes on calcimimetic treatment for patients who are refractory and non-refractory to treatment with PBVD alone. We consider this unlikely and therefore conducted subgroup analysis in which we varied the proportion of patients assumed to achieve > 30% reduction in PTH on PBVD alone, indicating how ‘refractory’ they might be to this treatment. The ICER for etelcalcetide versus PBVD alone was higher for patients with a higher probability of responding to PBVD alone. The ICER for etelcalcetide compared with cinacalcet rose more steeply for this easier-to-treat group.

The ERG-preferred base case differs from the company base case in two key respects: (1) the method of pooling results of the etelcalcetide trials (‘simple indirect treatment comparison’ rather than naive pooling); and (2) the method for estimating hazard ratios for clinical events from the EVOLVE trial (IPE rather than the lag-censored approach). Assuming a population in which 8.9% of patients would achieve > 30% reduction in PTH on standard treatment (the mean for placebo arms of the 20120229 and 20120230 trials) [8], the ICERs for etelcalcetide are £27,290 per QALY gained versus PBVD alone, or £22,400 per QALY gained versus cinacalcet. However, if we assume that patients who meet NICE criteria for treatment with cinacalcet (i.e. with refractory SHPT) are less likely to respond to PBVD alone (e.g. if 4.9% achieve > 30% reduction in PTH, as in the placebo arm of the company’s subgroup analysis for patients who have discontinued cinacalcet), the etelcalcetide versus cinacalcet ICER is lower at £16,244 per QALY gained. Conversely, patients being considered for treatment with PBVD alone (i.e. non-refractory) are more likely to respond (e.g. 17.1% achieve > 30% reduction in PTH, as in the placebo arm of the ERG meta-analysis of cinacalcet trials). In this group, the ERG base-case ICER for etelcalcetide versus PBVD is £28,626 per QALY gained. These patients would also be expected to respond more to cinacalcet.

Finally, we conducted analyses including appropriate sequenced strategies for refractory and non-refractory patients (4.9 vs. 17.1% responding to PBVD), following ERG base-case assumptions. None of the sequences produced superior cost effectiveness to etelcalcetide alone.

4 National Institute for Health and Care Excellence (NICE) Guidance

The NICE Appraisal Committee reviewed the clinical- and cost-effectiveness evidence available through the CS and the ERG report alongside testimony from clinical experts and patient representatives.

4.1 NICE Preliminary Guidance

Following consideration of the evidence submitted by the company and the ERG, and testimony from experts and other consultees, the NICE Appraisal Committee issued a preliminary recommendation. The committee recommended that etelcalcetide be an option for the treatment of SHPT in adults with CKD only if treatment with a calcimimetic is indicated, but cinacalcet is not suitable, and only if the company provides etelcalcetide with the discount agreed in the PAS.

4.2 Final NICE Guidance

After a period of public consultation, a second committee meeting was conducted in which the initial decision of the first Appraisal Committee was affirmed. The committee considered that the most appropriate comparator to etelcalcetide was cinacalcet in a refractory second-line therapy environment. The committee acknowledged the advantages of having an intravenous calcimimetic available in potentially improving adherence and lowering the pill burden of treatment. However, because of substantial uncertainty in establishing the long-term benefits of etelcalcetide compared with cinacalcet and higher associated costs, etelcalcetide is only recommended as an option for patients with SHPT for whom a calcimimetic is indicated but for whom cinacalcet is not considered suitable.

The committee noted that while the ICERs for etelcalcetide compared with cinacalcet were lower than £30,000 per QALY gained, the uncertainty in the estimates was high due to the lack of long-term efficacy data and the methodological assumptions that underpinned the analyses used to extrapolate long-term outcomes from short-term surrogate biochemical outcomes. Specifically, the committee concluded that the long-term projections of clinical events for etelcalcetide were dependent on a highly adjusted trial (EVOLVE) of a different treatment (cinacalcet) and an assumed directly proportional relationship between PTH reduction and clinical events that was not validated. Beyond the uncertainty in the method of extrapolating short-term biomarkers to long-term clinical outcomes, the committee emphasised that there was large uncertainty in the hazard ratio for mortality, which was the primary driver of cost effectiveness in the model. The committee stated that this uncertainty could increase the ICER by more than £10,000 per QALY gained without any consideration of the uncertainty in the validity of the extrapolation methods.

Additional uncertainty was acknowledged by the committee in relation to the appropriateness of the evidence base for a population of patients refractory to PBVD. The committee concluded that the trials used for clinical-effectiveness evidence included a broad population of patients with SHPT, rather than a population of patients who were refractory to PBVD—the population of patients who calcimimetics (etelcalcetide or cinacalcet) would typically be offered to in UK clinical practice.

The committee concluded that the most plausible ICER for the comparison of etelcalcetide to cinacalcet is between £14,778 and £26,647 per QALY gained, but with substantial uncertainty, as detailed above.

5 Conclusions

While etelcalcetide appears to be more effective in improving short-term surrogate outcomes, the long-term efficacy of etelcalcetide is highly uncertain. All of the studies for etelcalcetide and cinacalcet used in the evidence submission were from a broad SHPT population, not the narrower population of patients who are refractory to PBVD. The mapping of short-term surrogate outcomes to long-term clinical events (mortality, cardiovascular events, fractures, parathyroidectomies) was based on an assumed linear relationship between log hazards and percentage reduction in PTH that was not validated by any empirical data. Additionally, there was substantial uncertainty in the hazard ratio for mortality, which was the primary driver of the model. Our best estimates of the ICER for etelcalcetide compared with cinacalcet did not include uncertainty in the relationship between PTH reduction and longer-term clinical events. This means that our estimated ICERs underestimate decision uncertainty.

The model provided in the CS was well-constructed, transparent and free from any influential errors; however, its conclusions rest on unvalidated assumptions and should be interpreted with caution. Further use of EVOLVE trial data may allow validation of some assumptions, but this had not been undertaken at the time of the evaluation.

The committee’s recommendations echoed the concerns of the ERG but acknowledged that providing patients with multiple treatment options and modalities as well as further options when a treatment fails is important. The committee therefore recommended that etelcalcetide only be used in patients who are refractory to PBVD for whom cinacalcet is unsuitable.