FormalPara Key Points for Decision Makers

The main clinical-effectiveness data were derived from a multicentre randomised controlled trial (RCT) that demonstrated improvements in progression-free survival (PFS) and overall survival (OS) for crizotinib compared with pemetrexed chemotherapy in combination with carboplatin or cisplatin in patients with untreated anaplastic lymphoma kinase (ALK)-positive non-small-cell lung cancer (NSCLC).

The base-case incremental cost-effectiveness ratio (ICER) was estimated to be greater than £50,000 per quality-adjusted life-year (QALY) gained excluding the patient access scheme (PAS) discount.

The Evidence Review Group (ERG) critique of the evidence submitted by the company raised a number of concerns regarding the clinical data supporting the claimed OS benefits. The ERG also considered that the assumption made by the company regarding duration of treatment significantly underestimated total time on treatment (and hence total costs).

The ERG’s exploratory analyses focused on alternative approaches to analysing the OS data and reconfiguration of the economic model to better account for time spent on treatment.

Crizotinib was recommended as an option for untreated ALK-positive advanced NSCLC in adults once a PAS was agreed.

1 Introduction

Guidance on which treatments should be offered to patients on the NHS in England is issued by the National Institute for Health and Care Excellence (NICE). The role of the NICE technology appraisal programme is to assess treatments and make recommendations based on the cost effectiveness of the technology. Single technology appraisals (STAs) are designed to appraise a single technology with a single indication. The manufacturer of the technology submits evidence, which is reviewed and critiqued by an independent evidence review group (ERG). A NICE appraisal committee then considers that evidence, as well as additional evidence supplied by patient, clinical and NHS commissioning experts, and the ERG review.

NICE previously assessed crizotinib for patients with previously treated anaplastic lymphoma kinase (ALK)-positive locally advanced or metastatic non-small-cell lung cancer (NSCLC) [1]. It was not approved but was made available through the Cancer Drugs Fund (CDF). However, the company subsequently submitted evidence for crizotinib to be considered in a different population: patients with previously untreated ALK-positive locally advanced or metastatic NSCLC.

This article summarises the ERG critique of the company’s submission. The issues raised during the review and the committee’s decision-making process are also summarised. Full details of the appraisal and the relevant documents can be found on the NICE website [2].

2 The Decision Problem

Lung cancer is a highly prevalent disease, with 33,027 reported cases in England and Wales in 2014 [3]. It is characterised by abnormal or uncontrolled growth of lung cells [4] and is grouped into two categories, small-cell lung cancer (SCLC) and NSCLC, which make up 15% and 85% of patients, respectively [5]. Within NSCLC, there are three main sub-categories: large-cell undifferentiated carcinoma (10–15%), adenocarcinoma (40%) and squamous cell carcinoma (25–30%). The ALK fusion gene is active in approximately 3–5% of patients with NSCLC [6, 7], with the vast majority of cases being found in adenocarcinomas.

Lung cancer is a major cause of mortality and morbidity in the UK. Symptoms of NSCLC include coughing, chest pain, weight loss and loss of appetite, shortness of breath (dyspnoea) and fatigue. Once it has spread to distant organs, NSCLC can also result in bone pain, nervous system changes, yellowing of the skin (jaundice) and lumps near the surface of the body [4]. Therefore, the symptom burden has a significant impact on patients’ quality of life, with around 90% of patients experiencing two or more disease-related symptoms, which may result in psychological distress [8]. In some cases, the disease can result in the development of brain metastases, which severely increases patients’ mortality risk [9]. As symptom onset often occurs once the cancer has developed, prognosis is poor for those with advanced NSCLC, with current survival rates for lung cancer being the second lowest of 20 common cancers in England and Wales [5]: 7–24% of patients with stage 3 NSCLC and 2–13% of those with stage 4 NSCLC are estimated to survive for ≥5 years [10].

Current NICE guidance [11] recommends pemetrexed chemotherapy in combination with carboplatin or cisplatin given intravenously for first-line treatment in patients with inoperable advanced NSCLC and a good performance score. Several randomised controlled trials (RCTs) have shown median overall survival (OS) to be somewhere between 7.8 and 11.8 months for patients treated with pemetrexed chemotherapy [12–14]. Evidence for survival times amongst patients with ALK-positive NSCLC is scarce, but some limited data suggest that survival times for these patients may in fact be higher because many are younger and non-smokers. A small retrospective analysis of 36 patients with ALK-positive disease in a phase I clinical trial, many of whom had received prior therapy, reported a median OS of 20 months [95% confidence interval (CI) 13–26] [13].

Crizotinib is an oral receptor tyrosine kinase inhibitor that acts against ALK and its oncogenic variants. It is targeted at adults with ALK-positive advanced NSCLC and has been approved for use in 87 countries. NICE guidance TA296 [1] was issued in September 2013 and did not recommend crizotinib for people with previously treated ALK-positive NSCLC, but the treatment was made available through the CDF. Crizotinib has recently been granted EU marketing authorisation for use in the first-line setting. The manufacturer recommends 250 mg twice daily to be taken continuously until disease progression or unacceptable toxicity. Patients must undergo screening to identify whether they have ALK-positive disease before they can be offered crizotinib.

3 The Independent Evidence Review Group (ERG) Review

The company submitted evidence to NICE on the use of crizotinib in the first-line treatment of patients with non-squamous ALK-positive NSCLC. The ERG reviewed the submission received by NICE, assessed whether or not the submission conformed to NICE methodological guidelines, critiqued the company’s interpretation and analysis of the evidence and checked for the existence of other evidence or alternative interpretations of the evidence.

3.1 Clinical Evidence

The company’s submission included a systematic review conducted to identify RCT and non-RCT studies that investigated the efficacy and safety of crizotinib. One multicentre phase III open-label RCT was identified (PROFILE 1014) [15] that compared crizotinib with pemetrexed chemotherapy in combination with carboplatin or cisplatin in patients with untreated non-squamous ALK-positive NSCLC.

In the trial, crizotinib demonstrated a statistically significant improvement in progression-free survival (PFS): median 10.9 months (95% CI 8.3–13.9) versus 7.0 months (95% CI 6.8–8.2) for chemotherapy. Additionally, crizotinib had a statistically significant greater tumour response, with an objective response rate of 74% (95% CI 67–81) versus chemotherapy, which had a rate of 45% (95% CI 37–53). Additionally, crizotinib had a shorter time to response and a greater duration of response than chemotherapy.

At the time of the data-cut off, median OS had not been reached because just 26% of patients had died since randomisation. At the time of the data-cut off, median follow-up was 17.4 months for patients assigned to crizotinib and 16.7 months for those assigned to chemotherapy, a difference that was not statistically significant. However, 70% of patients who were initially assigned to chemotherapy were later permitted to cross over following progression and receive crizotinib. The company therefore utilised nine different methods to adjust for the presence of cross-over, using variations of the rank-preserving structural time model (RPSFTM), the iterative parameter estimation (IPE) method and the two-stage method. All nine resulted in hazard ratios (HRs) that demonstrated increased OS for patients receiving crizotinib relative to chemotherapy, with HRs ranging from 0.571 to 0.674. In most cases, the CIs were wide, with only four demonstrating a statistically significant benefit.

A number of different measures were utilised to measure health-related quality of life (HRQoL), including the EuroQol five Dimensions (EQ-5D) and the lung cancer-specific module of the European Organisation for Research and Treatment of Cancer Quality of Life Questionnaire (EORTC QLQ LC-13). These measures demonstrated that patients had improved quality of life whether receiving crizotinib or chemotherapy compared with baseline, with greater improvements for those treated with crizotinib.

There was little difference in the numbers of patients experiencing a treatment-related adverse event between the two randomised groups: 98.2% of crizotinib-treated and 92.9% of pemetrexed chemotherapy-treated patients experienced at least one. Within these patients, 35.1% in the crizotinib arm and 39.1% in the chemotherapy group experienced grade 3 or 4 adverse events. Adverse events leading to permanent discontinuation occurred in 12% and 14% of patients in the crizotinib and chemotherapy groups, respectively. The most frequently reported adverse events in the crizotinib group compared with the chemotherapy group were vision disorders (71 vs. 9%), diarrhoea (61 vs. 13%) and oedema (49 vs. 12%). Conversely, the most reported adverse events for chemotherapy-treated patients compared with those receiving crizotinib were fatigue (38 vs. 29%), anaemia (32 vs. 9%) and neutropenia (30 vs. 21%).

Two non-randomised studies were also included in the submission: PROFILE 1001 [16] and Davis et al. [17]. PROFILE 1001 was a single-arm, open-label phase I study in which 24 patients received first-line therapy of crizotinib 250 mg twice daily, resulting in a median PFS of 18.3 months (95% CI 8.3 to ‘not reached’) at the latest data cut-off point. Davis et al. [17] was a retrospective cohort study of 210 American and Canadian patients with confirmed ALK-positive NSCLC on both first and second-line crizotinib therapy that reported a median PFS of 9.6 months (95% CI 8.4–10.8) and a median OS of 2 years (95% CI 1.5 to ‘not reached’).

A pooled analysis of safety data was also conducted drawing on data from PROFILE 1014 and 1001 as well as PROFILE 1005 and 1007, which investigated crizotinib as a second-line therapy. The company submission reported that the pooled analysis (n = 1699) showed the safety profile was relatively consistent across all trials and lines of therapy. The most frequently reported adverse events experienced with crizotinib were vision disorders (62%), nausea (57%) and diarrhoea (54%). The European Medicines Agency (EMA) determined that the adverse event profile for crizotinib therefore presented a clinically significant but manageable burden to patients [18].

3.2 Critique of Clinical Evidence

The ERG highlighted a number of issues with the clinical evidence presented by the company.

3.2.1 Trial Design

PROFILE 1014 was a well-conducted trial, and the potential bias resulting from the open-label nature of the study was likely mitigated by the use of RECIST (Response Evaluation Criteria In Solid Tumors), an objective measure of disease progression,. However, in practice, progression is determined by the worsening of symptoms and not the RECIST criteria, which means the results seen in the trial may not be reflected in clinical practice. Additionally, the trial permitted treatment decisions after disease progression, which put the trial at high risk of bias for OS.

3.2.2 Overall Survival Data

The ERG highlighted four main issues with the survival data presented by the company.

First, the data—particularly OS data—were immature. At the data cut-off point at 18 months, death had occurred in just 26% of those who underwent randomisation.

Second, the methods by which the company adjusted for the issue of patients switching therapies beyond progression suffer from limitations and may result in bias. The RPSFTM and IPE methods both assume a common treatment effect, meaning the timing of a treatment does not affect its benefit. This may not hold, as those who switched on to the experimental drug when they were at a more advanced stage may not have experienced the same benefit as those who received the treatment from randomisation. The RPSFTM and IPE methods also have problems when the comparator treatment used in the RCT is active (i.e. it prolongs survival) [19]. Alternatively, the two-stage method assumes there are no unmeasured confounders and no time-dependant confounding between the time of disease progression and time of switching. The ERG could not select a method they considered to be the most appropriate with any certainty.

Third, the cross-over methods adjusted for patients who switched from chemotherapy to crizotinib; however, it did not adjust for patients who switched from crizotinib to other therapies.

Fourth, the two treatment arms are imbalanced in terms of the numbers who went on to receive follow-up therapy as well as the therapies people went on to receive. The aim of the appraisal was to evaluate the effect of first-line therapies alone, but the switching rules and the imbalances in follow-on therapy make it difficult to assess whether the outcomes of the trials reflect this.

Finally, the company analysed the survival data by fitting one parametric model to the data, therefore making the assumption of Cox proportional hazards, which assumes a constant proportional treatment effect between the two treatment arms. However, an inspection of the log-cumulative hazard plots for OS, which plot log hazards against the log of time, appeared to show the curves diverging, i.e. they were not proportional. This divergence may be explained by the different ways in which the two treatments are administered, as patients received crizotinib continuously for a mean period of 23.7 months, whereas those in the chemotherapy arm received treatment in fixed cycles for a mean of 3.9 months. The NICE Decision Support Unit (DSU) guidance on the interpretation of proportional hazards outlines that an assumption of proportional hazards is reasonable when the majority of events have taken place in the trial, and in the absence of patient-level data, neither of which is true in this scenario [19]. The ERG therefore considered it more appropriate to fit separate parametric models for each treatment arm.

3.2.3 Trial Population

In addition to these issues, the survival data results for chemotherapy from PROFILE 1014 differ from those reported in alternative studies, with PROFILE 1014 patients reporting longer survival times. After cross-over adjustment, 1-year survival probabilities were around 65–75%, whereas 18-month survival probabilities were around 65–70% depending on the method of adjustment. These estimates can be compared with those reported in other trials of advanced non-squamous NSCLC: 1-year survival for pemetrexed chemotherapy + cisplatin was 50% and 18-month survival was 35%, [12] whereas 1-year survival for pemetrexed chemotherapy + carboplatin was 40% and 18-month survival was 20% [14]. Whether these differences can be attributed to the patients in PROFILE 1014 being unrepresentative of the population seen in practice or whether the ALK-positive population performs better than the general non-squamous NSCLC population is unclear.

3.2.4 Trial Comparators

The ERG had concerns that not all comparators used in clinical practice were included in the analysis. Specifically, pemetrexed maintenance therapy was not included as a comparator. Pemetrexed maintenance therapy, which involves giving patients receiving chemotherapy further cycles of therapy, is available via the CDF to all patients who received pemetrexed chemotherapy with cisplatin (30% of all patients). The ERG considered that the exclusion of maintenance therapy means an important comparator used in a significant proportion of patients was excluded from consideration.

3.3 Cost-Effectiveness Evidence

The model submitted by the company was a three health state model (Fig. 1), which the company referred to as a semi-Markov ‘area under the curve’ analysis. The three states were progression free, progressed and death. Transitions between states were not explicitly incorporated into the analysis using probabilities, but the proportion of patients in each state was determined using estimates of survival over time. Patients were assumed to receive either crizotinib or pemetrexed chemotherapy in combination with cisplatin or carboplatin as first-line treatments. Once patients progressed (as defined by RECIST), they were moved onto second-line treatment (docetaxel) and then third-line best supportive care before death. However, in the model, patients who received crizotinib therapy first line could continue therapy beyond progression if the treating clinician deemed them to still be benefitting from treatment.

Fig. 1
figure 1

Model structure (from figure 17, page 122 of the company submission)

Full size image

The duration of treatment for patients who discontinued therapy at progression was based on time to progression, which was calculated from an extrapolation of PFS data from PROFILE 1014. However, this extrapolation was adjusted using the baseline characteristics of patients from the Davis et al. [17] study, as the company claimed these patients were more representative of patients typically found in UK practice. The duration of treatment for patients who continued treatment beyond progression was also linked to time to progression, but patients treated with crizotinib were assumed to receive a further four cycles of therapy based on data from the PROFILE 1014 trial.

The population analysed in the economic model were patients with ALK-positive NSCLC of non-squamous histology, which is consistent with PROFILE 1014. The economic perspective was the NHS and Personal Social Services (PSS) in accordance with the NICE reference case. The time horizon was 15 years, which was stated to represent a lifetime horizon, and costs and benefits in the model were discounted at an annual rate of 3.5% as per the NICE reference case.

Patients receiving crizotinib who were in the progression-free state were assumed to derive a greater utility benefit than those in the progression-free state treated with chemotherapy based on the EQ-5D data taken from the PROFILE 1014 trial. It was assumed that those who progressed but were still being treated with crizotinib would receive a higher utility than those who moved onto second-line therapy. A utility value was therefore applied that was a mid-point between the pre-progression utility and the post-progression second-line therapy utility value. Additionally, a mid-point transitional utility was applied for one cycle when patients moved between health states to reflect a gradual worsening of quality of life.

The economic model assumed patients received a dose of two 250-mg tablets daily, in line with PROFILE 1014. The dosing of pemetrexed chemotherapy with cisplatin was based on body surface area, whereas pemetrexed with carboplatin was selected using a target under the curve method. Administration costs were included for pemetrexed but not for crizotinib on the basis that it is an oral therapy that does not require hospital admission. Costs were also included for NHS resource use associated with routine medical care, monitoring and supportive care and management of adverse events.

Costs for ALK testing were also included in the base-case analysis, with a testing strategy that involved providing all patients with non-squamous NSCLC with an ImmunoHistoChemistry (IHC) test, and giving those who scored +1 or +2 a confirmatory fluorescence in situ hybridisation (FISH) test, which is considered the gold standard for testing.

The company presented results for the base-case analysis based on the November 2013 cut of the PROFILE 1014 data and supplied results with and without an unapproved confidential patient access scheme (PAS) that was applied to the list price of crizotinib. Without the PAS, crizotinib had an incremental cost-effectiveness ratio (ICER) of greater than £50,000 per QALY gained. The probabilistic analysis showed that crizotinib had a 0% chance of being considered cost effective at this threshold.

3.4 Critique of the Cost-Effectiveness Evidence

The ERG highlighted a number of issues with the company’s cost-effectiveness evidence.

3.4.1 Model Validation

The ERG identified a substantial number of errors and potential inconsistencies in the company’s de novo cost-effectiveness model, which they identified and corrected for early in the STA process. Many of the errors were minor, but they raised questions concerning the internal validity of the model and could easily have been identified before submission.

3.4.2 Proportional Hazards and Immature Survival Data

The issues surrounding OS around the variation in second-line therapy received, the assumption of Cox proportional hazards, the adjustment made for cross-over and the immaturity of the data have a profound impact on the cost effectiveness of crizotinib. To attempt to correct for these issues, the ERG fitted two independent parametric survival functions to the Kaplan–Meier plots of PFS and OS for the crizotinib and pemetrexed chemotherapy arms of PROFILE 1014. The ERG attempted to select the most appropriate parametric functions based on the Akaike information criterion (AIC) and Bayesian information criterion (BIC) criteria of goodness of statistical fit and clinical plausibility. However, there was little difference between the AIC and BIC values despite the curves producing a wide range of values for OS gain, and clinical plausibility was difficult to assess because of the lack of data available to inform the analysis.

3.4.3 Treatment Duration

The ERG made adjustments to the estimation of patients’ time on treatment beyond progression. The company model assumed that patients received a further four cycles of crizotinib (conditional on the patient being alive) based on a median of 3.1 months reported in PROFILE 1014. The ERG considered the use of a median time on treatment to be inappropriate and had concerns regarding the implementation of time on treatment in the model. In its analysis, the ERG sought to correct for these issues and modelled time on treatment using the Kaplan–Meier discontinuation curves to which the ERG fitted a series of parametric survival curves. On the basis of the AIC and BIC criteria, the exponential curve was selected, which generated a mean time on crizotinib post progression of 11.73 months. This overcame the issues of using a median instead of a mean value, which is incorrect practice, and the implementation issues in the company model.

Similar issues were identified in the model relating to time on second-line docetaxel (which, in the company model, was informed from PFS data from PROFILE 1007). However, the ERG was unable to obtain a treatment discontinuation curve for second-line therapy, and whether second-line therapy would in fact be offered in practice was uncertain. Given these issues, the ERG removed second-line treatment from the ERG base case, and patients were assumed to move directly onto best supportive care.

3.4.4 Administration Costs

The company’s model assumed crizotinib would not accrue any ongoing administration costs as it is an oral therapy that does not require hospital admission. The ERG questioned this but found the treatment of administration costs in previous NICE appraisals of oral therapies was inconsistent: the appraisal of oral nintedanib for previously treated, locally advanced, metastatic or locally recurrent NSCLC [20] did not include administration costs, whereas the appraisal of crizotinib for patients with previously treated ALK-positive NSCLC did [1]. The ERG considered the previous crizotinib appraisal to be the most relevant and—reflecting this—applied an administration cost of £163 in each cycle of the model.

3.4.5 Additional Issues

  1. 1.

    In addition to the principal issues outlined above, the ERG also identified a number of other issues: the choice of comparator therapies was a concern, particularly the failure to include pemetrexed maintenance therapy and crizotinib as a second-line therapy, which, while not included in the final NICE scope, are available via the CDF and are therefore representative of current UK practice.

  2. 2.

    The utility values for pre-progression pemetrexed chemotherapy patients who have completed treatment were likely to be underestimated because of limitations in the follow-up of these patients during the PROFILE 1014 trial.

  3. 3.

    The assumption that patients whose disease progressed would, for one cycle, experience a utility that was midway between the pre-progression and post-progression utility led to the potential double counting of utility.

  4. 4.

    The assumption, based on PROFILE 1014, that pemetrexed chemotherapy patients would receive a six-cycle regimen of chemotherapy differed from advice provided by the clinical advisors to the ERG, who stated that, although the aim may be to offer six cycles, this is not always achievable and patients typically receive four cycles of therapy. Furthermore, the summary of product characteristics for pemetrexed in combination with platinum-based chemotherapy allows for between four and six cycles of chemotherapy [21].

  5. 5.

    The company model assumed no drug wastage for either crizotinib or comparator therapies. The ERG considered this inappropriate and thought it was likely to underestimate the ICER because of the higher costs of crizotinib.

  6. 6.

    The ERG considered that the company had underestimated the cost of ALK testing and identified an alternative cost based on a large survey of UK testing facilities, which estimated much higher unit costs [22].

3.5 Conclusions of the ERG’s Review

PROFILE 1014 showed crizotinib had a significant benefit in terms of median PFS compared with pemetrexed chemotherapy (HR 0.45; 95% CI 0.35–0.60). The OS data were immature and median OS was not reached, with an unadjusted HR for death with crizotinib of 0.821 (95% CI 0.536–1.255). A number of methods of adjustment for cross-over were implemented, with HRs ranging from 0.571 to 0.674. However, not all were statistically significant, and there was substantial cross-over, which was not completely accounted for. Furthermore, evidence suggests that the assumption of Cox proportional hazards to model survival does not hold. Uncertainties also remain surrounding the clinical characteristics and prognosis of a typical population of patients with advanced non-squamous ALK-positive NSCLC and the comparability of the populations in PROFILE 1014 with the UK ALK-positive NSCLC population. The economic model contained a number of errors and potentially over-stated the cost effectiveness of crizotinib because of the way time spent on treatment, administration costs and ALK testing costs were implemented.

The ERG base-case analysis with a PAS applied found that, in a variety of scenarios, crizotinib could not be considered a cost-effective therapy. However, these results were highly uncertain because of the immaturity of the OS data, with different combinations of parametric functions used to model survival producing a wide range of ICER values.

4 National Institute for Health and Care Excellence (NICE) Guidance

4.1 Preliminary Guidance

The NICE Appraisal Committee considered whether crizotinib should be given NHS approval and whether it met the NICE end-of-life criteria. On balance, the committee concluded that crizotinib is clinically effective, increases PFS and likely increases OS compared with pemetrexed chemotherapy plus either cisplatin or carboplatin in people with ALK-positive NSCLC. However, the committee believed that the results presented in the ERG analysis most closely reflected the committee’s preferred assumptions, while also acknowledging that these results likely overestimated the ICER. The preliminary NICE recommendation was that, although crizotinib may meet the end-of-life criteria, it is not a cost-effective option for the first-line treatment of ALK-positive non-squamous NSCLC, even with the PAS applied.

4.1.1 Company’s Response to Preliminary Guidance

The company rejected the ERG’s argument regarding the utility value used for patients receiving pre-progression pemetrexed chemotherapy and presented new data to support a lower utility value, claiming that those receiving crizotinib experience a greater reduction in symptom burden. The company also revised upward the utility value used for patients continuing to receive crizotinib beyond progression, presenting new EQ-5D data to support the change. The company rejected the ERG’s estimate of ALK testing costs and instead pointed to information provided by a pathologist at the appraisal committee meeting who stated that an IHC test would likely cost £50–100. The company therefore assumed a midpoint of £75, which increased the cost per positively identified patient from the company’s original base case to £2380.

The company accepted that crizotinib may incur administration costs but reiterated the inconsistency in related NICE submissions and objected to the value used by the ERG. The company stated that, given the similarities between crizotinib and the recently appraised ceritinib [23], it would be appropriate to include the same administration cost for crizotinib that was accepted by NICE in this instance. Therefore, the company included a dispensary cost of £14.40 in each cycle of the model. The company also increased the PAS discount, which was applied to the list price of crizotinib.

The company revised the ERG’s changes to time on treatment, which were based on the survival curve for time on treatment from PROFILE 1014, by adjusting time on treatment using the ‘real-world’ patient characteristics identified in Davis et al. [17]. This ensured time on treatment was estimated in a way consistent with the modelling of PFS and OS.

The company considered the committee’s preferred analysis underestimated the OS gains from crizotinib as it implied a mean OS gain of just 0.8 months. The company stated that the OS gain would be at least 7.1 months, as this value was accepted by the committee in the second-line appraisal of crizotinib for NSCLC, and that the value of 0.8 months contradicted the opinion of clinical experts, who claimed a benefit of 7.1–13 months. The company supplied new analyses in which the proportional hazards assumption was relaxed but also included results where proportional hazards were assumed, as they believed the tests they had conducted indicated the proportional hazards assumption was appropriate. The curves were then adjusted using data from Davis et al. [17] to ensure the curves were more reflective of patients found in practice.

The company incorporated all of the above changes and used a range of parametric functions to model PFS and OS, assuming proportional hazards in some instances and modelling the curves independently in others. The company then excluded all scenarios that generated an OS gain of <7.1 months, because they deemed this clinically implausible, and all scenarios where the mean life expectancy while receiving pemetrexed chemotherapy was >24 months, as this would result in the end-of-life criteria no longer holding. This left 11 possible combinations of independent curves and four possible choices of parametric functions if proportional hazards were assumed, with ICERs ranging from £31,708 to £49,186 per QALY. The company further excluded curves that produced what they deemed implausibly large OS estimates for crizotinib and those that produced mean OS values greater than the median OS values, as they claimed this contradicted clinical expert opinion. A mean OS greater than the median OS would imply no tail in the survival curve, which seemed unlikely. This resulted in two possible curve pairings when separate curves were modelled and four when proportional hazards were assumed. The company’s preferred independent parametric curve analysis generated an ICER of £47,921 per QALY, and their preferred model using the proportional hazards assumption produced an ICER of £49,186 per QALY when the PAS was applied.

4.1.2 ERG Critique of the Response Submitted by the Company

Table 1 summarises the assumptions used in each version of the model, including the ERG’s re-analysis following the company’s appraisal consultation document (ACD). The ERG incorporated the new analysis presented by the company regarding time on treatment but rejected a number of other assumptions, including revisions made to administration costs and ALK testing costs and revisions to utility values for pre-progression pemetrexed patients following discontinuation of treatment and crizotinib patients continuing to receive treatment beyond progression.

Table 1 Summary of model assumptions
Full size table

Importantly, the ERG also rejected the company’s defence of the use of proportional hazards and highlighted that the assumption places a constraint on the data. The DSU document on this methods issue suggests that, where individual patient-level data are available, it is generally more appropriate to fit separate curves [19]. This is because it is generally a more conservative position and avoids the potential for bias resulting from imposing the proportional hazards assumption. Similarly, the ERG rejected the company’s claim that the independent covariate stratification of survival curves is inappropriate and highlighted that this method required fewer assumptions than the alternative method adopted by the company.

In interpreting the OS data and the criteria used by the company to select from all possible survival curves, the ERG also highlighted a number of issues. The company argued that mean OS for pemetrexed chemotherapy should not exceed 24 months as the committee had accepted that crizotinib met the end-of-life criteria. However, the ERG noted this was not based on evidence but simply the opinion of the committee on the basis of the information they had at the time. The company also argued that mean OS gain should exceed 7.1 months, as this was the committee’s opinion of the OS gains when crizotinib was given as a second-line therapy in PROFILE 1007. The ERG considered this assumption reasonable given evidence on the relative magnitude of PFS gains across PROFILE 1007 and 1014. However, this assumption was not entirely unproblematic because patients in second-line settings received therapy for different durations, the comparator was docetaxel rather than pemetrexed and the characteristics of patients who receive second-line treatment are likely to differ from those of patients who receive first-line therapy. The ERG also agreed with the company that mean OS should be plausible given expectations on the survival of patients with advanced ALK-positive NSCLC. However, given the lack of evidence on OS for patients with ALK-positive NSCLC, the ERG considered it difficult to judge what should be considered plausible. Therefore, the ERG considered a wider range of curves plausible than did the company in their response.

When proportional hazards were assumed, the ERGs revised base case using the updated PAS generated an ICER of £58,029 per QALY gained. When the assumption of proportional hazards was relaxed, assuming the mean OS gain exceeded 7.1 months, the estimated ICER was found to lie between £35,972 and £57,035 per QALY. The ICER assuming the company’s preferred curve selection was £55,131 per QALY.

4.2 Final NICE Guidance

Following the consultation on the preliminary guidance, the NICE Appraisal Committee released the following final guidance to the NHS (TA406) [2]:

“Crizotinib is recommended, within its marketing authorisation, as an option for untreated anaplastic lymphoma kinase-positive advanced non-small-cell lung cancer in adults. The drug is recommended only if the company provides it with the discount agreed in the patient access scheme.”

The committee agreed with many of the ERG’s adjustments but concluded that the company’s assumed utility values, administration costs and ALK testing costs were valid. It therefore concluded that the company’s independent parametric curve analysis, which generated an ICER of £47,291 per QALY gained, most closely reflected the committee’s preferred assumptions and noted that the ICERs for other alternative curves were similar. However, the committee acknowledged that the uncertainty surrounding the cost effectiveness of crizotinib was extensive and that the company’s analysis did not incorporate all of the committee’s preferred assumptions.

5 ERG Conclusion

This STA highlighted the important role the ERG plays in the appraisal process, particularly the role it can play in checking for errors, identifying sub-optimal methods and questioning the validity of assumptions. The ERG was able to identify a significant number of calculation errors in the submitted company model, which brought into question the internal validity of the analysis.

This STA also highlighted a number of general issues potentially important to future appraisals. First, as with many cancer models, OS was a key parameter in estimating cost effectiveness. However, the evidence provided in support of OS gains was subject to a number of limitations, many of which arose directly as a result of the design of the PROFILE 1014 trial. The most significant of these issues related to the immature dataset, which created significant uncertainty regarding the magnitude of OS gains, and—in the opinion of the ERG—meant OS could not be meaningfully estimated. This uncertainty led to a wide range of potential estimates of cost effectiveness, both above and below threshold values.

Second, crizotinib—unlike chemotherapy—is not given in a fixed treatment regimen, and patients are treated up to and even beyond disease progression. As such, time on treatment is an important determinant of overall drug acquisition costs and, consequently, estimated cost effectiveness. However, the ERG was critical of the approach taken by the company to model time on treatment. The ERG made significant revisions to the economic model to rectify these issues, which had a significant impact on the estimated ICER.

Finally, this STA highlighted past inconsistencies in previous NICE appraisals regarding the application of administration costs for oral therapies. While not instrumental in the current analysis, a more consistent approach by NICE committees would provide guidance for future submission and help avoid unnecessary confusion.