Introduction

One of the major cost drivers in the European healthcare systems is the pharmaceutical ‘innovation’; even considered more relevant than demographics [1]. At the same time, it is also recognized as one of the main contributors to the improvement of the population health status [2].

According to the most recent study from the Organization for Economic Co-operation and Development (OECD) [3], pharmaceutical expenditure accounts for a percentage that range between 11.4% (UK) and 19.1% (Spain) of total healthcare expenditure across the five largest European drug markets (France, Germany, Italy, Spain, and the UK). Specifically, the oncological and hematological drugs are leading the budget impact related to pharmaceutical innovation. The impact is driven by the expansion of multiple new indications normally based on a molecular definition that restricts the population to be treated and the drug ends up being designated as orphan-like medicines [4]. As estimated by a recent study [5, 6], the healthcare expenditures on cancer in the European Union member states represented roughly 6% of total healthcare expenditures. The steady increase of oncology costs is aligned with the disease increasing incidence, the progressive reduction of mortality as well as high prices, in contrast with the less robust evidence data on outcomes [7].

A recent study [5] estimated that 40% of the new orphan drugs authorized in Europe are related to rare neoplastic disorders, and compare to non-oncologic indications, the authorization is received at more advance stages of the clinical development and recognizing a higher potential clinical benefit. From 2009 to 2013, only 35% the 68 oncology indications approved by the European Medicines Agency (EMA) showed a significant prolongation of survival and only 10% showed an improvement in quality of life at the time of market approval. The magnitude of the benefit on overall survival ranged from 1.0 to 5.8 months (median 2.7 months). In the subsequent post marketing period (3,3 years later) there was evidence for extension of life in 7% of the previous authorizations and reported benefit on quality of life in 11% of the cases [8].

Occasionally, when the drug can cover clinical unmet needs with poor prognosis, the regulators trend to accept less and poorer evidence and include especial approvals, such as conditional approval related to further of adequate risk benefit rate in real world, after commercialization, or approval under exceptional circumstances when this may not be achieved, in order to ensure an earlier access to market. As described recently [9] the potential benefit of patients’ early access to new medicines in areas of high unmet medical need, and based on initial data only, have relevant implications in terms of medical and economic costs (opportunity costs of using alternative more efficient treatments available for patients). Several initiatives have been developed in Europe to address these challenges of funding premium priced products related to clear medical unmet needs but with limited evidence [10]. New access management models of these drugs have been promoted across Europe recently, especially for advance therapies, orphan drugs and medicines for cancer, and including innovative access schemes as value-based pricing, conditional reimbursement schemes or risk sharing approaches [11]. Despite the smooth increase of these new access schemes, the number of outcome-based solutions is still very limited being the lack of a systematic and harmonized value assessment methodology one of the main limitations [12].

Beyond the general awareness among healthcare authorities to ensure “value for money”, or the link between price and social or clinical value of the pharmaceutical innovation [13], the reimbursement process and value assessment of drugs is still an open debate in Europe [14]. Several methods have been developed to assess the value of drugs and set meaningful prices affordable to health-care systems [15]. These methods are normally based on the clinical benefits of the drugs and partially on value-based pricing (e.g. cost-effectiveness analysis). However, there is neither a consensus nor a European harmonization related to drug-pricing systems and, based on a comparative international policy analysis, value-based approaches to determine the prices of innovative products are diverse [16]: including the implicit clinical value of the quality-adjusted life-years (QALYS), mainly used in UK, Sweden or Australia, or the value classification based on innovation scales (used in France, Italy, Germany, Austria, Canada or Japan) [17]. Normally new drugs classified as an innovative medicine are reimbursed at a higher price than the current therapeutic alternatives; although the amount, type and methodology to set the premium is normally veiled by the healthcare authorities [7].

In Europe, the European Network for Health Technology Assessment (EUnetHTA) was set up in 2006 and includes all EU Member States to provide strategic guidance and policy orientation on the assessment of health technologies (including drugs), by developing policy papers and discussing areas of potential collaboration. During the last decade the network has focused the efforts on the development of common methodologies, piloting and producing joint early dialogues and Health Technology Assessment (HTA) joint assessment reports, as well as developing and maintaining common tools [18]. One of the most relevant tools developed by the network is the HTA Core Model for Rapid Relative Effectiveness Assessment (REA) [19]. The Model is a methodological framework for the collaborative production and sharing of HTA information that defines the content elements to be considered in an HTA and it enables standardized assessment reporting across Europe. Because of the objective of the framework is to share commonly required elements of information, only information that is considered both important and transferable is collected. The model brings a standardized framework that allows a common comparison of the drivers that lead pricing and reimbursement decisions among different European authorities.

HTA Network approach is focused on technical aspects while methods to support alignment of decisions with the compassionate impetus of healthcare systems is lacking [20]. In many countries, healthcare authorities are including a broader approach to assess the pharmaceutical products (especially in therapeutic areas like oncology and rare diseases) [21]. EVIDEM [22] (Evidence and Value Impact on Decision Making) was developed based on an analysis of the foundations of healthcare systems, the reasoning underlying decisions and fair processes, and has become a reference for multicriteria decision approaches in this setting. It includes the concept of reflective multicriteria assuming decision-makers are guided by a generic interpretative frame rooted in the baseline values of the healthcare systems, drawing on several domains of knowledge including healthcare ethics, evidenced-based medicine, health economics or health technology assessment approaches. A multicriteria analysis provides an effective approach to increase the legitimacy of decisions. Beyond a tool, reflective multicriteria pioneered by EVIDEM is geared to transform the vision of the value of healthcare interventions and how they might contribute to relevant, equitable and sustainable healthcare systems. EVIDEM can be used to compare various healthcare interventions and prioritize its implementation using a performance matrix underpinned in the several dimensions and criteria defined by the framework [20].

EVIDEM criteria overlap with EUnetHTA-core except for 4 non-contextual and 3 social criteria, which are absent or partially included in the EUnetHTA framework. Inversely, 2 EUnetHTA criteria are absent in the EVIDEM framework (Table 1).

Table 1 EVIDEM and EUnetHTA criteria correspondence
Full size table
Fig. 1
figure 1

EVIDEM alignment score by dimension

Full size image

Although multicriteria EVIDEM approach is now applied by several healthcare authorities [23], especially when the social and medical perception of need requires a more holistic assessment framework to support the payer’s decision, a formal and systematic comparison of EUnetHTA’s and EVIDEM’s methodological frameworks and whether European health technology assessment bodies (HTAb) are aligned with the EVIDEM methodology standards is lacking [24]. Since EUnetHTA and EVIDEM frameworks differ in a substantial number of criteria, it is of interest to know the extent of compliance with EVIDEM framework of HTAs as an additional way to explore potential reasons of assessment discrepancies. Despite the compliance of 37 European HTAb with using the supportive criteria for decision making proposed in the EUnetHTA-core framework has been previously reported [18], whether these HTAb do also comply with the wider EVIDEM multicriteria is unknown.

Thus, the main aim of this study is to describe the uptake and use of multicriteria approaches to appraise drug value by 37 European HTAb, using EUnetHTA and EVIDEM as reference frameworks.

Methods

A quantitative validation of the degree of alignment with the EUnetHTA’s standard framework of 37 European HTAb from 28 countries was done, based on a previous qualitative analysis conducted by the European Commission [18] and an additional thorough search of websites, publications and reports of HTAb. The criteria used for appraisal by the different HTAb were identified and classified, and the matching with the criteria described in the EVIDEM methodological framework were described semi-quantitatively using a heatmap of alignment.

The items reported included those criteria in the HTA Core Model, namely: Relative Effectiveness Assessment (REA) of pharmaceuticals, EUnetHTA methodological guidelines [25] and procedure descriptions [26, 27]. Also, criteria related to the types of technologies assessed, the administrative level (national, regional, institutional) and the formal background (legislation, formal agreement, internal guideline) of certain methodological requirements were also used.

An updated version of EVIDEM framework (v.10) was analyzed in order to assess how the dimensions and criteria included in the EUnetHTA methodological framework fitted within the EVIDEM’s methodological framework.

The EVIDEM framework includes 13 non-contextual dimensions and 6 contextual dimensions (Table 1). The non-contextual dimensions (EVIDEM core-model) include normative aspects combined with the description of the technical knowledge available. Contextual dimensions tailor the framework to the context of decision-making.

Fig. 2
figure 2

EVIDEM alignment score by HTAb. HTAb: Health Technology Assessment body. SESCS: Servicio de Evaluación del Servicio Canario de Salud; SBU: Swedish Agency for Health Technology Assessment and Assessment of Social Services; HVB: Hauptverband der Österreichischen Sozialversicherungsträger; KCE: Belgian Health Care Knowledge Centre; INAMI-RIZIV: National Institute for Health and Disability Insurance; NCPHA: National Center of Public Health and Analyses; SUKL: State Institute for Drug Control; FIMEA: Finnish Medicines Agency; HAS: Haute Autorité de Santé; IQWiG: Institute for Quality and Efficiency in Health Care; OGYÉI: National Institute of Pharmacy and Nutrition; HIQA: Health Information and Quality Authority; NCPE: National Centre for Pharmacoeconomics; AIFA: Italian Medicines Agency; UCSC: Università Cattolica del Sacro Cuore; ZIN: Zorginstituut Nederland; AOTMiT: Agencja Oceny Technologii Medycznych i Taryfikacji; INFARMED: National Authority of Medicines and Health Products; UHIF: Union Health Insurance Fund; AEMPS: Agencia Española de Medicamentos y Productos Sanitarios; AETSA: Agencia de Evaluación de Tecnologías Sanitarias de Andalucía; AQUAS: Agència de Qualitat i Avaluació Sanitàries de Catalunya; TLV: Dental and Pharmaceutical Benefits Agency; NICE: National Institute for Health and Care Excellence; SMC: Scottish Medicines Consortium; AWTTC: All Wales Therapeutics and Toxicology Centre; NIPH: Norwegian Institute of Public Health; NoMA: Norwegian Medicines Agency; LBI-HTA: Ludwig Boltzmann Institute of Health Technology Assessment; AAZ: Agency for Quality and Accreditation in Health Care and Social Welfare; UoT: University of Tartu; G-BA: Gemeinsamer Bundesausschuss; RER: Regione Emilia-Romagna; NVD: The National Health Service; MOH: Ministry of Health Malta; MOH: Ministry of Health Slovakia; JAZMP: Agency for Medicinal Products and Medical Devices

Full size image

An HTAb heatmap was developed, where heatmap categories were generated for each EVIDEM’s dimension using as a source the mentioned criteria in the EUnetHTA’s report [18], webs and reports available from the different HTAb analyzed (supplementary file). The contribution (weight) of each mentioned criterion to the final heatmap’s score by dimension was equal and proportioned to the number of criteria by dimension described in Table 1. Only when the mentioned criteria were not fully aligned with the EVIDEM’s criteria, the mention was weighted by 50% of contribution:

$$ Heat\ Score=\left[\left(\sum \# criteria\ mentioned\ by\ dimension\right)/\left(\sum \# total\ criteria\ by\ dimension\right)\right]\ast 100 $$

Descriptive statistics (mean, standard deviation, percentiles) were used to summarize the data and 95% confidence interval for each dimension and HTAb Figs. 1 and 2, and conditional formatting was used to automatically color code each cell using Microsoft Excel (Windows Office 365) so that graded colors were used with green coding for highest alignment (100) and red for lowest alignment (0). Values outside the interquartile range were used to assess alignment with the EVIDEM’s model [28]. HTAbs with and average heat score above the 75th percentile were considered “Aligned” with the EVIDEM model, and those below 25th percentile were considered “Misaligned”. The rest were classified as “Medium” in terms of EVIDEM model’s alignment.

Results

Most of the non-contextual criteria of EVIDEM are overlapped with the core model of EUnetHTA, except for the type of prevention benefits, non-medical comparative cost consequences, systematic use of expert consensus and use of clinical guidelines to define state-of-the-art, which are not or partially included on the EUnetHTA’s framework (Table 1). Regarding contextual criteria, the assessment of the system capacity and appropriate use of intervention is the most aligned criteria between both frameworks, followed by the political/historical/cultural context assessment, the mandate and scope of the healthcare system, the special population priorities and equity on access criteria. Other social criteria (stakeholders management, conflict of interest assessment or environmental impact assessment) are not reflected in the EUnetHTA’s framework. A systematic general description of the assessed technology and the request of clarification of the assessment process (guidelines and legislation) are key aspects considered by the EUnetHTA analysis that are not explicitly included in the EVIDEM framework.

Most of the non-contextual dimensions (such as disease severity, size of affected population, unmet needs, comparative effectiveness, comparative safety/tolerability or type of therapeutic benefit) show consistently high rates among the HTAb (mean above 85% and standard deviation below 16%); other non-contextual dimensions (type of preventive benefit, comparative non-medical costs, expert consensus) and relevant contextual dimensions (such as population priorities, common goal, environmental impact, system capacity or political/historical/cultural context) are systematically rated low (Table 2).

Table 2 HTAb heatmap of coincidence with EVIDEM framework
Full size table

All HTAb address consistently the health problem and current use of technology, technical characteristics, clinical effectiveness and safety criteria, which are included in the EUnetHTA core model. Choices on comparator, methodology of comparison, endpoints and methods of evidence search and synthesis, are consistently aligned. On the contrary, non-clinical domains, assessment approaches, methodology, modelling algorithms and data are consistently dis-aligned (Table 3).

Table 3 EVIDEM heat score by dimension
Full size table

None of the local HTAb had high heat scores with regards to the use of contextual criteria (Table 2). Considering alignment to EVIDEM-driven assessment framework, three patterns of HTAs emerged: “Aligned”, “Medium” and “Misaligned” (Table 4).

Table 4 EVIDEM heat score by HTAb
Full size table

Nine agencies in Bulgary, Hungary, Italy, Malta, Slovakia, Spain, Sweden and UK showed an “Aligned” profile (average heat score above the 75th percentile) with a consistent alignment on non-contextual dimensions and significantly high alignment scores on political/historical/cultural context, system capacity and appropriate use of the intervention.

Most HTAb (19/37; 51%) showed a “Medium” alignment profile. Alignment rates for non-contextual criteria were mainly high (e.g. patient perceived health and quality of evidence dimensions) in these HTAb, and also other contextual dimensions (such as the mandate and scope of the healthcare system, system capacity and appropriate use of the intervention) were rated high. On the contrary, population priorities and access dimension systematically rated below 50%, except for AEMPS.

In 9/37 (24%) HTAb the profile was considered “Misaligned”, with low scores on alignment (average score below 25th percentile) in dimensions such as patients perceived health methods, cost-consequence analysis (cost of intervention and other medical costs) and quality of the evidence. Considering the non-contextual perspective, the German G-BA and the NIPH in Norway show high scores focused and limited to the technical comparison of alternatives (effectiveness, safety and quality of evidence assessment). From the contextual perspective, all the HTAb of this group rated low on the mandate and scope of the healthcare system, population priorities on access, system capacity, appropriate use of the interventions and political/historical/cultural context.

HTAb with explicit responsibilities in providing specific advice on pricing and reimbursement (normally regional agencies in countries with more than one HTAb in place, such as Belgian KCE, German IQWIG, Irish HIQA, Italian UCSC, Portuguese INFARMA, Slovakian UHIF, Spanish SESCS or Swedish SBU) showed higher and similar scores on contextual and non-contextual dimensions.

Discussion and conclusions

The alignment between EVIDEM and EUnetHTA methodological frameworks is consistently high, especially when assessing domains related to health problem description, current use of the technology, technical characteristics, clinical effectiveness, and safety. However, other non-contextual dimensions of the EVIDEM framework and the EUnetHTA core model are consistently misaligned.

The main EUnetHTA core model criteria, such as clinical effectiveness, safety conditions, health problem description and current use of technology; are consistently addressed by all HTAb. As previously reported [18] the institutions go only partially beyond these criteria and it is normally dependent on the topic of assessment. For those European HTAb directly advising on price and reimbursement decisions, the reported criteria used to support their decisions show a more balanced alignment between both methodological approaches. That conclusion could explain why in many cases, the subnational HTAb in those countries with multiple agencies, are the ones showing a balanced profile among contextual and non-contextual dimensions.

EVIDEM provides a generic interpretive frame (MCDA – Multi-Criteria Decision Analysis – reflective grid) that can be used to elicit individual values and facilitate deliberations through a common structure that includes interpretive scores (quantitative criteria), qualitative impacts (qualitative criteria) as well as narrative comments (all criteria) [21]. EVIDEM framework was designed to minimize the limitations of the deliberation process by ensuring that: generic assessment criteria (either quantitative or qualitative) are included; evidence relevant to each criterion is made available through an efficient synthesis methodology; and face validity is checked at each step of the process (weights, scores and corresponding narratives, aggregated measures). EVIDEM framework is sufficiently flexible to be adapted to the local assessment context, although it also requires consistency in the identification of a set of criteria, scoring scale and weights when assessing a broad range of competing interventions in a specific local context [29, 30].

A holistic approach is required to consistently assess the social and medical needs to support payer’s decision on prices and reimbursement conditions of certain drugs, such as disruptive innovations or orphan drugs, broadening the need of using EVIDEM-like contextual assessment tools by European HTAb.