Introduction

In health technology and drug assessments conducted all over the industrialized world, quality of life (QoL) outcomes are increasingly gaining importance. Today, clinical studies routinely include QoL endpoints. Accounting for the growing importance of the QoL concept, in 2010, German parliament decided to make QoL one of the four main benefit criteria in its new system of drug benefit assessment.

Intending to decrease costs for pharmaceuticals and to set incentives for innovation, Germany revised prizing regulations for new drugs with the Act on the Reform of the Market for Medicinal Products (AMNOG) [1]. With the new legislation, German parliament linked reimbursement of drugs by statutory health insurance funds to evidence of added benefit, putting an end to free price setting. Many other European countries, like France or Sweden, had already introduced similar systems earlier [2].

Since January 2011, pharmaceutical companies are obliged to submit dossiers presenting their drug’s additional benefit over comparative standard treatment. The regulation applies to all reimbursable medicinal products with new or newly combined ingredients marketed in Germany and when new indications are authorized for these products. Comparative treatment is specified by the Federal Joint Committee (G-BA) with respect to the state-of-the-art medical treatment. The committee is the highest decision-making body in Germany’s system of joint self-government of the healthcare professions and decides which medical services are reimbursed by the statutory health insurance funds. Subsequently, these dossiers are scientifically evaluated by the Institute for Quality and Efficiency in Health Care (IQWiG) or, in case of drugs for orphan diseases, by the G-BA. Following a hearing process in which manufacturers may reply to the dossier evaluation and experts and federations contribute their perspectives, the G-BA decides on the drug’s additional benefit. Based on this decision, companies negotiate drug prices with the National Association of Statutory Health Insurance Funds (GKV-SV). If no additional benefit is demonstrated, the product is subject to reference pricing [3, 4]. Thus, decisions on additional benefit have a far-reaching budgetary effect on the German pharmaceutical landscape. As many neighboring countries use the German market for external reference pricing, these effects are likely to be of international scope [5].

Morbidity, mortality, and QoL endpoints as well as side effects constitute the new drug assessments’ main criteria. Yet QoL still is a relatively new concept, and there is no single generally accepted definition. Furthermore, neither AMNOG nor the accompanying regulation of the German ministry of health defines QoL [1, 6]. Hence, the aim of this study was to answer the following question: How is QoL defined in early benefit assessment (EBA) and which role does it play?

To trace this definition and to assess the concept’s relative importance within Germany’s benefit assessment, a qualitative content analysis (QCA) [7] was conducted. Additionally, quantitative data on the final G-BA decisions were collected. This paper focuses on findings that are related to the role and definition of QoL. Differing from other approaches, our analysis focuses on an individual endpoint and applies systematic qualitative methodology [3, 8–10].

Methods

Qualitative content analysis

All benefit assessments completed by the end of 2013 were processed (n = 66). Manufacturers’ dossiers (document “M1,” the summary of the dossier), the IQWiG’s or the G-BA’s evaluations of the dossiers, G-BA decisions (“main justifications”), and the protocols of the oral hearings were imported to the software “MaxQDA” and searched for the term “QoL” and synonyms.

Accounting for the fact that this study’s main data source consisted of texts, a “qualitative content analysis” (QCA) [7] was conducted. QCA highlights the importance of interpretation rules when dealing with information that is conveyed through language. The aim of this approach is to condense the text source to analytical categories (“codes”), which account for typical references and structures. It is an open approach; i.e., analytical categories are created during reduction, paraphrasing, and abstraction of text material; new findings are fed back into the analytical process (“iterative loop”) [7, 11, 12]. To ensure qualitative openness to the material, a pre-defined framework to organize the codes was not applied. Instead, codes were chronologically created during QCA in the sequence illustrated in the “Appendix”, Table 5.

Quantitative analysis

Additionally, quantitative data on the overall results of the benefit assessments were collected.

The G-BA exemplifies its resolution in detail (“main justifications”) and states which endpoints were pivotal regarding the declaration of additional drug benefit. It was determined how often outcomes regarding QoL, morbidity, mortality, and/or side effects were pivotal for the decision. To account for every single G-BA decision, pivotal endpoints, as well as extent and certainty of benefit, were surveyed on subgroup level.

Additionally, it was assessed how many dossiers included QoL data, how many dossiers were incomplete, in how many benefit assessments the manufacturers did not submit a dossier, and how many drugs targeted orphan diseases (Table 1).

Table 1 Examined variables on AMNOG drug benefit assessments
Full size table

Search terms and text extraction

The following search terms were employed: *lebensqualität* (German for QoL), *lq* (German abbreviation of Qol), *life*, *qol* und *ql*.

This study aimed to determine how different notions of the term QoL were communicated by the different stakeholders. Hence, every single reference to QoL or its synonyms was analyzed. Mere mentionings of the term “QoL” in tables of content or in lists of abbreviations or references were excluded from analysis. Relevant search hits were exported in whole sections (“excerpts”) rather than in individual sentences.

Identifying key content and coding

The resulting excerpts were reduced, summarized, and paraphrased independently by two researchers (CB and DL), omitting irrelevant passages. In a subsequent comparison of each pair of quintessential extractions of the original text passages (“reductions”), key content was discussed and identified. Consolidating both reductions, a consented reduced text version (“synthesis”) was created.

On the basis of these syntheses, the material was coded. Coding is a process of analytical categorization; i.e., codes (single words or short phrases) were created representing the respective text segment. For example, some codes referred to how QoL was (sometimes implicitly) defined. Others represented more general references to the concept’s overall importance or its relation to other endpoints like mortality or morbidity.

Again accounting for the growing degree of abstraction and interpretation, coding was performed by two researchers (CB and DL) independently reviewing the results. Once a researcher had completed the coding of a given benefit assessment procedure, the colleague would review the results and propose amendments or changes. New codes would be added to a coding list which included a description and an example of the type of text segment the respective code should be assigned to. Coding was done in a repetitive loop with a “learning” system of codes.

Finally, summarizing essays were written for each code compiling typical text fragments, summing up findings in a synopsis of the consented reductions of the original text passages.

Results

Quantitative results

In 51 of the first 66 EBAs, dossiers were judged complete. In eight cases, the documents were rated incomplete by the G-BA, because of deviations from the appropriate comparative therapy; in seven cases, the manufacturers did not submit any dossier. In eight cases, drugs for orphan diseases were assessed.

The G-BA often decided to split patient groups. It effectively decided in 114 cases or subgroups on benefit. As shown in Table 2, in 69 cases additional benefit was rated “not demonstrated” (60.5 %), and in 23 cases (20.2 %) “minor.” In 9 cases (7.9 %), it was rated “not quantifiable,” and in 12 (10.5 %) “considerable.” The highest possible extent of benefit (“substantial”) was not declared in any of the EBA’s.

Table 2 Extent of additional benefit according to G-BA (decisions in 114 subgroups in 66 early benefit assessments; January 2011–December 2013)
Full size table

As shown in Table 3, in 69 cases (60.5 %) the committee rated the certainty of benefit as “not demonstrated.” “Proof” (high certainty) for additional benefit was seen in 5 cases only. The categories “hint” (weakest certainty) or “indication” (medium certainty) were declared 14 times each (12.3 %).

Table 3 Certainty of additional benefit according to G-BA (decisions in 114 subgroups in 66 early benefit assessments; January 2011–December 2013)
Full size table

As explained above, the G-BA specifies the pivotal endpoints and explains the benefit resolution in its document “main justifications.” Among the 114 subgroup decisions, the G-BA only twice explicitly quoted QoL outcomes as crucial for the declaration of additional benefit (Crizotinib, Ivacaftor). Morbidity results were specified as crucial 21 times, data on side effects 17 times, and mortality results 12 times. As several endpoints could be declared pivotal for each subgroup, the G-BA identified 52 pivotal outcomes for the declaration of benefit in 44 subgroups. As shown in Table 2, in 70 subgroups there was no additional or a lower benefit.

Fifteen of the 59 (25.42 %) submitted dossiers did not include QoL data.

Qualitative results

The search led to 18,630 hits in the documents. Excluding mere mentions of the term “QoL” in tables of content or in lists of abbreviations or references, 1769 of these hits were selected for heuristic analysis. On this basis, 44 codes were created (complete list in the “Appendix”, Table 5).

Role of QoL outcomes in the early benefit assessment

Although QoL outcomes were pivotal for the declaration of additional benefit only twice and 15 of 59 submitted dossiers did not include any QoL data, IQWiG and G-BA frequently highlighted the need for QoL data in order to make adequate drug assessments (Elvitegravir; Aliskiren-Amlodipin; Brentuximab Vedotin; Aflibercept, metastatic colorectal cancer). Pharmaceutical companies were regularly criticized for not providing reliable QoL data (Elvitegravir; Colestilan; Apixaban; Aflibercept, metastatic colorectal cancer). Analysis of the final decisions also showed that 3 benefit decisions were subject to a time limit, demanding manufacturers to file in reliable QoL data prior to the scheduled re-assessment (Eribulin, Ocriplasmin, Pertuzumab). It was also argued by the G-BA that QoL outcomes are needed in order to put mortality and morbidity results as well as data on side effects into context (Elvitegravir/Cobicistat/Emtricitabin/Tenofovirdisoproxil). In one of the oral hearings, the G-BA chairperson pointed out that QoL outcomes have a “special value” and added: “I can only recommend that the industry put more spotlight on this aspect” (Pertuzumab). In the oral hearing on Decitabin, the G-BA patient representative emphasized: “From the perspective of patient representation, QoL is always very important.”

The understandings of QoL

Qualitative findings demonstrate “wide” and “narrow” notions of QoL. Manufacturers tended to apply wider concepts than IQWiG and G-BA.

The IQWiG drew on scientific sources [13, 14] to define QoL as a complex or multidimensional construct including physical, social, and psychological aspects (Aclidiniumbromid; Vandetanib, re-evaluation). In the assessment of Fingolimod, the IQWiG accepted evidence established with the instruments EQ-5D and PRIMUS QoL stating that these display subsets of QoL at least. In many cases, however, the institute did not comment on the manufacturers’ explicit or implicit definitions of QoL. This was usually the case when data were not statistically significant or the magnitude of the effect was clinically irrelevant. When dossiers were rated incomplete, IQWiG usually did not assess individual endpoints.

The G-BA did not explicitly define QoL. Instead, its decisions convey a pragmatic stance: QoL data have to be collected with validated instruments that are designed to assess QoL in order to substantiate additional benefit. Thus, the G-BA implicitly defined QoL as being something measurable with QoL instruments. It did not engage in definitional discussions transcending individual assessments.

One of the few external organizations explicitly defining QoL was the Medicines Committee of the German Medical Association (AKdÄ). In oral hearings, it distinguished between QoL and patient-reported outcomes: “Health-related QoL and patient-reported outcomes are absolutely not synonym”; while changes in QoL may have multiple causes, patient-reported outcomes such as pain are supposed to result with treatment and morbidity only (Crizotinib; Vemurafenib).

Several manufacturers, some G-BA personnel, and the AKdÄ argued that QoL can be heavily impaired through treatment burdens, conveying a “wider” QoL notion than the IQWiG. They concluded that less burdensome treatment options could be interpreted as patient-relevant QoL benefits. These include treatment effort (Aflibercept, metastatic colorectal cancer, pharmaceutical company = PC; Apixapan, G-BA staff), number of daily injections (Dapagliflozin, AKdÄ), the effect of single tablet regimes vs the need to use syringes (Emtricitabin, PC), the shortening of therapy duration (Telaprevir, PC), the frequency of injections and the need for higher insulin doses and glucose measurement (Saxagliptin/Metformin, PC), use of syringes and the need to keep an injection-food delay (Saxagliptin, PC), and reddening, swelling, and pain as a consequence of an injection (Fingolimod, oral hearing, G-BA staff).

In the assessment of Elvitegravir/Cobicistat/Emtricitabin/Tenofovirdisoproxil, the manufacturer argued that stigma-related side effects of HIV affect patient’s QoL and are likely to impair adherence, thus threatening the overall treatment success. In the dossiers for Emtricitabin/Rilpivirin/Tenofovirdisoproxil, it was argued that QoL would benefit from single tablet regimes, thus increasing adherence. Both cases illustrate typical QoL argumentations: manufacturers linked QoL to certain concepts or possible surrogates for QoL. Usually, the assessment institutions would question the patient relevance of these endpoints.

IQWiG and G-BA did not accept surrogates for QoL

In the benefit assessments of Aclidiniumbromid and Telaprevir, the IQWiG stated that patient satisfaction with treatment, patient preference, work productivity, and the “use of healthcare service” cannot be equated with QoL. The institute also pointed out that attending physicians’ QoL assessments is not patient-relevant and therefore not accepted as QoL evidence. Higher patient treatment satisfaction was not accepted as relevant endpoints, because they would ultimately have to reflect in QoL, hence should be proven with QoL instruments (Aclidiniumbromid; Fingolimod). With regard to progression-free survival and objective response rate, the IQWiG stated that a potential effect on QoL does not suffice to establish a surrogate parameter (Axitinib; Crizotinib; Vandetanib, re-evaluation).

The G-BA also demonstrated reluctance to accept surrogate parameters. Neither compliance nor patient satisfaction or patient preference was accepted as surrogates or aspects of QoL. Like the IQWiG, the G-BA did not comment on every argument made in the context of QoL, especially if the submitted data did not prove statistically significant. Drawing on its assessment of Ingenolmebutat, however, it can be derived that the committee is rather skeptical regarding the patient relevance of additional benefit if it is not demonstrated with the help of patient-relevant endpoints like QoL or morbidity. In the case of Ingenolmebutat, a higher therapy acceptance through a shortened duration of application was not accepted as a patient-relevant endpoint.

Morbidity and QoL

QoL understandings were also conveyed through distinguishing QoL from the other patient-relevant endpoints, morbidity, mortality, and side effects.

In some assessments, manufacturers argued that morbidity aspects directly impair QoL, for example spastics (Cannabis Sativa extract) and visual acuity (Ocriplasmin). While morbidity parameters were portrayed as “correlating” with QoL in the case of Pirfenidon, in the dossier for Lisdexamfetamindimesilat symptoms were equated with QoL. As explained above, IQWiG and G-BA did not systematically comment on all argumentation patterns, but for example in the oral hearing for Fingolimod, the IQWiG pointed out that benefits regarding morbidity should also be observable in QoL data, but did not equate these endpoints. Not all QoL instruments were judged appropriate for QoL assessments. As QoL was understood as a multidimensional construct, tools or subscales assessing symptoms only were assigned to the endpoint morbidity, even if the respective authors had introduced their questionnaires as QoL instruments (Crizotinib, IQWiG, and G-BA: European Organisation for Research and Treatment of Cancer—Core QoL questionnaire; Abirateronacetat, new therapeutic indication; IQWiG: Prostate Cancer Subscale, Functional Assessment of Cancer Therapy-Prostate Cancer), because the multidimensionality of QoL could not be captured through, for example, the mere assessment of pain. The IQWiG interpreted fatigue and pain as aspects of morbidity (Belimumab, Vemurafenib).

Mortality and QoL

Regarding the relation between mortality and QoL endpoints, there was no discussion of definitions, but it was asked how to balance lifetime extension versus QoL impairments. In this context, the AKdÄ stated that this balance was not only important from a patient’s perspective but also for society as a whole considering the treatment costs (Cabazitaxel, AKdÄ). Especially in palliative contexts, it would have to be discussed to what extent better mortality outcomes were “bought” at the costs of QoL impairments (Eribulin, AKdÄ, PC). The German Society for Hematology and Oncology (DGHO) questioned that benefit could be assessed solely drawing on mortality outcomes when dealing with severe diseases: “Two months of lifetime extension would not suffice for me. Dealing with these patients, other things such as QoL have to be taken into account” (Decitabin, oral hearing). According to the Top Oncological Centres, if there are no mortality benefits, it may be legitimate to focus on QoL data (Crizotinib). Also manufacturers demanded that in a balanced assessment, mortality endpoints would have to be weighed against QoL endpoints, simultaneously conceding the methodological challenges this poses for the institutions (Eribulin). Not disagreeing, the institutions pointed out that QoL data are of special importance in palliative situations.

Side effects and QoL

Regarding side effects, the different stakeholders' argumentations did not necessarily contradict each other. IQWiG argued that side effects would also have to be weighed against QoL outcomes (Crizotinib, IQWiG). Manufacturers accordingly argued that better QoL results were not “bought” at the expense of side effects (Aflibercept, metastatic colorectal cancer; Perampanel). On the other hand, side effects were expected to impair QoL (Decitabin, PC; Eribulin, AKdÄ), and it was highlighted when QoL impairments due to side effects were not considerable (Ipilimumab, PC). According to manufacturers, side effects were a primary factor affecting QoL in some indications (Colestilan; Elvitegravir/Cobicistat/Emtricitabin/Tenofovirdisoproxil). Both the G-BA chairperson and the AKdÄ spokesperson pointed out in oral hearings that QoL assessments are essential if side effects occur (Aflibercept, metastatic colorectal cancer; Eribulin). G-BA und IQWiG emphasized that QoL data are necessary to assess the impairments caused by side effects (Brentuximab Vedotin, G-BA; Vemurafenib, IQWiG). The G-BA also highlighted when reduced side effects did not reflect in better QoL results (Axitinib; Rilpivirin). In the eyes of the DGHO, QoL would be the endpoint ultimately linking morbidity and side effect outcomes (Eribulin, oral hearing, PC, and AKdÄ).

Table 4 sums up this study’s qualitative findings on the different definitions and understandings of QoL in EBA.

Table 4 Quality of life definitions in the early benefit assessment according to AMNOG (qualitative analysis of n = 66 early benefit assessments; January 2011–December 2013)
Full size table

Discussion

Quantitative analysis leads to the assumption that QoL plays only a minor role in Germany’s drug benefit assessment. Only in 2 of the first 66 EBA's completed by the end of 2013 did the G-BA explicitly derive additional benefit on the grounds of QoL results (Crizotinib; Ivacaftor). Qualitative analysis, however, shows that IQWiG and G-BA frequently highlighted the need for reliable QoL data, emphasizing the concept’s importance for drug assessment. The fact that final decisions on additional benefit have not often been justified with QoL endpoints mainly stems from the absence or the inappropriate presentation of respective data. It does, however, not reflect the G-BA’s or IQWiG’s disregard for QoL data. We found out that all key players—even if they do not necessarily share the same QoL understanding—highlight the concept’s importance for patients, evidence-based medicine in general, and the German system of drug benefit assessment in particular. At the same time, three decisions on benefit were restricted with a time limit due to a lack of reliable QoL data. In these cases, the G-BA set a date for the re-assessment of the drug, expecting the manufacturers to generate QoL results accordingly. It could be argued that in these cases QoL results were pivotal for the benefit decision, but in a negative fashion.

As qualitative analysis shows, QoL is not easily defined within the frameworks of EBA. Manufacturers tended to apply wider notions of the concept. The IQWiG, in contrast, referred to the multidimensional definition of QoL that is also widely used in research [13, 14]. The G-BA did not officially define QoL, but understands QoL as being something that can be assessed with validated QoL instruments. At the same time, they did not comment on or explicitly reject or confirm the IQWiG’s definition of QoL. It may be surprising that the G-BA did not officially define QoL. But bearing in mind that also in the scientific community there is no single, universally agreed-upon definition of the concept and that neither parliament nor ministry of health clarified the term QoL, the G-BA—having to trade legally—had good reason to act with caution.

IQWiG’s and G-BA’s understandings of QoL display a very methodological approach to their respective mandates. Focusing on the validity of specific instruments and the statistical and clinical relevance of the submitted data, they circumvented a debate that could possibly have complicated their task to assess drug benefit in individual cases as is specified by the law. At the same time, the G-BA avoided definitional precedence; i.e., it did not define QoL in a (legally or methodologically) binding way leaving scope for future drug benefit assessments.

As the IQWiG is an institute committed to evidence-based medicine, it unsurprisingly drew on scientific sources to define QoL. It could, however, be criticized that it failed to express its understanding of QoL prior to the first dossier evaluations. It was not until the publication of its “General Methods” [15] and the "Appendix", Table 5 of the first dossier evaluation conducted [16], in which the IQWiG exemplified its operationalization of benefit, that the IQWiG explained how it intended to conduct assessments. And it took even longer until QoL was for the first time explicitly defined in the dossier assessments of Aclidiniumbromid (02.01.2013) and Vandetanib (re-evaluation, 17.06.2013). These were the 32nd and the 47th benefit assessments, respectively.

The assessment institutions expect the manufacturers to submit reliable data on QoL. Qualitative analysis shows that shortcomings regarding the generation and presentation of QoL data are likely to impair the overall benefit assessment. QoL data meeting the standards that can be extracted from the IQWiG’s and G-BA’s assessments will help to demonstrate additional benefits, if there are any. What make QoL a special criterion are the definitional and empirical challenges and the fact that QoL results are consulted not only as an independent endpoint per se, but also to contextualize outcomes regarding mortality, morbidity, and side effects.

The G-BA’s cautious stance on QoL did not contribute to clarify or strengthen its role in EBA. Clearer guidance with respect to the generation and presentation of QoL data could have helped the cause of QoL. It was argued before that EBA was constructed as a “learning system” [17], but this does not necessarily justify regulatory shortcomings. The AMNOG was passed rather quickly. The cabinet decided in favor of the AMNOG initiative on June 29, 2010, and the Bundestag passed the law on November 11 the same year. But it was not before December 28, 2010, that the ministry of health had published its corresponding Regulation on the Benefit Assessment of Pharmaceuticals [6]. As AMNOG came into effect on January 1, 2011, this left G-BA und IQWiG with little time to prepare for their tasks. This may explain why IQWiG und G-BA could not provide firmer guidance from the start.

At the same time, many manufacturers did not provide QoL data at all or failed at adequately presenting them. This can in part be explained with unspecific guidelines. Nevertheless, companies could also be blamed for not filling the regulatory gap with convincing QoL argumentations. Simultaneously, QoL as an endpoint has historically only recently gained greater importance, which may be why manufacturers had not planned trials accordingly. This might also explain why the other key players may have had difficulties grasping the construct as well.

Conclusions

In Germany’s drug benefit assessment, there are empirical and regulatory challenges that at least partly stem from the fact that QoL is not easily or universally defined. Declaring QoL one of the EBA’s main criteria, political decision-makers presumably intended to strengthen the endpoint. But considering the tight time frame of EBA’s legislation and implementation, this intention was initially problematic, as the first results seemed to indicate that QoL as an assessment criterion had been marginalized and manufacturers as well as assessment institutions might have been overburdened with the appropriate generation, presentation, and assessment of QoL outcomes.

This German case study shows the importance of a clear definition of QoL and corresponding methodological guidelines from the start, when it is used as a criterion in drug benefit and health technology assessments.