Abstract
In 2014, the European Medicines Agency (EMA) released for comment a draft reflection paper on the use of patient-reported outcome (PRO) measures in oncology studies. A twelve-member International Society for Quality of Life Research (ISOQOL) taskforce was convened to coordinate the ISOQOL response. Twenty-one ISOQOL members provided detailed comments and suggestions on the paper: 81 % from academia and 19 % from industry. Taskforce members consolidated and further refined these comments and shared the recommendations with the wider ISOQOL membership. A final response was submitted to the EMA in November 2014. The impending publication of the EMA reflection paper presents a valuable opportunity for ISOQOL to comment on the current direction of EMA PRO guidance and strategy. The EMA paper, although focused on cancer, could serve as a model for using PROs in other conditions, as it provides a useful update surrounding some of the design issues common to all trial research including PRO endpoints. However, we believe there are a number of additional areas in need of greater consideration. The purpose of this commentary is therefore to highlight the strengths of this timely and potentially useful document, but also to outline areas that may warrant further discussion.
Similar content being viewed by others
Avoid common mistakes on your manuscript.
The European Medicines Agency (EMA) has released for comment a reflection paper on the use of patient-reported outcome (PRO) measures in oncology studies [1]. This updates their 2005 publication [2]. The purpose of the proposed reflection appears twofold: to ‘spur an open discussion on the value of PRO data in the development of medicinal products’ in oncology and to present recommendations surrounding optimal PRO trial design—both with a focus on the regulatory perspective.
The EMA invited public comments on the draft reflection paper in June 2014. An International Society for Quality of Life Research (ISOQOL) taskforce (authors listed on this commentary) was convened to coordinate the ISOQOL response. Twenty-one ISOQOL members provided detailed comments and suggestions on the EMA reflection paper: 81 % from academia and 19 % from industry. Taskforce members consolidated and further refined these comments and shared the recommendations with the ISOQOL members through its member listserv. A final response was submitted to the EMA in November 2014 [3].
The impending publication of the EMA reflection paper presents a valuable opportunity to comment on the current direction of EMA PRO guidance and strategy. The purpose of this commentary is therefore to highlight the strengths of this timely and potentially useful document, but also to outline areas that may warrant further discussion.
Signs of encouragement
We note that the EMA’s use of terminology has shifted from health-related quality of life (HRQL) to the umbrella term PROs. This change reflects the broader context for the capture of patient experiences and perspectives as, in addition to HRQL, they may also include such domains as symptom burden, functional impact, treatment concordance, treatment satisfaction and global health status.
Within the document, the EMA extols the virtues of rigorous PRO trial design. In particular, they highlight the importance of: a strong rationale, supporting both PRO collection itself and the timing of assessment; comprehensive training of trial staff and patients involved in PRO measurement; implementation of methods to maximize compliance; and the formulation of a detailed, PRO-specific, statistical analysis plan addressing special issues such as multiplicity and missing data. This approach is welcome: both experience and empirical research suggest that a failure to incorporate these design features during trial planning may result in PRO data that are uninformative or inappropriate for evaluating the harms and benefits of the intervention under study [4, 5]. The EMA recommendations also align with those presented in other contemporary PRO guidance documents, including those produced by the Center for Medical Technology Policy [6] and the US Food and Drug Administration (FDA) [7]. The apparent harmonization of EMA and FDA guidance is encouraging, and it is hoped that further alignment in the coming years may allow sponsors to adopt a unified PRO claim strategy across the two agencies. Harmonization on PRO guidance would also benefit from the involvement of perspectives from researchers in industry and academic institutions and from patient groups. As a good model, the US National Cancer Institute convened a Clinical Trials Planning Meeting in 2011 that included researchers, regulators and patient representatives to recommend a core set of symptoms to measure in adult cancer clinical trials [8]. The core set will promote consistent assessment of patient-centered and clinically relevant symptoms to capture in oncology research.
Areas requiring greater focus
Although the EMA reflection paper rightly highlights the importance of PRO trial design, a greater consideration of the issues surrounding PRO reporting is required. Poor reporting of PRO data—which limits their use to inform clinical care, guidelines and health policy—has been identified as a particular problem in trials research [9, 10]. Therefore, we believe that the EMA should also outline the importance of transparent and high-quality reporting of PRO endpoints in the final version of their reflection paper, and formally lend its support to the use of the 2013 CONSORT PRO extension [10] to address this issue. ISOQOL, through its ‘Best Practices for PROs in Randomized Clinical Trials’ taskforce [11], is currently undertaking work to tackle both poor PRO trial design and reporting: including the development of a protocol checklist which will facilitate optimal design of PRO endpoints in trials, and of user-centered tools for implementing the CONSORT PRO extension. Greater collaboration between the EMA and ISOQOL is encouraged to facilitate future improvements in PRO trial design, implementation and reporting.
In their draft reflection, the EMA questions the value of longitudinal PRO data, stating that they have ‘…rarely been informative from a licensing perspective… a main reason being the absence of demonstrated difference between the study arms’ [1]. We understand that lack of difference in PROs between study arms might be seen as a challenge. However, we also emphasize that if the PRO data (1) are of high quality and (2) arise from a robustly designed and adequately powered PRO substudy, with a clear and comprehensive trial protocol and (3) the results are appropriately reported in later publications, the information derived—even if it is a ‘no PRO difference’ result—may effectively inform clinical decision-making when considered with other clinical endpoints evaluating overall treatment impact. There are pivotal trials, for example, in brain cancer patients, where only marginal differences in PROs between treatment arms have been found; yet these have contributed to a better understanding of the ‘value’ of the new treatment under investigation [12]. We urge the EMA to recognize that the lack of difference in PROs between treatment arms should not be seen, per se, as a factor limiting the use of PRO data in informing licensing decisions. Further, a finding of no HRQL difference does not imply a lack of difference between treatment arms in relevant and more specific PRO domains, such as symptoms.
We also encourage the EMA to provide transparent data surrounding historical PRO labeling claims, alongside more detailed information regarding the final decision. Ideally, it would be useful to know how many products had PROs in the labels, but also how many had requested PROs, and the reasons why PRO labels were not approved. This information would be of major interest to readers, as it would shed light on the current value of PRO data in interpreting treatment effectiveness. Presentation of case studies, outlining successful PRO labeling claims, would also be of great benefit to the research community and would help guide future improvements in PRO trial design.
While we recognize that this is a reflection paper, it may also be a useful medium to consider contemporary challenges in oncology PRO trial design. For example, while it is quite straightforward to link PRO assessment to specific clinical events in case of a conventional chemotherapy-based trial (e.g., administering questionnaires in conjunction with the clinical visit), newer therapies pose challenges that investigators need to consider when developing a protocol. For instance, issues around ‘timing’ and adherence become more challenging in trials investigating modern targeted therapies such as tyrosine kinase inhibitors (TKIs), as these treatments are usually taken by patients on a daily basis (and in most cases for a prolonged period of months or years). We take for granted that the patient has received the recommended dose of chemotherapy or radiotherapy, as the patient has to attend their hospital and receive treatment in the clinic. However, anticancer-targeted therapies are typically administered orally, not requiring a hospital visit. It has been shown that adherence with targeted agents (e.g., leukemia patients) is not optimal and might undermine maximum benefit of therapy [13]. Patient-reported measures may be used to capture both the extent of medication adherence and reasons for non-adherence, which may include such issues as treatment toxicity, costs or forgetting the medication. Thus, EMA consideration of the challenges and opportunities associated with PRO evaluation in targeted therapies would be helpful.
Finally, the last decade has seen increasing interest in the contribution of patients as active partners in health research. Growing evidence reflects the beneficial impact of patient engagement in enhancing the quality, relevance and validity of such research [14, 15], and in particular within patient-centered outcomes research [16, 17]. For example, recent research in this area has sought to identify outcomes that really matter to patients [18] and improve the relevance and validity of PRO measures [19], with the aim of enhancing the acceptability of PRO-based assessment and improving compliance. The EMA reflection paper raises issues associated with ‘respondent burden’ and PRO selection, but fails to outline that these can be usefully explored and addressed with appropriate, active patient engagement. Of note, for many patients, completion of a relevant and appropriate measure may indeed be empowering; respondent burden may be more readily associated with completion of irrelevant and inappropriate measures [4]. We suggest that the EMA considers the value of involving patient stakeholders in the co-production of PRO trial components, with particular emphasis on: informing the selection of appropriate patient-centered endpoints; identifying relevant, acceptable and relatively un-burdensome measures of those endpoints; enhancing compliance with PRO assessment; and aiding interpretation of PRO findings and dissemination of the results.
Summary
The EMA draft reflection paper, although focused on cancer, could serve as a model for using PROs in other conditions: The paper provides a useful update surrounding some of the design issues common to all trial research including PRO endpoints. However, there are a number of additional areas in need of greater consideration, including the importance of the CONSORT PRO extension in driving up standards of reporting; the value of ‘negative’ PRO findings; the need for comprehensive information surrounding historical labeling decisions; and the role of patients in the PRO trial design and implementation. Importantly, there is also an opportunity for the EMA to outline how they might look to tackle future opportunities and barriers in the field of PROs research and how to make best use of PRO data.
References
EMA. (2014). Reflection Paper on the use of patient reported outcome (PRO) measures in oncology studies [Draft]. European Medicines Agency, Oncology Working Party; Doc. Ref. EMA/CHMP/292464/2014.
EMA. (2005). Reflection paper on the regulatory guidance for the use of health-related quality of life (HRQL) measures in the evaluation of medicinal products. European Medicines Agency, Committee for medicinal products for human use (CHMP); Doc. Ref. EMEA/CHMP/EWP/139391/2004.
ISOQOL. (2015). International Society for Quality of Life Research. http://www.isoqol.org/research/isoqol-publications. Accessed July 2015.
Kyte, D., Ives, J., Draper, H., Keeley, T., & Calvert, M. (2013). Inconsistencies in quality of life data collection in clinical trials: A potential source of bias? Interviews with research nurses and trialists. PLoS One, 8(10), e76625.
Kyte, D. G., Draper, H., Ives, J., Liles, C., Gheorghe, A., & Calvert, M. (2013). Patient reported outcomes (PROs) in clinical trials: is ‘in-trial’ guidance lacking? A systematic review. PLoS One, 8(4), e60684.
Basch, E. M., Abernethy, A., Mullins, C. D., Tiglao, M. R., & Tunis, S. R. (2011). Development of a guidance for including patient-reported outcomes (PROS) in post-approval clinical trials of oncology drugs for comparative effectiveness research (CER). Value in Health, 14(3), A10.
FDA. (2009). Guidance for industry: Patient-reported outcome measures: Use in medical product development to support labeling claims. http://www.fda.gov/downloads/Drugs/GuidanceComplianceRegulatoryInformation/Guidances/UCM193282.pdf
Reeve, B. B., Mitchell, S. A., Dueck, A. C., Basch, E., Cella, D., & Reilly, C. M. et al. (2014). Recommended patient-reported core set of symptoms to measure in adult cancer treatment trials. Journal of the National Cancer Institute, 106(7), dju129.
Brundage, M., Bass, B., Davidson, J., Queenan, J., Bezjak, A., Ringash, J., et al. (2011). Patterns of reporting health-related quality of life outcomes in randomized clinical trials: Implications for clinicians and quality of life researchers. Quality of Life Research, 20, 653–664.
Calvert, M., Blazeby, J., Altman, D. G., Revicki, D., Moher, D., & Brundage, M. (2013). Reporting of patient reported outcomes in randomised trials: The CONSORT PRO extension. JAMA, 309(8), 814–822.
International Society for Quality of Life Research. Best practices for PROs in randomized clinical trials taskforce. http://www.isoqol.org/about-isoqol/committees/best-practices-for-pros-in-randomized-clinical-trials. Accessed February 17, 2015.
Taphoorn, M. J., Stupp, R., Coens, C., Osoba, D., Kortmann, R., van den Bent, M. J., et al. (2005). Health-related quality of life in patients with glioblastoma: A randomised controlled trial. The lancet Oncology, 6(12), 937–944.
Marin, D., Bazeos, A., Mahon, F.-X., Eliasson, L., Milojkovic, D., Bua, M., et al. (2010). Adherence is the critical factor for achieving molecular responses in patients with chronic myeloid leukemia who achieve complete cytogenetic responses on imatinib. Journal of Clinical Oncology, 28(14), 2381–2388.
Brett, J., Staniszewska, S., Mockford, C., Herron‐Marx, S., Hughes, J., & Tysall, C. et al. (2014). Mapping the impact of patient and public involvement on health and social care research: A systematic review. Health Expectations, 17(5), 637–650.
Gradinger, F., Britten, N., Wyatt, K., Froggatt, K., Gibson, A., & Jacoby, A. et al. (2013). Values associated with public involvement in health and social care research: A narrative review. Health Expectations. doi:10.1111/hex.12158.
de Wit, M. P., Abma, T. A., Koelewijn-van Loon, M. S., Collins, S., & Kirwan, J. (2014). What has been the effect on trial outcome assessments of a decade of patient participation in OMERACT? The Journal of Rheumatology, 41(1), 177–184.
Staniszewska, S., Haywood, K. L., Brett, J., & Tutton, L. (2012). Patient and public involvement in patient-reported outcome measures. The Patient-Patient-Centered Outcomes Research, 5(2), 79–87.
Kirwan, J. R., Minnock, P., Adebajo, A., Bresnihan, B., Choy, E., De Wit, M., et al. (2007). Patient perspective: Fatigue as a recommended patient centered outcome measure in rheumatoid arthritis. The Journal of Rheumatology, 34(5), 1174–1177.
Nicklin, J., Cramp, F., Kirwan, J., Urban, M., & Hewlett, S. (2010). Collaboration with patients in the design of patient-reported outcome measures: Capturing the experience of fatigue in rheumatoid arthritis. Arthritis Care & Research, 62(11), 1552–1558.
Author information
Authors and Affiliations
Corresponding author
Additional information
Disclaimers: This paper was reviewed and endorsed by the International Society for Quality of Life Research (ISOQOL) Board of Directors as an ISOQOL publication and does not reflect an endorsement of the ISOQOL membership. The opinions or assertions in this article are the views of the authors and are not to be construed as official or as reflecting the views of the Medical Products Agency, Sweden.
Rights and permissions
About this article
Cite this article
Kyte, D., Reeve, B.B., Efficace, F. et al. International Society for Quality of Life Research commentary on the draft European Medicines Agency reflection paper on the use of patient-reported outcome (PRO) measures in oncology studies. Qual Life Res 25, 359–362 (2016). https://doi.org/10.1007/s11136-015-1099-z
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11136-015-1099-z