Introduction

Patient outcomes are essential to understand the impact of cancer and whether health care services and procedures make a difference to health outcomes. Routine assessment of how cancer patients feel and function is informally evaluated by clinical history, but can also be measured directly using patient-reported outcomes (PROs). PROs are patients’ direct reports of their health status including their symptoms, quality of functioning, and overall health-related quality of life [1, 2]. PROs have a long history of being measured in clinical trials (and other comparative research studies) to compare interventions in terms of their impact on health status from the patient’s perspective [3,4,5]. In order to inform the care of the individual patient, PROs, in conjunction with other clinical information, are increasingly being used to screen or monitor the progress of individual patients in conjunction with a clinical encounter [6,7,8,9,10].

Evaluations on the use of PROs in clinical practice in oncology have demonstrated important benefits. In many oncology settings, the use of PROs has been shown to promote patient–clinician communication, [11,12,13,14] assist with problem detection, [8, 9] affect patient management, [13, 15] and to improve symptom control, health-related quality of life, and functioning [10], although not all of these outcomes have been consistently demonstrated across studies. Most recently, randomized trials of implementing PROs in the medical oncology setting have demonstrated a survival benefit associated with their use, as well as a decrease in emergency department use [16,17,18]. Finally, PROs collected in the clinical encounter have the potential to be used as quality of care measures [19] and to inform health system improvements [20, 21].

Despite the existing evidence base on the value of PROs in oncology, it is not clear which of these benefits might be realized in the context of managing men with early-stage prostate cancer, since the majority of studies have been conducted in the context of systemic management of advanced disease [10]. Comparative studies evaluating PROs in the urologic oncologic setting have demonstrated that functional outcomes relating to urinary continence, bowel functioning, and sexual activity have treatment-specific patterns and are important drivers of prostate cancer patient satisfaction in the long-term [22,23,24,25,26]. A systematic review of PROs in clinical practice, however, identified prostate cancer as the least frequently addressed clinical setting [10].

This paper reports the findings of a pilot study evaluating the feasibility of implementing prostate cancer-specific PRO measure and reporting into routine uro-oncology clinical practice in four Regional Cancer Centers (RCCs) within the Cancer Care Ontario (CCO) health care jurisdiction. CCO is the organization responsible for cancer health services in the Province of Ontario, and is the sole provider of radiotherapy (delivered in 14 RCCs providing care to approximately 14 million Ontarians). Urologic health care services are provided both within RCC ambulatory clinics and by other regional academic and community hospitals. Since 2007, CCO RCCs have assessed patients’ self-reported outcomes using a general symptom assessment measure, namely the Edmonton Symptom Assessment Scale (ESASr) [27] across all 14 cancer centres [28]. Patients report on nine general symptoms (e.g. pain, shortness of breath, anxiety, fatigue), each on a scale of 0–10, where summary scores are then provided to clinicians (physicians and other health care professionals) for integration into the clinical encounter. A 2014 survey of oncology clinicians in Ontario regarding their attitudes towards the use of ESASr in ambulatory clinical practice revealed that 66% of 256 physicians and 81% of 353 nurse respondents considered the use of standardized tools to screen for symptoms as “best practice”. The top three recommendations to improve ESASr, identified by respondents, included the use of PROs specific for certain cancer populations [29].

The use of PROs specific to the prostate cancer population, in place of general symptom measures, is consistent with the theory of how PROs “work” to improve the quality of the clinical encounter as constructed by Greenhalgh et al. [30] in that the instrument item content is required to be consistent with the care priorities of both the patient and the clinician. That is, the ‘right’ instrument is required for the ‘right’ clinical encounter to maximize the value of eliciting PROs in practice. Given that the majority of men with early-stage prostate cancer have symptoms related to their cancer management (or co-morbid illnesses) rather than cancer-related symptoms, it is problematic that symptom domains such as urinary incontinence, irritative urinary symptoms, sexual and bowel functioning are neither screened for nor monitored with a general symptom instrument. Thus, the value of implementing PROs in practice is much more likely to be realized for patients with early-stage prostate cancer using a disease-specific PRO which assesses the impact of prostate cancer and treatment and better aligns with goals of care.

A previous single-centre study sought to determine the potential for a prostate-specific PRO (the EPIC-26 scale [31]) to be implemented in ambulatory uro-surgical and uro-radiotherapy clinics. Study outcomes described pragmatic process issues and qualitative assessments of clinicians’ attitudes [32]. Although qualitative findings were generally supportive of the use of a prostate-specific PRO in this clinical setting, no quantitative outcome data were collected regarding the patients’ impressions about the value of the intervention. We, therefore, undertook a larger cross-sectional study in purposefully sampled RCCs in order to better evaluate the feasibility and potential usefulness of implementing the EPIC instrument across all cancer centres as a prostate-specific PRO measure.

The overall purpose of this study was to conduct a single-arm evaluation of the implementation of EPIC-CP in clinical practice for men with early-stage prostate cancer treated in one of four RCCs. The specific study aims were to (1) evaluate the acceptability and usability of EPIC-CP through a patient ‘exit’ survey and (2) explore the clinicians’ acceptability of the EPIC-CP PRO data and use in clinical practice.

Methods

This mixed methods study was a multi-centre, cross-sectional evaluation of the feasibility of introducing the EPIC-CP instrument into urologic-oncology clinical practice (in addition to the already established use of ESASr). Feasibility was assessed in several dimensions including patient-reported evaluations (patient exit surveys), PRO completion rates, and qualitative interviews to assess clinician acceptability based on a qualitative description methodology as per Sandelowski [33]. Qualitative description aims to provide a rich, straight description of an experience or an event and answers questions about the what, why, and how of this experience while staying grounded in the data [34].

Instruments

ESASr is a validated assessment tool that was originally implemented in the palliative population. It assesses pain, tiredness, nausea, depression, anxiety, drowsiness appetite, well-being, and shortness of breath on an 11-point Likert scale (0–10). Severity of symptoms is scored at the time of each assessment, providing a symptom score profile for each patient over a period of time [35].

The EPIC-CP is a validated short-form 16-item questionnaire that measures urinary incontinence, urinary irritation, bowel, sexual, and hormonal function. It was adapted from the original 50- and 26-item EPIC questionnaires developed at the University of Michigan and UCLA based on an adaptation of the UCLA Prostate Cancer Index. It was created to measure health-related quality of life in several key urologic domains as well as overall vitality among men with early-stage prostate cancer managed with or without the use of adjuvant hormones [36]. Both questionnaires are available for public use and did not require special permission.

Study setting

Selection of participating RCCs was determined through a CCO Request for Proposals outlining pilot eligibility and study funding. Four RCCs were purposefully selected according to their academic/community focus, patient volumes, variation of patient demographics, and geographical location. A study operations committee was formed that included the CCO lead, study coordinators, and clinician leads from each participating centre.

Study population

Within each centre, a research associate recruited patients attending consultation or follow-up ambulatory clinics in surgery or radiotherapy. Patients attending radiotherapy review clinics were also able to participate. Patients were eligible if they were willing to consent, able to read English sufficiently well to complete questionnaires, and had not received palliative chemotherapy or radiotherapy (hormonal therapy was permitted). Clinicians attending the participating clinics were eligible if they had been exposed to the EPIC-CP by one or more of their patients participating in the study.

Study administration and evaluation

The EPIC-CP and ESASr instruments were completed by patients electronically on a tablet in advance of the clinical encounter. Summary scores were calculated for ‘real-time’ reporting, which were made available to the clinicians caring for each participating patient (Fig. 1). For study purposes, additional patient-reported evaluations were collected by means of a nine-item patient “exit survey” (PES). The PES included measures of patient acceptability (level of comfort, privacy, annoyance, time for completion, requirement for assistance, overall experience) and measures of EPIC-CP usability (use of tablet/touch screen, software, readability, location preference). The PES was developed for this study based on the original exit survey that was pilot tested in the smaller scale study [32]. The questions were pilot tested for clarity by patient volunteers prior to study initiation. Responses to each item were on a 5-point Likert scale anchored by “strongly agree” and “strongly disagree”. The PES was given to patients by the study coordinator during their last scheduled visit prior to the close of the study period. In some instances, due to scheduling conflicts, the patient was given the PES to take home to fill out, along with a return envelope. If surveys were not returned, a reminder phone call was made to the patient.

Fig. 1
figure 1

Sample of the EPIC-CP output for a given patient over time

Full size image

Study approach—quantitative

Quantitative analyses included descriptive statistics to summarize individual items on the PES, demographic information of participants, as well as their EPIC-CP and ESASr scores at the time of the clinical encounter. The main outcomes of interest for study feasibility from the patient survey perspective were patients’ overall ratings for using EPIC-CP in practice and their ratings on whether EPIC-CP improved their opportunities to communicate with their care providers. Given the exploratory nature of the study, a priori thresholds for the proportions of patients endorsing these items were not established.

EPIC-CP and ESASr items were dissimilar in item content, except for two areas: depression and vitality/fatigue. Instrument concordance was evaluated on the depression items (dichotomized to above or below the respective scale cut-points: four or higher on ESASr, range 0–10; two or higher on EPIC, range 0–4). This analysis was repeated for vitality (“lack of energy” on EPIC-CP and “fatigue” on ESASr). Percent agreement was used as the measure of concordance; correcting for chance agreement using the kappa statistic was not computed given the imbalance in marginal totals [37].

Study approach—qualitative

Clinician-reported perspectives of acceptability of the EPIC-CP were assessed through semi-structured, qualitative interviews (telephone or in-person) based on a topic guide. The topic guide included questions about (a) clinician’s perspective of the importance of the EPIC-CP PRO in clinical care; (b) how they used the PRO output report in their practice and specifically in treatment planning; and (c) their perception of patient receptivity to use of the tool. Interviews were conducted with a convenience sample of practitioners (i.e. urologists, radiation oncologists, and nurses), who were recruited via email invitations or in clinics. Interviews were approximately 45 min in length and were digitally recorded and subsequently transcribed verbatim, making note of verbal and non-verbal communication where relevant. Thematic analysis was performed by one member of the team using content-coding methodology based on the methods of Graneheim and Lundman, which involved examining the meaning units (based on manifest content), creating condensed meaning units, applying codes, and creating sub-themes and themes [38]. Qualitative data were analysed across participating RCCs (analyses stratified by clinic were not conducted). The research team undertook a consensus process for content validity, where the identified themes were discussed and agreed upon. The study was approved by the Ontario Cancer Research Ethics Board (OCREB) in 2014.

Results

Four RCCs were selected for participation—Princess Margaret Cancer Centre (PMCC), Grand River Regional Cancer Centre (GRRCC), Carlo Fidani Regional Cancer Centre—Trillium Health Partners (THP), and the Cancer Centre of South Eastern Ontario (CCSEO). These included one large (> 5000 new cases per year) and three smaller centres (all > 2000 new cases per year); two academic and two community-based centres, and three centres with genitourinary surgical oncology on-site.

In all, 333 patients completed the EPIC-CP tool, while 287 completed the PES. The demographic characteristics of patient participants are summarized in Table 1. The majority of the respondents were between the ages of 60 and 79 (77%) and many had completed a post-graduate degree, college, university, or trade school (62%). Additionally, most respondents were not undergoing any hormone therapy (74%).

Table 1 Demographic characteristics of patient participants (N = 287)
Full size table

During the course of the 6-month cross-sectional study, most patients had only one clinical encounter (Table 2), although the number of EPIC-CP assessments ranged from 1 to 11 per patient, for a total of 937 EPIC-CP questionnaires completed. Table 3 displays the item completion rates for each the EPIC-CP items across all completed questionnaires. The electronic platform was programmed to allow patients to skip questions if they so desired. Item completion rates, however, were generally high, ranging from 91 to 100%. Sexual health item completion rates were among the lowest rates, ranging from 91 to 92%.

Table 2 Number of EPIC-CP screens completed during the study duration across all sites (n = 333)
Full size table
Table 3 Completion rates for EPIC-CP items
Full size table

Figure 2 illustrates the distribution of scores (n = 937) on the ESASr and EPIC-CP instruments, respectively. The proportions of patients reporting high levels of symptom burden were low for most items on the ESASr (Fig. 2a). The proportions of patients reporting high levels of symptom burden were higher on the context-specific items of the EPIC-CP, including urinary and sexual functioning (Fig. 2b).

Fig. 2
figure 2

Distribution of scores on the ESASr (Panel A) and EPIC-CP instruments (Panel B, total number of assessments = 937)

Full size image

Table 4 illustrates the concordance between EPIC-CP and ESASr scores on the items addressing depression (Table 4, A) and vitality (Table 4, B). For depression, percent agreement of the instruments was 93%; where scores were discordant, the EPIC-CP cut-point was more sensitive to flagging depression than that of ESASr (7% above the EPIC-CP cut-point but less than 1% above the ESASr cut-point). The comparison of the vitality scores shows 88% agreement between instruments; for the discordant scores, a higher percentage of patients were above the cut-point for EPIC-detected vitality (8%) than were above the ESAS cut-point (4%).

Table 4 Concordance between EPIC-CP and ESASr scores for depression (4A) and vitality (4B)
Full size table

Table 5 describes the proportion of categorical responses on each item of the PES. The majority of patients (70%) agreed with the item: “Completing this questionnaire helped me tell the clinicians about how I have been feeling” and only a small proportion disagreed or strongly disagreed (2%). Figure 3 shows the distribution of these scores across participating centres, illustrating similar patterns of patient endorsement across different RCCs, with percentage positive scores ranging from 87 to 95%.

Table 5 Proportion of categorical responses on the PES
Full size table
Fig. 3
figure 3

Participants′ overall experience scores by participating centre and overall

Full size image

Thematic analysis from qualitative interviews with clinicians complemented the aforementioned quantitative findings regarding acceptability. A total of 31 clinicians agreed to participate in semi-structured interviews across the RCCs (range 6–11 clinicians per site). Figure 4 graphically summarizes the themes and sub-themes that emerged during discussions.

Fig. 4
figure 4

Summary of themes and sub-themes that emerged during qualitative interviews with clinicians

Full size image

One identified theme was that EPIC-CP completion fostered person-centred communication and discussion of sensitive topics. Many clinicians reported that the completion of the questionnaire prior to the clinic visit helped to structure their communication, as well as facilitate open dialogue and discussion of sensitive topics, such as urinary and sexual dysfunction. Clinicians often described reviewing the output report prior to entering or as they were entering the clinic exam room, so that patient scores were used to facilitate the conversation. When asked if they used the EPIC-CP results to inform discussion, a clinician commented,

Oh, all the time - absolutely. It was a really good tool to sit down and actually focus the discussion on the things that were pertinent to their particular situation. And you could bring up stuff…because they had reported it I could say to them more easily, “I see you’re having issues with…can you tell me about it” and then pull the information I needed to know how to treat them.

These qualitative findings complement the patient-reported results (Table 5), where 71% of patients endorsed that the questions regarding the impact of prostate cancer treatment on sexual health were important to facilitate discussion with clinicians. Additionally, 66% of patients reported that completing the EPIC-CP questionnaire helped them participate in discussions regarding their care, and 90% of patients agreed that they felt comfortable talking about their EPIC-CP responses with clinicians.

A second key theme involved clinicians identifying that patients were generally positive about their experience with the EPIC-CP questionnaire. Clinicians described the self-reported nature of the EPIC-CP questionnaire as a strength, which provided an unbiased assessment of treatment impact by the patient. Clinicians felt that the tool empowered the patient to tell their story and experience.

Well, why (is it important) because I think that the patients are often more truthful. And sometimes I think they feel…I put them on the spot when I ask them the questions and they feel pressured to answer. But if they have time to think about it when it’s on paper they may be more truthful or more willing to give that information out.

In the same vein, 70% of patients reported that completing the questionnaire helped them express how they had been feeling, while 62% stated that the questionnaire contributed to their satisfaction with the clinical encounter. Although 81.6% of patients agreed or strongly agreed that they were willing to complete a similar questionnaire at future clinic visits (Table 5), some patients voiced frustration with the technology used to collect PROs. One clinician raised concerns about language and computer literacy by stating,

…so the gentlemen in their 50 s and 60 s are quite literate, they have iPads at home….use to a touch screen…they fly through those questions…other men are less comfortable…it took a long time… and that would be a whole other challenge if you’ve got someone for whom English is not their first language…they need someone to help them in the absence of a medical translator.

Clinicians also sometimes articulated a positive experience using the EPIC-CP tool, as it improved communication and shared decision-making by allowing them to tailor the visit according to the patient scores and focus on the “red-flagged” items. For instance, if a patient did not trigger high symptom scores, then the visit length might only be 5 min, but if all domains were triggered, then the patient visit might require additional attention.

I guess prior to EPIC, the questions I tended to ask patients are quite random…Now that there is EPIC we actually have something to go by and know that there are issues that we need to cover with them….

Furthermore, clinicians mentioned that the completion of the EPIC-CP tool prior to the appointment improved overall clinic efficiency and workflow. This meant that there was more time to focus the appointment on the patient and create a more person-centred environment. One clinician expressed,

I think actually it improves, you know, the efficiency overall because you end up have a more focused discussion with some clearer information than if you were to take this out…the patient at the time, they walk into the room. So you already have a starting point.

However, only about half (47.4%) of the patients reported that their summary report was reviewed by their doctor during their appointment.

The issue of education for clinicians and patients on utilizing EPIC-CP was an emergent theme from interviews. Some clinicians felt that at minimum, patients should be oriented to the measure the first time they complete the tool. They should also be given specific education as to how to interpret the report and how it will be used by clinicians in patient care. With regard to clinician education, additional training on sexual health and the treatment strategies used for different stages of erectile dysfunction were mentioned to be potentially extremely beneficial areas for education.

Finally, the theme that EPIC-CP results encouraged involvement of the whole clinical team was also described in clinician interviews. One clinician explained that using the tool often meant coordinating with allied health professionals, such as dieticians or social workers, in order to assist patients with their symptom management. In other words, “…is it not just the physician…[the team] is truly multidisciplinary”.

Discussion

A growing body of evidence suggests that symptom screening and monitoring of ambulatory cancer patients can improve the quality of the clinical encounter between patient and clinician. This study demonstrates that in the context of ambulatory management of patients diagnosed with prostate cancer, a disease-specific PRO measure (EPIC-CP) can be successfully implemented and is generally valued by patients and clinicians. The majority of patients rated the use of EPIC-CP favourably, and qualitative evaluation of clinician’s attitudes towards the use of EPIC-CP was supportive, in that completion of the EPIC-CP appeared to foster person-centred care. Moreover, the EPIC-CP tool facilitated the clinical encounter and promoted open communication with respect to sensitive problem areas, such as urinary and sexual functioning.

The feasibility-related outcomes in this study were focused on acceptability of the EPIC-CP to patients and clinicians, as well as pragmatic usability measures designed to guide modifications to processes or infrastructure (e.g. screen display or need for assistance). Although no single outcome was used to define feasibility, we found that few problematic areas were identified from the perspectives of the respondents. Questions addressing sexual function had very high completion rates at over 90%, where 70% of patients endorsed routinely including these questions. Additionally, over 80% of patients expressed willingness to complete EPIC-CP on future clinic visits.

Our findings are consistent with the theory of how PROs in clinical practice positively impact the quality of the encounter; [30] many clinicians and patients reported that EPIC-CP use improved communication, focussed the content of the encounter, and created efficiencies. The findings in this study, however, cannot definitively determine the magnitude of improvement in these outcomes given the single-arm nature of the study, the use of ESASr in conjunction with EPIC-CP, and the potential for volunteer bias among patients and clinician participants. Moreover, given the study design, we did not include “down-stream” measures, such as impact on patient functioning, quality of life, or other outcomes. Also consistent with theory is that the use of EPIC-CP is unlikely to impact patient survival rates, since randomized trials demonstrating this type of impact have been conducted in advanced-stage cancer patients who are more likely to experience a survival benefit from early and effective symptom management. In the early-stage prostate cancer population, screening and monitoring for early urinary, bowel, sexual function, and mental well-being would be most likely to have impact on long-term quality of life and satisfaction with treatment choice, although further research is needed to confirm this hypothesis.

The study findings should be interpreted in the context of the study strengths and weaknesses. The study was performed in four ambulatory clinics purposefully selected to represent a variety of clinical settings and processes. Despite these differences, patient responses to exit-survey items were similar across centres, providing preliminary evidence of the consistency of findings (although the study was not designed to address between-centre outcome differences). The data were of high quality (few missing responses) and have the potential to be used as health-outcome measure in their own right going forward. In terms of potential limitations, bias may have been introduced by use of a single coder in the qualitative thematic analysis of the clinician interviews. Beyond the limitations inherent in a single-arm study, we cannot determine the potential for selection bias as it was not possible to analyse the proportion of invited patients who declined to participate, nor the degree to which clinician volunteers were biased towards a favourable evaluation of the experience. While research assistants were needed to enrol patients’ and obtain consent for the quantitative evaluations, it is unclear how or if they influenced the success of the implementation strategy. Additionally, we did not assess the costs of implementation in terms of technology and human resources required for successful integration of EPIC-CP into practice. Finally, since ESASr was already in clinical use in all participating clinics, it is unknown how our findings would have differed if EPIC-CP were to be implemented without the experience of ESASr use.

Future directions include a wider implementation of EPIC-CP across all 14 RCCs. In keeping with our qualitative findings, Cancer Care Ontario has undertaken development of symptom management guides for both patient self-management and for clinical management to support person-centred care and to promote best practice in clinical response to high EPIC-CP scores on specific items (https://www.cancercareontario.ca/en/symptom-management). Further, an implementation working group has been appointed to capitalize on lessons learned from this study for application in wider implementation. Important areas identified to address include the need to seamlessly identify patients for whom EPIC-CP completion is indicated; to protect the privacy of patients when completing the questionnaire, and improve the integration of the EPIC-CP scores for use by physicians and other health care professionals within existing clinic processes; and to ensure early engagement of clinicians in each site thereby developing ownership and accountability for the use of EPIC-CP in the clinical encounter. With regard to incomplete questionnaires, further research is required to better understand why some patients skip questions, and to determine strategies that would be most useful to minimize missing data while respecting patients’ autonomy to decline to answer specific items.

Conclusions

Implementing a disease-specific PRO measure in place of a generic measure in ambulatory genitourinary cancer practice can be successfully accomplished in a variety of clinical settings: a disease-specific PRO has the potential to improve the quality of the clinical encounter and to provide disease-specific outcome measures for further health services research. Provincial roll-out of this tool as a standard of care is recommended.