Introduction

The more than 15.5 million cancer survivors in the USA are at risk for long-term and late effects of cancer and its treatment (e.g., organ toxicity, compromised reproductive function, fear of recurrence and new cancers) [1]. These effects can be identified and addressed through coordinated care from follow-up care providers (e.g., primary care providers [PCPs], oncologists); however, coordination of survivors’ care is often poor, at times resulting in the duplication or omission of recommended services [2,3,4,5], poor health outcomes [4, 6,7,8,9], and significant costs to survivors and the healthcare system [10]. To improve care and outcomes for survivors, many organizations (e.g., the Commission on Cancer, the largest US cancer program accrediting organization) recommend or require cancer care providers to develop and deliver survivorship care plans (SCPs) to survivors and their follow-up care providers [10,11,12,13]. SCPs are written documents that ideally include treatment summary information (e.g., diagnosis, stage, treatments), plans for follow-up care (e.g., surveillance, preventive services), and recommended division of responsibilities among follow-up care providers.

Observational studies have found that survivors and PCPs benefit from SCPs [14]; however, to date, eight randomized controlled trials (RCTs) of SCPs’ effectiveness have reported mixed results [15,16,17,18,19,20,21,22,23,24,25,26,27,28]. Some RCT results may be null because they have evaluated outcomes on which SCPs should be expected to have limited influence. For example, SCPs may be unlikely to improve health-related quality of life (HRQOL) because HRQOL is determined by a complex set of clinical, demographic, and social determinants [29]. Stakeholders (e.g., survivors, oncologists) may expect more practically relevant outcomes than those assessed in RCTs. The US National Cancer Institute (NCI) has called for research to produce clearer evidence of SCPs’ effectiveness (PA-12-275, PA-16-012). A key component of such evidence involves the outcomes that stakeholders expect from SCP use. The objectives of this study were to (1) understand the outcomes that diverse stakeholders expect from SCP use and (2) compare them to outcomes assessed in extant RCTs. If stakeholders expect outcomes that differ from those assessed in extant RCTs, clearer evidence of SCPs’ effectiveness may be achieved by assessing the outcomes identified in this study. Further, SCPs’ effectiveness may be optimized when practitioners use SCPs with an eye toward outcomes that stakeholders expect from SCPs.

Methods

Study design

We identified outcomes assessed in extant RCTs, identified outcomes that stakeholders expect from SCPs, and mapped RCT outcomes onto stakeholder-identified outcomes.

Identifying RCTs of SCPs’ effectiveness

We began with a literature review in PubMed using the search terms in Appendix 1 through February 2017. We then hand-searched PubMed results to identify RCTs. Finally, three members of the research team reviewed articles to identify outcomes assessed in each RCT and the instruments used to assess them. When instruments were not included in publications, we used citations to identify the instruments.

Identifying outcomes that stakeholders expect from SCPs

We identified outcomes that stakeholders expect from SCPs via semistructured stakeholder interviews.

Recruitment

The North Carolina Cancer Hospital (NCCH), part of the University of North Carolina Healthcare System, is a tertiary academic NCI-designated comprehensive cancer center. Kaiser Permanente Southern California (KPSC) is an integrated healthcare system providing comprehensive care to over four million members. The Program of Care for Cancer, Nova Scotia Health Authority (NSHA), oversees all cancer services in the province of Nova Scotia, serving a population of approximately 940,000. We used snowball (NCCH, KPSC) and purposive (NSHA) sampling approaches to recruit interview participants with a stake in SCP use (see Table 1 for interview participants and their stake in SCP use). The Institutional Review Board at the University of North Carolina at Chapel Hill exempted the study from human subjects review. The Research Ethics Board at NSHA and the Institutional Review Board at KPSC approved the study.

Table 1 Interview participants and their stake in SCP use
Full size table

At NCCH and KPSC, we recruited administrative and provider stakeholders through clinical co-investigators. These stakeholders participated in initial interviews and were then asked to recommend other stakeholders. Survivors whom clinical co-investigators identified in turn identified their caregivers. At NSHA, one research team member directly contacted potential administrative and provider participants based on her knowledge of their role, practice, and/or involvement in SCPs for breast cancer survivors. Survivors and caregivers were recruited through distribution of study information/posters in clinics.

Instrument

We developed interview guides (Appendix 2) iteratively with input from the study team. We tailored guides to stakeholders’ roles and elicited information regarding the outcomes that they expected and/or desired from SCPs.

Procedure

We conducted individual, semistructured interviews in person (in a private setting at the clinic location or a researcher’s office) or over the telephone after obtaining informed consent from stakeholders. Each interview lasted 30–45 min and was conducted by the site principal investigator or research associate. We audio-recorded and transcribed all interviews verbatim.

Analysis

Four study team members used inductive analysis to identify outcomes that stakeholders expect from SCP use (hereafter “stakeholder-identified outcomes”). We developed a codebook based on emerging data (Appendix 3) to guide and document coding using the constant comparative method [30,31,32]. All coders collaboratively coded three interview transcripts, resolving discrepancies until consistency in coding was achieved. Two study team members then independently coded the remaining interview transcripts to identify salient themes related to expected outcomes of SCP use. We used qualitative analysis software (ATLAS.ti at NCCH and KPSC; NVivo at NSHA) to organize and manage the data. We had regular research team meetings to review, discuss, and confirm findings. Finally, we organized findings into service outcomes (i.e., services provided and stakeholders’ perceptions of services provided [33]; e.g., efficiency, effectiveness) and patient outcomes (i.e., clinical outcomes; e.g., satisfaction, function) [34].

Mapping of RCT instruments onto stakeholder-identified outcomes

First, we disaggregated the RCT outcomes by stakeholder group and outcome type (service; patient). Next, one study team member mapped stakeholder-identified outcomes onto RCT outcomes. Finally, the larger research team, including three survivorship experts and one patient-reported outcomes expert, discussed and iterated mapping until we reached consensus.

Results

Outcomes assessed in RCTs

We identified 14 manuscripts reporting results from eight RCTs (Table 2). Service outcomes included information provision and helpfulness [18, 20, 21, 23,24,25]; communication, continuity, and coordination [15, 16, 20, 21, 24,25,26]; knowledge of disease and follow-up [15, 16, 19, 22]; treatment satisfaction [15, 16, 18,19,20,21,22,23]; provider adherence to guidelines [22, 24]; and cost-effectiveness of SCPs [17].

Table 2 Outcomes assessed in extant randomized controlled trials (RCT) of survivorship care plan (SCP) usea,c
Full size table

Patient outcomes include cancer-related distress [15, 16, 19, 27], health-related quality of life [15,16,17, 19,20,21,22,23, 25, 26], psychological distress (not necessarily attributable to cancer) [15, 16, 19, 22, 26], survivor adherence to recommended care [16, 25], health literacy [19], and knowledge and/or self-efficacy related to survivorship [26, 27].

Most RCTs assessed outcomes in survivors; only three studies assessed outcomes among cancer care providers or PCPs [24, 27, 28], and none assessed outcomes among caregivers.

Outcomes that stakeholders expect from SCPs

We interviewed 27 stakeholders in eight groups across three sites (Table 1). Table 3 displays the stakeholder-identified outcomes with illustrative quotes by outcome type (service; patient) and stakeholder group.

Table 3 Stakeholders’ expected outcomes from survivorship care plan (SCP) use
Full size table

Service outcomes

Enhancing communication and role clarity

Members of all stakeholder groups valued SCPs for their potential to facilitate communication among providers, survivors, and caregivers. Stakeholders also reported that SCPs might help formalize relationships between cancer care providers and PCPs, clarifying the roles of each in follow-up care. One participant commented, “I think from the specialist perspective, it’s the comfort of being able to say, ‘okay, I’ve done my piece, everybody knows what they need to do, I can let go of this without worrying that something is going to happen and not get followed up.’”

Meeting providers’ educational and informational needs

Providers described SCPs as a clear resource for describing what kind of follow-up care survivors should receive as well as when and from whom they should receive it. Stakeholders reported that SCPs gave clear surveillance and follow-up care guidelines to PCPs, whose knowledge of cancer and its sequelae may be limited. For cancer care providers, who largely focus on treatment, SCPs offered information about aspects of health beyond cancer, giving them a more “global perspective of the patient.”

Mitigating provider anxiety

We conceptualize provider anxiety as closely tied to other service outcomes because of its influence on providers’ ability and willingness to provide services. A cancer care provider stated, “Oncologists don’t want to transfer their patients because they don’t trust that the [follow-up care] doctor will know how to take care of them.” Cancer care providers saw SCPs as a road map for moving forward and reassuring survivors that they were “not just…saying, ‘bye bye, go on,’” thus reducing cancer care providers’ anxiety and allowing them to transition survivors to nonspecialist care. PCPs also reported feeling “peace of mind knowing what’s next” with SCPs.

Facilitating efficient discharge to primary care; promoting equitable and appropriate follow-up care

By strengthening communication among providers, stakeholders hoped that SCPs might facilitate post-treatment discharge to primary care. Several cancer care providers reported that cancer survivors often return to oncology soon after post-treatment discharge because PCPs were unclear about their role and current recommendations for follow-up care. Stakeholders reported that improving the discharge process to primary care might reduce oncologist burden, mitigate over- and underutilization of tests and services, and “save money in the long run” by maximizing efficiency. One cancer care provider said, “…you would assume that if the patient, primary care physician and acute care physicians and team are clear, that you’d have appropriate transfer, testing, appointments.” Finally, some stakeholders at NSHA (but not KPSC or NCCH) noted that SCPs might promote equity in care by ensuring that “everybody’s getting the same message, [and] patients are being treated the same.”

Patient outcomes

Meeting educational and informational needs

Survivors described SCPs as a permanent, reliable source of information for them, their caregivers, and providers in the face of provider turnover and contradictory information. Survivors highlighted many informational needs that SCPs could meet, often describing them as a map that can guide them with respect to what kind of follow-up care they should seek, when they should seek it, and from whom they should seek it. An oncologist reported that survivors in her care are “…satisfied that there is...a roadmap for them…there’s such a satisfaction in the fact that ‘I know about what my cancer was [and] what I was treated [with]—I got all that information…for the rest of my life…’” Survivors also reported that SCPs containing lifestyle recommendations (e.g., diet, exercise) could promote holistic health long after treatment. Of note, several cancer care stakeholders reported not expecting SCPs to single-handedly address all of survivors’ educational and informational needs, stressing the importance of ongoing provider-survivor communication.

Mitigating fear and anxiety

Survivors and caregivers discussed the potential for SCPs to alleviate anxieties associated with a survivor’s transition from treatment to follow-up care. For example, fear of recurrence and fear of abandonment by cancer care providers were frequently mentioned as major sources of anxiety for survivors and caregivers. This domain was closely related to other outcome domains: by enhancing communication and role clarity, meeting educational needs, and building capacity for survivor self-management, SCPs were expected to mitigate survivor and caregiver anxiety.

Improving survivor capacity for self-management; sense of control

Stakeholders repeatedly described healthcare delivery systems as uncoordinated, thus limiting their capacity for self-management and sense of control. In this context, stakeholders credited SCPs with the potential to empower survivors to engage with their providers and manage their own health, thus shifting control toward survivors and caregivers. A cancer care provider echoed this, saying that having an SCP “gives patients more control over their cancer journey.” One survivor elaborated, “…I put [the SCP] away and…that makes me feel like I still have control because if I want to review what’s happening then I can pull it out and then I can go and ask questions…It’s that information that I would need to ask my question, it’s already there...”

Comparison of outcomes assessed in RCTs and stakeholder-identified outcomes

Figure 1 displays overlap (and lack thereof) between outcomes assessed in RCTs and stakeholder-identified outcomes. RCTs assessed 20 service and 15 patient outcomes; stakeholders identified 19 service and 10 patient outcomes. Only three of eight RCTs assessed cancer care provider or PCP outcomes [23, 24, 27], and none assessed caregiver outcomes. In contrast, our interviews showed that stakeholders expected SCPs to influence the outcomes of survivors, caregivers, cancer care providers, PCPs, and organizations/systems.

Fig. 1
figure 1

Overlap between stakeholder-identified and randomized controlled trial outcomes. S survivors, C caregivers, CCP cancer care providers, PCP primary care providers, O/S organization/system, RCT randomized controlled trial. Description: Venn diagram comparing RCT outcomes to stakeholder outcomes; overlapping outcomes (middle section of Venn diagram) are described in a text bubble adjacent to the diagram circles. Key describes stakeholder outcome categories, numbered 1–5, and abbreviations for stakeholders referred to in the diagram

Full size image

Two of four stakeholder-identified service outcomes were assessed in RCTs: enhancing communication and role clarity, and facilitating efficient discharge and promoting equitable and appropriate follow-up care. The remaining two stakeholder-identified service outcomes were not assessed in any RCT: mitigating provider anxiety and meeting providers’ educational and information needs.

All three stakeholder-identified patient outcomes were assessed in RCTs: mitigating fear and anxiety, meeting educational and informational needs, and improving self-management capacity. However, RCTs did not fully capture the nuances of the stakeholder-identified patient outcomes. For example, Hershman et al. [19] and Smith et al. [27] assessed fears related to cancer (e.g., of recurrence and death), yet no RCT assessed stakeholder-identified patient outcomes of survivors’ fear of abandonment by their cancer care provider or caregivers’ fear of cancer recurrence in their loved one. Additionally, no RCT assessed stakeholder-identified patient outcomes of survivors’ and caregivers’ sense of control. RCTs assessed several outcomes that stakeholders did not report expecting from SCPs, including treatment satisfaction [15, 16, 18, 22] and HRQOL (e.g., mood and functionality) [15, 16, 19, 22, 25,26,27].

Discussion

In this study, we compared the outcomes that diverse stakeholders expect from SCPs to outcomes assessed in extant RCTs of SCPs’ effectiveness. The premise of this study was that some RCTs’ results may be null because they have evaluated outcomes that stakeholders may not expect from SCPs and on which SCPs might be expected to have limited influence (e.g., HRQOL). We found that many of the outcomes assessed in RCTs have not captured the nuances of stakeholder-identified outcomes. This finding may shed light on why some RCTs have found positive effects of SCPs only with respect to secondary outcomes, such as physician implementation of recommended care [25] and survivors’ understanding of their providers’ respective roles [15], which relate to stakeholder-identified outcomes. In contrast, outcomes for which RCT results were not statistically significant, such as satisfaction with care [20] and functional status, are not outcomes that stakeholders report expecting from SCPs.

RCTs assessed more patient outcomes than service outcomes, whereas stakeholders expected more service outcomes than patient outcomes. For example, stakeholders did not report expecting SCPs to influence HRQOL, an outcome frequently assessed in RCTs (6/8). This calls into question recent emphasis on HRQOL as a critical outcome [35]. In some cases, HRQOL may be too distal an outcome in RCTs of SCPs’ effectiveness [36]. Stakeholders in this study identified more proximal outcomes (e.g., capacity for self-management, sense of control), which are consistent with themes identified in a study characterizing transitional readiness among survivors [37]. By assessing distal outcomes, RCTs may burden stakeholders by collecting data on outcomes not directly influenced by SCPs. Relatedly, HRQOL is a complex construct, and stakeholders may intuitively recognize that SCPs are unlikely to influence HRQOL independent of other interventions.

We also found that RCTs seldom assessed SCPs’ outcomes from the perspective of diverse stakeholders (e.g., caregivers). For example, we found that stakeholders expect SCPs to mitigate providers’ fears and meet their educational and informational needs, yet RCTs tended not to assess these outcomes from the perspective of providers. Our findings also suggest that stakeholders expect SCPs to promote efficiency and cost-savings in follow-up care for survivors, an organization-/system-level outcome seldom assessed in RCTs. Interestingly, these results were largely consistent across study sites. (A notable exception was the finding that stakeholders at NSHA but not KPSC or NCCH viewed SCPs as promoting equity in care, possibly reflecting a difference in ethos underlying Canada’s national and the USA’s employer-based health system.)

SCPs are intended to improve care and outcomes among survivors, but how they do so is unclear. Our study addresses calls for stakeholder engagement in research [38,39,40,41] on whether and how SCPs achieve their goal of improving care and outcomes among survivors, aligning with the efforts of the US Patient-Centered Outcomes Research Institute and Canadian Strategy for Patient-Oriented Research, which advocate for increased engagement of patients and other stakeholders throughout the research process [42, 43]. Some of this research has begun: the Canadian Partnership Against Cancer System Performance Initiative found that SCPs facilitated help-seeking among survivors; the extent to which help-seeking in turn improves care and outcomes should be assessed in future research [44].

Limitations of our study should be considered. We conducted the study with 27 stakeholders in three healthcare delivery settings, so results may not be generalizable beyond the stakeholders in these settings; however, each has features that may be generalizable. We included university-, institute-, and integrated health system-based cancer programs in the southwestern and southeastern USA and Canada. Further, the goal of qualitative research is depth, not breadth, of knowledge [45]. Stakeholder interviews may have been subject to social desirability bias. Stakeholders may have wanted to avoid sounding presumptuous about the outcomes that they expected from SCPs; however, we emphasized the importance of being forthright in the interest of sound research findings. Stakeholder interviews may also have been subject to selection bias: stakeholders who could speak to SCP use may have therefore believed that SCPs would yield greater outcomes; however, this would only make our findings more conservative. Further, some interview participants did not advocate SCP use, suggesting that selection bias was unlikely. Finally, some instruments that were used to assess outcomes were not publicly available, so RCTs may have assessed outcomes that we were unable to identify.

Despite these limitations, our study is among the first to assess the extent to which RCTs have addressed outcomes that stakeholders report expecting from an intervention [46]. In addition to the importance of evaluating interventions with respect to outcomes that are relevant to stakeholders, careful stewardship of limited research funding should compel us to design RCTs that assess stakeholder-identified outcomes. In comparing RCT outcomes to stakeholder-identified outcomes, our study makes a methodological contribution that may be applied to other interventions.

Conclusions

Overall, the outcomes that RCTs have assessed have been inconsistent with recommendations for key outcomes of SCP use. Scholars have emphasized proximal service outcomes at survivor, provider, and system levels (e.g., improved understanding, coordination, and communication) [36, 47]. Many RCTs addressed some dimensions of these outcomes; however, RCTs tended to assess patient outcomes more distal than recommendations suggest. Future RCTs should assess the outcomes that stakeholders identified in this study but have not been assessed in extant RCTs—largely service outcomes from diverse stakeholders’ perspectives (see Fig. 1). We recognize the challenges associated with measuring the kinds of outcomes identified in this study and emphasized by survivorship experts. Indeed, assessing the many nuances of stakeholder-identified outcomes from each of their perspectives may be costly and infeasible. Nevertheless, if RCTs continue to assess outcomes that break with stakeholders’ expectations and from a subset of stakeholders’ perspectives, their results are likely to continue to be mixed which may in turn limit SCP implementation. Future work is needed to identify or develop valid and reliable measures of the stakeholder-identified outcomes identified in this study so that clearer evidence of SCPs’ effectiveness may be achieved.