Background

With the increasing numbers of individuals surviving a diagnosis of cancer [1], an aging population, and more individuals experiencing multi-morbidity, primary care providers (PCPs) are seeing more patients with a history of cancer in their practice [1, 2]. Often called upon to help provide counseling for cancer prevention and screening strategies for their patients, PCPs will now need to know more about how to help support their patients through treatment and into long-term survivorship [3, 4]. Identifying and treating late and long-term effects of an expanding array of cancer treatments, managing comorbid conditions, incorporating health promotion strategies to maintain wellness, and continuing routine preventive care all necessitate strategies to adequately prepare the primary care workforce for the phase of cancer care now widely recognized as survivorship [5]. Calls have been growing for greater PCP involvement in the care of individuals going through cancer treatment [6].

The transition back to primary care after a cancer diagnoses and treatment has been identified as a source of high anxiety [7, 8]. From a cancer survivor perspective, gaps in care that may be within the scope of the primary care setting include informational needs, supportive and psychosocial support, and help with medical concerns [9]. The PCP community needs education and resources to help them meet the needs of the growing cancer survivor population. Though there are differing opinions and a lack of evidence on the best model of providing survivorship care within a health care system [2, 8], there is also acknowledgement that PCPs are willing to and should be involved [2, 11, 12] and specifically that educational offerings should be expanded in this area [13]. Survivorship care plans (SCPs) are an evolving standard of care in the follow-up of cancer patients [14, 15]; however, SCPs will likely not be sufficient alone to bridge the gap between oncology and primary care. In particular, SCPs will need to clarify the role of the PCP, especially in rural settings away from tertiary cancer treatment centers [16].

Educational interventions have been developed in the medical education or training setting [17, 18] or for institutions engaged in survivorship work [19]. Some online informational and continuing education modules on cancer survivorship are now available for practicing providers [20, 21], as are evolving competencies in survivorship care [22]. However, interventions to improve survivorship care which actively reach out to engage PCPs in their settings of care are lacking. Through the use of Boot Camp Translation methodology [23] and a quality improvement approach informed by appreciative inquiry [24], our team has developed and implemented a novel cancer survivorship training curriculum for rural primary care practices, which represent a site of care particularly relevant to individuals living in rural areas where primary care may be the predominant source of health care available. In Colorado, where the majority of counties are designated as rural or frontier, this is also relevant to patients and their PCPs living in areas designated as medically underserved or health professional shortage areas. This multi-modal intervention was delivered between 2014 and 2016 to 32 rural primary care practices participating in a rural practice-based research network (PBRN) in Colorado. The purpose of this paper is to describe the intervention and its development, as well as results of the program evaluation.

Methods

Setting and Context

Setting

Housed in the Department of Family Medicine at the University of Colorado Denver Anschutz Medical Campus, the High Plains Research Network (HPRN) is a community and practice-based research network (PBRN) that works with 54 primary care practices, 167 primary care clinicians, 16 hospitals, and 24 behavioral health clinics in the 16 counties in eastern Colorado. HPRN counties are medically underserved or health professional shortage areas. The HPRN region includes one local cancer center; however, there are no oncologists who live in the region. The local cancer center and several other communities have oncologists that visit once or twice per month. Patients typically travel 1–4 h for oncology care during acute treatment and follow-up. HPRN provides practice facilitation support across projects. The HPRN is guided by an active Community Advisory Council (C.A.C.) of local farmers, ranchers, schoolteachers, business owners, students, and health care providers to assure that the research is grounded in real patient and provider experiences.

Context

There are relatively few other resources such as peer support groups, patient education, wellness programs, or services specifically targeting the needs of survivors available in the HPRN. There is a strong sense of community in this rural, sparsely populated area of the state, and self-reliance and pride in taking care of one’s own needs is valued. Primary care providers are often the main providers available to support patients in post-cancer care needs. The strong history of work to improve the health in the region as part of the past collective accomplishments of the HPRN, including the longevity and commitment on the part of university-based investigators, made it possible for a new effort in cancer survivorship to be well-received by community members.

iSURVIVE Practice Team Training

The educational intervention consisted of a multimodal curriculum that addressed specific content outlined as relevant by the Institute of Medicine [6], including long-term sequelae; psychosocial concerns; statistics in health care access; quality assurance and models of care; prevention, detection, and treatment of recurrent and secondary cancers; and rehabilitation issues [6]. With this as a starting point, further refinement of the content and format of the curriculum were guided by the HPRN C.A.C. as well as a Scientific Advisory Board, which consisted of clinical experts at the University of Colorado School of Medicine from various disciplines relevant to survivorship care (for example, medical oncology, radiation oncology, nutrition, physical therapy, spiritual care, among others). Collectively, the intervention was named iSurvive by the HPRN C.A.C.

This study used SOuND Team Training™ to deliver four 1-h in-person sessions to full practice teams, including clinicians, nurse teams, medical assistants, front desk staff, care coordinators, behavioral health providers, and others. The training was both didactic and interactive and was delivered by an HPRN Health Educator and Community Research Liaison with existing relationships to the community and providers and who were trained in cancer survivorship issues by project staff. Appreciative inquiry, a quality improvement strategy which promotes the discovery of shared motivations, envisioning a transformed future, and learning around implementation of a change process (24), was a key strategy deployed in the curriculum delivery. In all sessions, participants were invited to share current successes in chronic care delivery and envision desired changes and ways to build on these successes to similarly achieve optimum survivorship care delivery (see Table 1 for description of intervention sessions and corresponding survivorship care domains). A supplemental series of 12 monthly 1-h webinars were provided via live, interactive sessions by a Scientific Advisory Board member and archived for access anytime on the iSURVIVE website (Table 2).

Table 1 In-person component of cancer survivorship curriculum
Full size table
Table 2 Participant demographics and practice type of questionnaire respondents
Full size table

Overview of Evaluation Methods

We hypothesized that the intervention would produce both short-term impacts (knowledge and awareness) as well as promote intention for behavior change in regards to survivorship care delivery. Therefore, the study used a triangulated mixed methods design [25], which included a questionnaire (quantitative component) to detect changes in knowledge and awareness and interviews (qualitative component) to capture findings not evident in the questionnaire as well as to solicit open-ended comments about the impact and perceived utility of the program. The qualitative inquiry was guided by a naturalistic approach [26], to uncover and describe impacts of the intervention, both intended and unintended.

Quantitative Evaluation Methods

Questionnaire Development

The research team developed a 14-item questionnaire to examine change in knowledge in practices participating in the iSURVIVE training. Knowledge questions addressed specific aspects of cancer survivorship care that were addressed in the iSURVIVE curriculum and webinars. Questions were a mix of straightforward knowledge, multiple choice questions, such as the definition of cancer survivorship and survivorship rates, and detailed cancer-specific clinical decision-making questions using case scenarios. The post-test questionnaire also included a question about attendance at each of four in-person sessions. Questionnaires were anonymous. The questionnaire was piloted with six primary care physicians at the University of Colorado School of Medicine campus and five primary care practice staff at a community health center. The questionnaire was modified for clarity based on this feedback.

Data Collection

Team members in any role at participating primary care practices were invited to complete a questionnaire. Baseline questionnaires were mailed 2 to 3 weeks prior to the first SOuND Team Training™ session to the main contact at each participating practice. Questionnaire packets included an information sheet, questionnaire, and postage-paid envelope to return to the study coordinator. The same questionnaire was administered at the end of the fourth and final in-person session. The main contact at the practice was asked to deliver questionnaire packets to team members not present at the final session. The study was approved by the Colorado Multiple Institutional Review Board.

Analysis

Completion rates were determined based upon the number of questionnaires that were provided to each participating site and the total number of completed questionnaires that were returned. For those participants who chose not to participate in or were unable to complete a questionnaire, no demographic information was collected.

The main outcome of this analysis was the change in percent of correct answers from baseline to post-test. We explored this outcome at three levels: (1) individual participant, to determine the aggregate change in knowledge from baseline to post-test; (2) practice role, to determine if the change in knowledge was significantly different in one group than in the others; and (3) by exposure to training (attendance), to determine if the change in knowledge was significantly different based on the number of training sessions attended. The practice role variable was categorized into three groups: prescribers (MD, DO, NP, or PA) in a primary care capacity, staff that interact with patients but are not prescribers (medical assistant, nurse staff, patient navigator/facilitator, pharmacist, behavioral health care provider, and others who interact with patients during care encounters), and other support staff (administrator, front office, medical records, billing).

Descriptive statistics (means, proportions, frequency distributions) were generated for demographics and knowledge variables. Baseline and post-test questionnaires were not linked between individuals to allow use of all information collected at both timepoints, regardless of staff changes over time, thus requiring an aggregate level of knowledge to be compared across time. Frequencies were used to assess demographic characteristics. t tests were used to evaluate baseline and post-test variance in the mean number of correct responses over time. The SAS GLM procedure was used to assess interactions between clinic role and changes in knowledge over time. The SAS Regression procedure was used to assess the difference in change in knowledge score among varying attendance levels by reporting values and 95% confidence intervals (CI). All statistical analysis was conducted using SAS version 9.4 (Copyright © 2013 SAS Institute Inc., Cary, NC, USA).

Qualitative Evaluation Methods

Approach

At a minimum of 1 year after the completion of the intervention, key informant interviews using an ethnographic approach were carried out in order to describe the impact of the training program and explore attitudes and practices regarding survivorship care delivery. Specifically, in the qualitative analyses reported in this article, we sought to describe the potential impact beyond change in knowledge and awareness in order to understand the impact of the intervention on change in practice as well as related contextual factors that affected implementation of knowledge from the intervention.

Data Collection

An interview guide was developed to standardize data collection across interviewers and settings. We inquired about workflows and practices in survivorship care covered in the intervention, including ways to systematically identify survivors and assess their needs; finding and using referral partners for typical survivorship late and long-term side effects (which were discussed in the sessions); organizational capacity and confidence to deliver team-based survivorship care using existing evidence; and barriers and facilitators to implementation. All questions were open-ended, including a final item asking them to offer any opinions or thoughts not covered in the interview. Interviews were conducted by project team members not involved in intervention delivery. A second team member took notes which were reconciled with the interviewer’s notes within 1 week of the interview to validate the findings.

Recruitment

Potential participants in the qualitative data collection were identified by research team, community liaisons, and health educators who delivered the intervention. The goal was to identify individuals at each practice site who could speak to organizational capacity. Initial recruitment was through electronic mail using a standardized template, which was followed by phone calls.

Analysis

Using a team-based approach, immersion crystallization was used to identify emergent themes from the primary data until thematic saturation was reached [27]. Three (3) analysis meetings were held with all research team members, with the participation of the interviewers to ensure accuracy.

Results

Intervention Delivery

Overall, four training sessions at 32 practices (128 total training sessions) were delivered to 355 unique participants by two health educators and two community liaisons over the 4-year period. Participating practices averaged 10.3 team members (clinicians and staff). The average number of attendees at each session was 7.3 for session 1, 5.4 for session 2, 5.2 for session 3, and 4.5 for session 4.

Participant Characteristics

Questionnaire

For the quantitative portion of the study, a total of 254 questionnaires were completed at baseline (71% response rate) and 218 at post-test (62% response rate) across 32 participating primary care practices. There were no significant differences in characteristics of respondents at baseline and post-test. At both baseline and post-test, 87% of respondents were female, and most reported being White (81% and 85%, respectively). About a third of respondents were of Hispanic ethnicity (35% and 33%). About a quarter described their role as a clinician prescriber (26%), slightly over a third were in non-prescribing roles in which they have clinical interaction with patients (35% and 39%), and just less than a quarter were in support roles (24% and 19%).

Interviews

For the qualitative portion of the study, a total of 21 practices and 44 individuals participated and provided demographic data (43% were clinicians/prescribing providers, 32% nurse care team, 23% administrative, 2% behavioral health). All invited clinics participated in the interviews.

Quantitative Results: Changes in Knowledge

At an aggregate level, the percent of correct responses to the knowledge questionnaire increased significantly between baseline and the completion of the curriculum, almost doubling between baseline and post-test (25% vs 46%, p < .001). The percent of correct responses varied by question, and all but one change were significant, as shown in Fig. 1.

Fig. 1
figure 1

Percent of correct responses per question and overall pre (n = 254) to post-test (n = 218) *p < .0001

Full size image

There was no significant interaction between role in the clinic and change in knowledge over time (F = 2.03, p = .13), indicating that the amount of change in knowledge from baseline to post-intervention did not differ significantly between roles. However, there was a significant and linear association between attendance at SOuND Team Training™ sessions and the number of correct answers at post-test (regression r squared = .26, p < .0001), as shown in Fig. 2.

Fig. 2
figure 2

Average knowledge score at post-test by attendance (n = 218) P < .0001

Full size image

Qualitative Results: Themes and Findings

High inter-rater reliability was observed between the interviewers [23] and several themes were identified by the research team from their notes, as described below, along with representative comments from the interviews.

Theme 1: Overall Positive Perspective and Immediate Impact of Training

There were multiple statements by providers describing and supporting the various types of positive impacts the intervention had on their practice. Most participants (at least 20 of the 21) stated that the iSURVIVE training was informative, educational, and useful in their clinical practice. Many providers (n = 19) made various statements indicating the iSURVIVE training altered their approach to cancer survivors as it increased empathy and awareness of cancer survivorship needs. One participant stated, “The training helped me to understand the implications of cancer even if the patient had it a long time ago”. A few providers (less than 10) stated that the training increased their understanding and awareness of the long-term effects of cancer. Most participants appreciated the practice-level and patient-level resources provided through the iSURVIVE training and that they were aware of the continual webinars, website resources, and service manuals. Other changes in practice noted by some providers included increased comfort and thoroughness during history taking and that the training normalized their comfort in asking questions and taking a proactive approach.

Theme 2: Taking Action

Implementation of cancer survivorship care varied across participating practices, which fits well with the iSURVIVE training curriculum because it was developed to encourage adaptable practice change. For example, some participants reiterated that execution of cancer survivorship care depends on the individual. When putting cancer survivorship care into action, specific attributes such as provider personal commitment, attitudes, and motivation were mentioned. Most participants mentioned making 1–2 small changes such as more comprehensive history taking, whereas a few mentioned more immediate changes in practice. Following the training, one provider stated “I ordered a PET scan for a patient who is a cancer survivor and hadn’t had a scan in 10 years”.

Theme 3: Training Created Intention to Make Future Changes in Survivorship Care

Participants revealed that the iSURVIVE training generated new ideas for cancer survivorship care in their practices. Many participants stated these opportunities could foster patient-provider shared responsibility in cancer survivorship care. Some providers encouraged ways for patient to self-disclose as a cancer survivor. A few participants thought posters within the clinic and exam room may encourage patients to advocate for themselves, though one provider mentioned it may not be necessary as she mentioned ‘they will tell us’ if they are cancer survivors. Many practice-level suggestions were shared that aimed to improve current processes around cancer survivorship care. For example, most practices are presently tracking other chronic conditions, and some plan to pursue cancer survivorship similarly. Many participants proposed ideas that focused on efforts within the electronic health record (EHR), such as generating cancer-specific reports to improve follow-up care, working with IT team to create an option to ‘flag’ cancer survivors in the EHR, and creating reminders in the system (similar to other conditions). Cancer survivor database/registry resources do not currently exist at any participating practices. Participants suggested/concluded this should be incorporated as other current chronic condition databases are in place. One practice developed plans to devote a medical assistant to this specific area of care and follow-up. One participant indicated, “We would like to create a formal process of follow-up and goals for our cancer survivors”.

Theme 4: Contextual Impact (Barriers and Facilitators) of Survivorship Care in Primary Care Setting

Participants identified several factors that have contributed to lack of execution of survivorship care. For example, many participants indicated that education is not enough. Sample statements from participants included “How do we put this into practice”? and “There are no models of integrated care to follow”. In general, practices do not have an established workflow or protocol for cancer survivorship nor is the survivorship care plan enough to generate practice-level change. There needs to be an obvious need for actionable items for the care plan to be implemented and/or utilized in order to change care delivery. Practices were unable to determine if their current cancer survivorship care was effective and that evaluation is limited, which presents a barrier in justifying the need for programs. Further, both practice level and organizational level limitations were expressed by the participants, including readiness for change, internal capacity, and lack of assigned staff or workflows (practice level). External and organizational factors noted included lack of required policy measures for cancer survivorship care, inadequate communication with Oncology, and lack of evidence to guide decision making. Some participants expressed barriers regarding time; competing priorities and pre-existing requirements make implement new workflows challenging.

Several facilitators of survivorship care change were also noted. Some participants stated that personal attributes of staff (e.g., commitment, attitudes) were what facilitated change in cancer survivorship care. Some participants felt the patient will initiate cancer survivorship care. A few providers relied on the patients as ‘they will tell me’. Reminders for both patients and providers were mentioned as a facilitating factor to cancer survivorship care. Some participants stated family support and involvement nurtured cancer care. In rural communities, continuity of care enables practices to see long-term effect and observe change. Having adaptable next steps and action items were suggested to incorporate into practice workflow.

Half the practices interviewed indicated they did not have a routine system established to identify cancer survivor needs. The majority of practices did not have existing personnel specifically designated for cancer follow-up care. However, many respondents (n = 12) identified opportunities for planned process or practice changes that may help in the care of cancer survivors, or they had incorporated such changes by the time of the interview. When asked if the changes were attributed to the training, 11 of the 12 indicated positively.

Discussion

Results of our novel practice team-based educational and practice change intervention (iSurvive) carried out in a rural primary care practice-based research network (PBRN) demonstrate that trainings targeting the healthcare team are feasible and acceptable and can be successfully carried out with positive impacts on knowledge and on survivorship care delivery across all practice team members. The results described here are novel in that they reflect the potential to increase knowledge of cancer survivorship issues in community primary care settings, outside of an academic medical center environment.

With regard to impacts on knowledge about cancer survivorship care, the positive changes were not unique to PCPs but were demonstrated for all primary care practice team members. Although not the primary purpose of our evaluation, this suggests that by including all clinical practice team members, the beneficial impact for patients can be greater than by focusing solely on clinicians and supports the concept of a primary care medical home and team-based care delivery.

The majority of practice respondents participating in this study indicated that they had not previously recognized the care needs of their patients who had a diagnosis of cancer as distinct survivorship issues. Additionally, our qualitative analysis demonstrated that the intervention created intention to change the way care is provided for cancer patients. Many forms of cancer and treatment-related late and long-term effects are increasingly recognized as essentially chronic conditions or risk factors for future disease; yet, successful strategies used in primary care practice aimed at managing common chronic conditions such as diabetes or cardiovascular health are not similarly applied to cancer survivorship. For example, several chronic disease management strategies that could be leveraged in delivering cancer survivorship care were noted by participants in our interviews including changes to the EHR to flag survivors and issue reminders and alerts; the establishment of registries so that survivors can be targeted for interventions and outcome measurement; and the development of evidence-based protocols for survivor follow-up care.

In keeping with these findings, respondents also noted that education and awareness are not enough to change survivorship care delivery in the primary care setting. Cancer survivorship is a very broad entity, and clinical guidance is still evolving. However, our team believes that the progress made with this intervention begins the conversation about caring for cancer survivors and sets the stage for more specific content to be developed, implemented, and evaluated. This curriculum was considered an introduction to survivorship care. Much of the teaching spoke to learning where to go for reliable information in real-time. The model described here to promote change in practice could be envisioned as a strategy to provide support and education that is complementary to existing efforts such as online modules or websites [20]. Our results confirm that a gap in care exists in the primary care setting around the phase of cancer care known as survivorship, which starts at the time of diagnosis. As with other chronic conditions and perhaps arguably more so, issues around care coordination and communication with various teams of oncologic specialists can be challenging as we learn more about the non-oncologic effects of a cancer diagnosis. Our study demonstrated that barriers do exist to implementing changes in survivorship care delivery, such as a need for more practical recommendations and a lack of strategies to incorporate into day to day practice. We hypothesize that the building of the evidence base around cancer survivorship care and development of more specific guidelines will create a context more conducive to survivorship care, and practices will benefit from training that includes ongoing practice facilitation support. The inclusion of survivorship in value-based care and other healthcare reform efforts will likely also play a significant role in promoting cancer survivorship care delivery in the primary care setting.

Our findings from the qualitative inquiry suggest that theory-driven factors which support primary care transformation and the dissemination of evidence-based approaches in healthcare delivery at the practice level likely play a similar role in survivorship care. Various theory-driven factors have been hypothesized in several dissemination and implementation science (D&I) frameworks [28], such as the Consolidated Framework for Implementation Research (CFIR) and the Interactive Systems Framework (ISF). While the specific relationship and definition of these factors may differ across theories, these models all point to internal factors (organizational climate, characteristics of individuals, and leadership), external factors (policies and incentives), and factors related to the intervention itself (knowledge and beliefs about the efficacy of the intervention among adopters, perceived relative advantage, and complexity of the intervention). Internal factors were discussed by many respondents, and the lack of availability of cancer survivorship guidelines (factor related to the intervention itself) was also discussed as a barrier by participants in our interviews. However, other external factors, such as requirements or regulations regarding the provision of survivorship care, the establishment of clear guidelines from third party payers for coverage of survivorship care, and the identification of suitable metrics for value-based survivorship care, were notably absent in our discussions with primary care providers in regard to drivers for cancer survivorship care delivery. This suggests that there is significant room for improvement at the system level to build in facilitators of survivorship care in the primary care setting and that further investigation is needed to understand how they may be applied.

Limitations

We describe evaluation of an educational intervention in rural Colorado among practices that chose to participate. While a majority of practices in the HPRN participated in iSurvive, they might represent a different group of practices and providers that are more willing or able to take on additional training. While we did not specifically measure clinical outcomes in this questionnaire, increased knowledge may be a crucial first step towards improved patient care and is a generally accepted component of practice quality improvement. Our mixed methods approach did provide additional context in which to evaluate overall impact of the intervention beyond considering changes in knowledge.

Some of the questions on the pre/post-test were challenging, which was intentional to avoid a potential ceiling effect. While there was improvement in overall scores, the fact that only 46% of post-test scores were correct raises the possibility that the curriculum developed for the training was overly broad, that important learning points were not emphasized strongly enough, or that questionnaire questions may have been perceived as irrelevant to the respondents’ roles. These suggest important points for continued study in this area as well as quality improvements in the intervention. Regardless, the results indicate a significant gain in knowledge of everyone in the practice, regardless of role, which can be a crucial step towards team-based care.

To our knowledge, iSurvive is the only cancer survivorship focused educational intervention that engages primary care practice teams by working within an established PBRN for intervention delivery. The participation of the community and providers helped to ensure that this training curriculum was well-accepted in rural communities in the HPRN and suggests that future survivorship care transformation in the primary care setting is feasible if key stakeholders are engaged. Additionally, the fact that the intervention is adaptable was noted as a positive benefit by many participants across our varied clinic settings. This is in keeping with current implementation theory [28], which suggests that modifiable intervention components facilitate adaptation of innovations in practice change. Combined with the high perceived value reported by respondents, the evidence from these interviews supports the potential wider dissemination of the iSurvive Program. These data also suggest the need for additional investigation into other ways beyond education that primary care practices can be supported to ensure the needs of the growing cancer survivor population in the US are met.