Introduction

An estimated 13.7 million Americans with a history of cancer were alive on January 1, 2012 [1]. The number of cancer survivors is expected to double by 2050 [2]. Cancer survivorship is a national public health concern. The Healthy People 2010 program of the Centers for Disease Control and Prevention [3] includes a benchmark for addressing cancer survivorship goals. The American Cancer Society (ACS) [4] has a 2015 goal to improve the quality of life of all those affected by cancer. Oeffinger and McCabe [5] asserted that advances in long-term cancer follow-up care would require the development of best practices in survivorship care training as patients transition from oncology to primary care providers. Primary care providers are uncertain about providing survivorship care, specifically frequency and duration of surveillance testing [2], clinical practice guidelines [6], long-term complications [7], and their own status of educational preparedness [8]. In a similar study on 399 nurses, 49 % reported a lack of survivorship knowledge [9].

The landmark Institute of Medicine (IOM) report resulted in ten recommendations addressing the ongoing needs of survivors. Priorities included raising awareness about cancer survivorship and providing guidance on the essential components of care. Emphasis on the development and widespread use of treatment summaries and survivorship care plans (SCP) by the IOM and others is intended to facilitate the transition from oncology to primary care. The IOM recommended that efforts be expanded to provide educational opportunities to health-care providers and equip them to address the health care and diverse psychosocial needs of survivors, including quality-of-life issues [10]. The present study provided an online cancer survivorship primer for primary care, assessed learning gain and educational quality and captured participants' intended practice changes upon completion.

Methods

Development of the Educational Intervention

An evidence-based cancer survivorship care course (primer) was developed suitable for continuing medical education/continuing education (CME/CE) fulfillment by participating US clinicians. The course covered relevant epidemiology, survivor issues, and survivorship care plans and provided links to resources and available guidelines. Four disease conditions were covered, including breast cancer, prostate cancer, colorectal cancer, and non-Hodgkin lymphoma (primarily due to the quality-of-life burden in this group). Upon acceptance and approval of a needs assessment, course content was aligned to objectives using the revised taxonomy for the cognitive domain [10]. The course was designed to address the following learning objectives:

  • evaluate late effects of cancer treatment for survivors of breast, colorectal, and prostate cancers and non-Hodgkin lymphoma;

  • identify psychosocial stressors and late effects faced by cancer survivors;

  • identify available sources for surveillance and prevention guidelines applicable to breast, colorectal, and prostate cancer and non-Hodgkin lymphoma survivors; and

  • indicate the importance of complementary and alternative medicine (CAM) use among survivors.

Epidemiological content included prevalence, lifetime risk, mortality, and incidence for each condition; content specific to survivorship included late effects of treatment, survivor concerns, and psychosocial stressors. Two patient-based case scenarios (breast cancer and prostate cancer patients) were presented. The patient scenarios were followed by discussion regarding diagnosis, patient evaluation, and management strategies. Non-graded questions followed the cases and contained targeted feedback for each response option. Surveillance and prevention guidelines and survivorship care plans were outlined with links to downloadable templates and additional information. A section on CAM was also included with a posttest question as this is considered an important area often overlooked by clinicians.

Instructional Design

Design of the cancer survivorship intervention was approached using the Analysis, Design, Development, Implementation and Evaluation or “ADDIE” instructional systems process model [11]. ADDIE provided a systematic method for analysis of the knowledge gap and learner characteristics, design of objectives and outcomes, development and implementation for online delivery, and evaluation of instructional goals. The Association of American Medical Colleges [12] recommends evidence-based educational features that promote effective learning, including the instructional media principles of Mayer [13] and the “Nine Events of Instruction” of Gagné [14]. Mayer's 12 evidence-based principles for multimedia–modality, interactivity, and spatial contiguity [13] are grounded in the psychology of human learning and informed the instructional design methodology for this course. The modality principle states that learning is facilitated when speech is used for complex topics rather than onscreen text. In the survivorship course, spoken words of survivors were carefully selected to convey quality-of-life impact and personal experience, rather than using text-based quotations. The interactivity principle indicates that learners retain content when they can control the pace of the presentation. Within the Medscape course, participants could navigate between sections of material at will, as well as exit and return to complete the course at a later time. The principle of spatial contiguity reduces cognitive load for multimedia learners through placement of on screen text and related visual components in close proximity. Contiguity regarding placement of text, graphics, and video clips on each screen was therefore considered carefully.

Instructional material was constructed in accord with the Gagné model, which emphasizes the following sequence: gain attention, present learning objectives, stimulate recall of prior knowledge, present educational content, provide learner guidance, measure performance, provide feedback, assess knowledge, and enhance retention and transfer [14]. The course opened with vivid portrayals of survivor stories to capture learner attention. Educational objectives were introduced and linked to prior knowledge through epidemiology for each disease condition. Guidance for learning was supplemented with IOM video clips highlighting leading survivorship experts. Learners were presented an opportunity to practice given non-graded case questions with targeted feedback for each option. Enhancement of retention and transfer was addressed though provision of direct links to survivorship care plan templates, current resources, and available guidelines. Guideline links were provided to Physician's Desk Reference (PDR), American Academy of Family Physicians (AAFP), ACS, American Society of Clinical Oncology (ASCO), National Comprehensive Cancer Network (NCCN), and National Cancer Institute (NCI) follow-up and surveillance. Validated screening scales gleaned from the Quality of Life (QoL) literature for measurement of psychosocial impact were provided as a clinical resource [14].

Case-based learning has shown promise for improvement of clinical decision-making skills [15, 16] and is an efficient methodology to provide integration of curriculum content. According to work done at the Johns Hopkins Evidence-based Practice Center, use of patient cases is associated with improvements in knowledge and courses that implement “multiple techniques rather than single media or print, increase the likelihood of significant learning gain” [17]. Case scenarios “challenge learners with unfamiliar problems that require adaptation or reconfiguration of prior knowledge to develop new strategies and solutions” [18]. Case development includes introduction of a real-world scenario that includes current symptoms and signs, past medical history, and medications followed by a discussion of relevant sequelae, surveillance, and follow-up guidance. Non-graded case questions that include detailed feedback for each response option engage the learner interactively with clinical decision making associated with the case. Care was taken in the development of the patient cases to present frequently encountered, clinically relevant problems.

Survey Content and Administration

The Medscape registration process is well defined, user-friendly, and provided the opportunity to collect research survey data by leveraging existing database structure and collection methods. Therefore, demographic data were simply culled from the user registration page. The Medscape terms of service agreement allows for collection and dissemination of de-identified course polling data to third parties, including researchers. The platform stratified domestic and international users, allowing for targeted domestic research.

All participants completed the four-question multiple choice survivorship pretest/posttest. Item one tested risk factors for breast cancer survival, item two tested late effects of prostate cancer treatment, and item three tested surveillance and follow-up for secondary cancers. These questions were previously validated with medical students and oncology fellows [19]. The fourth item tested knowledge regarding the use of CAM by survivors and was validated by expert opinion. Questions were matched to supporting content material under each educational objective.

Course evaluation included two quality measures after the posttest. The items were presented as yes/no and five-point Likert scale: 1 = strongly agree (SA), 2 = agree (A), 3 = no opinion (NO), 4 = disagree (D), and 5 = strongly disagree (SD). The first survey item captured participant responses regarding whether the course promoted improvement in survivorship care. Participants were asked to answer a second survey question regarding whether the material was organized in a manner appropriate for learning to occur. The first survey item was intended to confirm whether the survivorship topics were on target and met expectations. The second item was intended to measure efficacy of the instructional design and organization of the course. Finally, participants were asked if they intended to make practice changes, using a standard Medscape Education-validated list of practice changes.

Data Analysis

Data collection was conducted by data extraction of polling item responses to the four knowledge items and two program evaluation items. All data were collected through the Medscape Education secure computer system and supplied to the authors in de-identified files on September 24, 2012. Statistical comparison of overall pretest and posttest knowledge gain was made using the paired Student's t test for continuous variables including effect size by using Cohen's d. Significance was set at p < 0.05. Standardized mean difference from pretest to posttest for each survivorship knowledge question was assessed by clinical specialty with 95 % confidence intervals for d. Magnitude of the pre- to posttest effect was evaluated according to Cohen's criteria [20] where 0.20 = small effect, 0.50 = moderate effect, and 0.8 = large effect. Ancillary analysis included comparison of intended practice changes post-intervention across clinical specialty groups using Pearson's chi square.

Course Deployment

Medscape Education provides health-care continuing education, free of charge. Existing mechanisms are in place for provider registration, data collection, and delivery platform. The course entitled “Cancer Survivorship Primer for Primary Care” (Medscape course number 763570) was uploaded to the Medscape Education website (www.medscape.org) on May 19, 2012. The course provided physicians with 1.25 AMA Physician's Recognition Award (PRA) Category 1 Credit(s)™ and nurses with 1.25 ANCC Contact Hour(s). AOA level 2-B accreditation was obtained independently through coordination with the A.T. Still University Office of Continuing Education, since the Medscape system does not offer it by default.

Results

Course Participants

The population for this study included online participation by US clinicians voluntarily seeking online CME/CE credit. Inclusion criteria were applied to course completions to eliminate non-US responders and participants who did not complete the course in its entirety. Participants representing full cases on the survivorship survey items (no missing data) were retained for study. Recruitment was conducted by the hosting service using emails to registered users and website announcements through a variety of registered content subscriber communication channels tuned to specialty, including Medscape family medicine, internal medicine, hematology/oncology, and nursing education. The eligible sample (n = 1,521) consisted of MDs (n = 126), NPs (n = 183) and RNs (n = 1,168), PAs (n = 31), and DOs (n = 13). RNs represented the highest percentage of clinicians responding (l6.8) and DOs the lowest (0.9). Nearly three quarters (72 %) of the sample identified within the six categories are shown in Fig. 1.

Fig. 1
figure 1

Participant specialties by percentage and frequency. Oncology and hematology includes medical, pediatric, and surgical oncology. Other medical subspecialties includes cardiology, critical care, urology, and emergency medicine. Surgical subspecialties includes orthopedic and general surgery

Full size image

Knowledge Gain

Exposure to a comprehensive cancer survivorship educational intervention resulted in significantly higher knowledge scores. The correlated t test shows that the 1.4 point difference between the test means was statistically significant by a two-tailed test (p < 0.0005). The effect size (d = 1.72) was large and represented a substantive finding and high practical significance. Additionally, knowledge gains were broken down by pretest/posttest knowledge item and clinical practice specialty. The magnitude of change from pretest to posttest with 95 % confidence intervals is provided in Table 1. All effect sizes were positive; most were moderate to large.

Table 1 Knowledge item mean increase and effect size by clinical specialty
Full size table

According to the range of standardized mean differences, the largest effect sizes were observed in the other medical specialties group. This group accounted for the largest pretest to posttest learning gains in the course for question items 2 (late effects of prostate cancer treatment, d = 1.38), 3 (surveillance for secondary cancers, d = 1.12), and 4 (CAM use by survivors, d = 0.92).

For item 1 (risks factors for breast cancer survival), psychiatry and mental health showed the most improvement from pretest to posttest (d = 0.74), while primary care had the highest pretest mean on this item. The most challenging item for all specialties, including oncology/hematology, was item 2 (late effects of prostate cancer treatment). All specialty groups improved significantly on this question by the posttest with large mean differences (0.80 and above) indicated by 92 % and moderate gains for the remainder.

Educational Quality

Responses were collected for two program evaluation items at the conclusion of the course. The first survey item result captured participant responses regarding whether the course promoted improvement in cancer survivorship care. Nearly 100 % of the eligible participants (n = 1,521) selected yes to this question (98.9 %).

Participants were asked to respond to a second survey item on whether the material was organized clearly for learning to occur. This question was designed to assess the effectiveness of the instructional design and sequencing. Responses to this item demonstrated a 77.3 % (n = 1,176) response rate across all clinicians (345 did not answer this item), including leaving the strongly disagree (SD) option blank; 31 responded, yet had no opinion (NO); and only two RNs disagreed (D). Nearly three quarters of clinicians responding (97.2 %) strongly agreed (566) or agreed (577) that the course was organized clearly.

Since it was not possible to perform a follow-up survey, participants were asked about their intent to implement changes in clinical practice based upon participation in the intervention. As shown in Table 2, eligible study participants (959 or 63.1 %) reported on alternative communication strategies with patients and families as the most likely change they might make to clinical practice based on completion of the intervention. This was followed by modification of treatment plans (14.0 %), incorporation of different diagnostic strategies into patient evaluation (11.5 %), and changes to screening and prevention (11.4 %). Specific diagnostic strategy changes were not collected. There were no significant differences between clinicians (MD, DO, RN, PA, and NP) relative to practice change intentions (χ 2 (15) = 18.67, p = 0.229).

Table 2 Participant intention: “Based on completion, I plan to make the following changes to my practice”
Full size table

Intended Practice Changes

Discussion

While the course was developed and marketed toward primary care clinicians, a large number of non-primary care participants chose to participate. The practice disciplines of these participants included surgery and surgical subspecialties, mental health, critical care, emergency medicine, urology, cardiology, and “other, not specified.” This diversity of participants speaks to the prevalence of cancer and broad base of interest across many disciplines. There were significant gains in knowledge based on participation in the course. Further, most of these gains were moderate to large effect size when examined across specialties and test items, indicating practical significance for the cancer survivorship intervention. Pretest scores differed among the specialties, likely attributable to clinical exposure prior to the course. The observation that nurses were overrepresented in the sample was believed to reflect both the profile of Medscape CME/CE participants, as well as the national movement toward the provision of both primary and cancer survivorship care using a multidisciplinary team-based model.

Based on participant feedback, the intervention achieved the goal of promoting improvement in cancer survivorship care. All eligible participants answered the question about intent to implement practice changes, and the feedback was overwhelmingly positive. Recommendations from the IOM stressed the necessity for improved communication between survivors and providers, especially with reference to patient history, psychosocial impacts (financial, spiritual, etc.), and experimentation with alternative therapies. The results showed that the highest percentage of course participants (63.1 %) selected alternate communication strategies with patients and families as the primary change they intended to make upon completion of the course. Recognition of the importance of communication with patients and families by clinicians in this study supports a realization that survivors have unique psychosocial needs. Hudson et al. [21] recently studied 42 survivors who reported that their primary care provider was not engaged in their status as a survivor. Cheung et al. [7] found similar results in a study on 213 survivors. Evidence from 21 experimental studies showed that physician training in communication skills results in significant improvements in patient adherence [22] and enhanced cancer health outcomes [23].

Despite evidence highlighting patient-reported needs for improved communication [24, 25], training programs to date have focused predominantly on enhancing the quality of patient communication with providers [26–28]. The current study demonstrates strong interest in improving communication skills among both primary care providers and procedural specialists, which is a novel and encouraging finding, and underscores the need to continue to develop, disseminate, and evaluate additional training programs that seek to improve provider communication with cancer survivors.

The Survivorship Care Plan (SCP) promoted by the IOM and others was envisioned as the appropriate vehicle for communication of patient treatment history and follow-up. In a recent study using survivor focus groups and enrolling patients with five cancer conditions, Mayer et al. [29] found that improvement in care coordination was not fulfilled using a SCP alone and highlighted the importance of including transfer of care communication, discussion of recurrence, and management of long-term effects. Further, the authors recommended that communication occur in both verbal and written formats, in an ongoing manner over time. It has been suggested that providers may view survivorship care communication in the context of specific time frames or phases of care [7]. We focused the course material on the ongoing aspects of the survivorship continuum with particular emphasis on psychosocial impacts, delivered by actual survivors in compelling videos. Perhaps these compelling vignettes may have changed some perceptions regarding the survivor experience.

The American Association of Medical Colleges (AAMC) [12] recommends the use of systematic approaches to instructional design as essential for effective use of educational technology. As this intervention has demonstrated, implementation of instructional design best practices for multimedia with highly focused content can result in higher than average participant feedback (98.9 %) regarding educational quality. When measured against outcomes using other approaches, Srinivasan et al. [16] reported learner preferences for case-based learning. This appears to be confirmed by this study as demonstrated by highly positive feedback from participants regarding relevance and quality.

Meta-analyses reported by Marinopoulos et al. [17] indicated the potential for internet-based CME to effect lasting changes leading to long-term practice behavior change. Casebeer et al. [30] established that both intent to change practice and knowledge increases, especially for case-based formats, were associated with reported changes at follow-up. A limitation of our study was the inability to conduct personal follow-up to determine actual practice changes due to participant privacy.

Assessment of sustained practice changes using patient chart reviews or clinical observations after exposure to distance learning is rarely feasible. Coordination of survivorship care requires intention on the part of the provider community to provide these services. According to Cohen [20], while knowledge can be assessed using pre- and posttesting, performance must be evaluated subjectively through self-reported intention to change and/or objectively through observation. The propensity to change clinical practice to affect health outcomes has been studied based on various measures of behavioral intention; studies confirm that intention is significantly associated with subsequent behavior change [31]. Accordingly, we suggest that research be done to further explore clinician intentions to change practice after completing online educational interventions in cancer survivorship.

Conclusion

Care for cancer survivors can be enhanced through increased involvement of primary care providers. Internet-based delivery is a viable solution for provision of IOM recommended cancer survivorship education and training which is broadly applicable across professional disciplines. This effort provided the opportunity to survey a broad-based, large sample of clinicians on a comprehensive multimedia educational intervention. Improvement in knowledge gain in four survivorship topic areas was significant, notwithstanding prior exposure and specialty orientation. Intention to improve communication across groups including the traditionally more procedurally inclined specialties is particularly noteworthy.

Scenario-based cases were used, in conjunction with other evidence-based learning resources, as best practices to support integration of course material presented in an online format. The online delivery environment has inherent advantages for global reach and ease of electronic data collection. Inclusion of educational interventions focused on communication skill improvement related to survivorship care is especially important given the current findings.

In order to further disseminate the curricular materials developed in this course, the authors are preparing a teaching guide to accompany each case and plan to submit in tutorial format to the MedEdPORTAL iCollaborative of AAMC [12]. This repository of curricular materials is available free of charge for educators to adapt and use at their own institutions; there is a need for contributions on the topic of cancer survivorship, as a search query performed on August 3, 2013, identified only one curricular offering that addresses breast cancer survivorship.