Introduction

As national efforts focus on “ending cancer as we know it,” [1] conducting implementation research to accelerate the widespread adoption of evidence-based interventions (EBIs) remains critical to reduce morbidity and mortality due to cancer. Two high priority areas where evidence exists for prevention are for human papillomavirus (HPV) vaccination and colorectal cancer (CRC) screening.

Laboratory and epidemiological research has provided evidence that an infection with HPV is a necessary cause for nearly 100% of cervical cancers in addition to a subset of non-cervical cancers such as anal, oropharyngeal, penile, vaginal, and vulvar cancers [2]. It has been over a decade since an effective vaccine was licensed for use in the U.S. as a primary prevention strategy. The 2014 President’s Cancer Panel report highlighted research identifying barriers to vaccine uptake such as the lack of a strong recommendation by a health care provider, lack of awareness by parents, concerns about safety, and missed opportunities to administer the vaccine [3].

CRC is the third most commonly diagnosed cancer and third leading cause of death in both men and women in the United States (U.S.) [4]. The United States Preventive Services Task Force has found that screening with stool-based, direct visualization, or serology tests is effective in reducing the mortality associated with CRC [5]. Despite this evidence, in 2012, nearly one-third of U.S. adults were found to have never been screened for CRC [6]. A recent review suggests that cultural beliefs, socioeconomic status, health literacy and language barriers influenced the individuals decision to participate in CRC screening [7]. To achieve the National Colorectal Cancer Roundtable’s goal of increasing CRC screening to 80% by 2018, there have been calls for national multilevel initiatives that include diverse stakeholders such as policymakers, community organizations, and health care systems [8].

Findings in research cannot change health outcomes unless they are adopted among practitioners, settings and systems. The scientific study of methods to promote the systematic uptake of research findings into routine practice, in order to improve quality and effectiveness of health services is known as implementation research [9]. Implementation research is critically needed to accelerate the wider adoption of evidence-based implementation strategies for improving HPV vaccination rates and increasing CRC screening. Within this broad field, the recent focus has been on implementation strategies, which comprise the “how to” when changing health care practice. Implementation strategies have been conceptualized as the specific means or methods for adopting and sustaining EBIs [10].

The Comprehensive Cancer Control National Partnership (CCCNP) is a group of 19 organizations that are dedicated to supporting comprehensive cancer control (CCC) coalitions in the U.S. states, tribes, territories, and U.S. affiliated Pacific Island Jurisdictions [11]. With funding from the Centers for Disease Control and Prevention’s (CDC) National Comprehensive Cancer Control Program, all 50 states, the District of Columbia, seven tribes and tribal organizations, and seven territories and U.S. Pacific Island jurisdictions have produced cancer control plans [12]. The CCCNP provides support to CCC coalitions, and coordinates national CCC efforts. The CCCNP has set priority focus areas: (1) increasing HPV vaccination uptake among boys and girls aged 11 and 12 years old, and (2) increasing CRC screening to 80% by 2018 among adults 50 years of age and older. For HPV vaccination, the CCCNP HPV Vaccination Workgroup partnered with the CDC National Immunization Program and the American Cancer Society to identify and map out national partner HPV vaccine uptake activities. In May 2016, the CCCNP provided technical assistance to 11 states where national partner activities were limited and cervical cancer mortality and low vaccination rates were high. For CRC screening, the CCCNP CRC Screening Workgroup partnered with the National Cancer Institute (NCI) to convene 11 state teams with representation from the CCC coalitions and Federally Qualified Health Centers (FQHCs) to learn about EBIs and develop collaborative action plans in May and September 2016. State teams for the HPV and CRC workshops were selected through a competitive application process to attend the workshops.

State teams

For the HPV technical assistance, four individuals formed the state team which consisted of the CCC Program Director, the American Cancer Society (ACS) Health Systems Manager, the State Immunization Program Representative, and a HPV champion. A total of 11 teams were invited to participate in the technical assistance workshop for HPV, which included Arkansas, Colorado, Georgia, Florida, Illinios, South Carolina, Tennessee, South Dakota, Utah, Virginia, and West Virginia, for a total of 44 participants. For the CRC technical assistance, six individuals formed the state team and consisted of the CCC Program Director, American Cancer Society (ACS) Health Systems Manager, state Primary Care Association representative, FQHC representative, local health department representative, and the state Cancer Coalition Chairs. Eleven teams participated from Arizona, California, Florida, Georgia, Michigan, Mississippi, New York, Pennsylvania, South Dakota, Texas, and Vermont in the CRC workshop, for a total of 66 individuals. For the CRC workshops, the focus was on bringing the FQHCs to the table; for the HPV vaccination workshop, the goal was to bring together the immunization stakeholders along with the CCC coalitions.

Technical assistance components and delivery

For the purposes of this work, we considered “technical assistance” for HPV and CRC as a multicomponent strategy consisting of in-person sessions supported by subject matter experts, facilitated development of action plans by state team members, and follow-up support calls which included webinars with team members and partners that were involved in the implementation of the specific activities in their respective action plans. The in-person sessions and the facilitation action planning took place over a 2-day period in Atlanta, Georgia. After the in-person meeting (referred to as “workshops” hereafter), follow-up support included approximately 2–3 phone calls between team members and facilitators (that included trained professionals and subject matter experts) and webinars over a 6-month period. The technical assistance organizing and delivery team was led by the American Cancer Society and consisted of representatives from the individual organizations in the CCCNP and external consultants who also served as facilitators throughout the process.

For the HPV vaccination workshop, educational sessions focused on developing policy and system change; working with large health care systems; developing action plans specific to each state; and evaluating HPV vaccination policy and system changes. This workshop also included facilitated roundtables on advocating for policy changes, making system changes, working with physicians and providers, increasing vaccination among males, countering anti-vaccine press and reframing vaccine messages, and establishing linkages to immunization programs.

For the CRC workshop, the educational sessions were designed around the specific steps proposed in the toolkit developed by the NCCRT titled, “Steps for Increasing Colorectal Cancer Screening Rates: A Manual for Community Health Centers” [13]. Specific topics included implementing a quality screening navigation program, developing and sustaining partnerships, effectively using electronic health records, and increasing screening rates among Native American populations. Facilitated team discussions were carried out on planning and implementing strategies to increase CRC screening, screening support systems, moving patients to screening, and developing formal action plans specific to each state.

Technical assistance has been defined as an individualized, hands-on approach to building an entity’s capacity for the quality implementation of innovations, usually following a training [14]. Technical assistance can be delivered in different ways, including one-on-one consultations, small group facilitation, or through web-based programs [3]. Often times, technical assistance includes facilitation in some capacity, which has been considered as a “deliberate and valued process of interactive problem solving and support that occurs in the context of a recognized need for improvement and a supportive interpersonal relationship” [15]. A review has identified nine technical assistance facilitator roles—opinion leaders, coaches, champions, research facilitators, clinical/practice facilitators, outreach facilitators, linking agents, knowledge brokers, and external-internal facilitators. In addition, they also identified characteristics of facilitation and the process of facilitation [16].

Previous research about the impact of technical assistance and the application of facilitation and action planning has been limited. As national efforts continue to invest in providing technical assistance, there is a current gap in understanding how technical assistance functions as a strategy for the implementation of EBIs for cancer prevention. This study aims to evaluate the impact of technical assistance (including action planning and support calls) on workshop participants’ knowledge, attitudes, and skills related to use of strategies to implement EBIs in their local context for increasing HPV vaccination and CRC screening rates. The key study questions for this evaluation project were as follows: (1) how does participation in technical assistance influence the knowledge, attitudes and skills towards using EBIs to improve HPV vaccination and CRC screening rates? (2) What is the influence of technical assistance on the use of EBIs on action plans and the state cancer coalition activities? (3) How do state teams collaborate to implement EBIs to increase HPV vaccination and CRC screening rates after participating in the technical assistance workshops?

Methods

Logic model and study design

To conceptualize the impact of centralized technical assistance, the research team developed a logic model (Fig. 1). The research team hypothesized that participating in technical assistance would lead to increased knowledge of evidence-based cancer prevention and control interventions, improved attitude towards evidence base course content, improved attitudes towards use of EBIs, enhanced capacity and skill to plan and implement EBIs, increased perception of benefits on use of course content and decreased perception of barriers to use of course content. The research team also hypothesized that the level of collaborations within state coalition teams, increased use of EBIs in respondent’s agency, and increased uptake of EBIs for improving HPV vaccination rates and CRC screening rates. A post-test study design was chosen for this project based on feasibility using a quantitative approach.

Fig. 1
figure 1

Logic Model of the CCCNP-led CRC and HPV trainings

Full size image

Questionnaire development and data collection

Using the logic model, we created two separate questionnaires for participants in the HPV and CRC-related technical assistance cohorts. We asked respondents from both workshops, a set of common questions in addition to a set of questions that were specific to the content areas of the respective workshops. The common set of questions were about participants’ knowledge, attitudes, and skills around the use of EBIs; state coalition activities and team functioning; and demographics. The specific questions for participants in the HPV workshop incorporated the focus areas highlighted in the meeting agenda such as policies at the local and state levels, improving provider motivations, and working with immunization information systems. For the CRC participants, the specific questions were designed around monitoring and evaluating CRC screening interventions and learning new skills to develop and sustain partnerships.

The HPV questionnaire was composed of 64 items and the CRC workshop questionnaire had a total of 58 items (see Online Appendix A and B). Items had response options on a 4-point Likert-scale (5 = Strongly Agree, 1 = Strongly Disagree) and were designed to be completed by participants in approximately 15 min. The CRC workshop took place in September 2015 and the participants received the survey approximately 23 months (August 2017) after the workshop. The HPV workshop took place in May 2016 and the participants were invited to participate in the survey approximately 18 months (November 2017) after the workshop. A link to the online questionnaire was distributed via e-mail to all CRC and HPV workshop participants [17]. All non-respondents were sent e-mail reminders weekly for 4 weeks, with an extra reminder sent out the day before the questionnaire was closed. Despite our efforts, we still had a low response rate from CRC workshop participants, so we enhanced our follow-up methods for all HPV workshop participants. For participants in the HPV workshop, after 4 weeks of e-mail reminders, non-respondents were also called by a research assistant and asked to complete the questionnaire over the phone. This extra step was taken to increase the response rate to the survey (initial response rate of 27.2%, increased to 38.6% after the phone calls).

Data analysis

Basic descriptive statistics were conducted to determine the impact of technical assistance training on the following categories: participant knowledge, attitudes, and skills regarding the use of EBIs, influence of the workshop on state cancer coalition activities, and collaborations among state team members. Respondents were not required to answer every question of the survey and missing data were excluded from the analysis. In addition, because of the high numbers of agree and strongly agree, categories were dichotomized into disagree and agree.

Content analysis of action plans

To provide a comprehensive understanding of the impact of technical assistance, we also conducted a content analysis of the action plans that were produced by each of the state teams at the workshops. Content analysis is a research technique used for the objective and systematic description of the text in documents, in order to summarize the details concerning a set of documents [18]. Two coders reviewed the action plans independently and then met to achieve consensus on the main activities proposed on the action. Analyses were conducted separately for the action plans developed by HPV workshop participants and the CRC workshop participants.

Results

Technical assistance as an implementation strategy

Technical assistance in this context was a multicomponent strategy consisting of three distinct but related components: (1) educational sessions at the in-person workshop led by subject matter experts, (2) facilitated action planning sessions, and (3) follow-up calls and webinars to provide technical support for state teams as they implemented the specific activities of their respective action plans. Based on Proctor’s guidance on specifying the implementation strategies [10], Table 1 provides the details regarding each component of technical assistance.

Table 1 Components of technical assistance
Full size table

Participant characteristics

Of the 66 attendees for the CRC workshop, 22 (33.3%) responded to the survey and 17 (38.6%) of the 44 attendees for the HPV workshop responded to the survey. Team members from Alaska, Florida, Tennessee, South Dakota, Virginia, and West Virginia who participated in the HPV workshop responded to the survey. Members from California, Florida, Georgia, Michigan, Mississippi, Pennsylvania, South Dakota, Texas, and Vermont who participated in the CRC in-person meeting responded to the survey. Table 2 provides the details on the participant characteristics. Most respondents were state health department representatives (41%) for the HPV workshop and FQHC representatives (31%) for the CRC workshop.

Table 2 Characteristics of HPV and CRC workshop participants that responded to the survey
Full size table

Knowledge, attitudes, and skills developed after the workshop

As shown in Table 3, a total of 20 items were included in both the HPV and CRC questionnaires. Overall, respondents improved knowledge, attitudes, and skills about EBIs, implementation strategies, using local data, and resources and partnerships needed for implementing HPV and CRC-related EBIs. Respondents also noted knowledge and motivation to communicate and coordinate with partners across the state for implementing EBIs for improving HPV vaccination and CRC screening rates. Respondents from both groups (90% total respondents) agreed that it was important to use the action plans developed during the workshop.

Table 3 HPV and CRC workshop knowledge, attitudes, and skills
Full size table

For the participants in the HPV workshops, 18 additional items (see Online Appendix A and B) were included about the content areas of the HPV workshops. Participants agreed that the workshops improved knowledge about HPV vaccination-related policies at the local and state level (92.8%); improved provider’s motivations to recommend HPV vaccination (100%); enhanced attitudes to implementing immunization information systems (84.6%) and communication campaigns to increase public awareness of HPV vaccinations (80%); and built confidence in abilities to frame pro-HPV vaccine communication messages (92.9%).

For participants in the CRC workshops, seven additional items were included to assess knowledge, attitudes, and skills regarding CRC EBIs. After the training, participants agreed the workshop increased knowledge of designing EBIs (88.9%); monitoring and evaluating CRC programs (77.8%); developed more positive attitudes towards enhancing existing partnerships (88.2%); and increased confidence to use local data (88.2%), develop new partnerships (88.2%), and communicate with existing partnerships (83.3%).

Influence on state cancer coalition activities

As shown in Table 4, a total of eight items asked respondents about the influence of their participation on the state cancer coalition activities. 86% (86.2%) agreed that workshops helped the state cancer coalitions to implement EBIs to increase HPV vaccination and CRC screening.

Table 4 Extent to which HPV and CRC participants improved state cancer coalition activities after workshops
Full size table

Collaboration among state team members

The survey also assessed the communication and coordination among the state team members because of the workshops. A majority of both HPV (79%) and CRC (71%) respondents knew and had worked with at least one team member prior to participating in the workshops. All of the HPV workshop participants reported communicating with other members of the team; email (38%) and phone calls (38%) were the most common means of communication. Communication occurred on average every month (79%). Among the CRC workshop participants, only 2% of the respondents did not communicate with other members of the team. Among those that did communicate, email (40%) and phone calls (36%) were the most common means of communication and communications happened either every month (53%) or every 6 months (29%). About 50% of the HPV workshop respondents and 47% CRC workshop respondents had communicated with teams from other states since the workshop.

In terms of the specific activities that the team members engaged in, respondents reported using EBIs (16%), using implementation strategies (14–15%), monitoring and evaluating EBIs (12%), sharing resources (16%), and completing tasks on the action plan (16%). Specific to the workshops, approximately 13% of the HPV workshop respondents focused their efforts of developing and sustaining partnerships, while 14% of the CRC workshop participants used the STEPS manual [13].

Activities of the action plans

The action plans produced by the 11 state teams participating in the HPV workshop commonly prioritized targeting provider-level education and training (73%), conducting an environmental scan (27%), and strengthening partnerships to coordinate local efforts (27%). To achieve these priorities, tasks on the action plans included engaging key stakeholders (64%), developing provider-level training and education materials (55%), and identifying existing HPV vaccine activities within the state (45%). Although action plans assigned tasks to specific individuals, the most common resource need was additional staff needs (45%) and funding (36%). Through these specific action steps, state teams expected to increase HPV vaccination rates (55%), improve collaboration across state-level partners (36%), increase provider motivations to make strong recommendations for the vaccine (36%), and generate an environmental scan (27%).

For the 11 state teams that participated in the CRC workshop, prioritized increasing CRC screening rates by implementing FLU-FIT programs (36%), EBIs involving patient navigation and/or community health workers (36%), and EBIs for provider education (27%). Specific action steps included identifying partnerships (73%), evaluating or developing communication plans for the team needs (45%), and conducting trainings for clinical settings such as FQHCs (36%).

Discussion

Overall, our findings indicate that centralized technical assistance may be associated with changes in knowledge, attitudes, and skills of participants for implementing EBIs. As a result of technical assistance, respondents reported increased knowledge about EBIs (90%), importance of using EBIs (90.3%), and skills to implement EBIs (90%). Over 93% reported using resources from the CCCNP after the workshop. A 2010 survey of CCC partners’ on the use of and attitudes about evidence-based practices found similar results when considering the CDC-related technical assistance and more than 90% reported implementing EBIs or implementation strategies, which was similar to this study [18]. A similar survey was conducted with the CCC program directors that identified technical assistance needs in terms of adapting EBIs for cultural appropriateness, assessing current resources, and implementing an EBI [19, 20]. This study is one of the first to evaluate the impact of technical assistance strategies on implementation of EBIs. This study also is the first to generate evidence to support the use of technical assistance as an implementation strategy to promote the adoption and implementation of EBIs for comprehensive cancer control.

Most participants from both workshops agreed that the workshops had a strong influence on their cancer coalition’s activities and reported increased collaborations to either implementing EBIs or implementation strategies. A 2010 content analysis of state CCC plans reported a limited use of EBIs in the context of CRC screening [21]. In contrast, the action plans developed as a result of the 2015 CRC workshop included EBIs such as FLU-FIT program and patient navigation interventions in partnership with key stakeholders such as the FQHCs. Content analysis of action plans developed at the 2016 HPV workshop revealed the use of community education or provider education alone, which were not recommended by The Community Guide (http://www.thecommunityguide.org) [22]. The 2010 study also revealed the need for multicomponent strategies, and engagement with key stakeholders to successfully implement and sustain EBIs. After the 2016 HPV workshop, many team members reported increased knowledge, attitudes, and skills to engage and strengthen existing partnerships, including with the immunization programs. This was also reflected in the action plans.

Communicating and coordinating activities across the state level partners to reduce duplication of efforts and maximize efficiency was a key goal of the in-person workshops. Consequently, much of the in-person workshop time was spent in discussion about the existing efforts, identifying individual organizational capacities, and together coming up with action steps for future collaborative work. After the workshop, most team members communicated regularly via phone calls every month. This highlights explicitly the goals for CCC which is the use of a “systematic approach for maximizing resources, reducing duplication of efforts and focusing on EBIs across the continuum of cancer control”[23]. To achieve this goal, state experiences have suggested creating alliances and leveraging the involvement of key stakeholders as CCC implementation building blocks [24]. Each of these objectives have been clearly addressed with the use of technical assistance as an implementation strategy to promote the uptake of EBIs.

A review of evidence about technical assistance to enhance prevention capacity suggested that specific tasks and content varied widely across studies [25]. As described earlier, centralized technical assistance in this context has been a multicomponent strategy consisting of educational sessions, facilitation and the development of action plans. In addition, the review also mentioned the lack of an explicit model or an organizing framework to guide the planning, implementation and/or evaluation of the technical assistance. The logic model and the data presented in this paper, along with frameworks such as the one proposed by Leeman and colleagues [26], can help guide future efforts to deliver technical assistance. Like technical assistance, there is limited literature to suggest facilitation as a well-planned, supportive, and proactive role to enhance implementation of EBIs and further research is needed to explore the role of external facilitation as described in this study [27]. Although there is limited research in this context, a recent review of the comprehensive cancer plans suggests that there were limited details provided and several key component areas such as implementation processes were not represented in the action plans [28]. During the content analysis, this study also found limited details in the action plans developed.

Further research is required to better understand the specific strategies and/or combination of strategies utilized by state cancer control coalitions in their local context and operationalizing them which will help understand their impact in cancer prevention. There is ongoing work to explore the use of implementation strategies in the local context using a qualitative approach. A limitation of this study was the lack of baseline data, since the participants were queried after the workshop. There was an 18–23 months gap between when the workshops took place and the survey, which may lead to recall bias. To mitigate the impact of potential recall bias, the survey included a description of the workshop, referenced specific material presented in the workshop, and definition prompts for any items that might have been forgotten over time. We believe the gap between workshop delivery and survey administration might have also led to relatively low response rates. The study team attempted to increase participation by sending out multiple follow-up reminders and through phone calls. Finally, the study used self-reported measures with demand characteristics that might have elicited more positive responses, causing ceiling effects (highest possible score is always scored) across multiple items in both the HPV and CRC surveys. The study team attempted to ensure that the instrument accurately captured the respondents’ true perspective by using a third party, (i.e., non-funder, non-NCI administration or staff, non-workshop facilitator) as the administrator of the survey.

In conclusion, study findings provide evidence for the use of centralized technical assistance as a way to change individuals’ knowledge, attitudes, and skills regarding the implementation of EBIs for increasing rates of HPV vaccination and CRC screening. Future efforts can be improved by collecting baseline data, incorporating a guiding framework to inform the content of the technical assistance, describing the specific tasks in the facilitated support delivered through the process of implementation, and encouraging detailed action plans from state teams participating in the technical assistance.