INTRODUCTION

To achieve the goal of a high-quality, learning healthcare system in the USA, many physicians and leaders endorse the use of quality improvement (QI) methods to assess changes in areas needing improvement.1, 2 Engaging primary care practices, described as system gatekeepers,3 is essential to successful QI efforts focused on improving population health, enhancing patient experiences, and reducing costs.1, 2 Slow progress in these efforts has prompted calls to identify new ideas for QI, with many suggesting organizations solicit creative ideas from clinicians and staff. Clinicians and staff observe the wide-ranging realities of care, which positions them to generate and implement QI ideas.3, 4 However, engaging them in QI idea-related processes is challenging given time pressures and workload.5, 6 Often, clinicians and staff are tasked with guiding these efforts, becoming “QI leaders.” As QI leaders, they are advised to use evidence-based tactics, or specific approaches to address a goal, to promote engagement in the generation and implementation of QI ideas.7,8,9

Little is known about the tactics that QI leaders can use to solicit creative (defined as novel and useful)10, 11 QI ideas from primary care clinicians and staff, and nurture such ideas so that they are implemented, providing an opportunity to improve outcomes.9 In contrast, research exists on leader behavior, or words and deeds that leaders display during interactions, and styles, or an overall pattern of leader behaviors.12 Leader behaviors such as supporting staffs’ actions12,13,14 are positively associated with generating creative ideas and idea implementation. Participative leader styles are positively associated with creative idea generation9, 15, 16 and more directive styles with implementation.9

For QI leaders seeking to engage primary care clinicians and staff in QI idea-related processes, tactics could be more actionable and effective than leader behaviors and styles due to tactics’ focused nature.9 QI leader training often focuses on tactics to test interventions, and less on tactics to elicit ideas that form intervention inputs.7 An example tactic to elicit ideas is brainstorming, or rapid generation of ideas.8, 17 While studies have examined the association of brainstorming and creativity,18, 19 no work has assessed the relative effectiveness of leader tactics on idea-related processes.8, 19 The study objective was to assess the relationship between leader tactics and the generation of creative ideas, implementation, and evolution of implemented ideas by QI teams, to inform QI leaders’ selection of tactics to achieve their goals.

METHODS

Study Design

The study was conducted in 12 federally qualified community health practices in one US center over 18 months, serving 130,000 patients annually. Teams at each practice (primary care providers, nurses, behavioral health providers, and medical assistants) were asked to develop QI initiatives. The center assigned leaders from its QI department (nurses or PCPs with a department affiliation) to lead each team. The department provided 25 training hours upon department affiliation, which comprised training from department experts and outside consultants on methods to support improvement, address challenging team dynamics, and facilitate effective meetings. Trainings occurred in person and via video-conference, and all leaders received training prior to the study period. During the study, teams developed and implemented initiatives focused on care coordination. Leaders were encouraged to elicit ideas from team members using tactics taught at the department (meeting ground rules, brainstorming time, and reflection on team process). Leaders were free to choose which tactics they used with their teams, if any, and were in contact with department experts for ongoing support. We analyzed 220 QI ideas generated by teams.

We used two approaches to study leader tactics. The first was qualitative data analysis to identify leader tactics (four identified), ideas generated by teams (N = 220 ideas), ideas’ creativity and implementation outcome, and evolution pathways for implemented ideas. The second was quantitative analysis of leader tactics and idea creativity, implementation, and evolution.

Data Sources

QI Team Meeting Minutes

We analyzed minutes from 216 QI team meetings (monthly meetings for 18 months at 12 practices). QI team creation, clinical team roles, care coordination focus, and meeting organization (monthly for 1 h, written minutes taken by a member, and meeting minutes format) were required by the organization. Minutes included practice name, date, attendees, agenda, discussion, action items, and staff responsible. Meeting minutes was the organization’s norm and analysis for study purposes was not planned at the time of meetings, eliminating the possibility that leader tactics were influenced by note-taking related to our study. We analyzed minutes using a two-step content analysis process.20, 21 We identified leader tactics used across the 12 practices. Three tactics were used across nine leaders. Three leaders did not employ any specific tactic. The first author, a PhD-trained researcher with quantitative and qualitative methods expertise, coded QI ideas and the individual(s) who proposed each idea. Ideas were defined as: “A different alternative for a possible course of action to approach the task at hand.”22 Two MPH-trained researchers independently coded ideas in 30% and 70% of the minutes.23 There was 83% and 94% intercoder reliability between the first and second coders and first and third coders, considered “acceptable” (70%) to “exceptional” (94%).24 Idea identification discrepancies (5%) were discussed until consensus was reached. Atlas.ti 8 was used to code.

Staff Work Experience Surveys

We collected survey data on staff characteristics (professional role, tenure, and gender) and practice characteristics (psychologically safe work climate). We recruited and consented participants using leadership emails and meetings with researchers. At the start of the study, we administered the survey via email or paper to all 72 team members in the 12 practices (100% participation). Participants were full-time employees and majority were female (79%). The largest group of respondents were nurses (31%), with PCPs (27%), medical assistants (23%), and behavioral health providers (19%), and 61% of staff were employed by the organization for more than 2 years.

Measures

Leader Tactic

The first leader tactic observed was meeting ground rules, or agreed-upon rules about meeting process, participation, and respecting others’ opinions. We coded the leader as using this tactic if there was a ground rules discussion in minutes during the first meeting. The second leader tactic was brainstorming time, in which leaders provided team time to generate ideas. We coded a leader as using this tactic when brainstorming time was noted in minutes (this occurred in 100% of their meetings). The third leader tactic was reflection on team process, which involved leaders dedicating time to reflect collectively on the process in the meeting. Teams discussed whether the team was working effectively, how team members were communicating, and how the team was making decisions. Teams assigned a score out of 10 to the meeting and discussed what they could do to improve. We coded a leader as using this tactic when minutes indicated these discussions (this occurred in 90% of their meetings). When no tactic was observed, we coded the leader tactic as “no leader tactic” (reference category). No other tactics were mentioned beyond those described. Table 1 displays the tactics. One tactic was identified for each QI leader.

Table 1 Four Leader Tactics to Solicit Creativity from Staff
Full size table

Idea Creativity

Ideas were rated for creativity by an expert panel using the validated Consensual Assessment Technique. Experts familiar with the domain independently rate an idea for degree of novelty and usefulness, and ratings are multiplied for a creativity score.25 Five healthcare executives with at least 10 years of experience in administration (including QI) and graduate healthcare administration degrees were recruited as experts (60% were female). Experts were not aware of other participants.25 Experts were emailed a survey containing vignette idea descriptions and were asked to rate each idea (N = 220) for novelty (1 = least novel to 5 = most novel) and usefulness (1 = least useful to 5 = most useful).25 Idea order was randomized.25 Novelty and usefulness scores were multiplied for a score from 5 to 25, then divided by 5 for a creativity score out of 5 (1 = least creative to 5 = most creative).25 We averaged the experts’ scores for one score per idea. An intra-class correlation (ICC) was calculated between ratings26 (ICC1,k = 0.79), indicating a high degree of consistency acceptable for creativity assessments.25

Idea Implementation Outcome

Ideas were tracked over the course of the meetings to assess implementation. Implementation was defined when an idea was successfully transformed into a policy, process, or procedure by the organization, which was recorded in the meetings.27 We noted whether each idea was implemented (coded as 1, otherwise as 0) and the time elapsed (number of meetings) since the idea was first mentioned. Ideas not discussed in successive meetings or ideas discussed and rejected in a meeting were coded as not implemented. Right censoring occurred for ideas with no definitive outcome by the study end.28

Evolution of Implemented Ideas

The evolution of ideas implemented over the 18 months followed three distinct pathways. “Plug and Play” describes ideas implemented over 2 months or less, and were associated with low engagement on idea content. We coded an idea as low-engagement if the team did not discuss idea content, only the logistics for idea implementation. If the team discussed idea content plus implementation logistics, we coded the idea as having high engagement. We did not identify any ideas implemented over two meetings or less that garnered high engagement. “Slow Burn” describes ideas implemented over three meetings or more and were associated with low engagement. “Iterate and Generate” describes ideas implemented in three meetings or more, and were associated with high engagement. Table 2 shows the three pathways for all 220 ideas. We created dummy variables for analysis of the three pathways, with ideas not in a pathway as reference group.

Table 2 Three Pathways to Describe Evolution of Implemented Ideas
Full size table

Covariates

We included gender, professional role, and organizational tenure as staff characteristics that might influence ideas. Percentage of patients who were uninsured and patient visits per full-time employee captured differences in patient profile and workload per practice. We included a practice-level measure of psychological safety, the shared belief that the setting is safe for interpersonal risk taking,29 consisting of three items, including “In this team, if you make a mistake, it is often held against you” which were averaged and aggregated. This was included because it is critical to whether team members share creative ideas and organizations engage in learning required to implement new ideas.30

Analyses

We calculated descriptive statistics and estimated multivariate multi-level regression models to assess associations between leader tactics, covariates, and dependent variables of idea creativity, implementation outcome, and evolution of implemented ideas. We used procedures in SAS 9.4 for each dependent variable. We clustered standard errors at practice and staff levels to account for multi-level data (ideas associated with staff, in practices)31, 32 and calculated variance inflation factors to assess multi-collinearity.

First, we assessed the association between leader tactics and idea creativity, using the GLIMMIX procedure to account for multi-level data and continuous outcome variable.31 We examined standardized coefficients and p values to evaluate variable significance and interpret relative importance of each tactic on idea creativity.31 We assessed the association between leader tactics and idea implementation outcome using the PHREG procedure,31, 32 and a multivariate Cox proportional hazards model, which incorporated time to implementation.28 To check the proportional hazards assumption, we conducted likelihood ratio tests for each variable; these tests indicated support for this assumption.32 We examined hazard ratios and confidence intervals to evaluate likelihood that an idea associated with a leader tactic would be implemented over time. To assess association between leader tactics and evolution pathway for implemented ideas, we used the GENMOD procedure to perform a mixed model analysis. The association for each pathway was assessed in separate models, with pathway treated as a binary-dependent variable.31, 32 We examined odds ratios and confidence intervals to evaluate likelihood that an idea associated with a leader tactic was associated with each idea evolution pathway.

RESULTS

Table 3 presents descriptive statistics. Average idea creativity was 2.47 out of 5 (SD = 1.64) and 58% of ideas were implemented (23% were Slow Burn, 21% were Plug and Play, and 14% were Iterate and Generate idea evolution).

Table 3 Correlations of Key Variables (N = 220 ideas)
Full size table

Table 4 shows results of the leader tactic and idea creativity analyses. Compared with no leader tactic, brainstorming was associated with a 0.19 standard deviation increase in idea creativity. Meeting ground rules and reflection on team process were not significantly associated with idea creativity. Professional role (PCPs and nurses), shorter organizational tenure, and greater psychological safety were associated with greater idea creativity.

Table 4 Results of Multi-level Regression Analysis of Leader Tactics on Idea Creativity (N = 220 Ideas)
Full size table

Table 5 shows results of leader tactic and idea implementation analyses. Compared with no leader tactic, brainstorming and reflection on team process significantly increased hazard ratio for idea implementation by a factor of 1.2, and 1.84, respectively, whereas meeting ground rules was not significantly associated with idea implementation. Longer organizational tenure, more visits per FTE, and greater psychological safety were associated with greater idea implementation.

Table 5 Results of Cox Proportional Hazards Model Relating Leader Tactics to Idea Implementation (N = 220 Ideas)
Full size table

Table 6 shows results of the leader tactics and evolution for implemented ideas analyses. Model 1 shows that compared with no leader tactic, brainstorming increased odds ratio of “Plug and Play” idea evolution by a factor of 1.24. More visits per FTE was also significant. Model 2 shows that compared with no leader tactic, brainstorming, meeting ground rules, and reflection on team process were not significantly associated with “Slow Burn” idea evolution. Model 3 shows that compared with no leader tactic, reflection on team process significantly increased odds ratio of the “Iterate and Generate” pathway by a factor of 1.73. Longer organizational tenure was also significant. Variance inflation factors for all models were below 6 and independent variables were below 2.50 (threshold is 10); multi-collinearity was not an issue.

Table 6 Results of Cox Proportional Hazards Models Relating Leader Tactics to Idea Evolution Pathways (N = 220 Ideas)
Full size table

DISCUSSION

Our findings suggest that leader tactics have different effects on idea creativity, implementation outcome, and evolution of implemented ideas. Brainstorming was positively associated with idea creativity and “Plug and Play” idea evolution, which suggests this tactic promotes more creative, quickly implemented, low-engagement ideas. The association with idea creativity is consistent with studies linking brainstorming with higher creativity outside healthcare,18, 33 and extends past research by showing this tactic is more effective than other tactics in promoting creativity. The directive to generate creative ideas rapidly may promote divergent thinking needed for creativity.18 The evolution and implementation outcome results indicate that more creative ideas can also be implemented quickly, with low-engagement on the idea’s content, but with a focus on the logistics of implementation. For brainstorming success, leaders need skill in facilitating discussion around a shared purpose and enabling free flows of ideas such as allowing members to talk uninterrupted without fear of judgment to protect high-creativity ideas and promote rapid implementation.34, 35

Reflection on team process was positively associated with idea implementation and “Iterate and Generate” idea evolution, characterized by slower idea implementation with high team engagement about idea content. The ideas generated were less creative, however. No prior empirical study tested the effect of this tactic on our dependent variables, although theories suggest groups that reflect on their process may emphasize their collaborative output,17 and less creative ideas sustain more discussion over time as they are less likely to be criticized.36 Our work extends this by providing evidence to support the theory of less creativity in reflective teams, in a single study, and in healthcare QI.

There was not a significant association between meeting ground rules and idea creativity, evolution, or implementation. It may be that ground rules were used in teams where status differences were not prominent and had little effect. Future work should examine meeting ground rules to examine whether their effectiveness is conditional on other characteristics such as status differences.

Although we found a relationship between leader tactics and idea processes, we cannot determine a causal relationship. It is possible that leader tactics are indicative of practice dynamics, which we addressed by controlling for practice characteristics. Although only one leader tactic was listed in the minutes per practice, leaders may have used tactics not recorded. Nevertheless, this study suggests leaders should select tactics depending on their priorities. For some organizational challenges, the creativity imperative is urgent: existing approaches are insufficient, and rapidly implemented, creative ideas may be necessary for improvement.12 Preventing disruption to the status quo and stability of groups in the organization may be less important than the need for change.37 Leaders might consider employing brainstorming for these circumstances.

When circumstances suggests the value of incremental changes and deliberate problem solving,38, 39 promoting commitment over longer periods of time for idea implementation, and fostering engagement, discussion and evaluation of ideas with other characteristics (e.g., external environment fit) may be more important.39, 40 Leaders might consider employing reflection on team process for these circumstances, which emphasizes group process over rapid change, which may be desirable for leaders facing challenges requiring deep analysis.41 Leaders must be skilled at encouraging teams to notice the process and behaviors of teamwork, such as communication and the value of shared ideas.34 This work expands our understanding of how leaders facilitate QI,12 and suggests leaders employ tactics appropriate to their organizational goals.

This work contributes to implementation science literature that underscores the importance of “bottom up” efforts in change processes as a complement to “top down” QI interventions.42, 43 Healthcare teams play a critical role in the implementation of top-down approaches, including the implementation of evidence-based interventions44 and public policy responses.42 Research has clarified how critical their ownership of implementation is, but has neglected examination of how healthcare teams generate QI ideas that form interventions to be implemented, despite this being viewed as critical to implementation success.43 Even less research has examined how clinicians and staff generate ideas, though there is often an organizational mandate to do so.42

Our findings have implications for QI and implementation. For QI, it implies that on-the-ground improvement, specifically the generation and implementation of QI ideas, may complement research-generated evidence implemented in a “top-down” fashion.43, 44 Composition of on-the-ground teams matter: team members with shorter tenure were associated with greater idea creativity and longer tenure was associated with idea implementation. Both processes are critical. For QI leaders, leader tactics examined suggest optimal methods to foster engagement in QI by clinicians and staff under different conditions. For implementation researchers, leader tactics are strategies that can spread beyond a single setting, and can promote behavior change within primary care that can integrate into routine practice. Our findings address core issues at the nexus of implementation science and QI that advances the progress of primary care improvement.