Keywords

FormalPara Tools

Cause and effect matrix, SBAR, DMAIC, House Staff Patient Safety and Quality Council (HPSQC), “wow”factor, patient effect, outcomes, feasibility, multidisciplinary nature

Project Motivation (SBAR)

Situation

What

  • Compliance for the “global immunization” core measure for influenza and pneumococcal pneumonia was significantly below the institutional goals of >96 % compliance in 2012.

  • A comprehensive order set was created by the institution; however, compliance rates continued to be well below the goal of greater than 96 % [1].

How

  • Compliance rates were collected, compiled, and recorded by the central hospital based on core measure methodology.

Background

What

  • Global immunization for influenza and pneumococcal pneumonia became a core measure in January 2012.

  • The institution chose to adhere strictly to CDC guidelines for these vaccinations in order to avoid over-vaccination [2].

    • While guidelines for administration of influenza vaccination were relatively straightforward, guidelines for administration of pneumococcal vaccination were more complex and were often found to be a source of inappropriate ordering practices.

  • The Housestaff Patient Safety and Quality Council (HPSQC) was created in the summer of 2012 with a vision to create house staff-driven improvement of patient care and involve house staff in the existing quality improvement structure.

  • In its inaugural year, the HPSQC wanted to choose a QI project in order to meet its goals. A cause and effect matrix (Fig. 1) was used to determine which project would have the greatest impact and where we should direct our efforts.

    Fig. 1
    figure 1

    Cause and effect matrix

    Full size image

    • Effects that we were seeking included safety, outcomes, “wow” factor, feasibility, patient effect, and multidisciplinary nature.

      • Each effect was given a rank of importance (listed as “customer rank” on the example matrix below) from one to ten.

    • Each cause, or potential project in this case, was given a value that related to how much of an effect it would have for the respective category.

      • For example, the vaccination project was considered to be very feasible and was given a score of eight in that cell.

    • The “customer rank” of the effect was then multiplied by the value assigned to the effect. These values were added to give the project a rating and the highest rating project was given the highest rank.

  • The HPSQC chose improving compliance with “global immunizations” as an interdisciplinary quality improvement project based on the results of the cause and effect matrix exercise.

  • When the Council initiated its project by discussing vaccination screening with members of the hospital’s QI team, the following was revealed:

    • HPSQC members did not know why the existing vaccination order set had been put in place

    • HPSQC members were not aware of the core measure for immunization or the hospital’s compliance rate on this measure.

    • HPSQC members were not familiar with the specifics of the CDC vaccination guidelines

    • While a concurrent review process existed, when this was discussed with the HPSQC it was clear that the concurrent reviewers did not have a clear idea of whom to contact to order vaccination screening that had not been ordered correctly per the existing order set.

How

  • Compliance rates were already collected, compiled, and recorded by the hospital based on core measure methodology, and this practice continued through the implementation of the project.

  • The HPSQC met monthly. The HPSQC leadership had an additional monthly meeting in order to direct the goals of the project. The HPSQC leadership also met monthly with the institutional quality improvement vaccination team.

Assessment

  • Compliance rates were below goal because the purpose of the order set was poorly understood, there was little knowledge about the “global immunization core measures” by house staff who ordered the majority of these vaccinations, and communication regarding immunization failures was poorly understood.

Recommendations

  • Implementation of an education slide set created by house staff about the core measure and distributed by HPSQC members to their respective departmental house staff colleagues

  • Creation of a competition between departments for most compliant and most improved department

  • Partnering with the quality improvement staff in order to optimize the concurrent review process and communication plan for inadequate screening for vaccination

Project Implementation: DMAIC

Define

  • Problem: Compliance with “global immunizations” for influenza and pneumococcal pneumonia.

  • Goal: Improvement of compliance to >96 %.

  • Benefit: Decrease the potential for future morbidity of inpatients.

  • Scope: Inpatients of the Johns Hopkins Hospital.

Measure

What

  • Compliance rates for pneumococcal pneumonia vaccinations and influenza vaccinations by department.

  • Failure to order the vaccination correctly was also tracked and reviewed by the concurrent review team in conjunction with leadership of the HPSQC.

How

  • CMS vaccination core measure methodology [3]:

    • 104 medical record numbers of admitted patients were randomly selected and reviewed by the QI staff to determine compliance.

    • From those 104 medical record numbers, patients who were excluded from receiving the vaccination were removed.

    • Medical records of all patients meeting inclusion criteria were reviewed to determine if vaccinations for influenza and pneumococcal pneumonia were appropriately ordered and administered or for the presence of documentation that appropriately justified why the vaccination was not given (e.g., patient refusal).

    • Data were aggregated to determine monthly compliance.

Analyze

  • Compliance rates were reviewed monthly by the HPSQC.

  • Failures were reviewed monthly with the quality improvement team and the HPSQC leadership to determine if further changes needed to be made, such as further improvements to the communications plan or adjustments to the previously existing order set.

Results

See Fig. 2.

Fig. 2
figure 2

Percent compliance based on monthly audits

Full size image

For in-depth discussion and review of results from this project, please see the previously published article below:

Peterson S, Taylor R, Sawyer M, et al. The power of involving house staff in quality improvement. An interdisciplinary house staff- driven vaccination initiative. Am J Med Qual, first published online ahead of print May 9, 2014. doi:10.1177/1062860614532682

Improve

  • Based on reviewed failures with the quality improvement vaccination team, subsequent changes were made to the communication plan including escalation of communication from resident to senior resident to fellow to attending for vaccinations that were ordered incorrectly.

  • Based on reviewed failures, minor changes were made to the order set as it was noted that the most commonly missed comorbidity requiring pneumococcal vaccination was asthma and smoking. The order of the comorbidities was changed in the order set to better highlight these common comorbidities.

Control

  • Compliance rates continue to be reviewed by the HPSQC leadership.

  • The HPSQC leadership attended the quality improvement team vaccination meetings for the year following implementation. They continue to be involved at times that are concerning for a potential drop in compliance such as the beginning of the academic year when new house staff enter the system and in the fall when influenza compliance rates begin to be tracked.

Challenges

  • Scope involved every inpatient department. There are a small number of departments such as oncology that have a greater fellowship involvement in addition to strong feelings about vaccination management in their patient population that had to be delicately handled.

  • While there were representative members from every department on the HPSQC, not every department member was present at every meeting, requiring regular email communications regarding project implementation and progress.

  • There was a dependence on members of the HPSQC to distribute information to their respective departments. If there was concern that we were not getting responses from a HPSQC member, program directors were carbon copied for critical messages such as the education slide set, to ensure that the information was communicated to the involved residency program.

  • Significant and regular participation by the HPSQC leadership in both the regular HPSQC meetings and in meetings with the institutional QI team were necessary.

Successes

  • This project aided the institution in reaching goal compliance rates of >96 % while adhering to CDC guidelines.

Attending Comments

  • An important contributor to the success of this project was the selection of a problem that aligned with the hospital’s priorities for directing resources to produce improvement.

  • Additionally, the tenacity and interpersonal skills of the HPSQC chair, who directed the house staff effort on this project, was critical for its success.