Introduction

Quality improvement (QI) efforts have been proven to enhance quality and patient safety in the pediatric setting [1, 2]. The increased adoption of standardized clinical pathways and improvement initiatives leads to better outcomes for patients, including reduction in length of stay and cost [3, 4]. They have also been shown to improve clinical outcomes for common pediatric illnesses like asthma, bronchiolitis, fever, sepsis, spinal fusion, and ICU care [5, 6, 7, 8, 9].

Despite this evidence of improvement, there remain significant deficits in quality and patient safety in children’s hospitals [10, 11, 12]. Improvement work is difficult and requires a substantial investment in resources, including budgetary support, physician, other clinician and support staff time, data collection, and time away from clinical work. Training and engaging co-workers and colleagues in the process of quality improvement are time-consuming and resource-intensive and requires a sustained focus on the initiative over time [13•]. In addition, significant investment is required in data infrastructure and analytics to successfully measure and inform improvement efforts.

Non-healthcare industries have harnessed the power of data for decades but the ability to capture and manipulate electronic health record (EHR) data is a newer phenomenon [14, 15]. Data privacy, lack of documentation standardization, and surplus of unstructured medical notes make harnessing data for quality improvement difficult. There is limited literature demonstrating integration of data throughout the entire lifespan of a QI project, which includes proactively using data visualization to evaluate and improve operations instead of simply conducting a retrospective evaluation.

Across our institution and others, there has been growing interest in quality improvement from physicians, nursing, and other staff [16]. Incentives like maintenance of certification (MOC) credit, opportunities for publication, and personal motivation to make improvements at the local level in the inpatient and outpatient settings have made participation in QI activities appealing to clinical staff. A national focus on value-based care and changes in reimbursement structures, including the rise of consumer-directed healthcare plans, require hospitals to respond to these challenges quickly and with flexibility. However, the growing amount of data resources available to hospitals and an increased capability to incorporate data-driven decisions into clinical operations represent an opportunity to grow QI efforts.

The following describes how a large pediatric academic medical center has developed an infrastructure to build and transform the culture, scope, and value of quality improvement by leveraging a structured improvement methodology paired with strong data capability and trained human resources to successfully improve quality across the institution.

Structure: foundational needs

Improvement framework

Quality improvement in healthcare, championed by pioneers including Donabedian, Deming, and Berwick, is a more recent development compared to other industries such as aviation and automobile manufacturing [17••]. Standardized quality improvement methodologies developed in those industries include:

  • Total Quality Management (TQM): Popularized by the Navy in the 1980s and 1990s, TQM emphasized quality defined as meeting customer requirements. This approach used statistical process control and successive plan-do-check-act (PDCA) cycles to promote continuous improvement [18].

  • Lean: An evolution of the Toyota Production System, Lean is a manufacturing philosophy focused on improving productivity through the reduction of waste and “non-value added” steps or tasks in a process. Tools such as waste walks, spaghetti diagrams, and Kaizen events are hallmarks of this approach [17••].

  • Six Sigma: This approach focuses on reducing variability and improving quality through standardization and defect reduction. Statistical process control and continuous improvement are foundational elements of this methodology. Projects are conducted using a standard process that is an elaboration of the PDCA cycle called DMAIC (define, measure, analyze, improve, control). Six Sigma has a robust toolbox to aid teams in implementing their work. Such tools include process maps, driver diagrams, fishbone diagram, and Pareto analyses [17••].

Organizations such as the Institute of Healthcare Improvement (IHI) in the USA and the National Institute for Health and Care Excellence (NICE) in the UK have significantly advanced the impact and quality of improvement work in healthcare. The IHI developed what it calls the Model for Improvement [19]. The Model for Improvement is a three-stage process aimed specifically at healthcare efforts. The model asks three questions for teams to answer: (1) What are we trying to accomplish?, (2) How will we know a change is an improvement?, and (3) What changes can we make that will result in an improvement? Plan-do-study-act cycles are used to test those changes and achieve project aims.

All of these methodologies have advantages and disadvantages depending on the type of work to be done. However, at their base, all improvement methodologies are essentially the scientific method applied to everyday problems. It is less important which methodology is used so long as a methodology is used. Our hospital deliberately chose not to endorse any one approach, but rather to organize our work under a more generic framework that includes the essence of these models but encourages using tools from any of them as needed. The framework incorporates the classic cycle of process improvement with structured project management through each phase. Our framework emphasizes the method of initial data acquisition, cohort definition, data analysis, and continuous measurement over time through four separate phases: (1) define—what are we trying to accomplish?; (2) diagnose—what do we need to learn so we can narrow our focus?; (3) test and implement—what changes will we make that will result in an improvement?; and (4) sustain and spread—how will we ensure that changes are sustained and can they be spread to other areas? How this process is implemented will be reviewed in detail below.

Clinical data warehouse and data governance

In order to understand current performance and accurately assess improvements over time, access to sound clinical data is essential for any quality improvement program. Although small projects can achieve success through use of more labor-intensive data collection methods such as manual chart review, the implementation of a robust data acquisition, management, and storage infrastructure is essential for an enterprise-wide improvement program.

Data to support QI efforts at our hospital are retrieved primarily from our data warehouse. The data warehouse is a large database that stores data from across the enterprise, including clinical datasets from the electronic health record (EHR), financial and billing data, and legacy systems where the application is retired but the data are still available. Data is downloaded to the data warehouse daily, using a combination of key patient identifiers along with hospital admission dates, surgical dates, or other native source identifiers. Surrogate keys support multiple source systems in one environment and simplify the joining of tables within the data warehouse. At our institution, the data warehouse is viewed as the “one stop shop” to answer any questions about hospital operations. The data and daily load processes are maintained and updated by the Information Systems team and ad hoc consultation with the quality improvement team’s data analysts. Structured querying language (SQL) is used for data extraction from the CDW, and analysis and visualization are completed using tools such as Excel (Microsoft Excel, 2013), RStudio (RStudio, 2015), and QlikView (Radnor, PA). Data stewards are positioned across the enterprise to conduct data governance for the metrics and projects completed using data from the data warehouse, guiding interpretation within their domains and increasing awareness of already existing definitions.

Leadership support

The final piece of structure critical to the development of a powerful clinical quality improvement program is leadership support. This includes senior executive leadership as well as leadership from clinical departments and divisions. Developing an effective quality improvement program is expensive, requiring investment in the data and personnel infrastructure to ensure success. Leadership must be on board to provide not just the financial support but the accountability to ensure the resources are being used responsibly and the work is achieving the desired outcomes.

Personnel

All quality improvement projects depend on having the right personnel to implement the work. The following roles are essential for the work being done in our hospital.

Improvement advisors

Improvement advisors are the facilitators and project managers for the QI projects. They are experts in improvement methodology who are adept at helping teams appropriately scope projects, set achievable aims, and move work forward consistently. In our opinion, there is no one educational background that produces effective healthcare improvement advisors. Degree programs in industrial engineering and business administration address many of the tenets of improvement methodologies and project implementation. Framing that work within the context of a medical environment is a challenge. Understanding hospital hierarchies and clinical processes around patient is difficult. Those without hospital experience must be specifically trained on these factors as they impact improvement project implementation. Conversely, there are many clinicians and other hospital workers who understand the milieu but likely have no specific training in improvement. We have found that a team balanced between these groups—improvement experts who need training in healthcare and healthcare experts who need training in improvement methodologies—can be exceptionally effective. Over time, each group gains expertise in the original area of weakness while supporting the team in their area of strength. In addition to education and experience, we look for candidates who are intelligent, curious, show problem-solving ability and flexibility of thinking, and who have a professional maturity and presence that can effectively work with staff at all levels of the organization.

Data analysts

The data analyst is responsible for obtaining and visualizing data to support the clinical team’s understanding of processes and outcomes as they move through the improvement framework. The data analyst role pairs communication, curiosity, and problem-solving with technical skills. Our analysts must be able to interact directly with project teams to translate clinical concepts into discrete data elements, code, and analysis. Thus, a successful analyst has the ability to understand a problem, critically think about data, and communicate effectively, asking questions and guiding teams through data exploration and meaningful metric design. Similar to the improvement advisors, there is no one educational background that is required. Our data analysts primarily come from quantitative disciplines including public health/epidemiology, economics, statistics, and engineering. Because our analysts do not typically come from a clinical background, we look for analysts who possess the curiosity and independent motivation to learn the medical context of their work through interactions with clinical teams. Upon joining our team, analysts participate in a rigorous 4-week training program, during which they complete a sample project under the guidance of more experienced analysts. The program trains new hires in improvement methodology, expected interactions and common challenges with clinical teams, how to effectively partner with improvement advisors.

Clinical leadership

Clinical quality improvement project teams are always multidisciplinary and have clinical leads working with the improvement advisor and data analysts. At a minimum, these clinical leads should have interest and lend support to the projects. However, we believe it is a sound investment for hospitals to provide improvement education to clinicians who will be leading this work over time such that they can effectively influence their clinical colleagues in doing this work on an ongoing basis.

Prioritizing work: top-down versus bottom-up

Choosing where to deploy improvement resources is one of the most important decisions clinical QI leaders will make. In broad terms, work can be prioritized by leadership (“top-down” initiatives) or identified by the front-line staff who actually implement the work every day (“bottom-up” or grass roots initiatives). There are pros and cons for each approach.

Top-down

An obvious benefit of leadership-identified projects is that these initiatives will have the support and attention from hospital executives. These projects tend to have access to more resources, are publicized and discussed in important forums, and may be prioritized for other supports (e.g., IS builds) over other projects. Projects pushed from leadership to front-line staff can result in non-engagement and may fail in harnessing the creative energy of a dedicated clinician team. Thus, thoughtful messaging explaining the rationale of the project is critical, as is open two-way conversation. The grass roots approach can effectively engage clinician leaders, particularly physicians across the organization—which is critical to effect change.

Bottom-up or grass roots approaches

Bottom-up projects have the benefit of immediate engagement by the front-line staff as they are now having resources to solve problems that impact their daily work. Focusing on these problems provides direct impact on patient care. However, the problems front-line staff prioritize may be viewed as too focused and may not align with the overall operating plan to justify the assignment of expensive resources.

We believe that both types of work are necessary for healthcare systems to develop robust quality improvement programs. When our office was formed in 2014, there were already project teams focused on the top-down projects. We seized the opportunity to engage the front-line staff in improvement work through initiating a “request for proposals (RFP)” process that encouraged teams to apply for improvement and data support to solve clinical problems that directly affected patient care and their daily work. These smaller, more focused clinical opportunities for improvement, the “low hanging fruit,” helped to build capability and trust for this process throughout the organization. Project submissions needed to detail the information in Table 1 and were chosen based on overall merit of the application.

Table 1 Components of a clinical QI project proposal
Full size table

Over time, relationships, interest, and skills have grown and we are now able to engage front-line teams with leadership projects that spread across more divisions throughout our hospital and ambulatory network. The trust built in doing the smaller projects has reaped the benefit of an easier willingness to work with us on the top-down projects that signal the priorities of leadership. Further, individual divisions are financing dedicated time for embedded improvement advisor and data analyst (IA-DA) dyads to create local-level quality improvement units within each area; these IA-DA dyads are able to develop specific subject matter expertise, furthering their ability to use the principles of improvement science within their clinical area.

Project implementation

Clinical quality improvement projects should be implemented by multidisciplinary teams made up of the staff who have the fundamental knowledge of the problem to be addressed. Typically, teams have both a physician and a nursing lead. Our QI department assigns an IA and DA dyad to every project to pair appropriate data with the relevant phases of the framework. Prior to the development of this new design, our analytic model involved a data request process with limited interaction between the clinical subject matter expert (SME) and the data analyst. The context and clinical nuances were lost in translation and the resulting data output rarely provided insight into improvement opportunities for the proposed clinical problem. Conclusions were sometimes unreliable because the SME was not involved in the data validation. To address these issues, the IA-DA cross-functional dyad, along with clinical partners, integrates subject matter, improvement, and data expertise to help teams effectively conduct continuous quality improvement. Our approach embeds data into the problem definition, process change, and evaluation phases of the project. We are able to use focused data to quickly inform decision making, analyze the impact of tests of change, and iterate accordingly. Projects are then implemented using our improvement framework:

Define

During the define phase, the IA introduces the improvement framework to the clinical team and facilitates meetings to discuss and validate the problem the team is looking to address. Key stakeholders are identified, project governance is addressed, a project charter is created, a realistic timeline is generated, the overall project aim is finalized, and the patient cohort is defined. The IA and DA work with the clinical team to define the cohort of interest for the QI project through development of inclusion and exclusion criteria. The DA translates clinical concepts for cohort identification into data elements that can be extracted from the data warehouse. In addition to presenting descriptive statistics about this cohort, DAs provide patient lists for teams to review patient charts. These chart reviews build confidence in the data and help teams identify inclusion and exclusion criteria to develop an accurate patient cohort.

Diagnose

The diagnose phase requires extensive discussion with the clinical teams to analyze the current state of the improvement opportunity, pinpoint root causes of the problem, and determine an adequate measurement plan for the project. The IA guides the team in determining structure, process, and outcome metrics. They also lead the team in determining key drivers and utilize specific quality improvement tools (driver diagram, fishbone diagram, process/flow mapping, etc.) to facilitate discussion. As the team identifies drivers, the DA retrieves relevant data for process and outcome metrics for the defined patient cohort to reflect the opportunities for improvements. These metrics are analyzed for a baseline period, typically the 24 months prior to the project start, to understand the current state. The IA and DA collaborate to develop a data visualization tool that displays baseline performance and variation, as well as to further segment the data to identify opportunities for interventions. The goal is to produce a tool that makes the data actionable and steers the team to the most efficacious interventions to be tested.

Test and implement

The IA works with the team during the test and implement phase to conduct small tests of change to determine if the proposed solutions lead to improvement. Examples of tests of change include the creation of new order sets or improved documentation workflows in the EHR; iterative process changes for things like daily rounds, discharge planning, or service delivery; development of a clinical pathway; or changes to the mode and content of clinical communications. Evaluation and monitoring of progress through measurement are crucial during this phase. Ideally, the DA automates a measurement tool, providing daily updates of metric graphs and patient lists to the project team on an internal web page. When this is not possible, the IA and DA collaborate to obtain information and feed it back to the team so they can make a decision as to whether to continue the current trial, change it, or abandon it all together. Teams review the data throughout the project to see if interventions have been successful and review charts of individual patients to better understand causes of variation. Tracking rapidly available, high-quality data as part of the real-time evaluation process allows teams to quickly analyze the impact of tests of change and inform new ones.

Sustain

During the sustain phase, the IA leads the team in developing strategies to maintain improvements over time. Again, measurement is emphasized and the IA encourages teams to operationalize any successful changes permanently into their workflow. Teams are also encouraged to share their improvement stories with others in the hopes of spreading the effective change to other areas. For sustained measurement, the DA creates statistical process control (SPC) charts and educates the team around special cause variation versus random variation [20], ensuring that clinical teams react to statistically significant shifts in the data. The automated measurement tool continues to update with data even after formal project support has ended.

Outcomes

Using a standard approach to implement projects and provide the right staff and information to engage clinical teams in the work leads to remarkable improvements in the care clinicians provide every day. As physicians are not compensated for this work currently, recognition for this work in the promotions process is critical. Processes to use this work for maintenance of certification (MOC) requirements provide another incentive. Currently, our hospital has an extensive project portfolio that supports MOC. Additionally, well-done quality improvement projects are now recognized as important scholarly works in well-respected journals with the goal of bridging the gap between clinical practice and published evidence. Thus far in 2017 alone, project teams at our institution have published papers in Pediatrics [21, 22], Annals of Emergency Medicine [23], Child Abuse & Neglect [24], Shock [25], and Pediatric Emergency Care [26].

Conclusion

Successful improvement efforts require a strong combination of clinical leadership, trained improvement staff, a developed and flexible data infrastructure, and an institutional improvement framework. Improvement efforts at our hospital have evolved with the development of a new approach to problem-solving, relying heavily on the ability to use real-time, visualized data and improvement metrics that are validated, meaningful, and accessible to their clinical owners.

Hospitals that seek to better incorporate data into their improvement methodology must invest resources in data acquisition, visualization, and analyst expertise and should couple this investment with quality improvement trained staff who can translate clinical data into metrics to inform and evaluate actionable tests of change.