Keywords

Assessing Quality

Quality improvement begins with accurate quality assessment. The concept of assessing medical quality goes back as far as Florence Nightingale in the 1850s when she proposed evaluating outcomes of war casualties in the Crimean War [1]. In 1913 Earnest Codman a surgeon at the Massachusetts General Hospital recognized the importance of tracking outcomes of surgical procedures and introduced the “End Result Idea” [2]. Unfortunately his colleagues not only did not embrace his ideas but actually got him fired and run out of town. Few medical providers followed the concept of regularly measuring outcomes. Whether this disinterest in monitoring results was because variability in outcomes was not suspected or because some providers feared exposure of poor outcomes is not clear. Nevertheless, it was not until the mid-1960s that Avedis Donabedian, a physician at the University of Michigan, rekindled interest in medical outcomes. Donabedian introduced the field of health systems research and recommended that health care quality be evaluated in terms of structure, process, and outcomes.

Assessing Quality in Cardiac Surgery

Almost simultaneously with the development of health systems research, the field of cardiac surgery arose. In the early days of heart surgery, procedures were relatively few and mortality as high as 100 %, consistent with the often desperate conditions of those patients. Good outcomes were hoped for but uniformly unexpected. The emphasis at the time was more on achieving survival than on measuring quality. The field of cardiac surgery did mature, however, with improving results in adults with valve and coronary artery disease and in children with uncomplicated congenital heart defects. As the practice of cardiac surgery improved, the number of procedures increased significantly. Coronary artery bypass grafting became one of the most commonly performed surgical procedures in the world. As heart surgery became commonly accepted practice it was recognized that a reliable measure of quality was the ability of the patient to survive the operation. Whether a patient lives or dies after an operation is clearly definable with limited subjectivity [3, 4]. This measure can be compared to the same measure at other institutions or to an overall benchmark. In the 1970s and 1980s cardiac surgery centers would typically monitor their own mortality rates to assure “acceptable” outcomes. Unfortunately those subjective outcome reports were rarely shared between centers and often only disclosed in the form of a publication when a center was proud of exceptionally good results. Such mortality rates were unlikely to be representative of the majority of centers performing heart surgery and individual centers were uncertain of what an “acceptable” mortality should be. In 1987 the Health Care Financing Administration published the mortality rates for Medicare recipients undergoing coronary artery bypass surgery. A striking variation in mortality was revealed and many heart surgery programs were embarrassed [5]. Cardiac surgeons began to recognize with this public release of sensitive information from individual centers the importance of keeping track of their results. The Society of Thoracic Surgeons began to develop an Adult Cardiac Surgery Database. The purpose was to invite centers to submit their own data confidentially in a way that each center could assess its own outcomes and compare them to the outcomes of the other submitting centers. This form of registry data more accurately represented outcomes across multiple centers and permitted the setting of realistic benchmarks and outcome targets.

Lessons Learned in Pediatric Cardiac Surgery

Although many cardiac surgeons in the late 1980s performed both adult and pediatric heart surgery, the emphasis in pediatric cardiac surgery at that time was more related to achieving survival in some of the smallest patients with complex life threatening heart defects than to the overall assessment of quality. Outcomes were highly variable and complex malformations almost uniformly had bad outcomes. The concept of continuous quality assessment that was beginning to take hold for adult cardiac surgery had not taken hold in the world of pediatric heart surgery.

That attitude began to change with a courageous presentation by Marc de Leval at the 1993 meeting of the American Association for Thoracic Surgery. At that meeting de Leval presented an “Analysis of a cluster of surgical failures” in which he described his own experience with the neonatal arterial switch operation [6] (Fig. 16.1). Having experienced only one death in his first 52 arterial switch procedures, he was troubled by the occurrence of seven deaths in 16 subsequent cases. After losing patient no. 53 and no. 55 he “instinctively” sensed concern and visited another center known for low mortality in an effort to derive insights into his unanticipated significant change in outcomes. After losing patient no. 59, no. 63, and no. 64 he revisited the same institution. After patient no. 67 and no. 68 died, he ceased performance of the procedure at his own institution and did not resume it until he had retrained at a third institution. Upon resumption at his own institution he experienced only one death in his next 35 arterial switches. His retrospective “analysis” examined the full experience of 104 consecutive arterial switch operations in an effort to determine if the “cluster of failures” could have occurred by chance alone, and, if not, could the unfavorable trend have been detected earlier. In his manuscript, de Leval describes two techniques for identifying worrisome trends: the CUSUM procedure (cumulative sum) [7] and comparison to benchmarks derived from multicenter data [8]. Looking retrospectively with those techniques he indicated that had “a mechanism of continuous monitoring…been in place” at his institution the decision to retrain would have been reached sooner. By bravely divulging his own experience with failure and proposing techniques to detect failure early, de Leval deserves the credit for introducing quality assessment and quality improvement into the cloistered world of pediatric cardiac surgery.

Fig. 16.1
figure 1

Figure taken from de Leval et al., 1994, demonstrates cumulative sum (CUSUM) technique charting deaths as they occurred with comparison to a predicted rate determined by the Congenital Heart Surgeons’ Society (CHSS). If this type of continuous monitoring had been in place at Great Ormand Street (GOS) during this experience the abrupt change in failure rate would have been recognized earlier (Abbreviations: TGA transposition of great arteries, VSD ventricular septal defect) (Reprinted from de Leval et al. [6], with permission from Elsevier)

Full size image

Pediatric cardiac surgery is a complex and challenging endeavor that requires more than the deft surgical skills of a single individual. Successful cardiac surgery programs employ a teamwork approach with collaboration between surgeons, cardiologists, anesthesiologists, intensivists, nursing staff, other hospital staff, and hospital administrators. When a highly functional team is not established or breaks down because of poor communication or failed leadership, patients die unnecessarily. Around the time that de Leval was sensing his “cluster of failures” with the arterial switch operation, concerns were arising about the outcomes of infants undergoing heart surgery at the Bristol Royal Infirmary in Bristol, England [9]. Those concerns reached the point where infant heart surgery was discontinued at that center in 1995. A retrospective public inquiry into the adequacy of pediatric cardiac surgical services in Bristol was led by Sir Ian Kennedy and resulted in an extensive final report published in 2001. This thorough and incisive document delineates the multiple systemic and individual factors that resulted in excessive mortality at that center over that period. An important factor revealed in this report was the lack of a systematic mechanism for monitoring the clinical performance of the pediatric cardiac surgery program.

Bristol was not the only pediatric cardiac surgery program to suspend its services in 1995. In February of that year the pediatric cardiac surgery program at the Winnipeg Health Sciences Centre in Manitoba, Canada ceased providing children’s heart surgery following the death of 12 children in 1994. An Inquest was initiated in December 1995 and a final report published in 2000 [10–12]. This retrospective evaluation discovered problems in leadership, teamwork, communication, mentorship, and decision-making. Of primary importance was a failure to collect and analyze group data to track trends and to compare results with other programs. Less publicized but similar situations at centers providing pediatric heart surgery in the United States also occurred during and since. Because of the inherent vulnerability of the patient population and the complexity required to integrate resources and disciplines, pediatric cardiac surgery programs are especially sensitive to breakdowns. All three of the referenced situations point to the importance of collecting and tracking outcomes as well as the need to implement a human factors and safety systems approach to improving pediatric cardiac surgical outcomes.

Development of Multi-institutional Pediatric Cardiac Surgery Databases

While surgeons and programs may have monitored their own results, the assessment of those results was limited by the lack of agreed standards for comparison. If any standards were cited, they were derived from publications in the medical literature of exceptional results. Real data from multiple institutions was required to determine realistic standards. Thus began the development of databases in pediatric heart surgery. The first multi-institutional database for this group of patients was initiated in the upper Midwest of the United States to derive data to justify funding for children with congenital heart disease for the state of Minnesota and surrounding states. In 1982 the Northern Great Plains Regional Cardiac Program (NGPRCP) was begun under the leadership of Dr. James Moller of the University of Minnesota [13]. Initially data was collected from five centers: the Mayo Clinic, Minneapolis Children’s Medical Center, the University of Iowa, the University of Nebraska, and the University of Minnesota. Other centers outside of the upper Midwest began to voluntarily join the program. In 1990 the name was changed to the Pediatric Cardiac Care Consortium (PCCC) and the number of programs submitting data exceeded 40 by the year 2000. The database focused on two outcomes: death and length of hospital stay. Data was submitted to a central center with trained coders and was kept confidential with de-identified patient information. Annual center specific reports were made available to each submitting center permitting a comparison to aggregate data from the other participating centers. The PCCC was truly the first database providing data from multiple centers performing pediatric heart surgery allowing realistic information regarding mortality rates for a variety of pediatric cardiac surgical procedures. Benchmarks for mortality of individual operations could now be realistically derived [14].

Another database for patients with congenital heart defects was introduced in 1985 by the Congenital Heart Surgeons’ Society (CHSS). Two members of this group of pediatric heart surgeons, Dr. John Kirklin and Dr. Eugene Blackstone, proposed the pooling of data from the members’ institutions to assess the management outcomes of specific cardiac malformations. This database differed from the PCCC in that only select congenital heart lesions were tracked and each patient was followed annually. The first lesion to be studied was transposition of the great arteries. This was a timely choice that provided useful feedback permitting comparison of atrial redirection operations (Mustard and Senning procedures) to the newly introduced arterial switch operation. Data derived from the CHSS database provided valuable evidence favoring the latter procedure facilitating the pediatric surgical community’s transition to the newer operation for newborns with complete transposition [8]. Other cardiac lesions addressed by the CHSS Database have been pulmonary atresia with intact ventricular septum, pulmonary stenosis, interrupted aortic arch, coarctation of the aorta, critical aortic stenosis, aortic atresia, and tricuspid atresia. The CHSS database continues to serve as a repository of important data from which longer-term outcomes of rare congenital heart anomalies can be determined.

The PCCC and CHSS Databases are representative of the two forms of databases that have become valuable in the assessment of pediatric heart surgery outcomes [15]. The PCCC Database is a registry database in which some data is collected for all of the patients. The amount of data collected on each case is limited to a predetermined set of identifiers and early outcomes. This minimal dataset must be clearly assessable and easily and reliably entered for each patient. Registry databases like the PCCC help determine standard of care references from which benchmarks can be developed. The CHSS Database is an academic database in which “all of the data” is collected on some of the patients. Academic databases investigate specific populations or subgroups of patients to generate new knowledge. An academic database is much more amenable to longitudinal follow-up than a registry database and allows much more detailed studies. Both forms of databases continue to be important in the assessment and improvement of pediatric cardiac surgery quality.

In 1990 an informal group of European congenital heart surgeons (later formalized into what is presently the European Congenital Heart Surgeons Association (ECHSA)) recognized the importance of collecting data from all of the operations performed at their respective centers. This collaboration led to the birth of the European Congenital Heart Defects Database (ECHDD) in 1992, which began under the direction of Dr. Martin Elliot at Great Ormond Street Hospital for Children in London, England. By 1995 31 centers from 18 countries were submitting data. In 1998 the ECHDD relocated to the Children’s Memorial Health Institute in Warsaw, Poland under Dr. Bohdan Maruszewski as director. As congenital data was being collected in the ECHDD, the European Association for Cardio-Thoracic Surgery (EACTS) was developing the European Cardio-Thoracic Surgical Registry (ECSUR). In 1999 it was decided that the ECHDD would be part of the ECSUR. Initially termed the Pediatric ECSUR this registry database would soon be known as the EACTS Congenital Heart Surgery Database. By 2001 84 programs from 34 countries were represented. By 2012 the number of European centers had risen to 265 representing 36 countries. By that time another 147 centers outside of Europe from 43 countries all over the world had been added. Today, these databases include data from over 100,000 patients and 125,000 operations.

Contemporarily with what was happening in Europe, the US Society of Thoracic Surgeons was developing its own registry database for congenital heart surgery in North America. Dr. Constantine Mavroudis at Children’s Memorial Hospital in Chicago, Illinois was responsible for the initial development of the STS Congenital Heart Surgery Database. Centers began joining in 1994 and by 1997 24 North American centers had provided data that included mortality and length of stay derived from over 8,000 patient records. In the late 1990s Dr. Jeff Jacobs of St. Petersburg, Fl. assumed the Chairmanship of the STS Congenital Heart Surgery Database Task Force and has been responsible for the continued maturation and growth of this database [16]. By the end of 2011 there were over 100 participating centers representing more than 80 % of all congenital heart programs in the United States. Data from over 200,000 operations have been submitted from 1994 to today.

The key to the successful development of the congenital databases in Europe and North America was the establishment of a common language or nomenclature to describe the large number of disease entities treated and procedures performed. In addition, it was necessary to determine the minimal set of data required to allow valid linkage with other databases. In 1998 the EACTS and STS collaboratively initiated the International Congenital Heart Surgery Nomenclature and Database Project. By 2000 a minimum dataset was agreed upon and updated in 2001 [17, 18]. In the next few years the International Society for Nomenclature of Paediatric and Congenital Heart Disease (ISNPCHD) was created including surgeons, pediatric cardiologists, and congenital cardiac morphologists. By 2005 the nomenclature working group of the ISNPCHD had cross-mapped the nomenclature of the International Congenital Heart Surgery Nomenclature and Database Project of the STS and EACTS with the European Paediatric Cardiac Code (EPCC) of the Association for European Paediatric Cardiology (AEPC) creating the International Paediatric and Congenital Cardiac Code (IPCCC). With this common language and an agreed upon minimum dataset, the EACTS and STS Databases were up and running and able to combine information from two very large experiences in pediatric heart surgery. Furthermore, the stage was set for linkage with databases from other parts of the world, from other medical disciplines, and even with some administrative databases.

In pediatric cardiac surgery there is a broad spectrum of complexity and risk. The mix of cases at one institution may consist of simple low risk cases whereas the mix at another institution may contain more complex cases subject to significantly higher risk. Comparing the overall mortality of the two institutions would provide a misleading assessment of outcomes. Thus to provide fairer comparisons between centers, stratification schemes were developed. The first scheme (the Risk Adjustment in Congenital Heart Surgery-1 (RACHS-1) method) consisted of six risk categories for surgical procedures. This stratification, which was developed and championed by Dr. Kathy Jenkins at Boston Children’s Hospital, was derived from a consensus of pediatric cardiologists and cardiac surgeons [19]. The application of RACHS-1 to the PCCC dataset for the year 1996 confirmed a spread of mortality rates across RACHS-1 categories with category 1 having a mortality of 0.4 % and category 6 having a mortality of 41.5 %. Another approach to risk stratification, fostered by Dr. Francois Lacour-Gayet who worked at the Eppendorf University Hospital in Hamburg, Germany, applied scores and levels to each operation based on the perceived risk of mortality, morbidity, and the technical difficulty of the procedure [20, 21]. These values were referred to as Aristotle basic complexity (ABC) scores and levels. As with RACHS-1 the ABC scores were derived from expert opinions of experienced clinicians. The Aristotle approach was incorporated in the reports from the EACTS and STS Congenital Databases beginning in 2002 and the RACHS-1 stratification was added to the reports in both databases in 2006. Both approaches provided reasonable risk stratification, but the Aristotle levels classified more operations than did RACHS-1, whereas RACHS-1 provided more discrimination at the higher end of complexity [22]. By 2008 enough data had been collected in both the EACTS and STS Congenital Databases to permit an objective determination of risk from actual surgical outcomes. This new stratification scheme resulted in five “STS-EACTS Mortality Levels” which demonstrated better predictive value for mortality than the RACHS-1 or Aristotle systems [23]. This objectively derived system, now referred to as STAT Mortality Categories, has become the preferred stratification protocol for congenital heart operations and is currently incorporated in all EACTS and STS Congenital Database reports.

Now with an accepted nomenclature and means for risk stratification the EACTS and STS registry databases provide useful quality assessment for centers willing to participate. The formats of these databases continue to evolve to further improve the value of the information collected and analyzed. Periodically data fields are added and subtracted as experience with the information accrues and clinical questions increase and decrease in importance. The validity of the data in the databases must be assured with regular audits and other forms of data verification [24]. Measurement of outcomes has focused mostly on hospital or 30 day mortality. Mortality data alone, however, is insensitive to quality issues experienced by low risk procedures and says little about process failures that may lead to near misses and harm but not death. The hospital length of stay is available in these databases and has served as a proxy for the morbidity of pediatric cardiac surgical procedures. Other measures of morbidity are now being tracked in these databases including unplanned reoperations, postoperative renal failure, and postoperative complete heart block [25–27]. Hopefully tracking these measures will provide a more thorough assessment of process and outcomes for these patients. How these patients do after the initial 30 days following an operation is also important. Although registry databases are poorly suited for longitudinal follow-up, efforts are in progress to create Health Insurance Portability and Accountability Act (HIPAA)-compliant unique patient identifiers that would permit tracking of patients through a series of operations and for longer-term follow-up [28]. As many pediatric cardiac surgical conditions require staging of surgical procedures, the ability to keep track of the same patient through more than one procedure is important in assessing the outcomes of such sequenced approaches for individual patients. HIPAA-compliant unique patient identifiers may also permit linkage with administrative databases to enable assessment of costs or other nonclinical information or to assist with data verification regarding death, length of hospital stay, or other demographics [29].

From Quality Assessment to Quality Improvement

Pediatric cardiac surgery centers have the ability now to participate in highly developed multi-institutional databases such as the STS and EACTS Congenital Databases, with each center having the ability to compare its own outcomes with those of other database members. For the STS Congenital Heart Surgery Database each center’s outcomes are kept confidential but can be compared to the outcomes experienced in the aggregate of the other centers. In addition, graphical depictions are provided such that an individual center can visualize its outcomes in comparison with those of the other centers in a de-identified manner (Fig. 16.2). Outcome information is provided at a reasonable interval (every six months for the STS Congenital database) to each center who can continuously monitor its outcomes. Of course, not every center will feel assured, as its outcomes may not be satisfactory for every procedure. In fact, analysis of data from the STS Congenital Database has revealed that there is significant variation in outcomes among institutions providing pediatric heart surgery [30, 31]. The degree of variation in terms of mortality is minimal for low risk procedures, but tends to be as high as sixfold with increasing complexity and risk. Variation in outcomes is not only seen between centers but can occur within the same institution and with different surgeons with good results for one operation and much less satisfactory results for another operation [32]. These analyses indicate a spectrum of performance among and within programs providing pediatric cardiac surgery. These findings indicate a need for some programs to reevaluate their performance as a whole or for particular procedures.

Fig. 16.2
figure 2

Example of the graphical presentation of discharge mortality from a semiannual report from the Society of Thoracic Surgeons Congenital Heart Surgery Database. Each hospital, identified only to itself, can compare its results with the other unidentified hospitals. Similar graphical presentations are provided for each strata of risk (Abbreviation: Cl confidence limits)

Full size image

One approach to improving the overall outcomes in pediatric cardiac surgery is to identify the lowest performers and eliminate them, such as occurred in the cases of Bristol and Winnipeg, in the UK and Canada, respectively. A graphical depiction of this approach presented by the American Board of Pediatrics [33] is demonstrated in Fig. 16.3a. In this graphic variation in quality of care is represented as a bell shaped curve. When the tail of the curve (lower 5 % of performers) is eliminated, the improvement in quality realized by the system as a whole is only modest. On the other hand, if strategies for improvement are applied across all institutions, the variation between institutions can be significantly diminished (a narrower bell) and overall quality improved (bell moved to the right) (Fig. 16.3b). One such strategy is the application of standardized structures and processes to all programs.

Fig. 16.3
figure 3

(a) Removing poor performing providers has minimal effect on improving overall quality (From Miles [48]). (b) Applying a quality improvement initiative to all providers reduces variation among providers and improves overall quality (Reprinted from Miles [48]. Reproduced by permission of the American Board of Family Medicine)

Full size image

In 2007 under the leadership of its president at the time, Dr. John Mayer, the Society of Thoracic Surgeons created a task force to develop a list of quality measures for pediatric and congenital heart surgery. By 2011 this task force, which consisted of pediatric and congenital heart surgeons, had developed a set of 21 Quality Measures which was further vetted by four other STS committees and approved by the Executive Committee of the Society of Thoracic Surgeons. In the same year the same set of Quality Measures was reviewed and endorsed by the Congenital Heart Surgeons’ Society. This set of quality measures is listed in Table 16.1 and follows Donabedian’s principles for quality with five measures related to structure, six related to process, and ten different outcome measures. These Quality Measures for Pediatric and Congenital Heart Surgery were published in early 2012 where the details of each measure are described [34]. Most of these quality measures were derived from the experience and expert opinions of the STS task force and will require further evaluation in terms of reliability, validity and scientific acceptance. The STS Congenital Heart Surgery Database has added a Quality Module that will help determine if these measures are indeed associated with improved outcomes. The National Quality Forum, which also reviewed these measures, agreed that more data is required before it could support most of these measures, but it did endorse three measures: (1) participation in a national database, (2) measurement of total programmatic volume and programmatic volumes stratified by the five STAT mortality categories, and (3) operative mortality stratified by the five STAT mortality categories.

Table 16.1 Quality measures for congenital and pediatric cardiac surgery
Full size table

Minimizing variation in patient outcomes and improving the overall quality in pediatric heart surgery requires more than adoption and adherence to a set of quality measures. The Society of Thoracic Surgeons and the Congenital Heart Surgeons’ Society have both been instrumental in making quality improvement a priority for their members. The STS Congenital Heart Surgery Database has been the foundation for quality assessment and improvement in North America. The STS has also created separate workforces dedicated to Congenital Heart Surgery, Surgical Treatment of Adults with Congenital Heart Disease, and Peer Review and Evaluation. The Congenital Heart Surgeons’ Society has created the Committee on Quality Improvement and Outcomes to address these issues and to serve as a resource for centers seeking assistance.

Several areas of clinical research will prove valuable in improving the outcomes and minimizing variation. Studies in human factors, team performance, and the complex interactions required of a team providing pediatric heart surgery may lead to a substantial decrease in the number of errors that result in untoward outcomes [35–39]. The technical aspects of complex pediatric cardiac operations themselves are now being carefully assessed and are revealing a significant effect on results [40–42]. The ability to grade each operation with a Technical Performance Score will be valuable to surgeons who can use that feedback to hone their surgical skills [43]. Improvement in team dynamics and interactions as well as improvement in the performance of individual team members is expected to further improve the quality of pediatric heart surgery.

One other area that has promise for the field of pediatric heart surgery is the concept of learning collaboratives. Learning collaboratives involve the sharing of processes, approaches, and outcomes with other institutions. This approach was pioneered by a group of adult cardiac surgery programs in northern New England. When these hospitals noted that their outcomes for coronary artery surgery were unacceptable, they decided to collaborate rather than compete with each other [44]. The collaboration consisted of feedback of outcome data, training in continuous quality improvement techniques, and round robin site visits to each other’s institution. Site visiting teams consisted of surgeons, anesthesiologists, perfusionists, nursing staff, and others considered important to the delivery of coronary artery surgery at each institution. By observing practices at the other sites variation in processes diminished and variation in outcomes improved with a 24 % decrease in operative mortality for the programs as a whole [44]. Similar learning collaboratives have been successfully applied in other regions of the United States for the delivery of adult cardiac surgery [45, 46]. Such a learning collaborative coupled with a robust continuous quality improvement framework has yet to be attempted in pediatric cardiac surgery, but could result in substantial quality improvement for those centers courageous enough to participate in such a venture [47].