Keywords

1 Introduction

Evidence-based practice is defined as “the conscientious, explicit and judicious use of current best evidence in making decisions about the care of the individual patient. It means integrating individual clinical expertise with the best available external clinical evidence from systematic research” [1]. Evidence in different forms has thus led to many of the changes in practice of medicine that we see today.

For many disease processes and interventions, national and international registries have evolved as a preferred means of collecting data that contribute to this evidence. Clinical registries can be defined as “prospective, observational cohort study of patients with a particular disease and/or receiving a particular treatment/intervention” [2]. Registries thus provide a long-term opportunity to generate important disease-based and treatment-based information. As the available infrastructure for information technology develops, together with increasing demand for quality improvement and transparency, analysis of the data by statistical techniques can thus change the basis of surgery [3].

Clinical registries are considered the gold standard of observational data [4] although by their nature, they are inferior to randomized clinical trial (RCT) data, systematic reviews or meta-analyses. One of the main advantages of RCT data over registry data is that confounding variables that might lead to biased treatment effects have been formally accounted for by randomization.

2 Advantages and Lessons Learnt from Using Registries

One of the earliest registries in the management of obesity was the International (formerly National) Bariatric Surgery Registry that began collecting data in January 1986. It was created by Dr Edward Mason in the United States of America, and was active from 1986 to 2001. It contained data on 38,000 patients with a 30-day mortality rate of 0.24 % [5]. One of the first reports from this registry confirmed that the increasing weight of candidates for surgical treatment mandated earlier use of operative treatment before irreversible complications of obesity could develop. It further confirmed that the risk of obesity surgery was low and there was good evidence towards decreasing postoperative hospital stay. These impressive findings contributed to the evidence supporting the development and expansion of bariatric surgery [6].

Despite the increase in availability of bariatric surgery there are very few reports of long-term follow up. Published follow up reports after bariatric surgery are at best inadequate. A report by Higa et al. in 2011 confirmed that despite all reasonable attempts to make contact, only seven percent of patients were available to have their data recorded at 10 years after gastric bypass [7]. Similar follow up rates have been recorded from various units internationally. One of the aims of a registry therefore is to bridge the gap between good practice and good data maintenance. Not all good databases, however, have originated from registries. One of the best follow up programs in bariatric surgery is not based on a registry. O’Brien et al. recently published their 10 year series of over 3,000 gastric banding patients in which they achieved follow up in 81 % [8]. The collection of follow up data into registries, if appropriately funded, resourced and optimized could provide invaluable globally relevant data. There is a golden opportunity for publicly funded health services to collect such data. The example of cancer registries, where the infrastructure to collect basic observational data on quality of services is deeply embedded, is an obvious parallel. In contrast, it probably will always be especially challenging for countries that do not have publicly funded health services to collect registry-level data on bariatric surgery.

One of the mechanisms by which worldwide data collaboration can drive changes in practice is the generation of research questions that can lead to a better understanding of the problems based on the information obtained from the registries. Buchwald et al. published a review of the trends in bariatric surgery worldwide in 2011 [9]. This review identified the total number of bariatric procedures upto 2011 as 340,768 and the total global number of metabolic/bariatric surgeons as 6,705. The most commonly performed procedures were Roux-en-Y gastric bypass (RYGB) 46.6 %; sleeve gastrectomy (SG) 27.8 %; adjustable gastric banding (AGB) 17.8 %; and biliopancreatic diversion/duodenal switch (BPD/DS) 2.2 %. However, a limitation of this type of review is the lack of data on variables such as patient body mass index (BMI) ranges, sex, age and co-morbidities—data that can greatly affect the choice of procedure and the outcomes. For instance the burden of comorbidity between different bariatric surgery populations and the types of operations done for comorbidities such as type 2 diabetes are unknown. Such data can only be reliably obtained from an international collaborative registry.

3 The ‘Hawthorne’ Effect

The Hawthorne effect is a form of reactivity whereby subjects improve or modify an aspect of their behavior, which is being experimentally measured, in response to the fact that they know that they are being studied [10, 11]. Henry A. Landsberger, in 1950, coined the term Hawthorne effect [12] when he was analyzing older experiments from 1924 to 1932 at the Hawthorne Works (a Western Electric factory outside Chicago). The original purpose of the experiments was to study the effects of physical conditions on productivity. They were conducted for the most part under the supervision of Elton Mayo, an Australian-born sociologist who eventually became a professor of industrial research at Harvard.

Two groups of workers in the Hawthorne factory were used as guinea pigs. One day the lighting in the work area for one group was improved dramatically while the other group’s lighting remained unchanged. The researchers were surprised to find that the productivity of the workers who had better lighting increased much more than that of the control group. The employees’ working conditions were changed in other ways too (their working hours, rest breaks and others), and in all cases their productivity improved when a change was made. Indeed, their productivity even improved when the lights were dimmed again. Although at the end of the experiment everything had been returned to the way it was before the changes had begun, productivity at the factory was at its highest level and absenteeism had plummeted.

The experimenters concluded that it was not the changes in physical conditions that were affecting the workers’ productivity. Instead, it was the act of active observation that was responsible for the reported changes. Thus, in the context of registries, it is assumed that widespread adoption of data collection can encourage a higher level of effort from all personnel involved, thus potentially improving the overall results and the quality of the service provided.

4 Current National Bariatric Surgery Registries

4.1 Bariatric Outcomes Longitudinal Database (BOLD)

As a result of a previous collaboration with the American Society for Metabolic and Bariatric Surgery, the Surgical Review Corporation developed the Bariatric Outcomes Longitudinal Database (BOLD) [5], to provide observational data as a part of center of excellence initiative, with the intention to improve and optimize bariatric surgical care. The BOLD bariatric database is the world’s largest and most comprehensive repository of clinical bariatric surgery patient information, with data from more than 500,000 patients.

4.2 Michigan Bariatric Surgery Collaborative

Since this initiative was set up in 2006 [13], rates of venous thromboembolism have reduced by half. According to Blue Cross Blue Shield of Michigan actuaries, quality improvement in bariatric surgery saved Michigan payers $15 million between 2008 and 2010. This is an excellent example of changes in practice driven by a system of data collection in which there is independent verification of data entry and accuracy.

4.3 Scandinavian Obesity Surgery Register (SOReg)

Scandinavian Obesity Surgery Register: SOReg was first proposed by Hedenbro in 2000. It was ready for data entry trials in 2004 and became fully functional in 2007. Since then, the registry has recorded over 35,000 cases from over 40 centers in Sweden and Norway, and collects data on more than 9,000 annual procedures.

4.4 Ontario Obesity Bariatric Registry

The Ontario-based Bariatric Registry project aims to improve the care of the obese patients and increase the effectiveness of health care dollars. It consists of creation of a centralized referral process, to collect standard referral data on all patients, and direct their referral to their nearest bariatric center.

4.5 Italian National Registry

The Italian registry is one of the largest bariatric registries in the world. A study of data from an Italian national registry of 13,871 morbidly obese adults who underwent bariatric surgery between 1996 and 2006 demonstrated that the type of procedure significantly influenced mortality risk [14]. The risk ranged from 0.1 % for adjustable gastric banding to 0.8 % for biliopancreatic diversion.

4.6 Europe

In Europe there are several examples of national registries with limited follow up data. the International Federation of Surgery for Obesity (IFSO) European Chapter has also set up a Centre of Excellence program with a linked registry—the IBAR (International Bariatric Registry).

4.7 National Bariatric Surgery Registry (NBSR-UK)

The National Bariatric Surgery Registry was the result of a collaboration between Association of Laparoscopic Surgeons of Great Britain and Ireland (ALSGBI), Association of Upper Gastrointestinal Surgery (AUGIS), British Obesity & Metabolic Surgery Society (BOMSS), and Dendrite Clinical Systems Ltd. The key objective of the registry was to accumulate sufficient data to allow the publication of a comprehensive report on outcomes following bariatric surgery. This would include reportage on weight loss, co-morbidity and improvement of quality of life. The NBSR was set up in 2009 and by the ending of the year 2013 approximately 33,000 patient records have been accumulated. The first report of the surgeon-anonymized outcomes of over 8,000 patients was published in April 2011 [15].

5 Registries in Other Specialties

The Society for Cardiothoracic Surgery in Great Britain & Ireland has been actively involved in collecting, analyzing and benchmarking the data since 1977, and has been recognized as an international leader in this field; having published data down to individual surgeon level since 2005. Intermittent comprehensive reports of trends and outcomes in cardiac surgery (The Blue Books) are also published. The most recent Blue Book (Demonstrating Quality) was published in 2009.

On a Europe-wide level, the European database project was established in 2003 to collect, merge, and present cardiac surgical data throughout Europe. Over the last 10 years, the contributors towards this database have increased from 12 countries to 29 countries with over one million patient records.

6 Setting up a Clinical Registry and Sensitivities of Data Ownership

Information governance is the framework that brings together all the legal rules, guidance and best practice to ensure that the personal information is collected and stored safely. Thus an understanding of information governance is paramount before establishing any clinical registry. The European Data Protection Directive [16] regulates the transfer of personal data from an EU member state to a third country that has an adequate level of data protection. Understanding of these directives is essential when setting up an international registry.

Since a large number of organizations are usually involved in maintaining a registry, managing the intellectual property within it is a very complex process [17]. Also, if a registry is to be mandated post-hoc as a means of discriminating between the performances of individual surgeons or units, particular sensitivity needs to be used to encourage ‘buy-in’ to the process.

In 2013, there was a heated debate about the ownership of the bariatric surgery data that had been collected voluntarily by surgical teams into the UK National Bariatric Surgery Registry (NBSR). The Secretary of State held the view that data concerning National Health Service (publicly funded) patients should be available in the public domain even though there had been no administrative support or infrastructure available within the provider units for registry maintenance. Thus there was no way to ensure there were no missing/incomplete records or data inaccuracies. The resulting challenge to gain consent for data publication highlights the complexities of handling and storing large amounts of data on behalf of contributors whose view was near unanimous that the right to publish the data on their practice should remain with the surgeon, in the absence of public funding for data collection [15].

The resulting mandated report covered 106 consultant surgeons contributing to the NBSR for the financial year 2012/2013. One hundred and one surgeons (95 %) consented to have their individual outcomes for primary surgery published and the results showed no potential statistical outliers for mortality or length of stay. Using Hospital Episode Statistics codes, it was estimated that there were 138 NHS surgeons doing bariatric surgery in the 11 months between April 2012 to February 2013, and 5,656 operations were recorded. Most bariatric surgeons (77 %) were entering data and the great majority of NHS patients (up to 78 %) were being recorded into the registry.

There were three recorded deaths for an in-hospital mortality rate of 0.07 %, equivalent to an in-hospital survival rate of 99.93 %. The average length of hospital stay for all operations was 2.5 days.

The latest report from the National Bariatric Surgery Registry (NBSR) published on November 10, 2014. Following on from the success of previous years, this registry compiled information from 161 surgeons at 137 hospitals and reported figures on 16,956 primary operations and more than 1,327 planned follow-up procedures. Once again, excellent results with regards to the observed in-hospital mortality rate after primary surgery were notes (0.07 % overall).

7 Designing the Registry

7.1 Dataset Design and Collection

A well-designed and concise dataset design is central to the success of any clinical registry. Small datasets facilitate high participation rates and maximize rates of complete records. Registries with too many data fields are prohibitively time consuming and are inevitably limited by missing data points or inaccurate data entry. An example of the measures that can be used to analyze this is the proportion of records that are complete. For instance in the first report of 1 and 2 years outcome in 8,000 patients from the UK NBSR in 2011 it was found that 85 % of records had zero or only one baseline comorbidity data entry points missing [15]. The commonest comorbidity question that was missing was one that required a specific question to be asked that could not be gleaned already from the patient record—that is the functional status (‘how many flights of stairs can be climbed without resting?’). The use of traffic lighting to indicate missing data entry, incomplete or outstanding data can be very useful, as can the use of hover prompts and intuitive progression through the pages of data entry. It is also imperative that the terminology for each data point is defined, so as to avoid confusion between, for example, primary and revision surgery.

7.2 Data Validation

After collection, data validation is essential to ensure that the data are accurate for reporting and research purposes [18]. External cross checking can be performed by comparing events such as mortality reported by the registry to the Hospital Episode Statistics (HES) database. HES codes were analyzed in conjunction with the surgeon-level reporting described above and it was estimated that the overall in-hospital mortality rate for bariatric surgery was 0.11 % for the four financial years 2009–2013, equivalent to a survival rate of 99.89 %. This validated the very low mortality from bariatric surgery recorded by the consultant surgeons contributing to the NBSR.

8 Minimum Datasets and Definition of a Bariatric ‘Success’

There is currently no agreed minimum clinical outcomes dataset for bariatric surgery. To add to the ambiguity and lack of uniformity, there is currently no international agreement on how to calculate the excess weight loss or whether it should be completely replaced by excess BMI loss. There also needs to be increased emphasis on the accurate capture and reporting of complications and collation of patient-reported outcomes. Most importantly, there is no uniform definition of success of a bariatric procedure despite several decades of bariatric surgery and hundreds of thousands of patients it is still not known whether the outcome measures used should be weight loss, comorbidity outcomes, quality of life, or a varying combination of all these.

9 Data Pre-Processing

Clinical registries inevitably have unclean data. This is defined as an accumulation of transcription errors, logical inconsistencies, missing information, duplicate records and others. However, simply removing these data will potentially lead to an increase in bias and variance for any subsequent analysis conducted using the registry [19]. Thus appropriate resources should be allocated to this process which will usually require a close collaboration between clinicians and database managers.

10 Self Reporting vs. Independent Reporting?

The ideal data collection should be performed by an independent body which is not involved directly with the care or outcomes of bariatric patients. This would ensure that the collected data are not biased. It would also detach data collection from the clinical sources of funding thus making it more independent and credible.

Of the current registries, only the Michigan Bariatric Surgery Collaborative maintains a registry with independent reporting, and such quality assurance may currently be unique. The University of Michigan Health System serves as the Coordinating Centre and is responsible for collecting and analyzing comprehensive clinical data from the participating hospitals. It uses these analyses to examine practice patterns, generate new knowledge by linking processes of care to outcomes and by identifying best practices and opportunities for quality and efficiency improvement. The Centre further supports participants in establishing quality improvement goals and assists them in implementing best practices. The Metabolic and Bariatric Surgery Accreditation and Quality Improvement Program (MBSASQIP) of the ASMBS and the American College of Surgeons similarly aim to maintain and improve standards of data collection in the USA [20]. Currently, due to reasons such as lack of funding or resource, all or most of the other bariatric registries are based on self-reporting and are thus open to potential bias.

11 Using the Registry: Strengths and Limitations

National and international registries can be used for audit purposes, development of risk-prediction models, epidemiological and scientific research. They have also promoted significant developments in statistical methodology. Risk models are commonly developed and validated using registry data and are necessary when the data are used for assessment of performance of individual units or surgeons against statistical means for governance purposes.

One of the main concerns of large registries is that the analysis of results is only as good as the data entered. Thus, one of the major pitfalls in registries is the presence of incomplete data. This situation can be retrievable in certain circumstances, especially when duplicate data are being collected by other disciplines that share patient’s care. However, records with inaccurate data are more difficult to identify and verify, and if the patient had a poor outcome such as mortality because then the resulting error means that the whole dataset is misrepresentative. The available methods for correcting erroneous data entry are limited and ultimately the responsibility for accurate data entry lies wholly with the surgical team, assuming it is properly resourced.

12 Device Monitoring and Relevance to Bariatric Surgery

Several implantable devices are presently being used in bariatric patients. Adjustable gastric banding is the third commonest bariatric procedure performed worldwide with its market share currently being 17.8 % of all bariatric procedures performed. This represented 60,677 procedures in 2011 alone [9]. The intragastric balloon device procedure is done infrequently in individual countries but still potentially represents a large number of bariatric procedures when analyzed worldwide. The recent case of the Poly Implant Prothèse breast implant highlights the importance of device monitoring. These implants were shown to have a significantly higher rupture rate than other implants after 5 years [21]. Registries for such devices would have possibly led to earlier detection of the unacceptable failure rates of these implants.

Conclusion

In the past, bariatric surgery registries have attempted to capture data of clinical significance but more efforts need to be put in this aspect of bariatric surgery. Had we as surgeons participated in such programs from the beginning, the landscape would have been potentially very different than what it is today. The creation of bariatric registries internationally is a positive step towards a unified global bariatric database. Inferences from such a powerful tool will provide payers and patients with reassurance that quality control is evident in bariatric surgery. It could provide a basis for generating research questions and provide important observational data on differences in practice in bariatric surgery worldwide. More powerful data would be generated from the adoption of an internationally accepted clinical outcomes dataset, when one is published. “Knowledge is gained by gathering data, whereas, wisdom is earned by going through actual life experiences.

”—Master Jin Kwon

Key Learning Points

  • The Hawthorne effect is a psychological phenomenon that produces an improvement in human behavior or performance as a result of increased attention from superiors, clients or colleagues. Thus in the context of bariatric surgery registry, it is conceivable that the overall results may improve by the act of active data collection and monitoring

  • Registries can provide a long-term opportunity to generate important disease-based and treatment-based information

  • A well-designed and concise dataset design is central to the success of any clinical registry. Smaller datasets facilitate high participation rates and maximize rates of complete records. The data from registries is only as good as the data entered. Thus, one of the major pitfalls in registries is the presence of incomplete data.