Introduction

The German Cancer Society (Deutsche Krebsgesellschaft [DKG]) has been providing certification for oncological care institutions since 2003 in order to improve the quality of treatment provision for cancer patients. Certification creates a network of qualified and certified interdisciplinary and trans-sectoral institutions that reflects the entire chain of care for the affected patients [1]. Indicator figures constitute a central component of the certification process and are used to reflect qualification, quality, and collaboration between the specialist disciplines. The benchmarking reports of the certified colorectal cancer centers that are published since 2011 include the results of the respective indicators and provide an overview on the development of the quality of care over the course of time throughout Germany. At the same time, they enable the individual centers to compare their own performance and optimize it if necessary.

This article presents data from the third annual analysis of the DKG-certified colorectal cancer centers with a particular focus on indicators for colorectal cancer surgery. The overall development of the centers, including specific major topics, is discussed using selected indicators.

Methods

An annual presentation of treatment results in the form of indicator figures is obligatory for all DKG-certified centers. The data for these indicators are collected by the centers using an electronic questionnaire and are submitted to OnkoZert [2]. OnkoZert is an independent institute that organizes the auditing procedure on behalf of the DKG as the certification office. OnkoZert stores the data in a Microsoft SQL server express database, and using these data generates the analyses used for the annual benchmarking reports.

Quality assurance in data collection and analysis is ensured by various controls. Plausibility thresholds are provided for the indicators. When thresholds are exceeded, the center is obliged to comment on the results so that the questionnaire can be closed and sent to the certification institute. When the data are received by the certification institute, formal checking of the questionnaires is carried out to ensure completeness and consistency with the center’s other details that are documented for the auditing procedure. Prior to the audit, the auditor receives the questionnaires and carries out an initial check of the indicator results provided by the center. During the audit, further checks are carried out, e.g., by spot checks of specific patient files.

The on-site tumor documentation at the centers was carried out using a total of 12 different documentation systems. Support from the regional cancer registries was often available.

The 2013 benchmarking report presents the results of the completed auditing procedure for 2012. It does not include all 257 center sites certified by the DKG up to December 31, 2012. Sixteen sites were excluded that were certified for the first time in 2012 and for which data recording was not obligatory for the whole calendar year, as well as 11 sites in which the data system was being converted from one documentation system to another. The report is thus based on data for 19,567 CRC patients at a total of 230 sites who had a first diagnosis of CRC in 2011. It provides details for a total of 28 indicators. The indicators have clearly defined numerators and denominators, and in most cases target values (Table 1). The target values were defined by the experts of the certification commission and the guideline group based on published averages.

Table 1 Indicators for audit year 2012, showing medians and target values
Full size table

Results

The 2013 benchmarking report shows an increase in certified colorectal cancer centers from 188 to 247 centers during the years 2010 to 2012 (+31.4 %). Accordingly, the total number of primary cases treated — i.e., patients with a first diagnosis of colorectal carcinoma — has been continually increasing over the last 3 years. At the end of 2012, 32.4 % of incident cases of CRC in Germany were treated in DKG-certified centers. The numbers of primary cases per colorectal cancer center remained at a similar level to that of previous years, with a mean of 87 cases/year.

The following presentation of the results covers selected indicators. The complete analysis is available on the web sites of the German Cancer Society and OnkoZert [3]. All percentage figures refer to the median and to the audit year 2012, unless otherwise stated.

Figures for interdisciplinary collaboration

The numbers of primary cases presented at the multidisciplinary team conferences are covered by two indicators (Fig. 1). The rate for the indicator “Pre-treatment presentations of primary-case patients with rectal carcinoma or colon carcinoma in UICC stage IV” has increased by 2 % annually since 2010 and is at a very high level with 91.8 %. When the box and whisker diagram is examined, it is notable that the size of the box has continually decreased over the last 3 years. The interquartile distance corresponds to the area containing the mean 50 % of the data. The reduction in the length of the box indicates a reduced scattering of the values and thus assimilation of the proportion of patient presentations among the individual centers.

Fig. 1
figure 1

Indicators for interdisciplinary collaboration from the audit years 2010, 2011, and 2012: pretreatment case presentation; postoperative case presentation; psycho-oncologic care; counseling by social service; participation in research studies

Full size image

The indicator for postoperative case presentations was 98.1 % of cases (target ≥95 %). The results for this indicator have been at a very high level since 2010, at over 95 %. As in the results for pre-treatment case presentations, decreased heterogeneity among the centers was also evident: 95 % of the centers had presentation rates lying between 84 % and 100 % in 2010, whereas in 2012 between 93.1 % and 100 % of the patients were presented in the postoperative tumor conferences.

The indicators for psycho-oncological care (discussion period >25 min) and for counseling by the social service are also among the parameters measured to assess interdisciplinary collaboration in the centers. They are not subject to target values. The proportions of patients receiving psycho-oncological and social services are 54.8 % and 77.1 %, respectively. Following a marked increase from 2010 to 2011 (+45.4 %), the psycho-oncological care provided has remained at the same level, while counseling from the social service has continued to develop further from an initially already much higher level (2010: 73.4 %) during the last 3 years. Due to the absence of a target value, the implementation of these two indicators is extremely heterogeneous among the centers.

Participation in clinical trials is another parameter for interdisciplinary collaboration. During the last 3 years, the proportion of colorectal cancer patients who have been included in studies requiring a vote by local ethics committees has increased and was 13.1 % in audit year 2012 (target value ≥10 %). When this indicator is analyzed, it should be borne in mind that the numerator group may include all patients at the colorectal cancer center and not exclusively primary cases — i.e., also patients with recurrences or distant metastases. In addition, patients who are included in several different studies simultaneously may be counted several times. As the denominator includes the indicator for primary cases, results totaling more than 100 % may occur here.

Indicators for guideline compliance

The analyses of the guideline-based indicators or quality indicators provide information about how well the recommendations included in the guidelines are being implemented in the centers. In the certified centers adjuvant chemotherapy recommended in the German guidelines was received by 73.7 % of patients with UICC (Union for International Cancer Control) stage III colon carcinoma (Fig. 2). This value is similar to that in the previous year, but 4 % below that in 2010. The target value of ≥80 % was only achieved by 38.3 % of the sites.

Fig. 2
figure 2

Indicators for guideline compliance for the audit years 2010, 2011, and 2012: adjuvant chemotherapy for colon (UICC stage III); neoadjuvant radiotherapy or chemoradiotherapy for rectum (UICC stages II and III)

Full size image

The proportion of courses of neoadjuvant radiotherapy or chemoradiotherapy in patients with UICC stages II and III rectal carcinoma has remained consistently at 80 % in each of the last 3 years and thus meets the required target value (Fig. 3). With these two indicators, the low numbers of patients in the denominator need to be taken into account as a problem in the statistical analysis (ranges 2–46 and 1–47).

Fig. 3
figure 3

Box plots of indicators for guideline compliance for the audit years 2010, 2011, and 2012: adjuvant chemotherapy for colon (UICC stage III); neoadjuvant radiotherapy or chemoradiotherapy for rectum (UICC stages II and III)

Full size image

The number of good or moderate quality total mesorectal excision (TME) specimens (Mercury grades 1 and 2) is at an excellent level with 93 % in the centers (Fig. 4). Consequently, the target value of ≥70 % was met by 98.7 % of the sites. The target value was also clearly exceeded in audit years 2010 and 2011 as well. Increasing uniformity among the centers is clearly evident in the implementation of this indicator. Whereas 95 % of the centers had values between 58.9 % and 100 % in 2010, the range was between 76.6 % and 100 % in 2012.

Fig. 4
figure 4

Indicators for guideline compliance for the audit years 2010, 2011, and 2012: quality of the total mesorectal excision specimen (details from the pathology department); lymph-node examination (12 lymph nodes)

Full size image

Pathological examination of at least 12 lymph nodes in patients who did not receive neoadjuvant treatment was achieved by 96.6 % of the sites in 2012. This rate was at the same level as in the two previous years, continuing to meet the target value (≥95 %).

Indicators for specialist expertise

As shown in Fig. 5, standards in the respective centers are already rather high as far as local R0 resections for colon and rectum are concerned. Target values (≥90 %) were met by the majority of the centers and the figure remained at this high level between 2010 and 2012.

Fig. 5
figure 5

Indicators for specialist expertise for the audit years 2010, 2011, and 2012: local R0-resection rectum; local R0-resection colon

Full size image

The percentage of revision operations due to perioperative complications within 30d after elective operations stays nearly stable and did not exceed the target values for rectum and colon (<10 %) over the last 3 years. According to the prespecified target values clinically relevant anastomotic leakage requiring reinterventions following elective procedures should not exceed a figure of 6 % (colon) or 15 % (rectum) following elective procedure. The results for these indicators achieved by the centers are clearly below these target values over the previous 3 years, with almost constant median values, e.g., in audit year 2012 4.4 % and 7.6 %, respectively. The majority of the centers also met the target values (70.9 % and 84.3 %).

The mortality rate following elective procedures in patients with colon and rectal carcinomas is determined by individual factors in a given patient group (including age and comorbidities) as well as by the institution’s procedures for managing complications. The postoperative mortality should not exceed 5 % according to the certification rules. The median in the audit year 2012 was at 2.6 % in the certified centers (Fig. 6).

Fig. 6
figure 6

Indicators for specialist expertise from audit years 2010, 2011, and 2012: revision operations, colon; revision operations, rectum; anastomotic leakage in the colon; anastomotic leakage in the rectum; postoperative mortality within 30 days

Full size image

Discussion

The 2012 benchmarking report of the DKG-certified colorectal cancer centers provides for the first time an analysis of the results for 3 consecutive years The full benchmarking report has been published in the anonymized version presented here on the web pages of the German Cancer Society and OnkoZert [3]. The results of the colorectal centers may not reflect the quality of cancer care in Germany as a whole, because only theses facilities are included who underwent the process of certification and meet therefore the requirements of the certification system. Under the assumption that this represents a positive selection it might be speculated that population-based results would be worse.

In addition to the publicized report, each center has been provided with a set of PowerPoint slides in which its own results are marked. This dual form of reporting meets two essential goals of the certification process: on the one hand, the overall results and thus the quality of care are presented in a transparent way and can be used by the guidelines group or by the certification commissions. On the other hand, individual center get a benchmarking with all of the other centers, allowing for effective quality assurance, self-reflection, and quality improvement. In this context, improvements that are needed can be agreed, and in particular their implementation can be checked in the subsequent audits. In addition, the individual benchmark report provides an indispensable source of information for clarifying internal multidisciplinary processes (e.g., preoperative chemoradiotherapy).

Discussion of data input

The quality of the data entry has been constantly improving in recent years. This is due on the one hand to the centers’ increasing experience with documentation issues, but also to the further development of data management within the certification system. This includes the published hints for interpretation and precise specifications of the indicators, which allow the centers to avoid false interpretations of indicators [4]. The electronic questionnaires introduced in 2011 also minimize incorrect entries and at the same time require statements by the center if pre-set plausibility thresholds are not met [5]. In addition, auditors have become much more confident in handling the questionnaires and are giving more attention to the assessment and discussion of the indicator results before and after the audit. Online training is carried out to familiarize auditors with innovations in the data management system.

However, a potential shortcoming of the data is the fact that only summary data per center, not individual patient data were entered in the database. Risk adjustment of outcome data is therefore impossible. Nevertheless, due to the intensity of the audit including check of individual patient records the validity of the data should be high.

Conditions for data analysis

A total of 230 sites with 19,567 primary cases are included in the 2013 benchmarking report. The mean number of primary cases per site was 87, slightly lower than in the previous years. This value is important when analyzing the indicator results, as it frequently defines the basic overall group — i.e., the denominator for the indicators. However, there are a number of indicators that only take a subset of this basic overall group into account, such as the frequency of adjuvant chemotherapy in patients with UICC stages III colon cancer. The median for the denominator in the overall analysis is 13 (range 2–46), so that a change in the denominator even just by a single patient would have substantial effects on the rate and thus on the extent to which the target value is met. There are also other reasons plausibly explaining failure to reach individual target values, such as a selected group of patients, or patients who decline systemic therapies in particular. For this reason, it is indispensable for the indicator results to be discussed with the specialists during the audits in a structured dialogue, so that any failure to reach the target value can be correctly understood. In the future, it will be possible to record reasons for not carrying out a treatment, such as the patient declining it or when there are contraindications, using the XML OncoBox intestine [4].

Collaboration within the certified network

The indicators in the benchmarking report can be divided into three groups that reflect fundamental aspects of the certification system. The first group includes the areas that reflect interdisciplinary and professional collaboration among partners at a certified center. This principally involves the holding of pretreatment and postoperative multidisciplinary conferences. The postoperative presentation of surgical and endoscopic primary cases has become an established standard for collaboration in almost all of the centers (with 88 % of the centers meeting the target value), and the rate of patients presented also varies only slightly among the certified centers in comparison with other indicators. The situation is slightly different with the preoperative presentation of primary cases and of patients with new recurrences or metachronous metastases. Although the median presentation rate for these indicators is high, there are a number of centers that do not meet the target values, and in addition there are also wide differences among the centers. A precise analysis of the explanations provided by the centers and of the audit reports shows that willingness on the part of individual specialties varies. The centers are aware that there is a clear potential for improvement of internal communications here, and this group of topics is notably often described as a frequently recurring topic of discussion at management conferences. Nevertheless, the figures of primary and secondary resections of liver metastases in the certified centers are well within the expected figures. Furthermore, it should be noted in connection with indicators for the presentation of primary cases at the tumor conference that the figures — particularly for the lower percentile ranges — have clearly increased over 3 years. The differences between the centers are declining and performance is generally improving.

The figures for psycho-oncologic care (with conversations lasting over 25 min) and for contact with the social service are indicators that do not have target values and accordingly show a wide range of implementation among the centers. In the absence of a target value, there is no reason for the auditor to refer to any divergence in the audit report. A classic plan–do–check–act cycle can therefore only be implemented to a limited extent. The median for care by the social service is higher. Analysis of the audit reports shows that social-work staffs are firmly integrated partners in the network of certified centers and that psycho-oncology has been developing in a similar direction in recent years. The denominator for these two indicators refers to primary cases in all tumor entities. The certification commission was aware from the very start that patients with metastases or recurrences — and thus patients with a greater need for care — were not taken into account in this calculation. This deliberate acceptance was due to the initially still very limited experience with documentation at the centers. Whereas the centers had a very good grasp of their primary case numbers from the very start, it was not possible to validly define the overall case numbers — i.e., including patients with recurrences and metastases — during the initial documentation years. As this situation has now fundamentally changed, as described above, the denominators for these two indicators will in the future be expanded to include all patients.

The final indicator in this group is that for participation in clinical trials. Although a majority of the centers (68.4 %) achieved the target value, and the development of the median and upper percentile values have shown a positive trend in recent years towards greater participation in research studies by the centers, when one looks at the audit reports more carefully. This situation is similar for all tumor entities in the certification system, with the exception of malignant skin tumors. All of the certification commissions are in agreement on the importance of active participation by certified centers in clinical trials — partly with the aim of promoting the initiation of clinical trials through this requirement. However, the legal conditions applying in Germany, the absence of clinical trials for the adjuvant situation and the relatively small group of patients who can be included in individual studies in colorectal cancer centers make this requirement difficult to implement. In response to this difficult situation, which affects every type of tumor, the German Cancer Society in collaboration with the certification commissions is developing a joint approach to finding new ways of implementing the research study requirement in the centers.

Guideline-based indicators

The second group of indicators covers the area of guideline-compliant work. The analysis here deals mainly with the indicators for systemic therapies and for pathology, in collaboration with colorectal surgery. The results for the implementation of adjuvant chemotherapy in patients with colon carcinoma and of neoadjuvant radiotherapy or chemoradiotherapy in those with rectal carcinoma have only changed marginally during the 3 years covered, and the target values are not reached by many of the centers. When the centers’ comments are analyzed it becomes clear that the most frequent cause for failure to meet the target value, is patients decline of an offered treatment or that contraindications are present. The centers are therefore able to provide plausible explanations for not reaching individual targets. As mentioned earlier, it will be possible in the future to record reasons for non-implementation using the XML OncoBox intestine. Another limitation is the fact that chemotherapy is frequently carried out in a non hospital-based outpatient setting in Germany. For all of the tumor entities, the sector transition is an interface at which the quality of the data may decline, as the outpatient health-care providers are often not closely integrated into the certified network and treatment data are thus only reported to a limited extent. There is a clear potential for improvements for the majority of the certified centers. The problem of the small denominator, discussed at the beginning of the Discussion section, also needs to be taken into account in this context.

The good results for the indicator “quality of the TME rectal specimen” provide exemplary evidence for the way in which a method and a process can become successfully established in everyday clinical work over the last 3 years. Similarly good results were seen in relation to the indicator for the number of lymph nodes examined. All four indicators mentioned are components of the set of quality indicators described in the current S3 guideline on colorectal carcinoma [6]. In the update to the guideline, ten quality indicators have been derived from the strong recommendations given in the guideline, on the basis of the methodology of the German Guideline Program in Oncology (GGPO), and have been fully transferred to the questionnaire for the colorectal cancer centers [7]. Using quality indicators, the content of the guidelines can be introduced into everyday clinical work and its application can be critically reflected on in the on-site audits. The aim of implementing the quality indicators is always to improve the quality of treatment. A quality indicator must therefore always allow scope for potential improvement in the results that are measured [8–10]. The annual analyses of the certified centers are presented to the guideline groups during the update process, and it is known from experience with the quality indicators for the S3 guideline on breast carcinoma that indicators can be deleted from the indicator set over the course of time once their implementation starts to be carried out on a standard basis [11]. The very good development in the indicators for assessing the TME specimen and the number of lymph nodes examined will lead to these quality indicators being subject to critical examination during the next guideline update, so that a decision can be taken on whether there is still potential for improvement in the everyday clinical practice or whether the indicators should be replaced. Regarding TME quality it may be wise to raise the bar in the sense that either a higher percentage of “good/moderate” quality specimen is defined as target or that only the percentage of “good” (Mercury grade 1) specimen is counted.

Expertise of treatment partners

The third group of indicators covers the expertise and the results of the main treatment partners. This area is heavily influenced by surgery for all of the tumor entities [12]. For all disciplines there are requirements in the catalogue of requirements that reflect the qualification of the medical staff, the technical facilities and the number of treatments that must be carried out within the network. But indicators for quality of radiotherapy or systemic treatment expertise like the number of side effects are only used for individual tumor entities and are not included in the questionnaire for colorectal cancer centers. The reasons for this are firstly, the difficulty of recording side effects for specialist disciplines that treat all tumor entities, such as radiotherapy in particular. Secondly, standardized scores for side effects in systemic therapy and in radiotherapy are not well established among all of the health-care providers. By contrast, complications associated with surgical interventions usually occur in the shorter term and are clearly recordable and evaluable using the hospital information systems.

Analysis of the surgical indicators, such as the number of anastomotic leakage or local R0 resections, recorded separately for rectal and colon carcinoma shows a multifaceted picture. While target values for R0 resection of colon or rectum cancer were met by the vast majority of the centers right from the beginning, the situation for anastomotic leakages is more complex.

With regard to anastomotic leakage, and even more so for the number of revision operations, the target values for all four indicators are reached both in longitudinal and cross-sectional views if the median for the centers analyzed is considered alone. Despite that, many of the centers do not meet the target values. Analysis of the centers’ explanations and the corresponding audit reports shows that these indicators are often discussed in quality circles and at morbidity and mortality conferences. As a result measures for improvements are taken by medical and nursing staff in the surgical department and the operating theatre. These measures cover, for example, the further development of anastomotic techniques, focused training courses, and the development of internal standard operating procedures in the hospital. Analysis of the causes, particularly in connection with the indicators for revision surgery, is often carried out on an interprofessional basis in collaboration with hygiene specialists. In a classic “plan–do–check–act” cycle, this illustrates the improvement potential resulting from the certification. In parallel with these established mechanisms, a joint project involving the German Cancer Society, the German Society for General and Visceral Surgery, and the Working Group of Colorectal cancer Centers is developing a coaching program that will be offered to centers that repeatedly have unusual results for certain indicators mentioned above.

Future developments

The published benchmarking report presents the therapeutic reality of medical care in the certified colorectal cancer centers in Germany. Decisive aspects for improving quality include a structured dialogue conducted with the auditor both inside and outside the network concerning the results of the indicators. In the future, systematic recording via the XML OncoBox [5] of reasons why a certain treatment or procedure was not carried out will provide further support for a detailed analysis.

Adopting the quality indicators from the evidence-based guideline on colorectal carcinoma established an important prerequisite for implementing the content of the guidelines safely in the everyday clinical practice. During update processes, the annual analyses will provide the guideline group with crucial information on the extent to which the guideline recommendations are being implemented, and in the long term — together with analyses of the clinical cancer registries — this will allow the guidelines to be evaluated.