Introduction

Traumatic brain injury (TBI) is the most important cause of disability for people younger than 45 years of age and a leading cause of acquired disability and death in children, affecting 475,000 US children annually [1, 2]. An international TBI workforce stressed the importance of comparative effectiveness research as a strategy to improve outcomes after TBI [2]. Such research can utilize a number of study designs—including randomized trials, observational studies and others—to determine the effectiveness of treatments on relevant outcomes across a population to ultimately improve health care delivery [3]. Since large, definitive studies to determine best practices must include a number of institutions, understanding of the characteristics of clinical sites can be an important factor in judging how treatments are applied to subjects and how the treatments are related to the observed outcomes.

The Approaches and Decisions for Acute Pediatric TBI (ADAPT) Trial is an observational, cohort study of 1000 children being conducted to determine the relative effectiveness of different approaches used in three aspects of care for children with severe TBI (www.ADAPTTrial.org). The study includes 50 clinical sites from 8 different countries (US, UK, Spain, the Netherlands, New Zealand, Australia, South Africa, and India). The aspects of care under investigation are (i) first-line intracranial pressure management strategies (cerebrospinal fluid diversion and hyperosmolar therapies), (ii) strategies to mitigate secondary insults (hyperventilation without intracranial hypertension and hypoxia detection/treatment using interstitial brain oxygen monitoring [PbO2]), and (iii) strategies to provide metabolic support (nutritional support and glucose management). As an observational cohort study using propensity scoring to control for covariates between subjects, ADAPT requires that an assessment of clinical sites be performed to account for site differences in the statistical analysis. A recent review of nearly 10,000 adult TBI patients showed that clinical site was the most significant predictor of patient outcomes [4], forming a significant part of the rationale for the current emphasis on comparative effectiveness research in TBI.

Since clinical center characteristics can affect overall outcomes for patients with severe TBI, and these differences may have implications for overall clinical care as well as the conduct of our research study, we set out to design a survey to characterize differences in the structure, processes, and cultures of clinical sites by identifying organizational factors potentially associated with quality of care delivery at each center [5–8]. We developed an organizational assessment survey that queried site personnel on patient population; clinical staffing; and institutional structure, process, and culture characteristics. The ultimate goal of this survey tool is to characterize center differences that may be relevant in contemporary clinical care of all children with severe TBI, as well as providing information that will be used in the final statistical analysis of between-site variations in the three aspects of care studied.

Materials and Methods

Overall Study Design

The Data Coordinating Center of the ADAPT Trial at the University of Pittsburgh and all clinical sites obtained IRB approval to perform the study, assess outcomes, and provide data relevant to this manuscript. Clinical site participation in this study came from a variety of sources including the Pediatric Neurocritical Care Research Group (PNCRG) membership, the Collaborative Pediatric Critical Care Research Network (CPCCRN), the Paediatric Intensive Care Society (PICS), and the European Society of Paediatric and Neonatal Intensive Care (ESPNIC). Since the purpose of the ADAPT Trial is to determine optimal management strategies for the care of children with severe TBI, the leadership of ADAPT chose to limit site participation to those with the sufficient resources to choose between the various strategies. For example, PbO2 monitoring can be cost prohibitive in many centers around the world—costing several thousand dollars for the monitor as well as the technology to use the device. Therefore, clinical sites who simply could not consider such contemporary therapeutic modalities were not approached to participate in the ADAPT Trial. As a result, the clinical sites were located in relatively large clinical centers with significant resources at their disposal.

Survey Development and Analysis

To better understand the organizational structure, process, and culture of the participating clinical sites, we created an organizational assessment survey. This survey focused on six domains: (i) hospital characteristics, (ii) pediatric intensive care unit (PICU) factors, (iii) medical staff information, (iv) elements related to quality of care, (v) medication safety considerations, and (vi) the culture of safety regarding teamwork and communication (See Online Appendix 1). This survey included elements from previously published tools, such as the Safety Attitude Questionnaire, a scoring system for ICU quality of care developed by Najjar-Pellet and colleagues [9], and the Pediatric HCAHPS™ [10] (Hospital Consumer Assessment of Healthcare Providers), and modified them to shorten the time required for completion [9, 11]. The survey was specifically designed to facilitate comparisons between hospitals in different countries.

The survey, containing a total of forty items, was developed in consultation with experts at the University of Utah and the University of Pittsburgh. After the initial survey was developed, nursing and medical directors of the pediatric intensive care unit (PICU) from Primary Children’s Hospital (University of Utah) were provided an opportunity to give feedback on the survey as their clinical site was not participating in the ADAPT Trial at that time. Alterations of the survey with respect to wording and question structure were incorporated, but no questions were removed during this revision procedure.

The principal investigators of ADAPT sites that enrolled subjects within the first year of the study were asked to complete the ADAPT Organizational Assessment to the best of their ability. They were encouraged to use institutional personnel, particularly the medical and nursing directors of their institution, as a resource to provide the most accurate information available. Respondents were instructed to leave blank those items that were not applicable to their clinical site. In particular, data regarding Pediatric HCAHPS™ were expected from clinical sites within the United States only. All analyses were conducted using IBM SPSS Premium 20 (SPSS, IBM, Armonk, NY) or Stata 11.1 (StataCorp, College Station, TX). Summary statistics are provided with median and interquartile ranges.

Results

Structure: Hospital Characteristics, ICU Factors and Medical Staff Information

Forty-three clinical sites enrolled subjects in the ADAPT Trial within the first year, and therefore, survey information from these clinical sites represents the data for this manuscript. Of the clinical sites, 32 are US-based and 11 are international (See Table 1). Of these sites, 60.5 % (n = 26) are free-standing children’s hospitals, all 43 clinical sites have a University affiliation, and 37 (86 %) have “closed” units where critical care services are provided by intensivists for each child.

Table 1 Clinical sites participating in the ADAPT Organizational Assessment
Full size table

Site characteristics are shown in Table 2. The size of the pediatric intensive care units (PICUs) varied widely, with a range of 8–55 beds. Correspondingly, the number of admissions in the previous year ranged from 537–2623 patients. PICU attending physician and neurosurgeon staffing was also quite variable. Nursing characteristics were less variable. For example, nursing staff experience was predominantly modest, and most units had either a 1:1 or a 1:2 nursing-to-patient ratio. The majority of PICUs within the US had achieved “magnet status”—an accreditation by the American Nurses Credentialing Center that recognizes organizations for quality patient care, nursing excellence, and innovations in professional nursing practice. Thirty-four (79 %) of the sites had clinical pharmacists participating in daily PICU rounds.

Table 2 PICU characteristics
Full size table

Process: Quality of Care and Medication Safety Considerations

Quality of care and achievement of care goals may be measured with scoring systems. Nearly all clinical sites utilized scoring systems to monitor levels of sedation, pain, and withdrawal in their patients. However, a variety of scoring systems were used, such as the State Behavioral Scale (SBS), Richmond Agitation Sedation Scale (RASS), and the Glasgow Coma Scale (GCS) score, with GCS being the most common (58 %). The reported frequency of sedation assessment varied widely, from hourly to every 12 h; more than 50 % of sites reported that usual practices included performing assessments of sedation level every 2–4 h, with more frequent assessments of infants. Only four sites assessed for delirium as part of routine nursing practice.

Data pertaining to quality improvement practices are summarized in Table 3. A significant majority of clinical sites have implemented care bundles to prevent ventilator-associated pneumonia (VAP) and central line-associated bloodstream infections (CLABSI) and reported low rates of these complications. Importantly, the range for hand-washing compliance was high overall but ranged from 35 to >95 % in this self-reported survey. Fewer than 50 % of clinical sites reported that an explicit protocol for caring for children with severe TBI in the PICU was utilized (n = 20, 46 %).

Table 3 Quality improvement data
Full size table

Regarding five common interventions that address medication safety (computerized physician order entry, standard order sets, electronic medical record, RN bar coding of medications, and “smart” IV infusion pumps); 63 % (n = 35) of the centers had implemented all five interventions. Another 20 % (n = 8) implemented four interventions, and 13 % (n = 5) have implemented a single intervention listed above.

Culture of Safety Regarding Teamwork and Communication

For issues regarding the culture of US hospitals within the clinical sites, 28 % (n = 12) reported the results of their Pediatric HCAHPS™ scores on two items. For those reporting, the median hospital score for the previous year was 80.5 (IQR 18.0, 86.8, n = 12) and a median score for physician courtesy and respect reported toward families was 84.0 (IQR 35.0, 90.0, n = 13), indicating good to high satisfaction of patient experiences. A large majority of sites (79 %, n = 34) reported an existing process to rapidly disclose medical errors to families as well as implementing routine leadership meetings to identify potential problems (70 %, n = 30). Teamwork between services was relatively strong, with 27.9 % (n = 12) and 53.5 % (n = 23) of sites “strongly agreeing” or “agreeing,” respectively, that Neurosurgery and ICU teams work well together as a coordinated team. Many sites (63 %. n = 27) provided high fidelity (simulation) communication training for their nurses and physicians.

Discussion

To our knowledge, ours is the first attempt to describe site-specific characteristics that are likely to play an important role in differences found in patient outcomes between pediatric trauma centers. In this descriptive survey, we found a wide variety of differences between sites particularly in terms of structure (patient volume, medical staffing), process (types of safety and quality of care measures implemented), and culture within this international consortium of pediatric trauma centers. These differences illustrate intrinsic differences between sites, representing countries with individualized health care systems and unique challenges, which may begin to explain why interventional studies have failed to prove efficacy of experimental therapies. Ultimately, understanding these differences is a potentially important factor for the future analysis of data obtained from the ADAPT trial and will be important for determining best practices for children with severe TBI.

The importance of clinical site characteristics on outcomes after TBI has a reasonably long history but is relatively under-studied. More than a decade ago, Clifton and colleagues failed to demonstrate a beneficial effect of therapeutic hypothermia in a randomized, controlled trial including four clinical sites. In a secondary analysis, significant inter-center variability in clinical practice related to fluid management was observed. This variation—whereby a single clinical center used increased amount of early diuretics and caused some degree of hypovolemia in the experimental group—has been postulated to contribute to the failure of the trial to prove its primary hypothesis [12, 13].

More recently, Lingsma and colleagues performed a systematic analysis on the effect of clinical centers on outcomes after TBI [4]. The IMPACT study combined 13 clinical studies—including interventional and observational studies—to form a database of 9578 subjects enrolled at 265 clinical centers. After controlling for various patient characteristics and accounting for random chance, they found a significant inter-center effect on outcomes. They determined sites with the highest and lowest rates of favorable outcomes—finding that sites within Europe had greater degrees of variation than in the US. This larger amount of variability has spurred the European Commission, the US National Institute of Health, and other funding agencies to establish the International Initiative for Traumatic Brain Injury Research (InTBIR) to advance TBI research, treatment, and care.

Prior to the inception of ADAPT, we explored some aspects of center variability in pediatric TBI [14]. Specifically, we surveyed 32 clinical sites regarding the institutional goals of intracranial pressure management, mitigation of secondary injuries, and metabolic support as preliminary information for the design of the ADAPT Trial. We found that the overall goals of care varied dramatically between—and even within—clinical sites. This study builds on our previous efforts by surveying and describing key aspects of institutional differences between the clinical sites. We believe it is likely that factors measured within this manuscript—ICU size, number of admissions, implementation of clinical care bundles, and any number of other variables—will play a role in patient outcomes observed in children within the ADAPT Trial. Moreover, we believe that this work represents the first comprehensive attempt to quantify institutional characteristics in major trauma centers for children that may have implications for system-wide evaluation of trauma centers within and between countries.

The Institute of Medicine defines “quality of care” within six dimensions—safe, effective, patient-centered, timely, efficient, and equitable [15]. An approach to study quality of care in hospital units is via a global assessment of culture with a focus on the human resources, organizational structure, patient care management, culture of safety, and consistency of approach toward specific issues (e.g., infection control, medication safety, or pain management) [7, 11, 16–18]. Therefore, we conducted a global assessment of structure, process, and culture, with specific domains potentially relevant for pediatric TBI. Using similar methodologies as ours, others have demonstrated that institutional characteristics are associated with important clinical outcomes. As an example, the Extended Prevalence of Infection in Intensive Care (EPIC) study characterized the impact of ICU organizational factors on mortality and other outcomes in a large cohort of ICU patients from 75 countries, representing seven different global geographic regions. The authors identified a significant relationship with particular organizational factors after adjusting for confounding, specifically a nurse-to-patient ratio greater than 1:1.5, admission to medical or mixed medical/surgical ICU (vs. surgical ICU), and a trend toward the presence of an in-house intensivist 24 h per day [8]. In our survey, we found that the most common nursing staffing ratios ranged between 1:1 and 1:2. It is possible that the ADAPT Trial could confirm that this factor is associated with outcomes in the pediatric trauma population. Moreover, given the large disparity in sizes of the clinical sites and other variables within our survey, it is likely that other factors associated with outcomes will also be determined in the analysis of the ADAPT Trial.

There are a number of limitations to our study. Primarily, the data gathered from the clinical sites are self-reported and may be subject to bias. Much of the information gathered consists of simple facts (number of ICU beds/physician and nursing staffing) that are available routinely within the institutions. However, some information, such as compliance with care bundles and other variables, is more difficult to measure and report. We endeavored to control for this by requesting that the leadership of the institutions be an integral part of the data collection process. Nevertheless, it is possible that difficulties inherent to measuring some variables introduced bias into the data. In addition, nearly 10 % of the participating sites left many of the survey items unanswered. Another limitation of our data may be our choice of variables selected to differentiate the institutions. In an effort to mitigate any bias toward a particular aspect of institutional characteristics, we collaborated with experts in quality improvement and in pediatric TBI to select the variables queried in the survey. Moreover, we chose to include nationally based standards, such as the Peds HCAHPS™ data, in an effort to gather standardized information that is used in other settings. Finally, we believe our survey has significant face validity, as evidenced by the review at the University of Utah that was performed prior to its implementation. A significant strength of our survey is its prospective nature. We believe that incorporating the results of the survey into our future analysis of the ADAPT data will decrease the bias inherent in a retrospective review of potential site differences. In addition, the results of this survey provide real-time information about site culture that would be difficult, if not impossible, to accurately collect retrospectively.

In summary, we report results of a unique survey that was designed to differentiate clinical centers in the US and Europe caring for children with severe TBI. The information garnered from this survey illustrates differences that already exist between large institutions and we believe will be useful in the statistical analysis of our observational cohort study. Furthermore, we believe that others performing such observational studies should consider methodologies such as ours to control for center differences in studies regarding TBI or other critical illnesses.