Introduction

Efforts to formally measure the quality of medical care have evolved over the past two decades, spurred on in part by the Institute of Medicine’s call to action, Crossing the Quality Chasm [1]. A substantive aspect of these efforts has been the development of quality indicators or performance measures (PM), which offer organizations and healthcare providers a specific structure by which to measure, evaluate, and improve care. The Agency for Healthcare Research and Quality (AHRQ) defines quality indicators as “standardized, evidence-based measures of healthcare quality that can be used with readily available hospital inpatient administrative data to measure and track clinical performance and outcomes” [2]. The development and use of quality indicators or PMs are intended to promote the delivery of high quality and safe patient care. For the purposes of this document, the terms quality indicators and PM are considered interchangeable, and PMs will be used for clarity.

The delivery of neurocritical care encompasses multiple medical conditions, occurs in a variety of patient locations within the healthcare system, and involves a multitude of providers across different specialties. While PMs exist for some aspects of neurocritical care conditions such as stroke and seizures, there is no formal organized set of PMs specifically designated for neurocritical care. Recognizing this, the Neurocritical Care Society (NCS) convened a writing group in 2016 to develop PMs relevant across the spectrum of neurocritical care and the various inpatient acute hospital settings in which care is delivered (i.e., not limited to dedicated neurocritical care units or specialty-trained neurocritical care providers). The writing group aimed to develop a unified Neurocritical Care Performance Measure Set based on: (1) a systematic review of existing clinical practice guideline (CPG) recommendations that could be developed into new PMs, and (2) the identification and vetting of existing PMs for inclusion in the measure set. This initiative is part of a multifaceted effort to expand quality improvement resources for clinicians caring for neurocritically ill patients and to improve outcomes for these patients. This includes a recent publication on standards for Neurologic Critical Care Units that should be seen as complementary to this PM work, but addresses structural elements of care, whereas PMs address the process and outcomes of care provided [3].

The target audience for this Neurocritical Care Performance Measure Set is practitioners who care for neurocritically ill patients worldwide and the purpose is to improve the care of patients. The intent is to optimize patient-centered outcomes, though the implementation of PMs is often impacted by regulatory and/or financial drivers. Consumers, providers, accountable care organizations and payors are increasingly using PMs to demonstrate and measure the quality of healthcare. In the USA, payors sometimes use PMs to align financial incentives and penalties [4]. To be used in this manner, PMs must be well-developed, vetted, and tested to ensure that they are evidence-based, meaningful, valid, measurable, and reliable. Before these PMs are used in a regulatory setting (especially those that are new), beta testing is necessary. However, this should not delay their implementation as a framework for practitioners to improve the care of their patients.

Methods

Although there is no uniformly accepted process for developing PMs, there are examples from organizations and published medical literature that review and describe the components of reasonable methodologies that could be considered best practices [5,6,7,8]. Because this is the first set of PMs developed principally by the NCS, the first task of the writing group was to define a standard methodology for the selection and development of individual PMs and the collective measure set. The writing group employed a methodology that included eight key steps (Fig. 1). As established by the group a priori, PMs were derived solely from published evidence-based CPG recommendations or existing PMs, as opposed to reports utilizing expert consensus or intuition-based methodologies designed to drive aspects of care. A standardized form was used to document each PM, and includes a PM statement or definition, numerator, denominator, period of assessment, sources of data, rationale, sources of clinical recommendations, methods of reporting/type of score, type of PM, quality strategy domain, challenges to implementation, and analogous PMs endorsed by other organizations. The full PM set of measure specification forms (MSF) is included in the Appendix.

Fig. 1
figure 1

NCS performance measure set development methodology

Full size image

Composition of the Writing Group

An international, multidisciplinary writing group was formed through the NCS Guidelines Committee, and provided oversight for the PM development process. Writing group members were identified according to their expertise in neurocritical care and previous experience developing PM or leading healthcare quality initiatives. The group was diverse, representing multiple geographic areas across the USA and Europe. The writing group included pharmacy and nursing representatives, as well as physician neurointensivists from neurology, anesthesia, and neurosurgery. Work was conducted during regular conference calls and two in-person meetings. All authors disclosed relationships with industry and any other conflict of interest at the outset of the project and any potential conflicts were addressed according to NCS policy.

Scope of PM

For this set of PMs, neurocritical care was defined according to disease process and acuity of care. Thirteen medical conditions were identified from the modules covered in the Emergency Neurological Life Support program (Table 1) and included for consideration in the development of neurocritical care PMs. Topics relevant to multiple neurocritical care disease conditions and represented by existing CPGs developed by NCS were also considered for inclusion (coagulopathy reversal, venous thromboembolism prophylaxis, external ventricular drain (EVD) management, management of devastating brain injury, and multimodality monitoring). In order to focus the scope of this neurocritical care PM Set, general critical care topics such as pain, sedation, and vascular access were not included. Also, scope was limited to include adults (age > 18 years) only.

Table 1 Disease processes considered for performance measure development
Full size table

As the purpose of these PMs is to improve the quality of care for patients with neurocritical care diseases, PMs were specifically designed to be patient-centered, as opposed to focused on medical practitioner specialty or the physical location in which care is provided. This was considered particularly important because different care models in the USA and worldwide may involve different types of providers and physical structures depending on resource allocation and distribution. Thus, these measures do not apply, and are not intended to apply, solely to practitioners who self-identify as neurointensivists or only to patients cared for in a specifically designated neurocritical care unit. Applying these PMs broadly to all patients with the included conditions will necessitate collaborative work within and across hospitals to ensure that patients receive the appropriate quality of care regardless of the nature of the provider or the location of care. Therefore, these measures would be expected to be implemented in neurocritical care units, emergency departments, general intensive care units (ICU), or general hospital wards based on the customs, practices, infrastructure, and resources of the system in which the patient is receiving care. Because of the nature of neurocritical care, these PMs apply only to the inpatient acute care hospitalization related to the primary condition.

Since the target audience for these PMs is providers caring for neurocritically ill patients throughout the world, and in accordance with NCS’s status as an international organization, CPGs and existing PMs from any country were included in the systematic review, provided the publication was in English. As such, the collective measure set reflects a diversity of neurocritical disease conditions from a global perspective. The burden of specific neurocritical care disease conditions varies considerably from country to country, as does quality measurement in healthcare and use of PMs. Consequently, certain PMs may have the potential for greater or lesser impact on quality of care, depending on country and disease condition prevalence.

Results

We conducted a systematic database search in Ovid Medline/PubMed/Cochrane, CINAHL, and EMBASE to identify all CPGs and published PMs related to the neurocritical care management of the conditions included (Table 1). Only CPGs and PMs specifically related to the neurocritical care management of the diseases mentioned were considered for PM development. CPGs and PMs related to general critical care topics (e.g., pain or sedation) were not considered for PM development. Table 3 in appendix provides an overview of search terms used.

The websites of relevant professional societies and regulatory agencies including the Centers for Medicare and Medicaid (CMS), National Quality Forum (NQF), The Joint Commission (TJC), Det Norske Veritas (DNV) Healthcare, and the National Institute for Health and Care Excellence in the United Kingdom were hand-searched to identify existing PMs not published in an indexed database. Documents published in English between 2000 and 2018 were included. In order to be considered as a source document for potential PM development, only CPGs describing a robust consensus methodology used for generating recommendations were included, in accordance with AHRQ guideline criteria. Specifically, recommendations were included if rigorously developed, evidence-based and related to health outcomes, and there was clarity about the source(s) from which the review criteria were derived. In order for existing PMs to be considered for inclusion as a PM in this measure set, they had to include proposed inclusion and exclusion criteria and a measurable numerator and denominator, with or without clear abstraction guidelines.

The search identified 50,257 citations (Fig. 2). Each document underwent title and abstract review by two writing group members to identify CPGs and publications describing PMs. Of the 50,257 citations identified, 398 documents were taken to full text review, which included validation of the document as a CPG or a publication describing a PM. This stage included an analysis of each CPG using a modified Appraisal of Guidelines for Research and Evaluation II tool to ensure that the document met minimum established criteria for CPG quality [9]. This level of review identified 150 documents that underwent data extraction. Each phase of review was completed independently by two writing group members. Conflicts were discussed between the reviewers with a third writing group member adjudicating the conflict, if necessary. The systematic review was conducted using DistillerSR software™.

Fig. 2
figure 2

PRIMSA systematic review diagram

Full size image

Extraction of Recommendations

The writing group employed a rigorous process to extract only the strongest recommendations based on the specific methodology used in the specific CPG. Strong recommendations were prioritized as most guideline methodologies give stronger ranking to recommendations with the highest quality of evidence to support the recommendation. Examples would include a Strong recommendation if the GRADE methodology [10] was used or a Class I recommendation of the American Heart Association/American Stroke Association (AHA/ASA) methodology [11] was used. Lesser recommendations were not considered for development into PMs. The most recent iterations of a guideline were prioritized for extraction. However, all guidelines were reviewed. Each extracted recommendation from the CPGs was collated with other similar recommendations according to disease and topic. To determine which recommendations should be proposed for development into a PM, each recommendation was critically reviewed using the following criteria:

  • Importance of the recommendation to neurocritical care

  • Scientific acceptability and evidence base is well-established

  • Feasibility—the data required for the PM is likely to be obtained at a reasonable cost and during the period allowed for data collection

  • Actionability—the degree to which a practitioner can influence the quality of care being delivered by a health system

  • Denominator—the patient group to whom the PM applies is clinically meaningful

  • Validity and reliability of the recommendation and resulting PM

Each recommendation was evaluated for inclusion according to these criteria. Consequently, there are some aspects of care that are likely reasonable and may represent current best practices, but based on the above criteria were not felt suitable as a formal PM. This should not be interpreted as suggesting that aspects of care that are not formal PMs are inappropriate or should not be performed. Rather, it should be understood that the writing group did not feel that inclusion among the relatively small group of rigorously developed PMs was warranted according to the methodology used.

Development and Review of MSF

Recommendations were developed into PMs using the criteria outlined in the MSF. Fifty-one proposed PMs were presented for discussion at an in-person writing group meeting in June 2018. Over the next 6 months, candidate PMs underwent an iterative process of development, discussion, and revision. The writing group then voted on the PMs using a predefined worksheet with a five-point Likert scale for each of the criteria described above and an additional question regarding overall suitability as a PM. Any PM with a score in any category less than four was reviewed and discussed further by the writing group. A draft set of PMs was presented at the NCS annual meeting in September 2018. Attendees were invited to provide written feedback to the writing group that prompted further review and editing of the PMs.

The revised candidate PMs were then posted for a 30-day general public comment period in December 2018. Relevant organizations and societies were invited via e-mail to review and comment on the candidate PMs. All comments were reviewed by the writing group to determine if changes to the PMs were warranted. The PMs were edited based on this public feedback a final time prior to drafting this manuscript. The final PM Set and accompanying manuscript underwent peer review prior to publication according to NCS policy. The final NCS PM Set and manuscript were approved by the NCS Guidelines Committee and the NCS Board of Directors prior to publication.

Patient Population and Care Period

The patient population is adults (age 18 years or older) with the primary discharge diagnosis of the relevant disease from the title of each specific PM. The care period is the acute hospitalization for diagnosis and management of that condition. This includes emergency department care and management in the ICU and hospital wards, with the specific period of assessment (e.g., entire hospitalization, first 24 h of care) specified in each individual PM. Patients with neurocritical care conditions may receive significant care in other locations such as the pre-hospital or post-acute care rehabilitation setting. However, these care periods were not included because of the focus of this initial PM Set. Likewise, children (age less than 18 years) were excluded as were patients who developed neurocritical conditions subsequent to an admission for another primary disease condition (e.g., in-hospital stroke following admission for myocardial infarction, or status epilepticus occurring after admission for sepsis). This should not be taken to mean that relevant guidelines and treatments do not apply to these patients, but rather that specific decisions that were expected to capture the most relevant scope for these PMs had to be made at the beginning of the process. Hospital admissions with length of stay > 120 days were excluded, as is done in many NQF-endorsed PMs, to avoid double counting patients when generating quarterly reports. These PMs do apply to patients transferred from one acute care setting to another, with the sending and receiving hospital responsible for the appropriate aspects of the PM.

Discharge administrative records should be used when possible to identify eligible patients. In the USA, International Classification of Diseases version 10 codes, standardized disease registries, or surveillance of hospital admission logs may be used for this purpose. In other countries, codes used in national administrative or billing databases relevant to that country, disease registries, or admission surveillance logs are recommended. If none exist, then primary discharge diagnosis from chart review of hospital records should be used.

Brief Summary of the Neurocritical Care Performance Measure Set

Table 2 lists the full Neurocritical Care Performance Measure Set. The set consists of 21 PMs: Six that are similar or the same as stroke measures developed by the AHA/ASA and/or TJC, five that are similar or the same as the American Academy of Neurology (AAN) Inpatient and Emergency Neurology Quality Measurement Set (the development of which NCS was a participant), and ten newly proposed PMs. When including existing PMs, the writing group considered the evidence in support of the PM and whether revisions or adaptations were warranted to improve feasibility or actionability with the desire to minimize suggested changes in an effort to harmonize with prior efforts. Overall, ten PMs were related to stroke (including ischemic stroke, intracerebral hemorrhage, and subarachnoid hemorrhage), two involved neuromuscular diseases (Guillain–Barre Syndrome and myasthenia gravis), three were related to neuroinfectious diseases, two concerned status epilepticus, one with traumatic brain injury, one with global cerebral ischemia after cardiac arrest, and two with processes of neurocritical care delivery relevant to more than one disease condition. Of the 13 topics described in Table 1, PMs were not developed for five diseases (coma, intracranial hypertension, non-traumatic spinal cord compression, traumatic spinal cord injury, and brain death) due to lack of meeting criteria for inclusion.

Table 2 NCS performance measure set
Full size table

The writing group developed a detailed Performance Measurement Specification Form for each PM (see Appendix) that provides a PM statement, numerator, denominator, period of assessment, sources of data, rationale, sources of clinical recommendations, methods of reporting/type of score, type of PM, quality strategy domain, challenges to implementation, and analogous PMs endorsed by other organizations. The Discussion section of this manuscript provides a brief summary of challenges related to the development of the PM Set, addresses concerns brought forth by the writing group and in public comment, and gives recommendations for further testing of the PMs. References for source documents or rationale for each PM are included in the individual MSF’s to allow for ease of review and are also included here [5, 11,12,13,14,15,16,17,18,19,20,21,22,23,24,25,26,27,28,29,30,31,32,33,34,35,36,37,38,39,40,41,42,43,44,45,46,47,48,49,50,51,52,53,54,55,56,57,58,59,60,61,62,63,64,65,66,67,68,69,70,71,72,73,74,75,76,77,78,79,80,81,82,83,84,85,86,87,88,89,90,91,92,93,94,95,96,97,98,99,100,101,102,103,104,105,106,107,108,109].

Discussion

The 21 PMs in the NCS PM Set are the result of a systematic review of existing PMs and CPGs with extraction of the strongest recommendations into PMs. These PMs reflect an 18-month effort to vet the best evidence in neurocritical care and create PMs relevant to patients with neurocritical illness. The responsible development of PMs requires a careful balance: PMs aim to provide a framework to ensure that the best medical evidence is systematically applied in patient care, while also considering the intended and unintended consequences of the proposed PM. This is of particular concern in the USA where healthcare financial reimbursement may be aligned with PMs. PM must be validated in a real-world context prior to alignment with financial or accreditation outcomes. While creating the neurocritical care PM Set, we encountered a number of challenges across several domains including the scope of neurocritical care, establishing measurement criteria in the absence of clear evidence for specific criteria, harmonization with PMs from other organizations, and accepting that not all topics of interest to neurocritical care providers lend themselves to PM development. We believe a review of these challenges is relevant to understanding the current draft neurocritical care PM Set and setting a direction for ongoing PM development and refinement.

Scope Challenges

Ensuring a defined scope of neurocritical care for PM development was a priority for the writing group at the outset and continued to be revisited throughout the development of the neurocritical care PM Set. The perceived range and scope of neurocritical care differ by various providers and organizations; this was particularly evident from feedback during the public comment period. For example, some providers felt that a particular PM should not be included because it did not fall under care delivered within their specialty, such as care most often delivered in an emergency department (e.g., status epilepticus) or by providers that might not identify as neurointensivists (e.g., neurointerventionalists). Others felt that inclusion criteria should be limited to patients cared for in a neurocritical care unit, and therefore patients with an identical condition would be excluded if they were cared for in a general ICU (e.g., traumatic brain injury patient in a surgical ICU). Others suggested that because they did not see many patients with a specific condition (e.g., tuberculous meningitis) in their practice, it was not of sufficient concern to include as a PM. Defining neurocritical care from each of these lenses could result in different priorities in PM development. We chose specifically to define neurocritical care from the perspective of a patient who has a specific disease process, rather than based on the provider or location of care. The writing group maintained throughout the development of the neurocritical care PM Set that neurocritical care was defined by the acuity of illness in each of the diseases outlined in Table 1. Therefore, the PMs in our PM Set should be applicable to patients in multiple care settings, including the emergency department, ICU, or even acute care units in some instances. For example, PMs related to status epilepticus management, bacterial meningitis and acute stroke may be more likely to be measured in the emergency department than the ICU. However, due to the critical acuity of the neurologic illness, we included these measures as a part of the neurocritical care PM Set. Similarly, targeted temperature management (TTM) after cardiac arrest may be managed by different provider teams and in different ICUs depending on the organizational structure of different hospitals. However, hypoxic-ischemic encephalopathy after cardiac arrest is a critical neurologic illness and, therefore, included in the neurocritical care PM Set. Finally, some programs engage their neurocritical care providers in the acute management of stroke and other illnesses in the emergency department, telemedicine programs, or consult teams. By defining the scope of neurocritical care by the nature and acuity of illness, the focus remains on the patient and enables the highest number of patients worldwide to be helped by these PMs.

Measurement Criteria

We aimed to be evidence-based in all aspects of PM development. This approach directed our methodology in the systematic review and extraction of recommendations from CPGs. However, this was challenging at times during the development of the MSF’s. Performance MSF requires clear measurement criteria (e.g., timeframes and frequency of assessment). This proved difficult in several instances where evidence clearly supported a clinical management approach, but extracted CPGs and their source documents did not specify certain information required to create a rigorous PM. For example, urgent treatment of status epilepticus resulting in prompt seizure cessation is recommended. However, the administration of benzodiazepines in status epilepticus and the treatment of ongoing status epilepticus with anticonvulsant medications can only be a measurable PM if a timeframe for administration is specified, and such a precise timeframe is not clearly specified in strong recommendations from CPGs. After much discussion, the writing group decided to adopt the American Epilepsy Society timeline that accompanies their status epilepticus CPG [44].

Conversely, the writing group felt that existing evidence did not lend itself to more specific measurement criteria for when and how often screening for vasospasm should occur in aneurysmal subarachnoid hemorrhage (aSAH). Even so, it was felt that having a PM for vasospasm (and delayed cerebral ischemia) screening was merited by existing CPG recommendations and that future efforts should focus on better refining the time window for screening. Likewise, numerous strong recommendations from several CPGs support the use of TTM in comatose patients after cardiac arrest. However, specific metrics for effectiveness of the intervention at achieving and maintaining a temperature target are not sufficiently evidence-based to become part of a PM. Finally, while the randomized trials assessing decompressive craniectomy in large hemispheric infarction demonstrate improved patient outcome and thirteen separate strong CPG recommendations are provided as sources in the MSF, it is recognized that some patients or their families would choose not to have this procedure even if discussed in a manner that did not reflect a bias of the practitioner providing the information. Thus, compliance with this measure can be met as either performance of decompressive craniectomy or clear documentation in the medical record of why the procedure was not performed, which should include documentation of patient or family preferences regarding medical decision-making. As more providers become aware of and begin to use PMs to improve their practice, this tension between implementing a CPG-based recommendation and the specific criteria by which compliance is measured will become important when considering the spirit and intent of a specific PM.

Performance Measure Harmonization

The writing group felt that it was both important and appropriate to include existing PMs put forth by other organizations, if they met the predefined methodological criteria, rather than assuming that their presence in other documents was sufficient. The purpose of this was to provide a comprehensive PM measure set across the a priori defined scope of neurocritical care even if these aspects had been considered separately in another context. Overall harmonization was prioritized unless evidence was identified that changes to the PM were warranted. For example, the measure regarding nimodipine administration in aSAH advocates for a shorter timeframe for administration (within 24 h of hospital arrival) than the clinical trials from which the CPG source recommendations were developed. In this case, we noted this discrepancy and opted to endorse the measure as written to harmonize with existing measures that are already in use in programs across the USA.

Similarly, we endorsed existing PMs related to severity scoring in ischemic stroke, intracerebral hemorrhage (ICH) and aSAH by combining these into one PM. However, we concur with public comments that the timeframes specified in the PMs are subjective and that beta testing should be part of next steps. The writing group chose to endorse the existing AHA PM measuring the rate of symptomatic intracranial hemorrhage after ischemic stroke. After an exhaustive search of published CPGs and the existing TJC PM specifying this, we found no CPG recommendations from which this measure might have been extracted. However, the PM evaluates patient outcomes rather than process of care, and as an existing PM that is actively used as part of stroke center certification, at least in the USA, it may not be subject to CPG development. Therefore, we elected to endorse the measure, especially in the absence of information that existing use of this PM is inappropriate.

Worldwide Use of the Neurocritical Care PM Set

The intent was to develop a PM Set that could be utilized worldwide and the literature review and included CPG source documents reflect this. However, it is recognized that patient populations, resources, availability of specific medications and procedures, and custom and practice may vary and potentially influence the ability to precisely define measurement criteria for a PM even if the spirit of a specific CPG-based recommendation is upheld. Tuberculous meningitis is more prevalent in countries outside the USA, but is associated with significant morbidity and mortality. Given the strong CPG recommendation for the administration of corticosteroids in this patient population, a PM addressing the use of dexamethasone in tuberculous meningitis was warranted based on our PM development methodology.

The PM involving stroke severity score assessment specifies the use of the Hunt and Hess scale so as to harmonize with the existing TJC PM in place in the USA. However, it is recognized that the World Federation of Neurosurgical Societies (WFNS) scale may be more commonly used around the world. Beta testing is appropriate to assess whether this PM should be modified to include the WFNS scale in order to ensure worldwide implementation. Because of the lack of observational data regarding current practices, the prevalence of corticosteroid use throughout the world in ICH and traumatic brain injury is unclear. PMs discouraging the use of corticosteroids in these diseases were developed based on strong CPG recommendations even though actionability may be limited if compliance is already very high. A similar situation exists for the administration of intravenous acyclovir for the treatment of herpes simplex virus encephalitis as few other pharmacologic treatment options are available. Field testing will be useful to determine the impact of these and other PMs on treatment practices.

Excluded Recommendations

Finally, the neurocritical care PM Set reflects evidence-based PMs that were determined to be feasible, actionable and valid. There are a number of additional PMs published by other organizations that were not included in this PM Set. When reviewed, they were either not supported by strong CPG recommendations, or were excluded because they were assessed to be less feasible, less actionable, or lacking validity. There are also many other recommendations published in the 150 documents reviewed that were not developed into PMs because they either did not meet the minimum strength of recommendation or were not feasible, actionable, or valid after further consideration. For example, we extracted multiple strong recommendations for early rehabilitation after stroke. Given the recent support for early mobilization in the ICU, a PM supporting early rehabilitation and mobilization in the ICU was strongly considered. After discussion, the group felt that a PM could not move forward given the mixed outcomes in studies evaluating early mobilization after stroke and lack of randomized-controlled trials or strong recommendations supporting the practice in other populations. Other topics that the writing group considered were brain death determination, reversal of direct oral anticoagulant medications, screening for blunt cerebrovascular injury after trauma, and advanced care planning including palliative care, but none had strong recommendations from current CPGs that would allow PM development. This approach emphasizes the view of the writing group that PMs should generally not be “aspirational” or intended to create new approaches to care, but rather should be achievable and expected given current evidence-based care.

Future Considerations

Feasibility testing, often referred to as beta testing, is a key step in the process of PM development. Feasibility testing further evaluates the feasibility, actionability, validity, and reliability of proposed PMs through field testing with participating organizations. This process is critical to the further development of a data extraction algorithm, identifying a concurrent or retrospective process for data collection and identifying patients for data extraction, often through diagnosis-related groups. All proposed PMs in the neurocritical care PM Set, especially the ten newly proposed PMs, should undergo feasibility testing prior to further action. Additionally, members are invited to review these PMs within their own organization and report on the feasibility of data collection.

As with any new PMs, documentation at the patient level may need to be expanded or templated to enhance communication and ensure accurate data collection. We expect a number of measures may require expanded documentation, including the EVD insertion bundle, decompressive craniectomy for large hemispheric infarction, and avoidance of steroids in ICH. We believe this documentation will enhance communication at the bedside surrounding key clinical practice issues and facilitate measurement of the proposed PMs.

After reasonable feasibility testing, PMs may be considered for regulatory endorsement. Regulatory endorsement may include several programs through CMS, NQF, or other organizations that certify programs specific to neurosciences such as TJC, DNV, or the American College of Surgeons. Feasibility testing and a future course for regulatory endorsement will be part of the next steps coordinated by the NCS Quality Committee. Feasibility testing may be considered in conjunction with other partnering organizations interested in improving quality for neuroscience patients.

PM development should be ongoing and iterative in nature. In our effort to be rigorous and evidence-based, we may have excluded recommendations that would yield reasonable PMs. Further NCS PM writing groups may consider broadening the minimum criteria for strength of recommendation. As CPGs are developed, we suggest that all strong recommendations continue to be considered for PM development. Finally, most of these PMs focus on process, and this is a recognized limitation of many current PM. PM that evaluate patient outcome directly should be sought and developed.

Conclusions

Neurocritical care has advanced to a mature field in which CPGs exist for many aspects of care. PM can be a valuable tool in measuring quality of care and improving that care. This neurocritical care PM Set represents the first organized effort to develop formal PMs that extend across the scope of neurocritical care delivery for adults. The fact that this PM Set includes many new PMs and that half are identical or analogous to existing PMs emphasizes that collaboration across organizations may yield synergy. Next steps include field testing of new and existing PMs in order to refine inclusion and measurement criteria. In so doing, we must remind ourselves that the purpose of these PMs, and hopefully all PMs, is to improve patient care.