Introduction

Cerebral vasospasm is a common and feared complication of SAH. Up to 70% of patients have radiographic evidence of vasospasm, whereas 30% develop symptomatic vasospasm [1, 2]. It is associated with delayed cerebral ischemia and cerebral infarction [3,4,5]. A third of patients with vasospasm either die or become disabled, and it is thus a major cause of death and poor outcome [6, 7].

Cerebral angiography (CA) has been the gold standard for the diagnosis of cerebral vasospasm [8]. Other modalities used for diagnosis include computed tomography angiography (CTA), CT perfusion (CTP), and transcranial Doppler (TCD). CA, CTA, and CTP are invasive, require contrast administration (nephrotoxicity), entail radiation exposure, and require that a critically ill patient be moved to the CT scanner or the angiography suite. [8] Therefore, these modalities are not generally ordered in the absence of symptoms, potentially missing silent vasospasm. Rather, these are triggered for confirmation of vasospasm when the patient exhibits signs of symptomatic vasospasm or sometimes routinely performed on ~ day 7 from the onset of symptoms. Thus, these techniques are neither designed nor deployed for detection of subclinical vasospasm. Additionally, dynamic surveillance of vasospasm is not possible with these angiographic modalities given lack of daily (or more frequent) monitoring. CA, CTA, and CTP are “snapshot studies” as they capture pictures of the cerebral circulation at a given point in time [9].

TCD, in contrast, is portable, available at the bedside, safe (no radiation or contrast), repeatable, and allows noninvasive diagnosis of vasospasm [10,11,12,13,14]. Like CA/CTA/CTP, TCD is also a point measurement but unlike these, TCD has monitoring capabilities that allow it to be repeated more frequently to provide hemodynamic data on a timeline for closer surveillance of vasospasm. We have previously reported pooled weighted estimates of sensitivity (SN), specificity (SP), positive predictive value (PPV) and negative predictive value (NPV) for prediction of delayed cerebral ischemia (DCI) based on TCD evidence of vasospasm in a previous meta-analysis [15]. Here, we present a systematic review and random-effects meta-analysis of prediction of DCI with cerebral angiography, reconciling its accuracy against data previously collated for TCD. We also report quality of evidence for the value of TCD and cerebral angiography in subarachnoid hemorrhage (SAH) based on pooled data from our meta-analyses.

Methods

We followed the PRISMA guideline for the meta-analysis [16].

Search Strategy and Study Selection

A comprehensive literature search of three databases, PUBMED, the Cochrane database, and Clinicaltrials.gov, was performed through May 2017, using the combination of search terms provided in the supplement. Articles were not excluded on the basis of year or language of publication. All study designs were allowed, including randomized trials, clinical trials, and observational studies, both prospective and retrospective designs. Observational studies that included at least 10 consecutive patients were eligible. Studies were considered for inclusion if they had used catheter angiography to diagnose vasospasm and reported DCI as measured with CT and/or magnetic resonance imaging (MRI) in addition to clinical assessment. All follow-up durations for assessment of DCI on neuroimaging studies were allowed. Letters, case reports, commentaries, review articles, meta-analyses, abstract-only publications, and studies on animals or children were excluded. Two independent authors (OMD and CCC) performed literature search and study selection. Titles and abstracts were first screened for relevance and bibliographies of seminal articles and reviews manually searched for eligible publications. Full-text reviews were performed when the initial screening was indecisive about inclusion. Studies that provided data to permit computation of SN, SP, PPV, and NPV were included. If more than one study provided data on the same set of patients, then the study that provided most complete data was included and the rest excluded. Conflicts were resolved with discussion and feedback from one or more senior authors (GK, COC, AVA).

Data Abstraction and Study Quality

Two independent authors (OMD, CCC) extracted and tabulated data. Data regarding the author, year of publication, modality used for diagnosis of vasospasm, true positives (angiographic spasm + and DCI +), false positives (angiographic spasm + , DCI −), true negatives (angiographic spasm-, DCI-), false negative (angiographic spasm −, DCI +), and study design (prospective, retrospective, RCT) were abstracted. Raw numbers were extracted and were used to calculate SN, SP, PPV, and NPV for each study. Metrics relating to study design and study quality were also retrieved and charted. Study quality assessment was done using the Cochrane bias assessment tool [17].

Statistical Analysis

Data Synthesis

SN, SP, PPV, and NPV were transformed to the log scale (log OR) and standard errors (SEs) calculated from the log-transformed confidence intervals. The log ORs and SEs were transformed back to ORs for synthesis. Data synthesis was performed using the DerSimonian and Laird random-effects model [18]. Forest plots were generated and results presented as ORs and 95% confidence intervals.

Heterogeneity

Heterogeneity was calculated using the Cochran Q test (χ2 heterogeneity test), and a P < 0.5 (concurrent to Q > df) was taken as the presence of statistically significant heterogeneity. The equation used to define heterogeneity was I2 = (Q − fd) × 100/Q, where fd = i − 1, Q = Cochran Q, fd = degrees of freedom, and i = number of studies) [19]. The between-study variance was expressed as τ2. Heterogeneity was quantified using I2 and regarded as significant when I2 > 50%.

Publication Bias

We evaluated publication bias with the Begg’s and Egger’s tests (significant P < 0.05) [20, 21]. Funnel plots were rendered for visual assessment of publication bias.

Subgroup and Sensitivity Analyses

Pooled accuracy estimates in randomized versus nonrandomized studies were assessed in subgroup syntheses. The degree to which each trial impacted the result of a given synthesis was evaluated with an exclusion sensitivity analysis. This was performed by individually excluding each study at a time and computing the new, combined value at each exclusionary step. [22]

Test Accuracy and Quality of Evidence

We analyzed test accuracy and quality of evidence for cerebral angiography and TCD for prediction of DCI. These estimates were generated using the GRADEpro GDT: GRADEpro Guideline Development Tool [Software]. McMaster University, 2015 (developed by Evidence Prime, Inc.). Available from gradepro.org.

Bax L: MIX 2.0–Pro, version 2.0.1.4 (BiostatXL) was used for statistical analysis.

Results

PRISMA flowchart depicts study selection and literature search in Fig. 1. Characteristics of included studies are provided in Table 1. PRISMA checklist and study quality are available in the supplement.

Fig. 1
figure 1

PRISMA flowchart of literature search and study selection

Full size image
Table 1 Study characteristics
Full size table

A total of 15 studies [23,24,25,26,27,28,29,30,31,32,33,34,35,36,37] (5463 patients) were selected for inclusion in the meta-analysis. Eleven studies provided data for synthesis of sensitivity (4640 patients). All 15 provided data for synthesis of specificity and PPV. Ten studies provided data for synthesis of NPV (1899 patients). No significant publication bias was found except in the synthesis of specificity. Significant heterogeneity was found in all syntheses (I2 > 50%). Results of primary syntheses and subgroup analysis are provided in Table 2. Forest plots of data syntheses and funnel plots for appraisal of publication bias are provided in Fig. 2.

Table 2 Pooled weighted estimates from primary and subgroup syntheses
Full size table
Fig. 2
figure 2

Forest plots and funnel plots for the syntheses of SN (a, b), SP (c, d), PPV (e, f), and NPV (g, h)

Full size image

Subgroup and Sensitivity Analysis

Confidence intervals for the pooled sensitivity were wide in the subgroup of randomized studies indicating that the small number of studies was the likely source of lack of precision (Table 2). The included randomized trials assessed various treatment approaches which were another source of heterogeneity. Syntheses remained robust to exclusion sensitivity assessment. Sensitivity exclusion plots are available in the supplement.

Quality of Evidence and Test Accuracy for Cerebral Angiography and TCD for Prediction of DCI

In Tables 3 and 4, we provide quality of evidence for angiography and TCD with regard to prediction of DCI [38]. Test accuracy estimates are “moderate” for TCD and “low” for angiography based on the pooled data from our meta-analyses.

Table 3 Should angiography be used to diagnose vasospasm for prediction of DCI in SAH?
Full size table
Table 4 Should TCD be used to diagnose vasospasm for prediction of DCI in SAH?
Full size table

Discussion

Our systematic review and meta-analysis showed that cerebral angiography has low accuracy in the prediction of DCI, with SN 57%, SP 68%, PPV 32%, and NPV 90%. We have previously reported pooled accuracy data for the prediction of DCI based on TCD evidence of vasospasm [15]. We report here that test accuracy estimates are low for cerebral angiography and moderate for TCD.

Prediction of eventual DCI is a more meaningful measure, than prediction of angiographic vasospasm, of the utility of these diagnostic studies as it informs an actual clinical outcome and also allows comparison between cerebral angiography and TCD. It is counterintuitive to test the accuracy of TCD, a dynamic and repeatable modality, against cerebral angiography—a snapshot study, for predicting vasospasm—a dynamic phenomenon that can change over a short period of time. A strong correlation for vasospasm detection between TCD and angiography is therefore unlikely, as even a brief lag in time between the two can alter their concordance.

For prediction of DCI, our study shows that pooled weighted estimates for sensitivity, specificity, PPV, and NPV are inferior for cerebral angiography as compared to TCD (Table 5). Caution is advised as these are indirect comparisons and the definition of DCI, as with our previous meta-analysis, was heterogeneous across the studies included. Most studies defined DCI as infarction on follow-up neuroimaging but few included clinical diagnosis of cerebral infarction (persistent new neurological deficit) as evidence of DCI. The reason for this heterogeneity is that studies did not consistently provide complete data for either outcome individually. Moreover, there was significant statistical heterogeneity in these syntheses which is not unexpected given the variety of management paradigms, protocols, and variation in practice that exists in the management of SAH. We explored heterogeneity with subgroup analysis which indicated that study type (randomized vs. nonrandomized) was a source of heterogeneity. The percent cutoffs for luminal reduction used for diagnosis and grading of vasospasm on CA varied across the studies (Table 1). The angiographic assessment is subjective and rests on visual appraisal of the vessel diameter by the interpreting neuroradiologist. This variability introduces heterogeneity in the synthesis. Angiographic diagnosis and grading of vasospasm contrast with the assessment of vasospasm on TCD, which uses objective mean flow velocity (MFV) cutoffs and intracranial-to-extracranial MFV ratios for diagnosing and grading vasospasm. That said, heterogeneity also exists in the diagnostic cutoffs used for detection of vasospasm on TCD. Several other sources of heterogeneity were identified, such as variety of study designs, variable timing of follow-up neuroimaging to diagnose cerebral infarction, inconsistency in the definition of DCI, year of publication, variety of approaches applied in the treatment of vasospasm, and variety of protocols for triggering a diagnostic study for diagnosing vasospasm, etc.

Table 5 Prediction of DCI with TCD and CA
Full size table

Multiple vasospasm-independent causes of DCI have been proposed and investigated such as cortical spreading ischemia, microcirculatory constriction, and microthrombosis [39]. Development of DCI also depends on the extent and severity of vasospasm, cerebral collateral circulation, cerebral metabolic demand, and blood pressure among other parameters [39]. Our study addresses prediction of DCI based on angiographic evidence of vasospasm. Our study does not address the vasospasm-independent predictors or causes of DCI.

Aside from these drawbacks, it is important to recognize that our meta-analysis synthesized data from a fairly large number of patients and provided estimates with relatively narrow confidence intervals. An ideal surveillance device has a sensitivity of 100% and NPV 100%. High sensitivity ensures that all pathological events are captured and high NPV guarantees that a negative test will exclude pathology/poor outcome with certainty. Cerebral angiography has low sensitivity but fair NPV. On the other hand, high SN and high NPV make TCD an ideal modality for surveillance.

Here, we demonstrate that test accuracy estimates were low for cerebral angiography and moderate for TCD. The litmus test of the usefulness of a diagnostic study is its translation into improved clinical outcomes demonstrated in a clinical trial. However, it is logistically and ethically challenging, notwithstanding cost-prohibition, to demonstrate in a randomized trial that a given diagnostic procedure (e.g., angiography, MRI, TCD, or even bedside clinical exam) will translate into improved clinical outcome. It is unlikely that a randomized trial will ever be designed to solely examine the clinical utility of angiography or TCD (or any other imaging study). A prospective evaluation of 186 patients found that routine TCD examinations positively contributed to the diagnosis in 72% of patients with DCI and led to changes in management that benefited the patient in 43% [40]. In 9% of patients, authors concluded that the outcome might have been better if the TCD result had been acted upon appropriately [40]. More recently, our meta-analysis of intra-arterial vasodilators for treatment of vasospasm in SAH found that rates of immediate neurological response and eventual outcome were better when TCD was used for selecting patients for treatment [41]. Neurological response was defined by resolution of, or improvement in, vasospasm-related neurological deficits after treatment, within 24 h. The study defined clinical outcome by modified Rankin scale of 0–2 or Glasgow Outcome Scale of 5–6. TCD not only allows superior prediction of eventual DCI, there is also evidence that it can translate into better outcomes when used for patient selection for treatment of vasospasm.

In conclusion, we provide confirmation that TCD evidence of vasospasm provides better prediction of DCI than angiographic evidence of vasospasm. TCD is therefore recommended for surveillance of vasospasm. Standardizing the approach to grading vasospasm on CA as well as standardizing the treatment of vasospasm with the use of protocols and standardized regimens would reduce heterogeneity in practice and overall improve the care of SAH patients. Future comparative effectiveness studies can better define the value of these diagnostic tools in patients with SAH.