Introduction

Intracranial aneurysms are an important health problem worldwide, affecting about 2% of the population [1]. Treatment consists of the exclusion of the malformation from the intracranial circulation to eliminate the risk of (re)bleeding. The introduction of the Guglielmi detachable coil (GDC) in 1991 [2, 3] was a breakthrough in the treatment of intracranial aneurysms [4–6]. Currently, more than 200,000 patients have been treated worldwide using this technique [6]. However, aneurysms that have been treated with GDCs may recur. Data from the International Subarachnoid Aneurysm Trial show that the rebleed rate is 0.2% per patient year with follow-up from 1 to 8 years with a mean of 4 years [5]. Thus, vigilant follow-up is needed to detect residual aneurysmal flow [7–11], since re-coiling may be indicated.

Digital subtraction angiography (DSA) has been established as the method of reference for aneurysm evaluation after coiling. However, DSA is an invasive method and has the risk of neurological complications. Large prospective studies have revealed a combined transient and reversible neurological complication rate in the range 0.4% to 2.3%, and a permanent neurological complication rate in the range 0.1% to 0.5% [12–16]. This risk accumulates, because multiple follow-up examinations are mandatory [7–11].

GDCs are compatible with MR imaging in terms of safety and image quality at static magnetic field strengths up to 3.0 T [17–20]. Magnetic resonance angiography (MRA) may therefore be an excellent alternative to DSA: it is noninvasive, does not carry the risk of neurological complications, does not require anaesthesia, and is radiation-free. Moreover, the brain parenchyma can be imaged at the same time.

The purpose of this study was therefore to systematically review the contemporary literature and to compare the diagnostic performance of MRA with DSA as the standard of reference in the follow-up of intracranial aneurysms treated with GDCs.

Methods

Search strategy

A computer-aided search of the PubMed/MEDLINE and Embase databases was conducted to find relevant published articles on the diagnostic performance of MRA in the follow-up of patients with intracranial aneurysms treated with GDCs, and in whom DSA was used as the standard of reference. The search strategy is presented in Table 1. No beginning date limit was used. The search was updated until 31 January 2007. To expand our search, bibliographies of articles which finally remained after the selection process were screened for potentially suitable references.

Table 1 Search strategy and results as of 31 January 2007
Full size table

Study selection

In the first stage, two researchers (T.C.K., R.M.K.) independently reviewed the titles and abstracts of the retrieved articles. Studies not comparing MRA with DSA in the follow-up of patients with intracranial aneurysms treated with GDCs, review articles, meta-analyses, abstracts, editorials or letters, case reports, tutorials, and guidelines for management were excluded. Studies performed in animals and studies involving ten or fewer patients were also excluded. Only English-, German-, and French-language studies were included because the investigators were familiar with these languages. Articles were rejected if they were clearly ineligible.

In the second stage, the same researchers (T.C.K., R.M.K.) independently evaluated the full-text version of all articles that were found to be potentially eligible for inclusion, using the same inclusion and exclusion criteria as mentioned above. In the data analysis, nonenhanced time-of-flight MRA (TOF-MRA) and contrast-enhanced MRA (CE-MRA) were analysed separately. When the results of TOF-MRA and CE-MRA were mixed together in the same study, making separate analysis impossible, that study was excluded. Studies which provided insufficient data to construct a 2×2 contingency table to calculate sensitivity and specificity were also excluded. When data were presented in more than one article, the article with the most details, or the most recent article, was chosen.

At all stages, disagreements between the two researchers (T.C.K., R.M.K.) were discussed and resolved by consensus.

Study quality

For each study, the methodological quality was assessed by using the Quality Assessment of Studies of Diagnostic Accuracy Included in Systematic Reviews (QUADAS) criteria, which is a 14-item instrument [21, 22]. The item “Was the reference standard independent of the index test (i.e. the index test did not form part of the reference standard)?” did not apply and was thus removed from the standard QUADAS list. In some of the included studies, the interpreters of the MRA images might have been aware of the findings of DSA performed immediately after the coiling procedure. However, we believe that this method of image analysis resembles that in clinical practice and is justified. Therefore, we also removed the item “Were the same clinical data available when test results were interpreted as would be available when the test is used in practice?” from the standard QUADAS list. Two items were added to the standard QUADAS list: “Was comparator review bias avoided?” and “Was the reproducibility (interobserver) of MRA described?” The complete list of quality items is presented in Table 2. For each item, the two researchers (T.C.K., R.M.K.) independently assessed whether it was fulfilled (yes or no). If it was unclear from the information provided in an article as to whether an item was fulfilled, the item was rated as “unclear”. Both “no” and “unclear” responses were interpreted as indicating that the quality criterion was not met. Disagreements were discussed and resolved by consensus. Agreement between the two reviewers was quantified by Cohen’s κ [23], using the Statistical Package for the Social Sciences version 11.5 (SPSS, Chicago, Ill.). Total quality score was expressed as a percentage of the maximum score of 14.

Table 2 Criteria list used to assess the methodological quality of the studies
Full size table

Data analysis

A distinction was made between studies (or subsets in studies) evaluating TOF-MRA, and those evaluating CE-MRA. Sensitivity and specificity of both types of MRA for the detection of residual flow (within the aneurysmal neck and/or coil mesh), with corresponding 95% CIs, were calculated from the original numbers given in the included studies. If the necessary data could be extracted from the included studies, we also performed separate analyses for the detection of residual flow within the aneurysmal neck and for the detection of residual flow within the coil mesh. If multiple MRA examinations were performed in the same patient during follow-up (and comparison with DSA was available), each of those examinations was regarded as a single evaluation and separately taken into the analysis. The number of uninterpretable results per study was extracted.

Meta-analysis was performed using a bivariate random effects approach to pool the sensitivity and specificity [24]. This model assumes a bivariate normal distribution for the logit-transformed sensitivity and specificity values across studies, allowing for heterogeneity beyond chance due to clinical or methodological differences between studies. It incorporates and estimates the correlation that might exist between estimates of sensitivity and specificity within studies. A standard correction of adding 0.5 to all cells of the 2×2 contingency table was applied if the true-positive rate, false-positive rate, false-negative rate or true-negative rate was zero. Estimates of the mean logit-transformed sensitivity and specificity were then obtained. Pooled estimates of sensitivity and specificity with 95% CIs were calculated after antilogarithm transformation of these logit estimates. A Chi-squared test was performed to test for heterogeneity between studies. Heterogeneity was defined as P<0.10. Differences in pooled sensitivity and specificity between TOF-MRA and CE-MRA were determined by using an F-test, with the level of statistical significance set at P<0.05. To improve visualization of the results, the 95% coverage region of the estimated bivariate distribution of the logit sensitivity and specificity was transformed back to receiver operating characteristic (ROC) axes [24]. The results of the included original studies were also plotted in ROC space. Statistical analyses were executed with the SAS statistical software package version 9.1.3 (SAS Institute, Cary, N.C.).

Results

Literature search

The computer-aided search revealed 146 articles from PubMed/MEDLINE and 309 articles from Embase (Table 1). Reviewing titles and abstracts from PubMed/MEDLINE revealed 21 articles potentially eligible for inclusion. Reviewing titles and abstracts from Embase revealed 23 articles potentially eligible for inclusion, of which 18 were already identified by the PubMed/MEDLINE search. Thus, 26 studies [25–50] remained for possible inclusion and were retrieved in full text version. Screening of the references of these 26 articles did not bring up new articles. After reviewing the full article, four studies [26, 40, 43, 47] were excluded because they did not supply sufficient information to construct a 2×2 contingency table to calculate sensitivity and specificity. Two studies [32, 36] were excluded because the same data were used in later studies. One study [35] was excluded because clipped and coiled aneurysms were analyzed together. One study [49] was excluded because the results of TOF-MRA and CE-MRA were mixed together, making separate analysis impossible. Another study [25] was excluded because it was a review article, and one study [50] was excluded because it was a case series involving five patients. Eventually, 16 studies [27–31, 33, 34, 37–39, 41, 42, 44–46, 48] were included in this systematic review, of which 14 investigated TOF-MRA [28–31, 33, 34, 37–39, 42, 44–46, 48] and 7 investigated CE-MRA [27, 30, 31, 37, 38, 41, 44]. The characteristics of the included studies are outlined in Table 3.

Table 3 Characteristics of the 16 included studies
Full size table

Methodological quality assessment

For each of the 16 included studies, 14 methodological quality items were assessed (Table 4). There was disagreement in 7 of 224 scores with a Cohen’s κ of 0.93. Disagreements occurred in items 1, 2, 7, and 10, and were caused by reading errors and differences in interpretation. The total quality score, expressed as a fraction of the maximum score, ranged from 43% to 79%, with a median of 64%. Most studies (81%) were assigned a positive score for inclusion of the correct spectrum of patients. All studies avoided verification bias (quality items 5 and 6) and the majority of studies (88%) explicitly mentioned that MRA was interpreted without knowledge of the DSA findings. However, most studies (94%) did not clearly describe how patients were selected for inclusion. In ten studies (63%), the time between MRA and DSA (possibly) exceeded 48 h. The majority of studies did not sufficiently describe how MRA and DSA were performed (69% and 94%, respectively). Ten studies (63%) did not explicitly mention whether DSA was interpreted without knowledge of the MRA findings. Only three studies (19%) described the interobserver reproducibility of MRA.

Table 4 Quality assessment of the 16 studies included in the present review
Full size table

Data analysis

The results of the TOF-MRA studies are presented in Table 5, and the corresponding ROC plot is displayed in Fig. 1. Gonner et al. [48] applied two sequences in all patients (Table 3), with equal results; they were regarded as one result in our analysis. Sensitivity and specificity of TOF-MRA for the detection of residual flow (within the aneurysmal neck and/or coil mesh) ranged from 29.4% to 100% and from 50.0% to 100%, with pooled estimates of 83.3% (95% CI 70.3–91.3%) and 90.6% (95% CI 80.4–95.8%), respectively. Both pooled estimates were subject to heterogeneity (P<0.001 for both), which is also demonstrated by the degree of scatter of the results of the original studies in the ROC plot (Fig. 1). Results of studies in which separate analyses were performed for the detection of residual flow within the aneurysmal neck and for the detection of residual flow within the coil mesh are also presented in Table 5. Of 14 TOF-MRA studies, 6 had uninterpretable results, which ranged from 1.4% to 11.1% of their total results [29, 30, 34, 37, 44, 45]. The percentage of uninterpretable results in all TOF-MRA studies was 3.4%. Uninterpretable results were caused by clip-related artifacts in 22.2%. The causes of the remaining uninterpretable results were not further specified.

Table 5 Sensitivity and specificity of non-enhanced time-of-flight magnetic resonance angiography (TOF-MRA) for the detection of residual aneurysmal flow
Full size table
Fig. 1
figure 1

Graph of pooled sensitivities and specificities (including 95% confidence ellipses) and original studies for the detection of residual flow (within the aneurysmal neck and/or coil mesh) with non-enhanced time-of-flight magnetic resonance angiography (TOF-MRA) and contrast-enhanced magnetic resonance angiography (CE-MRA)

Full size image

The results of the CE-MRA studies are presented in Table 6, and the corresponding ROC plot is shown in Fig. 1. Sensitivity and specificity of CE-MRA for the detection of residual flow (within the aneurysmal neck and/or coil mesh) ranged from 72.0% to 100% and from 73.7% to 100%, with pooled estimates of 86.8% (95% CI 71.4–94.5%) and 91.9% (95% CI 79.8–97.0%), respectively. Both pooled estimates were subject to heterogeneity (P=0.049 and P=0.002, respectively), which is also demonstrated by the degree of scatter of the results of the original studies in the ROC plot (Fig. 1). Gauvrit et al. [27] were the only ones who performed separate analyses for the detection of residual flow within the aneurysmal neck and for the detection of residual flow within the coil mesh, and their results are also presented in Table 6. Four of seven CE-MRA studies [27, 37, 41, 44] had uninterpretable results, which ranged from 1.2% to 6.3% of their total results. The percentage of uninterpretable results in all CE-MRA studies was 3.0%. Uninterpretable results were caused by clip-related artifacts in 41.7% and motion artifacts in 8.3%. The causes of the remaining uninterpretable results were not further specified.

Table 6 Sensitivity and specificity of contrast-enhanced magnetic resonance angiography (CE-MRA) for the detection of residual aneurysmal flow
Full size table

The pooled sensitivity and specificity of TOF-MRA were not statistically different from those of CE-MRA (P=0.66 and P=0.82, respectively).

Discussion

This systematic review and meta-analysis included 16 studies concerning the value of MRA for the detection of residual aneurysmal flow in the follow-up of patients treated with GDCs. The included studies had a moderate methodological quality score, with a median total quality score of 64%. Pooled sensitivity and specificity of TOF-MRA for the detection of residual aneurysmal flow (within the aneurysmal neck and/or coil mesh) were 83.3% and 90.6%, respectively. For CE-MRA, pooled sensitivity and specificity were 86.8% and 91.9%, respectively.

The results of this systematic review should be interpreted with caution, because of several limitations. First, significant heterogeneity was identified between the results of the included studies. There were insufficient studies to use meta-regression to examine the causes for this heterogeneity. Subjective review of Table 5 and the ROC plot in Fig. 1 shows that the study of Farb et al. [30] was an outlier with low sensitivity (29.4%) of TOF-MRA. A possible explanation may be that many patients in that study had large aneurysmal remnants (≥3 mm), which could not be detected by TOF-MRA. Other potential sources of variability among the included studies are variations in scanning protocols, patient characteristics, differences in implicit threshold, and the experience of image interpreters. Second, the included studies had limited methodological quality. In 15 studies (94%), there was a poor reporting of the method of cohort assembly. Furthermore, in 10 studies (63%) the time between DSA and MRA (possibly) exceeded 48 h, up to a maximum of 6 weeks [48], making modification of the aneurysmal packing between the two tests possible. Moreover, in ten studies (63%) it was not clear whether the reference test was interpreted independently of the index test, which might have lead to diagnostic bias. Three studies [37, 38, 44] (19%) investigated both TOF-MRA and CE-MRA. However, they did not mention whether the evaluation of the two tests was done blinded from each other, which might have lead to comparator review bias. Source images should always be consulted, because all postprocessing algorithms inevitably produce some degree of data reduction. However, two studies [27, 38] did not include source images for interpretation, but only maximum intensity projections. Additionally, in two studies [28, 48] (13%) it was not clear whether source images were evaluated. Orthogonal plane (2-D) DSA was used in 13 studies [27, 29–31, 37–39, 41, 42, 44–46, 48] (81%) as the standard of reference. However, a remnant cavity of an embolized aneurysm may be masked by the radio-opaque coil mass and missed on 2-D DSA [30]. The rate at which this occurs is unknown. Rotational (3-D) DSA may overcome this problem, but was used in only three studies (19%) as the standard of reference [28, 33, 34]. In clinical practice, there is usually a single interpreter of the MRA images, whereas in most of the included studies there was a consensus review by two or more interpreters, which could have resulted in a positive bias. Also, only three studies (19%) described the interobserver reproducibility of MRA, with Cohen’s κ ranging between 0.13 and 0.92 [27, 31, 38]. None of the included studies described the intraobserver reproducibility of MRA. Finally, a potential limitation of any meta-analysis is the possibility of publication bias; studies that show a positive effect tend to be published more often than those that do not [51].

We also performed separate analyses for the detection of residual flow within the aneurysmal neck and for the detection of residual flow within the coil mesh, if the necessary data could be extracted from the included studies. However, various definitions were used for the two terms, making a comparison between studies difficult. Seven studies [28, 33, 34, 39, 42, 44, 46] applied their own definitions and two studies [27, 29] applied the classification according to Roy et al. [52]. According to the classification of Roy et al., a residual neck is defined as the persistence of any portion of the original defect of the arterial wall but without opacification of the aneurysmal sac, and residual flow within the coil mesh is defined as any opacification of the aneurysmal sac. However, the distinction between a residual neck and residual flow within the coil mesh cannot be reliably made [53]. For these reasons, we omitted calculation of pooled sensitivity and specificity for the detection of residual flow within the aneurysmal neck and coil mesh separately.

Two types of MRA have been studied for the follow-up of aneurysms treated with GDCs: TOF-MRA and CE-MRA. TOF-MRA is performed without the use of a contrast agent, whereas in CE-MRA, a gadolinium chelate is administered intravenously. Both types of MRA use a T1 effect, with the flow of unsaturated spins into the saturated static image volume producing an increased signal [54]. The main disadvantage of TOF-MRA is that turbulent and slow residual flow in a coiled aneurysm may result in signal loss, because of intravoxel dephasing and spin saturation, respectively. This signal loss may particularly occur in larger aneurysmal remnants. Intravoxel spin dephasing may be overcome by the use of smaller voxel sizes, thinner slices, and a short TE. Placement of the coiled aneurysm near the inflow portion of the image volume, and the use of lower flip angles in combination with longer TRs, the shortest possible TE, and thinner slices or the multiple overlapping thin slab acquisition (MOTSA) technique, will minimize spin saturation effects [54]. The remaining spin saturation effects can be further eliminated by intravenous injection of paramagnetic contrast material (CE-MRA), which shortens the T1 of flowing blood. However, disadvantages of CE-MRA compared to TOF-MRA are increased cost and possible superimposition of veins [54]. Venous contamination may be minimized with an automated triggering tool to ensure optimal synchronization of peak arterial contrast concentration, with initiation of a k-space elliptic central acquisition scan [30]. Another disadvantage of CE-MRA is the possibility of peripheral contrast enhancement of the organized thrombus or of the vasa vasorum within the adventitial layer of the aneurysm wall, leading to false-positive results [27, 32]. A limitation of both TOF-MRA and CE-MRA is that a subacute thrombus containing methaemoglobin may simulate flow, because it has a short T1. If a high signal-intensity thrombus is suspected on the basis of MRA findings, black-blood MRA can be performed to confirm it or rule it out [33, 54, 55].

One should further keep in mind that MRA is limited in patients with coiled aneurysms adjacent to previously surgically clipped aneurysms due to susceptibility artifacts, and it is contraindicated in patients with claustrophobia, pacemakers, defibrillators or other implanted electronic devices.

MRA in the follow-up of GDC-treated aneurysms at 3.0 T may potentially be superior to MRA at 1.0 T or 1.5 T because of higher spatial resolution, more efficient suppression of the background tissue, and higher signal-to-noise ratio. On the other hand, MRA at 3.0 T may lead to greater coil-related susceptibility artifacts and, as a consequence, impaired image quality. Of the included studies in the present review, Majoie et al. were the only ones who investigated MRA at 3.0 T, both for TOF-MRA and CE-MRA. In their study, there was full agreement between MRA (both TOF-MRA and CE-MRA) and DSA in 17 (81%) of 21 coiled aneurysms. High-signal intensity rim artifacts were seen on 3 (14%) of 21 MRA images, but did not interfere with interpretation of the occlusion status of the aneurysms [31]. More 3.0-T MRA studies making a direct comparison with MRA at lower field strengths are needed to determine its exact value.

Neuroform stent-assisted coil embolization is a promising technique for the treatment of wide-necked aneurysms [56]. Neuroform stents are compatible with MR imaging in terms of safety and image quality at static magnetic field strengths up to 3.0 T [57]. Patients treated with this technique may therefore also benefit from noninvasive follow-up using MRA [58]. Future studies are needed to determine the value of MRA as an alternative to DSA in this patient population.

In conclusion, the results of this systematic review and meta-analysis suggest that both TOF-MRA and CE-MRA achieve a moderate to high diagnostic performance in the detection of residual aneurysmal flow in the follow-up of patients treated with GDCs. It should be noted, however, that the studies included in this review were of moderate methodological quality and that considerable heterogeneity was identified across the included studies. More well-designed studies are required to confirm the current results and MRA at higher field strength (>1.5 T) needs to be further explored.