Introduction

Coil embolisation (CE) has proved an effective method of treating intracranial aneurysms but there remains a category of aneurysms in which endosaccular packing with coils is either not feasible or not durable [1–3]. Such aneurysms include: (1) very large or giant, wide-necked saccular aneurysms and (2) fusiform-shaped or circumferential aneurysms. Altering the blood flow into aneurysms using a stent placed in the parent vessel was identified as an effective alternative treatment strategy for such aneurysms [4, 5]. This concept has been developed with the introduction of flow diverters (FDs), which are low porosity stents with a greater metal surface area than conventional stents designed to reconstruct the aneurysm bearing artery so that sac thrombosis occurs. Whereas CE aims to fill the aneurysm sac with coils and occlude it by promoting intra-aneurysmal thrombosis, FDs are deployed in the parent artery across the aneurysm neck to redirect blood flow in the parent artery away from the aneurysm sac. Disruption of both inflow and outflow of blood by the FD deployed at the parent artery-aneurysm interface creates an intra-aneurysmal environment favourable for thrombosis, which, over a period of time, leads to gradual aneurysm occlusion [6]. The flow diversion treatment of intracranial aneurysm is relatively new and experience with it is limited. However, published reports identify it as an effective treatment technique particularly for those aneurysms for which CE may not offer complete cure [6–8].

In order to assess the effectiveness of any treatment option, an objective and reproducible measure of efficacy is required. For CE, the usual measure has for several years been a three-point scale, which classifies the completeness of aneurysm occlusion based on the angiographic appearance as complete, residual neck, or residual aneurysm [9, 10]. This grading scheme is simple, easy to administer, and distinguishes three anatomical outcomes, presumed to have different prognostic significances. For instance, it separates completely or near completely occluded aneurysms from aneurysms with residual filling of the sac, which are associated with higher risks of future rupture and the completely occluded aneurysm from those with remnants, which are associated with a higher incidence of subsequent recurrence [11].

Although the treatment goal is the same, i.e., complete obliteration of the aneurysm, the principle of aneurysm treatment using FDs is different from endosaccular CE. The differences can be summarised as: the effect on the parent artery, coverage of the aneurysm neck, coverage of branch arteries, and delayed sac thrombosis. FD deployment potentially compromises blood flow in the parent artery and requires the use of prophylaxis with antiplatelet drugs. Coverage of the aneurysm neck and delayed progressive thrombosis changes the chronology of sac occlusion compared to CE. In the latter, occlusion is established in a few days and surveillance imaging is directed at monitoring for re-opening due to coil compaction. After FD treatment, monitoring in the short- and medium-term documents progressive sac thrombosis and remodelling of the parent artery. Once achieved, neck closure is presumed to be permanent; whereas with CE, a modest degree of neck remnant may be an inevitable part of clot maturation and retraction around the coils [2, 12]. Thus, with CE, surveillance imaging is concerned with the long-term durability of the treatment and the detection of recurrence. This is not the case for FD treatments (at least initially), where the intention is to monitor progression of aneurysm occlusion.

Oxford Neurovascular and Neuroradiology Research Unit (ONNRU) identified these issues when conducting a registry to audit outcomes of patients treated using a particular FD (SILK flow diverter; Balt International, Montmorency, France) [8]. It quickly became clear that the classification system in current use, designed to assess effectiveness of CE, was inadequate for defining the effects of the new FD devices. A new schema was therefore developed to assess the angiographic outcomes of aneurysms treated with FDs. The purpose of this study is to assess the feasibility and reproducibility this new grading system.

Materials and methods

Angiograms collected at 11 centres (see acknowledgements) which contributed to the SILK registry were used to assess the new grading schema. The registry was conducted to assess the safety and efficacy of the SILK flow diverter (Balt Extrusion, Montmorency, France) used with or without endosaccular coils in a range of aneurysms considered unsuitable for conventional endosaccular packing [8]. Image data from these treatments, submitted to ONNRU for central evaluation, were searched to identify two dimensional digital subtraction angiograms (2D DSA) showing comparable views of aneurysms at treatment and on follow-up.

Fifty-five image sets, all of them acquired in 2008 and 2009, were independently reviewed by two experienced interventional neuroradiologists using the grading schema described below. Of the 55 angiograms reviewed, 39 showed saccular aneurysms while 16 angiograms were of fusiform aneurysms. The data sets had been anonymised prior to their entry in the registry and coded labels were used throughout this study.

A trained researcher explained the grading schema to readers A (JVB) and B (IQG). Reader A had more than 20 years and reader B had ∼10 years of experience in neuroradiology. They were shown illustrations explaining the categories and the two axes of the grading schema, prior to the test images being presented in a random order and in separate sessions. Using the grading schema, the degrees of contrast filling of the aneurysm sac and parent artery were rated from the presented angiograms. The readers used pre-treatment images to obtain baseline dimensions of the treated aneurysm and rated the effects of FD deployment on end-of-treatment and follow-up angiograms.

Grading schema for the assessment of angiographic outcomes following endovascular FD deployment

The proposed grading schema documents the angiographic outcomes following endovascular FD deployment on two axes. Axis-I grades the degree of aneurysm occlusion (for both saccular and fusiform morphologies) on a five-point scale, assigning a higher score to greater aneurysm occlusion. Parent artery status following FD deployment is assessed on Axis-II using a simple three-point scale. Scores for the two axes are reported separately. Details of the grading system are discussed below and illustrated in the Figs. 1, 2, 3, 4, 5, 6.

Fig. 1
figure 1

Axis I grading for saccular aneurysms. Grading the degrees of aneurysm occlusion before (a) and after treatment (b–f) using a flow diverter device. Grade 0 (b) represents no change in sac filling despite device deployment; grade 1 (c) represents any change in endosaccular blood flow but with at least 50% of the sac filing; grade 2 (d) represents less than 50% of the sac filling and grade 3 (e) is the same but filling is restricted to a region which is smaller than the width of the original neck. Grade 4 (f) represents complete sac occlusion with a patent parent artery. The effects of the flow diverter on the parent artery diameter are scored separately (see text)

Full size image
Fig. 2
figure 2

Axis I grading for fusiform aneurysms. Grading the degrees of aneurysm occlusion before (a) and after treatment (b–f) with a flow diverter. Scoring is performed on 2-D angiograms and the sac dimensions are recorded as the maximum width (W), the maximum length (L), and the width of the endovascular FD (FD w ). Following treatment, width of the remnant sac (w) is measured on either side of the deployed FD (w 1 and w 2) and the two measurements are added to obtain an overall estimate of the width of the remnant sac \( \left( {w = {w_1} + {w_2}} \right) \); residual sac length (l) at the point of maximum contrast filling is also measured. W-FD w and L are used as denominators for comparing the dimensions of the remnant sac (w and l) on subsequent angiograms. The degree of contrast filling after treatment is graded 0 (b) when there is no perceptible change in sac filling; grade 1 (c) when filling exceeds 50% of the denominator sac width (W) and 50% of the denominator sac length (L); Grade 2 (d) when contrast fills less than 50% of the denominator sac width (W) or less than 50% of the denominator sac length (L); Grade 3 for contrast filling less than 50% of the denominator sac width (W) and less than 50% of the denominator sac length (L), while grade 4 (e) is complete aneurysm occlusion with parent artery reconstruction

Full size image
Fig. 3
figure 3

Illustrative angiograms. a Pre-treatment 2D-DSA; b end-of-treatment control angiogram showing no reduction in flow (grade 0 on axis-I); c 3-months follow-up angiogram showing neck remnant (grade 3 on axis-I) and a patent parent artery (grade a on axis-II)

Full size image
Fig. 4
figure 4

Illustrative angiograms. a Pre-treatment 2D-DSA; b end-of-treatment control angiogram showing flow reduction with contrast filling greater than 50% of the denominator width and length of the fusiform aneurysm (grade 1 on axis-I); and c 6-months follow-up angiogram showing complete obliteration of the aneurysm (grade 4 on axis-I) with a patent parent artery (grade a on axis-II)

Full size image
Fig. 5
figure 5

Illustrative angiograms. a Pre-treatment 2D-DSA; b end-of-treatment control angiogram showing flow reduction with contrast filling greater than 50% of the pre-treatment aneurysm volume (grade 1 on axis-I) and a patent parent artery (grade a on axis-II); c 3-months follow-up angiogram showing contrast filling less than 50% of pre-treatment aneurysm volume; residual contrast filling is mainly in the neck region but extends beyond the width of aneurysm neck (grade 2 on axis-I) and narrowing of the parent artery (grade b on axis-II)

Full size image
Fig. 6
figure 6

Illustrative angiograms. a Pre-treatment 2D-DSA; b end-of-treatment control angiogram showing flow reduction with contrast filling less than 50% of the pre-treatment aneurysm volume (grade 2 on axis-I) and a patent parent artery (grade a on axis-II); c 3-months follow-up angiogram showing complete obliteration of the aneurysm (grade 4 on axis-I) with a patent parent artery (grade a on axis-II)

Full size image

Axis I: degree of aneurysm occlusion

  1. (a)

    Saccular aneurysms

    1. 0

      No change in the endoaneurysmal flow

    2. 1

      Residual contrast filling greater than 50% of the pre-treatment aneurysm volume (includes contrast medium stasis or a reduction in the contrast density inside the aneurysm sac)

    3. 2

      Residual contrast filling less than 50% of the pre-treatment aneurysm volume

    4. 3

      Residual filling confined to the neck region and not extending beyond the width of neck

    5. 4

      No residual filling, i.e., complete obliteration of the aneurysm

  2. (b)

    Fusiform or circumferential aneurysms

    For the fusiform aneurysms, sac dimensions are recorded as the maximum width (W), the maximum length (L), and the width of the endovascular FD (FD w ). Following treatment, width of the remnant sac (w) is measured on either side of the deployed FD (w 1 and w 2 ) and the two measurements are added to obtain an overall estimate of the width of the remnant sac \( \left( {w = {w_1} + {w_2}} \right) \); residual sac length (l) at the point of maximum contrast filling is also measured. W-FD w and L are used as denominators for comparing the dimensions of the remnant sac (w and l) on subsequent angiograms (Fig. 2).

    1. 0

      No change in the endoaneurysmal flow

    2. 1

      Residual contrast filling >50% the pre-treatment length and >50% the pre-treatment width of the aneurysm (l > 50% L and w > 50% W-FD w )

    3. 2

      Residual contrast filling either <50% the pre-treatment length or <50% pre-treatment width (l < 50% L or w < 50% W-FD w )

    4. 3

      Residual contrast filling <50% pre-treatment length and <50% pre-treatment width (l < 50% L and w < 50% W-FD w )

    5. 4

      No residual contrast filling, i.e., complete obliteration of the aneurysm (l = 0 and w = 0)

Axis II: patency status of the parent artery

  1. a

    No change in the parent artery diameter

  2. b

    Narrowing of the parent artery

  3. c

    Parent artery occlusion

Statistical analysis

Cohen's kappa coefficient was calculated using a statistical software (SPSS v 18.0, SPSS Inc., Chicago, IL) to assess the inter-observer agreement on the two axes of the ONNRU grading schema. When calculating the kappa statistic, the ordinal nature of the grading system and the size of categories on the two axes were taken into account by assigning proportional weights to individual categories. Quadratic weights were found appropriate for the axis I (5 degrees of residual sac filling), while linear weights were assigned to the axis II (3 degrees of parent artery patency). The resulting kappa statistic was interpreted for the inter-observer agreement using the following criteria: ≤0.40 poor agreement, 0.40–0.75 fair to good agreement, and ≥0.75 excellent agreement [13].

Results

After the initial explanation by the researcher, both readers found the grading schema easy to apply. Compared with axis I (degree of aneurysm occlusion on a five-point scale), the inter-observer agreement was better for axis II (assessment of parent artery status on a three-point scale). Observers’ ratings on the two axes are summarised in Tables 1 and 2.

Table 1 Axis I: grading of the saccular occlusions following deployment of the FD devices
Full size table
Table 2 Axis II: grading of the parent artery patency following deployment of the FD devices
Full size table

Inter-observer agreements were excellent both for the assessment of degree of aneurysm occlusion (kappa statistic 0.89; 95% CI 0.81–0.99) and the parent artery patency status (kappa statistic 0.90; 95% CI 0.76–1.0). Overall, there were ten discordant readings for the assessment of aneurysm occlusion and two for the assessment of the parent artery patency.

Discussion

We propose a new grading system to assess the angiographic outcomes following placement of endovascular FD devices which obstruct blood inflow at the aneurysm neck in order to induce intra-aneurysmal thrombosis. Two interventional neuroradiologists used the schema to evaluate representative follow-up angiograms and good inter-observer agreements, reflected by kappa coefficients, were observed. We acknowledge that in proposing a new grading scale for clinical use, one must first justify why precedent scales are not sufficient and provide reasons for implementing a new system. The classification schemes most frequently used for the assessment of angiographic outcomes following conventional CE rely on either a three- or a four-grading [9]. The three-point scale, now most commonly used, describes treated intracranial aneurysms as completely occluded, with neck remnant, or with saccular filling. Proposed by Jean Raymond and colleagues, it primarily separates the aneurysms with a patent sac from those completely or near completely occluded (neck remnant). This distinction is considered important as the patent sac category is assumed to have a prognosis different from the other categories, i.e., a higher risk of future growth or bleeding [11, 14]. The durability of aneurysm occlusion following CE is a concern, so the aim of treatment is to fill the sac with as many coils as possible and achieve complete blood flow occlusion because completely occluded aneurysms have been shown to have lower recurrence rates than incompletely occluded aneurysms [10, 15–17]. However, even if an apparently complete occlusion is achieved after endovascular treatment, delayed recurrence may still occur because the same haemodynamic stress applies to the coil-thrombus complex via the patent aneurysm neck [18, 19]. Thus the grading scale used to categorise surveillance imaging has to be able to identify any change in sac filling and is primarily concerned with increases in the size of any remnant. Thus the neck remnant category is included to capture this situation but separately from larger remnants.

For treatments using the FD, the method and the sequence of events that follow their deployment in the parent artery is worth considering in detail because of inherent differences from those that follow packing the aneurysm with coils only. FDs partially obstruct blood flow into the aneurysm sac by redirecting it in the parent artery so that it is concentrated along the course of the parent vessel. Flow diversion, which starts as soon as the FD is deployed, causes blood stasis inside the sac and aneurysm thrombosis is initiated. Once begun, successful sac occlusion depends on a growing thrombus extending to fill the sac; however, the process is gradual and may take weeks depending upon the effective reduction in sac filling. This depends on several different factors such as the porosity of the FD device, size of the aneurysm, location, and the haematological status of the patient. The growth of an endothelial tissue lining on the FD and neointima formation across the aneurysm neck completes the healing process forming a seal across the neck region [20]. The FD treatment strategy is new and there is very little published data on its biological effects. The process of aneurysmal sac occlusion is assumed to be unidirectional (progressive thrombosis and reduction in patent sac) and recanalisation of the sac is generally not discussed in the published reports. However, the SILK registry reported one case where a patient developed delayed recanalisation of a previously partially occluded aneurysm sac which subsequently ruptured [8].

The mechanism and time course of FD action thus requires a grading scale that can be used to document different degrees of occlusion of the treated aneurysm. Since the occlusion evolves over time, any new grading scale needs to be sufficiently sensitive to register changes on interval examinations. One solution would be to use a linear scale based simply on the percentage of the original aneurysm volume occluded. This approach has proved difficult in practice when applied to endosaccular CE because determining aneurysm sac volumes has not been standardised, making comparisons difficult. Remodelling of the sac in response to reduced blood inflow over a period of time, as seen after successful aneurysm thrombosis induced by parent artery occlusion or FD devices, makes it difficult to compare volumes in such a system. The ordinal scale, to some extent, avoids these difficulties but also needs to be responsive to varying aneurysm sac size, morphology, and the degree of any residual filling. The grading scheme of FD treatments, like the Raymond scale, needs to recognize the possibility of recanalisation of a near completely occluded aneurysm. It is currently not known if recanalisation occurs for a completely occluded aneurysm with a remodelled parent vessel.

In addition, the endovascular placement of an FD device may be associated with narrowing or occlusion of the parent artery and can affect branch arteries whose origins are covered by the device. An evaluation scale should allow for these possibilities, as well as being applicable to the more complex shapes of larger aneurysms that are currently the subject of the new treatment. The three-point grading system was designed for endosaccular CE; it fails for fusiform-shaped aneurysms and does not address the effects of the device on parent or branch arteries. Its limitations became obvious during the evaluation of angiograms submitted as part of the SILK registry when the need for a more sensitive and more widely applicable grading scheme became apparent.

For the new schema, saccular occlusion and change in the parent vessel patency following endovascular deployment of the FD were identified as two independent processes and were therefore independently rated. Although the scores on the two axes could be combined into a single score, we consider that it would be clinically more useful to report the scores separately. The number of categories on axis-I was kept enough to register serial changes in flow following the placement of FD. The five major categories, i.e., no reduction in flow (grade 0), less than 50% flow reduction (grade 1), more than 50% flow reduction (grade 2), and almost complete occlusion or complete (grades 3 and 4), were found sufficient in evaluating the effectiveness of FD in treating the target aneurysm and documenting changes in contrast filling at multiple time-points. Although it appears intuitive to have more categories than five to allow detection of minor changes in flow, such a strategy might reduce the reproducibility of scores and smaller differences between adjacent categories on the scale may not be needed for predicting the final outcome.

There is obviously a subjective component to our definitions which requires testing by larger numbers of readers and proof in practice. The possibility of recanalisation of a completely or near completely occluded aneurysm required the neck remnant category as used in the three-point scale for CE; though theoretically, if a substantial portion of the aneurysm neck has been covered by a new endothelium as a result of the FD treatment, the haemodynamic factors that cause coil compaction and recurrence after CE treatment have been so altered that recurrence is less likely. For similar reasons, the cut-offs for other categories and their number may need to be revised as more data on the follow-up of FD-treated aneurysms becomes available. Axis II was introduced to identify changes in parent artery status, and it adds to an overall evaluation of the effects of FD on the parent artery-aneurysm complex. The three-point categorisation of axis II is intuitive and recognises parent artery narrowing or occlusion. We considered adding an evaluation of branch artery patency; however, since it does not directly or indirectly affect the degree of aneurysm occlusion, it was decided that it would unnecessarily add to the complexity of the schema. Whether or not this scale is easy to administer will be decided by the neuroradiologist using FD treatment strategy; however, in its current form the scale is reproducible enough to be used as shown by the kappa coefficient.

Conclusion

We acknowledge that the choice of categories for the new grading schema is arbitrary and that the various parameters defining the proposed schema, e.g., number of intervals, their size, and break points are broad in order to accommodate the relatively subjective nature of angiogram reading. For a scale to be clinically useful, it should be (1) simple and easy to administer, (2) sensitive to detect meaningful changes, (3) reproducible, and (4) able to distinguish outcomes which have different prognosis. These aims served as the guidelines in adjusting the parameters and developing the grading schema reported here. For instance, increasing the number of intervals could have made the scale very sensitive to detecting very small changes in flow reduction or parent artery calibre but would have increased the inter-observer variability and risked over-splitting error which might adversely affect its prognostic power. Such trade offs were recognised and the chosen categories appeared intuitive and with practical meaning.