Introduction

Quantitative coronary angiography (QCA) has been used for more than 20 years as a standard method to obtain objective data on the severity of coronary artery stenosis and the dimensions of the vessel in the majority of clinical trials in the field of percutaneous coronary intervention (PCI) [1]. QAngio XA (Medis Medical Imaging System BV, Leiden, the Netherlands) is one of the most popular computed-QCA systems, previous versions of which were validated in patients [2–6] and in vitro and in vivo phantom studies [4, 5, 7]. Although QAngio XA version 7.3 is used worldwide, no proper study of agreement or reproducibility has been performed on this most recent software update, which introduced substantial changes in the algorithm for calculating the minimum lumen diameter of stenosis (MLD). This is despite the fact that the algorithms of a QCA system define its reliability. In the old version, there may have been some overestimation due to the slants of the arterial and reference centrelines of the case. In the new algorithm, in contrast, the arterial diameters are taken perpendicular to the arterial centreline, and the reference diameters perpendicular to the reference centreline (Fig. 1). Because QCA analysis is conducted at a specific core laboratory in many clinical trials, it is possible that a QCA system might be updated unavoidably to a newer version before a trial is completed. It is important to know how extent of the systematic difference in measurements of the previous version exists compared to the current version of QAngio XA. This study therefore aimed to investigate the agreement of averages between versions 7.3 and 6.0 of QAngio XA by comparing the variables measured using the two versions with a sufficient sample size.

Fig. 1
figure 1

Algorithm of QCA system

Full size image

Methods

Population

This study used imaging data collected in the OUCH-TL [8] and OUCH-Pro [9] studies, which were prospective non-randomized single-arm registries designed to assess the clinical and angiographic outcomes of drug-eluting stents (DES) in haemodialysis patients. OUCH-TL [8], utilizing paclitaxel-eluting coronary stents, enrolled 154 lesions in 116 patients, and OUCH-Pro [9], utilizing everolimus-eluting coronary stents, included 160 lesions in 123 patients in 28 institutions in Japan. Both studies followed the same protocol as the OUCH registry [10]. In brief, consecutive haemodialysis patients who underwent PCI using assigned DES were enrolled, and the exclusion criteria were as follows: history of surviving an episode of cardiac arrest, cardiogenic shock, emergency PCI, ST-segment elevation myocardial infarction, intolerance to anti-platelet drugs, coronary stenting within 6 months, in-stent restenosis following DES implantation, severe valvular disease, critical limb ischemia, and chronic total occlusion of the target vessel.

Study design

In this study, we selected patients who had a single target lesion. Total occlusion, bifurcation lesions were excluded from the analysis because of the difficulty in measuring the lesion length (LL) of total occlusions. Left main trunk lesions were also excluded because of the inaccuracy of the interpolated reference diameter (RefD) before and after a large bifurcation. The primary endpoint of this study was MLD, because it is usually used to evaluate QCA parameters and is easily affected by changes in algorithm, as shown in Fig. 1. The secondary endpoints were RefD and LL.

After random sampling from the OUCH-TL and OUCH-Pro studies, two experienced analysts independently analysed lesions with both versions of QAngio XA. The study was designed so that each lesion could cross over to a different version of QAngio XA. The order in which to the lesions would be analysed was randomly (1:1) allocated with random number by computer stratified by target vessels (right coronary artery, left anterior descending coronary artery, and left circumflex coronary artery) centrally by the first author. The version of QAngio XA could be recognised by two analysts. The same frame was analysed using the two versions at least a month apart to prevent the use of knowledge from the first analysis [11].

QCA analysis

Before analysis, the first author selected a frame that (1) was filled with contrast agent, (2) provided a sufficiently clear image with which to analyse the lesion, (3) did not overlap with branches of the coronary tree, and (4) enabled specification of the segments and their edges 5 mm apart from the stenotic lesion. Two other analysts at the core laboratory, Cardiocore Japan (Yokohama, Japan), analysed the three parameters described above for the same frame with both software versions 6.0 and 7.3.

The entire analytical procedure of the QCA system consists of the following steps [12]:

  1. 1.

    Calibration of the image data in an appropriate frame using catheter size information;

  2. 2.

    Assignment of the two edges of the coronary segment selected;

  3. 3.

    Automated path line detection;

  4. 4.

    Automated contour detection of the arterial segment with adjustment by an analyst;

  5. 5.

    Derivation of the QCA parameters.

Data and management

Anonymous data and case report forms were collected from the participating centres. All patients gave written informed consent for the OUCH-series studies and for comprehensive usage of their data. Both OUCH-TL [8] and OUCH-Pro [9] registries have been approved by institutional ethics committees and both studies have been registered in the open-access clinical trials registry (as UMIN000002594 and UMIN000006684). Subsequently, the secondary use of the data in this study was approved by the research ethics committee at the University of Tokyo (Examination No.: 11001). Cardiocore Japan managed all angiographic data as the administrator of these studies.

Sample size calculation

We determined our sample size via a random-intercept analysis-of-variance model. We set the “permissible range of difference” between versions to be a ±0.2 mm (the typical value per pixel in practice [12] that is considered acceptable range if the observed difference between versions was within the permissible range of difference [12]). To calculate the sample size required to maintain a power of 0.9 with two-sided alpha of 0.05, at least 93 lesions would be required based on the previous evidences [13]. In this study, 100 patients were randomly sampled accounting for uncertainties in the presumed values of the parameters in sample size calculation. Simple random sampling was performed using random number generated by a computer. The details of calculating the sample size are described in the Appendix.

Statistical analyses

Reproducibility of measurements between versions was graphically summarized by scatter diagrams with Lim’s coefficient of orthogonal regression between versions without considering the variance of lesions and Bland–Altman plots with intra-class correlations [14].

In the main analysis, we analysed the data by the following random-intercept analysis-of-variance model to assess the agreement of the mean measurements between QAngio XA version 7.3 and 6.0. The assumed model was

$${{Z}_{ijk}}=\mu +{{\alpha }_{i}}+{{\beta }_{j}}+{{\gamma }_{k}}+{{\varepsilon }_{ijk}},$$
$${{a}_{i}}\tilde{\ }N\left( 0,\sigma _{\alpha }^{2} \right)~~i.i.d.,{{\varepsilon }_{ij}}\tilde{\ }N\left( 0,\sigma _{\varepsilon }^{2} \right)~~i.i.d.$$

where \({{Z}_{ijk}}={{Y}_{ij2}}-{{Y}_{ij1}}\) is the difference between measurements of lesion \(i=1,\,~2,\,\ldots \,,\,n\) read by two analysts \(~j=1,~\,2\) with version 6.0 (\({{Y}_{ij1}}\)) and version 7.3 (\({{Y}_{ij2}}\)). We considered lesion effects \({{\alpha }_{i}}\) as random effects, and rater effects \({{\beta }_{j}}\), which are equivalent to inter-observer differences, and order effects \({{\gamma }_{k}}~\)of the randomised reading-order k = 1 (version 6.0 was first), 2 (version 7.3 was first) as a fixed effect. Figure 2 explains our study design as well as relating parameters in our statistical model.

Fig. 2
figure 2

Study design. Order of analysis was randomized for each lesion. The two versions were analysed at least a month apart to prevent the use of knowledge from the first analysis

Full size image

The average difference between the two versions was estimated with SAS/MIXED (SAS Institute, Cary, NC) with the robust variance. Subgroup analyses were performed by dividing target vessels into curved vessels (the right coronary artery and left circumflex coronary artery) and relatively straight vessels (the left anterior descending coronary artery), and then by using other extraneous factors as exploratory analyses. All analyses were performed with SAS version 9.4.

Results

The patient flow diagram is shown in Fig. 3. From 170 lesions which fulfilled the selection criteria, 100 lesions were randomly sampled. Patient and lesion characteristics are listed in Table 1, which shows no significant difference between OUCH-TL [8] and OUCH-Pro [9] with reference to the integration of the data from the two studies.

Fig. 3
figure 3

Patient flow. LAD left anterior descending coronary artery, LCX left circumflex coronary artery, LMT left main trunk, RCA right coronary artery

Full size image
Table 1 Patient characteristics
Full size table

Table 2 lists the mean values of all measurements. None of the measured values (MLD, RefD, and LL) showed any obviously large differences between versions 6.0 and 7.3 (Fig. 4). Lim’s coefficient of orthogonal regression between versions of MLD, RefD, and LL was calculated to be 1.00 (95% CI 0.93, 1.08), 0.99 (95% CI 0.96, 1.02), and 0.96 (95% CI 0.87, 1.06), respectively. Bland–Altman plots of MLD, RefD, and LL showed that the measured values had no large systematic difference between versions in a series of analyses (Fig. 5). However, variation in the difference in MLD between versions was slightly larger for Analyst 2 than for Analyst 1. Intra-class correlation, which was a ratio of lesion variance to the whole variance, for MLD, RefD, and LL was 0.76, 0.96, and 0.59, respectively.

Table 2 Measurements by both analysts (mean ± SD)
Full size table
Fig. 4
figure 4

Scatter plots and correlation coefficients for each measurement read by version 6.0 against that read by version 7.3. Left, minimum lumen diameter; middle, reference diameter; right, lesion length

Full size image
Fig. 5
figure 5

Bland–Altman plots. Left, minimum lumen diameter; middle, reference diameter; right, lesion length. Difference is defined by the difference of measurement by ver. 6.0 and by ver. 7.3

Full size image

Table 3 presents the parameter estimates for the fixed and random effects in the model. The average difference between the two versions in MLD was −0.022 (95% CI −0.045, −0.0001) and the 95% confidence interval was within ±0.2 mm, which was the permissible range of difference between versions. Although all effects of the 95% confidence intervals in MLD and RefD were within ±0.2 mm, those in LL were calculated to be larger than ±0.2 mm.

Table 3 Parameter estimates for random-intercept analysis-of-variance models
Full size table

Table 4 presents estimates of the average difference between the two versions in MLD by subgroup analyses related to coronary location, history of myocardial infarction, history of coronary artery bypass graft, and diabetes mellitus. The estimates of all subgroup analyses showed negative values and the 95% confidence intervals of the estimates were within ±0.2 mm. Although the upper limit of 95% confidence interval of the difference at the straight lesions was less than zero, those at the curved lesions straddled zero.

Table 4 Subgroup analyses for the contrast of average MLD difference between versions from random-intercept analysis-of-variance models
Full size table

Discussion

This study demonstrated for the first time that MLD measured by the latest QCA system, i.e., the measurements of MLD with QAngio XA version 7.3, were statistically equivalent in average from those obtained with the previous version (version 6.0). The measured values of RefD also showed agreement between versions of QAngio XA. Since these average differences between the versions were considered small, results of the QAngio XA version 6.0 is transportable into those of the version 7.3 even if a QCA system is updated before a trial is completed. In Phase 3 trials or large-scale studies intended to evaluate the performance of coronary stents, clinical outcomes such as cardiac death, myocardial infarction, and target vessel revascularization are required as a primary end point. However, QCA variables such as late lumen loss are still one of the most important end points for pivotal clinical studies and Phase 2 trials [15–20]. Most validation studies of QCA systems were conducted in the early 1990s; recent QCA systems have not been validated in spite of changes to their algorithms.

The new version of QAngio XA differs from the old one in its operability improvements and algorithm changes. The major change in the algorithm is method of calculating MLD, as explained in the Introduction. It is important to say that the new QCA system can measure MLD without any overestimation. This may illustrate that the estimate obtained by using the equation of MLD measured by version 7.3 in Table 3 is significantly shorter than that obtained by version 6.0. Therefore, it is reasonable that the MLD difference between versions was found to be larger in straight vessels compared with curved vessels, because atherosclerotic plaque that causes stenosis tends to form at an inner curve and might not be corrected as shown in Fig. 1. Figure 6 shows a representative model in a curved vessel. However, the difference may not matter clinically, as in this study equivalence was not defined by the 95% confidence interval of the difference straddling zero, but instead by lying in a ±0.2 mm margin.

Fig. 6
figure 6

Representative model in a curved vessel

Full size image

Similarly, the change in algorithm may affect rater effects as inter-observer difference. If an analyst tends to correct the contour manually at the most severe stenosis to accommodate the measured value as visually estimated, inter-observer difference in version 7.3 might increase in comparison with version 6.0. To examine this variation, parameters of automated measurements need to be compared with adjustments by analysts.

The 95% confidence intervals of the mean difference in LL exceeded ±0.2 mm. Analysis of LL in particular tended to vary depending on the analyst, because analysts select the region of interest in consideration of the subsequent PCI procedure. In addition, the absolute values of LL were usually much larger than those of MLD and RefD. This means that the mean difference in LL between versions resulted from the variance of LL rather than the change in algorithm. We reason that ±0.2 mm is an inappropriate margin of LL, in contrast with MLD and RefD. However, it would be appropriate to confirm these values in real clinical trials in order to accommodate QCA measurements in the assessment of the clinical trials.

QAngio XA version 7.3, evaluated in this study, is a two-dimensional (2D) QCA system, whereas three-dimensional (3D) QCA systems have been developed to overcome the imaging limitations of 2D QCA systems [21]. Some studies have shown that 3D QCA systems provided accurate QCA measurements [22, 23]. An additional advantage of a 3D QCA system is thought to be its ability to more precisely predict the area of vessels and its effectiveness in analysing bifurcations. In the future, 3D QCA systems are expected to supplant 2D ones. However, 3D QCA systems can reconstruct 3D arteries using location information from biplane angiography systems, which may not be applicable to routine clinical practice. Therefore, the validation of 2D QCA systems continues to be necessary.

There are several limitations in this study. First, it was not designed to evaluate the reproducibility of each measurement. However, the programming of the QCA system is already known to be different between versions, and it is not surprising if the value itself is different between versions. The most important point to consider is whether the mean measurements between versions are appreciably different, because mean measurements have been used in several clinical studies. Therefore, the agreement of averages was thought to be a more practical measure. Second, this study was conducted on single vessels excluding bifurcation lesions. QAngio XA has a bifurcation algorithm, which showed in vivo robustness and reproducibility in two studies [24, 25] and was compared with other QCA systems [26]. Third, analysis of lesions was not repeated by a different analyst under the same conditions; strictly speaking, intra-observer difference was not investigated in this study.

Conclusion

The QCA parameters MLD and RefD of QAngio XA showed no major differences between versions 6.0 and 7.3.