Introduction

Recent years have witnessed the increasing diffusion of cardiac positron emission tomography (PET) for myocardial perfusion imaging. With tracers like 15O-water (H2 15O), 13N-ammonia (13NH3) and 82Rubidium (82Rb) [1–4], cardiac PET has gained acceptance in the detection of physiologic severity of coronary artery disease (CAD) and microcirculatory impairment, which both represent important information with regards to patient treatment and risk stratification [5–10].

One of the major advantages of cardiac PET is its ability to quantify myocardial blood flow (MBF) at rest and during stress in absolute terms (ml/g/min) and thus to calculate the myocardial flow reserve (MFR) [6, 11]. The availability of reliable tracer kinetic models and several software packages has enabled this kind of measurement quickly and accurately also in clinical routine practice [11, 12]. Non-invasive MBF and MFR quantification is currently considered part of a comprehensive approach to evaluate CAD, improving the identification of early-stage disease, microvascular dysfunction and multivessel CAD, and also improving the characterization of CAD burden [6, 13–15].

Recent studies have shown the incremental diagnostic value of quantitative MBF measurements as an adjunct to the visual analysis of cardiac PET images, in demonstrating the perfusion defect area [5, 6, 13–17].

With all perfusion tracers, MBF quantification with cardiac PET has always demonstrated good repeatability [18–20] and reproducibility [12, 19, 21–24]. Moreover, MBF and MFR measurements have shown great diagnostic performance in predicting significant CAD, with levels of sensitivity and specificity higher than 85 % [5, 17, 25, 26].

The goal of our study is to determine the accuracy of segmental MBF/MFR measurements assessed with quantitative cardiac PET imaging in the evaluation of CAD in a cohort of patients that participated to the Evaluation of Integrated Cardiac Imaging for the Detection and Characterization of Ischaemic Heart Disease (EVINCI) study [27]. The peculiarity of this study is that the diagnostic accuracy of absolute segmental MBF/MFR values was assessed blinded to patients’ clinical data and to visual analysis of PET images, using MBF/MRF cut-off values from literature, by two independent observers, in order to evaluate both the inter-observer reproducibility of clinical decisions and the objective performance of MBF/MFR segmental values independently of clinical information.

Methods

Patients

All patients included in this study participated in the EVINCI study, which prospectively enrolled patients with intermediate probability of CAD (20–90 %) [27]. All patients’ data were anonymised.

PET image acquisition and analysis

Image acquisition procedures were based on international guidelines [28], defined by the PET core lab centers and implemented throughout the consortium prior to the beginning of the study [27].

PET data were analyzed with the cardiac PET modeling software package (PCARDP) available in PMOD (PMOD version 3.5, PMOD Technologies, Zurich, Switzerland). MBF at rest and during stress was calculated from model fitting of the arterial input function and tissue time–activity curve, according to the correct method for each tracer; in particular, the Hermannsen et al. method was used for H2 15O PET processing [29], the de Grado et al. method for 13NH3 PET processing [22, 30] and the Lortie et al. method for 82Rb PET processing [31]. The left ventricular wall was divided into 17 segments: septal (apical septal, midinferoseptal, midanteroseptal, basal inferoseptal and basal anteroseptal), anterior (apical anterior, midanter- ior and basal anterior), lateral (apical lateral, midlateral and basal lateral), inferior (apical inferior, midinferolateral, midinferior, basal inferolateral and basal inferior) and apical [32]. MFR was calculated as the ratio of stress MBF to rest MBF.

All PET images were analyzed by two independent, expert observers (R.S., V.B.), who, at the time of PET analysis, were completely unaware of the patients’ demographic, clinical and instrumental data.

Myocardial perfusion PET studies were classified as positive or negative for significant CAD solely on the basis of segmental MBF/MFR measurements, without visual analysis of PET images and without considering global or regional values. Such classification was performed according to criteria derived from the most established literature for each tracer. Regarding H2 15O, PET studies were considered positive for significant CAD when more than one contiguous segment showed stress MBF comprised between 2.0 and 2.3 ml/g/min or if at least one segment showed stress MBF <2.0 ml/g/min [26]. For 13NH3, PET studies were considered positive when more than one contiguous segment showed stress MBF <1.85 ml/g/min or when at least one segment showed stress MBF < 1.85 ml/g/min and MFR < 2.0 [5, 17]. Regarding 82Rb, PET studies were considered positive if at least one segment showed MFR <1.7 [33, 34].

For H2 15O and 13NH3, these literature cut-offs were confirmed by receiver operating characteristic (ROC) curve analysis in the present study. In particular, the H2 15O cut-off was stress MBF < 2.05 ml/mg/min, and the 13NH3 cut-offs were stress MBF < 1.79 ml/g/min and MFR < 2.06. Therefore, the cut-offs derived from literature were used at a segment level for both the per-patient and the per-vessel analyses. The attribution of each segment to vascular territory was based on standard assumptions [32].

Coronary anatomy study

Coronary anatomy was evaluated by coronary computed tomography angiography (CCTA) or invasive coronary angiography (ICA). Both procedures were performed according to international guidelines [35], defined by the CCTA and ICA core lab centers and implemented throughout the EVINCI consortium prior to the beginning of the study [27].

The positivity or negativity of ICA and CCTA examinations were defined by the respective core-lab centers, according to the EVINCI protocol [27]. In brief, at ICA, coronary stenoses were considered significant when occurring in at least one major coronary artery (left main, left anterior descending [LAD], left circumflex [LCX] and right coronary [RCA] arteries) with a reduction in lumen diameter >50 % in the left main stem or >70 % elsewhere or between 30 % and 70 % with a fractional flow reserve (FFR) ≤ 0.80. At CCTA, the examinations were considered abnormal if at least one major coronary artery had a diameter stenosis >50 %.

Statistical analysis

Quantitative variables were expressed as mean ± standard deviation (SD), and categorical variables as percentages. Inter-observer agreement has been assessed as percentage of concordant PET results and using the kappa statistic. A P value of less than 0.05 was considered statistically significant. SPSS 22 software was used for statistical analysis.

Results

Description of the population

A total of 126 patients (Table 1) underwent myocardial perfusion imaging by PET and images were available and suitable for the analyses. Of them, 77 patients underwent cardiac PET with H2 15O, 42 patients with 13NH3 and 7 patients with 82Rb. These patients were included in the evaluation of inter-observer agreement.

Table 1 Characteristics of study population (N = 126)
Full size table

Of all 126 subjects, 5 patients had nor CCTA neither ICA results available, 23 patients had a positive CCTA result but no ICA result to confirm it, 13 patients had a negative CCTA result (no ICA) [36] and for the remaining 85 patients, ICA results were available. The final analysis on diagnostic accuracy included only patients for which cardiac PET measurements and either ICA result (N = 85) or a negative CCTA result (N = 13, in consideration of the high NPV of CCTA) [36] were available. Therefore, 98 patients (Table 1) were included in the final analysis on diagnostic accuracy, consisting of 73 subjects with H2 15O PET and 25 subjects with 13NH3 PET (no subject with 82Rb PET). Among them, obstructive CAD was diagnosed at ICA in 29 (30 %) patients, while non-obstructive CAD (at CCTA or ICA) was observed in 69 (70 %) patients.

Inter-observer agreement

Patient-based analysis

Overall, the inter-observer agreement on PET results was 90 %, with a kappa of 0.79, which indicates substantial agreement. In particular, the inter-observer agreement (and kappa coefficients) on H2 15O, 13NH3 and 82Rb PET results were, respectively, 86 % (k = 0.71), 95 % (k = 0.89) and 100 % (k = 1), indicating, respectively, substantial, almost perfect and perfect agreements. The observational data of the two observers are shown in Table 2.

Table 2 Patient-level observational data of the two observers for H2 15O, 13NH3 and 82Rb PET results
Full size table

Vessel-based analysis

The overall inter-observer agreement on PET results was 88 % (kappa of 0.74) in the RCA territory, 94 % (kappa of 0.84) in the LAD territory and 94 % (kappa of 0.85) in the LCX territory.

The detailed inter-observer agreement and kappa coefficients on H2 15O, 13NH3 and 82Rb PET results for all vascular territories are shown in Table 3.

Table 3 Inter-observer agreement and kappa coefficients on H2 15O, 13NH3 and 82Rb PET results in each vascular territory
Full size table

Diagnostic accuracy

Patient-based analysis

Segmental MBF PET measurements correctly identified 83 (85 %) patients with significant coronary artery stenoses at ICA or without significant stenoses either at CCTA or ICA, resulting in a global sensitivity of 86 %, a specificity of 84 %, a PPV of 69 % and an NPV of 93 % (Fig. 1).

Fig. 1
figure 1

Sensitivity, specificity, positive predictive value (PPV), negative predictive value (NPV) and accuracy on a per-patient basis of segmental quantitative PET perfusion parameters. Legend – 13NH3: 13N-ammonia; H2 15O: 15O-water; NPV: negative predictive value; PPV: positive predictive value

Full size image

Considering MBF measurements derived from H2 15O PET, segmental parameters correctly classified a total of 64 (88 %) patients with significant coronary stenoses at ICA or without significant stenoses either at CCTA or ICA, of which 21 of the 22 patients had positive findings and 43 of the 51 had negative findings. Quantitative analysis with H2 15O resulted in a sensitivity of 95 %, a specificity of 84 %, a PPV of 72 % and an NPV of 98 % (Fig. 1). Considering MBF derived from 13NH3 PET, segmental parameters correctly classified a total of 19 (76 %) patients with significant coronary stenoses at ICA or without significant stenoses either at CCTA or ICA, of which 4 of the 7 had positive findings, and 15 of the 18 patients had negative findings. Quantitative analysis with 13NH3 resulted in a sensitivity of 57 %, a specificity of 83 %, a PPV of 57 % and an NPV of 83 % (Fig. 1).

Vessel-based analysis

In vessel-based analyses, overall quantitative MBF had a sensitivity of 92 %, a specificity of 82 %, a PPV of 42 % and an NPV of 99 % for the detection of significant coronary stenoses in all major coronary arteries. Sensitivity, specificity and predictive values for quantitative parameters derived from H2 15O and 13NH3 in each vascular territory are shown in Table 4 and Fig. 2.

Table 4 Diagnostic accuracy of classification using segmental quantitative values for detecting CAD in each vascular territory
Full size table
Fig. 2
figure 2

Sensitivity, specificity, positive predictive value (PPV), negative predictive value (NPV) and accuracy on a per-vessel basis of segmental quantitative PET perfusion parameters. Legend – 13NH3: 13N-ammonia; H2 15O: 15O-water; LAD: left anterior descending artery; LCX: left circumflex artery; NPV: negative predictive value; PPV: positive predictive value; RCA: right coronary artery

Full size image

Discussion

The present study was conducted to evaluate the reliability and technical accuracy of PET-derived segmental MBF and MFR measurements to detect significant obstructive CAD in the particular and unfavourable condition of knowing solely the quantitative data, in order to test their performance by themselves and not filtered by clinical interpretations. PET perfusion values were calculated by two independent observers, blinded to demographic and clinical data, and examinations were classified as positive or negative according to cut-off values suggested throughout the literature [5, 17, 26, 33, 34].

Overall, classification of PET examinations from segmental perfusion values provided reliable results, with an inter-rater agreement from substantial to almost perfect at both patient- and vessel-level analyses. This result is in line with previous findings [5], and suggests that, even if segmental parameters may seem more variable and thus less reproducible than regional or global values, the diagnostic classification derived from them proved to be very reliable.

Segmental MBF/MFR measurements provided an accuracy of 85 % to classify patients with or without significant CAD. Besides, also the vessel-level analysis yielded good accuracy (83 %), with overall sensitivity and specificity of 92 % and 82 %, respectively.

In clinical practice, quantitative PET myocardial perfusion imaging has been demonstrated as a useful tool for the detection of significant CAD. Quantitative MBF values integrate and enhance the qualitative assessment of myocardial perfusion imaging [5, 9, 13, 14, 17, 25, 26]. Traditionally, MBF rest values are also corrected for blood pressure and cardiac rate at the moment of the examination, specifically in order to avoid overestimation in anxious or overly excited patients, with consequent undue decrease in MFR. Conversely, our study assesses the reliability and accuracy of MBF/MFR values by themselves and without any of these possible corrective factors. In spite of this, our results show levels of accuracy, sensitivity and specificity greater than 80 %, which mirrors previously published studies that demonstrated sensitivity and specificity ranging from 76 % to 96 %, and from 80 % to 90 %, respectively [5, 6, 25, 26, 37, 38]. Our findings suggest segmental values by themselves represent a robust and reliable measurement.

Our levels of sensitivity both on a per-patient (86 %) and on a per-vessel basis (92 %) are higher than that reported in other similar studies [5], suggesting that the use of segmental values could be suitable to highlight subtle alterations that, with global values, would be otherwise missed.

Among patients with significant stenosis at ICA, only 4 % were not correctly classified by quantitative perfusion parameters (1/73 for H2 15O PET, 3/25 on 13NH3 PET). On a per-vessel analysis, quantitative values did not correctly identify only 3 of the 37 significantly stenotic vessels (8 %).

Interestingly, the NPV is very high in both patient- and vessel-level analyses, confirming the important role of quantitative PET in ruling out obstructive CAD [26]. However, the PPV is rather low in all analyses, suggesting that the presence of a positive PET examination (according to the cut-offs in literature) does not necessarily stand for flow-limiting CAD. This finding is in line with previous studies [26], and reflects the well-known discrepancy between the impairment in myocardial perfusion and the presence of epicardial disease. Such a mismatch could not represent the presence of real false positive results, but more likely represents the effects of coronary microvascular disease and/or non critical atherosclerotic lesions on myocardial perfusion as measured by PET [15, 39]. It is well know that coronary stenosis (even with FFR measurement) and perfusion provide different and complementary information, which could be discordant [40–42].

One of the strengths of our study is the inclusion of subjects from the EVINCI study, which is a large multicenter European study designed to compare the diagnostic accuracy of noninvasive anatomic and functional imaging in identifying patients with significant CAD defined by ICA. PET examinations were performed with different PET perfusion tracers, thus enabling us to evaluate diagnostic performance with all available tracers, considered together and separately.

Another strength of the present study is the fact that all PET examinations have been evaluated by two independent observers that were unaware of both clinical and CTA/ICA information. This ensures that all measurements were performed objectively and no subjective interpretation has been applied to the results; thus, the accuracy values are related on the performance of only segmental MBF/MFR quantitative parameters.

This study has some limitations. The decision to not be aware of patients’ clinical and instrumental data and thus to not consider confounding factors could be a limitation of this study, leading us to be too conservative in the results. Previous studies have shown MBF is related to age, sex and several CAD risk factors; so, the choice to not consider these data in the interpretation could have lowered the accuracy values [26, 43]. Besides, the choice to not visually evaluate PET images could have limited the capacity to test the incremental value of quantitative MBF/MFR measurements over visual or semiquantitative evaluation. However, our goal was to determine the real reliability and performance of PET segmental quantitative parameters “per se”, without the added value of subjective clinician interpretation, and we found that even in this unfavourable situation the levels of accuracy remain high.

The selection of cut-off values from the literature is a critical point to be addressed. We choose, to the best of our knowledge, cut-off values derived from the most ascertained works in the literature for each tracer. In some cases (H2 15O, 13NH3), cut-off values derived from numerous populations could be retrieved [17, 26] and they were confirmed by post-hoc ROC curve analysis in the present studies. In other cases (82Rb), more variable data were found. We choose the most used threshold for CAD, which is MFR < 1.7 for positivity [33, 34], and not a threshold of stress MBF. However, even though we had chosen a stress MBF threshold [34], our results would not have changed. Naturally, since for the 82Rb patients we had no angiographic data available for comparison, in this particular group, the interobserver classification agreement does not imply an accurate diagnosis of obstructive CAD.

Another limitation could be the fact that 13 patients with negative CCTA included in this analysis were not referred to ICA. Actually, according to the EVINCI protocol, these patients should have had also negative functional stress tests for regional ischemia. All these patients had no regional perfusion defects at qualitative PET and also negative stress echocardiography or stress magnetic resonance results.

Moreover, the limited number of patients with positive ICA, especially in the subgroup who underwent 13NH3 PET, could be a potential limitation, and may have contributed to lower sensitivity and PPV. Finally, the vessel-level analysis has been conducted under the assumption of standard vascular territory distribution. This assumption could have led to less accurate assignment of segmental perfusion alteration to culprit vessels, given the morphologic variability of the coronary tree [44].

Conclusions

Quantitative measurements of MBF and MFR parameters with cardiac PET represent a valuable tool for clinical practice to better identify patients with functionally relevant coronary stenoses. Such measurements could have great impact on decision-making and on patients’ risk stratification. This is the first study to assess the performance of absolute quantitative perfusion parameters isolated from clinical information in order to assess their technical reliability. We demonstrated that the assessment of MBFs and MFR absolute values at a segment level lead to reliable and accurate identification of patients with significant coronary stenosis at ICA. This could represent a further step toward the use of quantitative PET myocardial perfusion imaging in clinical practice.