Introduction

Assessment of ventricular performance has become a key parameter to appraise cardiac disease severity, to evaluate therapeutic response and to risk stratify patients [1]. Whereas ejection fraction (EF) is a well-established and generally accepted parameter to appreciate global ventricular systolic performance, it fails to provide data with regard to regional contractility and lacks sensitivity to depict subtle alternations in ventricular function [2]. Alternatively, myocardial strain, defined as the percentage change of myocardial dimension in a specific direction, is a reliable parameter to assess myocardial performance and has shown to be an early marker of systolic dysfunction [3, 4].

Although speckle tracking echocardiography is the reference to assess myocardial strain, in recent years cardiovascular magnetic resonance (CMR) feature tracking (FT) has become an appealing technique for myocardial strain assessment as well [5,6,7]. Cine steady-state free-precession images, which are part of a standard study CMR protocol, can be used for CMR-FT based myocardial strain analysis and quantification. An advantage of this approach is that the technique does not require additional CMR sequences, such as myocardial tagging, but can be applied to standard cine steady-state free-precession exams, thereby leaving the potential to retrospectively analyse strain in patients in whom no specific strain image acquisition sequence was performed [8]. Currently, several vendors provide dedicated CMR-FT analysis software. Although several papers have been published providing data with regard to intra- and inter-observer reproducibility of the different software packages, to our knowledge only two studies so far have focused on the inter-vendor agreement in selective populations, i.e. healthy volunteers undergoing dobutamine stress CMR, and segmental RV strain analysis in patients with overt or preclinical arrhythmogenic right ventricular dysplasia/cardiomyopathy [9,10,11,12,13,14,15,16]. Therefore, it was our aim to evaluate intra-/inter-observer, and inter-vendor agreement of four commercially available software packages including subjects with a wide variation in cardiac performance. For the present study, we used TomTec, Medis, Circle cvi42 and Segment Medviso. For the purposes of this study, the different software tools are referred to as “TomTec”, “Circle”, “Medis” and “Medviso”. TomTec, Medis and Circle rely on optical flow technology to discriminate ventricular boundaries, while Medviso uses a non-rigid, elastic algorithm for image registration and segmentation [9, 17,18,19].

Methods

Study population

From the CMR database, we selected a cohort of 45 patients (all CMR studies were performed in 2016 or later). A first group (n = 15) yielded normal CMR findings, i.e. LV ejection fraction ≥ 50%, end-diastolic LV wall thickness ≤12 mm and no myocardial enhancement at late gadolinium enhancement imaging (control group). A second group consisted of 15 patients with a reperfused acute ST-elevation myocardial infarction (infarct group). A third group consisted of 15 patients with a dilated cardiomyopathy (DCM group) (see Table 1). More information with regard to the specifics of the three groups can be found in the Electronic supplementary material (ESM). Only CMR studies were selected yielding a good to excellent image quality, as judged by an experienced reader >10 years of CMR experience. All subjects were in sinus rhythm at the time of the CMR exam. The study protocol conformed to the Declaration of Helsinki and the study was approved by the ethical committee of our hospital (S51789). Because of the retrospective nature of this study, patient informed consent was waived.

Table 1 Clinical characteristics and CMR findings for the three groups
Full size table

Cardiovascular magnetic resonance acquisition

All CMR studies were performed on a 1.5-T unit (Ingenia; Philips Healthcare, Erlangen, Germany) by using commercially available CMR imaging software, electrocardiographic triggering, and a cardiac-dedicated phase-array coil. For assessment of LV dimensions and function steady-state free-precession breath-hold cine images were acquired in the following orientations: vertical and horizontal long-axis, short-axis and left ventricular (LV) outflow tract view. Standard parameters were: repetition time/echo time, 3.6/1.8 ms; sense factor, 2, flip angle, 60°; section thickness, 8 mm; matrix, 160 × 256; field of view, 300 mm; pixel size, 1.6 × 1.6 mm; number of phases, 30; phase percentage, 67%. The vertical long-axis was determined on the transverse images by positioning an image plane connecting the middle of the mitral valve with the LV apex. On the vertical long-axis, the same anatomical landmark points were used to define the horizontal long-axis plane. The cardiac short-axis was defined on the horizontal long-axis using an image plane perpendicular to the interventricular septum. Care was taken to position the most basal short-axis slice at end-diastole exactly through the mitral valve ring. The set of short-axis images encompassed the left ventricle entirely. Between slices a gap of 2 mm was used. Per breath-hold, two short-axis CMR slices were acquired. For the late gadolinium enhancement studies, a dose of 0.15 mL of gadobutrol (Gadovist; Bayer, Leverkusen, Germany) per kilogram of body weight, was administered, and we used breath-hold T1-weighted three-dimensional contrast-enhanced phase-sensitive inversion-recovery (PSIR) gradient-echo sequence in cardiac short-axis, vertical long-axis and horizontal long-axis view (TR, 4.8 ms; TE, 2.3 ms; flip angle, 15°; PSIR flip angle, 5°; acquisition matrix, 168 × 159 reconstructed to 256 × 256 × 20; parallel imaging acceleration factor, 2.5 and 2; fold-over suppression, 2 × 25 mm; BW, 365 Hz; FOV, 320 × 320 × 100 mm; pixel size; 1.9 × 2.0 × 5 mm; reconstructed pixel size 1.3 × 1.3 × 5.0 mm). PSIR-images were acquired 10-15 min post-contrast administration. To reduce total acquisition time, we performed cine imaging in SA and LV outflow tract after contrast administration using the available time before start of late gadolinium enhancement imaging.

Ventricular volumes and function

All images were sent to an off-line workstation and analysed using commercially available software package (ViewForum; Philips Medical Systems, Best, Netherlands) by a single observer with more than 2 years of experience in CMR (MBP). LV volumes, mass and function were measured using manual planimetry of the endocardial and epicardial borders from the short-axis set at end-diastole and end-systole (see Table 1 and ESM). The papillary muscles were considered part of the LV cavity. In the dilated cardiomyopathy group, presence of ventricular asynchrony and apical rocking were visually scored. A similar visual approach was used to score the presence of left valvar disease. Late gadolinium enhancement images were analysed for the presence of myocardial enhancement, and the number of segments was quantified using the 17-segment classification as defined by the American Heart Association [20]. In the infarct group the location of enhancement was determined as anteroseptal, inferior or lateral.

LV myocardial feature tracking analysis

For myocardial feature tracking analysis cine images were uploaded in the different CMR-FT software packages, i.e. (1) TomTec Arena (ver. 1.3.0.124), (2) QStrain Medis (ver. 2.0.12.0), (3) Circle cvi42 (ver. 5.5) and (4) Segment Medviso (ver. 2.0 R4988). In a first step, in cardiac short-axis, a basal, mid and apical LV slice was defined for each patient which was used to calculate global circumferential and radial myocardial strain (GCS and GRS, respectively). Global long-axis strain (GLS) calculation was performed in vertical and horizontal long-axis and in the LV outflow tract view. The analysis of cine images was performed as recommended by the different vendor user manuals. In cardiac short-axis all software packages required drawing of an endo- and epicardial contour. In long-axis direction, TomTec and Medis required only an endocardial contour while Circle and Medviso required drawing of an endo- and epicardial contour. According to the user manual, Medviso and Circle required drawing of LV contours on an end-diastolic image, Medis and TomTec drawing of contours on an end-systolic image with subsequent tracking of the contours over the remainder of the images. If tracking was suboptimal, Medviso allowed adaptation of the contours on the (initial) end-diastolic image whereas the other packages allowed end-diastolic and end-systolic contour correction before re-running the tracking analysis. Moreover, Circle allowed if the strain borders visible in the overlay were suboptimal to draw contours in a different cardiac phase, before re-running the analysis. All CMR studies were analysed by two skilled CMR readers (M.B.P. and D.C.), one reader (M.B.P.) repeated the analyses respecting an interval of one week between repeated analyses. Before starting the CMR-FT analyses in the study group, both readers performed together analyses with the different software packages in five subjects not belonging to the study group. To compare the practical performance of the different software packages, we measured the total time employed, the number of mouse clicks (Mousotron, Blacksun software) and the number of tracking corrections needed in short-axis and long-axis direction. These analyses were performed in ten randomly selected subjects.

Statistics

Summary statistics for continuous variables are presented as mean ± standard deviation (SD) or medians with interquartile ranges (IQR), as appropriate. Categorical variables are presented as numbers with percentages in parentheses. Differences in clinical characteristics and CMR findings between the three groups were tested with one-way analysis of variance (ANOVA) with post hoc unpaired t-tests with Bonferroni correction. Differences in post-processing time and in global myocardial strain values between the different vendors were tested with repeated measurements ANOVA with post hoc paired t-tests with Bonferroni correction. Because there is no gold-standard measure of global myocardial strain, the measurements of each vendor were compared with the average global myocardial strain of the four vendors using Bland-Altman analysis with 95% limits of agreement. Differences between vendors were analysed with one-way ANOVA with post hoc paired t-tests with Bonferroni correction. For this analysis, the results of the first reading of the first reader (M.B.P.) were used. Inter-observer (two readers) and intra-observer (two readings) reproducibility for the different vendors were assessed by using the absolute mean error, intraclass correlation coefficients (ICCs), and the coefficient of variation (CV). Reproducibility was defined as poor (ICC <0.4), good (0.4< ICC <0.75) or excellent (ICC >0.75) [21]. The CV was calculated as the SD of the differences between repeated analyses normalised to the mean. A p value <0.05 was considered to indicate a statistically significant difference. Statistical analyses were performed by using R Statistical Software v.3.4.0 (Foundation for Statistical Computing) and SPSS Statistics v.20.0 (IBM Corp).

Results

Mean age of the study population was 55 ± 14 years (range, 25-79 years; 58% males) with a large range of LV volumes and LV EF (Table 1). GLS, GCS, and GRS values were -13.9% ± 5.4% (range, -23.8% to -3.9%), -12.2% ± 5.8% (range, -25.1 to -1.0%), and 32.0% ± 14.7% (range, 3.6-67.8%), respectively. Amongst vendors, we found significant differences with regard to the time employed and number of mouse clicks needed to analyse LV strain in short- and long-axis direction (Table 2). TomTec and Medis were significantly faster and required a lower number of mouse clicks than the two other vendors (p < 0.01). Also, the number of tracking corrections differed amongst vendors. Overall, Circle needed most contour corrections for adequate tracking. Medviso needed initial tracking corrections in particular on long-axis cine images at the basal LV segments. Figure 1 shows the strain overlay at end-diastole and peak-systole in a control subject by the different vendors (see also online video 1-4). In Fig. 2 is shown an example of the strain-time curve obtained with the different CMR-FT algorithms in a control subject, an infarct, and DCM patient.

Table 2 Post-processing characteristics for each software obtained in ten randomly selected study subjects
Full size table
Fig. 1
figure 1

Midventricular short-axis in a normal subject showing the endocardial and epicardial contours and strain overlay at end-diastole and peak-systole for the different vendors

Full size image
Fig. 2
figure 2

Strain versus time curves obtained with the four different software packages in a normal subject, an DCM and infarct patient. Medviso (green), Circle (blue), TomTec (purple) and Medis (red). All values in the y-axis represent percentages (%). DCM dilated cardiomyopathy, GCS global circumferential strain, GLS global longitudinal strain, GRS global radial strain

Full size image

Inter-vendor reproducibility

The mean global strain values per vendor are summarised in Fig. 3. Bland-Altman analyses with 95% limits of agreement confirmed no significant bias between the different vendors and the mean of all vendors (Fig. 4). ANOVA showed significant differences between vendors for GRS (p < 0.001) and GLS (p = 0.018), not for GCS (p = 0.379). For GRS, post-hoc analysis showed significant differences between Medviso on the one hand and TomTec and Medis on the other hand, and between Circle and TomTec (Fig. 3). For GLS, post-hoc analysis showed significant differences between Medviso on the one hand and TomTec and Medis on the other hand (Fig. 3).

Fig. 3
figure 3

Boxplots show global circumferential strain, global longitudinal strain and global radial strain per software package. One-way ANOVA showed significant differences between vendors for average GRS and GLS. GCS global circumferential strain, GLS global longitudinal strain, GRS global radial strain

Full size image
Fig. 4
figure 4

Bland-Altman plots show the inter-vendor reproducibility of global circumferential strain (a-d), global longitudinal strain (e-h), and global radial strain (i-l). DCM dilated cardiomyopathy, GCS global circumferential strain, GLS global longitudinal strain, GRS global radial strain

Full size image

Intra-observer and inter-observer reproducibility

No significant bias was found for both intra- and inter-observer reproducibility for all four software packages (Figs. 5 and 6, Table 3). ICC for inter- and intra-observer reproducibility ranged from 0.828 to 0.991 and 0.902 to 0.997, respectively. Medviso showed the lowest CV ranging from 2.8 to 5.0% for intra-observer reproducibility and 4.0 to 5.2% for inter-observer reproducibility. The other vendors showed higher CV values. In particular GRS measured by TomTec and Medis showed high CV values (up to 28.8%).

Fig. 5
figure 5

Bland-Altman plots show the intra-observer reproducibility of global circumferential strain (a-d), global longitudinal strain (e-h) and global radial strain (i-l). DCM dilated cardiomyopathy, GCS global circumferential strain, GLS global longitudinal strain, GRS global radial strain

Full size image
Fig. 6
figure 6

Bland-Altman plots show the inter-observer reproducibility of global circumferential strain (a-d), global longitudinal strain (e-h) and global radial strain (i-l). DCM dilated cardiomyopathy, GCS global circumferential strain, GLS global longitudinal strain, GRS global radial strain

Full size image
Table 3 Inter- and intra-observer reproducibility for global circumferential strain, global longitudinal strain and global radial strain
Full size table

Discussion

As CMR-FT software is increasingly used to assess myocardial strain as part of a comprehensive CMR exam, it is essential to appraise how accurate and reproducible the different CMR-FT software packages are to assess myocardial strain. As we had no “gold standard” (e.g. calibrated cardiac phantom) to assess accuracy, only the reproducibility issue could be addressed which, however, is crucial when it comes to repeated studies or when comparing strain values obtained by different vendors, readers or readings. Our study results show significant differences for global myocardial strain measurements between vendors, in particular for assessment of global longitudinal and radial strain. With regard to intra- and inter-observer reproducibility, overall all packages perform very well (ICC > 0.75) [21]. While Medviso is superior with the highest ICC and lowest CV values, some vendors suffer to reproducibly measure global radial strain.

Currently, several vendors offer CMR-FT dedicated software, but only a few studies have been reported on the inter-vendor agreement [9, 16]. For the current study, we compared four CMR-FT algorithms commercialised by TomTec, Medis, Circle and Medviso. The first three vendors use optical flow technology for CMR-FT. Following contouring of a first, reliable image, the traced contours are then followed in time by searching the same features in the following frames. The tracked features can be the myocardial-cavity boundary or anatomical elements that are different along the tissue [6]. Both TomTec and Medis use Advanced Medical Imaging Development (AMID) software for their CMR-FT program. The fourth vendor, Medviso, uses a non-rigid, elastic algorithm for segmentation and registration for myocardial strain quantification [17,18,19]. Although Bland-Altman analyses showed no bias between the different vendors and the mean of all vendors, ANOVA showed significant differences in GRS and GLS between vendors. For GRS, Medviso yielded significantly lower values than TomTec and Medis, while Circle yielded lower values than TomTec (Fig. 3). A similar finding was noted for GLS with lower strain values for Medviso compared to TomTec and Medis. With regard to the inter- and intra-observer agreement, all software packages perform excellent with regard to circumferential and longitudinal strain measurement, but some of them (Medis, TomTec) suffer to reproducibly measure radial strain. A clear-cut explanation for these observations is not available, but as Medis and TomTec use a similar algorithm, it is plausible that this algorithm performs less well for radial strain measurement. Previous studies on CMR-FT reproducibility—mainly using TomTec software—showed similar findings [9, 15, 22]. Most likely, small differences in the manual contouring of endo/epicardial borders between readers or readings has a major impact on radial strain values, whereas the impact on circumferential/longitudinal strain reproducibility is much smaller. Here, Medviso outperforms clearly the other packages in terms of inter- and intra-observer reproducibility, with excellent ICC values and CV values of 5% or lower. This suggests that a non-rigid algorithm is likely superior to the optical flow methods to reproducibly trace contours over time, independent of the reading/reader, as well as the level of experience of the reader as recently reported by Morais et al. [19] comparing the impact of observer experience on CMR-FT reproducibility using a non-rigid algorithm.

We found significant differences in total analysis time between vendors—the shortest for TomTec and Medis—which is likely related to the design of the contouring program (e.g. number of clicks needed), the number and need for contour correction, and to a large extent to the complexity of the used CMR-FT algorithm (e.g. blood-myocardial border tracing versus non-rigid, elastic registration algorithm). TomTec and Medis do not need epicardial contouring on long-axis and require contour tracing of short-axis images at end systole yielding good tracking propagation towards end diastole, with only minimal corrections. In contrast, Medviso and Circle necessitate both endocardial and epicardial contouring at end diastole rather than end systole, substantially increasing the number of mouse clicks. The Circle software often needed substantial contour adjustments, while Medviso needed regularly corrections at the basal LV borders in long-axis direction. Finally, the time needed to perform the strain analysis is significantly longer for Medviso (>50 s on long-axis and >100 s on short-axis) compared to the other software packages (all <10 s).

Limitations

To assess the reproducibility of different CMR-FT packages, and in contrast to most previous studies, we selected a cohort of patients with a wide range of cardiac performances, including patients with normal findings at CMR (‘control group’) as well as patients with focal pathology (‘acute myocardial infarction group’) and diffuse pathology (‘dilated cardiomyopathy group’). The control group does not necessarily imply that this group can be considered as a normal group as all subjects were referred for magnetic resonance imaging (MRI) with complaints (see ESM). Moreover, we cannot extrapolate our findings to other cardiac pathologies such as patients presenting thick-walled left ventricles (e.g. hypertrophic cardiomyopathy, cardiac amyloidosis). Kuetting et al. [12] reported, using TomTec software, lower myocardial strains and impeded reproducibility on post-contrast cine images as a result of the blood-myocardium contrast. In our routine clinical setting, we perform part of the cine studies post-contrast administration. The rationale is twofold, first to shorten total imaging time using the time between intravenous contrast administration and LGE imaging to perform cine imaging in short-axis and LV outflow tract, and second because of the improved tissue characterisation of contrast-enhanced cine images in the presence of focal myocardial pathology such as acute myocardial infarction [16, 23, 24]. As our study was not designed to address this issue, we cannot appropriately evaluate the impact of contrast administration on strain values and reproducibility by the different vendors. In the present study, we reported only on global myocardial strain. As assessment of segmental strain is important as well, certainly in the presence focal pathology, studies comparing this issue amongst vendors is crucial too. A recent study by Bourfiss et al. [16] studying RV myocardial strain in subjects with preclinical presentation of arrhythmogenic right ventricular cardiomyopathy/dysplasia with a similar set-up (comparing four vendors) showed that segmental strain values are not interchangeable between vendors. Similarly, recent work in the field of speckle tracking echocardiography in patients with a history of previous myocardial infarction has shown substantial bias in segmental strain calculation amongst vendors [25, 26]. Finally, to fully appreciate the reproducibility of strain measurements, not only the different CMR-FT tracking software algorithms should be compared but also the impact of the source images should be addressed comparing the different CMR vendors, such an initiative was recently realised in the field of speckle tracking strain imaging [27].

Conclusions

In conclusion, comparing four commercially available software package for CMR-FT analysis in a cohort of subjects with a wide range of cardiac performances, we found that global myocardial strain values, in particular longitudinal and radial strain values, are not interchangeable between vendors, and that inter-observer and intra-observer reproducibility for GRS remains challenging for some vendors.