Abstract
Myocardial pathology segmentation in cardiac magnetic resonance (CMR) is an important step for patients suffering from myocardial infarction. In this paper, we present a cascaded framework with complementary information for infarcted and edema regions segmentation in CMR sequences. Specifically, instead of using all the three CMR sequences as joint inputs, we first use a 2D U-Net with balanced-Steady State Free Precession (bSSFP) cine sequence to segment the whole heart (left ventricle and myocardium) because bSSFP can capture cardiac motions and present clear boundaries. Then, we crop the whole heart as a region of interest (ROI). Finally, we segment the scar and edema regions in the late gadolinium enhancement (LGE) and T2 CMR sequence ROI. We evaluate the proposed method on MICCAI 2020 MyoPS testing set and achieve Dice scores 0.6283 ± 0.2772 for scar and 0.5419 ± 0.2406 for the combination of edema and scar, which is better than the inter-observer variation of manual scar segmentation (0.5243 ± 0.1578).
Access provided by Autonomous University of Puebla. Download conference paper PDF
Similar content being viewed by others
Keywords
1 Introduction
Quantitative assessment of myocardial viability is essential in the diagnosis and treatment management for patients suffering from myocardial infarction (MI). Cardiac magnetic resonance (CMR) is particularly used to provide imaging anatomical and functional information of heart, such as the late gadolinium enhancement (LGE) CMR sequence which visualizes MI, the T2-weighted CMR which images the acute injury and ischemic regions, and the balancedSteady State Free Precession (bSSFP) cine sequence which captures cardiac motions and presents clear boundaries. Combining these multi-sequence CMR data can provide rich and reliable information as well as morphological information of the myocardium [9].
One of the important tasks is to segment the myocardium into different regions, including normal myocardium, infarction and edema, from multi-sequence CMR dataset. Manual annotation is generally time-consuming, tedious and subjects to inter- and intra-observer variations. Thus, fully automatic segmentation method is highly desired in clinical practice. Figure 1 presents some images from different CMR sequences and the corresponding edema and scars annotations. It can be observed that the intensity appearances vary significantly among different sequences, and the both edema and scars have ambiguous boundaries and low contrast. Thus, it is very challenging to automatically segment them.
To the best of our knowledge, most of CMR segmentation related studies focus on left ventricle, right ventricle, and myocardium segmentation [1, 2, 10], little work has been done in the fully automatic cardiac pathology segmentation [4, 5, 8]. Zhuang [8] proposed a multivariate mixture model and maximum of log-likelihood framework for simultaneous registration and segmentation of multi-source CMR images, achieving a Dice score of 0.4779 ± 0.1855 for scars segmentation. Recently, Li et al. [5] proposed a new framework of scar quantification based on surface projection and graph-cuts framework, achieving a mean accuracy of 0.856 ± 0.033 and mean Dice score of 0.702 ± 0.071 for LA scar quantification.
2 Method
This paper focuses on myocardial scar and edema segmentation from the following three CMR sequences
-
C0 sequence; It is a balanced steady-state, free precession (bSSFP) cine sequence, which captures cardiac motions and presents clear boundaries;
-
DE sequence; It is late gadolinium enhancement (LGE) CMR sequence, which visualizes myocardial infarction (MI);
-
T2 sequence; It is T2-weighted CMR, which visualizes acute injury and ischemic regions.
One of the main challenges is how to combine these multi-sequence CMR data and exploit rich and reliable information regarding to the pathological as well as morphological information of the myocardium.
Motivated by the characteristics of different CMR sequences, we propose a cascaded framework for myocardial edema and scar segmentation, which can exploit the complementary information of the three CMR sequences. Figure 2 presents the whole pipeline of the proposed method. Specifically, the proposed method contains three stepsFootnote 1:
-
Step 1 (whole LV segmentation). Train a 2D U-Net [6] to segment the whole LV (including left ventricular blood pool and myocardium) from C0 sequence, because the heart boundary is clear in this sequence;
-
Step 2 (creating ROI). Crop LV region of interest (ROI) from DE and T2 sequence based on the segmentation results in step 1. In this way, the unrelated background can be excluded;
-
Step 3 (scar and edema segmentation). Train a new 2D U-Net to segment the scar and edema from DE and T2 sequences because the pathology is more clear in the two sequences. Specifically, DE and T2 sequences are combined as two channels and then input to the network.
3 Experiments and Results
3.1 Dataset and Training Protocols
Dataset. Three-sequence CMR from 45 patients [7, 8] are involved in this study. Specifically,
-
C0 sequence generally consists of 8 to 12 contiguous slices, covering the full ventricles from the apex to the basal plane of the mitral valve, with some cases having several slices beyond the ventricles. The typical parameters are as follows, TR/TE: 2.7/1.4 ms; slice thickness: 8–13 mm; inplane resolution: reconstructed into 1.25 \(\times \) 1.25 mm.
-
DE sequence consists of 10 to 18 slices, covering the main body of the ventricles. The typical parameters are as follows, TR/TE: 3.6/1.8 ms; slice thickness: 5 mm; in-plane resolution: reconstructed into 0.75 \(\times \) 0.75 mm.
-
T2 sequence generally consists of a small number of slices. For example, among the 35 cases, 13 have only three slices, and the others have five (13 subjects), six (8 subjects) or seven (one subject) slices. The typical parameters are as follows, TR/TE: 2000/90 ms; slice thickness: 12–20 mm; in-plane resolution: reconstructed into 1.35 \(\times \) 1.35 mm.
The number of training cases is 25, and the remained 20 cases are used for testing. During preprocessing, we apply z-score to separately normalize each sequence.
We employ nnU-Net [3] as the main network. Due to the fact that the CMR data has large slice thickness, 2D U-Net is more suitable in this task. During training, the patch size is \(112\times 112\) and batch size is 6. We apply five-fold cross validation in all experiments. Each fold is trained on a TITAN V100 GPU.
3.2 Five-Fold Cross Validation Results of the Whole LV Segmentation
Table 1 shows five-fold cross validation results for the whole LV segmentation, and Fig. 3 presents some examples of the segmentation results. It can be found that the segmentation results are quite accurate, where the average Dice score in each fold is more than 0.95. The high LV segmentation accuracy can insure that all the myocardial lesions (scar and edema) are included in the segmentation ROI. Thus, when we crop the LV ROI from DE and T2 CMR sequences based on the segmentation results.
3.3 Five-Fold Cross Validation Results of the Pathology Segmentation
Table 2 shows the five-fold cross validation results of scar and edema segmentation. We conduct two groups of experiments: only using DE CMR sequence and using both DE and T2 sequence. Results show that using two sequences can obtain better performance, especially for Edema + Scar, with up to 10% improvements in terms of Dice.
Figure 4 presents some examples of the scar and edema segmentation results. The boundaries of edema and scar are very unclear as show in Fig. 4-(a), which are extremely challenging. There are obvious errors in the segmentation results, which is in accordance with the relatively low Dice scores in Table 2.
3.4 Pathology Segmentation Results on Testing Set
Table 3 shows the quantitative segmentation results for each case in testing set. Some cases (e.g., myops_2204, myops_2215) obtain good segmentation performance for scar segmentation, with 0.8+ in Dice. However, some cases (e.g.., myops_2207, myops_2218) are failed with almost zero Dice. Figure 5 presents the box plots to visualize the quantitative results. It should be noted that the Dice of Edema + Scar is significantly worse than the Dice of Scar, indicating that the segmentation results of edema is much more worse than scar. Figure 6 presents some visualized segmentation results of edema and scar.
4 Conclusion
Myocardial pathology segmentation is a challenging task due to its unclear boundaries and low contrast in CMR sequences. In this paper, we designed a cascaded framework that enables to utilize the complementary informations in different CMR sequences. Experiments on MICCAI 2020 MyoPS testing dataset show that the proposed method can achieve better performance than the inter-observer variation.
Notes
- 1.
In step 1 and step 3, the networks are trained end-to-end, while the whole framework is not end-to-end.
References
Bernard, O., et al.: Deep learning techniques for automatic MRI cardiac multi-structures segmentation and diagnosis: is the problem solved? IEEE Trans. Med. Imaging 37(11), 2514–2525 (2018)
Chen, C., et al.: Deep learning for cardiac image segmentation: a review. Front. Cardiovas. Med. 7, 25 (2020)
Isensee, F., Jäger, P.F., Kohl, S.A., Petersen, J., Maier-Hein, K.H.: Automated design of deep learning methods for biomedical image segmentation. arXiv preprint arXiv:1904.08128 (2020)
Li, L., Weng, X., Schnabel, J.A., Zhuang, X.: Joint left atrial segmentation and scar quantification based on a dnn with spatial encoding and shape attention. arXiv preprint arXiv:2006.13011 (2020)
Li, L., et al.: Atrial scar quantification via multi-scale CNN in the graph-cuts framework. Med. Image Anal. 60, 101595 (2020)
Ronneberger, O., Fischer, P., Brox, T.: U-net: convolutional networks for biomedical image segmentation. In: Navab, N., Hornegger, J., Wells, W.M., Frangi, A.F. (eds.) Medical Image Computing and Computer-Assisted Intervention, pp. 234–241 (2015)
Zhuang, X.: Multivariate mixture model for cardiac segmentation from multi-sequence MRI. In: Ourselin, S., Joskowicz, L., Sabuncu, M.R., Unal, G., Wells, W. (eds.) MICCAI 2016, Part II. LNCS, vol. 9901, pp. 581–588. Springer, Cham (2016). https://doi.org/10.1007/978-3-319-46723-8_67
Zhuang, X.: Multivariate mixture model for myocardial segmentation combining multi-source images. IEEE Trans. Pattern Anal. Mach. Intell. 41(12), 2933–2946 (2019)
Zhuang, X., Li, L.: Multi-sequence CMR based myocardial pathology segmentation challenge (2020). https://doi.org/10.5281/zenodo.3715932
Zhuang, X., et al.: Evaluation of algorithms for multi-modality whole heart segmentation: an open-access grand challenge. Med. Image Anal. 58, 101537 (2019)
Acknowledgement
This work is supported by the National Natural Science Foundation of China (No. 91630311, No.11971229). The author highly appreciates the organizers of Myocardial pathology segmentation combining multi-sequence CMR (MyoPS 2020) for their public dataset and organizing the great challenge.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2020 Springer Nature Switzerland AG
About this paper
Cite this paper
Ma, J. (2020). Cascaded Framework with Complementary CMR Information for Myocardial Pathology Segmentation. In: Zhuang, X., Li, L. (eds) Myocardial Pathology Segmentation Combining Multi-Sequence Cardiac Magnetic Resonance Images. MyoPS 2020. Lecture Notes in Computer Science(), vol 12554. Springer, Cham. https://doi.org/10.1007/978-3-030-65651-5_15
Download citation
DOI: https://doi.org/10.1007/978-3-030-65651-5_15
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-65650-8
Online ISBN: 978-3-030-65651-5
eBook Packages: Computer ScienceComputer Science (R0)