Keywords

1 Introduction

Cardiovascular diseases (CVDs) are the leading cause of death in the world [1]. Among the common CVDs, myocardial infarction (MI) is a specific cardiovascular disease. The state of the heart after myocardial infarction receiving revascularization requires careful evaluation of the myocardial segment’s functionality. It requires assessment of myocardial infarction and myocardial no-reflow (persistent microvascular obstruction area) regions using magnetic resonance imaging (MRI). This is often done using a delayed enhancement-MRI (DE-MRI), which involves an MRI examination following several minutes of a contrast agent injection. Accurate and automatic patient classification is profoundly essential in this scenario. Moreover, accurate and automatic segmentation of the clinically essential regions such as myocardial infarction and myocardial no-reflow can help to assess tissue functionality and thereby make more accurate treatments based on the severity of the disease. Developing an accurate and automatic cardiac image segmentation framework is a challenging task due to low contrast boundary regions, different sizes and shapes of targets, and respiratory motion. Indeed, variations in image acquisition settings in different clinical setups are other challenges in developing accurate and robust medical image segmentation methods [2].

Recently, deep learning approaches using convolutional neural networks have provided the state of the art results in medical image analysis tasks. They have successfully been applied in classification and segmentation tasks. In this paper, we propose a deep learning framework for automatic and accurate evaluation of myocardial infarction from clinical information with and without the delayed-enhancement cardiac-MRI. The proposed framework has two main applications. The first application is to automatically detect the myocardial contours, the infarcted area, the permanent microvascular obstruction area (no-reflow area), and segments the left ventricular cavity from a short-axis DE-MRI. The second application is to automatically classify a given case into normal or pathological using the provided clinical information with or without DE-MRI. The provided clinical information includes sex, age, overweight, arterial hypertension, diabetes, familial history of coronary artery disease, Troponin, Killip Max, ejection fraction, ventricular natriuretic peptide, and the ST segment.

We evaluated our method on a MICCAI 2020 challenge dataset, named automatic Evaluation of Myocardial Infarction from the Delayed-Enhancement Cardiac MRI (EMIDEC) [3]. The experimental results show that the proposed method can be used to automatically evaluate the myocardial infarction from cardiac DE-MRI as well as from clinical information. Moreover, the proposed method can accurately segment the anatomical structures such as the myocardium and the left ventricular cavity.

2 Proposed Framework

Segmentation. To segment the anatomical and pathological (if present) structures from DE-MRI, we proposed a new deep learning framework (Fig. 1). It employed two encoder-decoder style convolutional neural networks [4]. The first network, the anatomical network, was used to segment the anatomical structures such as the myocardium and the left ventricular cavity. The second network, the pathological network, was used to segment the areas such as the myocardium infarction, the no-reflow, and the normal or healthy myocardium regions. The segmentation results from the pathological network are then masked by the segmented myocardium region from the anatomical network. It enabled us to constrain the segmentation results of the pathological network to be inside the myocardium area. However, note that the anatomical and pathological networks were trained separately. The segmented myocardium area from the anatomical network was the final myocardium segmentation. The predicted segmentation from the anatomical structure segmentation network and the masked pathological areas from the pathological network are then merged to yield the final four label segmentation output.

The encoder-decoder networks’ building block consists of a repeated application of 2D \(3\) \(x\) \(3\) convolution, exponential linear unit (ELU) activation function, batch normalization followed by squeeze-and-excitation network [5], and a \(2\) \(x\) \(2\) max pooling operation with stride 2 for down-sampling. Starting with 32 feature maps, after each max-pooling layer the feature channels are then doubled. The up-sampling part consists of \(2\) \(x\) \(2\) up-convolution and concatenation layer, followed by repeated 2D \(3\) \(x\) \(3\) convolution, exponential linear unit (ELU) activation function, batch normalization, and squeeze-and-excitation network [5]. The output 2D convolutional layer activation function was softmax for both pathological and anatomical networks. Each network was trained by calculating the average of Dice and cross-entropy loss.

Fig. 1.
figure 1

The schematic representation of the proposed segmentation framework. The anatomical network is used to segment the left ventricular cavity and the myocardium. The pathological network is used to detect the pathological areas (myocardium infarction (MI) and no-reflow) and the healthy or normal myocardium regions. The red, green, and blue colors, respectively, show the left ventricular cavity, the myocardium, and the myocardium infarction. (Color figure online)

Full size image

Classification. The classification task revolves around the clinical information for each patient. The proposed approach used a Support Vector Machine (SVM) with a linear kernel and a hinge loss [6] as an initial feature selection step. The selected features were then passed into another SVM with a radial basis function kernel to refine the separation boundary between the two classes (i.e., normal or pathological). The cascaded SVM was designed to remove redundant features at the first step and provide robust features for final classification in the second step. Moreover, classification using the DE-MRI was performed from the segmentation results by checking if the segmented areas consist of either the myocardium infarction or the no-reflow. If a given case consists of pathological areas in two or more slices, it was considered pathological otherwise normal. The classification pipeline is shown in Fig. 2.

Fig. 2.
figure 2

The schematic representation of the proposed classification framework using the provided clinical information and cardiac DE-MRI.

Full size image

Implementation Details. The proposed framework was implemented on an i7 computer with 32 GB RAM and a dedicated GPU (NVIDIA TITAN Xp, 12 GB). Python, Keras API with Tensorflow backend, was used to implement the method. We trained the model using the ADAM optimizer for 500 epochs with a learning rate of \(10^{-3}\) and early stopping criteria of 200 [7]. All convolutional neural networks’ parameters were initialized using He initialization [8]. Moreover, other hyper-parameters values, such as the ADAM optimizer parameters, were Keras’ default values.

3 Experimental Setup and Results

Dataset and Evaluation Criteria. To evaluate the proposed method, we selected the EMIDEC MICCAI 2020 challenge dataset [3]. It consisted of 100 training DE-MRI images, of which 67 are with pathological and 33 normal cases. The 50 held out testing data consisted of 33 pathological conditions and 17 normal cases. The training dataset was divided randomly into 85 training and 15 validation cases. The validation cases were composed of 10 pathological and five normal cases. As the provided EMIDEC dataset was with different image resolutions, we resized it to 256x256 before feeding it to the network. Each given case was also pre-processed by zero-centering the intensity values and normalizing it by the standard deviation. Although the proposed segmentation networks are in 2D, we evaluated the results in 3D after stacking each predicted 2D images per case. Thus, the evaluation was in 3D using the Dice index (DSC), Hausdorff distance (HD), and relative volume difference (RVD) for the segmentation task. For the classification task, we used the accuracy metric.

Experimental Results. Experimental results show that the proposed method can segment the left ventricular cavity and the myocardium cardiac structures with an average Dice index of 0.93 and 0.84, and an average 3D HD value of 6.5 mm and 8.9 mm, respectively. Quantitative segmentation results are shown in Table 1. The proposed SVM-based classification method yielded a classification accuracy of 80% using only the clinical information in 5-fold cross-validation. Moreover, our method based on the segmented pathological areas from the pathological network classifies with 93.3% accuracy using DE-MRI only. However, although the proposed method detects the pathological areas, it appeared to under segment them yielding less overlapping area ratio. For example, it yielded an average Dice index of 0.4 and 0.67 and an average relative volume difference of 0.242 and 0.375 for the myocardial infarction and no-reflow, respectively.

Table 1. Quantitative segmentation results on the EMIDEC dataset (15 validation sets). Values are expressed in mean ± standard deviation (std), minimum (Min.), and maximum (Max.).
Full size table

The proposed method showed promising performance to segment the anatomical structures such as the myocardium and the left ventricular cavity from DE-MRI. To extend this approach, shape prior based deep learning methods could help constrain the segmentation of the anatomical structures [9,10,11]. Post processing methods using convolutional auto-encoders [12, 13] could also constrain the segmentation process by guaranteeing the correct shape of the anatomical structures. Moreover, the clinical information could be used to constrain the segmentation process of the pathological areas.

Fig. 3.
figure 3

Examples of segmentation results on the EMIDEC validation set. [Column 1] Input DE-MRI; [Column 2] Predicted left ventricular cavity (LV) and myocardium (MYO); [Column 3] Predicted normal MYO areas and myocardium infarction (MI) areas; [Column 4] All predicted structures including the LV, MYO, and MI; [Column 5] Ground truth level from expert. The red, green, and blue colors respectively show the LV, the MYO, and the MI. (Color figure online)

Full size image

4 Discussion and Conclusion

In this paper, we proposed a deep learning framework to automatically segment the left ventricular cavity and the myocardium regions from DE-MRI. We also proposed a deep learning framework to automatically detect pathological areas, particularly the myocardium infarction and the no-reflow areas, from the DE-MRI. Moreover, the classification of a given case into pathological or normal was performed using the provided clinical information as well as using the DE-MRI.

Experimental results, obtained using the EMIDEC MICCAI 2020 challenge dataset, revealed that the proposed framework can accurately segment the heart structures. Moreover, our method can detect pathological areas. Besides, the classification of a given case can be performed using either clinical information or DE-MRI, which could benefit practitioners in assessing the state of the heart. However, the classification accuracy from DE-MRI seems to outperform the approach using only the clinical information. Next, shape prior based methods will be explored to constrain the segmentation process of anatomical structures such as the myocardium and the left ventricular cavity. The clinical information could also be used to constrain the pathological areas’ segmentation process.