Abstract
Lung image analysis is an essential part in the assessment of pulmonary diseases. Through visual inspection of CT scans, radiologists detect abnormal patterns in the lung parenchyma, aiming to establish a timely diagnosis and thus improving patient outcome. However, in a generalized disorder of the lungs, such as pulmonary hypertension, the changes in organ tissue can be elusive, requiring additional invasive studies to confirm the diagnosis. We present a graph model that quantifies lung texture in a holistic approach enhancing the analysis between pathologies with similar local changes. The approach extracts local state-of-the-art 3D texture descriptors from an automatically generated geometric parcellation of the lungs. The global texture distribution is encoded in a weighted graph that characterizes the correlations among neighboring organ regions. A data set of 125 patients with suspicion of having a pulmonary vascular pathology was used to evaluate our method. Three classes containing 47 pulmonary hypertension, 31 pulmonary embolism and 47 control cases were classified in a one vs. one setup. An area under the curve of up to 0.85 was obtained adding directionality to the edges of the graph architecture. The approach was able to identify diseased patients, and to distinguish pathologies with abnormal local and global blood perfusion defects.
You have full access to this open access chapter, Download conference paper PDF
Similar content being viewed by others
Keywords
1 Introduction
An important task in the radiology workflow is detecting subtle alterations in patient scans that could help to correctly identify and diagnose diseases. However, when there is a widespread distribution of the disease in an organ, i.e. the lungs, the pathological changes can be so elusive that require more (invasive) studies to establish a diagnosis [16]. Particularly in the case of pulmonary hypertension (PH), an invasive catheterization procedure is the gold standard for diagnosis and allows the differentiation of PH from pulmonary embolism (PE) [12]. A correct interpretation of the cases in their early stages is relevant since both pathologies have similar local manifestations, despite requiring different treatments [10]. Moreover, the pulmonary perfusion changes in PH and PE might not only be present in small regions, but also have a profound impact on the entire pulmonary circulatory network [11]. A holistic lung blood perfusion analysis, instead of local independent assessments of haemodynamic alterations, could improve the clinical evaluation of a patient by providing a global pathological status of the lungs.
Graph modeling is a complete framework that has been previously proposed for brain connectivity analysis, but has rarely been applied to other organs [14]. In short, graph methods divide the brain into fixed anatomical regions, and compare neural activations between different regions [15]. Based on these approaches, we propose a graph model of the lungs built from 3D local texture descriptors extracted on an atlas-based parcellation, with the purpose of encoding pulmonary blood perfusion relations between neighboring regions. An early version of this method was proposed in [8], combining simple intensity features in a 30 patients dataset of dual energy CT scans, combining 11 energy levels. The approach was also compared against 8 other methods, including deep learning approaches in the ImageCLEF tuberculosis challenge 2017 [4], obtaining the 3 top scores for the lung image analysis task in drug resistance detection [7].
In this paper we exploit the graph architecture for the analysis of the whole pulmonary circulatory network in a relatively large and heterogeneous dataset of patients with pulmonary vascular pathologies. The global analysis of the lungs was refined to detect and differentiate between PH and PE, relying only on the CT scans taken in an emergency radiology service. A novel visual interpretation of the biomedical tissue texture is presented as well, which can help radiologists to promptly interpret and localize 3D textural changes in anatomical structures.
2 Methods
2.1 Dataset
Experiments were carried out on contrast-enhanced chest CT images at 70 KeV of 125 patients: 31 with diagnosed PE, 47 with diagnosed PH and 47 control cases (CC) (see Fig. 1). The resolution of the CT slices in x- and y-directions varied from 0.5742 to 0.9766 mm, while the inter-slice distance was 1.00 mm. Since 3D rotation-invariant texture features were used in our approach, all CT images were converted into isometric volumes, with a voxel size of \(1\times 1\times 1\) mm\(^3\).
2.2 Holistic Graph Model of the Lungs
A pipeline composed of four steps was developed for building a distinctive graph per patient (see Fig. 2). Initially, the lung fields were automatically extracted using the method explained in [6]. Then, the lung masks were geometrically divided into a 36-region atlas [3, 5] derived from the 3D model of the human lung presented by Zrimec et al. [17]. For each region r of this atlas two texture-based feature descriptors were extracted: the Fourier histograms of oriented gradients (FHOG) [13] and the locally-oriented 3D Riesz-wavelet transform (3DRiesz) [9]. These descriptors have been successful for multiple biomedical texture analysis applications [1, 2]. FHOG was computed using 28 3D directions for the histogram, obtaining a 28-dimensional feature vector per image voxel v (\(\mathbf {f}_{H}(v) \in \mathbb {R}^{28}\)). For 3DRiesz we used the 3rd-order Riesz-wavelet transform, with 4 scales and 1st-order alignment (see [9]). The feature vector for a single voxel was defined as the weighted sum of the absolute Riesz response along the 4 scales, obtaining a 10-dimensional feature vector (\(\mathbf {f}_\mathcal {R}(v) \in \mathbb {R}^{10}\)). Finally, the average and standard deviation of these descriptors were obtained for each region r: \(\varvec{\mu }_{H}(r)\), \(\varvec{\sigma }_{H}(r)\), \(\varvec{\mu }_{\mathcal {R}}(r)\), and \(\varvec{\sigma }_{\mathcal {R}}(r)\).
2.3 Undirected Weighted Graph Model of the Lung
In [4] Dicente et al. tested several graph configurations for lung modeling of tuberculosis cases using the 36-regions atlas. The graph architecture obtaining the best scores contained 84 edges, based on a region adjacency defined on the atlas. Formally, there exists an undirected edge \(E_{i,j}\) between nodes \(N_i\) and \(N_j\) if regions \(r_i\) and \(r_j\) are 3D adjacent in the atlas or symmetric with respect the left-right division of the lungs (see Fig. 2). Considering \(\mathbf {f}_i\) and \(\mathbf {f}_j\) the feature vectors of regions \(r_i\) and \(r_j\) respectively, the weight \(w_{i,j}\) of an edge \(E_{i,j}\) was defined as: \(w_{i,j} = 1- corr (\mathbf {f}_i, \mathbf {f}_j) \in \left[ 0,2\right] \). The use of rotation-invariant texture descriptors where each component corresponds to a texture-direction implies that regions with similar texture will be highly correlated, and thus, with \(w_{i,j} \approx 0\). Since the edges used for this graph were undirected, then \(w_{i,j} = w_{j,i}\) and this graph has 84 weights. For a patient p this graph will be referred as \(G^{U}_p\).
2.4 Directed Weighted Graph-Model of the Lungs
Using the same graph architecture (36 nodes and 84 edges), we introduce the notion of directionality. let \(\mathbf {f}_i\) and \(\mathbf {f}_j\) be the feature vectors of regions \(r_i\) and \(r_j\), respectively. The weight of an edge \(E_{i,j}\) between nodes \(N_i\) and \(N_j\) is then defined as \(w_{i,j}=1- corr (\mathbf {f}_i, \mathbf {f}_j)\) if \(||\mathbf {f}_i|| \le ||\mathbf {f}_j||\), and 0 otherwise. Since \(w_{i,j} \ne w_{j,i}\) (except when \(||\mathbf {f}_i|| = ||\mathbf {f}_j||\)), this graph contains 168 weights, of which 84 are equal to 0. However, for each patient these 84 weights equal to 0 may be at different positions since the condition for being 0 relies on the properties of each patient lung tissue. The idea of this approach is to better exploit the same graph architecture adding more information about the texture in the regions (nodes), i.e. which node contained more/less texture than its neighbors For a patient p this graph is referred as \(G^{D}_p\).
2.5 Graph-Based Patient Descriptor
As mention before, the undirected graph \(G^{U}_p\) of a patient p only contained 84 weights. Then, we defined the graph-based patient descriptor \(\varvec{w}^{U}_p\) as the vector containing the 84 weights ordered by their location in the graph. For the directed graph, \(\varvec{w}^{D}_p\) contained the 168 weights ordered by their graph location.
Patient Descriptor Normalization. The descriptor vectors \(\varvec{w}_p\) were normalized based on the training set, obtaining \(\varvec{\hat{w}}_p\). However, since each component of \(\varvec{w}_p\) corresponded to a weight in the graph, these can not be seen independently and the normalization was done for all components simultaneously.
Concatenation of Patient Descriptors. As mentioned before, four different feature vectors were computed in each atlas region r(\(\varvec{\mu }_{H}(r)\), \(\varvec{\sigma }_{H}(r)\), \(\varvec{\mu }_{\mathcal {R}}(r)\), and \(\varvec{\sigma }_{\mathcal {R}}(r)\)) providing complementary information about the texture and its variability. Given a patient p, a different graph was obtained from each of these feature vectors. The final patient descriptor vector \(\varvec{W}_p\) used in our experiments was defined as the concatenation of the four normalized graph-based descriptors: \(\varvec{W}_p = (\varvec{\hat{w}}_{\varvec{\mu }_{H}, p} || \varvec{\hat{w}}_{\varvec{\sigma }_{H},p} || \varvec{\hat{w}}_{\varvec{\mu }_{\mathcal {R}}, p} || \varvec{\hat{w}}_{\varvec{\sigma }_{\mathcal {R}}, p} )\). After this concatenation, \(\varvec{W}^{U}_p \in \mathbb {R}^{336}\) when using the undirected graph and \(\varvec{W}^{D}_p \in \mathbb {R}^{672}\) for the directed approach.
3 Experimental Setup
Four binary classification experiments were performed: (a) PH vs. PE, (b) PH vs. CC, (c) PE vs. CC, and (d) (PH+PE) vs. CC. In experiment (d) PH and PE patients were considered to belong to same class. The dataset was divided in each experiment in \(\sim \)70% for training and \(\sim \)30% for testing. This results in 34 PH, 22 PE and 34 CC for training and 13 PH, 9 PE and 13 CC for testing. As the classes were not balanced in the test sets, the random accuracy (when assigning the most frequent label to all the patients) was 59.09% for experiments (a) and (c), 50.00% for experiment (b) and 62.86% for experiment (d). 2-class SVM classifier with Gaussian kernel was used in each experiment. The parameter optimization was performed using 10-fold cross-validation on the training set. All experiments were performed using both the undirected and the directed graph models.
The descriptor vectors for a given patient \(\varvec{W}^{U}_p\) and \(\varvec{W}^{D}_p\) had a dimension significantly higher than the size of our dataset (336 and 672 vs. 125). To avoid the known overfitting problems when using large feature spaces, feature dimensionality reduction was performed in the training phase. This consisted on selecting those dimensions that correlate above the average with the training labels. This method reduced the feature space dimension approximately by two, both when using the undirected and directed graph models. Moreover, since the random split of the dataset in training and test sets may have a strong effect in the classification performance, we ran each experiment using five different random splits to ensure the robustness of our models.
4 Results
Figure 3 shows the average and standard deviation of the accuracy and the AUC over the five random splits for each of the four experiments. The accuracy and the confusion matrices in Fig. 4 are reported using the standard classification decision threshold of 0.5, and not in the optimal threshold provided by the ROC curves. The results of each experiment were above the random performance in all cases (see red line in Fig. 3). When analyzing all the performance measures, the directed graph model performed better than the undirected in the experiments (b) and (c). However, both models performed almost equivalent in experiments (a) and (b). The highest benefit of the directed graph among the undirected is shown in experiment (b), where the true positive rate increases from 69.23% and 75.38% to 76.92% and 81.54% in the PH and CC classes respectively. In experiment (d) the increment is similar, however, the directed graph is still not able to identify the CC class above random. These results suggest that PH and PE do not form a clear cluster in the feature space.
5 Discussion
This work presents a comprehensive graph model of the lungs that connects 3D regional texture nodes enclosing the whole circulatory network of lungs. The method was evaluated in a large dataset of 125 CT thorax scans from patients with clinical symptoms commonly associated to vascular pathologies, such as pulmonary hypertension and pulmonary embolism. Even for experienced radiologists detecting and differentiating pulmonary hypertension is a challenging task diagnosed only by using a catheterization procedure. The proposed approach was able to correctly identify most of the patients with these pulmonary perfusion defects in a standard CT scan, even without any prior clinical information.
The proposed composite interpretation of the lung circulatory network was able to distinguish between local defects (PE) and elusive global pathological patterns (PH). By analyzing the confusion matrices in Fig. 4, it can be seen that the method had the worse classification scores with the control class. Nonetheless, it is important to highlight that all the patients in the dataset were obtained from an emergency department and may well have other lung malfunctions. A straightforward 3D visualization of the textural changes in the lungs was additionally generated from the obtained graph architectures. The graph depicting both lungs could be useful for radiologists to understand intrinsic parenchymal texture distributions that might not be apparent in 2D renderings of the organ.
6 Conclusions and Future Work
A precise assessment of pulmonary perfusion can lead to a fast diagnosis and optimal treatment in the presence of haemodynamic changes. A directed weighted graph model of the lungs encoding blood perfusion relations from an automatically generated geometric atlas of the lungs was evaluated. The approach was able to not only recognize diseased patients, but also to classify correctly similar abnormal local and global patterns in the pulmonary parenchymal texture. This holistic graph descriptor can be expanded to include more lung pathologies and potentially be applied to other anatomical structures as well.
References
Chalkidou, A., O’Doherty, M.J., Marsden, P.K.: False discovery rates in PET and CT Studies with texture features: a systematic review. PLOS ONE 10(5), 1–18 (2015)
Depeursinge, A., Al-Kadi, O.S., Mitchell, J.R.: Biomedical Texture Analysis: Fundamentals, Applications and Tools. Elsevier-MICCAI Society Book series. Elsevier, October 2017. https://www.elsevier.com/books/title/author/9780128121337
Depeursinge, A., Zrimec, T., Busayarat, S., Müller, H.: 3D lung image retrieval using localized features. In: Medical Imaging 2011: Computer-Aided Diagnosis, vol. 7963, p. 79632E. SPIE, February 2011
Dicente Cid, Y., Batmanghelich, K., Müller, H.: Textured graph-model of the lungs for tuberculosis type classification and drug resistance prediction: participation in ImageCLEF 2017. In: CLEF2017 Working Notes. CEUR Workshop Proceedings, CEUR-WS.org, Dublin, Ireland, 11–14 September 2017 (2017). http://ceur-ws.org
Dicente Cid, Y., Depeursinge, A., Foncubierta-Rodríguez, Platon, A., Poletti, P.A., Müller, H.: Pulmonary embolism detection using localized vessel-based features in dual energy CT. In: SPIE Medical Imaging. International Society for Optics and Photonics (2015)
Dicente Cid, Y., Jimenez-del-Toro, O., Depeursinge, A., Müller, H.: Efficient and fully automatic segmentation of the lungs in CT volumes. In: Goksel, O., Jimenez-del-Toro, O., Foncubierta-Rodriguez, A., Müller, H. (eds.) Proceedings of the VISCERAL Challenge at ISBI, pp. 31–35. No. 1390 in CEUR Workshop Proceedings, April 2015
Dicente Cid, Y., Kalinovsky, A., Liauchuk, V., Kovalev, V., Müller, H.: Overview of ImageCLEFtuberculosis 2017 - predicting tuberculosis type and drug resistances. In: CLEF 2017 Labs Working Notes. CEUR Workshop Proceedings, CEUR-WS.org, Dublin, Ireland, 11–14 September 2017 (2017). http://ceur-ws.org
Dicente Cid, Y., et al.: A lung graph–model for pulmonary hypertension and pulmonary embolism detection on DECT images. In: Müller, H. (ed.) MCV/BAMBI -2016. LNCS, vol. 10081, pp. 58–68. Springer, Cham (2017). https://doi.org/10.1007/978-3-319-61188-4_6
Dicente Cid, Y., Müller, H., Platon, A., Poletti, P.A., Depeursinge, A.: 3-D solid texture classification using locally-oriented wavelet transforms. IEEE Trans. Image Process. 26(4), 1899–1910 (2017)
Farber, H.: Pulmonary circulation: diseases and their treatment. Eur. Respir. Rev. 21(123), 78 (2012). 3rd edition
Galiè, N., et al.: Treatment of patients with mildly symptomatic pulmonary arterial hypertension with bosentan (early study): a double-blind, randomised controlled trial. Lancet 371(9630), 2093–2100 (2008)
Lador, F., Herve, P.: A practical approach of pulmonary hypertension in the elderly. Semin. Respir. Crit. Care Med. 34(5), 654–664 (2013)
Liu, K., et al.: Rotation-invariant hog descriptors using fourier analysis in polar and spherical coordinates. Int. J. Comput. Vis. 106(3), 342–364 (2014)
Richiardi, J., Bunke, H., Van De Ville, D., Achard, S.: Machine learning with brain graphs. IEEE Signal Process. Mag. 30, 58 (2013)
Richiardi, J., Eryilmaz, H., Schwartz, S., Vuilleumier, P., Van De Ville, D.: Decoding brain states from fMRI connectivity graphs. NeuroImage 56(2), 616–626 (2011)
Tuder, R.M.: Relevant issues in the pathology and pathobiology of pulmonary hypertension. J. Am. Coll. Cardiol. 62(25 SUPPL.), D4–D12 (2013)
Zrimec, T., Busayarat, S., Wilson, P.: A 3D model of the human lung with lung regions characterization. In: ICIP 2004 Proceedings of the IEEE International Conference on Image Processing, vol. 2, pp. 1149–1152 (2004)
Acknowledgments
This work was partly supported by the Swiss National Science Foundation in the PH4D project (grant agreement 320030-146804).
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2018 Springer Nature Switzerland AG
About this paper
Cite this paper
Dicente Cid, Y., Jiménez-del-Toro, O., Platon, A., Müller, H., Poletti, PA. (2018). From Local to Global: A Holistic Lung Graph Model. In: Frangi, A., Schnabel, J., Davatzikos, C., Alberola-López, C., Fichtinger, G. (eds) Medical Image Computing and Computer Assisted Intervention – MICCAI 2018. MICCAI 2018. Lecture Notes in Computer Science(), vol 11071. Springer, Cham. https://doi.org/10.1007/978-3-030-00934-2_87
Download citation
DOI: https://doi.org/10.1007/978-3-030-00934-2_87
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-00933-5
Online ISBN: 978-3-030-00934-2
eBook Packages: Computer ScienceComputer Science (R0)