Double Grad-CAM Guidance for Improved MRI-based Pseudo-CT Synthesis

Abstract

Deep learning approaches are capable of learning the mapping function from MR to CT images and thus allow synthesizing pseudo-CTs. However, due to the lack of visual information in the source MRI domain, they often fail to generate bone regions accurately. To address this issue, we propose a double Grad-CAM guidance for U-Net (DGCG U-Net), which indirectly forces the network to focus more on bone structures. More specifically, we first train a Grad-CAM guided classification model in such a way that it distinguishes MR and CT images based solely on bone regions. After that, we utilize this pre-trained classifier and again use Grad-CAM technique to guide our U-Net model by forcing it to focus on bone regions. The performance of the proposed approach is evaluated on the publicly available RIRE data set. The results demonstrate that our model, compared to the baseline GCG U-Net, generates more accurate pseudo-CTs, resulting in approximately 2.5% and 5.1% improvement for MAE and MSE in the bone regions, respectively. The corresponding Grad-CAM guided classifier achieved an accuracy of 99%.