Super-Resolution Reconstruction of Fetal Brain MRI with Prior Anatomical Knowledge

Abstract

Super-resolution reconstruction (SRR) of fetal brain MRI from motion-corrupted thick-slice stacks can provide high-resolution isotropic 3D images that are vital for prenatal examination and quantification of brain development. Existing fetal brain SRR methods generally rely on a two-stage optimization procedure by performing rigid slice-to-volume registration and volumetric reconstruction in an alternating manner. Despite their advantages, these methods have not considered additional guidance from external anatomical priors, resulting in unsatisfactory performance in various challenging cases. To address this issue, we propose a novel Prior Anatomical Knowledge guided fetal brain Super-Resolution Reconstruction method, namely PAK-SRR. In PAK-SRR, we consider two key kinds of prior anatomical information. First, we integrate the anatomical prior provided by tissue segmentation into both the slice-to-volume registration and volumetric reconstruction to enforce registration consistency on boundaries, effectively alleviating misregistration caused by blurry tissue boundaries of brain. Second, to enrich the structural details of the reconstructed images, we further employ longitudinal fetal brain atlases to guide volumetric reconstruction. Extensive experiments on multi-site clinical datasets demonstrate that our PAK-SRR significantly outperforms the state-of-the-art SRR methods for fetal brain MRI, quantitatively and qualitatively. Our code is publicly available at https://github.com/sj-huang/PAK-SRR for reproducibility and further research.

S. Huang and G. Chen— Contributed equally.

1 Introduction

Acquiring isotropic high-resolution (HR) magnetic resonance (MR) images from fetal brain is essential for prenatal examination and brain development studies. However, it is clinically difficult to acquire such isotropic HR volumetric images due to large irregular motion of fetal subjects. In this way, fetal brains are often scanned in a short time with fast imaging protocols (e.g., SSFSE: single-shot fast spin echo [4]), which acquire multiple thick-slice stacks from different views, as shown in Fig. 1. These different-view stacks are further integrated using super-resolution reconstruction (SRR) techniques at the post-acquisition stage to obtain desired isotropic HR volumetric images. However, it is challenging to conduct fetal brain SRR due to 1) poor slice image quality caused by large irregular motion, 2) misalignment among intra- and inter-stack slices, and 3) large variation of fetal brains in the acquired MR images [13].

To tackle these challenges, Rousseau et al. [13] proposed an approach to reconstruct 3D isotropic HR volumetric images for fetal brain from multi-view (direction) and multi-resolution 2D slices, by iteratively performing slice-to-volume registration and volumetric reconstruction, after preprocessing of acquired data. Kuklisova-Murgasova et al. [9] used expectation-maximization statistics to discard the potential outliers for more robust SRR. Moreover, they applied intensity matching to compensate for the inconsistency of multi-slices. Rousseau et al. [14] later improved the regularization procedure associated with SRR to preserve better boundaries. However, these approaches are semi-automatic since fetal brain region needs to be predefined [2]. To resolve this issue, Ebner et al. [2] proposed an automatic approach to integrate localization, segmentation, and SRR into a unified framework. Besides, deep learning has been employed to improve the performance of fetal brain SRR [7, 15, 18]. However, deep learning-based methods rely on large-scale high-quality training data, and suffer from generalization issues, thus limiting their practical applications. Despite the progress of existing methods [2, 5, 9, 13, 14, 19], they all ignore the use of anatomical prior, which can provide vital information to improve the SRR of fetal brain MRI. This leads to unsatisfactory performance for various challenging cases with large irregular motions and low image quality, as shown in Fig. 1.

To this end, we propose a novel method, namely PAK-SRR, to employ rich prior anatomical knowledge (PAK), including tissue segmentation maps (i.e., seg-priors) and longitudinal fetal brain atlases (i.e., atlas-priors), as strong guidance to improve the SRR of fetal brain MRI (Fig. 1). Specifically, we employ seg-priors from tissue segmentation maps to guide more accurate slice-to-volume registration, which can improve the quality of reconstructed HR volumetric images. Furthermore, we employ atlas-priors (with rich anatomical priors) to address image degradation caused by motion artifacts and thick-slice acquisition. These two kinds of prior anatomical information are integrated into a unified regularization framework, which is optimized by an effective linear least-squares solver, called LSMR [3]. Experimental results on multi-site clinical datasets demonstrate that our PAK-SRR achieves state-of-the-art reconstruction performance along with strong robustness against motion artifacts.

Fig. 1.

Comparison of the existing framework (right top) [2] with our PAK-SRR (right bottom).

2 Background

The SRR of fetal brain MRI utilizes stacks of acquired 2D slices to reconstruct 3D super-resolution image and is usually formulated by the slice acquisition model [5] as below:

$$\begin{aligned} \textbf{y}_k=\textbf{A}_k(\textbf{x})+\textbf{e}_k, \end{aligned}$$

(1)

where $\textbf{y}_{k}$ is the acquired ${k}_{th}$ slice and $\textbf{x}$ is the tentative 3D HR image. $\textbf{A}_k$ is an operator including slice-to-volume transformation $\boldsymbol{T}_{k}$ and blurring operator $\boldsymbol{B}$ (based on a certain point spread function (PSF) [11]). Generally, we can formulate $\textbf{A}_k(\cdot ):=[\boldsymbol{T}_{k}\circ \boldsymbol{B}(\cdot )]_{k}$, where the symbol $\circ$ denotes resampling operation, and $[\cdot ]_{k}$ indicates the corresponding plane of the slice $\textbf{y}_k$ in the 3D image. The vector $\textbf{e}_k$ represents an additive noise.

Due to the ill-posedness of the above problem in Eq. (1), the commonly used methods for SRR of fetal brain MRI are based on a two-stage iterative registration-reconstruction framework, including 1) slice-to-volume registration and 2) volumetric reconstruction. In each iteration i, slice-to-volume registration is performed to align each slice $\textbf{y}_{k}$ with the tentatively reconstructed volumetric image $\textbf{x}$ usually by minimizing the following objective function:

$$\begin{aligned} {\boldsymbol{T}}^{(i)}_k=\underset{\boldsymbol{T}_{k}}{{\text {argmin}}}\left( ~\mathcal {S}(\textbf{x}^{(i-1)}, \textbf{y}_{k} ; \boldsymbol{T}_k)+\mathcal {R_T}(\boldsymbol{T}_k)\right) , \end{aligned}$$

(2)

where $\mathcal {S}$ represents the similarity measure between slice $\textbf{y}_{k}$ and the transformed volume $\textbf{x}$, and $\mathcal {R_T}(\cdot )$ is the regularization term for the transformation $\boldsymbol{T}_{k}$. It should be noted that transformation is usually applied on volume $\textbf{x}$ instead of slice $\textbf{y}_{k}$, since resampling a 2D slice in 3D space is not as well defined as sampling a volume. Rigid transformation is usually employed for registration [2, 9, 13, 14], which includes six Degrees of Freedom (DoF) for each individual slice. Mathematically, we omit subscript k for clarity and can formulate the transformation as a $4 \times 4$ matrix $\boldsymbol{T}=\left[ \begin{array}{cc} \boldsymbol{R(\theta )} &{} \boldsymbol{d} \\ 0 &{} 1 \end{array}\right]$, where $\boldsymbol{R(\theta )}$ denotes a rotation matrix parameterized by three rotational parameters $\boldsymbol{\theta } = (\theta _u , \theta _v , \theta _w )^T$, and $\boldsymbol{d} = (d_u,d_v,d_w)^T$ is a vector of three translation parameters in the 3D space.

Volumetric reconstruction is performed when the transformations $\{\boldsymbol{T}_k^{(i)}\}_{k=1}^{K}$ are updated in each iteration. Assuming $\textbf{e}_k$ being an additive white Gaussian noise, the image $\textbf{x}$ can be updated as below:

$$\begin{aligned} \textbf{x}^{(i)}=\underset{\textbf{x}}{{\text {argmin}}}\left( \sum _{k} \frac{1}{2}\left\| \textbf{y}_{k}-\textbf{A}_{k}^{(i)} (\textbf{x})\right\| _{2}^{2}+\mathcal {R}_{\textbf{x}}(\textbf{x})\right) , \end{aligned}$$

(3)

where $\mathcal {R}_{\textbf{x}}(\cdot )$ regularizes the solution space of the latent 3D image $\textbf{x}$. The optimization problem can be solved by popularly used least-square solver. The above-described two-stage framework performs alternating update of transformations $\{\boldsymbol{T}^{(i)}_k\}_{k=1}^{K}$ and 3D image $\textbf{x}^{(i)}$ until convergence.

Fig. 2.

Overview of our proposed PAK-SRR for fetal brain MRI. (I) Extracting brain tissue maps and the corresponding seg-priors (i.e., $\{\textbf{w}_k\}_{k=1}^{K}$). (II) Aligning multi-view stacks (i.e., $\{\textbf{y}_k\}_{k=1}^{K}$) into the same space for obtaining an initial coarse HR volumetric image. (III) The two-stage iterative registration-reconstruction stage, including a seg-priors guided slice-to-volume registration for updating $\{\boldsymbol{T}^{(i)}_k\}_{k=1}^{K}$ and a seg-priors and atlas-priors jointly guided (with $\{\textbf{w}_k\}_{k=1}^{K}$ and $\nabla \textbf{x}_{\text {a}}^{(i)}$) volumetric reconstruction for updating $\textbf{x}^{(i)}$. (IV) Reorienting the final reconstructed 3D image into standard anatomical space defined by longitudinal atlases.

3 Method

An overview of our proposed PAK-SRR is shown in Fig. 2. Different from the existing methods for SRR of fetal brain MRI, we consider two key kinds of prior anatomical information from 1) tissue segmentation maps (seg-priors) and 2) longitudinal fetal brain atlases (atlas-priors), and integrate them into the two-stage iterative registration-reconstruction framework for improving the registration accuracy and reconstruction performance. The effectiveness of our seg-priors and atlas-priors will be demonstrated in Sect. 4.

3.1 Slice-to-Volume Registration Guided by Seg-priors

The slice-to-volume registration stage aligns multi-resolution and multi-view input slices with the tentatively-reconstructed volume by rigid registration. However, accurate registration is challenging due to large irregular motion and blurry tissue boundaries in the input stacks, which may cause misalignment and result in unsatisfactory SRR. To resolve this issue, we perform tissue segmentation and employ the seg-priors to assist the slice-to-volume registration. Specially, we perform segmentation for six types of tissues, including cerebrospinal fluid, grey matter, white matter, ventricles, cerebellum, and brainstem. Based on the segmentation results, we calculate the individual Signed Distance Map (SDM) [1] for each of six types of tissues, and then calculate the mean of six SDMs ($N=6$) as the seg-priors $\{\textbf{w}_k\}_{k=1}^{K}$ as follows:

$$\begin{aligned} \textbf{w}_k=\frac{1}{N}\sum ^N_{t=1} c^{\frac{-\left| SDM_k^{(t)}\right| }{\left\| SDM_k^{(t)}\right\| _{\infty }}}, \end{aligned}$$

(4)

where c is a constant used to control the contrast of the map (the larger c, the larger the contrast), and in the experiments, we set $c = 1e^3$. The operators $\left| \cdot \right|$ and $\left\| \cdot \right\| _{\infty }$ calculate the absolute value and $\ell _{\infty }$-norm, respectively. The calculation of seg-priors for white matter is demonstrated in Fig. 3.

Fig. 3.

Demonstration of the stages for calculation of the seg-priors. (a) Brain image; (b) Segmented white matter; (c) Color-coded SDM for white matter; (d) Color-coded seg-priors for white matter. (Color figure online)

We incorporate the seg-priors $\{\textbf{w}_k\}_{k=1}^{K}$ into the calculation of similarity measure, i.e., normalized cross-correlation coefficient (NCC), and obtain our proposed weighted NCC as

$$\begin{aligned} \textrm{NCC}_{\textbf{w}}(I_{1}, I_{2}):=\frac{\sum _{j} \textbf{w}^{(j)} {\left( I_{1}^{(j)}-\bar{I}_{1}\right) \left( I_{2}^{(j)}-\bar{I}_{2}\right) }}{\sqrt{\sum _{j}\left( I_{1}^{(j)}-\bar{I}_{1}\right) ^{2}} \sqrt{\sum _{j}\left( I_{2}^{(j)}-\bar{I}_{2}\right) ^{2}}}, \end{aligned}$$

(5)

where $I_{1}$ and $I_{2}$ denote the vectorized images and $\textbf{w}$ is the vectorized seg-priors. The index j denotes the $j_{th}$ element of the vector. Therefore, we can update the transformation $\boldsymbol{T}_{k}^{(i)}$ of the $k_{th}$ slice at iteration i with

$$\begin{aligned} \boldsymbol{T}_{k}^{(i)}=\underset{\boldsymbol{T}_k}{{\text {argmin}}}\left( -\textrm{NCC}_{\textbf{w}_k}(\textbf{y}_{k}, [\boldsymbol{T}_{k}\circ \boldsymbol{B} (\textbf{x}^{(i-1)})]_{k})\right) . \end{aligned}$$

(6)

In this way, we penalize more on boundary misalignment, and can obtain improved registration performance on boundary areas, which usually convey more structural information and are thus critical for downstream applications.

Fig. 4.

Visualization of the reconstructed volume ($\textbf{x}^{(i)}$), the registered atlas ($\textbf{x}_{\text {a}}^{(i)}$), and the corresponding gradient maps ($\nabla \textbf{x}^{(i)}$ and $\nabla \textbf{x}_{\text {a}}^{(i)}$) by both methods (shown in first two rows for PAR-SRR, and third row for NiftyMIC) at three update iterations (shown horizontally with two columns for each iteration).

3.2 Volumetric Reconstruction Jointly Guided by Seg-priors and Atlas-priors

The volumetric reconstruction stage aims to reconstruct a super-resolution volumetric image based on the aligned thick-slice stacks. In this stage, we intend to employ the seg-priors and atlas-priors jointly. First, we integrate the calculated seg-priors $\{\textbf{w}_k\}_{k=1}^{K}$ into the maximum a posteriori (MAP)-based volumetric reconstruction stage:

$$\begin{aligned} \textbf{x}^{(i)}=\underset{\textbf{x}}{{\text {argmin}}}\left( \sum _{k \in \mathscr {K}_{\delta }^{(i)}} \frac{1}{2}\left\| \textbf{y}_{k}-\textbf{A}_{k}^{(i)}(\textbf{x})\right\| _{\textbf{w}_{k}}^{2}+\frac{\alpha }{2}\Vert \nabla \textbf{x}\Vert _{2}^{2}\right) , \end{aligned}$$

(7)

where $\left\| \cdot \right\| _{\textbf{w}_{k}}^{2}$ represents the weighted $\ell _2$-norm with the weights $\textbf{w}_{k}$ being diagonal matrix for the $k_{th}$ slice. Following the work of [2], we eliminate the outlier slices which are severely corrupted by large motion according to the following rule:

$$\begin{aligned} \mathscr {K}_{\delta }^{(i)}:=\left\{ 1 \le k \le K: \textrm{NCC}(\textbf{y}_{k}, \textbf{A}_{k}^{(i)}(\textbf{x}^{(i-1)})) \ge \delta \right\} , \end{aligned}$$

(8)

where $\mathscr {K}_{\delta }^{(i)}$ denotes a set of indices of inlier slices. $\mathrm {NCC(\cdot )}$ measures the slice similarity such that the slices with similarity less than a predefined threshold $\delta$ are considered as outliers and will be discarded in the volumetric reconstruction stage.

Second, to compensate for the information loss due to motion artifacts, we resort to longitudinal fetal brain atlases and use gradient maps of certain time-point atlases as auxiliary information for volumetric reconstruction. In Fig. 4, we demonstrate the gradient maps of a reconstructed volumetric image and the atlas [6]. As can be observed, the atlas usually has sharper boundaries and sharper gradient maps than the initially reconstructed volumetric image, implying that atlas information can be utilized to compensate for blurry boundaries and improve quality of the reconstructed images. Therefore, we register the atlas to the tentatively reconstructed volumetric image, denoted as $\textbf{x}_{\text {a}}$, and impose similarity match between the gradient of the volume $\nabla \textbf{x}$ and the corresponding one of the atlas-priors (i.e., $\nabla \textbf{x}_{\text {a}}$) to obtain clearer boundaries in the reconstructed images. Mathematically, we can use the following formulation:

$$\begin{aligned} \textbf{x}^{(i)}=\underset{\textbf{x}}{{\text {argmin}}} \biggl (\sum _{k \in \mathscr {K}_{\delta }^{(i)}} \frac{1}{2}\left\| \textbf{y}_{k}-\textbf{A}_{k}^{(i)} (\textbf{x})\right\| _{\textbf{w}_{k}}^{2} +\frac{\alpha }{2}\Vert \nabla \textbf{x}-\varepsilon \nabla \textbf{x}_{\text {a}}\Vert _{2}^{2}\biggr ), \end{aligned}$$

(9)

where $\varepsilon$ and $\alpha$ are scalar parameters. The prior knowledge from the atlases can effectively guide the volumetric reconstruction from low-quality slices. Those slices with severe quality degradation usually decrease reconstruction performance. It is worth noting that, when introducing atlas-priors into the regularization, the overall intensities of the reconstructed volumetric image $\textbf{x}$ might slightly deviate from the input slices during the optimization, as shown in Fig. 4. To alleviate this effect, we perform histogram matching between the tentatively reconstructed volumetric image and its corresponding input stacks during slice-to-volume registration in each iteration.

4 Experiments

4.1 Dataset

We have evaluated our proposed PAK-SRR on fetal brain MRI of 66 subjects covering 20 to 38 gestational weeks (GWs) from multi-site clinic centers, with totally 213 stacks. Each subject was scanned under at least three different-view (direction) stacks by the SSFSE sequence with different spatial resolutions including $1.3\times 1.3\times 5$, $1.5\times 1.5\times 4.2$, $0.625\times 0.625\times 3$, and $0.55\times 0.55\times 4.4$ mm. We use bias field correction [8] to preprocess all different-view stacks.

4.2 Experimental Settings

We do the following settings for our experiments. 1) For selecting a target stack used initially to build a tentative HR 3D MR image of each fetal brain, we first use U-Net to segment brain tissue map for each different-view stack, and then convert them to seg-priors as shown in Fig. 3. Next, we calculate an overlap ratio for each stack with all other stacks according to their respective brain tissue maps. In this way, we can select a stack with the maximum overlap ratio with all other stacks. Finally, by using this selected stack as a target space and aligning all other stacks to it, we can build a tentative HR 3D MR image (for each fetal brain). 2) For parameters used in Eq. 9, we set the outlier-threshold $\delta$ as 0.6, 0.7, and 0.8, as well as the hyper-parameter $\varepsilon$ as 0.8, 0.7, and 0.6, for the three iterations (for our case of using three iterations), respectively. We set the regularization parameter $\alpha$ to 0.01 for all iterations. 3) The isotropic resolution for the final HR MR image can be set to a range of 0.5 mm to 1.0 mm. We choose 0.8 mm since the resolution of our employed longitudinal atlases is 0.8 mm (Fig. 5).

Table 1. Quantitative comparison between PAK-SRR and other methods on multi-site datasets of 66 subjects. The best results are in bold.

Fig. 5.

Qualitative results by NiftyMIC and PAK-SRR for subjects at 20, 24, 27, 30, 35, and 38 GWs, shown in 6 columns, respectively.

4.3 Implementation Details

The following pipeline is conducted for all the experiments. 1) Intensity correction: following the work in [2], we employ linear regression to make intensities of all other stacks consistent with the intensities of the selected target stack. 2) Volume-to-volume rigid registration: we employ the symmetric block-matching algorithm REGALADIN [12] to align all stacks with the target stack. 3) Initial HR volume Estimation: we apply the scattered data approximation approach [2] on the aligned stacks to obtain an initial coarse HR volume. 4) Slice-to-volume registration: our proposed seg-priors guided slice-to-volume registration algorithm (as described in Sect. 3.1) is used to improve registration accuracy (especially in challenging areas) by introducing tissue seg-priors. 5) Volumetric reconstruction: we employ the longitudinal atlases from Gholipour et al. [6] and register the corresponding time-point atlas (according to GWs) to the tentatively-estimated HR volume. Based on the guidance of atlas-priors (as described in Sect. 3.2), we update the reconstructed HR volume using LSMR algorithm [3]. 6) Alternatingly optimizing 4) and 5) until convergence: the above-described two-stage framework performs the alternating update of transformation $\boldsymbol{T}^{(i)}_k$ and 3D image $\textbf{x}^{(i)}$ until convergence. 7) Reorienting HR image: the reconstructed HR volumetric image is further reoriented into the standard anatomical space (defined by longitudinal atlases) for downstream analysis.

Table 2. Quantitative results by NiftyMIC and PAK-SRR for normal (N) and abnormal (ABN) subjects from 20 to 38 GWs.

4.4 Experimental Results

We compare our PAK-SRR with four state-of-the-art methods [2, 9, 13, 14] in terms of peak-signal-to-noise ratio (PSNR), structural similarity index measure [17] (SSIM), and NCC. As shown in Table 1, our PAK-SRR obtains the best mean performance and the best standard deviation in SSIM and NCC. The second-best method, NiftyMIC, outperforms the other three methods also by a large margin. Therefore, in the following experiments, we will focus on the comparison between our PAK-SRR and NiftyMIC.

Table 3. Quantitative results of NiftyMIC and PAK-SRR for a normal 31 GWs subject under different combinations of input stacks. Note that A, C, and S denote stacks that are acquired with high-resolution in axial, coronal, and sagittal views, respectively.

Fig. 6.

Qualitative results by NiftyMIC and PAK-SRR on 31-GWs subject. A, C, and S denote stacks that are acquired with high-resolution in axial, coronal, and sagittal views, respectively.

Table 2 summarizes the statistical results of different GWs for all 66 subjects. We divide the data into normal (clinically defined normal fetal brain growth and development, with appearance close to the atlases) and abnormal (as opposed) cases for better evaluation. As can be observed, for all the cases, our PAK-SRR achieves promising performance in all the evaluation metrics. For the cases mainly containing normal subjects, our proposed PAK-SRR obtains significant improvement over NiftyMIC, especially in SSIM and NCC, which is attributed to the use of additional information from the atlas-priors.

For in-depth analysis, we employ different combinations of axial (A), coronal (C), and sagittal (S) views as input to evaluate the effectiveness of our proposed methods. The qualitative performance is illustrated in Fig. 6. It can be seen that NiftyMIC generates blurring boundaries and gets worse when the number of input views reduces. In contrast, PAK-SRR obtains high-quality isotropic images, even using only the axial view from three angles (3A). In fact, in Table 3, we show that, using only 3A views, PAK-SRR generates much better quantitative performance than NiftyMIC using the combinations of all views. Interestingly, the result of 1A+1C+1S outperforms 3A+2C+2S for both methods, indicating that the redundant or possibly-damaged inlier slices may degrade the performance due to misregistration, and another reason might be that the use of more angular stacks also causes errors in slice-to-volume registration. Overall, based on both qualitative and quantitative evaluations, our PAK-SRR achieves much clearer anatomical structures, especially for the challenging samples, and outperforms NiftyMIC for all cases of different GWs and different acquisition views, in terms of all the evaluation metrics.

4.5 Ablation Study

In PAK-SRR, we propose two key components, i.e., seg-priors (SP) and atlas-priors (AP). To demonstrate their effectiveness, we conduct ablation study on two variants, SP-SRR and AP-SRR, for evaluating seg-priors and atlas-priors (as shown in Table 4), respectively.

Table 4. Ablation study of key components of our PAK-SRR, ($\mathcal {P}$) denotes the paired t-test p-values for comparison of group means before and after intervention.

Fig. 7.

Qualitative performance of NiftyMIC and SP-SRR with the assistance of tissue segmentation in two cases (Left: 27 GWs; Right: 31 GWs).

Effectiveness of Seg-priors: The seg-priors are obtained by tissue segmentation which is one of the key contributions. Figure 7 demonstrates the results of NiftyMIC and SP-SRR for two fetal brains of 27 GWs and 31 GWs. We can see that both methods obtain reasonable HR images for two fetal brains. However, NiftyMIC reconstructs HR images not only with blurry boundaries but also with less fidelity compared to the reference images. On the contrary, SP-SRR achieves successful reconstruction in these challenging regions as masked and zoomed-up in Fig. 7, demonstrating that the seg-priors extracted from the tissue segmentation maps indeed improve both the registration accuracy and the reconstruction performance.

Effectiveness of Atlas-priors: To show the effectiveness of AP-SRR, we provide a challenging case with large motion and artifacts in Fig. 8. In this case, the brain cannot be identified in the other two views. The NiftyMIC fails to reconstruct the $k_{th}$ slice in the sagittal view, and shows severe streak artifacts in the other two views. In contrast, with the guidance of longitudinal atlases (i.e., atlas-priors), AP-SRR achieves promising reconstruction in all three views. It is worth noting that since the super-resolved sagittal view provided by AP-SRR intends to compensate for the motion artifact and generates more plausible images than the corrupted input slice, which is usually considered as a reference, it may reduce quantitative assessment for this particular case. In some cases, there might be significant differences between the appearances of the atlas and the reference. Even in this way, the anatomical structures of the reconstructed slices using atlas-priors can still be consistent with the references as shown in Fig. 9. This demonstrates the robustness of our proposed method.

Fig. 8.

Qualitative performance of NiftyMIC (middle three columns) and AP-SRR (right three columns) for a 35 GWs subject. The $k_{th}$ reference slice of the sagittal view is corrupted with large motion and artifacts (left three column).

Fig. 9.

Visualization of the initial image ($\textbf{x}^{(1)}$), initially-registered atlas ($\textbf{x}_{\text {a}}^{(1)}$), reconstructed HR image ($\textbf{x}$), and reference.

5 Conclusion

In this paper, we proposed a novel prior-anatomical-knowledge assisted method for super-resolution reconstruction (SRR) of fetal brain MRI, namely PAK-SSR. Our method is built on a two-stage iterative registration-reconstruction scheme. Different from the existing methods, we introduce the seg-priors, derived from the segmentation maps of six tissue types in fetal brains, to guide the slice-to-volume registration and volumetric reconstruction. Moreover, we propose to employ the atlas-priors, derived from the longitudinal fetal brain atlases, to exploit developmental characteristics for assisting the volumetric reconstruction stage. We performed extensive experiments to evaluate our PAK-SRR. Experimental results show that PAK-SRR outperforms the state-of-the-art SRR methods by a large margin in terms of both quantitative and qualitative evaluations. In the ablation study, we further show that our proposed seg-priors and atlas-priors can effectively improve slice-to-volume registration and volumetric reconstruction, eventually providing reconstructed 3D images with high fidelity and clear tissue details. In the future, we will explore to leverage these high-quality fetal brain images to improve existing brain development studies [10, 16] by extending the temporal dimension to pro-gestation period.