Keywords

1 Introduction

Normal growth and development of the human embryonic and fetal brain during the first trimester is of crucial importance for current and future health of the fetus [14, 17]. Currently, this is monitored by manual measurements, such as the circumference and volume of the brain [13, 15]. However, these measurements lack overview: it is unclear how the different measurements relate. The availability of an atlas i.e., a set of brain templates for a range of gestational ages, could overcome these challenges by offering a unified and automatic framework to compare development across subjects.

In literature several atlases are available [5,6,7,8, 11, 12, 16, 18, 19]. However, these are based on magnetic resonance imaging and/or acquired during the second and third trimester of pregnancy. Here, we present to the best of our knowledge the first framework for the development of a brain atlas describing growth of the human embryo and fetus between 56 and 90 days gestational age (GA) based on ultrasound imaging.

2 Method

The atlas is generated from three-dimensional (3D) ultrasound images \(I_{i,t}\), for subject i imaged at time t, where t is the GA in days. The atlas \(A_t\) is obtained by groupwise registration of \(I_{i,t}\) for every pregnancy \(i=1,...,k\) on every time point t, followed by taking the mean over the deformed images: \(A_t = \frac{1}{k} \sum _{i} I_{i,t} \circ \phi _{i}\). Hereby the constraint \(\sum _{i,t} \phi _{i,t} \approx 0\) is applied, as proposed by Balci et al. and Bhatia et al. [1, 3]. To ensure invertibility of the deformations we used diffeomorphic non-rigid deformations with the deformation field \(\phi _{i,t}\), obtained by integrating the velocity field \(\nu _{i,t}\).

The framework is based on the extension of Voxelmorph for learning conditional atlases by Dalca et al. [4]. An overview of the framework can be found in Fig. 1. Here, we only train the registration framework and we initialize the atlas for every time t as the voxelwise median over all images \(I_{i} \forall i\). The median was chosen over the mean, since this resulted in a sharper initial atlas. Next, the atlas is updated for iteration n as the mean of \(I_{i,t} \circ \phi _{i,t}^{n}\) for every time t. Subsequently, the network is trained until \(A_{t}^{n} \approx A_{t}^{n-1}\).

The loss function is defined as follows:

$$\begin{aligned} \begin{aligned} \mathcal {L} \left( A_t, I_{i,t},\phi _{i,t},\phi _{i,t}^{-1}\right)&= \lambda _{\text {sim}}\mathcal {L}_{\text {similarity}}\left( A_t\circ \phi _{i,t}^{-1},I_{i,t}\right) + \lambda _{\text {group}}\mathcal {L}_{\text {groupwise}}\left( \phi _{i,t}\right) \\ {}&+\,\lambda _{\text {mag}}\mathcal {L}_{\text {magnitude}}\left( \phi _{i,t}^{-1}\right) + \lambda _{\text {dif}}\mathcal {L}_{\text {diffusion}}\left( \phi _{i,t}^{-1}\right) \end{aligned} \end{aligned}$$
(1)

The first term computes the similarly between the atlas and image, we used the local squared normalized cross-correlation, which was used before on this dataset [2]. The second term approximates the constraint for groupwise registration by minimizing the running average over the last c deformation fields obtained during training. To balance the influence of this constraint with respect to time, we sorted the data based on day GA within every epoch and took as window c the average number of images per day GA in the dataset. Finally, the deformations are regularized by: \(\mathcal {L}_{\text {mag}} = \Vert \phi _{i,t}^{-1} \Vert _2^2\) and \(\mathcal {L}_{\text {dif}} = \Vert \nabla \phi _{i,t}^{-1} \Vert _2^2\).

Fig. 1.
figure 1

Overview of the proposed framework and characteristics of the used dataset.

Full size image

3 Data and Experiments

The Rotterdam Periconceptional Cohort (Predict study) is a large hospital-based cohort study conducted at the Erasmus MC, University Medical Center Rotterdam, the Netherlands. This prospective cohort focuses on the relationships between periconceptional maternal and paternal health and embryonic and fetal growth and development [14, 17]. 3D ultrasound scans are acquired at multiple points in time during the first trimester. Here, to model normal development, we included only singleton pregnancies with no adverse outcome and spontaneous conception with a regular menstrual cycle.

We included 871 ultrasound images of 398 pregnancies acquired between 56 and 90 days GA. For each day GA, we have at least 10 ultrasound images, as shown in top-right graph in Fig. 1. The data was split such that for every day GA \(80\%\) of the data is in the training set and \(20\%\) in the test set. We first spatially aligned and segmented the brain using our previously developed algorithm for multi-atlas segmentation and registration of the embryo [2]. Next, we resized all images to a standard voxelsize per day GA, to ensure that the brain always filled a similar field of view despite the fast growth of the brain. This standard voxelsize per day GA was determined by linear interpolation of the median voxelsize per week GA. We trained the network using the default hyperparameters proposed by Dalca et al. [4] for \(\lambda _{\text {group}} \in \{0,1, 10, 100\}\). We reported the mean percentage of voxels having a non-positive Jacobian determinant \(\% |J|\le 0\), the groupwise loss \(\mathcal {L}_{\text {group}}\) and the similarity loss \(\mathcal {L}_{\text {sim}}\). Finally, for the best set of hyperparameters the atlas was updated iteratively, and we visually analyzed the result.

4 Results

From the results given in Table 1 for iteration \(n=1\) we concluded that all tested hyperparameters resulted in smooth deformation fields, since the percentage of voxels with a non-positive Jacobian determinant \(\%|J|\le 0\) over the whole dataset was less then one percent. Furthermore, we observe that for \(\lambda _{\text {group}}=1\) \(\mathcal {L}_{\text {group}}\) is similar to not enforcing the groupwise constraint. For \(\lambda _{\text {group}}=100\), we observed that \(\mathcal {L}_{\text {sim}}\) deteriorated, indicating that the deformation fields are excessively restricted by the groupwise constraint. Hence, \(\lambda _{\text {group}}=10\) was used to iteratively update the atlas. Finally, note that the difference between results for training and testing are minimal: indicating a limited degree of overfitting. In Fig. 2 a visualization of the results can be found for \(t=68\) and \(t=82\). In the showed axial slices the choroid plexus and the fourth ventricle can be observed.

Table 1. Results for different hyperparameters, with the standard deviation given between brackets.
Full size table
Fig. 2.
figure 2

Axial slice of the atlas for different GA and iterations 0, 1, 2 and 3.

Full size image

5 Discussion and Conclusion

We propose a deep learning approach for creation of a 4D spatio-temporal atlas of the embryonic and fetal brain using groupwise image registration. Here, we trained the registration network iteratively and visually inspected the resulting atlas. We found that the registration network results in smooth deformation field, and that several relevant brain structures were visible in the atlas.

In this work, the window c of the groupwise loss term was set to the mean number of samples per day GA, in future work this hyperparameter will be varied to study its influence. As shown in Fig. 1, in future work also the atlas generator network will be incorporated, where constraints for temporal smoothness and sharp edges in the atlas can directly be incorporated in the loss. Finally, we will evaluate if the relevant brain measurements of the atlas are close to clinically known values and we will analyze if the morphology of the brain, modelled by the deformations \(\phi _{i,t}\), shows the known correlation with maternal periconceptional health factors found in previous research [9, 10].