Keywords

These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

1 Introduction

Following the rapid advancement of medical imaging techniques, medical image analysis plays an increasingly important role in the field of medical science. In neuroimaging studies, since brain structures of different subjects have variant shapes, brain images of different subjects are often transformed into a common space to facilitate statistical image analysis. Such a common space is typically defined by a brain image atlas, e.g., the MNI brain and the Talairach atlas [1].

Recently, groupwise image registration methods for registering multiple images have attracted interests due to their efficacy [2–11]. Instead of registering each individual image to a specific atlas image, the groupwise image registration methods register images to a common space identified automatically from the images to be registered. Such an image registration strategy can effectively reduce registration bias introduced by a specific image atlas, and therefore might be able to improve the subsequent image analysis. In this chapter, different kinds of groupwise image registration methods are introduced, and their performance in the registration of 3D MR brain images is evaluated.

Many methods for the groupwise image registration have been proposed in the last decade [2–10]. Except the congealing method that registers images by optimizing a groupwise image similarity, i.e., sum of entropies of pixel stacks [10, 12], most of the existing groupwise image registration methods are built upon pairwise image registration algorithms and achieve the final registration in two different ways: (1) registering all images directly to a template image that can be a representative image selected from the images to be registered (typically a group center image) or iteratively to their evolving average image [2, 5, 6], referred to as direct pairwise image registration based groupwise image registration methods; and (2) registering images to their similar images and then composing the resulting deformation fields to achieve the registration of all images to the representative image [3, 4, 7–9, 13, 14], referred to as intermediate template based groupwise image registration methods.

This chapter is organized as follows. Pairwise image registration methods are firstly briefly introduced, followed by introductions of groupwise image registration methods, including the congealing method, the direct pairwise image registration based, and the intermediate templates based groupwise image registration methods. Finally, the chapter is concluded with a brief discussion.

2 Pairwise Image Registration

The pairwise image registration methods find a spatial transformation for registering one image, often referred to as floating image, to another image, referred to as target image. A survey of image registration methods has classified image registration methods into feature based and area based methods [15]. To register two images (target and floating images), a feature based image registration method typically follows three steps. First, each image’s feature points are identified and characterized by simple image features such as line intersections [16] and corners [17] or more sophisticated image features, e.g., histogram of local image patch and SIFT features [18]. Second, matched feature points of the images to be registered are obtained by matching their image features. Finally, parameters of a transformation model, e.g., rigid transformation or projection mapping, are estimated to establish the correspondence between the matched feature points. The resulting parameters of the transformation model can be used to spatially transform the floating image to the target image’s space. Figure 1 shows two 2D images, each of them containing an arrow with different orientations. Once one gets at least two pairs of matched feature points, parameters of a rigid transformation (the rotation angle and translation) can be estimated to establish their spatial correspondence for achieving image registration.

Fig. 1
figure 1

Images contain an arrow shape with different orientations. By finding the matched feature points (marked as red points), one is able to estimate the rotation angle and translation of a rigid transformation for registering the images

Full size image

In the area based image registration methods, instead of feature points, image regions are considered. In other words, all image pixels within a region of interest are regarded as one feature “point”. Typically, parameters of certain transformation model in the area based image registration are determined based on a predefined cost function. The most commonly adopted cost functions include Pearson correlation coefficient for modeling linear relationship between intensities of images to be registered and mutual information for modeling their non-linear relationship [15].

The image registration methods can also be classified as either parametric image registration or non-parametric image registration based on the transformation model used in the registration. The advantage of parametric image registration methods is their high computational efficiency. In particular, linear parametric transformation models used in image registration include rigid transformation, affine transformation, and projection mapping, while Thin Plate Spline (TPS) and Free-Form Deformations (FFD) are typically adopted as non-linear parametric transformation models in image registration [19, 20].

Since the deformation modeled by a parametric model typically has low degrees of freedom, non-parametric image registration methods are often used for registering images with diffuse, local differences, e.g., brain images [21]. Among the non-parametric image registration methods, Thirion’s Demons algorithm [22] is one well-known method. The Demons algorithm is derived from the optical flow equation [23], and the deformation vector for each pixel is formulated as

$$\begin{aligned} {\varvec{\nu }}(x) = \left\{ {\begin{array}{ll} {\frac{{(I_{f} (x) - I_{t} (x)) \cdot \nabla I_{t} (x)}}{{\parallel \nabla I_{t} (x)\parallel ^{2} + (I_{f} (x) - I_{t} (x))^{2} }},} &{} {{\text {if}} \parallel \nabla I_{t} (x)\parallel ^{2} + (I_{f} (x) - I_{t} (x))^{2} > \varepsilon },\\ \qquad \qquad \qquad {0,} &{} \qquad \qquad \qquad {\text {otherwise}} \\ \end{array} } \right. \end{aligned}$$
(1)

where \(\varvec{\nu }\) is the deformation vector at each pixel \(x\), \(I_{f}\) and \(I_{t}\) are floating and target images, and \(\varepsilon \) is a small tolerance to improve the solution’s numerical stability. The Demons algorithm achieves image registration in an iterative way. In particular, at each iteration step, deformation vectors for all pixels are calculated and regularized by a Gaussian filter, and then the resulting deformation vectors are added to an accumulated deformation field. At the end, the accumulated deformation field is used to warp \(I_{f}\) to \(I_{t}\).

Fig. 2
figure 2

Illustration of the Demons. Left images with a ramp structure of gray values. Right top intensity profile and the deformation vectors of pixels at each column for registering image A to image B. Right bottom intensity profile and the deformation vectors of pixels at each column for registering image A to image C

Full size image

Figure 2 illustrates how the deformation vector is calculated by Eq. (1) for registering one image (denoted by A) to two different images (denoted by B and C). Each image contains a different ramp structure of gray values. For registering image A to image B, since the gray value of each column of A is smaller than the corresponding column in B, \(I_{f}(x)-I_{t}(x)\) is negative according to the Eq. (1). Therefore, the directions of deformation vectors for each column of A is opposite to the directions of image gradient of B as shown in the right top of Fig. 2. On the contrary, for registering image A to image C, the deformation vectors and the image gradient of C have the same direction since the gray value of each column of A is bigger than the corresponding column in C, as shown in the right bottom of Fig. 2.

The Demons algorithm has been improved for achieving better accuracy and higher efficiency in different ways and validated in different applications [22, 24–70]. Among its improved versions, the Diffeomorphic Demons algorithm has been widely adopted in the registration of brain images [65]. Since deformation vectors are projected onto the Lie groups by exponential maps, the Diffeomorphic Demons algorithm is able to obtain a diffeomorphic deformation field, i.e., no folding and invertible. This algorithm has also been used in groupwise image registration methods that are built upon pairwise image registration [8, 11, 71].

The pairwise image registration has been extended for registering multiple images using different strategies, referred to as groupwise image registration. The remainder of this chapter reviews representative groupwise image registration methods, including the congealing method, the direct pairwise image registration based and the intermediate templates based groupwise image registration. In particular, graph based groupwise image registration methods are introduced as special cases of the intermediate template based groupwise image registration.

3 Groupwise Image Registration Using Congealing

The congealing method was first proposed by Miller [72] for the task of image classification, and then was adopted for groupwise image registration [73]. Parametric deformation models are typically used in the congealing based groupwise image registration, including affine transformation and B-spline deformation [12, 73]. The congealing method in conjunction with the Diffeomorphic Demons algorithm has also been adopted in fMRI data registration [74].

In the congealing method, no template is required. The similarity among images to be registered is measured by the sum of pixel wise (voxelwise) entropies across the whole image space

$$\begin{aligned} E=\sum _{x=1}^{n}H(s(x)), \end{aligned}$$
(2)

where x is the spatial location of a pixel, n is the number of pixels in the image space, \(s(x)\) is a variable defined by the intensities of a pixel x across all of the images to be registered, and \(H(\cdot )\) is the entropy function. The variable defined by the intensities of a pixel x across all of the images is often referred to as an image stack, as illustrated by Fig. 3. The congealing method achieves groupwise image registration by iteratively deforming images one by one for minimizing the cost function defined by Eq. (2) until convergence. Its computational cost is very high since it requires one evaluation of the cost function defined by Eq. (2) for every update of the deformation parameters for each image.

Fig. 3
figure 3

Image stack used in the congealing method. The rectangles stand for input images and the red circles constitute a pixel stack at one pixel location \(x\)

Full size image

To achieve efficient groupwise image registration, many groupwise image registration methods are built upon pairwise image registration. Basically, pairwise image registration is adopted to achieve groupwise image registration in two ways: (1) registering all images directly to a template image that can be a representative image selected from the images to be registered (typically a group center image) or iteratively to their evolving average image [2, 5, 6]; and (2) registering images to their similar images, known as intermediate templates, and then composing the resulting deformation fields to achieve the registration of all images to the representative image [3, 4, 7–9, 13, 14].

4 Groupwise Image Registration Using Direct Pairwise Image Registration

It is an intuitive way to achieving the groupwise image registration by directly registering all images to a template image [2, 5, 6]. The basic idea is to choose or to produce an image serving as a template from a given group of images. Then all the images in the given image group are registered to the template to achieve the final alignment (Fig. 4). The key of this kind of methods is how to determine an optimal template.

Fig. 4
figure 4

Illustration of groupwise registration using direct pairwise image registration. Left input images; Right one input image is selected as a template and other images are registered to the template for achieving the final groupwise registration

Full size image

The template image can be defined as the geometric mean of the input images based on their pairwise registration results [5]. In particular, each image is first registered to all the other images, then the geometric mean is estimated using the Multi-Dimensional Scaling (MDS) based on the registration result of all image pairs, and finally the closest image to the geometric mean is chosen as the template. In [75], the template image was defined as the average image of deformed input images, and each deformed input image was produced by warping the input image with the average deformation field resulted from averaging all deformation fields determined from pairwise image registrations between this image and all the other images. However, the template selection methods have relatively high computational cost since the pairwise image registration is required for every possible pair of images. As mentioned in [75], it would take 4096 h, i.e., 170 days, to get the template for an image group containing 64 images using one processor.

Fig. 5
figure 5

Illustration of the iterative process of the groupwise registration method using the group average image as the template

Full size image

Another efficient way to obtain the template is to generate the average image from the input images iteratively [6]. Particularly, an average image of registered input images using affine transformation is first generated as the template image and then all images are registered to the template image using a nonlinear registration method. The registered images can be averaged again to generate a new template image so that the groupwise image registration is improved iteratively. The iteration process stops when all images have been well aligned or the maximum number of iterations exceeded. The flowchart of such a groupwise image registration algorithm is shown in Fig. 5.

At the very beginning of the iterative groupwise image registration, images are typically not well aligned and the resulting average image is blurred. Such a blurred template image could hamper the groupwise image registration. To overcome this problem, a sharp average image can be generated using an adaptive weighting strategy at each iteration step [7]. The weight for each image’s pixel/voxel at each iteration step is inverse proportional to the squared intensity difference between image patches centered at the pixel/voxel in the image considered and the group mean image produced in the previous iteration step, so that the pixel/voxel in an image more similar (i.e., similar image intensity) to that in the group mean image contributes more to the pixel/voxel intensity in the new group mean image. The resulting group mean image is often with sharp contrast.

5 Groupwise Image Registration Using Intermediate Templates

Since the registration of similar images is typically easier than the registration of images that are much different from each other, several groupwise image registration algorithms have been proposed to register images to their similar images, referred to as intermediate templates, and then composing the resulting deformation fields for achieving the registration of all images to a representative image of input images [3, 4, 7–9, 13, 14]. The basic idea of this kind of registration methods for registering image A to final template R via intermediate template B is illustrated by Fig. 6. To determine the optimal intermediate template for each image, different strategies have been proposed [14, 71].

Fig. 6
figure 6

Illustration of groupwise registration using intermediate templates. Red points indicate images. A is the image to be registered, B is the intermediate template, and R is the final template

Full size image

A statistical deformation model has been used to generate the intermediate templates in a brain image registration method called RABBIT [14]. In particular, the method first gets a set of deformation fields that register a template brain image to individual brain images, and then applies Principal Component Analysis (PCA) to the deformation fields for generating a statistical deformation model that is able to characterize a brain image deformation field with a small number of parameters. To register the template brain image to a brain image, the statistical deformation model is used to generate a deformation field that warp the template image to produce an intermediate template close to the brain image considered, then a deformation field that registers the intermediate template and the brain image considered is estimated by a pairwise image registration algorithm, and finally the obtained deformation field is composed with that generated by the statistical deformation model to register the template brain image to the brain image considered. Since the difference between the intermediate template image and the image considered is relatively small, a better registration result can be obtained.

Fig. 7
figure 7

Illustration of the registration strategy proposed in ABSORB. Given a group of image, represented by red points, to be registered, each image (e.g., A) is warped toward the group center with a deformation field obtained by averaging deformation fields warping A to its neighboring images (\({{N}_{1}},{{N}_{2}}\) and \({{N}_{3}}\)) that are closer to the group center than A itself. The procedure is iterated to warp all images to the group center image that is updated after each iteration step

Full size image

In most of groupwise image registration methods, the intermediate templates for each input image are its neighboring images chosen from the rest of input images. For example, in a method called ABSORB [71], each image’s neighboring images serve as its intermediate templates if the neighboring images are closer than itself to a group center image. The center image is determined by choosing the image near the group geodesic mean on an estimated manifold of input images using graph theoretic techniques (discussed in the Sect. 6). The groupwise image registration is then achieved by iteratively warping each input image with a mean deformation field obtained by averaging deformation fields that map the input image to its intermediate templates, as illustrated by Fig. 7.

Recently, graph theoretic techniques have been adopted in the groupwise image registration for learning a manifold of input images so that a large deformation between images can be decomposed into a series of small and anatomically meaningful deformations between similar images [3, 4, 8, 9]. The graph based groupwise image registration methods are intermediate template based registration methods too since some input images are used as intermediate templates of other images for achieving the final groupwise image registration. The representative groupwise image registration methods using graph theoretic techniques are introduced in Sect. 6.

6 Groupwise Image Registration Using Graphs

In the graph based methods, a graph of input images is constructed. In such a graph, each node corresponds to an input image and similar nodes are connected with edges. The weight of each edge connecting two images is determined by their similarity measure. The graph of images helps estimate the manifold of input images and the geodesic distance between two images can be approximated by the their shortest path in the graph identified using graphical algorithms, such as Dijkstra method [76]. Images on the shortest path between a pair of images can serve as intermediate templates and the registration between the pair of images can be achieved by subsequently composing deformations between all adjacent image pairs along their shortest path. Typically, a root image, which serves as the template, is first determined in the graph based methods, and then all images are registered to the root image along their corresponding shortest paths to the root. The template (or root) in the graph based methods is usually identified as a pseudo-geodesic mean of the images [8] by

$$\begin{aligned} I_{root}=\underset{I_{i}}{{argmin}} \sum _{j=1}^{n} g(I_{i},I_{j}),i=1,\cdots ,n, \end{aligned}$$
(3)

where \(g(I_{i},I_{j})\) is the length of the shortest path between images \(I_{i}\) and \(I_{j}\). Once the root image is determined and the shortest paths from non-root images to the root are fixed, a tree of images is obtained and the root of the tree is the group center of images. An example of such a tree with 6 images is illustrated in Fig. 8.

Fig. 8
figure 8

Illustration of the graph based groupwise image registration. Each circle denotes one image, and \(D_{xy},x, y=a,b,c,d,e,r\), \(x \ne y\), denotes a deformation field that registers image x to image y

Full size image

As shown in Fig. 8, there are 5 non-root images (denoted as \({a},{b},{c},{d},{e}\)) and each of them has a corresponding shortest path, following which the non-root image can be registered to the root image by subsequently composing pairwise deformation fields along the shortest path, i.e.,

$$\begin{aligned} D_{xr}=D_{xk_{1}}\circ D_{k_{1}k_{2}}\circ \cdots \circ D_{k_{n}r}, x,k_{1},k_{2},\cdots ,k_{n},r\in P_{xr}, \end{aligned}$$
(4)

where \(P_{xr}=(x,k_{1},k_{2},\cdots ,k_{n},r)\) is the shortest path from image x to the root image r, and \(D_{xr}\) denotes the deformation field mapping x to r. For instance, in Fig. 8 the shortest path of image c to the root image r is \(P_{cr}=(c,a,r)\) and the registration of image c to the root image can be achieved by subsequently composing deformation fields \(D_{ca}\) and \(D_{ar}\) , i.e., \( D_{cr} = D_{ca} \circ D_{ar}\). The graph based registration methods help decompose a large deformation into small ones, and it is relatively easier to achieve accurate results for the registration of similar images than the registration of images with larger differences.

In the rest of this section, typical graph based methods are discussed in detail, and their performance in the registration of MR brain images is evaluated.

6.1 kNN Graph Based Groupwise Image Registration

Most of the graph based groupwise image registration methods adopt \(k{\text {NN}}\) graph to build the graph of images [4, 8]. In a \(k{\text {NN}}\) graph, each node is connected to its k nearest neighboring nodes that are determined based on an image similarity measurement. The typically used image similarity is the Euclidean distance between image intensities defined as

$$\begin{aligned} d_{Euclidean}(I_{i},I_{j}) =\left( \sum _{x=1}^{n}(I_{i}(x)-I_{j}(x))^2\right) ^{1/2}, \end{aligned}$$
(5)

where n is the total number of pixels/voxels in each image, \(I_{i}(x)\) and \(I_{j}(x)\) are the image intensities at location \(x\) in images \(I_{i}\) and \(I_{j}\) and respectively. An example kNN graph is shown in Fig. 9.

Fig. 9
figure 9

An example of the graph construction using \({k}{\text{ N }N}\) algorithm. Each node is connected to its k nearest neighbors (\({k}=3\)). For instance, node A is connected to B, C and D

Full size image
Fig. 10
figure 10

A set of synthetic images that have two peaks of different heights

Full size image

The \(k{\text {NN}}\) graph has been demonstrated to be able to estimate the manifold of high-dimensional data such as images [77]. However, the performance of \({k}{\text {NN}}\) graph based method is sensitive to the selection of k. Therefore, the resulting shortest paths identified in a kNN graph, i.e., geodesic distances measured in the estimated manifold, are unstable. The sensitivity of \({k}{\text {NN}}\) graph based method is illustrated by the estimation of the manifold of synthetic images shown in Fig. 10. These images lie on a one dimensional manifold. The \({k}{\text {NN}}\) graph based estimation result varies very much with the parameter k, as indicated by the identified shortest paths of images to a group center image shown in Fig. 11. In particular, \({k}{\text {NN}}\) graphs with different settings of k are constructed based on similarity measures defined by Eq. (5), and then Dijkstra method is used to identify the shortest path from each image to the group center determined using Eq. (3). It is clear that the shortest paths for some images are changed dramatically with different settings of k. It is worth noting that the number in each circle shown in Fig. 11 corresponds to the image with the same number in Fig. 10.

Fig. 11
figure 11

The shortest paths derived from \({k}{\text {NN}}\) graphs with different values of k. The root image is marked as a gray circle and the ID in each circle corresponds to the image with the same ID in Fig. 10

Full size image

The graph of images can also be constructed using a so-called \(\epsilon \)-ball method in a similar way to the \({k}{\text {NN}}\) graph method. Different from the \({k}{\text {NN}}\) graph method, the \(\epsilon \)-ball method determines the neighboring images with connections to a given image in the graph using a distance threshold. In particular, given a group of images, each image’s neighboring images are those within a ball centered at the image considered with radius \(\epsilon \). Unfortunately, the \(\epsilon \)-ball method has the same problem as the \({k}{\text {NN}}\) graph method.

6.2 kNN+MST Based Groupwise Image Registration

Built upon the \({k}{\text {NN}}\) graph method, the Minimum Spanning Tree (MST) method has also been adopted to identify the shortest paths for images in groupwise image registration [4, 7]. The MST is a tree with minimal sum of edge weights. Two commonly used algorithms for constructing a MST are Prim’s algorithm [78] and Kruskal’s algorithm [79], and the latter can also be used to construct a minimum spanning forest if the graph is not connected, i.e., a MST for each connected component. The pseudo-code of Kruskal’s algorithm for connected graphs is summarized in Algorithm 9.1.

figure a

Figure 12 shows an example how to find the MST in a graph where each edge is weighted by a distance measure as indicted by the number next to itself. Starting from a void set, the Kruskal’s algorithm finds the MST in the graph by adding one edge with the minimal distance step by step.

It is worth noting that the MST is usually generated from a graph obtained by the \(k{\text {NN}}\) graph method in the groupwise image registration methods [4]. Therefore, it suffers from the same problem as the \({k}{\text {NN}}\) graph based groupwise image registration methods.

Fig. 12
figure 12

Illustration of the construction of MST in a graph where each edge has a distance measure as indicated by the number next to itself. Kruskal’s algorithm is applied to find the MST by adding one edge (in black) with the minimal distance step by step as illustrated by the figures shown from top left to top right, then bottom left to bottom right. The resulting MST is shown in the bottom right

Full size image

6.3 Sparse Graph Based Groupwise Image Registration

A sparse coding based algorithm was proposed recently for constructing a sparse graph of images, referred to as \(\ell ^{1}\)-graph [80]. Particularly, given a group of images \(I_{i},i = 1, \cdots ,m\), each with n voxels, the algorithm measures similarities among images using sparse coding [81]. Mathematically, the similarity measures for one image, e.g., \(I_{i}\), to all the other images can be derived from the solution of the sparse coding problem

$$\begin{aligned} \underset{\alpha }{{min}}\parallel I_{i} - D{\alpha } \parallel +\lambda \parallel \alpha \parallel _{1}, \end{aligned}$$
(6)

where \(I_{i}\) is an n dimensional column vector of image intensities, \(D = [I_{1} , \ldots ,I_{i-1} , \ldots ,I_{i+1} , \ldots ,I_{m} ] \in {\mathbb {R}}^{{n \times m-1}}\) is a matrix containing all the other images, \(\alpha \) is a (m-1)-dimensional vector containing the coefficients corresponding to images in D. Since each element of \(\alpha \) is proportional to the similarity measure between \(I_{i}\) and its corresponding image in D, its inverse or a monotonically increasing function of its inverse can be used to measure the image distance. According to the sparse coding based image distance measures, one can build a directed graph of images in which each node’s outgoing edges to other images are weighted by its distance measures to other images calculated based on the sparse coding. The advantage of the \(\ell ^{1}\)-graph method over the kNN graph is that each image’s neighbors are determined adaptively to the image itself and the resulting graph is sparse due to the L1 norm constraint. Furthermore, this method is more robust to its parameter \(\lambda \) . As shown in Fig. 13, the shortest paths (using Dijkstra method) and the geodesic group center derive from the \(\ell ^{1}\)-graph with different settings of \(\lambda \) for the images shown in Fig. 10 are stable and the shortest paths are consistent with the underlying manifold of the images.

Fig. 13
figure 13

The shortest paths derived from \(\ell ^{1}\)-graph with different values of \(\lambda \). The root image is marked as a gray circle and the number in each circle correspond to the image with the same number in Fig. 10

Full size image

A groupwise image registration method built upon the sparse graph of images, referred to as dynamic sparse graph based method in this chapter, has been proposed in [11]. In particular, a sparse graph of images is constructed based on an adaptively weighted sparse coding method for estimating image similarity measures among different images. Different from the sparse coding problem formulated by Eq. (6), a weight vector is introduced in the adaptively weighted sparse coding

$$\begin{aligned} \mathop {\min }\limits _{\varvec{\alpha }} \left\| {I_{i} - D\varvec{\alpha }} \right\| + \lambda \left\| {diag({{\varvec{w}}})\varvec{\alpha }} \right\| _{1}. \end{aligned}$$
(7)

where \(I_{i}\) is an n-dimensional column vector of image intensities, \(D = [I_{1}, \ldots ,I_{i} , \ldots ,I_{m} ] \in {\mathbb {R}}^{{n \times m}}\) is a matrix containing all the input images, \(\alpha \) is an m-dimensional vector containing the coefficients corresponding to images in D, and \({\varvec{w}} = (w_{1} , \ldots , w_{m} ) \in {\mathbb {R}}^{m}\) is a weight vector for defining the participation of images in D. To avoid a trivial similarity measurement of \(I_{i}\), its corresponding weight \(w_{i}\) in w is set large enough to prevent \(I_{i}\) from representing itself.

Different from the existing graph based groupwise image registration methods [4, 8], this method then achieves the groupwise image registration in an iterative fashion. Once a graph of images is obtained, each image is registered to its direct parent image, and the graph of images is updated based on the registration results. Then, according to the updated graph of images, each image is again registered to its direct parent image in the current graph. This procedure is repeated until all images are registered to the root image. At every iteration step, images with the same directed parent image become close to each other and form a cluster after they are registered to the same parent image. If the sparse coding as formulated in the Eq. (7) with the same setting, i.e., fixed weight w, is used to estimate image similarity measures for the registered images, the images in the same cluster might represent one another, resulting in larger values of similarity measures among themselves and small values of similarity measures (mostly 0) with the other images outside of the cluster. Therefore, some of images in the same cluster may have the shortest paths traversing others in the same cluster as illustrated by Fig. 14b, or all the images in one cluster are isolated from images in other different clusters, thus generating a disconnected graph as illustrated by Fig. 14c. The registration of one image with the shortest path traversing multiple images highly similar to it requires multiple pairwise image registration between similar images, which increases the computation cost with little improvement of the overall registration accuracy. In a graph with disconnected components, one is not able to find a root image that is reachable from all the other images. To solve the problems illustrated in Fig. 14, the weight w is adaptively updated along with the image registration. Particularly, for calculating image similarity of image , large values are set to weight elements in w corresponding to the images that have the same direct parent image as \(I_{i}\) in the previous iteration step. Due to the constraint of the L1 norm in Eq. (7), images having the large weights will not participate in the representing of \(I_{i}\). Therefore, images in the same cluster are isolated from each other and the problems illustrated in Fig. 14b and c can be solved, as illustrated by Fig. 14d.

Fig. 14
figure 14

Illustration of the shortest path of each image to the root identified in the updated graph of images with the image similarity measures estimated using the sparse coding with fixed and adaptive weight w. Red dots numbered 1–8 and R denote different images, and R is the root image identified in the graph of images. a The shortest paths in the directed graph at one of the iteration steps. Images 1–4 have the same parent image 5, and form a cluster. Images 5–8 have the same parent image R and form another cluster. b The shortest path for image 4 derived from updated graph using the sparse coding with fixed weight w. c The updated graph becomes disconnected if the image similarity measures are estimated using the sparse coding with fixed weight w. d The shortest paths of images derived from the updated graph with image similarity measures estimated using the sparse coding with adaptive weight w

Full size image

7 Groupwise Registration of Brain Images

In this section, applications of groupwise image registration methods to neuroimaging studies are presented. In particular, three graph based groupwise image registration methods, including kNN, kNN+MST and sparse graph, are evaluated based on two public 3D MR brain image datasets, including LPBA40 [82] and NIREP-NA0 [83]. The same pairwise image registration, namely Diffeomorphic Demons [65], is used in all of the three methods.

The LPBA40 dataset consists of 3D MR brain images obtained from 40 normal subjects and each of them has a manually labeled image with 54 brain regions. The NIREP-NA0 dataset contains 3D MR brain images obtained from 16 normal subjects and corresponding manually labeled images with 32 regions. The overlap for the labeled regions across different images can be used to evaluate the performance of image registration algorithms. Some images and their corresponding label images from both datasets are shown in Figs. 15 and 16, respectively.

Fig. 15
figure 15

2D slices of 8 randomly selected 3D MR brain images from LPBA40 and their corresponding label images

Full size image
Fig. 16
figure 16

2D slices of 8 randomly selected 3D MR brain images from NIREP-NA0 and their corresponding label images

Full size image

All the T1-weighted images are affine-registered to MNI152 space using FLIRT after preprocessed using histogram matching, and the transforms are then applied to their corresponding individual label images [84]. For each dataset, an average image is obtained and its 3D surface rendering is shown in Fig. 17. As the 3D surface rendering results shown, the average images are blurred, especially in gyri, indicating that the images cannot be well aligned using an affine image registration algorithm due to large inter-subject differences. The registered images using affine transformation are used as input images in the graph based groupwise image registration methods.

Fig. 17
figure 17

3D surface rendering results of average images of LPBA40 (left) and NIREP-NA0 (right)

Full size image

7.1 kNN Graph Based Groupwise Image Registration

Images in the datasets of LPBA40 and NIREP-NA0 are registered separately using the kNN graph based groupwise image registration method [8]. In particular, the kNN graph was adopted to estimate the underlying manifold of input images and Dijkstra method was used to find the shortest path between each pair of images. After the determination of the root image using Eq. (3), deformation fields mapping images to their direct parent images along the corresponding shortest paths were calculated. The deformation field mapping each image to the root image was produced by composing deformation fields along its shortest path as formulated in Eq. (4). It is worth noting that the image distance in the construction of kNN is the Euclidean distance defined in Eq. (5).

Figure 18 left shows the tree of images formed by the shortest paths derived from the kNN graph of LPBA40 dataset with k set to 3. These 2D Points in Fig. 18 are projection results of 3D MR brain images in LPBA40 dataset by applying Principal Component Analysis (PCA) to reduce the high dimensional original images to 2D points. The surface rendering results of average of the registered images and the root image are shown in Fig. 18 right. Compared with the surface rendering result of the average image shown in Fig. 17, the one shown in Fig. 18 has sharper contrast, indicating better registration results have been achieved by the kNN graph based groupwise image registration algorithm. Registration results of NIREP-NA0 dataset are shown in Fig. 19 with the same layout as Fig. 18.

Fig. 18
figure 18

Left Tree of images formed by the shortest paths from each image (red point) to the root image (blue point) derived from the kNN graph of LPBA40 dataset. Right Surface rendering results of average image of registered images and the root image (within the red rectangle on the right bottom)

Full size image
Fig. 19
figure 19

Left Tree of images formed by the shortest paths from each image (red point) to the root image (blue point) derived from the kNN graph of NIREP-NA0 dataset. Right Surface rendering results of average image of registered images and the root image (within the red rectangle on the right bottom)

Full size image

7.2 kNN+MST Based Groupwise Image Registration

Images in the datasets of LPBA40 and NIREP-NA0 are registered separately using the kNN+MST based groupwise image registration method [4]. Figure 20 left shows the resulting kNN+MST of LPBA40 dataset and surface rendering results of the average image of registered images and the root image. The results of NIREP-NA0 dataset are shown in Fig. 21 with the same layout as Fig. 20. Visually, these results are similar to those obtained by thekNN graph based groupwise image registration method.

Fig. 20
figure 20

Left kNN+MST of LPBA40 dataset. Right Surface rendering results of average image of registered images and the root image (within the red rectangle on the right bottom)

Full size image
Fig. 21
figure 21

Left kNN+MST of NIREP-NA0 dataset. Right Surface rendering results of average image of registered images and the root image (within the red rectangle on the right bottom)

Full size image

7.3 Sparse Graph Based Groupwise Image Registration

Images in the datasets of LPBA40 and NIREP-NA0 are registered separately using the dynamic sparse graph based groupwise image registration method [11]. Figure 22 shows the tree formed by the shortest paths derived from the graph at each iteration step for the registration of images in LPBA40 dataset.

Fig. 22
figure 22

Left Trees formed by the shortest paths derived from corresponding graphs of LPBA40 dataset at four different iteration steps. Right Surface rendering results of average image of registered images and the root image (within the red rectangle on the right bottom)

Full size image
Fig. 23
figure 23

Left Trees formed by shortest paths derived from corresponding graphs of NIREP-NA0 dataset at two different iteration steps. Right Surface rendering results of average image of registered images and the root image (within the red rectangle on the right bottom)

Full size image

Registration results of NIREP-NA0 dataset can be found in Fig. 23 with the same layout as Fig. 22. These results are not visually different from those obtained by the kNN graph based groupwise image registration method and kNN+MST graph based groupwise image registration method. However, it seems that the group average images obtained by the sparse graph based groupwise image registration method are closer to their corresponding root images for both LPBA40 and NIREP-NA0 datasets. Such the observation is supported by the quantitative evaluation results, presented in the following section.

7.4 Evaluation of Graph Based Groupwise Image Registration

The registration performance of different graph based groupwise image registration methods has been evaluated quantitatively using overlap measures among different labeled regions of registered images. Particularly, the overlap between corresponding regions of registered images is evaluated using Dice index [85]. Given two regions X and Y, their overlap can be measured by \(DI = \frac{{2\left| {X \cap Y} \right| }}{{\left| X \right| + \left| Y \right| }}\) . To eavluate the registation performance of a given method, we obtain the overlap measure for each labeled region between every pair of images. So, given registration results obtained by one registration method, we obtain Dice index values for \(\left( {\begin{array}{c} {40} \\ 2 \\ \end{array} } \right) = 780\) image pairs and \(\left( {\begin{array}{c} {16} \\ 2 \\ \end{array} } \right) = 120\) image pairs for the LPBA40 and NIREP-NA0 datasets, respectively.

Besides the overlap measures for different regions, we also calculated the weighted average Dice index value across of all the labeled regions for each pair of registered images for the LPBA40 and NIREP-NA0 datasets, which is formulated as

$$\begin{aligned} DI_{w} = \mathop \sum \limits _{{i = 1}}^{n} \frac{{Vol_{i} }}{{Vol_{w} }}DI_{i} , \end{aligned}$$
(8)

where \(Vol_{w}\) is the volume of all labeled regions, \(Vol_{i}\) is the volume of the i-th region, n is the total number of labeled regions, and \(DI_{i}\) is Dice index value of the i-th region.

Fig. 24
figure 24

Dice indexes of registered brain regions of LPBA40 dataset using kNN, kNN+MST and dynamic sparse graph based groupwise image registration methods. The region ID’s corresponding brain region name is following. 1/2: L/R superior frontal gyrus; 3/4: L/R middle frontal gyrus; 5/6: L/R inferior frontal gyrus; 7/8: L/R precentral gyrus; 9/10: L/R middle orbitofrontal gyrus; 11/12: L/R lateral orbitofrontal gyrus; 13/14: L/R gyrus rectus; 15/16: L/R postcentral gyrus; 17/18: L/R superior parietal gyrus; 19/20: L/R supramarginal gyrus; 21/22: L/R angular gyrus; 23/24: L/R precuneus; 25/26: L/R superior occipital gyrus; 27/28: L/R middle occipital gyrus; 29/30: L/R inferior occipital gyrus; 31/32: L/R cuneus; 33/34: L/R superior temporal gyrus; 35/36: L/R middle temporal gyrus; 37/38: L/R inferior temporal gyrus; 39/40: L/R parahippocampal gyrus; 41/42: L/R lingual gyrus; 43/44: L/R fusiform gyrus; 45/46: L/R insular cortex; 47/48: L/R cingulate gyrus; 49/50: L/R caudate; 51/52: L/R putamen; 53/54: L/R hippocampus; and 55: weighted average Dice index of all the brain regions

Full size image
Fig. 25
figure 25

Dice indexes of registered brain regions of NIREP-NA0 dataset using kNN, kNN+MST and dynamic sparse graph based groupwise image registration methods. The region ID’s corresponding brain region name is following. 1/2: L/R occipital lobe; 3/4: L/R cingulate gyrus; 5/6: L/R insular gyrus; 7/8: L/R temporal pole; 9/10: L/R superior temporal gyrus; 11/12: L/R inferior temporal gyrus; 13/14: L/R parahippocampal gyrus; 15/16: L/R frontal pole; 17/18: L/R superior frontal gyrus; 19/20: L/R middle frontal gyrus; 21/22: L/R inferior gyrus; 23/24: L/R orbital frontal gyrus; 25/26: L/R precentral gyrus; 27/28: L/R superior parietal gyrus; 29/30: L/R inferior parietal lobule; 31/32: L/R postcentral gyrus; and 33: weighted average Dice index of all the brain regions

Full size image

Figure 24 shows the evaluation results based on the LPBA40 dataset for kNN, kNN+MST, and dynamic sparse graph based groupwise image registration methods. The evaluation results include Dice index values for 54 brain regions (ID: 1–54) and the weighted average Dice index value of all the brain regions (ID: 55).

Figure 25 shows the evaluation results based on the NIREP-NA0 dataset for methods including kNN, kNN+MST, and dynamic sparse graph based groupwise image registration methods, including Dice index values for all 32 brain regions (ID: 1–32) and the weighted average Dice index value of all the brain regions (ID: 33).

In summary, the average value of the weighted average Dice index values of all the brain regions of LPBA40 dataset are 0.692 (kNN), 0.691 (kNN+MST), and 0.709 (dynamic sparse graph). The paired t-tests revealed that the dynamic sparse graph performed better than kNN and kNN+MST with p values \(< 1.0 \times 10^{-20}\). For the NIREP-NA0 dataset the values are 0.628 (kNN), 0.622 (kNN+MST), and 0.639 (dynamic sparse graph). Again, the paired t-tests revealed that the dynamic sparse graph performed better than kNN and kNN+MST with p values \(<1.0 \times 10^{-20}\). It is evident that the registration accuracy of the dynamic sparse graph based method is higher than the kNN and kNN+MST based methods. Since Diffeomorphic Demons [65] is used in all of the three methods, the applied image similarity measures and graph construction strategies contribute to the performance difference of these methods.

8 Discussion and Conclusion

In this chapter, several groupwise image registration methods have been discussed. These groupwise image registration methods are typically used to generate a common template or to model the shape variation of objects of interest. Except the congealing method, most groupwise image registration methods achieve the image registration by registering images to a template image using pairwise image registration algorithms. The key points in the groupwise image registrations methods are template determination, registration path identification, and pairwise image registration. For the template determination, the ultimate objective is to choose an unbiased image as the template to which all images can be registered with almost equal effort. Several strategies for the template determination have been reviewed in this chapter, including the geometric mean, the geodesic mean, and the iterative average group image. For the registration path identification, the simplest way is to directly register image to the template. However, it is difficult to achieve accurate image registration when images to be registered have large difference. It typically leads to better performance if intermediate template images are adopted to reach to the final template since large difference can be decomposed into many small ones. For the pairwise image registration, which is used to register images to the intermediate template or the final template in the groupwise registration methods, many algorithms could be adopted, not limited to Diffeomorphic Demons adopted in this study.

We have been focusing on the graph based groupwise image registration methods in this chapter, including kNN, MST and sparse graph (including dynamic sparse graph), due to their high computation efficiency and accuracy. The basic idea of these methods is to estimate the underlying manifold of input images so that the shortest paths that approximate the geodesic distance between each pair of images can be derived. Based on all of the geodesic distances between images, the geodesic center of image can be chosen as a template image. Final deformation field mapping each image to the template can be obtained by composing deformation fields along its corresponding shortest paths. For such methods, image similarity measurement plays an important role. Therefore, in this chapter, the pairwise (e.g., Euclidean distance) and groupwise (e.g., sparse coding and adaptively weighted sparse coding) image similarity measurements have been reviewed. Evaluation of both kinds of similarity measurements using the LPBA40 and NIREP-NA0 datasets has demonstrated that the groupwise image similarity measurement can lead to better groupwise image registration performance than the pairwise image similarity measurements due to that the groupwise image similarity measures are estimated from a global point of view, while the pairwise image similarity measures considers only two images to be measured and no information of the other images is utilized.

Graph based manifold learning techniques have been increasingly adopted in the groupwise image registration methods to capture the distribution of images to be registered so that a group center image can be identified and the deformation between the group center and other images can be decomposed into small ones that register similar images pairwisely. For the graph based manifold learning techniques, a critical parameter is the neighborhood size for constructing a graph of images. Since the distribution of images to be registered is not necessarily uniform, especially in applications of medical image analysis with a limited number of images, the manifold of images estimated using kNN graph might be sensitive to its parameter, which inevitably renders the groupwise image registration unstable. Although the sparse coding based graph construction could be robust for estimating manifold of images with a non-uniform distribution, it merits further investigation how the image registration performance is hinged on its sparsity parameter.

Groupwise image registration algorithms have been successful in many applications of brain image analysis [4, 7, 10–12, 74]. However, all these algorithms have been focusing on imaging data of single image modality, either structural imaging data or functional imaging data. Groupwise registration of both structural and functional data might be helpful to better align the brain structure and function units across subjects [74, 86]. How to effectively integrate both structural and functional information of multimodality images for groupwise image registration merits investigation.