Registration of Range Images Using a Novel Technique of Centroid Alignment

Abstract

Here, the problem of 3D face registration across poses is addressed using the concept of alignment of principal components. First the alignment is done based on some coarse registration and then fine registration is computed using Iterative Closest Point. The registration scheme used is novel because for computing the coarse registration the distance between the centroids has been used. Subjective evaluation of registered images shows excellent registration.

1 Introduction

Recognition of human faces has been an important field of research. Past few years have found 3D face recognition to be more important than 2D face recognition, because of its enormous capability to handle pose, expressions, and occlusions [1–3]. Earlier, the problem of face recognition in humans had mainly relied on 2D images. The approach had certain limitations. Moreover, the information of a 2D face is often confined to pixel information only, thus making it difficult to assess the role of 3D shape processing. Also, a 3D face has more views that are necessary to register for effective face recognition. The main characteristics of the present work have been enlisted below:

• 1. A new approach has been attempted which registers 3D faces based on their centroids.

• 2. This approach of centroid-based registration works well on poses.

• 3. The present method of registration is fully automatic and less complex, unlike landmark-based registration methods.

In Sect. 2, a literature survey has been given on 3D face registration. The present method has been discussed in Sect. 3. A comparative analysis with other methods existing in the literature has been demonstrated in Sect. 4. In Sect. 5, conclusion and future scope have been enlisted.

2 Literature Survey on 3D Face Registration

Illumination, expression, occlusions, and pose easily affect the performance of recognition rates in case of 2D face images because, 2D face images are incapable of capturing the depth information that is usually found only in case of 3D face images. Nowadays, it is possible to overcome these problems by 3D technology. Here, a survey is presented, which aims to address the various challenges and issues challenged by different 3D face registration techniques. Image Registration is defined as the process to align two different point clouds to each other. One point cloud is selected to be the reference, i.e., in frontal pose, and the other point cloud is the non-frontal, which is to be registered to the frontal point cloud. The process of registration continues till a minimum distance is attained between the frontal and the registered point cloud. There are two broad classifications of face registration:

• (1) Coarse Registration Methods

• (2) Fine Registration Methods

• (a) Coarse Registration Methods: This form of registration could be categorized into the following:

  • (i) Point Signature—This [4] descriptor is only for finding correspondences between two frontal and unregistered point clouds. The process is quite fast. However, the algorithm is very complex, and it is the main drawback of this algorithm.

  • (ii) Spin Image—This [5] is a two-dimensional visualization. Initially, the method was used for recognition. One problem of the method is the resolution of the image. So, a filter is used, in this case, to remove false triangles and to give a smooth finish to the interpolated image.

  • (iii) Principal Component Analysis—PCA [6] is usually used as a popular method for dimensionality reduction. This process is used for the alignment of two point clouds. The disadvantage of principal component analysis is to cope with surfaces containing symmetries.

  • (iv) Algebraic Surface Model—Here [7], at first, from the two point clouds two implicit polynomial models are determined based on least squares method. However, this method is far better than (i) and (ii) above because it requires less time. The model is computed on the basis of a function of the distance between polynomial models and the points.

• (b) Fine Registration Methods: This form of registration is used when a coarse registration has already been computed. The fine registration is computed based on an initial guess to converge to an optimal solution. The optimal solution is that the distance function between the two point clouds must be minimized after fine registration. Following are the categories of fine registration:

  • (i) Iterative Closest Point

  • (ii) Matching Signed Distance Fields

  • (iii) Procrustes Analysis.

  Fine registration methods could be categorized into the following:

  • (a) ICP—Besl and McKay first presented the ICP [8] method. Here, the main aim is to obtain a solution in terms of fine registration between two different point clouds. In this case, an initial estimation between the two different point clouds is known, and so, all the points are thereby transformed to a common reference frame that is the frame for the registered image.

  • (b) Matching Signed Distance Fields—The method [7] is based on matching of signed distance fields and that is a multiview type of registration. First, all views are aligned with a common system based on an initial estimation of motion. Then, some key-points are generated on the three-dimensional mesh grid. Then, the closest key point would be searched to find out the necessary translational and rotational parameters required for the purpose of registration.

  • (c) Procrustes Analysis—Landmark forms the basis of the registration technique called Procrustes [8] analysis. For the alignment of two 3D shapes, landmarks are essential. So, the comparison of two shapes forms the basis of Procrustes analysis. Generalized orthogonal Procrustes analysis (GPA) is the registration of “k” sets of configurations.

3 Present Method

A depth map image (also called as a range image) is a 2.5D image. The speciality of the image is that at each (x, y) position of the image depth values of the image is stored, i.e., how far from the camera each point is situated. The 3D face registration is at this moment defined to be the process, by which, an unregistered point cloud is aligned to a registered or a frontal point cloud.

3.1 3D Face Registration

Face registration [1, 2] is defined to be the process to align an unregistered face image to a frontal face image. Registration requires some landmark points. So, registration is necessarily a transformation, which would align an unregistered point cloud to a registered point cloud (Fig. 1).

Fig. 1

Image for a Unregistered Image b Registered Image of a person

Face registration is important both in case of 2D and 3D face recognition. For registration, it is necessary to have a set of landmarks. For registration, translation of a probe image to a gallery image is necessary. The probe image is the unregistered point cloud. The gallery image is the image to which the probe image has to be registered. Point cloud is thus a 3D view of an image. The process of image registration can be of three different types [9]:

• (i) Registration of one face to another.

• (ii) Registration to a fixed face model.

• (iii) Registration to a coordinate system using facial landmarks.

The first approach registers two point clouds with an iterative procedure. One of the point clouds is the unregistered image while the other is the frontal image to which the unregistered image is to be registered. The second approach deals with the registration of an unregistered point cloud to a model that has been learned from a training set. Registration to a coordinate system using facial landmarks requires a mapping of the current landmarks of the facial system to the intrinsic coordinate system and thereby computing necessary translational and rotational parameters. The present approach may be visualized in the form of a block diagram in Fig. 2.

Fig. 2

Diagram showing the proposed approach (Range images are shown besides the 3D mesh for visualization purpose only)

The following are the steps of the proposed algorithm:

• (i) 2.5D Range Image Acquisition: The images acquired or downloaded are in the form of a Virtual Reality Modeling Language (VRML) file. The 3D data points to be visualized should be in the mesh format.

• (ii) 3D Face Registration by ICP [ 5, 6 ]: ICP is a method to register an unregistered face with a mean face template. In the present work, a simpler version of the ICP algorithm has been used. The ICP algorithm implemented in the present work inputs two 3D mesh images, one image in frontal pose and an unregistered 3D mesh image from the 3D database, and tries to find a closest match between the two oriented models to find correspondences between the two. At first, the smallest average distance between the frontal and the unregistered model is used as an input for a coarse registration, after which a fine registration is sought.

  ICP uses a transformation using a combination of:

  • a. PCA

  • b. SVD

  At first, a coarse transformation is computed by aligning two point clouds based on the basic initial estimation of the closest points. Then the principal components are extracted. Next, the principal components are aligned using a fine transformation, for which it is necessary to compute the rotation matrix, which is required to register the unregistered point cloud to the frontal one.

  • To perform the coarse registration, it is necessary to input the distance between corresponding points. The distance between the corresponding points (i.e., between the centroids) is passed as an input to the ICP algorithm, for the purpose of an initial coarse registration, after which the fine final registration is sought.

  • When the distance between the closest points is passed to the ICP algorithm, the first initial transformation is found between the principal component of the unregistered and the mean frontal point cloud using reduction by principal component analysis.

  • Next, a fine registration transformation is sought by aligning the two principal components and finally the 3D mesh is converted to the range image.

    The main contribution of the present work lies in the fact that, at first the smallest distance has been found between the two models for coarse registration. This distance is the distance between the centroids of the two point clouds.

Algorithm 1

ICP_Centroid_Align(model X, object P) //To Align Object P to model X

Input: Initialize by finding the smallest correspondence between each point on the three-dimensional surface between principal components of the unregistered image and frontal image. This smallest distance is taken to be the distance between the centroids of the two point clouds.

Output: Registered Image

Coarse Registration:

• 1. At first, compute the distance between the centroids of the registered and the unregistered point clouds and save the distance in a variable that called as cen.

• 2. Roughly align the principal components of the unregistered mesh to that of the neutral mesh by applying steps 2.1 to 2.3

  • 2.1 Compute the closest points in the unregistered mesh and find out the rotational and translational parameters

  • 2.2 Discard points in the unregistered meshes that are greater than cen.

  • 2.3 Find out the translational and rotational parameters by matching the closest points of the unregistered mesh and the neutral frontal mesh to which the unregistered model is to be registered

  • 2.4 Execute steps 2.1 to 2.3 for a fixed number of iterations

Fine Registration:

Input: Principal components of the coarsely registered mesh X, the neutral frontal mesh P.

Output : The final fully registered mesh.

• 3. Repeat steps 4 to 5 for a specified threshold.

• 4. For all points that belong to P find closest x ε X.

• 5. Transform each point of the coarsely aligned point cloud X to minimize distances between P and X.

• 6. Output the registered image X

Figure 3 shows some images from GavabDB database, which has been registered using centroid alignment technique for poses. Figure 4 shows the corresponding principal components for the registered faces.

Fig. 3

Some registered face images from GavabDB database with pose variations

Fig. 4

A snapshot of the registration technique applied: first column showing principal components of unregistered mesh, second column showing the principal components to which the unregistered point clouds is to be registered and the third column shows principal components of the final registered image

• (iv) Noise Removal: After registration of 3D mesh images have been done, the surfaces of 3D mesh images are processed for noise removal. For this, two steps are followed:

  • (a) Interpolation: Here, entire 3D mesh image is preprocessed using an interpolation [4, 10] technique namely the trilinear interpolation.

  • (b) Surface smoothing: Gaussian filter has been used for smoothing here. Gaussian smoothing uses a 2D distribution as a point-spread function, and convolution achieves this. Once a kernel has been calculated, then Gaussian smoothing can be performed.

• (v) 3D Face Recognition: The final stage, in the face recognition process, is feature extraction and classification. Here in this proposed work some efficient feature detection and extraction scheme is be used for final recognition preferably WLD feature extraction technique.

4 Comparative Analysis of the Present Approach with Other Methods Existing in the Literature

Table 1 depicts the manner in which the present method of face registration outperforms the already existing methods in the literature.

Table 1 A comparative analysis of present technique over other methods existing in the literature

Analysis: In Table 1 some of the most recent works on ICP and some other methods are enlisted. Table 1 clearly shows that the present method of 3D face registration outperforms all the other methods present in the literature. Most methods that are present in the literature have registered small poses using ICP, but here the present method is quite an innovative one because it registers extreme poses too using ICP by centroid alignment.

5 Conclusion and Future Scope of the Work

The proposed work demonstrates the fact that registration is an essential transformation used for ill-posed subjects. In this work, ICP has been utilized in an entirely novel way, and the method is proposed to register extreme poses also. So, here, in this proposed work, 3D face registration, its interpretability is expressed in registering extreme poses. Registration is a transformation that is used as a tool to convert an ill-posed object to its frontal one. Once a subject is registered in a frontal pose, after that, effective recognition could be done. Though the present method has been tried on GavabDB dataset, the same is proposed in future to be tried on some other existing datasets too. There are many advantages of 3D face recognition over 2D face recognition, and registration, as well as recognition, is an important field of research work. As a part of future work, a proposal is being made to register and recognize 3D faces across extreme poses by registering input face images using the registration technique mentioned in this present work.