1 Introduction

Relief is a sculpting technique that models the shape of an object from a flat background. There are several relief types that can be classified based on the depth degree of the sculpture. Unlike high-relief that usually contains more than 50 % of the depth, bas-relief is a special type of a relief that has sculpted elements in a highly compressed form so that the shape of the sculpture has only shallow depth. Bas-relief has applications of producing commemorative medals, coins, souvenirs, or tactile artwork for the blind.

With the rapid development of 3D printing technology, bas-relief generation algorithms that allow users to automatically or semi-automatically create bas-relief of commemorative objects have become necessary. Most of the previous approaches for bas-relief generation receive their input in the format of 3D scenes. However, as smart phones and digital cameras become popular, algorithms that can receive their input in the format of a photographic image are becoming increasingly important. Bas-relief generation from a face photograph is especially useful because a human face is one of the most common subject in a photograph and subsequent artwork. For instance, we often see the face of politicians or celebrities on coins and souvenirs.

One of the simplest methods is to take each pixel brightness as a depth value and elevate the resulting surface according to the depth value. Although this approach generates a surface with realistic textures, its depth estimation is often wrong in terms of 3D shape, especially in the areas of facial features. Because estimating accurate depth information from a single photographic image is difficult, an algorithm that can overcome this problem is necessary.

From this motivation, we propose a new method that generates a bas-relief surface from the photographic image of a human face. Our method has the following unique characteristics and advantages:

  • Unlike previous approaches, we provide facial feature detection and enhancement (i.e. construction of the hair, eyes, eyebrows, nose, lips and etc.), thus making the resulting surface reflect the 3D shape of a facial features more accurately and naturally.

  • Our method maintains the advantage of representing realistic surface textures by reflecting all the pixel intensities of an input image for modeling the resulting surface.

  • Our method does not require a separate training or image database for facial shape modeling, thus making our method suitable for fast and convenient prototyping for further manual processing.

The proposed approach largely includes two phases. First, we detect facial features such as hair, eyes, eyebrows, nose, and lip. A top-hat transformation algorithm coupled with basic image processing techniques (i.e. Otsu’s method) is used to extract the small features of a face. We build the Face Parts Map Region (FPM-R) to locate the position and area of each feature on the face. Second, we adjust the pixel values in the area of the detected facial features so that the dark areas of the facial features can be elevated in the resulting relief surface. Finally, image smoothing and basic Shape-From-Shading (SFS) methods [18] are used to create a polygonal mesh of the bas-relief surface. The resulting bas-relief surface can be 3D-printed for manufacturing. The experimental results demonstrate that our method can generate a bas-relief that more accurately reflects the 3D shape of a human face where the facial features are more clearly and naturally expressed on the surface.

The remainder of this paper is organized as follows. After reviewing related papers in section 2, the overall process of our method for bas-relief generation is explained in section 3. In section 4, the techniques for facial feature detection are described. Section 5 describes our method for hair part extraction from face photograph. Section 6 describes our algorithm for extracting and enhancing facial features from a face photographic image for the generation of bas-relief surfaces. In section 7, experimental results are presented. Finally, section 8 forms the conclusion of this paper.

2 Related work

There are many types of reliefs such as bas-relief or low relief, high relief, counter-relief, and sunken relief. Since ancient times, humans have made bas-reliefs to transmit information on life experiences and express gratitude to their heroes or gods. Therefore, bas-relief has high artistic values. Currently, we can obtain bas-relief anywhere such as on coins, museum pieces, or home decorations. From this motivation, a significant amount of research has been conducted for bas-relief generation with focus on two input types: 3D scenes and images.

The first computer algorithm for generating bas-reliefs and high reliefs appeared in [2]. The authors proposed a method that describes the elevation of figures in bas-relief or high relief sculptures, and presented a simple algorithm. Bas-relief of a 3D scene can be regarded as a height field from the camera’s perspective. In [15], the authors presented a simple way that is able to produce bas-reliefs from input 3D scenes by finding the vantage point while conforming to depth compression. Both [11] and [5] used unsharp masking for emphasizing salient features before linear compression for exporting to a 3D mesh. [11] is based on differential coordinates, whereas [5] is based on the gradient domain. The methods [15] and [5] used a High Dynamic Range (HDR) imaging technique that compresses a large range of values while maintaining significant visual features. The method [15] was extended to [6], which applied non-linear scaling and used bilateral filtering to decompose gradients. A different approach [12] was developed that used and modified the adaptive histogram equalization method for effective depth compression. Although this method is simple compared with the method [15] in terms of concept and implementation, the quality of the results is equivalent or better depending on the input data. Finally, [19] developed a spatial bilateral filtering technique that can extract the feature details of an input 3D objects and generate a map with all the information on height domains. The algorithm in this paper also uses an adaptive compression method to adjust the Z-depth of the bas-relief mesh.

All the aforementioned methods start by making a depth map for the 3D scene, and perform linear or non-linear compression of depths in order to create the bas-relief surface. However, using these methods to produce high quality bas-relief requires obtaining suitable input 3D models, which can be a difficult process [1]. From this motivation, methods for bas-relief generation from images that can be easily and ubiquitously created using a smartphone have also been developed. However, a monocular image cannot provide sufficient information for retrieving the 3D depth information. In a monocular image, we only have information on pixel color intensities.

There are several approaches for creating bas-reliefs from input images. In [7], the authors provided an overview on how to easily export bas-reliefs from rubbing images. They used a modified SFS method with height constraints to create the relief surface. The base relief that is represented as the low frequency component is estimated. A partial differential equation (PDE) based mesh deformation is performed to generate the base relief. The high frequency height map is also estimated by inverting the input image pixel values and combining them with the base relief to generate the final height map that can be directly converted to a bas-relief mesh. Therefore, this method cannot be applied to our work because we want to control the height for each facial features. In [16], the authors performed image-based training through a feed-forward neural network. One of the main limitations of this method is its requirement for data training that requires significant amount of time. On the other hand, the paper in [17] is based on pixel intensity and gradient information in the image for dividing an image into distinct regions and elevating the bas-relief surface based on the boundaries of each region. The weakness of this approach is that it requires a process for region segmentation that is often difficult to automate accurately in complex images. For example, the face can be divided into many parts. However, the location of nose, mouth, and eyes is unknown. Hence, their approach has difficulty of making such parts distinctive. Our previous work [14] proposed a new method that uses facial part detection combined with the equation of line method for bas-relief generation from a face image. Techniques for bas-relief generation can be also similarly applied to creating tactile surface models from paintings [3, 10]. Object boundaries are extracted and depth of each region is computed or assigned to create the bas-relief surface.

3 Overview

The main goal of our approach is to generate a bas-relief surface from the photographic image of a human face. We assume that the input is an RGB-colored image. The color image should first be converted to the grayscale. As the initial step, a facial detection technique is applied to the input image. Using the facial feature detection techniques described in [9] and [13], we can compute the positions of features in a face photograph. As a result, we can extract rectangular regions that contain the facial features (i.e. hair, eyes, eyebrows, nose, and lip). Figure 1 (2) shows the result of the facial feature detection method. Image processing techniques, such as top-hat transformation and Otsu’s method, are used for the extraction of the exact feature areas. The goal of this step is to partition the image into small facial features clearly. Subsequently, FPM-R is built. The results obtained in this step are shown in Fig. 1 (4) and (5). When combined with facial feature detection, we can determine the exact area of each small face feature (i.e. eyebrows, eyes, nose, philtrum, mouth, teeth, face skin, and hair). Next, we adjust the z-depth values for each region by inverting the pixel values of the facial feature areas. Image smoothing methods are performed on the depth map to reduce the effect of noise. The generated depth map is then used for elevating and generating the bas-relief surface, which is our final result.

Fig. 1
figure 1

System overview. (1) an input lena image, (2) result of the facial feature detection, (3) grayscale image, (4) result of Otsu’s method, (5) result of top-hat transformation for feature enhancement, (6) Face Parts Map Region (FPM-R) generated by using the results of (4) and (5), and (7) bas-relief surface which is our final result

Full size image

4 Facial feature detection

Our approach utilizes a technique for facial feature detection [9, 13]. The technique is used to locate facial features in the form of rectangular regions that contain the facial features, that will be later utilized to extract more accurate area of the features using our method. The algorithm for the feature detection requires a 2D image as an input. Then, estimation of the pixel values in the entire face area is performed. A threshold value can be used to specify the face skin.

The first step of our framework is to detect the face and its components based on [9, 13]. By following this method, we can obtain individual features such as hair, and eyes and nose.

Based on YCbCr color, we can detect the face skin area with a given input face photograph. This helps us estimate the facial feature areas, and then we can detect the lips, mouth, right and left eyes. To manage this problem, we need to use a technique which is called Support Vector Machine (SVM). SVM is a concept of analyzing the data and then generating a decision plane for classification. The proposed algorithm uses the linear SVM based on the intensity values of RGB layers. After obtaining the position and size of the eyes, nose, and mouth regions, we continue by adjusting the brightness of the dark parts. Figure 1 (1) and (2) show an example of input image and the result of facial feature detection. The detected regions are obtained in the form of rectangular boxes where accurate feature areas can be extracted in subsequent steps.

5 Hair area extraction

Our goal here is to present a method for extracting the hair part from a face photograph by making a contrast between hair and other parts. In this regard, we use morphological operations as image processing for grouping pixels so that subsequent thresholding can be used to extract the hair area. We explain our approach step by step in this section. A problem might arise if we use a threshold for the input grayscale image. Our method aims to group all pixels into large areas. We perform dilation of the areas until obtaining the two largest areas using a threshold for the image generated by the last step of the morphological operations.

In order to detect the hair part (and other parts outside the face, such as a hat or the surrounding scene), we create a morphological structuring element with a disk shape. We set the size of the disk as 20 in our experimentation. Then, we use morphological erosion for the image. Let I and B denote an original image and a structuring element, respectively. The geodesic erosion of size n of an input image I with respect to the structuring element B is denoted as \( {E}_B^{(n)}(I) \). Through this step, the pixels in the image can be grouped.

$$ {I}_e={E}_B^{(n)}(I)={E}_B^{(1)}(I)\left[{E}_B^{\left(n-1\right)}(I)\right]=I\ominus B $$
(1)

where I e represents image erosion. The result is shown in Fig. 2a.

Fig. 2
figure 2

Results of morphological operations on input image I. Figure 1 (2) shows the input image I. (a) morphological erosion, (b) opening-by-reconstruction, (c) morphological dilation from the result of (b), and (d) opening-closing by reconstruction

Full size image

In the next step, we reconstruct the image with parameter I e . The opening-by-reconstruction I obr is computed as follows:

$$ {I}_{obr}={R}_I\left({I}_e\right)={R}_I\left(I\ominus B\right) $$
(2)

The result is shown in Fig. 2b. R represents mophological reconstruction.

Next, we compute the morphological dilation for I obr . The geodesic dilation of size n of an image I with respect to a structuring element B is denoted by D (n) B (I).

$$ {I}_{obrd}={D}_B^{(n)}\left({I}_{obr}\right)={D}_B^{(1)}\left({I}_{obr}\right)\left[{D}_B^{\left(n-1\right)}\left({I}_{obr}\right)\right]={I}_{obr}\oplus B $$
(3)

where \( {I}_{obrd} \) is calculated by morphological dilation of \( {I}_{obr} \). The result of this step is shown in Fig. 2c.

After completing the previous steps, we perform reconstruction again with two parameters that are the complement of \( {I}_{obrd} \) and \( {I}_{obr} \) to obtain opening-closing by reconstruction \( {I}_{obrcbr} \) as follows.

$$ {I}_{obrcbr}={R}_{C\left({I}_{obr}\right)}\left(C\left({I}_{obrd}\right)\right) $$
(4)

where C(X) denotes the complement of an image X. The result obtained in this step is shown in Fig. 2d.

For the last step, we choose a suitable threshold in order to obtain the desired hair part. Depending on the selected parameter, we can obtain the desired binary image as shown in Fig. 3a. We elevate the hair part on the depth map by inverting the colors in the regions that have zero pixel values in the resulting binary image as shown in Fig. 3b. Moreover, by applying this method to the detected eyes parts, we can decrease the intensity of the white color in the eyes and invert the eyebrows with a suitable threshold.

Fig. 3
figure 3

Extraction and depth adjustment of hair area. a Binary image. b Depth adjustment

Full size image

Through the method described in this section, we can reduce the noise in the input image and also group pixels into a large area. This way, we can obtain the desired area of hair. If we only use thresholding for the input grayscale image, we will have the problem of not being able to control noise or group pixels into a large area. For instance, in this case, acne or freckles on the face might appear if we threshold the original image. The result would be not being able to obtain a good hair part.

6 Facial feature enhancement

The rectangular region of each facial feature can be extracted using the facial detection technique described in the previous section. We define El as the box region for the left eye, Er for the right eye, N for the nose, and M for the mouth. We refine the region for finding the exact area of other smaller parts. In this section, we describe how to compute the exact area of all small parts on the face. The method is divided into two small steps: 1) computing a binary image from a gray image, called Binary Image from Gray image (BIG), and 2) using top-hat transformation and converting the result into a binary image, called Binary Image Top-Hat (BITH). Overall flows of the processing are shown in Fig. 4.

Fig. 4
figure 4

Overall flows of facial feature enhancement

Full size image

6.1 Binary image from gray image

We use a basic thresholding method, called Otsu’s method [8], to convert a grayscale image into a binary image. The average intensity is used to determine an initial value. Next, we partition each region into two groups, \( {R}_1 \) and \( {R}_2 \), using a threshold \( T \). Subsequently, we calculate the respective mean gray values \( {m}_1 \) and \( {m}_2 \) of the partitions \( {R}_1 \) and \( {R}_2 \). Then, we select the new threshold value \( {T}^{\prime }=\frac{\left({m}_1+{m}_2\right)}{2} \). By repeating the previous steps, we obtain the desired threshold value for generating BIG. At this time, the binary image might contain all the information of the dark face parts of the face photograph. Figure 5 shows the results of this method with different thresholds. As can be seen in the figures, the binary image that is automatically generated by Otsu’s method clearly expresses features and boundary of features in the original image.

Fig. 5
figure 5

Conversion from a grayscale image to binary images with different threshold values. a threshold: T = 0.3. b threshold: T = 0.5. c threshold: T = 0.7. d threshold based on Otsu’s method

Full size image

Otsu’s method is automatic and quite effective because we do not need to select the threshold. In addition, the obtained binary image contains most of the information about the dark parts of the gray image. By selecting the gray threshold manually, we have to dedicate a significant amount of time to obtain the dark parts. On the other hand, we can see such parts using Otsu’s method, and thus the processing time is shortened. The result contains all the information about dark parts on the face of the face photograph (black partition).

6.2 Binary image top-hat

Top hat transform is an operation where a small element can be extracted from an input image [4]. In the field of image processing, there are two types of top-hat transform methods: white top-hat transform, also called top-hat, and black top-hat transformation, also called bottom-hat. It is described as image opening by structuring elements such as a line or a disk. The result of the white top-hat transform provides an image that contains objects or elements, where the element is used to loop an image in order to obtain the area where all the pixels within the element are brighter than their surroundings. For instance, we can see the area between an eyebrow and an eye. The face skin within that area is brighter than the eyebrow and eye area.

By using the line-shaped structuring element, the loop is processed for all pixels of image. This can help find the line that is brighter than the surroundings. On the face, we use the line-shaped structuring element, and thus it has an angle and length. Then, we can obtain objects or elements that are brighter than the surroundings. See Figs. 6 and 7 for more details. Using a line for the eye helps obtain the skin area between the eyebrow and eye.

Fig. 6
figure 6

Line structuring element for eye areas

Full size image
Fig. 7
figure 7

Results of top-hat transform with different parameters

Full size image

We use the line-shaped structuring element. The results obtained when using a length of 10 and angle of 45 ° are better than when using the other lengths. When using an angle of 0°, we miss the eye area between the eyebrows and eye. When using an angle of 90°, we miss the nose area. However, with an angle of 45°, we obtain full information about the white area on the face.

After applying the top-hat algorithm with the line-shaped structuring element, we continue to use Otsu’s method to obtain a binary image. The results are shown in Fig. 7.

6.3 FPM-R computation

As indicated before, we have two binary images: BIG and BITH. As shown in Fig. 9 (1), we define the parameters El and Er for the eye region (left eye and right eye), N for the nose, M for the mouth and F for the face. Now we will label all the parts of BIG and BITH that have intersection with the region (El∪Er∪N ∪M).

The procedure for labeling BIG with each part (El,Er,N,M) is described as below and illustrated in Fig. 8.

Fig. 8
figure 8

Illustration of labelling method

Full size image
  1. 1.

    Find a starting point and set a number.

  2. 2.

    Find an adjacent pixel and set index.

  3. 3.

    Find a next starting point and repeat the process.

figure c

Above algorithm is proposed in order to solve this problem. In the algorithm, we define the input variables src (Binary image BIG), dst (output labeling image whose size is the same as src), El (Eye left box), Er (Eye right box), N (Nose box), and M (Mouth box). All of the box regions are obtained by using the facial detection technique that is described in section 4. The input is a BIG image and the output is dst which is the result of using our labeling method for the BIG image. As shown in Fig. 8 (1), we use the loop statement in each row of pixels in the binary image until we obtain the value 1. At that time, we push adjacent pixels into the LStack. (see Fig. 8 (2)). Similarly, we continue to set an index to other parts, such as 2, 3, and 4 (see Fig. 8 (3)). In addition, we do not need to label areas with a small size. In this case, we delete all those areas with a size smaller than 15 pixels. This means that we reduce noise in the image and obtain a new result where the noise-points are lost. All the areas of size less than 15 pixels are removed.

The input result from BIG and BITH are shown in Fig. 9 (2). Figure 9 (4) shows the result of labeling BITH. We combine Fig. 9 (2) and (4) into Fig. 9 (5). The algorithm for creating FPM-R is described in the following algorithm.

Fig. 9
figure 9

Face Part Map Region (FPM-R) (1) the result of using Facial Detection Technique, (2) the result of labeling BIG, (3) the result obtained using top-hat transformation, (4) result of labeling (3), and (5) FPM-R

Full size image
figure d

We can limit the size of the parts on the face. We calculate the length and width of each rectangle that represents each feature area. This way, we can determine the size of each part on the face. The result is shown in Fig. 9 (5).

6.4 Bas relief generation using FPM-R

We adjust the pixel values in the area of the extracted facial features so that the dark areas of the facial features can be elevated in the resulting relief surface. FPM-R formed using the method described in the previous section represents the exact area of each facial feature. A depth map that represents the bas-relief surface, \( {D}^{\prime}\left(x,y\right) \), can be formulated as follows:

$$ D\left(x,y\right)=I\left(x,y\right)+a\left(FPM-R\right),\ {D}^{\prime}\left(x,y\right)={S}_{\alpha}\left(D\left(x,y\right)\right) $$
(5)

where \( D\left(x,y\right) \) is an intermediate depth map, I is an original grayscale image, a is a function that adjusts pixel brightness on FPM-R (i.e. facial feature map), and \( {S}_{\alpha } \) is an image smoothing function with smoothness degree \( \upalpha \). In order to create the intermediate depth map \( D\left(x,y\right) \), the steps described in this section are implemented.

The red colored area of Fig. 10 (2) represents the eyebrows. Using the result of the input gray image, we invert the pixel intensities of the eyebrows area. We make slight adjustments to the gray image between eyebrow and eye areas. The yellow color area represents the dark areas near the iris, which is adjusted to appear lighter. On the other hand, the green color area indicates that the white iris is adjusted to appear darker. We have the location and size of the nose area, which is colored blue in Fig. 10 (2). We use our algorithm to adjust the nose brightness, as described in [14]. This means that we create an equation of lines and then make the area such that the brightness steadily decreases from the lightest points. The violet color area represents the philtrum area between nose and mouth. After capturing a picture, the light in the philtrum area is usually quite strong. Therefore, we can adjust the depth map for the philtrum similar to the depth for the face skin. In the mouth area, we also control the dark intensities. The result of the depth map is obtained and shown in Fig. 10 (3). Finally, an image smoothing and a basic Shape-From-Shading (SFS) method are applied to the depth map in order to generate a mesh of the bas-relief surface.

Fig. 10
figure 10

Intermediate and final results of bas-relief generation from Lena face photograph. a original input gray image, b Face Parts Map Region (FPM-R) computed using our method, c Depth map adjusted based on FPM-R and hair area extracted. Smoothing process is subsequently performed to reduce noise. d Bas-relief surface that is our final result

Full size image

7 Experimental results

We implemented and tested our algorithms, and performed an evaluation of the results. Our implementation is based on MATLAB for feature detection and hair area extraction, and C++ coding with QT library for the proposed algorithms. The code was tested on a PC equipped with a 2.80 GHz AMD Phenom (tm) II X6 CPU and 8GB main memory, and applied to two data sets. The information of the test data sets and their processing time are summarized in Table 1.

Table 1 Processing time for each step (feature extraction, hair extraction, and FPM-R generation) of bas-relief generation. (unit: second)
Full size table

Figure 10 shows the bas-relief surface generated for the Lena image. Figure 10a is a gray image converted from the original image. The gray image is used to make a depth map in the subsequent steps. Figure 10b is the FPM-R that was generated using our algorithm. FPM-R contains the information for the location and size of all parts on the face. For instance, based on FPM-R, we made a depth map as shown in Fig. 10c, where the extracted eyebrow areas are inverted with the white color area, the irises are adjusted with a suitable parameter \( \upbeta \), and the white iris is also limited with a parameter \( \upgamma =-15 \). We chose the \( \upgamma \) value to obtain the eye area in the bas-relief mesh for realism. The philtrum area is adjusted to equalize the average values of the face skin. Moreover, in Fig. 10c, we use \( \upalpha =1 \) for the hat and hair area on the left side (corresponding to the black color area on the left side of Fig. 10c). For depth map generation, we also used the hair extraction method [14]. After a brightness adjustment on all parts of the face, we have to use a method to smooth the depth map image in order to create a standard depth map. We can obtain various depth maps depending on the number of times we apply the smoothing method. In our experimentation, we applied the smoothing method four times, and then the depth map was obtained as shown in Fig. 10c. Figure 10d shows the bas-relief mesh obtained as the final result. The generated bas-relief surface has the correct front-to-back relationships on the face and smooth boundaries, which makes the bas relief look more natural and realistic.

Figure 11 illustrates the variations in our results that can be generated by modifying the parameters. α is used to adjust the number of times that the smoothing method is used. \( \upbeta \) and \( \upgamma \) are the parameters for adjusting the dark areas and white iris, respectively. Figure 11a illustrates the quality of the resulting bas-relief surface when \( \upalpha =1 \). We can see that the hair part looks noisy and unnatural. Moreover, if the \( \upgamma \) value is poorly chosen, the iris areas also look unnatural, too. Therefore, selecting appropriate level of smoothing is necessary for obtaining satisfactory results. In Fig. 11b, we analyze the philtrum area. In this result, the philtrum area is higher than the mouth area, which is not true in reality. Therefore, with the suitable values and parameters, the result appears more realistic, as shown in Fig. 11c.

Fig. 11
figure 11

Bas-reliefs generation by using different parameters α and γ. α represents the number of iteration for smoothing operation. γ represents the degree of adjusting pixel intensities in white iris area. a α = 1 and γ = − 20. b α = 2 and γ = − 15. c α = 4 and γ = − 15

Full size image

In the results, we can see front-to-back relationships more clearly. The eyebrows and nose areas are higher than the iris and white iris. In addition, the height of iris and white iris is equivalent. On the face skin area, for instance, the philtrum is lower than the nose and mouth. This is true in reality.

We compare our results with those of the previous height elevation method. The height elevation method considers pixel intensities of input image as height of bas-relief surface and triangulates the image accordingly to generate bas-relief surface. It is the same as the method introduced in [17] when we set the parameters \( \uplambda =0 \) and \( \upmu =0 \) in the equation \( R\left(x,y\right)=\delta I\left(x,y\right)+\lambda G\left(x,y\right)+\mu D\left(x,y\right) \) where \( R \) represents relief height, G represents gradient, and \( D \) represents depth map. Figures 12 and 13 show the test and comparison results. Both methods reflect pixel intensities for adjusting positions of surface elements, which results in realistic bas-relief surfaces. Some of the facial features in our resulting bas-relief surface appear clearer, especially in the hair, eyebrows and nose areas. We have suitable height relationships among the parts on the face. The reason for this is that we computed FPM-R, which can provide the location and size of all the parts, and the brightness of each part is adjusted according to the correct front-to-back relationships.

Fig. 12
figure 12

The results of bas-relief generation and comparison. a input photograph image of a movie star, b the result of the previous height elevation method, c FPM-R image (our intermediate result), and d the bas-relief result of our method

Full size image
Fig. 13
figure 13

The results of bas-relief generation and comparison. a input photograph image of a man, b the result of the previous height elevation method, and c the bas-relief result of our method

Full size image

The main problem of the previous height elevation method is inaccurate height estimation in the area of dark pixel intensities such as hair and eyebrow areas where the bas-relief surface is sunken even though it should be slightly protruded. The improvement of our method was obtained by avoiding and adjusting the inaccurate height estimation as shown in Figs. 12 and 13. In order to quantitatively measure the accuracy, we measured average height values of the bas-relief surface in the area of the facial features and skin areas as shown in Table 2. Relative height of the features to the skin indicate partial information on the accuracy. The relative height value should be positive in the areas of hairs, eyebrows, and nose as the surfaces on the areas should be slightly protruded. The result of our method generates positive values in the areas while previous method generates negative values in the areas of hairs and eyebrows making the surface in the areas look sunken. This indicates our method estimated the height of the feature areas more accurately. However, this measurement method has limitation of showing only partial information on accuracy (i.e. relative height of feature areas). Since the characteristics of bas-reliefs involve artistic components, the quantitative evaluation of performance in terms of accuracy is a challenging problem that is planned as our future work.

Table 2 Average height values of bas-relief surface in the areas of facial features relative to skin area
Full size table

8 Conclusion

In this paper, we proposed a new approach for creating a bas-relief surface from a face photograph used as the input based on FPM-R. From the face photograph, we build the feature map called FPM-R. In this regard, we can know the location and size of each feature area on the face. This provides us with information for adjusting the brightness values of each feature part. Our method can construct a bas-relief surface with suitable front-to-back depth relationships among the facial features based on the FPM-R. The experimental results showed that our method can produce a bas relief surface where the facial features are enhanced such that the surface appears more natural and realistic. Our method also has limitation that requires users to choose appropriate parameter values to generate satisfactory results. Automation of this process is considered as future work.