Keywords

1 Introduction

Real time captured picture recognition often encloses more information than required to communicate proposed information. So image abstraction can be used to reduce unnecessary information in an image and preserving relevant information for interpretation. More formally, image abstraction refers to the process of simplifying complex scène by removing irrelevant information that is not mandatory for particular event [1,2]. Image abstraction has been advanced technology under NPR and has been an effective visual tool for many applications. Image abstraction has talented to suggest the assured aspects and real time informative scene more effectively.

In this paper, an integrated filter based approach for image abstraction is proposed. We made an attempt to effectively integrate anisotropic filter, Difference of Gaussian filter and coherence shock filter followed by Mean Curvature Flow (MCF) and dithering to produce more effective abstracted image. We have exploited the features of these filters through integration for better image abstraction useful for many applications. The abstraction process is not confined to any shape or color. The proposed abstraction technique is capable to convey colors, regions, shapes as well as a particular event in an effective manner. Any image abstraction technique is essentially suppress the random and real time noise, preserving image structure, shape and handle poor lighting conditions in an image and hence it is a challenging task in the research area of image processing.

Image abstraction is most useful for numerous applications such as smoothing isophote curves, stipple drawing, mosaics engraving, simplifying visual cues, cubist rendering, cartoon rendering of 3D object, animated movies, noise suppression, pen and pencil sketch illustration to name a few. It is also useful to solve the optimization problem in engineering and scientific applications.

2 Related Works

Several researchers have made an attempt to propose effective techniques for image abstraction. Image segmentation, color quantization, or feature preserving smoothing techniques are used for image abstraction. Edge preserving filters [3] are also used to automatically create stylized abstractions from images or videos.

A familiar approach to create non photo realistic representation of an image is to transform an image into abstracted image using an interactive or automatic technique [4]. Another interactive tool approach, where the brush strokes are placed automatically was introduced by Hays and Essa [5] in the year 2004.

DeCarlo et al. [6] proposed stylization and abstraction of photographs based on mean shift color image segmentation. The technique transforms images into a line-drawing style using bold edges and large regions of constant color. But the method may not produce satisfactory results for images with complex structures. Comaniciu et al. [7] and Collomosse et al. [8] extended the mean shift segmentation to video to produce a temporally coherent cartoon like image sequence. Wang et al. [9] used anisotropic mean shift filter for handling elongated structures often found along temporal axis of a video. Wen et al. [10] also used mean shift segmentation algorithm for generating a colored sketch from a photograph. Lecot and Levy [11] developed a triangle based image segmentation algorithm for abstract and stylistic bitmap-to-vector image conversion. Though the image segmentation is a natural choice of tool for the task of image abstraction, a crude segmentation often results with incomplete abstraction, which further requires some post processing like curve fitting, editing, smoothing and stylizing.

Filter based image abstraction methods have also been proposed in the literature. Winnemoller et al. [12] showed that bilateral filter can be used to abstract color images as well as video. Orzan et al. [13] developed a multi-scale image abstraction system based on gradient reconstruction. Gooch et al. [14] proposed Artistic vision: Painterly Rendering Using Computer Vision Techniques based on Difference-of-Gaussians (DOG) filter. In their work, they have considered raster image as input and obtained a painting—like image composed of strokes rather than pixels. Kang and Lee [15] proposed shape simplifying image abstraction method for producing stylistic abstraction of a photograph. The method used mean curvature flow in conjunction with shock filter to simplify both shape and color simultaneously. But an obvious limitation of this method is that the curvature flow contracts small circular shapes very quickly. If the circular shape is of high importance in certain cases, it needs to be masked before running this algorithm. Kang et al. [16] proposed image and video abstraction using anisotropic Kuwahara filter. This filter effectively removes the details in high contrast edges by preserving shape boundaries in low contrast region. But the Kuwahara filter is unstable in the presence of high noise in the source image and suffers from block of artifacts. Kang et al. [17] proposed Flow-Based Image Abstraction technique based on line and region extraction filters guided by Edge Tangent Flow (ETF) that describes the flow of salient features in the image. The method may not perform satisfactorily when there are large number of irregularities and random noise in a poor intensity images.

From the literature survey, it is found that the methods proposed for image abstraction possess certain limitations in terms of quality output or in handling type of input image. Also the methods have been implemented using GPU devices and CUDA languages, which demands high computation and sophisticated environment for implementation. Since the implementation of the proposed work is carried out in MatLab environment using local library functions, it is simple and does not require any sophisticated high computing environment.

Rest of the paper is organized as follows: Sect. 3 describes the proposed methodology, Sect. 4 explains the experimental analysis and Sect. 5 presents the conclusion.

3 Proposed Method

The proposed method of image abstraction involves five major steps. First, anisotropic filter is applied to suppress the irregularities and to preserve the prominent edge boundaries in an image. However, some edge information may not be clear and need to be enhanced. In the second step, a Difference of Gaussian filter is applied to make the uncleared edges more clear. In the third step, we apply coherence shock filter iteratively to provide sharpening effect to the image. The image obtained at this stage may contain surface irregularities due to sharpening and need to be suppressed. Hence in the fourth step, we apply MCF to remove the irregularities in the image surface. Finally, we apply dithering to create the misapprehension or non realistic of color depth in an image with a limited color palette, which produces artistic effect without any upsetting effect to human eye. The following sections provide a brief description about the filters used in the proposed methodology.

3.1 2D Anisotropic Filter

Anisotropic filter is used to smooth and enhance edges and texture in an image. This filter effectively suppresses the unwanted noises from input images and produces best rendering and noise free images. Implementation of anisotropic filter involves various steps as follows:

The first step is to calculate the local gradient information of a color image I where I = (x, y) denotes pixel information.

$$ {\text{I}}\left( {\text{f}} \right):{\text{R}}^{ 2} \,{\text{grad}} \to {\text{R}}^{ 3} \,{\text{grad}} $$

Subsequently, the local adoptive smoothing based on the local gradient information of an image is performed. Further, the smoothing kernel is locally calculated choosing Gaussian forms given in [17–20]

$$ G(dir) = \exp \left\{ {\frac{1}{2}\left( {\frac{{dir.a^{ + } }}{{sigma_{1}^{2} }}} \right) + \left( {\frac{{dir.a^{ - } }}{{sigma_{2}^{2} }}} \right)} \right\} $$
(1)

Here, dir is the location vector, where a+ and a− are the vectors perpendicular to the local gradient. Values σ1 and σ2 are appropriately chosen. The Color image orientation and region direction derivatives information is calculated using the formula.

$$ \frac{\partial f}{\partial x} = \left( {\frac{\partial R}{\partial x}.\frac{\partial G}{\partial x}.\frac{\partial B}{\partial x}} \right)\quad \frac{\partial f}{\partial y} = \left( {\frac{\partial R}{\partial y}.\frac{\partial G}{\partial y}.\frac{\partial B}{\partial y}} \right) $$
(2)

The local gradient is calculated to estimate important portions in an image by means of Eigen vector and Eigen values. It can be mathematically represented as follows.

$$ (dI)^{2} = \left( {\frac{{dx}}{{dy}}} \right)^{T} = \left( \begin{aligned} & \left( {\frac{{\partial f}}{{\partial x}}} \right)^{2} \;\frac{{\partial f}}{{\partial x}}\frac{{\partial f}}{{\partial y}} \\ & \frac{{\partial f}}{{\partial x}}\frac{{\partial f}}{{\partial y}}.\left( {\frac{{\partial f}}{{\partial x}}} \right)^{2} \\ \end{aligned} \right) = \left( {\frac{{dx}}{{dy}}} \right) $$
(3)

With the help of sequential gradient magnitude, we calculate the largest Eigen value in a given matrix. This can be mathematically represented as

$$ \alpha = \left( {\frac{\lambda 1 - \lambda 2}{\lambda 1 + \lambda 2}} \right) $$
(4)

The gradient magnitude is computed using finite difference approximation for partial differential equation as follows.

$$ ||\nabla f|| = \left\{ {\left( {\frac{\partial f}{\partial x}} \right)^{2} + \left( {\frac{\partial f}{\partial y}} \right)^{2} } \right\}^{\frac{1}{2}} $$
(5)

Anisotropic filter not only smooth the image but it also gives more importance to preserve the edges gradient strength. In this work, we estimate the corner and edge strength for the purpose of preserving corners and also to smooth edges by using the equation,

$$ C = (1 - \alpha )||\nabla I||^{2} $$
(6)

During smoothing and local orientation protection process, corners should be preserved. A corner is identified as an isotropic with large local gradient strength. We can estimate the corner strength by:

$$ C = (1 - \alpha )||I^{2} || $$
(7)

How To preserve corners, we divide the standard deviations by another 1 + C. The final variances in (1) are then given by:

$$ \sigma_{1} = \frac{(1 - \alpha ).\text{a} }{(1 + c)}\quad \sigma_{2} = \frac{\text{a} }{(1 + c)} $$
(8)

Where a is an image noise factor. The variance of the image noise σ can be estimated globally by calculating the images local variances. Based on local orientation estimation, we can measure the noise factor. Anisotropic filter preserves large homogeneous and heterogeneous noise free regions.

3.2 2D Difference of Gaussian Filter

Difference of Gaussians filter [12] is used to smooth an image as well as useful for line extraction. The filter is also used to show significant high discontinuities. Two Gaussian filters with different blurring radius are formed and the respective images after Gaussian filter are obtained. The resulting images are subjected to subtraction operation to obtain the result. Image smoothing can be performed by convolution using appropriate spatial mask. Applying the spatial mask to an input image suppresses the high frequency spatial information. A general DoG is used to remove unwanted noise and for line extraction. The most important parameters for DoG filter are the smoothing radii (σ) for the two Gaussian blurs [21,22]. It is observed that a small increasing in the radius tends to give thicker appearing edges and a small decreasing tends to increase the threshold for recognizing something as an edge. In most cases, a best result is obtained when the value for (radius-2) is smaller than the (radius-1). DoG equation can be mathematically represented as follows.

$$ f\left( {I,\,\mu ,\,\sigma_{1} ,\,\sigma_{{_{2} }} } \right) = \frac{1}{{\sigma_{1,} \sqrt {2\prod } }}{ \exp }\left( { - \frac{{\left( {I - \mu } \right)^{2} }}{{2\sigma_{1}^{2} }}} \right) - \frac{1}{{\sigma_{2} \sqrt {2\prod } }}{ \exp }\left( { - \frac{{\left( {I - \mu } \right)^{2} }}{{2\sigma_{2}^{2} }}} \right) $$
(9)

3.3 Modified 2D Coherence Shock Filter

2D Coherence Shock Filter [23–25] is an edge preserving and smoothing filter, which gives more importance to direction of the edges in an image. It involves either a dilation or erosion process depending on whether the pixel is present in the maximum or minimum influence zone.

$$ It = - sign(\Updelta u)|\nabla u| $$
(10)

The filter creates shocks between maximum and minimum influence zone and it represents that the shock filter is within the range of original image. The modified version of the coherence shock filter makes the edges more sharp and helps more accurate segmentation of region of interest. A slight modification to coherence shock filter is accomplished as follows:

$$ I_{t} = - sign(\Updelta u^{*}I)|\nabla u|^{*}I_{\text{Smooth}} $$
(11)

Here, I is the gradient image and Ismooth is an anisotropic filtered image. The modified coherence shock filter combines the gradient image, anisotropic filtered image and −sign (∆u). The modified coherence shock filter gives a shining effect to the image by preserving hidden edges.

3.4 Mean Curvature flow

Mean Curvature flow (MCF) is used to remove the irregularities from complex background images. It effectively protects and conveys directional characteristics of shapes, features and textures. MCF is capable to identify luminance contour on the image for suppressing unwanted irregular curves, removing noise and irregular peaks. It suppresses and elaborates the irregular curves in an input image. MCF can be mathematically expressed as [15],

$$ \begin{gathered} I_{MCF} = \kappa |\nabla I| \hfill \\ \hfill \\ \end{gathered} $$
(12)

3.5 Dithering

It is a technique used to create the misapprehension/non-realistic of color depth in images with a limited color palette. The aim of dithering is to decrease the number of colors and to provide artistic effect to an image without any upsetting effect to human eye. It contributes more to convert original image into cartoon, half tone like image. The idea of dithering is mainly based on quantization and color approximation technique [26]. In this work, we used the dithering function supported by Mat Lab.

The proposed method of integrated filter-based approach to image abstraction can be algorithmically expressed as follows:

Algorithm:

An integrated filter based approach for image abstraction

  • Input: Raw image

  • Output: Abstracted image

  • Method:

    Step 1: :

    Apply anisotropic filter for preserving low contrast regions in an image.

    Step 2: :

    Apply Difference of Gaussian filter for extracting dominant edges.

    Step 3: :

    Apply shock filter for recovering hidden edges.

    Step 4: :

    Apply mean curvature flow to regularize the irregular isotope curves in an image.

    Step 5: :

    Apply dithering for reducing the color space.

  • Algorithm ends.

4 Experimentation

In order to study the efficacy of the proposed technique for image abstraction, we conducted experiments on various types of natural images. In this section, we are presenting few of the results obtained from the proposed method. In our experimentation, the kernel value σ for anisotropic filter is set to 3.0 and anisotropic filtering half-width to 5.0. In Gaussian filter, the σ1 and σ2 values are set to 3.0 to 2.9 and the deviation value\( \mu \) is set to 0.5. For shock filter, the deviation value is set to 3.02 and the number of iterations is set to 40. The MCF is designed using local isophote curvature and an 8 bit dithering function is applied to reduce the color space. Figure  1 shows the results of varies stages of the proposed method of image abstraction.

Fig. 1
figure 1

a Input image.b Anisotropic filter output.c Difference of Gaussian filter (DoG).d Shock filter (90 iterations).e Mean curvature flow (MCF).f 8 bit dithering

Full size image

We have also conducted an experiment on the image used in [15]. Figure  2a is the input image and Fig.  2b, c shows the result of the method in [15] and the proposed method respectively. It can be observed that the proposed method produced the better output when compared to the method used in [15].

Fig. 2
figure 2

a Input image.b Shape simplifying image abstraction output.c Our method

Full size image

We have also conducted an experiment on natural image and the proposed method has produced the encouraging result. Figures  3a and 4a is an input image and Figs.  3b– f and 4b– h show the result of various stages of the proposed method of image abstraction and stylization.

Fig. 3
figure 3

a Natural image.b Anisotropic filter output.c DoG filter.d Shock filter (30 iterations).e Mean curvature flow (MCF).f 16 bit Dithering output

Full size image
Fig. 4
figure 4

a Church image.b Anisotropic filter output.c DoG filter output.d Shock filter (5 iterations) output.e Shock filter (10 iterations) output.f. Mean curvature flow (MCF) output.g 24 bit Dithering output.h 8 bit Dithering output

Full size image

5 Conclusion

this paper, we have presented an integrated filter-based approach to produce abstracted and stylized images. The proposed method incorporates the features of various filters to produce the better result. The method does not require any individual brush strokes to produce abstraction and stylized effect. The proposed method is found to be effective for all types of blurred images, high contrast images, and complex background images. The experimental results obtained for various types of images are highly encouraging and are comparable with the other abstraction techniques. The approach is found to be computationally efficient in producing effective cartoon like images being simple in terms of its implementation.