1 Introduction

Red-eye effect is a popular problem in consumer photography which is usually appeared in photographs taken with a built-in camera flash. The main reason for this problem is derived from the reflection of the blood vessels in the retina which happens when the eye receives an abrupt and intense light. Recently, compact cameras and camera phones exacerbate this problem due to the vicinity of the flash device and the lens [1].

Red-eye depends on at least three cases [2]:

  1. 1.

    The value of the enclosed light: low value makes the person’s pupil open wider to allow the entrance of more light, exposing the retina.

  2. 2.

    Age of the person: the younger the person, the broader the retina, since the greater the red-eye effect.

  3. 3.

    Flash beam angle: the light is rebound off the subject (person) back to the camera. More adjacent outgoing beam to the reflected beam which makes the greater red-eye effect.

Red-eye is omnipresent in photography by flash. Photographers usually eliminate some forms of red-eye which is visually displeasing, since the ability to discard this undesirable effect is a worthwhile feature in cameras. Unwanted red-eye effect can be corrected by using image processing and machine vision. This kind of system can be driven in camera-enabled cell phones to allow the red-eye avoidance features to be generated clearly into cameras [3]. By using image processing, we can make a red-eye correction system which can be completely automatic, modifying pictures as they are taken. Sometimes red-eye correction techniques fail to detect the red-eye effects which make a visually poor correction, or alter a region of the photograph that is not an eye, such as circles or dots on clothing; hence, proposing a proper algorithm which decreases such fails can be helpful to reach a good system. Figure 1 shows some of these fails.

Fig. 1
figure 1

Some examples of images which failed in red-eye detection and correction: a A region of the photograph detected as eye which is not, b A region of the photograph detected as eye which is closing, and c A region of eye which is not detected as eye

Full size image

Different methods have been introduced for red-eye detection and its correction; mostly used algorithms are those utilized in semi-automatic or manual approaches; this kind of algorithms require the user to define the red-eye pixels or draw a box containing the red-eye region before applying the red-eye removal algorithm. Some other procedures contain automatic approaches which can correct the red-eye with no need for region selection [4, 5]. Several methodologies are introduced for semi-automatic and automatic red-eye methodologies. All existing procedures include at least two parts: red-eye detection and red-eye correction. Detection part contains different executes like human face or/and eye detection, segmentation and selecting the red-eye regions, feature extractions and classification of regions by them, glint detection and pair verification [6]. Ulichney improved one-click system called RedBot which is ready for use free online [7]. Substantial work in red-eye detection and red-eye correction was performed by Hardeberg et al. [8]. At the beginning, Hardeberg proposed a semi-automatic method where the eye region had to be selected manually [9]. After that, Hardeberg [10] developed this semiautomatic approach and improved a fully automatic method which was used as a preliminary color segmentation based on crossing the threshold in different color models to identify the skin regions [11]. Matthew Gaubatz and Robert Ulichney presented an automatic system to detect and correct the red-eye which is divided into two stages: face detection by using a multi-scale classifier and then searching for the eyes in the candidate face regions by using constraints based on colors, red variance and glint. Schildkraut and Gray [12] used an automatic algorithm to detect pairs of eye regions, which is regulated to near-frontal face images. The pair verification procedure was used to decrease false positive. However, photographs which have single red-eye cannot be corrected with this approach. Detected red-eyes are then removed to preserve the natural appearance of the eyes. Luo et al. [13] also used a two-stage system for the red-eye system; they modeled a feature-based object detection which was trained by an AdaBoost classifier. Some new features were also designed for indicating the orientation-dependent problem with directing to the digital image features traditionally utilized for object recognition purposes; detected red-eye regions then de-saturated and modified by darkening over the red-eye region. Ioffe proposed an automatic red-eye detection system including a red-eye detector which finds red-eye candidates; a novel face detector used to detect red-eye outline and discard most image regions [14]. Marchesotti et al. have closely considered the problem of red-eye correction, mostly less analyzed than red-eye detection [15]. They introduced three correction approaches for red-eye correcting; after that, a comparison is applied among these approaches due to their image demotion risk and their expected perceptual quality improvement. The final result was achieved by selecting the correction method belonging to those measures and the detection confidence. Battiato et al. [16] proposed a feature-based approach to analysis and classification considering the bag-of-key points paradigm; their procedure includes extraction of local image characteristics, quantization of the feature space into a codebook through clustering and extraction of code word distribution histograms.

All of the mentioned methods have some weaknesses which make them to get some errors and decrease their accuracy; this study presents a new evolutionary-based classifier based on a new introduced imperialist competitive algorithm (ICA) to enhance the system accuracy to reach a high-performance system.

Imperialist competitive algorithm was firstly introduced in 2007 to deal with continuous optimization problems by Esmaeil and Lucas [17]. ICA is a new meta-heuristic algorithm which is introduced to solve the optimization problems. Despite algorithms like PSO, ACO and other swarm intelligence which are designed by imitating the natural behaviors, ICA was inspired by the socio-political behaviors. The main advantage of ICA is the weaker empire which is collapsed in the imperialistic competition process. Since, the probability of achieving the optimal parameters is considerably increased to avoid for trapping in the local optimum.

ICA has been also successfully employed in image processing fields like segmentation [18], feature extraction [19], edge detection [20] and neural networks optimization [21, 22].

In this paper, we mainly focused on a new type of neuro-fuzzy trained by ICA and utilize this hybrid method to employ in the red-eye removal purposes.

2 Adaptive Neuro-Fuzzy Inference System (ANFIS)

A neuro-fuzzy (ANFIS) system is a join of fuzzy systems and neural networks in such a way that neural network is used to describe the parameters of fuzzy system [23]. ANFIS generously removes the requirement for manual optimization of the fuzzy system parameters. A neural network is used to adjust the system parameters automatically, for example, the membership functions bounds, leading to developed performance with no expert [24]. The neuro-fuzzy systems with the learning ability of neural networks and with the advantages of the rule-based fuzzy systems can develop the performance in an important manner and also can make a mechanism to feedback past observations into the classification process. In neural networks, the training basically builds the system. However, using a neuro-fuzzy system, the scheme is built by fuzzy rule-based knowledge definitions and is then refined using neural networks training algorithms. Some advantages of the ANFIS are given as follows:

  1. 1.

    Purify fuzzy if–then rules to define the behavior of a complicated system.

  2. 2.

    Requires no prior human expertise, in which uses membership functions and desired dataset to approximate.

  3. 3.

    Larger choice of membership functions to use.

  4. 4.

    High convergence and elapsed time.

2.1 ANFIS Architecture

ANFIS is a Takagi–Sugeno fuzzy interface system put in the framework of adaptive systems to make learning and adaptation easy [25]. This sort of framework makes the ANFIS modeling to be organized by less dependent on expert knowledge.

To demonstrate the ANFIS architecture, three fuzzy if–then rules based on a first-order Sugeno model are considered:

$$\begin{array}{l} {{\text{Rule }}1:{\text{If }}\left( {x{\text{ is }}A_{1} } \right){\text{ and }}\left( {y{\text{ is }}B_{1} } \right){\text{ and }}\left( {z{\text{ is }}C_{1} } \right){\text{ then }}\left( {f_{1} = \, p_{1} x \, + \, q_{1} y \, + \, r_{1} } \right)} \hfill \\ {{\text{Rule }}2:{\text{If }}\left( {x{\text{ is }}A_{1} } \right){\text{ and }}\left( {y{\text{ is }}B_{1} } \right){\text{ and }}\left( {z{\text{ is }}C_{2} } \right){\text{ then }}\left( {f_{1} = \, p_{2} x \, + \, q_{2} y \, + \, r_{2} } \right)} \hfill \\ {{\text{Rule }}3:{\text{If }}\left( {x{\text{ is }}A_{1} } \right){\text{ and }}\left( {y{\text{ is }}B_{2} } \right){\text{ and }}\left( {z{\text{ is }}C_{1} } \right){\text{ then }}\left( {f_{1} = \, p_{3} x \, + \, q_{3} y \, + \, r_{3} } \right)} \hfill \\ {{\text{Rule }}4:{\text{If }}\left( {x{\text{ is }}A_{1} } \right){\text{ and }}\left( {y{\text{ is }}B_{2} } \right){\text{ and }}\left( {z{\text{ is }}C_{2} } \right){\text{ then }}\left( {f_{1} = \, p_{4} x \, + \, q_{4} y \, + \, r_{4} } \right)} \hfill \\ {{\text{Rule }}5:{\text{If }}\left( {x{\text{ is }}A_{2} } \right){\text{ and }}\left( {y{\text{ is }}B_{1} } \right){\text{ and }}\left( {z{\text{ is }}C_{1} } \right){\text{ then }}\left( {f_{1} = \, p_{5} x \, + \, q_{5} y \, + \, r_{5} } \right)} \hfill \\ {{\text{Rule }}6:{\text{If }}\left( {x{\text{ is }}A_{2} } \right){\text{ and }}\left( {y{\text{ is }}B_{1} } \right){\text{ and }}\left( {z{\text{ is }}C_{2} } \right){\text{ then }}\left( {f_{1} = \, p_{6} x \, + \, q_{6} y \, + \, r_{6} } \right)} \hfill \\ {{\text{Rule }}7:{\text{If }}\left( {x{\text{ is }}A_{2} } \right){\text{ and }}\left( {y{\text{ is }}B_{2} } \right){\text{ and }}\left( {z{\text{ is }}C_{1} } \right){\text{ then }}\left( {f_{1} = \, p_{7} x \, + \, q_{7} y \, + \, r_{7} } \right)} \hfill \\ {{\text{Rule }}8:{\text{If }}\left( {x{\text{ is }}A_{2} } \right){\text{ and }}\left( {y{\text{ is }}B_{2} } \right){\text{ and }}\left( {z{\text{ is }}C_{2} } \right){\text{ then }}\left( {f_{1} = \, p_{8} x \, + \, q_{8} y \, + \, r_{8} } \right)} \hfill \\ \end{array}$$

where x and y are the inputs, Ai and Bi are the fuzzy sets, pi are the outputs within the fuzzy region specified by the fuzzy rule, pi, qi and ri are the design parameters that are determined during the training process [26]. In the ANFIS architecture, a circle shows a fixed node, where a rectangular shows an adaptive node.

Here, the Takagi–Sugeno fuzzy model-based ANFIS architecture has three inputs and one output which will be adjusted online using the error backpropagation method. Inputs are the introduced color space for the image, and the output illustrates the considered objective.

Membership function for all inputs and the hybrid learning is Gaussian function. This selection is to increase the identification speed of the parameters.

From the above, each node in the layer 1 is adaptive. The outputs of layer 1 are the fuzzy membership explanation of the inputs, which can be represented as,

$$O_{i}^{1} = \mu_{{A_{i} }} (x),i = 1,2$$
(1)
$$O_{i}^{1} = \mu_{{B_{ - 2i} }} (y),i = 3,4$$
(2)

where x, y and z are the inputs to node i, A is a linguistic label (small, large) and \(\mu_{{A_{i} }} (x)\), \(\mu_{{B_{ - 2i} }} (y)\) and \(\mu_{{C_{ - 4i} }} (y)\) can entertain any fuzzy membership function. Usually we choose \(\mu_{{A_{i} }} (x)\) to be bell-shaped with a maximum equal to 1 and minimum equal to 0, such as:

$$\mu_{{A_{i} }} (x) = \frac{1}{{1 + \left| {\frac{{x - c_{i} }}{{a_{i} }}} \right|^{{2b_{i} }} }}$$
(3)

where a i , b i and c i are the parameters of the membership function. The parameter c i locates the center of the curve, and b i is usually positive.

The second layer contains the fixed nodes (which are the so-called firing strengths of the rules) and the product obtained as output from each node of this layer can be represented as follows:

$$O_{i}^{2} = \omega_{i} = \mu_{{A_{i} }} (x)\mu_{{B_{i} }} (y),i = 1,2$$
(4)

Layer 3 contains fixed nodes which calculate the ratio of the firing strengths of the rules and are the so-called normalized firing strengths. It can be represented as below:

$$O_{i}^{3} = \overline{{\omega_{i} }} = \frac{{\omega_{i} }}{{\omega_{1} + \omega_{2} }},i = 1,2$$
(5)

All nodes in fourth layer are adaptive nodes with a node function:

$$O_{i}^{4} = \overline{{\omega_{i} }} O_{i} = \overline{{\omega_{i} }} \left( {p_{i} (x) + q_{i} (y) + r_{i} } \right),i = 1,2$$
(6)

where ω i is the normalized firing strength from layer 3, {p i , q i , r i } are the parameters set of this node. These are referred to consequent parameters.

In the fifth layer, the overall output is calculated as the summation of all incoming input signals. Hence, the overall output of the model can be obtained as follows:

$$O_{i}^{5} = {\text{overall}}\_{\text{output}} = \sum\limits_{i} {\bar{\omega }_{i} O_{i} = \frac{{\sum\nolimits_{i} {\omega_{i} O_{i} } }}{{\sum\nolimits_{i} {\omega_{i} } }}}$$
(7)

In the equation above, \(O_{i}^{5}\) presents the ANFIS system output for the given input x, y and z.

In the proposed method, ANFIS is employed for both two phases of the system: skin detection and the red-eye detection. The input image is derived as a skin (red-eye) region if the output value is less than 1; otherwise, it is inferred as a background. In this way, ANFIS is trained to classify different types of images from various datasets. In testing process, the same approach is followed for all images and the ANFIS classifies the images performance.

2.2 Learning Algorithm of ANFIS

Learning algorithm has been used to adjust all changeable parameters, namely {a i , b i , c i } and {p i , q i , r i }, to match the output result by training data [25]. Once the assume parameters a i , b i and c i of the membership function are arranged, the output of the ANFIS model can be presented as follows:

$$O = \frac{{\omega_{1} }}{{\omega_{1} + \omega_{2} }}O_{1} + \frac{{\omega_{2} }}{{\omega_{1} + \omega_{2} }}O_{2}$$
(8)

By substituting Eq. (5) into Eq. (8), we have:

$$O = \overline{{\omega_{1} }} O_{1} + \overline{{\omega_{2} }} O_{2}$$
(9)

And substituting the fuzzy if–then rules into Eq. (9) gives:

$$O = \overline{{\omega_{1} }} \left( {p_{1} (x) + q_{1} (y) + r_{1} } \right) + \overline{{\omega_{2} }} \left( {p_{2} (x) + q_{2} (y) + r_{2} } \right)$$
(10)

To recognize the optimal values of these parameters, the least squares method can be used easily. Until the considered parameters are not determined, the search space becomes larger and the convergence of the training becomes slower.

A hybrid procedure which includes both the least squares method and the gradient descent method together is adopted to solve the problem. The hybrid procedure includes a forward and a backward pass. The least squares method (forward pass) is used to optimize the consequent parameters by the defined considered parameters. At the moment that the optimal consequent parameters are found, the backward pass commences immediately.

The gradient descent method (backward pass) is employed to modify the assumed parameters optimally corresponding to the fuzzy sets in the input domain. The output of ANFIS classifier is estimated by using the consequent parameters found in the forward pass [26].

In the following, the paper is organized as follows: Sect. 3.1 describes detection of the skin-like areas in the color image with fast segmentation algorithms based on adaptive neuro-fuzzy (ANFIS) in HIS’ color space. Morphological operations are then applied to eliminate the extra areas not including the skin. Section 3.2 describes all the works on red-eye detection which includes: some new features to find a proper area which can reduce the image domain to be searched to find red-eye artifacts; morphological operations are then applied again to eliminate the extra areas which do not include the red-eye.

3 Imperialist Competitive Algorithm

Imperialist competitive algorithm (ICA) is a new computational and heuristic algorithm to solve the optimization problems. Like other optimization algorithms, ICA starts with an initial population as the country; country comprises two parts: colonies and imperialists where imperialist countries try to overcome the other countries and turning them to their colonies [17]. Imperialist countries are also competing strongly with together for getting possessions of other countries. During the competition, the weakest empire collapses and the stronger empire becomes stronger [27]. The pseudo-code of imperialist competitive algorithm is illustrated as follows [28]:

1. Select some random points on the function and initialize the empires

2. Move the colonies toward their relevant imperialist (Assimilation)

3. Randomly change the position of some colonies (Revolution)

4. If there is a colony in an empire which has lower cost than the imperialist, exchange the positions of that colony and the imperialist

5. Unite the similar empires

6. Compute the total cost of all empires

7. Pick the weakest colony (colonies) from the weakest empires and give it (them) to one of the empires (Imperialistic competition)

8. Eliminate the powerless empires

9. If stop conditions satisfied, stop, if not go to 2

In this study, ICA is applied to optimize the parameters of the ANFIS. The countries of the ICA contain the range of the membership functions in both skin detection and red-eye detection.

3.1 Skin Color Detection

Color is one of the key features in more face detection algorithms. When color is employed as a feature in face detection, a good selection of color space and modeling of the skin color distribution has become two main stages. HSI (Hue–Saturation–Intensity) color model as perceptual features of color is a good choice for our purpose; since the transformation of RGB to HSI is highly invariant to high intensity at white lights, ambient light and surface orientations depend on the light source [29, 30].

By using the following nonlinear equations, RGB color model transforms to HSI color space, where the intensity component is detached from the chrominance components:

$$H^{\prime} = \left\{ {\begin{array}{ll} {{\text{undefined}}},& \quad {\text{if}}\,C = 0 \\ {\frac{G - B}{C}\bmod 6}, & \quad {\text{if}}\,M = R \\ {\frac{B - R}{C} + 2}, & \quad {\text{if}} \, M = G \\ {\frac{R - G}{C} + 4}, &\quad {\text{if}}\,M = B \end{array} } \right.$$
(11)
$$H = 60^{ \circ } \times H^{\prime}$$
(12)
$$S = \left\{ {\begin{array}{ll} 0 & {{\text{if}}\,\,C = 0} \\ {\frac{C}{V},} & {\text{otherwise}} \\ \end{array} } \right.$$
(13)
$$I = \frac{1}{3}(R + G + B)$$
(14)

To obtain an appropriate color model, a manual adjustment is applied on the HIS model as follows [31]:

$$H_{\text{new}} = {1 \mathord{\left/ {\vphantom {1 2}} \right. \kern-0pt} 2} \times H$$
(15)
$$S_{\text{new}} = S$$
(16)
$$I_{\text{new}} = 3 \times I$$
(17)

The classified results of the skin images are used for training phase. From the skin image, the skin regions are manually extracted. Then by using MATLAB function, the skin parameters are described. Here, region parameters include: area, convex area and centroid (Fig. 2). The achieved region parameter values are given to the ANFIS for training process, and the result of ANFIS is set as greater than 1. To decide on the number of membership functions (MFs) and rules, subtractive clustering is applied on the input data (containing 10,000 skin and non-skin pixels). Subtractive clustering is a fast, one-pass algorithm which is used for estimating the number of clusters and the cluster centers in a set of data [32]. Figure 3 shows the membership function of inputs.

Fig. 2
figure 2

ANFIS architecture which is tantamount to Sugeno fuzzy model and its input space. ANFIS architecture for a two-input Sugeno fuzzy model with two rules. X, y and Z are the inputs, A i , B i and C i are linguistic labels (small, medium, large), M and N are the hidden layer neurons, o i is the firing strength of a fuzzy rule and w is the weighted given to each input

Full size image
Fig. 3
figure 3

a–c Final generalized bell-shaped membership function of three inputs for skin detection

Full size image

For transforming the gravy output of the ANFIS into a binary mask (which represents only the skin and non-skin regions), Kapur’s method based on the entropy theory is employed [33].

It includes the maximization of the class entropies which is clarified as a measure of class compactness and accordingly, of class separability. The probability distribution of gray levels over the black part can be presented as follows:

$$\frac{{p_{0} }}{{P_{B} }},\frac{{p_{1} }}{{P_{B} }} \ldots \frac{{p_{s} }}{{P_{B} }},$$
(18)

and the white part as follows:

$$\frac{{p_{s + 1} }}{{1 - P_{B} }},\frac{{p_{s + 2} }}{{1 - P_{B} }} \ldots \frac{{p_{n + 1} }}{{1 - P_{B} }},$$
(19)

in which s is the threshold; p i (i = 0, 1…, n − 1) is the statistical probability of pixels with gray level i in the perfect image; P B is the probability of pixels with gray level less than or equal to the threshold value s.

$$P_{B} = \sum\limits_{i = 0}^{s} {p_{i} } ,$$
(20)

The entropy of the black part (object) of the image is:

$$H_{B}^{(S)} = - \sum\limits_{i = 0}^{s} {\frac{{p_{i} }}{{P_{B} }}\log_{2} (\frac{{p_{i} }}{{P_{B} }})}$$
(21)

and for the white part is:

$$H_{W}^{(S)} = - \sum\limits_{i = s + 1}^{n - 1} {\frac{{p_{i} }}{{1 - P_{B} }}\log_{2} \left(\frac{{p_{i} }}{{1 - P_{B} }}\right)} ,$$
(22)

The entire entropy of the image is then described as follows:

$$H_{T}^{(S)} = H_{B}^{(S)} + H_{W}^{(S)} ,$$
(23)

The threshold s must be chosen such that \(H_{T}^{(S)}\) becomes maximized (Fig. 4). After thresholding, morphological operation (filling holes and opening followed by closing) is applied on the recognized skin pixel regions [34]. Region filling is a complicated algorithm including set dilation, complementation and intersections:

$$X_{k} = (X_{k - 1} \oplus B) \cap A^{c} , \, k = 1,2,3 \ldots$$
(24)

where A is a set of boundary and B is structuring element. The algorithms terminate at iteration step k if \(X_{k} = X_{k - 1}\). The opening of A by B is obtained by the erosion of A by B, followed by dilation of the resulting image by B:

Fig. 4
figure 4

Skin detection: a input image and b output after ANFIS classification and mathematical morphology

Full size image
$$A \circ B = (A\varTheta B) \oplus B$$
(25)

Opening is used to remove small area noises which cannot become a part of skin. Closing typically fuses narrow breaks, makes counters smooth and long thin gulfs, eliminates small holes, and fills gaps in the contour. The closing of set A by structuring element B is denoted by A⋅B. Closing mixes thin distances as below:

$$A \cdot B = (A \oplus B)\varTheta B$$
(26)

A flat, diamond-shaped by radius equal to 4 (this shape is good for all size of images) is selected as the structuring element.

3.2 Red-eye Detection

In this study, the main procedure of red-eye detection is like to the skin procedure. Some new features are selected to red-eye color detection:

$$F1 = R - 0.5G - 0.5B - 0.7\hbox{min} \left\{ {G,B} \right\}$$
(27)
$$F2 = Cr - 1.1 \times Cb$$
(28)
$$F3 = Cr$$
(29)

where

$$C_{b} = 128 + \frac{{ - 37.945 \times R^{\prime}_{D} }}{256} + \frac{{74.494 \times G^{\prime}_{D} }}{256} + \frac{{112 \times B^{\prime}_{D} }}{256}$$
(30)
$$Cr = 128 + \frac{{112.439 \times R^{\prime}_{D} }}{256} + \frac{{94.154 \times G^{\prime}_{D} }}{256} + \frac{{18.258 \times B^{\prime}_{D} }}{256}$$
(31)

The prime (′) symbol presents that the Gamma Correction utilization; thus, R′, G′ and B′ nominally range from 0 to 1 such that 0 and 1 represent the minimum and maximum intensity, respectively.

Red-eye information more depends on the image colors. Since, one of the proper color models for describing the red-eye information is the RGB. Because the main color for detecting the red-eye is R (red), the models G and B are decreased by a one a half coefficient. The reason for using the\(0.7\hbox{min} \left\{ {G,B} \right\}\)is to decrease the unrelated color information as far as possible to ease the red-eye detection step.

C B and C R are practical approximation to color processing and perceptual uniformity, to process the primary colors corresponding roughly to red, green and blue into the perceptually meaningful information. Actually, C B and C R are the blue-difference and red-difference chroma components.

So, in order to decrease the importance of G as the main factor of disturbance and B after that, the F2 is proposed as the selected color space by trials and errors. C R is also used as a separate feature because of characterizing the red-difference chroma.

Therewith, to limit the number of hits, the algorithm uses the following geometric constraints:

$${\text{F4}} = \left\{ {\begin{array}{ll} {\text{True}} & {\frac{{{\text{length}}\;{\text{of}}\;{\text{the}}\;{\text{label}}\;{\text{image}}}}{{{\text{width}}\;{\text{of}}\;{\text{the}}\;{\text{label}}\;{\text{image}}}} \ge 1.5} \\ {\text{False}} & {{\text{O}} . {\text{W}} .} \\ \end{array} } \right.$$
(32)
$${\text{F5}} = \frac{\text{proportion of the labeled region }}{\text{proportion of the pixels in the whole image}} \le 0.25$$
(33)

F4 is finally used to remove those with a somewhat more ratio of length to width and were not removed in the classification stage. F5 is another feature for eliminating the big objects which have about 25% of the whole image and were not removed by skin detection and also red-eye color detection. The value of 25% is attained after some trial and errors in different images. Afterward, five presented features are trained to an ANFIS. Figure 5 shows the membership function of inputs.

Fig. 5
figure 5

a, b Final generalized bell-shaped membership function of three inputs for red-eye detection

Full size image

Similar to the skin detection system, the result of this stage is a gray red-eye-likelihood image. Therefore, threshold value is achieved by Kapur’s again. The thresholding algorithm on the red-eye-likelihood image makes binary image of the red-eye regions. The output has two values again: 1 for red-eye regions and 0 for irrelevant objects. Morphological processing includes: filling holes, and opening followed by closing is applied again to a better performance. A graphical abstract for the proposed method is shown in Fig. 6.

Fig. 6
figure 6

Graphical abstract for the proposed method

Full size image

3.3 Red-Eye Correction

Once a candidate red-eye region is detected, the artifact can be corrected by replacing the color of red pixels within an eye-sized neighborhood.

Red-eye correction step is to absorb the light of this region and make it dark. A linear transform is employed for this purpose. This calculation can be represented by Eq. (34).

$${\text{RGB}}_{\text{new}} = {\text{RGB}}_{\text{old}} - {\text{mean}}\;2\left\{ {{\text{RGB}}_{\text{skin}} } \right\} \times {\text{Mask}}_{\text{final}}$$
(34)

where \({\text{RGB}}_{\text{old}}\) is the original input image, \({\text{mean}}\,2\) is two-dimensional mean, \({\text{RGB}}_{\text{skin}}\) comprises skin regions of the original image and finally \({\text{Mask}}_{\text{final}}\) is the mask of red-eye after morphological operations. After red-eye detection, Eq. (34) is implemented to correct red-eye pixels. Figure 7 shows some results of red-eye correction.

Fig. 7
figure 7

Skin correction: a input image and b corrected image

Full size image

4 Experimental Results

In preparation for testing of red-eye diagnosis system, a synthetic database was collected. The database includes 100 subjects, where 70 subjects are used to train. Our approach can be divided into two phases; in the first phase, candidate skin regions are identified using fuzzy rule-based classifier automatically. In the second phase, red-eye zones are directly detected by scanning totally image with no previous reduction of the search space. In this case, the initial candidate red-eye regions are yielded to further verification steps to obtain a final confirmation based on the proposed low-level features and on fuzzy classifier. Both of these zones are justified with considering the great color variability of the red-eye artifacts, where red-eyes of different subjects obtained from different lighting conditions and with different digital cameras are depicted.

For estimation of system quality, three quantitative quality criterions are proposed and compared with existing automatic solutions. The first quality criterion is the correct detection rate (CDR) and is given in Eq. (35). The false acceptance rate (FAR) is the percentage of recognition moments in which false acceptance occurs. The false rejection rate (FRR) is the percentage of recognition moments in which false rejection occurs. The FAR and FRR are expressed in Eqs. (36) and (37), respectively; these methods are applied to a custom database which comprises different databases by 60 images.

$${\text{CDR}} = \frac{\text{Number of pixels correctly classified }}{\text{Total pixels in the test dataset}}$$
(35)
$${\text{FAR}} = \frac{\text{ Number of non red-eye pixels classified as red-eye pixels classified}}{\text{Total pixels in the test dataset}}$$
(36)
$${\text{FRR}} = \frac{\text{Number of red-eye pixels classified as non red-eye pixels classified}}{\text{Total pixels in the test dataset}}$$
(37)

The proposed red-eye correction has been compared with the following algorithms:

Red-eye gone [35], pho.to [36], stop red-eye [37], HP RedBot [7], Microsoft Photo Editor (it exists on the Windows OS in default), CyberLink [38], Arc Soft [39]. These algorithms have been applied on 100 images, and the results are presented in Table 1.

Table 1 Classification comparison of performance in the presented procedures
Full size table

We can derive from Table 1 that the proposed algorithm has achieved the best performances in terms of correct detection, the false acceptance ratio and the false rejection ratio. Notwithstanding the images collection employed in the experiments is not publicly available. Therefore, most of the photographs are taken from private collections. Different topologies are applied to the ANN, and the topology with the best results is selected. The utilized topologies are given in Table 2. The parameters of ANFIS for skin and red-eye detection are presented as follows:

Table 2 ANFIS parameters of the skin and red-eye detection
Full size table

Afterward, to evaluate the proposed ANFIS system, a comparative analysis is performed between the optimized ANFIS and ordinary ANFIS based on classification accuracy and error rate

Experimental results showed that the classification accuracy for the ordinary backpropagation type is comparatively less than the classification accuracy of the proposed ANFIS-based method. Figure 7 shows that the mean square error of BP is considerably high. It also does not guarantee high classification accuracy because it requires large training dataset which is not practically feasible. Since the ANFIS classifier combines the merits of both the neural classifier and the fuzzy classifier, it yields superior results than the other classifiers [35].

5 Comparative Discussion

The main superiority of the proposed approach is divided into two phases: the first phase includes five new features which help to enhance the accuracy percent. The next phase consists of the application of the imperialist competitive algorithm to optimize the neural network weights for red-eye correction application. We compared our algorithm with some online and available methods like: red-eye gone, phixr, stop red-eye, HP RedBot, Microsoft Photo Editor, CyberLink and Arc Soft. The test was executed on 100 images:

Table 1 shows that the proposed approach performs better than the other analyzed algorithms with regard to the correct detection, the false acceptance and the false rejection ratio.

The false acceptance ratio of our algorithm has better performance rather than the other algorithms. This metric shows that the experiments we applied to certify the region are really an eye and do not affect the results; the main reason for this issue is because of employing numerous images for training phase. The false positive rate is also very low, since a region has to pass a great number of tests to be observed as an eye. If the skin regions cannot be detected, our algorithm faces more problems in red-eye correction.

We can also conclude that the proposed approach has a prominent superiority than the ordinary neuro-fuzzy classifier in all metrics of classification accuracy (see Table 3).

Table 3 Classification comparison of performance in two presented classifiers
Full size table

6 Conclusion

In this paper, the application of imperialist competitive algorithm is proposed for optimizing the neuro-fuzzy system in red-eye correction applications. Experimental results yield promising results for optimized ANFIS as an image classifier. The classification accuracy of the approach is comparatively higher than other methods and also neural classifiers. Our approach uses optimized ANFIS to classify the red-eye artifacts in two steps: first, skin regions are detected by the system. After that by using five new features (which contain both color and geometric features) and employing them to train the system, red-eye areas are detected. The results showed 86% accuracy, 8% FAR and 6% FAR.