1 Introduction

Nowadays, computer-aided systems have been applied in many kinds of real-world problems, for example, finance (Bezděk 2014; Bennouna and Tkiouat 2018; Pislaru et al. 2019), control problems(Qiu et al. 2019; Sun et al. 2019), decision making (Hendiani and Bagherpour 2019), urban problems (Hawas et al. 2019), fault detection (Dhimish et al. 2018; Calderon-Mendoza et al. 2019) and medical diagnosis (Hu et al. 2011, 2018; Froelich 2017; Lahsasna and Seng 2017; Pota et al. 2018; Ahmadi et al. 2018; Fu et al. 2019; Ontiveros-Robles and Melin 2019a, b). Some of these applications have been developed based on fuzzy logic concepts (Mendel et al. 2006; Abu Arqub et al. 2016; Ontiveros-Robles et al. 2017; Arqub et al. 2017; Arqub and Al-Smadi 2020) and other intelligent techniques. However, these kinds of methods can also have a lot of potential for improving other methods because they are very versatile when they are combined in a hybrid fashion, especially Sugeno fuzzy systems. In addition, with the uncertainty handling capabilities provided by type-2 fuzzy logic, the potential advantages of fuzzy logic have now been augmented. For this reason, we propose combining uncertainty handling of type-2 fuzzy logic with the Support Vector Machine (SVM) model.

Based on the flexibility of type-2 fuzzy logic, the main contribution of the present paper is the proposal of two approaches for hybrid classifiers based on general type-2 fuzzy logic and Support Vector Machines. Both approaches have been compared with respect to the original methods and other conventional methods, for example, artificial neural networks and statistical methods. The reason to select SVM for building the proposed hybrid approaches is because it has been shown to be one of the better classifiers as reported in (Ghaddar and Naoum-Sawaya 2018; Xie et al. 2018; Richhariya and Tanveer 2018; Xu et al. 2019; Leong et al. 2019; Battineni et al. 2019; Saigal et al. 2019). It is worth mentioning that previous existing approaches similar to this work are only using interval type-2 fuzzy systems, and not general type-2 fuzzy, like in this work. A comparative study has been realized focused on diagnosis problems in order to put forward our proposed approaches in a relevant context under several uncertainty sources.

The organization of the paper is explained as follows: Sect. 2 contains an outline of the relevant topics that are the core of the proposed approach, Sect. 3 explains the proposed hybrid classifiers, Sect. 4 presents the experimental results and a preliminary discussion and finally Sect. 5 contains the conclusion of the paper.

2 Literature review

In this section, the relevant basic concepts about the proposed approach are introduced. In this case, type-2 fuzzy logic focused on Sugeno Fuzzy Inference Systems, and the Support Vector Machines are briefly described.

2.1 Type-2 fuzzy logic

Recently, type-2 fuzzy logic has demonstrated to be very useful in different kinds of problems, for example: industrial problems (Bukhari et al. 2018; Roy et al. 2019; Al-Jamimi and Saleh 2019), in fuzzy control (Castillo et al. 2016; Bai and Wang 2018; Castillo and Amador-Angulo 2018), in pattern recognition (Melin and Castillo 2013; Ramirez et al. 2019), in medical applications (Nguyen et al. 2015; Ontiveros-Robles and Melin 2019a), and many other areas. The ability of this kind of an approach to consider the uncertainty improves the performance that can be obtained for real- world applications.

Fuzzy logic was originally introduced by Zadeh in (Zadeh 1965), and allows the modeling of linguistic variables through mathematical functions called membership functions. In fuzzy logic, the concept of membership degree is not binary, as Zadeh introduced the concept of membership degree as a number in a continuous range from 0 to 1. These concepts were used as building blocks, for the so-called Fuzzy Inference Systems. For example, the Mamdani Fuzzy Inference System (Mamdani 1974) or the Takagi–Sugeno-Kang Fuzzy Inference System (Takagi and Sugeno 1993). Equation (1) describes the mathematical representation of a fuzzy set that is actually called a type-1 fuzzy set:

$$ {\text{A}} = \{\upmu_{\text{A}} \left( x \right)|\forall x \in X\} $$
(1)

where \( \upmu_{\text{A}} \left( x \right) \) is called membership function in the domain of X.

On the other hand, in recent years, fuzzy logic has been giving increasing attention to the concepts of type-2 fuzzy logic and its applications. type-2 fuzzy logic is an extension to type-1 fuzzy logic, but with the ability to consider the uncertainty in its mathematical model. The main advantage of using type-2 fuzzy logic over type-1 fuzzy logic is its ability for improving the performance of these systems in real-world applications with several uncertainty sources, and some examples can be found in (Bai and Wang 2018; Bukhari et al. 2018; Ramirez et al. 2019).

Type-2 fuzzy logic can be categorized based on its uncertainty modeling approach, there are interval type-2 fuzzy logic (Qilian Liang and Mendel 2000; Mendel et al. 2006; Li et al. 2018) and general type-2 fuzzy logic (Lucas et al. 2007; Wagner and Hagras 2010). However, the more complete uncertainty model can be described by the so-called general type-2 fuzzy systems, as this kind of systems model the uncertainty through a secondary membership function for every value of the primary membership function, obtaining in this way a three-dimensional membership function. The mathematical expression of these kinds of Fuzzy Sets is presented in Eq. (2):

$$ {\tilde{\text{A}}} = \left\{ {\left( {\left( {x,u} \right)} \right)\left| {{\text{u}} \in \left[ {0,1} \right],\quad\upmu_{{{\tilde{\text{A}}}}} \left( {x,u} \right)} \right\rangle 0} \right\} $$
(2)

where \( \upmu_{{{\tilde{\text{A}}}}} \left( {x,u} \right) \) is the type-2 membership function and \( u \) is the uncertainty domain.

These kinds of fuzzy systems demand more computational resources, but there exist alternatives to better approximate the model thus reducing the computational cost, and some examples of these approaches are the geometric approach, the z-slices approach and finally α-planes approach.

The α-planes approach was selected in this work to be used in order to enable the use of general type-2 fuzzy logic in the proposed approaches. This approximation of GT2 FIS consists on the discretization of the GT2 FS in horizontal slices called α-planes (Mendel et al. 2009) and then solving of these slices in a separate fashion, and after this, the α-plane outputs are aggregated in order to compute the final output. The mathematical equation of an α-plane and the aggregation of the α-planes are expressed in (3) and (4), respectively:

$$ \tilde{A}_{\alpha } = \left\{ {\left( {\left( {x,u} \right)} \right)|{\text{u}} \in \left[ {0,1} \right],\upmu_{{\tilde{A}}} \left( x \right) = \alpha } \right\} $$
(3)
$$ \tilde{Z} = \cup \tilde{Z}_{\alpha } $$
(4)

where \( \tilde{A}_{\alpha } \) is the \( \alpha \)-plane and \( \tilde{Z}_{\alpha } \) is the estimated output for the corresponding \( \alpha \)-plane.

In a general type-2 fuzzy inference system approximated by the α-planes representation, every α-plane can be solved as an interval type-2 fuzzy inference system.

An example of a general type-2 membership function can be found in Fig. 1.

Fig. 1
figure 1

General type-2 membership function

Full size image

As was mentioned previously every α-plane is solved as an interval type-2 fuzzy system and this implies a high computational cost. The stages of an interval type-2 Sugeno Fuzzy Inference System can be observed in Fig. 2.

Fig. 2
figure 2

IT2 sugeno FIS

Full size image

One of the main reasons because this kind of systems requires a higher computational cost with respect type-1 fuzzy systems is the type-reduction, there exist alternatives that reduces this process for example in Nie and Tan (2008) and Ontiveros-Robles et al. (2017), these processes are computed for every α-plane in a separate way and after aggregated according to Eq. (4). Even when this kind of Fuzzy System requires a high computational effort, it has demonstrated to provide good results because of the uncertainty handling in real-world problems (Ontiveros-Robles et al. 2018; Ontiveros et al. 2020).

2.2 Sugeno fuzzy inference systems

For this paper, the Takagi–Sugeno-Kang Fuzzy Inference System (TSK FIS) is proposed to be used for the design of the hybrid classifiers. The reason for using this kind of FIS is its versatility for different kinds of problems, for example (Shokouhifar and Jalali 2017; Krokavec and Filasová 2018; Dhimish et al. 2018; Tsai and Chen 2018; Bemani-N and Akbarzadeh-T 2019), and the flexibility of these systems to be easily combined with other methods, for example (Rezakazemi et al. 2017; Reddy and Sudhakar 2019; Rajab 2019).

The structure of a type-1 TSK FIS is illustrated in Fig. 3.

Fig. 3
figure 3

Structure of a type-1 TSK FIS

Full size image

As can be noted, the structure of the fuzzy rules is similar to the Mamdani fuzzy rules, but the difference is on the consequent. The rules of TSK FIS are not associated with a consequent membership functions, they are associated with mathematical functions. These functions are frequently linear polynomials, where the consequent function of the ith rule is expressed in (5) and the system output is presented in (6):

$$ f_{i} = \mathop \sum \limits_{{{\text{j}} = 2}}^{m} c_{i,j - 1} x_{j} + c_{0} $$
(5)
$$ {\text{O}} = \mathop \sum \limits_{i = 1}^{n} \varPhi_{i} f_{i} \left( {x_{1} \ldots x_{n} } \right) $$
(6)

where \( f_{i} \) is the linear function associated with the ith rule, \( c_{i,j - 1} \) is the called Sugeno coefficient, and \( \varPhi_{i} \) is the normalized firing force of the ith rule.

As can be noted, the Sugeno coefficients provide this approach with a lot of potential to be applied in different kinds of problems, and can be obtained based on learning or optimization methods.

On the other hand, there exist several approaches of type-2 TSK FISs, but is difficult to select which is the best approach, and some examples of these systems are presented in (Sanchez et al. 2017; Ontiveros-Robles and Melin 2019a).

For this paper, we propose to use an approach inspired on the general type-2 fuzzy inference systems designed in Ontiveros-Robles and Melin (2019a), and the main idea is the hybridation of this approach with the conventional binary Support Vector Machine, in order to evaluate the performance of the new hybrid approach.

2.3 Support vector machines

The main goal of a support vector machine (Fig. 4) is to find the optimal hyperplane that separates the data into two classes (binary SVM) (Ghaddar and Naoum-Sawaya 2018; Xie et al. 2018). Some relevant applications of SVM in real-world problems are energy management in hotels (Shao et al. 2020), hyperspectral image classification (Okwuashi and Ndehedehe 2020), detection the evolution of malwares (Wadkar et al. 2020), diagnosis of Alzheimer’s disease (Richhariya et al. 2020), and others.

Fig. 4
figure 4

Support vector machine

Full size image

On the other hand, this method allows the implementation of a strategy for increasing the dimensionality of the data in order to perform the best separation of the classes, and this process is obtained by the use of special functions called Kernels (Fig. 5).

Fig. 5
figure 5

Kernel example

Full size image

In the present paper, the equation of the hyperplane is expressed in (7):

$$ \begin{array}{*{20}c} {h_{1} x + h_{2} y + h_{3} + h_{4} x + h_{5} y + h_{6} \ge 1} \\ {h_{1} x + h_{2} y + h_{3} + h_{4} x + h_{5} y + h_{6} \le - 1} \\ \end{array} $$
(7)

where \( h_{\text{i}} \) is the ith coefficient of the hyperplane.

In this example, the hyperplane is for two attributes, but in the practice the number of attributes depends of the problem. Also, this kind of SVM approach does not apply any Kernel because in one of the approaches the Kernel functions are the fuzzy rules.

3 Hybrid classifier designing

This section explains the proposed approaches and methodology to generate the hybrid classifiers. In the proposed approaches we use the method for reducing the computational cost of type-2 fuzzy systems introduced in (Ontiveros et al. 2018).

To generate the membership functions, based on the training data, the method introduced in Ontiveros-Robles and Melin (2019a) is used. This method consists on generating the GT2 MFs based on the concept of embedded type-1 MFs. By the way, the uncertainty in the GT2 MFs is selected in order to be correlated with respect the training data. An example of this can be observed in Fig. 6.

Fig. 6
figure 6

Example of GT2 MFs generation

Full size image

3.1 GT2 + SVM approach

The first approach is mainly focused on the GT2 Fuzzy Classifier and its improvement through the SVM method. The approach consists on obtaining the Sugeno coefficients of the GT2 FIS through the SVM method. This approach is illustrated in Fig. 7.

Fig. 7
figure 7

GT2 + SVM hybrid approach

Full size image

For example, consider the following example: The output of the ith α-plane of a system with two fuzzy rules is expressed in (8):

$$ {\text{O}} = \overbrace {{\varphi_{1} \left( {c_{11} x + c_{12} y + c_{10} } \right)}}^{{{\text{Rule }}1}} + \overbrace {{\varphi_{2} \left( {c_{21} x + c_{22} y + c_{20} } \right)}}^{{{\text{Rule }}1}} $$
(8)

where \( \varphi_{1} \) is the first rule firing force, \( c_{11} \) is the Sugeno coefficient of the first rule and first input, \( c_{11} \) are the Sugeno coefficients of the system and the inputs x and y.

After a mathematical derivation, we obtain (9)

$$ {\text{O}} = c_{11} \varphi_{1} x + c_{12} \varphi_{1} y + c_{10} \varphi_{1} + c_{21} \varphi_{2} x + c_{22} \varphi_{2} y + c_{20} \varphi_{2} $$
(9)

As can be noted in (8), the equation is in fact a polynomial and based on this, we can obtain the Sugeno coefficients through the SVM method obtaining the following expressions (10):

$$ \begin{array}{*{20}c} {h_{1} \varphi_{1} x + h_{2} \varphi_{1} y + h_{3} \varphi_{1} + h_{4} \varphi_{2} x + h_{5} \varphi_{2} y + h_{6} \varphi_{2} \ge 1} \\ {h_{1} \varphi_{1} x + h_{2} \varphi_{1} y + h_{3} \varphi_{1} + h_{4} \varphi_{2} x + h_{5} \varphi_{2} y + h_{6} \varphi_{2} \le - 1} \\ \end{array} $$
(10)

where the coefficients \( h_{\text{i}} \) are the hyperplane coefficients obtained by the SVM methodology.

This approach considers the outputs of every α-plane as a hyperplane that separates the data in two classes.

3.2 SVM + GT2 approach

The second approach is mainly focused on the improvement of the SVM method through GT2 fuzzy logic. The main goal is the use of the fuzzy firing force of the rules as additional features for the SVM classifier, and in this way is possible to improve the separability of the data achieved by the SVM.

The concept is very similar to the Kernel functions, but the fuzzy firing forces of the rules can have interpretability.

Figure 8 illustrates the structure of the proposed approach.

Fig. 8
figure 8

SVM + GT2 hybrid approach

Full size image

As can be noted, this figure corresponds to one of the α-planes. However, for the generalized type-2 fuzzy system is necessary to perform the computation for the corresponding number of α-planes before the aggregation of these results.

This approach is similar to the GT2 + SVM approach, but with the difference in the inputs of the SVM method.

An equivalent example of this approach can be observed in (11):

$$ {\text{O}} = \overbrace {{\varphi_{1} c_{1} }}^{{{\text{Rule }}1}} + \overbrace {{\varphi_{2} c_{2} }}^{{{\text{Rule }}1}} + xc_{3} + yc_{4} $$
(11)

where the coefficients \( \varphi_{\text{i}} \) are the normalized firing forces of every rule.

As can be noted in this approach, the firing forces of the rules are introduced in order to increase the dimensionality of the data and expecting with this to increase the accuracy of the classifier. This approach demands less computational cost than the first introduced approach because the number of parameters is lower.

4 Experimental results

The benchmark problems selected for the comparison of the proposed approaches with respect to the original methods are the ones presented in (Ontiveros-Robles and Melin 2019a), considering that one of the references for comparison is the general type-2 fuzzy classifiers introduced in that paper. Table 1 summarizes these datasets.

Table 1 Diagnosis Datasets
Full size table

4.1 Hold out validation

This first validation is focused on the statistical comparison of the proposed approach with respect to an approach of GT2 classification. In this case, this is the approach that inspired the fuzzy inference systems used in the proposed approaches and introduced in (Ontiveros-Robles and Melin 2019a). The statistical test parameters are summarized in Table 2.

Table 2 Z-test parameters
Full size table

Tables 3 and 4 are presenting the accuracy performance for thirty experiments by using 60% for training and 40% for testing, the mean and the standard deviation and the Z-value. If the Z value is over 1.645 the statistical test provides sufficient evidence to accept the Ha, and this means that the proposed approach is better. Green cells indicate that the proposed approach is better, yellow means a draw and red means that the proposed approach is not the best.

Table 3 Statistical comparison of GT2 versus GT2-SVM
Full size table
Table 4 Statistical comparison of GT2 vs. SVM-GT2
Full size table

As can be noted in Table 3, the first hybrid approach (GT2-SVM) fails in showing an improvement in comparison with respect to the original GT2 approach, only in two of ten cases shows an improvement, and in six of ten the GT2 approach is better, and this can be related with the overfitting effect.

As can be noted, for the second hybrid approach, the statistical test shows an improvement in four of the ten datasets. On the other hand, the proposed approach is worse than the conventional GT2 approach in three cases, but the general average performance is better with the hybrid approach.

4.2 Cross-validation

The second validation consists in a cross-validation with different K values. The performance of the proposed hybrid approaches and also the original methods performance are reported. On the other hand, other results in the literature based on fuzzy logic are also reported.

Starting with K = 3, Tables 5, 6 and 7 summarize the results of the proposed approach and other fuzzy approaches of the literature. These tables document the accuracy as a performance measurement, this accuracy is obtained based on the average of ten experiments, and the standard deviation is also documented.

Table 5 K = 3 cross-validation results
Full size table
Table 6 K = 5 cross-validation results
Full size table
Table 7 K = 10 cross-validation results
Full size table

Based on the ten presented datasets, we summarize the results by a simple comparison explained as follows: for each dataset the best method obtains one point, if two or more obtain the same performance the point will be divided into the number of approaches that are in a draw. Tables 8, 9 and 10 show the results for K = 3, K = 5 and K = 10, respectively. In these tables we assign points for every dataset performance, 1 point if the method is the best, 0.5 points in a double draw and 0.333 points in a triple draw.

Table 8 Performance comparison K = 3
Full size table
Table 9 Performance comparison for K = 5
Full size table
Table 10 Performance comparison for K = 10
Full size table

As can be noted in the comparison, the results for cross-validation are different with the variation of K, and the conclusion of these is that the proposed hybrid GT2 SVM approach is not showing an improvement because obtains worst results than the original methods. In addition, we can observe that the hybrid SVM GT2 approach obtains better results with more percentage of training data (K = 5 and K = 10) and presents competitive performance and in some cases better than the original methods.

5 Conclusions

Based on the experimental results we observe an interesting effect in the proposed hybrid approach. First, in the proposed approach of the GT2-SVM classifiers, we observe that the performance tends to decrease in most of the cases, and in this case we cannot find a significant improvement in the hybridation. However, in the case of the hybrid SVM-GT2 approach, the performance of this approach is better than the original methods when compared in a separate fashion.

We can assume that the advantage of the SVM of being robust and avoiding the problem of overfitting help this approach to be better than the original GT2 classifier and the increase of the dimensionality of the data helps this approach to overcome the SVM method.

On the other hand, we can note that the proposed approaches are very competitive with respect to other fuzzy logic approaches that can be found in the literature, and even in many cases the proposed SVM-GT2 approach is the best method considering the proposed approaches and the listed references.