Keywords

These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

1 Fuzzy System Models

Here first, historically significant fuzzy system model developments are reviewed in order to identify their unique structures and to point out how they differ from each other. Then we show the details of our FULL TYPE 2 Fuzzy System developments with a new algorithm.

2 Type 1 Fuzzy Rule Base Models

The most commonly applied fuzzy system models are fuzzy rule bases. Here, we only deal with Multi-Input Single Output (MISO) systems. Generally fuzzy system models represent relationships between the input and output variables which are expressed as a collection of IF-THEN rules that utilize linguistic labels, which are represented with fuzzy sets. The general fuzzy rule base structure which is known as Zadeh- Fuzzy Rule Base, Z-FRB, can be written as follows:

$$ R:\mathop {\text{ALSO}}\limits_{\text{i = 1}}^{\text{c*}} \left( {{\text{IF}}\;antecedent_{i} {\text{THEN}}consequent_{i} } \right), $$

where c* is the number of rules in a rule base either given by experts or it is determined by a fuzzy clustering algorithm such as FCM, Fuzzy-C-Means (Bezdek [1]) or IFC, Improved Fuzzy Clustering (Çelikyılmaz and Türkşen [2]). The fuzzy rule base structures determined by various alternatives mainly differ in the representation of the consequents. If the consequent is represented with fuzzy sets then the fuzzy rule base is known as Zadeh [13] version which is originally applied by Mamdani, et al., [3], and a modified version is proposed by Sugeno and Yasukawa, SY-FRB, [6]. Whereas, if the consequents are represented with linear equations of input variables, then the rule base structure is the Takagi-Sugeno Fuzzy Rule Base, TS-FRB, [5] structure. These are the main models amongst others which we do not review in this paper. In particular Zadeh Fuzzy Rule Bases, Z-FRB can be formalized as: \( R: \, \mathop {ALSO}\limits_{i = 1}^{c*} \left( {{\text{IF }}x \in X{\text{ isr }}A_{i} {\text{ THEN y}} \in Y{\text{ isr }}B_{i} } \right) \)

In general, let nv be the number of selected input variables in the system. Then, the multidimensional antecedent, x, can be defined as x = (x 1 ,x 2 ,…,x nv ), where x j is the j th input variable of the antecedent and the domain of x in X, can be defined as X = X 1  × X 1  × … × X nv , X j  ⊆ ℜ.

In particular, the Z-FRB structure can be expressed as follow, where the multi-dimensional antecedent fuzzy subset of i th rule is Ai. This multi-dimensional antecedent fuzzy subset determination eliminates the search for the appropriate t-norm for the combination of antecedent fuzzy subsets with “AND”.

Thus, variations of Z-FRB are Sugeno-Yasukawa, SY-FRB, and Takagi-Sugeno (TS-FRB) Fuzzy Rule Base structures:

$$ \left( {{\text{SY-FRB}}} \right)\quad \quad \quad \quad \quad R: \, \mathop {ALSO}\limits_{i = 1}^{c*} \left( {{\text{IF }}x \in X{\text{ isr }}A_{i} {\text{ THEN y}} \in Y{\text{ isr }}B_{i} } \right) $$
$$ \left( {{\text{TS-FRB}}} \right)\quad \quad \quad \quad \quad R:\mathop {ALSO}\limits_{i = 1}^{c*} \;\,(IF\;antecedent_{i} \;THEN\;y_{i} { = }a_{i} x^{T} \, + \,b_{i} ) $$

where, antecedenti = x ∈ X isr A i, and a i  = (a i,1 ,…, a i,NV ) is the regression coefficient vector associated with the i th rule together with b i which is the scalar associated with the i th rule. For these special cases of Z-FRB, again each degree of firing, \( d_{i} \), associated with the-i th rule, is determined directly from the corresponding i th multi-dimensional antecedent fuzzy subset A i and applied to the consequent fuzzy subset for the SY-FRB or to the classical ordinary regression for the case of TS-FRB.

3 Fuzzy Regression Functions

There are a number of variations of the proposed Fuzzy Regression Functions. We discuss here only one alternative in this paper, namely, Fuzzy Regression Functions which we have proposed with LSE.

3.1 Fuzzy Regression Functions with LSE (FF-LSE)

In ordinary LSE (Least Square Estimation) method, the dependent variable, y, is assumed to be a linear function of input, variables, x, plus an error component:

$$ y = \beta_{0} + \beta_{1} x_{1} + \ldots + \beta_{nv} x_{nv} + \varepsilon $$

where y is the dependent output, x j’s are the explanatory variables input, for j = 1,…, nv, nv is the number of selected inputs and ε is the independent error term which is typically assumed to be normally distributed. The goal of the least squares method is to obtain estimates of the unknown parameters, β j ’s, j = 0,1,…, nv, which indicate how a change in one of the independent variables affects the dependent variable.

$$ \beta = \left( {X^{T} X} \right)^{ -1} X^{T} y $$

The proposed generalization of LSE as FF-LSE (Fuzzy Functions with LSE, more appropriately know as Fuzzy Regression Functions with LSE), requires that a fuzzy clustering algorithm, such as FCM, or IFC be available to determine the interactive (joint) membership values of input-output variables in each of the fuzzy clusters that can be identified for a given training data set. Let (X k ,Y k ), k = 1,…, nd, be the set of observations in a training data set, such that X k  = (x jk | j = 1,…, nv).First, one determines the optimal (m*, c*) pair for a particular performance measure, i.e., a cluster validity indeces such as Bezdek […], and Celikyılmaz and Türkşen […] with an iterative search and an application of FCM or IFC algorithm, where m is the level of fuzziness (in our experiments we usually take m = 1.4,…,2.5), Ozkan and Turksen […]) and c is the number of clusters (in our experiments we usually take c = 2,…,10). The well known FCM (Bezdek 1973) algorithm can be stated as follows:

$$ \begin{aligned} & \hbox{min} \quad {{J(U,V) = }}\sum\limits_{\text{k = 1}}^{\text{nd}} {\sum\limits_{i = 1}^{c} {(u_{ik} )^{m} (\left\| {x_{k} - v_{i} } \right\|} )_{A} } \\ & s.t. \, \quad \quad \;\,0 \le u_{ik} \le 1,\forall i,k \\ \, & \quad \quad \quad \quad \sum\limits_{i = 1}^{c} {u_{ik} } = 1,\forall k \\ & \quad \quad \quad \quad 0 \le \sum\limits_{k = 1}^{nd} {u_{ik} } \le nd,\forall i \\ \end{aligned}, $$

where J is objective function to be minimized, ||.|| A is a norm that specifies a distance based similarity between the data vector x k and a fuzzy cluster center v i . In particular, A = I is the Euclidian Norm and A = C −1 is the Mahalonobis Norm, etc.

Once the optimal pair (m*, c*) is determined with the application of FCM algorithm, and a cluster validity index, one next identifies the

cluster centers for m = m* and c = 1,…,c* as:

$$ \mathop {v_{X|Y,\,j} }\limits_{m*} = (x_{1,\,\,j}^{c} ,x_{2,\,j}^{c} , \cdots ,x_{nv,\,j}^{c} ,y_{\,j}^{c} ) $$

From this, we identify the cluster centers of the input space again for m = m* and c = 1,…,c* as: \( \mathop {v_{X,\,j} }\limits_{m*} = (x_{1,\,j}^{c} ,x_{2,\,j}^{c} , \cdots ,x_{nv,\,j}^{c} ). \)

Next, one computes the normalized membership values of each vector of observations in the training data set with the use of the cluster center values determined in the previous step. There are generally two steps in this calculations:

First we determine the (local) optimum membership values u ik ‘s and then determine µ ik ‘s that are above an α- cut in order to eliminate harmonics generated by FCM as:

$$ u_{ik} = \left( {\sum\limits_{j = 1}^{c} {\left( {\frac{{\left\| {x_{k} - v_{X,i} } \right\|}}{{\left\| {x_{k} - v_{X,\,j} } \right\|}}} \right)}^{{\frac{2}{m - 1}}} } \right)^{ - 1} ,\mu_{ik} \ge \alpha , $$

where µ ik denotes the membership value of the k th vector, k = 1,…,nd, in the i th rule, i = 1,…,c* and x k denotes the k th vector and for all the input variables j = 1,…, nv, in the input space. (2) Next, we normalize them as:

$$ \gamma_{ij} (x_{j} ) = \frac{{\mu_{ij} (x_{j} )}}{{\sum\limits_{i' = 1}^{c} {\mu_{i'j} (x_{j} )} }} $$

where \( \gamma \) ij is the normalized membership value of x j , j = 1,…, nv, in the i th rule, i = 1,…,c*, which in turn will indicate the membership value that will constitute an new input variable in our proposed scheme of function identification for the representation of i th cluster. Let \( \Gamma _{i} = (\gamma_{ij} |i = 1, \ldots ,c*; j = 1, \ldots ,nv) \) be the membership values of X in the i th cluster, i.e., rule.

Next we determine a new augmented input matrix X for each of the clusters which could take on several forms depending on which transformations of membership values we want to or need to include in our system structure identification for our intended system analyses. Examples of these are:

$$ X_{i}^{\prime } = \left[ {1,\Gamma _{i} ,X} \right],\quad \quad X_{i}^{\prime '} = \left[ {1,\Gamma _{i}^{2} ,X} \right],\quad \quad X_{i}^{\prime \prime \prime } = \left[ {1,\Gamma _{i}^{2} ,\Gamma_{i}^{m} ,\exp (\Gamma _{i} ),X} \right], $$

etc., where X i ′, X i ″, X i ″′ are the new input matrices to be used in least squares estimation of a new system structure identification where

$$ \Gamma _{i} = (\gamma_{ij} |i = 1, \ldots ,c*;j = 1, \ldots ,nv). $$

The choice depends on whether we want to or need to include just the membership values or some of their transformations as new input variables in order to obtain a best representation of a system behavior. In particular, this is done in order to get a higher value of R2 to show that a better model is obtained for an application. A new augmented input matrix, say X i ′, would look as shown below for the special case of X = X j , i.e., the matrix X is just a vector of a single variable, X j  = (x jk |k = 1,…,nd) for the j th variable:

$$ X_{{^{ij} }}^{\prime} = [1,\Gamma _{i} ,X_{ij} ] = \left[ {\begin{array}{*{20}c} 1 & {\gamma_{i1} } & {x_{ij1} } \\ \vdots & \vdots & \vdots \\ 1 & {\gamma_{ind} } & {x_{ijnd} } \\ \end{array} } \right] $$

Thus the fuzzy regression function, \( Y_{i} = \beta_{i0} + \beta_{i1}\Gamma _{i} + \beta_{i2} X_{ij} \), that represents the i th rule corresponding to the i th interactive (joint) cluster in space \( (Y_{i} ,\Gamma _{i} ,X_{j} ) \), \( \beta_{i}^{*} = (X_{ij}{\prime}^{T} X_{ij}^{\prime})^{ - 1} (X_{ij}{\prime}^{T} Y_{i} ),\quad \quad X_{ij}^{\prime } = \left[ {1,\Gamma _{i} ,X_{ij} } \right]. \)

Such that \( \beta_{i}^{*} = (\beta_{i0}^{*} ,\beta_{i1}^{*} ,\beta_{i2}^{*} ) \) and the estimate of Y i would be obtained as \( Y^{*}_{i} = \beta_{i0}^{*} + \beta_{i1}^{*}\Gamma _{i} + \beta_{i2}^{*} X_{ij}. \)

Within the proposed framework, the general form of the shape of a cluster can be conceptually captured by a second order (cone) in the space of U × X × Y which can be illustrated with a prototype shown in Fig. 1.

Fig. 1
figure 1

A Fuzzy cluster in U × X × Y space

Full size image

One usually determines Type 1 membership values with an application of FCM […] algorithm shown below:

where Eq. 1 stated in the algorithm above is:

$$ \mu_{ik}^{(t)} = \left[ {\sum\limits_{j = 1}^{c} {\left( {\frac{{d\left( {x_{k} ,\upsilon_{i}^{(t - 1)} } \right)}}{{d\left( {x_{k} ,\upsilon_{j}^{(t - 1)} } \right)}}} \right)}^{{\frac{2}{m - 1}}} } \right]^{ - 1} $$

And Eq. 2 is:

$$ \upsilon_{i}^{(t)} = {{\left( {\sum\limits_{k = 1}^{n} {\left( {\mu_{ik}^{(t)} } \right)^{m} x_{k} } } \right)} \mathord{\left/ {\vphantom {{\left( {\sum\limits_{k = 1}^{n} {\left( {\mu_{ik}^{(t)} } \right)^{m} x_{k} } } \right)} {\sum\limits_{k = 1}^{n} {\left( {\mu_{ik}^{(t)} } \right)^{m} } }}} \right.} {\sum\limits_{k = 1}^{n} {\left( {\mu_{ik}^{(t)} } \right)^{m} } }},\forall i = 1, \ldots ,c $$

4 Generation of Full Type 2 Membership Values

For this purpose, we propose and hence introduce an new algorithm in order to generate Full Type 2 membership value distribution from the results obtained with an application of FCM which produce a Type 1 membership value distribution for our studies of Full Type 2 investigations.

5 Full Type 2 Fuzziness i.e., Membership of Membership

Here we want to show how one determines the second order degree of fuzziness in order to develop Full Type 2 fuzzy system models.

It should be noted that depending on where \( x \in X \) is there may be more than one second order membership value distribution.

6 Full Type 2 Fuzzy Set Extraction Algorithms

We propose the following Full Type 2 fuzzy set extraction algorithm from a given data set called FT2FCM (Türkşen, 2012):

Full Type 2 Fuzzy Clustering Algorithm

$$ Min\,j{}^\prime (U^{\prime } (U),\,W) = $$
$$ = \sum\limits_{k = 1}^{nd} {\sum\limits_{i = 1}^{c\prime } {\sum\limits_{l = 0}^{1} {\left( {\mu_{{\mu i\left( {x_{k} } \right)}} \left( Z \right)} \right)} } } \left( {\left\| {\mu_{{\mu i\left( {x_{k} } \right)}} \left( {Z_{l} } \right) - \bar{\mu }_{{\left( {x_{k} } \right)}} \left( {Z_{l} } \right)\left\| A \right.} \right.} \right),k = 1, \ldots ,\,nd; $$
$$ i = 1, \ldots ,c^{\prime } $$
$$ st.\,0 \le \mu_{{\mu i\left( {xk} \right)}} (Z) \le 1 $$
$$ 0 \le \mu_{i} \left( {x_{k} } \right) \le 1 $$
$$ 0 \le \sum\limits_{k = 1}^{nd} {\mu_{i} \left( {x_{k} } \right) \le nd} $$
$$ \mu_{i} \left( {x_{k} } \right) \mathrel\backepsilon \left[ {0,1} \right];\;\;\mu_{{\mu i(x_{k} )}} (Z) \mathrel\backepsilon \left[ {0,1} \right];\;\;l \mathrel\backepsilon \left[ {0,1} \right] $$

where \( j^{\prime} \) is the objective function to be minimized for a given \( x_{k} \in X,\left\| . \right\|_{\text{A}} \) is a norm, i.e., Euclidian or Mahalanobis, that specifies a distance measure based on a membership values for a given \( x_{k} \in X \) and its second order fuzzy cluster center \( \bar{\mu }_{i} \left( {x_{k} } \right) \).

Next one computes the normalized membership values of these Full Type 2 membership values for each vector of membership values obtained in an initial application of the original FCM or IFC algorithm in the first stage.

There are generally two steps in these calculations:

We first determine (local) optimum membership of membership values \( \mu_{{\mu_{i} }} \left( {x_{k} } \right) \)’s and then apply an α-cut in order to eliminate the second order harmonics generated by an application of FT2FCM as:

$$ \begin{array}{*{20}c} {\mu_{{\mu_{i} }} \left( {x_{k} } \right)} \\ {x_{k} \in X} \\ \end{array} = \left[ {\left( {\sum\limits_{i = 1}^{c\prime } {\frac{{\left\| {\mu_{\mu i} \left( {x_{k} } \right) - \bar{\mu }_{i} \left( {x_{k} } \right)} \right\|}}{{\left\| {\mu_{\mu i} \left( {x_{k} } \right) - \mu_{j} \left( {x_{k} } \right)} \right\|}}} } \right)^{{\frac{2}{m - 1}}} } \right]^{ - 1} $$
$$ \gamma^{\prime }_{{\mu_{{\mu_{i} }} (x_{k} )}} = \frac{{\mu_{{\mu_{i} }} (x_{k} )\left| {_{{x_{k} \in X}} } \right.}}{{\sum\nolimits_{i = 1}^{{c^{\prime } }} {\mu_{{\mu_{i} }} (x_{k} )} }},\mu_{{\mu_{i} }} (x_{k} ) \ge \alpha , \gamma^{\prime }_{{\mu_{{\mu_{i} }} (x_{k} )}} \ge \alpha $$

where \( \gamma^{\prime }_{{\mu_{{\mu_{i} }} (x_{k} )}} \) denotes the membership values of the membership values of the kth vector k = 1,..,nd in the ith rule, or ith fuzzy regression function (Türkşen, 2012) and \( x_{k} \in X \) denotes the kth vector and for all the input variables, k = 1,.., nd in the input space.

Recall that we are able to obtain the membership value distribution as:

$$ X^{\prime}_{ij} = [1,\Gamma _{i} ,X_{ij} ] = \left[ {\begin{array}{*{20}c} 1 & {\gamma_{i1} } & {x_{i1} } \\ \vdots & \vdots & \vdots \\ 1 & {\gamma_{ind} } & {x_{ind} } \\ \end{array} } \right] $$
$$ \Gamma _{i} = (\gamma_{ik} |i = 1, \ldots ,c*;k = 1, \ldots ,nd) $$
$$ \Gamma _{i} = \left( {\gamma_{ij} |i = 1, \ldots ,c^{*} ;j = 1, \ldots ,nd} \right)\quad \quad\Gamma _{i} = \left[ {\begin{array}{*{20}c} {\gamma_{11} } & {\gamma_{21} } & { \cdots \gamma_{{c^{*} 1}} } \\ \vdots & \vdots & \vdots \\ {\gamma_{1nd} } & {\gamma_{2nd} } & { \cdots \gamma_{{c^{*} nd}} } \\ \end{array} } \right] $$

We process each \( \gamma_{i} \) via our Full Type 2 clustering algorithm given above, called FT2FCM, to determine Full Type 2 distribution for each cluster i, \( \Gamma _{i} = \left( {\gamma_{ij} |i = 1, \ldots ,c^{*} ;j = 1, \ldots ,nd} \right) \).

Thus we apply to each \( \Gamma _{i} \), ALGORITHM 2 given below to generate Full Type 2 membership, values, i.e., membership of membership.

7 Experimental Results

We present here our experimental results for TD_Stock Price Data set that is available for all researchers on the internet.

Çelikyılmaz-Türkşen’s validity index results for TD_Stockprice data:

Fuzzy classification of TD_Stockprice data: (c* = 2,m* = 1.8)

Cluster-2 view for TD_Stockprice data:

Çelikyılmaz-Türkşen’s Validity Index for µ ik data:

Cluster-2 results of TD-Stockprice data (c* = 2,m* = 1.8)

According Çelikyılmaz-Türkşen index, the suitable number of cluster should be chosen as c’ = 2 (µ ik data is the membership values of first study’s cluster-2). Where c’ = 2, m’ = 1.8.

µ lk (µ ik ) for Cluster1 and 2 are shown above:

A possible three cluster view:

TD_Stockprice Data set:

According to Bezdek’s validity index results (shown as follows), the suitable number of cluster was chosen as c* = 3:

Fuzzy classification of TD_Stockprice data: (c* = 3,m* = 2.0)

Cluster-2 view for TD_Stockprice data:

Çelikyılmaz-Türkşen’s Validity Index for µ ik data:

Cluster-2 results of TD-Stockprice data (c* = 3,m* = 2.0) for membership of membership.

According Çelikyılmaz-Türkşen index, the suitable number of cluster should be chosen as c’ = 2 (the µ ik data is the membership values of first study’s cluster-2). Where c’ = 2,m’ = 2.0.

A possible three cluster view:

8 Conclusions

In this paper, we have first review the essentials fuzzy system models: such as (1) Zadeh’s rulebase model, (2) Takagi and Sugeno’s partly a rule base and partly a regression function model and (3) Türkşen’s “Fuzzy Regression Functions” model where a fuzzy regression function correspond to each fuzzy rule and thus a fuzzy rule base is replaced with “Fuzzy Regression Functions” model. Next we review the well known FCM algorithm which lets one to extract Type 1 membership values from a given data set for the development of “Type 1” fuzzy system models as a foundation for the development of “Full Type 2” fuzzy system models. For this purpose, we provide an algorithm which lets one to generate Full Type 2 membership value distributions for a development of second order fuzzy system models with our proposed second order data analysis. If required one can generate Full Type 3,…, Full Type n fuzzy system models with an iterative execution of our algorithm. Finally we present our results graphically for TD_Stockprice data with respect to two validity indeces, namely: (1) Çelikyılmaz-Türkşen and (2) Bezdek indeces. Based on our development, we expect in the future new results would be obtained in “Full Type 3,…, Full Type n” fuzzy system model analyses.