Extending Neuro-fuzzy Techniques with Grey-Based Hybridisation

Abstract

Real-world problems are commonly composed by interrelated components in many complex ways. They are usually dynamic, that is, they change with time through a series of interactions among related components. Classical decision-making techniques cannot support these kinds of challenges. For that reason, this paper focused on the extension of neuro-fuzzy techniques as Fuzzy Cognitive Maps with Grey Systems Theory.

4.1 Introduction

Real-world problems are commonly composed by interrelated components in many complex ways. They are usually dynamic, that is, they change with time through a series of interactions among related components (Salmeron, 2012). Classical decision-making techniques cannot support these kinds of challenges. For that reason, this paper focused on the extension of neuro-fuzzy techniques as Fuzzy Cognitive Maps with Grey Systems Theory.

Fuzzy Cognitive Maps (FCMs) constitute neuro-fuzzy systems, which are able to include experts’ knowledge (Kosko, 1986; Lee et al., 2002; Salmeron, 2009a, 2009b). From an Artificial Intelligence point of view, FCMs are supervised learning neural systems, whereas more and more data is available to model the problem, the system becomes more accurate at adapting itself and reaching a solution (Papageorgiou & Groumpos, 2005; Rodriguez-Repiso et al., 2007). FCM is a modelling technique for complex systems generated from mixing fuzzy logic and artificial neural networks (Salmeron, 2009a, 2009b).

FCMs have some benefits over classical modelling methods; they offer more information in the relationships between variables or concepts. Moreover, the FCM model is dynamic, and customizable and capture nonlinear relationships. The final fuzzy model is worthy for simulation, analysis and check the influence of variables and forecast the behaviour of the whole modelled system.

FCM have been used in many different fields as medicine (Papageorgiou, 2011; Salmeron, 2012), failure modes effects analysis (Pelaez & Bowles, 1995; Salmeron & Gutierrez, 2012) simulation (Fu, 1991), affective computing (Salmeron, 2012), engineering (Salmeron & Papageorgiou, 2014; Stylios & Groumpos, 2000), space exploration (Furfaro et al., 2010), relationship management in airline services (Kang et al., 2004), web-mining inference amplification (Lee et al., 2002) and others. For a deep review of the FCM research see (Papageorgiou & Salmeron, 2013).

4.2 Fuzzy Cognitive Maps

4.2.1 Theoretical Background

Fuzzy Cognitive Maps (Kosko, 1996) are a worthy technique for modelling and analysing the systems and people’s behaviour. FCMs are a set of nodes linked by edges. The nodes model concepts relevant to a given problem. The causal links between these concepts are modelled by the edges showing the direction of the influence. The other attribute of an edge is its sign, which can be positive (a promoting effect) or negative (an inhibitory effect).

The main aim of a FCM around a problem is to be able to forecast the outcome by letting the critical issues interact with one another. These forecasts can be used for checking whether an initial scenario is consistent with the entire collection of stated causal assertions (Bueno & Salmeron, 2009). Causal relationships between the nodes can have different weights, represented in a range [−1, +1] (or [0, +1]). FCM substitutes the signs by a fuzzy value between −1 and +1 where the zero value indicates the absence of causality. Moreover, it involves feedback, where the impact of the change in a node may affect other nodes (Lopez & Salmeron, 2013) (Fig. 4.1).

Fig. 4.1

Gaussian membership functions

The FCM nodes (c_i) would model such concepts such as heat, radiation, temperature or marketing strategy, among others. An edge linking two nodes models the causal influence of the causal node on the effect node (e.g.: the influence of the temperature to failure).

In a more formal way, an FCM can be represented as a 4-tuple;

$$\Omega = ({{\mathbf{N}}},{{\mathbf{A}}},f,r)$$

(4.1)

where N is the set of nodes $N={\{{n}_{i}\}}_{i=1}^{m}$ with m the number of them, A = [w_ij]_n×n is the adjacency matrix modelling the edges between nodes as ${\{{w}_{ij}\}}_{i,j=1}^{m}$, f(·) the activation function, and r the nodes’ range.

An adjacency matrix A represents the FCM nodes relationships. FCMs measure the strength of the causal relation between two nodes and if no causal relation be it is denoted by 0 in the adjacency matrix.

$$A=\begin{array}{cc}\begin{array}{c}{C}_{1}\\ \vdots \\ {C}_{n}\end{array}& \begin{array}{c}\begin{array}{ccc}{C}_{1}& \dots & {C}_{n}\end{array}\\ \left(\begin{array}{ccc}{w}_{11}& \cdots & {w}_{1n}\\ \vdots & \ddots & \vdots \\ {w}_{n1}& \cdots & {w}_{nn}\end{array}\right)\end{array}\end{array}$$

(4.2)

4.2.2 FCM Dynamic Analysis

FCMs are dynamical systems involving feedback, where the effect of change in a node may affect other nodes, which in turn can affect the node initiating the change. The analysis starts with the design of the initial vector state (c(0)), which models the initial value of each variable or concept (node) (Salmeron & Lopez, 2012). The initial vector state c(0) with n nodes is denoted as

$${\text{c}}\left( 0 \right) \, = \, \left( {{\text{c1}}\left( 0 \right),{\text{ c2}}\left( 0 \right), \, . \, . \, . \, ,{\text{ cn}}\left( 0 \right)} \right)$$

(4.3)

where c₁(0) is the value of the concept i = 1 at instant t = 0.

The new states of the nodes are computed in an evolutive updating process with an activation function, which is applied to map monotonically the node state into a normalized range [0, 1] or [−1, +1]. The sigmoid function is the most used one (Bueno & Salmeron, 2009) when the node value maps in the range [0, 1]. The vector state c(t + 1) at time t + 1 would be

$$\begin{array}{ll}c(t+1)& =f(c(t)\cdot A)\\ & =f({c}_{1}(t)\cdot A), \ldots ,f( {c}_{n}(t)\cdot A))\\ & ={c}_{1}\left(t+1\right)),\ldots ,({c}_{n}\left(t+1\right)\end{array}$$

(4.4)

The element i of the vector state c_i(t + 1) at time t + 1 can be computed as shown in Eqs. 4.5 and 4.6. Equation 4.5 computes c_i(t + 1) just with pre-synaptic computation.

$$\overbrace {C_i \left( {t + 1} \right)}^{post-synaptic} = f\left( {\overbrace {\mathop \sum \limits_{j = 1}^n W_{ji} .C_j \left( t \right)}^{pre-synaptic}} \right)$$

(4.5)

Equation 4.6 computes the pre-synaptic influence and aggregate the state of the post-synaptic node in the previous iteration. In that situation, the node has memory of its former state.

$$\overbrace {C_i ~\left( {t + 1} \right)}^{post - synaptic} = f\left( {\overbrace {C_i \left( t \right)}^{memory} + \overbrace {\mathop \sum \limits_{j = 1}^n W_{ji~} .C_j \left( t \right)}^{pre - synaptic}} \right)$$

(4.6)

If f(·) is the unipolar sigmoid function and the updating rule is as Eq. 4.6, then the element i of the vector state c(t) at time t would be

$${C}_{i}\left(t+1\right)=\frac{1}{1+{e}^{-\lambda.({C}_{i }\left(t\right)+{\sum }_{j+1}^{n}{W}_{ji }.{C}_{j}(t))}}$$

(4.7)

where λ is the constant modelling the degree of normalization (function slope). If f(·) is the hyperbolic tangent function and Eq. 4.6 the updating rule, then the element i of the vector state c(t) at time t would be as follows

$${C}_{i}\left(t+1\right)=\frac{{e}^{2}.\lambda.\left({c}_{i}\left(t\right)+{\sum }_{j=1}^{n}{w}_{ji}.{c}_{j}\left(t\right)\right)-1}{{e}^{2}.\lambda.\left({c}_{i}\left(t\right)+{\sum }_{j=1}^{n}{w}_{ji}.{c}_{j}\left(t\right)\right)+1}$$

(4.8)

After an inference process, the FCM reaches either one of two states after the iterations. It settles down to a fixed pattern of node values, the so-called hidden pattern or fixed-point attractor (as c(t) ≈ c(t − 1)).

Alternatively, it keeps cycling between several fixed states, known as a limit cycle. Using a continuous activation function, a third possibility known as a chaotic attractor exists. This occurs when, instead of stabilizing, the FCM keeps producing different state vector values for each iteration (Salmeron & Froelich, 2016).

4.2.3 FCM Consensus

It is possible to build FCMs from real-world raw data (e.g.: with evolutionary techniques) or from a panel of experts (Salmeron, 2009a, 2009b). Delphi methodology it is a methodology applied to manage the communication of a panel of experts in order to reach a consensus regarding a problem. One of the main features of Delphi method is when the experts get feedback reports, they could modify their own judgement based on the feedback (Bueno & Salmeron, 2009).

A more analytical way is the Augmented FCM. It doesn’t require that experts change their initial judgement for consensus as Delphi method needs. The augmented adjacency matrix is built adding the adjacency matrix of each expert (Salmeron, 2009a, 2009b). Let k FCMs with common nodes. The augmented adjacency matrix (A*) would be computed as follows

$$A*=\left(\begin{array}{ccc}{w}_{11=\frac{1}{k}}^{*}.{\sum }_{m=1}^{k}{w}_{11}.m& \cdots \frac{1}{k}.& {\sum }_{m=1}^{k}{w}_{1n}.m\\ \vdots & \ddots & \vdots \\ {w}_{n1=\frac{1}{k}}^{*}.{\sum }_{m=1}^{k}{w}_{1}.m& \cdots \frac{1}{k}.& {\sum }_{m=1}^{k}{w}_{nn}.m\end{array}\right)$$

(4.9)

If the FCMs have not common nodes, then the adjacency matrix would be added including zero rows and columns. Let a couple of FCMs ({w′_ij} and {w″_ij}) without common nodes, the augmented adjacency matrix would be

$$A*=\left(\begin{array}{ccc}{w}_{11=\frac{1}{2}}^{*}.{w^{{\prime}}}_{11}& \cdots & 0\\ \vdots & \ddots & \vdots \\ 0& {w}_{nn=}^{*}& \frac{1}{2}.{w}_{nn}^{{{\prime\prime}}}\end{array}\right)$$

(4.10)

Figure 4.2c shows an example of Augmented FCM building process of a couple of FCMs with common and not common nodes. Figure 4.2a, b detail the FCMs from experts A and B and Fig. 4.2c is the Augmented FCM.

Fig. 4.2

Augmented FCM

4.3 Fuzzy Grey Cognitive Maps

Fuzzy Grey Cognitive Maps (FGCMs) are a FCM generalization (proposed in Salmeron (2010)) for modelling complex systems with high uncertainty, under discrete incomplete and small data sets and it is based on Grey Systems Theory (Deng, 1989).

4.3.1 Theoretical Background

4.3.1.1 Grey Systems Theory

Grey Systems Theory (GST) is a useful approach for solving problems with high uncertainty and discrete small and incomplete datasets (Liu & Lin, 2006). GST has been successfully applied in fields such as military science, agriculture, medicine, meteorology, business, industry, energy, transportation, geology and so on (Yamaguchi et al., 2007).

GST categorizes the information according to the degree of the available knowledge in three-fold. The system is white if the knowledge of that system is fully known (whole understanding), but it is black if the system is completely unknown. Finally, a system with partial knowledge known and partial knowledge unknown is called a grey system (Deng, 1989).

GST includes data fuzziness because it can consider it (Liu & Lin, 2006). Furthermore, the fuzzy theory holds information usually based on experience; while GST is concerned with objective data, they do not need any information different than the datasets to be disposed.

In a more formal way, let Θ be the universal set, then a grey set Φ ⊂ Θ is defined by its both mappings. Note that $\left\{{\mu }_{\Phi }^{+}\left(\cdot \right), {\mu }_{\Phi }^{-}\left(\cdot \right)\right\}\in [\mathrm{0,1}]$, where ${\mu }_{\Phi }^{-}\left(\cdot \right)$ is the lower membership function, ${\mu }_{\Phi }^{+}\left(\cdot \right)$ is the upper one and ${\mu }_{\Phi }^{-}\left(\cdot \right)\le {\mu }_{\Phi }^{+}\left(\cdot \right)$. Also, GST extends fuzzy logic, since the grey set Φ becomes a fuzzy set when ${\mu }_{\Phi }^{-}\left(\cdot \right)={\mu }_{\Phi }^{+}\left(\cdot \right)$. The crisp value of a grey number is unknown, but the range in which the value is found is known.

An interval grey number is denoted as x^± ∈ [x⁻, x⁺] | x⁻ ≤ x⁺ and it has an upper (x⁺) and a lower (x⁻) limit (Yang & John, 2012). Both limits are crisp numbers in first order interval grey numbers.

When the grey number x^± has just an upper limit is as follows x^± ∈ (−∞, x⁺] and when it has just a lower limit is x^± ∈ [x⁻, + ∞). A black number has both unknown limits x^± ∈ (−∞, + ∞) and it becomes a white number when both limits are the same x⁻ = x⁺. The transformation of grey numbers in crisp ones $\widehat{x}$ is known as whitenisation (Liu & Lin, 2006) and it is computed as follows

$$\widehat{x}=\mathrm{ x}+ \cdot \upxi + (1 -\upxi ) \cdot \mathrm{ x}- |\upxi \in [0, 1]$$

(4.11)

where ξ is a parameter to control the position of the crisp values according to the limits. If ξ = 0.5, then it is called equal mean whitenisation. The length of a grey number is computed as l(x^±) = | x⁺ − x⁻|. In that sense, if the length of the grey number is zero (l(x^±) = 0), then it is a white number.

Despite the length of a grey number with only one limit known (lower or upper), x^± ∈ [x⁻, +∞) or x^± ∈ (−∞, x⁺] being infinite, the grey number is not a black number because it is possible to know one of the two limits.

Equations 4.12–4.15 compute the grey arithmetic operations

$$x\frac{+}{1}+x\frac{+}{2}\in \left[{x}_{1}^{-}+{x}_{1}^{-},{x}_{1}^{+}+{x}_{2}^{+}\right]$$

(4.12)

$$x\frac{+}{1}\_x\frac{+}{2}\in \left[{x}_{1}^{-}+{x}_{2}^{-},{x}_{1}^{+}+{x}_{2}\right]$$

(4.13)

$$\begin{aligned}x\frac{+}{1} \cdot x\frac{+}{2}\in& [min\left\{{x}_{1}^{-}\cdot{x}_{2}^{-},{x}_{1}^{+}\cdot{x}_{2}^{+},{x}_{1}^{-}\cdot{x}_{2}^{+},{x}_{1}^{+}\cdot{x}_{2}^{-}\right\}, \\ & \; max\left\{{x}_{1}^{-}\cdot{x}_{2}^{-},{x}_{1}^{+}\cdot{x}_{2}^{+},{x}_{1}^{-}\cdot{x}_{2}^{+},{x}_{1}^{+}\cdot{x}_{2}^{-}\right\}] \end{aligned}$$

(4.14)

$$\frac{{x\frac{+}{1}}}{{x\frac{+}{2}}}\in \left[min\left\{\frac{{x}_{1}^{-}}{{x}_{2}^{-}},\frac{{x}_{1}^{+}}{{x}_{2}^{+}},\frac{{x}_{1}^{-}}{{x}_{2}^{+}},\frac{{x}_{1}^{+}}{{x}_{2}^{+\_}}\right\},max\left\{\frac{{x}_{1}^{-}}{{x}_{2}^{-}},\frac{{x}_{1}^{+}}{{x}_{2}^{+}},\frac{{x}_{1}^{-}}{{x}_{2}^{+}},\frac{{x}_{1}^{+}}{{x}_{2}^{+\_}}\right\}\right]\left|\left\{{x}_{i}^{\_},{x}_{i}^{+}\right\}{}_{i=1}{}^{2}Y\ne 0\right|$$

(4.15)

4.3.1.2 FGCM Fundamentals

The FGCM nodes are concepts or variables and the relationships between them are modelled by grey weighted and directed edges (Nápoles et al., 2021; Salmeron, 2010). Formally, a FGCM is denoted as a 4-tuple

$$\Omega \, = \, \left( {\zeta ,{\text{ W}},{\text{ f}}^\pm \left( \cdot \right),{\text{ l}}\left( \uppsi \right)} \right)$$

(4.16)

where ζ = ${\{{\alpha }_{i}^{\pm }\}}_{i=1}^{n}$ is the set of nodes with grey numbers as grey states, W = $\{{w}_{ij}^{\pm }\}$ is the set of edges linking the nodes weighted with grey weights, f^±(·) the grey activation function and l(ψ) the range of the information space.

An interval grey weight between the nodes α_i^± and α_j^± is denoted as w_ij^± ∈ [w_ij⁻, w_ij⁺] and it has a lower (w_ij⁻) and an upper (w_ij⁺) limit.

FGCM includes greyness as an uncertainty score (Salmeron, 2010; Salmeron & Gutierrez, 2012; Salmeron & Papageorgiou, 2014). If a grey state (or weight) has a high score of greyness, then it means that the grey state have a high uncertainty associated. It is computed as follows

$$\emptyset \left(\alpha \frac{+}{i}\right)=\frac{\left|l(\alpha \frac{+}{i})\left| \right.\right.}{l(\psi )}$$

(4.17)

where |l(α_i^±)| = |α_i⁺ − α_i⁻| is the length of grey node state α_i^± in absolute value and l(ψ) is the length of the information space, denoted by ψ. It is computed as follows

$$l\left(\psi \right)=\left\{\begin{array}{c}1 if\, \left\{\alpha \frac{+}{i},\varpi \frac{+}{i}\right\} \subseteq \left[\mathrm{0,1}\right]\\ 2 if \, \left\{\alpha \frac{+}{i},\varpi \frac{+}{i}\right\} \subseteq \left[-1,+1\right]\end{array}\right.$$

(4.18)

where l(ψ) = 1 for unipolar FGCMs and l(ψ) = 2 for bipolar FGCMs.

FGCMs model the human reasoning in a more realistic way than FCM does, because it is able to manage the uncertainty between the concepts and the incomplete information about the modelled system as FCMs cannot do (Salmeron & Palos-Sanchez, 2019).

In Fig. 4.3a, the relationship (ϖ₁₃^±) between α₁^± and α₃^± is a white one, between α₁^± and α₂^± is a grey one (w₁₂^±), and between α₃^± and α₁^± is a black one (w₃₂^±).

Fig. 4.3

Potential relationships of the nodes in FGCM (a) and FCM (b)

FCMs are just able to model white (crisp) relationships (Fig. 4.3b). As a result, a huge amount of information is lost. Another advantage of FGCMs over FCM is related with the uncertainty associated to the elements of the problem (or system) modelled (Salmeron & Papageorgiou, 2012).

4.3.2 FGCM Dynamic Analysis

FGCMs are dynamical systems involving feedback just as FCMs are, where the influence of change in a grey node may affect other grey nodes, which in turn can impact the grey node starting the dynamics. The dynamic analysis begins with an initial grey vector state (α^±(0)), which models the initial grey value of each concept (grey node). The initial grey vector state α ± (0) with n grey nodes is denoted as

$$\alpha^\pm \left( 0 \right) \, = \, \left( {\alpha^\pm \left( 0 \right), \, \alpha^\pm \left( 0 \right),. \, . \, . \, ,\alpha^\pm \left( 0 \right)} \right)$$

(4.19)

where α^±(0) is the state of the concept i = 1 at time t = 0.

The grey state values of the grey nodes are updated in an evolutive process with a grey activation function, which is applied to map monotonically the grey node state into the information space (Salmeron, 2010). The grey state is updated as follows

$$\begin{array}{ll}{\alpha }^{\pm }& ={f}^{\pm }\left({\alpha }^{\pm }(t)\cdot {A}^{\pm }\right)\\ & =\left({f}^{\pm }\left({\alpha }_{1}^{\pm }(t)\cdot {A}^{\pm }\right),\ldots ,{f}^{\pm }\left({\alpha }_{n}^{\pm }(t)\cdot {A}^{\pm }\right)\right)\\ & =\left({\alpha }_{1}^{\pm }\left(t+1\right), \ldots {, \alpha }_{n}^{\pm }\left(t+1\right)\right)\end{array}$$

(4.20)

where A^± is the grey adjacency matrix and f^±(·) the grey activation function. Mostly, the grey activation function used to be the grey unipolar sigmoid (Eq. 4.21)

$$\alpha \frac{+}{i}\left(t+1\right)=\left[\frac{1}{{1+e}^{-.\lambda.\left(a\frac{+}{i}\left(t\right)+\sum_{j=1}^{n}{w\frac{+}{ji}}.\alpha \frac{+}{j}\left(t\right)\right)}},\frac{1}{{1+e}^{-.\lambda. \left(a\frac{+}{i}\left(t\right)+\sum_{j=1}^{n}{w\frac{+}{ji}}.\alpha \frac{+}{j}\left(t\right)\right)}}\right]$$

(4.21)

or the grey hyperbolic tangent (Eq. 4.22) as follows

$$\alpha \frac{+}{i}\left(t+1\right)=\left[\frac{{e}^{2.\lambda.\left(a\frac{+}{i}\left(t\right)+\sum_{j=1}^{n}{w}_{ji}.\alpha \frac{+}{j}\left(t\right)\right)}-1}{{e}^{2.\lambda.\left(a\frac{+}{i}\left(t\right)+\sum_{j=1}^{n}{w}_{ji}.\alpha \frac{+}{j}\left(t\right)\right)}+1},\frac{{e}^{2.\lambda.\left(a\frac{+}{i}\left(t\right)+\sum_{j=1}^{n}{w}_{ji}.\alpha \frac{+}{j}\left(t\right)\right)}-1}{{e}^{2.\lambda.\left(a\frac{+}{i}\left(t\right)+\sum_{j=1}^{n}{w}_{ji}.\alpha \frac{+}{j}\left(t\right)\right)}+1}\right]$$

(4.22)

4.3.3 FGCM Consensus

It is possible to build FGCMs from real-world raw data (e.g.: using evolutionary techniques) or from a panel of experts. As far as the author knows, there have been not used Delphi methodology for building FGCMs yet, but augmented FGCMs have been used. The augmented adjacency grey matrix is built adding the adjacency grey matrix of each expert (Salmeron & Palos-Sanchez, 2019).

Let k FGCMs with common grey nodes. The augmented adjacency grey matrix (A^±*) would be computed as follows

$${A}^{\pm *} =\left(\begin{array}{ccc}{\varpi }_{11}^{*}=\frac{1}{k}{.\sum_{m=1}^{k}\varpi \frac{+}{n1}.m}& \cdots & \varpi \frac{*}{1n}=\frac{1}{k} .\sum_{m=1}^{k}\varpi \frac{+}{1n}.m\\ \vdots & \ddots & \vdots \\ {\varpi }_{n1}^{*}=\frac{1}{k}{.\sum_{m=1}^{k}\varpi \frac{+}{n1}.m}& \dots & \varpi \frac{*}{nn} = \frac{1}{k} .\sum_{m=1}^{k}\varpi \frac{+}{nn }. m\end{array}\right)$$

(4.23)

If the FGCMs have not common grey nodes, then the adjacency grey matrix would be added including null rows and columns. Let a couple of FGCMs {w_ij^±(a)} and {w_ij^±(b)} without common grey nodes, the augmented adjacency grey matrix would be

$${A}^{\pm *} =\left(\begin{array}{ccc}{\varpi }_{11}^{+\left(a\right)*}=\frac{1}{2}.{\varpi }_{11}^{+\left(a\right)}& \cdots & 0\\ \vdots & \ddots & \vdots \\ 0& \dots & \varpi \frac{+\left(b\right)*}{nn} = \frac{1}{2} . \varpi \frac{+(b)}{nn}\end{array}\right)$$

(4.24)

4.4 Conclusions

This work shows the extension of neuro-fuzzy techniques with Grey Systems Theory. FGCMs are a generalization for modelling systems with high uncertainty, under discrete incomplete and small data sets.

FGCMs model the human reasoning in a more realistic way than FCM does, because it is able to process the uncertainty between the nodes and the incomplete information about the modelled problem as FCMs cannot do.