Abstract
It has become essential to develop machine learning techniques due to the automation of various tasks. At present, several tasks need manual intervention for better reliability of the system. In this work, fuzzy-based approach has been proposed where systems are trained based on initial data sets. In several data sets, the data is either partially available or unavailable. When data sets need to be used on real time systems, the non-availability of data may lead to catastrophe. In this approach, a fuzzy-based rule set is formulated. The rule strength is used to determine the effectiveness. Rules with similar strengths are clustered together. The learning is carried out by determining a threshold for the formulated rule set. Based on the threshold computed, a modified rule set is formulated with rule strengths greater than the computed threshold. A semi-supervised learning approach that uses an activation function is employed. The fuzzy learning approach proposed in this work reduces the error by 20% compared to conventional approaches.
Access provided by Autonomous University of Puebla. Download conference paper PDF
Similar content being viewed by others
Keywords
- Threshold
- Activation function
- Learning rule matrix
- Machine learning
- Rule strength
- Smart systems
- Semi-supervised learning
1 Introduction
In today’s world, there is a need to design gadgets and systems with automation. To enable automation of gadgets, it is essential that systems handle decisions independently based on the environmental conditions.
Systems should adapt to the external environment and handle tasks based on the underlying behaviour. Systems should also handle exceptional tasks. To fulfil the same, the system needs to be intelligent to take decisions in the absence of manual intervention.
Machine learning is one way of implementing these features of Artificial Intelligence where systems are trained based on an initial data set.
The system forms the initial dataset by collecting the information from the external environment through sensors and actuators. System is trained to learn information and take decisions based on iteration of datasets. The time needed by the system to learn information and take decisions from the datasets should be finite and small so as to avoid failures that can lead to catastrophe.
In this work, a fuzzy rule matrix is constructed and the threshold of the entire rule set is also computed. The modified rule set is constructed by selecting rules whose rule strength is greater than the threshold. These are the rules used by the system to learn the data. Other rules are not considered.
The paper is organised as follows: Sect. 2 discusses the literature on machine learning algorithms. Section 3 explains the fuzzy model proposed, illustrates the same with an example and gives the algorithm. Section 4 explains the simulation environment and the results. Section 5 concludes the work and highlights the scope for the future work.
The block diagram of the entire procedure is shown in Fig. 1.
2 Literature Review
The most conventional machine learning strategy is the Naive Bayes classifier. Several literatures report the inaccuracies in Bayes’ conditional independence. This is due to the suboptimal nature of the probability estimates as reported in [1,2,3]. Even when multidimensional tables are used to compute probabilities in real-time scenarios, many errors are observed as discussed in [2]. Support Vector method uses a hyperplane to classify the data. Based on the classification a number of support vectors are generated as explained in [4]. But when data sets are widely spread and have a large range of deviation, this method does not yield accurate results. The choice of the hyperplane is the key factor associated with the data classification as discussed in [5, 6]. Artificial neural networks give reliability but again this depends on the choice of the activation function used to train the data set as explained in [7, 8]. Linear and Logistic regression as explained in [9,10,11] uses a cost function and estimates the gradient. The reduction in the gradient is used to learn the datasets. The regression analysis depends on the effectiveness of the cost function.
Several deep learning neural network algorithms model a data set as a function using a mathematical model and train the same to interpolate or extrapolate the data value for a particular value. The Stochastic Gradient Descent technique as explained in [12] attempts to minimise the loss of the model by incorporating additional parameters and increments these proportional to the gradient estimated. But this makes the data noisy due and filters needs to be used to eliminate the noise associated. The method of steepest descent as discussed on [13] which when used on a very large data set requires an infinite number of iterations to get an optimum solution. It may so happen that an optimum solution may not exist also. Another approach which works in a different dimension is Reinforcement learning as explained in [14]. This method arrives at an optimal solution by trial and error in infinite time. But the time of training in an unsupervised scenario would be infinite and the optimal solution cannot be guaranteed. The drawbacks highlighted in the above techniques are overcome in the fuzzy-based analysis. In the fuzzy-based analysis a precise value for the rule strength are arrived and the optimal solution is arrived by training the same with an activation function in a finite period of time. It does not require any mathematical model and infinite number of trials to arrive at an optimal solution.
3 Proposed Work
In this work a fuzzy-based learning approach using Mamdani’s engine as discussed by Venkat and Pradheep in [15] has been used. For each rule, a rule strength is computed. The rule strength is computed based on a set of sub-factors and their corresponding weights. For each linguistic variable used in the rule, the mid-value of the range is computed. The weighted average of each variable of the rule gives the rule strength. A typical rule is given as follows:
The Rule Strength is computed using the expression
The rule strengths have been computed for all rules in the fuzzy rule set using the above expression. The threshold of the fuzzy rule set is computed using the following expression
where n is the total number of rules.
The threshold computed is a measure of the upper bound till completion of the training of the system. The modified rule set is constructed by selecting rules, which have a rule strength greater than the computed threshold. After this a matrix is constructed in the form (RS − mT, RS, RS + mT), where m = 1 to n and n is the number of rules in the original rule set. The matrix is formulated for different values of m in such a manner that RS − T is positive. This criterion decides the number of columns of the matrix.
The matrix now is non-symmetrical as the number of rows would be greater than the number of columns. The matrix is split into a number of sub-symmetrical matrices by matching the number of rows with the columns.
The procedure has been illustrated with an example where RS − T is positive and RS − 2T, RS − 3T, etc. are negative. With only RS − T, RS and RS + T, the matrix has been formulated. Hence, the modified rule set should be split into 3 X 3 matrices based on the modified rule set. The actual output matrix is accepted as input. The error, which is the difference between the output and modified rule matrix, is computed. An activation function is used to iteratively train the modified rule set till the error is non-negative. The final value is the optimal rule set.
For computing the trained matrix for antivirus check, the different sub-factors and their corresponding weights are shown in Table 1.
The different Linguistic variables with their ranges and corresponding mid-values are indicated in Table 2.
A typical rule is shown below.
If (Phishing Check is Excellent) and (Spyware Check is Good) and (Malware check is Average) and (Trojan Check is Fair) and (Rootkit scan is Poor) Then (Antivirus software is Average).
The entire rule set comprises of 125 rules. Few sample rules from the entire rule set is shown below in Table 3.
The rule strength of the rule If (Phishing Check is Excellent) and (Spyware Check is Good) and (Malware check is Average) and (Trojan Check is Fair) and (Rootkit scan is Poor), then (Antivirus software is Average) is 36.2. The Rule strengths of the above rule base would be computed as indicated in Table 4.
Using the expression, the threshold has been computed as 62.02. Based on this threshold modified rule set is shown in Table 5.
Hence RS − T is only non-negative and the rest of RS − mT is negative. From the above discussion 14 (3 X 3 Symmetrical) sub-matrices have been formulated. The last row of the 14th matrix would be zeroes. The modified rule set is trained using the activation function \(E = \left( {\frac{1}{{1 - e^{ - T} }}} \right)\) as shown in Table 6.
The modified rule matrix is LRM. The iteration matrix is LRM1. Given Output matrix is OM. The error is (OM − LRM1). The iteration is carried out till all elements of OM − LRM1 becomes negative.
The algorithm has been discussed below
-
Form the fuzzy rule set
-
Compute the rule strength and Threshold
-
Form the Learning rule matrix (LRM) with selected rules (RS > T)
-
Form modified rule matrix (RS − mT, RS, RS + mT)
-
Accept the output matrix (OM)
-
Start the iteration by computing Error E = (OM − LRM)
-
Use activation function \(E = \left( {\frac{1}{{1 - e^{ - T} }}} \right)\) to minimise the error.
-
LRM1 = LRM + E
-
If (LRM1 <= OM) then
-
Final output O1 = LRM1
Else
-
L1: LRM1 = LRM + E
-
-
If (LRM1 > OM)
-
O1 = LRM1
Else
-
(LRM1 = LRM1 + E)
-
-
Repeat iteration until LRM1 value is > OM.
-
Goto L1.
-
Compute RMSE, RRSE, RAE and MAE for final value of O1.
-
End the procedure.
4 Simulation Results
The algorithm was simulated on Java platform with several IT-related issues like Antivirus, Data backup strategies, hardware maintenance. Simulations were carried out on fuzzy rule sets with about 500 rules.
-
Root Mean Square Error (RMSE)
-
Root Relative Square Error (RRSE)
-
Relative Absolute Error (RAE)
-
Mean Absolute Error (MAE).
4.1 Root Mean Square Error (RMSE)
It is the square root of the mean of the squared difference between the Output Matrix (OM) and Trained Learning Rule Matrix (LRM1). It is given by the equation
where n is the number of sample values of the data set under consideration.
4.2 Root Relative Square Error (RRSE)
It is defined as the normalised version of the total squared error to the total squared error of the prediction made.
where A~ is the average of the n sample values.
4.3 Relative Absolute Error (RAE)
It is defined as the total absolute error normalised to the total absolute error of the prediction made.
where A ~ is the average of the n sample values.
4.4 Mean Absolute Error (MAE)
It is defined as the error difference for the n samples considered.
The results of the fuzzy-based approach proposed in this work have been compared with the four conventional machine learning algorithms listed below:
-
1.
Naïve Bayes Classifier
-
2.
Support Vector Method (SVM)
-
3.
Regression Analysis
-
4.
Artificial Neural Networks (ANN).
4.5 Comparison with Naïve Bayes Classifier
The results of the proposed fuzzy-based approach are compared with Naïve Bayes Classifier. The reduction in Error for the parameters RMSE, RRSE, RAE and MAE when compared with the Naives Classifier method are 21, 17, 18 and 27% respectively as indicated in Table 7.
4.6 Comparison with Support Vector Method (SVM)
The reduction in Error for the parameters RMSE, RRSE, RAE and MAE for the proposed work when compared with the Support Vector Method are 32, 10, 17 and 17% respectively as indicated in Table 8.
4.7 Comparison with Regression Analysis
The reduction in Error for the parameters RMSE, RRSE, RAE and MAE for the proposed fuzzy-based approach when compared with the Regression Analysis Method are 13, 4, 7 and 23% respectively as indicated in Table 9.
4.8 Comparison with Artificial Neural Networks (ANN)
The reduction in Error for the parameters RMSE, RRSE, RAE and MAE for the proposed fuzzy-based approach when compared with the Artificial Neural Networks method are 34, 21, 17 and 32% respectively as indicated in Table 10.
4.9 Summary
The reduction in error for the proposed work compared to conventional approaches is shown in Table 11.
The average reduction by using the fuzzy learning method for RMSE, RRSE, RAE and MAE are 25, 13, 15 and 25% respectively as indicated in Table 11. The average of these values would be 20%. Hence, the fuzzy learning method reduces error compared to conventional approaches by 20%.
5 Conclusion
A fuzzy-based learning approach has been proposed in this work. The fuzzy learning approach reduces the error by 20%. This approach computes the rule strength and chooses rules with rule strengths greater than the threshold limit for effective learning. The learning approach uses a predictive range for the linguistic variable, which is chosen before commencing the training process.
6 Future Work
The work can be extended by making this training process fully unsupervised by generating rule strengths using random function and then completing the entire process. The random unsupervised learning approach could later be implemented using Raspberry Pi as hardware for several applications like healthcare analytics, stock market, etc.
References
Taheri, S., Mammadov, M., & Bagirov, A. M. (2011). Improving Naïve Bayes classifier using conditional probabilities. In 9th Australian Data Mining Conference, pp. 63–69.
Liu, H., Yin, X., & Han, J. (2005). An efficient multi-relational Naïve Bayesian classifier based on semantic relationship graph. In MRDM, pp. 39–49.
Ziddi, N. A., Cerquides, J., Carman, M. J., & Webb, G. I. (2013). Alleviating Naïve bayes attribute independence assumption by attribute weighting. Journal of Machine Learning Research, 1947–1988.
George, A., Poornachandran, P., & Kaimal, M. R. (2012). adsvm: Pre-processor plug-in using support vector machine algorithm for Snort. ACM 978-1-4503-1822-8/12/08, pp. 179–184.
Milchevski, A., Rozza, A., & Taskovski, D. (2015). Multimodal affective analysis combining regularised linear regression and boosted regression trees. ACM 978-1-4503-3743-4/15/10, pp. 33-39, AVEC’15.
Zikos, D., & Vandeliwala, I. (2015). Using binary logistic regression coefficients for the dynamic quantification of comorbidities. ACM 978-1-4503-3452-5/15/07 PETRA’15, pp. 12–14.
Plagge, M. (2013). Using artificial neural networks to predict first-year traditional students second year retention rates. ACM 978-1-4503-1901-0/13/14, ACMSE’13, pp. 13–18.
Trujillo, L., Martinez, Y., & Melin, P. (2011). How many neurons? A genetic programming answer. ACM 978-1-4503-0690-4-11-07/GEECO’11, pp. 175–176.
Gunasekara, M., Dharmaratne, A., & Sandaruwan, D. (2014). A feature-point based approach for pose variant face recognition. ACM 978-1-4503-2765-7/14/08, VINCI’14, pp. 242–243.
Kalles, D., Verykios, V. S., Feretzakis, G., & Papagelis, A. (2016). Data set operations to hide decision tree rules. ACM 978-1-4503-4304-6/16/08, PrAISe’16, pp. 12–20.
Ram, P., & Gray, A. G. (2011). Density estimation trees. ACM 978-1-4503-0813-7/11/08, pp. 627–635, KDD’11.
Senior, A., Heigold, G., & Ranzato, M. A. An empirical study of learning rates in deep neural networks for speech recognition (pp. 12–17). IEEE.
Wang, X. Method of steepest descent and its applications (pp. 1–3). IEEE.
Li, Y. Deep reinforcement learning: An overview. Thesis work: Arxiv, pp. 12–42.
Subramanian, D. V., Dr. & Kumar, K. P., Dr. (2016). Fuzzy based modelling for an effective it security policy management. In SAI Computing Conference 2016, 978-1-4673-8460-5/16, pp. 1–9. IEEE.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2019 Springer Nature Singapore Pte Ltd.
About this paper
Cite this paper
Pradheep Kumar, K. (2019). Fuzzy-Based Machine Learning Algorithm for Intelligent Systems. In: Balas, V., Sharma, N., Chakrabarti, A. (eds) Data Management, Analytics and Innovation. Advances in Intelligent Systems and Computing, vol 808. Springer, Singapore. https://doi.org/10.1007/978-981-13-1402-5_25
Download citation
DOI: https://doi.org/10.1007/978-981-13-1402-5_25
Published:
Publisher Name: Springer, Singapore
Print ISBN: 978-981-13-1401-8
Online ISBN: 978-981-13-1402-5
eBook Packages: EngineeringEngineering (R0)