Abstract
In this paper we examine several methods for improving the performance of MLP neural networks by eliminating the influence of outliers and compare them experimentally on several classification and regression tasks. The examined method include: pre-training outlier elimination, use of different error measures during network training, replacing the weighted input sum with weighted median in the neuron input functions and various combinations of them. We show how these methods influence the network prediction. Based on the experimental results, we also present a novel hybrid approach improving the network performance.
Access provided by Autonomous University of Puebla. Download to read the full chapter text
Chapter PDF
Similar content being viewed by others
Keywords
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.
References
Chen, D., Jain, R.: A robust backpropagation learning algorithm for function approximation. IEEE Transactions on Neural Networks 5(3), 467–479 (1994)
Chuang, C.C., Su, S.F., Hsiao, C.C.: The annealing robust backpropagation (arbp) learning algorithm. IEEE Transactions on Neural Networks 11(5), 1067–1077 (2000)
El-Melegy, M.T., Essai, M.H., Ali, A.A.: Robust training of artificial feedforward neural networks. In: Hassanien, A.-E., Abraham, A., Vasilakos, A.V., Pedrycz, W. (eds.) Foundations of Computational, Intelligence Volume 1. SCI, vol. 201, pp. 217–242. Springer, Heidelberg (2009), http://dx.doi.org/10.1007/978-3-642-01082-8_9
El-Melegy, M.: Random sampler m-estimator algorithm for robust function approximation via feed-forward neural networks. In: The 2011 International Joint Conference on Neural Networks (IJCNN), pp. 3134–3140 (2011)
El-Melegy, M.: Ransac algorithm with sequential probability ratio test for robust training of feed-forward neural networks. In: In: The 2011 International Joint Conference on Neural Networks (IJCNN), pp. 3256–3263 (2011)
El-Melegy, M.: Random sampler m-estimator algorithm with sequential probability ratio test for robust function approximation via feed-forward neural networks. IEEE Transactions on Neural Networks and Learning Systems 24(7), 1074–1085 (2013)
Golak, S., Burchart-Korol, D., Czaplicka-Kolarz, K., Wieczorek, T.: Application of neural network for the prediction of eco-efficiency. In: Liu, D., Zhang, H., Polycarpou, M., Alippi, C., He, H. (eds.) ISNN 2011, Part III. LNCS, vol. 6677, pp. 380–387. Springer, Heidelberg (2011), http://dx.doi.org/10.1007/978-3-642-21111-9_43
Guillen, A.: Applying mutual information for prototype or instance selection in regression problems. In: ESANN 2009 (2009)
Hampel, F.R., Ronchetti, E.M., Rousseeuw, P.J., Stahel, W.A.: Robust Statistics: The Approach Based on Influence Functions (Wiley Series in Probability and Statistics), revisied edn. Wiley Interscience, New York (April 2005), http://www.worldcat.org/isbn/0471735779
Hart, P.: The condensed nearest neighbor rule (corresp.). IEEE Transactions on Information Theory 14(3), 515–516 (1968)
Huber, P.J.: Robust Statistics. Wiley Series in Probability and Statistics. Wiley-Interscience (1981), http://www.worldcat.org/isbn/0471418056
Ben-Gal, I.: Outlier detection. Kluwer Academic Publishers (2005)
Kordos, M., Białka, S., Blachnik, M.: Instance Selection in Logical Rule Extraction for Regression Problems. In: Rutkowski, L., Korytkowski, M., Scherer, R., Tadeusiewicz, R., Zadeh, L.A., Zurada, J.M. (eds.) ICAISC 2013, Part II. LNCS, vol. 7895, pp. 167–175. Springer, Heidelberg (2013), http://dx.doi.org/10.1007/978-3-642-38610-7_16
Kordos, M., Blachnik, M., Wieczorek, T.: Temperature prediction in electric arc furnace with neural network tree. In: Honkela, T., Duch, W., Girolami, M., Kaski, S. (eds.) ICANN 2011, Part II. LNCS, vol. 6792, pp. 71–78. Springer, Heidelberg (2011), http://dx.doi.org/10.1007/978-3-642-21738-8_10
Kordos, M., Duch, W.: Variable step search algorithm for feedforward networks. Neurocomputing 71(13–15), 2470–2480 (2008), http://www.sciencedirect.com/science/article/pii/S0925231208002099
Kordos, M.: source code and datasets used in the experiments (2013), http://www.ath.bielsko.pl/~mkordos/tpnc2013.html
Liano, K.: Robust error measure for supervised neural network learning with outliers. IEEE Transactions on Neural Networks 7(1), 246–250 (1996)
Merz, C., Murphy, P.: Uci repository of machine learning databases (2013), http://www.ics.uci.edu/mlearn/MLRepository.html
Olive, D.J., Hawkins, D.M.: Robustifying robust estimators (2007)
Perní-Espinoza, A.V., Ordieres-Meré, J.B., de Poisón, F.J.M., González-Marcos, A.: Tao-robust backpropagation learning algorithm. Neural Networks 18(2), 191–204 (2005), http://www.sciencedirect.com/science/article/pii/S0893608004002345
Riedmiller, M., Braun, H.: A direct adaptive method for faster backpropagation learning: the rprop algorithm. IEEE International Conference on Neural Networks 1, 586–591 (1993)
Rusiecki, A.: Fault tolerant feedforward neural network with median neuron input function. Electronics Letters 41(10), 603–605 (2005)
Rusiecki, A.: Robust LTS backpropagation learning algorithm. In: Sandoval, F., Prieto, A.G., Cabestany, J., Graña, M. (eds.) IWANN 2007. LNCS, vol. 4507, pp. 102–109. Springer, Heidelberg (2007), http://dx.doi.org/10.1007/978-3-540-73007-1_13
Rusiecki, A.: Robust learning algorithm based on iterative least median of squares. Neural Processing Letters 36(2), 145–160 (2012), http://dx.doi.org/10.1007/s11063-012-9227-z
Salvador, G., Derrac, J., Ramon, C.: Prototype selection for nearest neighbor classification: Taxonomy and empirical study. IEEE Transactions on Pattern Analysis and Machine Intelligence 34, 417–435 (2012)
Tolvi, J.: Genetic algorithms for outlier detection and variable selection in linear regression models. Soft Computing 8, 527–533 (2004)
Wilson, D.L.: Asymptotic properties of nearest neighbor rules using edited data. IEEE Transactions on SMC Systems, Man and Cybernetics 2(3), 408–421 (1972)
Zhang, J.: Intelligent selection of instances for prediction functions in lazy learning algorithms. Artifcial Intelligence Review 11, 175–191 (1997)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2013 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Kordos, M., Rusiecki, A. (2013). Improving MLP Neural Network Performance by Noise Reduction. In: Dediu, AH., Martín-Vide, C., Truthe, B., Vega-Rodríguez, M.A. (eds) Theory and Practice of Natural Computing. TPNC 2013. Lecture Notes in Computer Science, vol 8273. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-45008-2_11
Download citation
DOI: https://doi.org/10.1007/978-3-642-45008-2_11
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-642-45007-5
Online ISBN: 978-3-642-45008-2
eBook Packages: Computer ScienceComputer Science (R0)