Abstract
In this paper, we propose an extended version of the principle of minimizing empirical risk (ER) based on the use of averaging aggregating functions (AAF) for calculating the ER instead of the arithmetic mean. This is expedient if the distribution of losses has outliers and hence risk assessments are biased. Therefore, a robust estimate of the average risk should be used for optimization of the parameters. Such estimates can be constructed by using AAF that are solutions of the problem of minimizing the penalty function for deviating from the mean value. We also propose an iterative reweighting scheme for the numerical solution of the ER minimization problem. We give examples of constructing a robust procedure for estimating parameters in a linear regression problem and a linear separation problem for two classes based on the use of an averaging aggregating function that replaces the α-quantile.
Article PDF
Similar content being viewed by others
Avoid common mistakes on your manuscript.
References
G. Beliakov, H. Sola, and T. Calvo, A Practical Guide to Averaging Functions, Springer-Verlag (2016).
T. Calvo and G. Beliakov, “Aggregation functions based on penalties,” Fuzzy Sets Syst., 161, No. 10, 1420–1436 (2010).
M. Grabich, J.-L. Marichal, E. Pap, “Aggregation Functions,” in: Encycl. Math. Appl., 127, Cambridge Univ. Press (2009).
P. J. Huber, Robust Statistics, Wiley, New York (1981).
D. P. Knigma and Ba J., Adam: A method for stochastic optimization, e-print arXiv1412.6980.
R. Koenker, Quantile Regression, Cambridge Univ. Press, New York (2005).
Y. Ma, L. Li, X. Huang, and S. Wang, “Robust support vector machine using least median loss penalty,” IFAC Proc. Vols. 18th IFAC World Congr., 44, No. 1, 11208–11213 (2011).
R. Mesiar, M. Komornikova, A. Kolesarova, and T. Calvo, “Aggregation functions: A revision,” in: Fuzzy Sets and Their Extensions: Representation, Aggregation and Models (H. Bustince, F. Herrera, and J. Montero, eds.), Springer-Verlag, Berlin–Heidelberg (2008).
W. Newey and J. Powell, “Asymmetric least square estimation and testing,” Econometrica, 55, No. 4, 819–847 (1987).
P. J. Rousseeuw, “Least median of square regression,” J. Am. Stat. Assoc., 79, 871–880 (1984).
P. J. Rousseeuw and A. M. Leroy, Robust Regression and Outlier Detection, Wiley, New York (1987).
M. Schmidt, N. Le Roux, and F. Bach, Minimizing finite sums with the stochastic average gradient, e-print arXiv1309.2388.
Z. M. Shibzukhov, “Correct aggregate operations with algorithms,” Pattern Recogn. Image Anal., 24, No. 3, 377–382 (2014).
Z. M. Shibzukhov, “Aggregating correct operations on algorithms,” Dokl. Ross. Akad. Nauk, 462, No. 6, 649–652 (2015).
V. Vapnik, The Nature of Statistical Learning Theory, Springer-Verlag (2000).
V. J. Yohai, “High breakdown-point and high efficiency robust estimates for regression,” Ann. Stat., 15, 642–656 (1987).
Author information
Authors and Affiliations
Corresponding author
Additional information
Translated from Itogi Nauki i Tekhniki, Seriya Sovremennaya Matematika i Ee Prilozheniya. Tematicheskie Obzory, Vol. 154, Proceedings of the International Conference “Actual Problems of Applied Mathematics and Physics,” Kabardino-Balkaria, Nalchik, May 17–21, 2017, 2018.
Rights and permissions
About this article
Cite this article
Shibzukhov, Z.M. Principle of Minimizing Empirical Risk and Averaging Aggregate Functions. J Math Sci 253, 583–598 (2021). https://doi.org/10.1007/s10958-021-05256-y
Published:
Issue Date:
DOI: https://doi.org/10.1007/s10958-021-05256-y