L1-Norm Support Vector Regression in Primal Based on Huber Loss Function

Abstract

Support vector regression (SVR) method becomes the state of the art machine learning method for data regression due to its excellent generalization performance on many real-world problems. It is well-known that the standard SVR determines the regressor using a predefined epsilon tube around the data points in which the points lying outside the tube contribute to the errors whereas the inside points are simply ignored. To measure the data misfit as stated, the epsilon insensitive function is introduced as a loss function. In comparison with the popular quadratic loss function, it is robust but only continuous and therefore numerical minimization is difficult. However, as a combination of robust treatment to large errors and showing quadratic treatment to small errors, Huber function is used in the literature to measure the data misfit having the smooth property that it is differentiable everywhere. In this study, we propose a novel robust Huber SVR (HSVR) formulation in primal where the regressor is made as flat as possible by considering the regularization term in L1-norm. Since the regularization term is non-smooth and therefore by replacing it with smooth approximation functions, new problem formulation is obtained which is solved then by functional iterative method. Tests were performed on few synthetic and real world data sets whose results confirm the suitability and effectiveness of the proposed robust model.