Abstract: In this paper a finite Newton iterative method of solution for solving the implicit Lagrangian Support Vector Regression (SVR) formulation has been proposed. Unlike solving a quadratic programming problem for the case of the standard SVR the solution of the proposed method is obtained by solving a system of linear equations at each iteration of the algorithm. For the linear or nonlinear SVR the finite termination of the proposed method has been established. The algorithm converges from any starting point and does not need any optimization packages. Experiments have been performed on a number of interesting synthetic and real-world datasets.…The results obtained by the proposed method are compared with the standard SVR. Similar or better generalization performance of the proposed method clearly demonstrates its effectiveness and applicability.
Show more
Keywords: Implicit Lagrangian support vector machines, Newton method, Support vector regression
Abstract: A new smoothing approach for the implicit Lagrangian twin support vector regression is proposed in this paper. Our formulation leads to solving a pair of unconstrained quadratic programming problems of smaller size than in the classical support vector regression and their solutions are obtained using Newton-Armijo algorithm. This approach has the advantage that a system of linear equations is solved in each iteration of the algorithm. Numerical experiments on several synthetic and real-world datasets are performed and, their results and training time are compared with both the support vector regression and twin support vector regression to verify the effectiveness of…the proposed method.
Show more
Keywords: Implicit Lagrangian support vector machines, nonparallel planes, support vector regression, smoothing technique, twin support vector regression
Abstract: Construction of robust regression learning models to fit training data corrupted by noise is an important and challenging research problem in machine learning. It is well-known that loss functions play an important role in reducing the effect of noise present in the input data. With the objective of obtaining a robust regression model, motivated by the link between the pinball loss and quantile regression, a novel squared pinball loss twin support vector machine for regression (SPTSVR) is proposed in this work. Further with the introduction of a regularization term, our proposed model solves a pair of strongly convex minimization problems…having unique solutions by simple functional iterative method. Experiments were performed on synthetic datasets with different noise models and on real world datasets and those results were compared with support vector regression (SVR), least squares support vector regression (LS-SVR) and twin support vector regression (TSVR) methods. The comparative results clearly show that our proposed SPTSVR is an effective and a useful addition in the machine learning literature.
Show more
Keywords: Kernel methods, pinball loss, robust support vector regression