Lorenz, Nicole : Optimization problems in statistical learning: duality and optimality conditions

Author(s):

Lorenz, Nicole

Title:

Optimization problems in statistical learning: duality and optimality conditions

Electronic source:

application/pdf

Preprint series:

Technische Universität Chemnitz, Fakultät für Mathematik (Germany). Preprint 8, 2008

Mathematics Subject Classification:

47A52			[ Ill-posed problems, regularization ]
90C25			[ Convex programming ]
49N15			[ Duality theory ]

Abstract:

Regularization methods are techniques for learning functions from given data. We consider regularization problems that consist of a loss and a regularization term with the aim of selecting a prediction function with a finite representation which minimizes the error of prediction, whereas the regulizer avoids overfitting. In general, these are convex optimization problems, for which we construct conjugate duals, by means of which we derive necessary and sufficient optimality conditions. In the second part of the paper we consider some particular cases of the general problem, namely the Support Vector Machines problem and Support Vector Regression problem. Our approach allows to avoid the use of pseudo-inverse matrices in case of finitely positive semidefinite kernel functions.

Keywords:

machine learning, regularization, convex analysis, duality

Language:

English

Publication time:

1 / 2009