Conference paper
Multiplicative updates for the LASSO
Cognitive Systems, Department of Informatics and Mathematical Modeling, Technical University of Denmark1
Department of Informatics and Mathematical Modeling, Technical University of Denmark2
Image Analysis and Computer Graphics, Department of Informatics and Mathematical Modeling, Technical University of Denmark3
Multiplicative updates have proven useful for non-negativity constrained optimization. Presently, we demonstrate how multiplicative updates also can be used for unconstrained optimization. This is for instance useful when estimating the least absolute shrinkage and selection operator (LASSO), i.e. least squares minimization with $L_1$-norm regularization, since the multiplicative updates (MU) can efficiently exploit the structure of the problem traditionally solved using quadratic programming (QP).
We derive an algorithm based on MU for the LASSO and compare the performance to Matlabs standard QP solver as well as the basis pursuit denoising algorithm (BP) which can be obtained from www.sparselab.stanford.edu. The algorithms were tested on three benchmark bio-informatic datasets: A small scale data set where the number of observations is larger than the number of variables estimated ($M
| Language: | English |
|---|---|
| Publisher: | IEEE |
| Year: | 2007 |
| Pages: | 33-38 |
| Proceedings: | 2007 17th IEEE Workshop on Machine Learning for Signal Processing |
| Journal subtitle: | Mlsp2007 |
| ISBN: | 1424415659 , 9781424415656 , 1424415667 and 9781424415663 |
| ISSN: | 21610363 and 15512541 |
| Types: | Conference paper |
| DOI: | 10.1109/MLSP.2007.4414278 |
| ORCIDs: | Mørup, Morten and Clemmensen, Line Katrine Harder |
Basis Pursuit Denoising (BPD) Least Absolute Shrinkage and Selection Operator (LASSO) Multiplicative Updates
Benchmark testing Bridges Computer languages Constraint optimization L<sub>1</sub>-norm regularization LASSO Large-scale systems Least squares approximation Mathematical model Noise reduction Pursuit algorithms Quadratic programming basis pursuit denoising algorithm benchmark bio-informatic datasets convergence of numerical methods large scale microarray data sets least absolute shrinkage and selection operator least mean squares methods least squares minimization multiplicative updates optimisation quadratic programming small scale data set unconstrained optimization
