Conference paper
PASS-GP: Predictive active set selection for Gaussian processes
We propose a new approximation method for Gaussian process (GP) learning for large data sets that combines inline active set selection with hyperparameter optimization. The predictive probability of the label is used for ranking the data points. We use the leave-one-out predictive probability available in GPs to make a common ranking for both active and inactive points, allowing points to be removed again from the active set.
This is important for keeping the complexity down and at the same time focusing on points close to the decision boundary. We lend both theoretical and empirical support to the active set selection strategy and marginal likelihood optimization on the active set. We make extensive tests on the USPS and MNIST digit classification databases with and without incorporating invariances, demonstrating that we can get state-of-the-art results (e.g.0.86% error on MNIST) with reasonable time complexity.
Language: | English |
---|---|
Publisher: | IEEE |
Year: | 2010 |
Pages: | 148-153 |
Proceedings: | 2010 IEEE International Workshop on Machine Learning for Signal Processing |
ISBN: | 1424478758 , 1424478766 , 1424478774 , 9781424478750 , 9781424478767 and 9781424478774 |
ISSN: | 21610363 and 15512541 |
Types: | Conference paper |
DOI: | 10.1109/MLSP.2010.5589264 |
ORCIDs: | Winther, Ole |
Approximation methods Cavity resonators GP Gaussian processes Optimization PASS-GP Prediction algorithms Support vector machines Training hyperparameter optimization marginal likelihood optimization optimisation predictive active set selection predictive probability set theory time complexity