Conference paper
Attention: A Machine Learning Perspective
We review a statistical machine learning model of top-down task driven attention based on the notion of ‘gist’. In this framework we consider the task to be represented as a classification problem with two sets of features — a gist of coarse grained global features and a larger set of low-level local features.
Attention is modeled as the choice process over the low-level features given the gist. The model takes its departure in a classical information theoretic framework for experimental design. This approach requires the evaluation over marginalized and conditional distributions. By implementing the classifier within a Gaussian Discrete mixture it is straightforward to marginalize and condition, hence, we obtained a relatively simple expression for the feature dependent information gain — the top-down saliency.
As the top-down attention mechanism is modeled as a simple classification problem, we can evaluate the strategy simply by estimating error rates on a test data set. We illustrate the attention mechanism on a simple simulated visual domain in which the choice is over nine patches in which a binary pattern has to be classified.
The performance of the classifier equipped with the attention mechanism is almost as good as one that has access to all low-level features and clearly improving over a simple ‘random attention’ alternative.
Language: | English |
---|---|
Publisher: | IEEE |
Year: | 2012 |
Pages: | 1-6 |
Proceedings: | 3rd International Workshop on Cognitive Information Processing (CIP) |
ISBN: | 1467318779 , 9781467318778 , 1467318760 , 1467318787 , 9781467318761 and 9781467318785 |
Types: | Conference paper |
DOI: | 10.1109/CIP.2012.6232894 |
ORCIDs: | Hansen, Lars Kai |
Computational modeling Conferences Error analysis Gaussian Discrete mixture Information processing Machine learning Training Visualization binary pattern classical information theoretic framework classification classification problem coarse grained global features feature dependent information gain gist information theory knowledge representation learning (artificial intelligence) machine learning perspective random attention alternative simulated visual domain statistical analysis statistical machine learning model task representation top-down attention mechanism top-down task driven attention