Approximating methods for intractable probabilistic models: Applications in neuroscience

This thesis investigates various methods for carrying out approximate inference in intractable probabilistic models. By capturing the relationships between random variables, the framework of graphical models hints at which sets of random variables pose a problem to the inferential step. The approximating techniques used in this thesis originate from the field of statistical physics which for decades has been facing the same type of intractable computations when analyzing large systems of interacting variables e.g. magnetic spin systems.

In general, these approximating techniques are known as mean field methods. The thesis provides a brief introduction to the basic methodology of learning and inference in graphical models as well as a short review of the various types of mean field approximations which recently have been shown to be efficient for carrying out approximate inference in intractable probabilistic models.

Starting from the naive mean field approximation we derive for the independent component analysis (ICA) model with instantaneous mixing general expressions for the posterior quantities needed to perform learning by Expectation-Maximization (EM). Furthermore, we explore the feasibility of going beyond the naive mean field approximation for this model.

In fact, it turns out that the overcomplete ICA problem can be solved using a simple linear response correction to the mean sufficient statistics obtained by naive mean field approximation. In addition, we apply to the ICA problem an adaptive version of the Thouless, Anderson and Palmer (TAP) mean field approach which is due to Opper and Winther.

To illustrate the methodology on a real world problem, an explorative analysis of a functional magnetic resonance imaging (fMRI) dataset from a visual activation study is carried out using ICA with binary sources. It is shown this approach, which is computationally efficient, infers reasonable brain activation functions.

Finally, we outline various ways of carrying out approximate message passing in probabilistic models for which marginalization over some of the clique variables is intractable.