Geometrical Aspects of Manifold Learning

One of the most common assumptions in machine learning is that the data lie near a non-linear low dimensional manifold in the ambient data space. In this setting, machine learning methods should be developed in such a way to respect the underlying geometric structure implied by the manifold. When the data lie on such a curved space, then the shortest path between two points is actually a curve on the manifold and its length is the most natural distance measure.

Hence, a suitable way to measure such distances is to model the curved space as a Riemannian manifold. The reason is that by learning a Riemannian metric in the ambient space where the data lives, we transform this Euclidean space into a Riemannian manifold, which enables us to compute the shortest path.

In this thesis, we study how statistical models can be generalized on these curved spaces, also we develop methods to learn the Riemannian manifold directly from the data, as well as an efficient method to compute shortest paths. Usually, statistical models depend on the way we measure distances between points.

Therefore, we are able to develop statistical models that respect the underlying geometrical structure of the data, by replacing the way we measure distances. Under this consideration the Gaussian distribution can be transformed into a flexible locally adaptive normal distribution. However, learning a Riemannian metric which captures the underlying geometry of the given data is a challenging task.

Here, we develop a parametric and non-parametric model to learn such a metric. In the non-parametric case we provide a systematic way to estimate the hyper-parameters of the metric that maximize the data likelihood. Regarding the parametric case, we show that the latent space of the deep generative models can be considered as a stochastic Riemannian manifold, and thus, we are able to learn the corresponding expected Riemannian metric directly from the data.

Finally, the shortest path on these manifolds is given by solving a system of 2nd order non-linear ordinary differential equations. Since data driven Riemannain manifolds often entail high curvature and an unstable Riemannian metric, a specialized fast and robust solver that is able to solve the system is a necessity.

Therefore, we provide such a solver inspired by probabilistic numerics.