Optimal transport for non-convex optimization in machine learning

Abstract: Function approximation is a classical task in both classical numerical analysis and machine learning. Elements of the recently popular class of neural networks depend nonlinearly on a finite set of parameters. This nonlinearity gives the function class immense approximation power, but causes parameter optimization problems to be non-convex. In fact, generically the set of global minimizers is a (curved) manifold of positive dimension. Despite this non-convexity, gradient descent based algorithms empirically find good minimizers in many applications. We discuss this surprising success of simple optimization algorithms from the perspective of Wasserstein gradient flows in the case of shallow neural networks in the infinite parameter limit.

mathematical physicsanalysis of PDEsclassical analysis and ODEsdifferential geometryfunctional analysismetric geometrynumerical analysisoptimization and control

Audience: researchers in the topic

( paper )

---

Online Seminar "Geometric Analysis"

Series comments: We discuss recent trends related to geometric analysis in a broad sense. The general idea is to solve geometric problems by means of advanced tools in analysis. We will include a wide range of topics such as geometric flows, curvature functionals, discrete differential geometry, and numerical simulation.

Registration and links to videos available at blatt.sbg.ac.at/onlineseminar.php

---