Mathematical aspects of neural network learning through measure dynamics

Joan Bruna (Courant Institute and Center for Data Science, NYU)

04-Nov-2020, 18:00-19:00 (3 years ago)

Abstract: High-dimensional learning remains an outstanding phenomena where experimental evidence outpaces our current mathematical understanding, mostly due to the recent empirical successes of Deep Learning algorithms. Neural Networks provide a rich yet intricate class of functions with statistical abilities to break the curse of dimensionality, and where physical priors can be tightly integrated into the architecture to improve sample efficiency. Despite these advantages, an outstanding theoretical challenge in these models is computational, ie providing an analysis that explains successful optimization and generalization in the face of existing worst-case computational hardness results.

In this talk, I will focus on the framework that lifts parameter optimization to an appropriate measure space. I will cover existing results that guarantee global convergence of the resulting Wasserstein gradient flows, as well as recent results that study typical fluctuations of the dynamics around their mean field evolution. We will also discuss extensions of this framework beyond vanilla supervised learning, to account for symmetries in the function, as well as for competitive optimization.

data structures and algorithmsmachine learningmathematical physicsinformation theoryoptimization and controldata analysis, statistics and probability

Audience: researchers in the topic

( video )


Mathematics, Physics and Machine Learning (IST, Lisbon)

Series comments: To receive the series announcements, please register in:
mpml.tecnico.ulisboa.pt
mpml.tecnico.ulisboa.pt/registration
Zoom link: videoconf-colibri.zoom.us/j/91599759679

Organizers: Mário Figueiredo, Tiago Domingos, Francisco Melo, Jose Mourao*, Cláudia Nunes, Yasser Omar, Pedro Alexandre Santos, João Seixas, Cláudia Soares, João Xavier
*contact for this listing

Export talk to