BEGIN:VCALENDAR
VERSION:2.0
PRODID:researchseminars.org
CALSCALE:GREGORIAN
X-WR-CALNAME:researchseminars.org
BEGIN:VEVENT
SUMMARY:Bruno Loureiro (École Polytechnique Fédérale de Lausanne (EPFL)
 )
DTSTART:20221215T170000Z
DTEND:20221215T180000Z
DTSTAMP:20260423T003242Z
UID:MPML/88
DESCRIPTION:Title: <a href="https://researchseminars.org/talk/MPML/88/">Ph
 ase diagram of Stochastic Gradient Descent in high-dimensional two-layer n
 eural networks</a>\nby Bruno Loureiro (École Polytechnique Fédérale de 
 Lausanne (EPFL)) as part of Mathematics\, Physics and Machine Learning (IS
 T\, Lisbon)\n\n\nAbstract\nDespite the non-convex optimization landscape\,
  over-parametrized shallow networks are able to achieve global convergence
  under gradient descent. The picture can be radically different for narrow
  networks\, which tend to get stuck in badly-generalizing local minima. He
 re we investigate the cross-over between these two regimes in the high-dim
 ensional setting\, and in particular investigate the connection between th
 e so-called mean-field/hydrodynamic regime and the seminal approach of Saa
 d & Solla. Focusing on the case of Gaussian data\, we study the interplay 
 between the learning rate\, the time scale\, and the number of hidden unit
 s in the high-dimensional dynamics of stochastic gradient descent (SGD). O
 ur work builds on a deterministic description of SGD in high-dimensions fr
 om statistical physics\, which we extend and for which we provide rigorous
  convergence rates.\n
LOCATION:https://researchseminars.org/talk/MPML/88/
END:VEVENT
END:VCALENDAR