BEGIN:VCALENDAR
VERSION:2.0
PRODID:researchseminars.org
CALSCALE:GREGORIAN
X-WR-CALNAME:researchseminars.org
BEGIN:VEVENT
SUMMARY:Mikhail Belkin (Halicioğlu Data Science Institute\, University of
  California San Diego)
DTSTART:20210428T170000Z
DTEND:20210428T180000Z
DTSTAMP:20260423T003249Z
UID:MPML/42
DESCRIPTION:Title: <a href="https://researchseminars.org/talk/MPML/42/">Tw
 o mathematical lessons of deep learning</a>\nby Mikhail Belkin (Halicioğl
 u Data Science Institute\, University of California San Diego) as part of 
 Mathematics\, Physics and Machine Learning (IST\, Lisbon)\n\n\nAbstract\nR
 ecent empirical successes of deep learning have exposed significant gaps i
 n our fundamental understanding of learning and optimization mechanisms. M
 odern best practices for model selection are in direct contradiction to th
 e methodologies suggested by classical analyses. Similarly\, the efficienc
 y of SGD-based local methods used in training modern models\, appeared at 
 odds with the standard intuitions on optimization.\n\nFirst\, I will prese
 nt evidence\, empirical and mathematical\, that necessitates revisiting cl
 assical notions\, such as over-fitting. I will continue to discuss the eme
 rging understanding of generalization\, and\, in particular\, the "double 
 descent" risk curve\, which extends the classical U-shaped generalization 
 curve beyond the point of interpolation.\n\nSecond\, I will discuss why th
 e landscapes of over-parameterized neural networks are generically never c
 onvex\, even locally. Instead\, as I will argue\, they satisfy the Polyak-
 Lojasiewicz condition across most of the parameter space instead\, which a
 llows SGD-type methods to converge to a global minimum.\n\nA key piece of 
 the puzzle remains - how does optimization align with statistics to form t
 he complete mathematical picture of modern ML?\n
LOCATION:https://researchseminars.org/talk/MPML/42/
END:VEVENT
END:VCALENDAR