BEGIN:VCALENDAR VERSION:2.0 PRODID:researchseminars.org CALSCALE:GREGORIAN X-WR-CALNAME:researchseminars.org BEGIN:VEVENT SUMMARY:Rong Ge (Duke University) DTSTART:20201204T160500Z DTEND:20201204T170500Z DTSTAMP:20260423T005713Z UID:sss/15 DESCRIPTION:Title: A L ocal Convergence Theory for Mildly Over-Parameterized Two-Layer Neural Net \nby Rong Ge (Duke University) as part of Stochastics and Statistics S eminar Series\n\n\nAbstract\nThe training of neural networks optimizes com plex non-convex objective functions\, yet in practice simple algorithms ac hieve great performances. Recent works suggest that over-parametrization c ould be a key ingredient in explaining this discrepancy. However\, curren t theories could not fully explain the role of over-parameterization. In p articular\, they either work in a regime where neurons don't move much\, o r require large number of neurons. In this paper we develop a local conver gence theory for mildly over-parameterized two-layer neural net. We show t hat as long as the loss is already lower than a threshold (polynomial in r elevant parameters)\, all student neurons in an over-parametrized two-laye r neural network will converge to one of teacher neurons\, and the loss wi ll go to 0. Our result holds for any number of student neurons as long as it's at least as large as the number of teacher neurons\, and gives explic it bounds on convergence rates that is independent of the number of studen t neurons. Based on joint work with Mo Zhou and Chi Jin.\n LOCATION:https://researchseminars.org/talk/sss/15/ END:VEVENT END:VCALENDAR