BEGIN:VCALENDAR VERSION:2.0 PRODID:researchseminars.org CALSCALE:GREGORIAN X-WR-CALNAME:researchseminars.org BEGIN:VEVENT SUMMARY:Jérôme Bolte (Toulouse 1 University Capitole) DTSTART:20200706T130000Z DTEND:20200706T140000Z DTSTAMP:20260423T035054Z UID:OWOS/13 DESCRIPTION:Title: A Variational Model for Automatic Differentiation with Applications to Deep Learning\nby Jérôme Bolte (Toulouse 1 University Capitole) as part o f One World Optimization seminar\n\n\nAbstract\nAutomatic differentiation is an automatized implementation of differential calculus\, it plays a key computational role in several fields as machine learning\, design optimiz ation\, fluid dynamics\, physical modeling\, mechanics\, finance. It is al so efficient for nonsmooth problems despite the occurence of spurious beha viors. In that case\, one indeed observes the apparition of calculus artif acts and artificial critical points that have no variational nature. Our goal is to provide a simple mathematical model for this differentiation pr ocess. Our motivation comes from deep learning which will also serve as an illustrative model for our ideas and results.\nThe first easy\, but someh ow unexpected fact\, is that there is no\n«subdifferentiation» operator modeling nonsmooth nonconvex automatic differentiation. This fact motivate s the introduction of a family of multivalued mappings generalizing gradie nt-like behaviors that we call conservative fields. We shall review their salient properties and show how they allow us to study rigorously forward and backward automatic differentiation. We will also try to clarify the s purious behavior of automatic differentiation and study the role of what w e call «artificial critical points». We apply our findings to show that the training of feedforward neural networks through mini-batch stochastic «subgradient» methods comes with rigorous convergence guarantees.\nJoint work with E. Pauwels\n\nthe address and password of the zoom room of the seminar are sent by e-mail on the mailinglist of the seminar one day befor e each talk\n LOCATION:https://researchseminars.org/talk/OWOS/13/ END:VEVENT END:VCALENDAR