BEGIN:VCALENDAR VERSION:2.0 PRODID:researchseminars.org CALSCALE:GREGORIAN X-WR-CALNAME:researchseminars.org BEGIN:VEVENT SUMMARY:Sharan Vaswani (Simon Fraser University) DTSTART:20231019T210000Z DTEND:20231019T220000Z DTSTAMP:20260513T193639Z UID:SFUOR/23 DESCRIPTION:Title: E xploiting Problem Structure for Efficient Optimization in Machine Learning \nby Sharan Vaswani (Simon Fraser University) as part of PIMS-CORDS SF U Operations Research Seminar\n\nLecture held in ASB 10908.\n\nAbstract\nS tochastic gradient descent (SGD) is the standard optimization method for t raining machine learning (ML) models. SGD requires a step-size that depend s on unknown problem-dependent quantities\, and the choice of this step-si ze heavily influences the algorithm's practical performance. By exploiting the interpolation property satisfied by over-parameterized ML models\, we design a stochastic line-search procedure that can automatically set the SGD step-size. The resulting algorithm exhibits improved theoretical and e mpirical convergence\, without requiring the knowledge of any problem-depe ndent constants. Next\, we consider efficient optimization for imitation l earning (IL) and reinforcement learning. These settings involve optimizing functions for which it is expensive to compute the gradient. We propose a n optimization framework that uses the expensive gradient computation to c onstruct surrogate functions that can then be minimized efficiently. This allows for multiple model updates\, thus amortizing the cost of the gradie nt computation. The resulting majorization-minimization algorithm is equip ped with strong theoretical guarantees and exhibits fast convergence on st andard IL problems.\n LOCATION:https://researchseminars.org/talk/SFUOR/23/ END:VEVENT END:VCALENDAR