BEGIN:VCALENDAR VERSION:2.0 PRODID:researchseminars.org CALSCALE:GREGORIAN X-WR-CALNAME:researchseminars.org BEGIN:VEVENT SUMMARY:Leonid Pastur (B.Verkin Institute for Low Temperature Physics and Engineering\, Kharkiv\, Ukraine) DTSTART:20210611T133000Z DTEND:20210611T143000Z DTSTAMP:20260423T024507Z UID:MEGA/29 DESCRIPTION:Title: On Random Matrices Arising in Deep Neural Networks\nby Leonid Pastur (B. Verkin Institute for Low Temperature Physics and Engineering\, Kharkiv\, U kraine) as part of Séminaire MEGA\n\n\nAbstract\nWe study the distributio n of singular values of product of random matrices pertinent to the analys is of deep neural networks. The matrices resemble the product of the sampl e covariance matrices. However\, an important dierence is that the analog the of the population covariance matrices\, assumed to be non-random or r andom but independent of the random data matrix in statistics and random m atrix theory\, are now certain functions of random data matrices (synaptic weight matrices in the deep neural network terminology). For the Gaussian synaptic weight matrices the problem has been treated in recent work [1] and certain subsequent works by using the techniques of free probability t heory. Since\, however\, free probability theory deals with population cov ariance matrices which are independent of the data matrices\, its applicab ility to this case has to be justi\ned. We use a version of the techniques of random matrix theory to justify and generalize the results of [1] to t he case where the entries of the synaptic weight matrices are just indepen dent identically distributed random variables with zero mean and \nnite fo urth moment [2]. This\, in particular\, extends the property of the so-cal led macroscopic universality to the considered random matrices.\n\n[1] J. Pennington\, S. Schoenholz\, and S. Ganguli\, The emergence of spectral un iversality In: Proc. Mach. Learn. Res. (PMLR 70) 84 (2018) 1924-1932\, htt p://arxiv.org/abs/1802.09979\n\n[2] L. Pastur and V. Slavin\, On Random Ma trices Arising in Deep Neural Networks: General I.I.D. Case\, http://arxiv .org/abs/2011.11439.\n LOCATION:https://researchseminars.org/talk/MEGA/29/ END:VEVENT END:VCALENDAR