The edge of chaos: quantum field theory and deep neural networks

Abstract: The nascent NN-QFT correspondence offers an interesting, physics-based approach towards a theoretical foundation for deep neural networks. In this talk, I will discuss recent work in which we explicitly construct the QFT corresponding to a general class of networks encompassing both recurrent and feedforward architectures. The point at which the network exhibits critical behaviour can then be obtained by examining the largest Lyapunov exponent in a proper mean field theory treatment. However, this and previous results in the literature hold only at infinite width, while empirical evidence suggests that the correlation length receives significant corrections in real-world (finite width) networks. Hence, going beyond the mean field regime, we develop a perturbative approach formally analogous to that in the O(N) vector model, in which the weight variance plays the role of the 't Hooft coupling. In particular, we compute both the O(1) corrections quantifying fluctuations from typicality in the ensemble of networks, and the subleading O(depth/width) corrections due to finite-width effects. I will introduce the use of Feynman diagrams in this context for linear models, and discuss the generalization to nonlinear models if time permits.

HEP - theorymathematical physicsquantum physics

Audience: researchers in the topic

Purdue HET

Series comments: The recorded talks will be available on YouTube here: www.youtube.com/playlist?list=PLxU3vHZccQj64m9zsQR74D5WP1z1g4t1J