Wilsonian Renormalization of Neural Network Field Theories

Neural scaling laws and hyperparameter transfer scaling laws play an increasingly dominant role in deep learning. In physics, scaling behaviour pointed directly to the Wilsonian Renormalisation

Group (RG). In this seminar talk, I'll present our ongoing efforts to build an RG framework for deep learning, based on the neural-network field-theory correspondence. In particular, I'll explain what neural network field theories are, write down their action close to a solvable Gaussian point, and suggest a Wilsonian RG approach based on integrating out non-learnable field modes—the data-driven analogue of UV modes. Interestingly, the lack of translation invariance and locality is balanced by large-dimension effects. The RG treatment recovers a celebrated result in ML (Benign overfitting). It also allows us to define analogues of scaling dimensions, namely, scaling

intervals and, with them, the notions of relevancy and irrelevancy. Finally, I'll discuss open questions and future directions involving strong non-Gaussian behavior and feature learning.