Statistical physics of learning with two-layer neural networks

          @misc{ pirsa_PIRSA:25040093,
            doi = {10.48660/25040093},
            url = {https://pirsa.org/25040093},
            author = {Loureiro, Bruno},
            keywords = {},
            language = {en},
            title = {Statistical physics of learning with two-layer neural networks},
            publisher = {Perimeter Institute},
            year = {2025},
            month = {apr},
            note = {PIRSA:25040093 see, \url{https://pirsa.org}}
          }
          

Abstract

Feature learning - or the capacity of neural networks to adapt to the data during training - is often quoted as one of the fundamental reasons behind their unreasonable effectiveness. Yet, making mathematical sense of this seemingly clear intuition is still a largely open question. In this talk, I will discuss a simple setting where we can precisely characterise how features are learned by a two-layer neural network during the very first few steps of training, and how these features are essential for the network to efficiently generalise under limited availability of data.