Towards a “Theoretical Minimum” for Physicists in AI

          @misc{ pirsa_PIRSA:25040089,
            doi = {10.48660/25040089},
            url = {https://pirsa.org/25040089},
            author = {Kahn, Yonatan},
            keywords = {},
            language = {en},
            title = {Towards a {\textquotedblleft}Theoretical Minimum{\textquotedblright} for Physicists in AI},
            publisher = {Perimeter Institute},
            year = {2025},
            month = {apr},
            note = {PIRSA:25040089 see, \url{https://pirsa.org}}
          }
          

Abstract

As progress in AI hurtles forward at a speed seldom seen in the history of science, theorists who wish to gain a first-principles understanding of AI can be overwhelmed by the enormous number of papers, notational choices, and assumptions in the literature. I will make a pitch for developing a “Theoretical Minimum” for theoretical physicists aiming to study AI, with the goal of getting members of our community up to speed as quickly as possible with a suite of standard results whose validity can be checked by numerical experiments requiring only modest compute. In particular, this will require close collaboration between statistical physics, condensed matter physics, and high-energy physics, three communities that all have important perspectives to bring to the table but whose notation must be harmonized in order to be accessible to new researchers. I will focus my discussion on (a) the various approaches to the infinite-width limit, which seems like the best entry point for theoretical physicists who first encounter neural networks, and (b) the need for benchmark datasets from physics complex enough to capture aspects of natural-language data but which are nonetheless “calculable” from first-principles using tools of theoretical physics.