Talks

In this talk, I will discuss how physics can help improve our understanding of deep learning systems and guide improvements to their scaling strategies. I will first discuss mathematical results based on mean-field techniques from statistical physics to analyze the feature learning dynamics of neural networks as well as posteriors of large Bayesian neural networks. This theory will provide insights to develop initialization and optimization schemes for neural networks that admit well defined infinite width and depth limits and behave consistently across model scales, providing practical advantages. These limits also enable a theoretical characterization of the types of learned solutions reached by deep networks, and provide a starting point to characterize generalization and neural scaling laws (see Cengiz Pehlevan's talk).

Literature reviews are an essential component of scientific research, but they remain time-intensive and challenging to write, especially due to the recent influx of research papers. In this talk, we will explore the zero-shot abilities of recent Large Language Models (LLMs) in assisting with the writing of literature reviews based on an abstract. We will decompose the task into two components: 1. Retrieving related works given a query abstract, and 2. Writing a literature review based on the retrieved results. We will then analyze how effective LLMs are for both components. For retrieval, we will discuss a novel two-step search strategy that first uses an LLM to extract meaningful keywords from the abstract of a paper and then retrieves potentially relevant papers by querying an external knowledge base. Additionally, we will study a prompting-based re-ranking mechanism with attribution and show that re-ranking doubles the normalized recall compared to naive search methods, while providing insights into the LLM's decision-making process. We will then discuss the two-step generation phase that first outlines a plan for the review and then executes steps in the plan to generate the actual review. To evaluate different LLM-based literature review methods, we create test sets from arXiv papers using a protocol designed for rolling use with newly released LLMs to avoid test set contamination in zero-shot evaluations. We will also see a quick demo of LitLLM in action towards the end.

We present AlphaGeometry2, a significantly improved version of AlphaGeometry introduced in Trinh et al. (2024), which has now surpassed an average gold medalist in solving Olympiad geometry problems. To achieve this, we first extend the original AlphaGeometry language to tackle harder problems involving movements of objects, and problems containing linear equations of angles, ratios, and distances. This, together with support for non-constructive problems, has markedly improved the coverage rate of the AlphaGeometry language on International Math Olympiads (IMO) 2000-2024 geometry problems from 66% to 88%. The search process of AlphaGeometry2 has also been greatly improved through the use of Gemini architecture for better language modeling, and a novel knowledge-sharing mechanism that enables effective communication between search trees. Together with further enhancements to the symbolic engine and synthetic data generation, we have significantly boosted the overall solving rate of AlphaGeometry2 to 84% for all geometry problems over the last 25 years, compared to 54% previously. AlphaGeometry2 was also part of the system that achieved silver-medal standard at IMO 2024 this https URL. Last but not least, we report progress towards using AlphaGeometry2 as a part of a fully automated system that reliably solves geometry problems directly from natural language input.

Exactly solvable models have tremendously helped our understanding of condensed matter systems. A notable number of them are "frustration-free" in the sense that all local terms of the Hamiltonian can be minimized simultaneously. It has been particularly successful at describing the physics of gapped phases of matter, such as symmetry protected topological phases and topologically ordered phases. On the other hand, relatively little has been understood about gapless frustration-free Hamiltonians, and their ability to teach us about more generic systems. In this talk, we derive a constraint on the spectrum of frustration-free Hamiltonians. Their dynamical exponent z, which captures the scaling of the energy gap versus the system size, is bounded from below to be z >= 2. This proves that frustration-free Hamiltonians are incapable of describing conformal critical points with z = 1. Further, by a well-known mapping from Markov processes to frustration-free Hamiltonians, we show that the relaxation time for many Markov processes also scale with z >=2. This improves the previously known bound on the relaxation time scaling of z >= 7/4. The talk is based on works with Rintaro Masaoka and Haruki Watanabe.

Scientific research is facing mounting challenges: overwhelmed reviewers, fragmented expertise, an outdated and inefficient system for allocating resources, and dissemination tools that no longer match the complexity and scale of modern scientific output. In this talk, I speak both as a researcher and as the founder of a successful startup to explore what a more coherent, future-ready scientific ecosystem could look like. Drawing on real prototypes and emerging tools, I’ll outline how we might reshape the way we publish, collaborate, and share not just to fix what’s broken, but to unlock what science could become tomorrow.

In the past two years, the LLM has made significant progress in math and reasoning, but it has not been applied widely in scientific research tasks. In this talk I will give a brief introduction to our on-going efforts on building the first AI scientist platform, where all researchers in different fields can contribute to teaching the AI scientists via contributing benchmarks and contributing specialized tools. We believe that by providing AI with the real-time updates of benchmarks and research tools, we are starting to enter an era with innovation driven by new types of human-AI collaboration.