Condensed Matter

A scientific understanding of modern deep learning is still in its early stages. As a first step towards understanding the learning dynamics of neural networks, one can simplify the problem by studying limits that might have theoretical tractability and practical relevance. I’ll begin with a brief survey of our earlier body of work that has investigated the infinite width limit of deep networks, a topic of active study recently. With these results in hand, it nonetheless appears there is still a gap towards theoretically describing neural networks at finite width. I’ll argue that the choice of learning rate is one crucial factor in dynamics away from the infinite width limit and naturally classifies deep networks into two classes separated by a sharp transition. This is elucidated in a class of solvable simple models we present, which give quantitative predictions for the two classes. Quite remarkably, we test these predictions empirically in practical settings and find excellent agreement. 

Yasaman Bahri is a research scientist on the Google Brain team. Her current research program is to build a scientific understanding of deep learning using a combination of theoretical analysis and empirical investigation. Prior to Google, she was at the University of California, Berkeley, where she received her Ph.D. in physics in 2017, specializing in theoretical quantum condensed matter.

While the concept of topology is often introduced by contrasting oranges with bagels, the idea of topologically distinct quantum phases of matter is far more abstract. In this talk, we will focus on a more tangible form of topology that also arises in quantum condensed matter system: when the sites in a lattice are dragged around in a symmetric manner, what attributes of the lattice would remain unchanged? Such deformation can be viewed as the lattice analog of the familiar (mental) exercise of transforming a bagel into a coffee mug. We will first introduce the mathematical framework for identifying the topological classes of lattices, and then discuss how these invariants can constrain and inform the more abstract notion of topological phases of matter that emerge.

We discuss a new class of quantum phase transitions --- Deconfined Mott Transition (DMT) --- that describe a continuous transition between a Fermi liquid metal with a generic electronic Fermi surface and an insulator without emergent neutral Fermi surface. We construct a unified U(2) gauge theory to describe a variety of metallic and insulating phases, which include Fermi liquids, fractionalized Fermi liquids (FL*), conventional insulators and quantum spin liquids, as well as the quantum phase transitions between them. Using the DMT as a basic building block, we propose a distinct quantum phase transition --- Deconfined Metal-Metal Transition (DM2T) --- that describes a continuous transition between two metallic phases, accompanied by a jump in the size of their Fermi surfaces (also dubbed a 'Fermi transition'). We study these new classes of deconfined metallic quantum critical points using a renormalization group framework at the leading nontrivial order in a controlled double-expansion and comment on the various interesting scenarios that can emerge going beyond this leading order calculation.

I’ll talk about two independent works on classical and quantum neural networks connected by information theory. In the first part of the talk, I’ll treat sequence models as one-dimensional classical statistical mechanical systems and analyze the scaling behavior of mutual information. I'll provide a new perspective on why recurrent neural networks are not good at natural language processing. In the second part of the talk, I’ll study information scrambling dynamics when quantum neural networks are trained by classical gradient descent algorithm. For many problems, this hybrid quantum-classical training process consists of two stages where information scrambles very differently in the network. 

I propose [1] to use the residual anyons of overscreened Kondo physics for quantum computation. A superconducting proximity gap of Δ<TK can be utilized to isolate the anyon from the continuum of excitations and stabilize the non-trivial fixed point. We use the dynamical large-N technique [2] and bosonization to show that the residual entropy survives in a superconductor and suggest a charge Kondo setup for isolating and detecting the Majorana fermion in the two-channel Kondo impurity.
I will then conjecture that topological defects in a multichannel Kondo lattice carry anyons. Motivated by this, we look at a two-channel SU(N) Kondo lattice in the large-N limit [3].  In this model, the continuous channel-symmetry is spontaneously broken, forming a “channel ferromagnet” and realizing the so-called fractionalized order parameter [4]. By integrating out the fermions we derive an effective action that describes the symmetry breaking and its emergent collective modes. Remarkably, topological defects in the order parameter carry a U(1) flux, manifested in the Aharonov-Bohm phase picked by electrons that orbit the defect. We argue that the phase diagram contains a non-magnetic transition between a large and a small Fermi surface.
I will also briefly highlight our recent results [5] on a magnetically frustrated Kondo-screened triangle which contains two symmetry-preserving phases, transcending the Landau-Ginzburg paradigm. The quantum phase transition is driven by the proliferation of instantons of the emergent gauge theory and can be regarded as a toy model for the deconfined criticality.

Meta-learning involves learning mathematical devices using problem instances as training data. In this talk, we first describe recent meta-learning approaches involving the learning of objects such as: initial weights, parameterized losses, hyper-parameter search strategies, and samplers. We then discuss learned optimizers in further detail and their applications towards optimizing variational circuits. This talk also covers some lessons learned starting a spin-off from academia.