Expressiveness in Deep Learning via Tensor Networks and Quantum Entanglement - Nadev Cohen

Three fundamental factors determine the quality of a statistical learning algorithm: expressiveness, generalization and optimization.  The classic strategy for handling these factors is relatively well understood.  In contrast, the radically different approach of deep learning, which in the last few years has revolutionized the world of artificial intelligence, is shrouded by mystery.  This talk will describe a series of works aimed at unraveling some of the mysteries revolving expressiveness, arguably the most prominent factor behind the success of deep learning.  I will begin by showing that state of the art deep learning architectures, such as convolutional networks, can be represented as tensor networks -- a computational model commonly employed in quantum physics.  This connection will inspire the use of quantum entanglement for defining measures of data correlations modeled by deep networks.  Next, I will turn to a quantum max-flow / min-cut theorem characterizing the entanglement captured by tensor networks.  This theorem will give rise to new results that shed light on expressiveness in deep learning, and in addition, provide new tools for deep network design.

Works covered in the talk were in collaboration with Yoav Levine, Or Sharir, David Yakira and Amnon Shashua.