PIRSA:16110030

A conceptual viewpoint on information decomposition

Perrone, P. (2016). A conceptual viewpoint on information decomposition. Perimeter Institute. https://pirsa.org/16110030

Perrone, Paolo. A conceptual viewpoint on information decomposition. Perimeter Institute, Nov. 08, 2016, https://pirsa.org/16110030 

          @misc{ pirsa_PIRSA:16110030,
            doi = {10.48660/16110030},
            url = {https://pirsa.org/16110030},
            author = {Perrone, Paolo},
            keywords = {Quantum Foundations},
            language = {en},
            title = {A conceptual viewpoint on information decomposition},
            publisher = {Perimeter Institute},
            year = {2016},
            month = {nov},
            note = {PIRSA:16110030 see, \url{https://pirsa.org}}
          }

Paolo Perrone Max Planck Institute for Mathematics in the Sciences

DOI 10.48660/16110030
Collection
Talk Type Scientific Series
Subject

Abstract

Can we decompose the information of a composite system into terms arising from its parts and their interactions?
For a bipartite system (X,Y), the joint entropy can be written as an algebraic sum of three terms: the entropy of X alone, the entropy of Y alone, and the mutual information of X and Y, which comes with an opposite sign. This suggests a set-theoretical analogy: mutual information is a sort of "intersection", and joint entropy is a sort of "union".
The same picture cannot be generalized to three or more parts in a straightforward way, and the problem is still considered open. Is there a deep reason for why the set-theoretical analogy fails?
Category theory can give an alternative, conceptual point of view on the problem. As Shannon already noted, information appears to be related to symmetry. This suggests a natural lattice structure for information, which is compatible with a set-theoretical picture only for bipartite systems.
The categorical approach favors objects with a structure in place of just numbers to describe information quantities. We hope that this can clarify the mathematical structure underlying information theory, and leave it open to wider generalizations.