PIRSA:16110030 Scientific Series Quantum Foundations

DOI10.48660/16110030

Series: Quantum Foundations

Perrone, P. (2016). A conceptual viewpoint on information decomposition. Perimeter Institute. https://pirsa.org/16110030

Perrone, Paolo. A conceptual viewpoint on information decomposition. Perimeter Institute, Nov. 08, 2016, https://pirsa.org/16110030 

@misc{ pirsa_PIRSA:16110030,
  doi = {10.48660/16110030},
  url = {https://pirsa.org/16110030},
  author = {Perrone, Paolo},
  keywords = {Quantum Foundations},
  language = {en},
  title = {A conceptual viewpoint on information decomposition},
  publisher = {Perimeter Institute},
  year = {2016},
  month = {nov},
  note = {PIRSA:16110030 see, \url{https://pirsa.org}}
}

Abstract

Can we decompose the information of a composite system into terms arising from its parts and their interactions?
For a bipartite system (X,Y), the joint entropy can be written as an algebraic sum of three terms: the entropy of X alone, the entropy of Y alone, and the mutual information of X and Y, which comes with an opposite sign. This suggests a set-theoretical analogy: mutual information is a sort of "intersection", and joint entropy is a sort of "union".
The same picture cannot be generalized to three or more parts in a straightforward way, and the problem is still considered open. Is there a deep reason for why the set-theoretical analogy fails?
Category theory can give an alternative, conceptual point of view on the problem. As Shannon already noted, information appears to be related to symmetry. This suggests a natural lattice structure for information, which is compatible with a set-theoretical picture only for bipartite systems.
The categorical approach favors objects with a structure in place of just numbers to describe information quantities. We hope that this can clarify the mathematical structure underlying information theory, and leave it open to wider generalizations.

Resources

Next talk

Related Talks