Introduction
What this guide covers
Introduction Rigorous Reference Technical Treatise
Overview
Topics covered
- Introduction
- Information Theory and Asymptotic Behavior
- Fundamental measures of information and mathematical entropy
- 1.1 Shannon entropy for a discrete variable
- 1.2 Joint, conditional entropy and the chain rule
- 1.3 Mutual information and Kullback-Leibler divergence
Related concepts
Main connections
Compatible notation
Mathematical sample in LaTeX
The expression is rendered accessibly, with its LaTeX source preserved below.
View LaTeX source
H(X) = -\sum_{x \in \mathcal{X}} p(x) \log_b p(x)