Skip to content

Advanced & Applied Topics · Information Theory & Entropy

Information theory in continuous spaces · Advanced mathematical treatise

Public introduction · full guide in Eikos

Introduction

What this guide covers

Introduction Mathematical theory of Claude Shannon

Overview

Topics covered

Introduction
Information Theory: Continuous Spaces
Limit of discrete entropy and transition to continuum
Differential Entropy (Shannon Entropy in Continuous Spaces)
Behavior before Transformations and Coordinate Changes
Differential entropy of classical analytic distributions

Related concepts

Main connections

Compatible notation

Mathematical sample in LaTeX

The expression is rendered accessibly, with its LaTeX source preserved below.

H(X) = -\sum_{x \in \mathcal{X}} P(X=x) \log_2 P(X=x)

View LaTeX source

H(X) = -\sum_{x \in \mathcal{X}} P(X=x) \log_2 P(X=x)