Advanced & Applied Topics · Information Theory & Entropy

Information theory: Foundations, measures, and statistical inference

Public introduction · full guide in Eikos

Introduction

What this guide covers

Introduction // Mathematical Methods of Information Theory

Overview

Topics covered

  • Introduction
  • Information Theory: Foundations and Measurements
  • Epistemological and Historical Introduction
  • Shannon Axiomatics and Faddeev's Theorem
  • The Faddeev-Khinchin Axioms
  • 1. Continuity

Related concepts

Main connections

  • Calculus & differentiation
  • Mean & expected value
  • Normal (Gaussian) distribution
  • Variance & standard deviation
  • Linear algebra & matrices
  • Conditional probability & independence

Compatible notation

Mathematical sample in LaTeX

The expression is rendered accessibly, with its LaTeX source preserved below.

HHartley=log(SN)=Nlog(S)
View LaTeX sourceH_{\text{Hartley}} = \log(S^N) = N \log(S)