Programming & Scientific Computing · Statistical & Parallel Computing

Statistical computing: Numerical optimization

Public introduction · full guide in Eikos

Introduction

What this guide covers

Introduction Introduction Statistical computing & mathematical optimization

Overview

Topics covered

  • Introduction
  • Numerical Optimization in Inference and Modeling
  • Introduction and Formulation of the Problem in Statistics
  • The Statistical Paradigm as an Optimization Problem
  • Classification of Optimization Problems
  • Softness and Conditioning

Related concepts

Main connections

  • Maximum likelihood estimation
  • Calculus & differentiation
  • Linear algebra & matrices
  • Regularization (Lasso / Ridge / Elastic Net)
  • Mathematical optimization & gradients
  • Bayes' theorem & priors

Compatible notation

Mathematical sample in LaTeX

The expression is rendered accessibly, with its LaTeX source preserved below.

θ^MLE=argmaxθ∈ΘL(θ;𝐗)=argmaxθ∈Θ∏i=1np(Xi∣θ)
View LaTeX source\hat{\theta}_{\text{MLE}} = \arg\max_{\theta \in \Theta} L(\theta; \mathbf{X}) = \arg\max_{\theta \in \Theta} \prod_{i=1}^{n} p(X_i \mid \theta)