Advanced & Applied Topics · Numerical Optimization & Scientific Methods

Numerical methods · numerical optimization — algorithmic and mathematical foundations

Public introduction · full guide in Eikos

Introduction

What this guide covers

Introduction Chart from the original guide Numerical Methods · High Performance Computing

Overview

Topics covered

  • Introduction
  • Numeric and Algorithmic Optimization
  • Formulation and Optimality Conditions
  • 1.1 Necessary and Sufficient Conditions in Unrestricted Optimization
  • 1.2 The Fundamental Role of Convexity
  • Geometry of Loss Surfaces

Related concepts

Main connections

  • Linear algebra & matrices
  • Mathematical optimization & gradients
  • Calculus & differentiation
  • Regularization (Lasso / Ridge / Elastic Net)
  • Variance & standard deviation
  • Bayes' theorem & priors

Compatible notation

Mathematical sample in LaTeX

The expression is rendered accessibly, with its LaTeX source preserved below.

min𝐱∈ℝdf(𝐱)subject to𝐱∈Ω
View LaTeX source\min_{\mathbf{x} \in \mathbb{R}^d} f(\mathbf{x}) \quad \text{subject to} \quad \mathbf{x} \in \Omega