Machine Learning: Advanced Linear Models and Regularization Methods

Introduction

What this guide covers

Introduction \mathcal{L}(\beta) = \|Y - X\beta\|_2^2 + \lambda \|\beta\|_p^p |> argmin_\beta (\text{Regularization}) Statistical Theory and Algorithmic Implementation

Overview

Topics covered

• Introduction
• Advanced Linear Models and Regularization Methods
• Loss Perspective and Probability in Least Squares
• Geometric Foundation of Ordinary Least Squares (OLS)
• Maximum Likelihood Estimation and Gauss-Markov Hypothesis
• Multicolinearity and Singular Value Decomposition (SVD)

Related concepts

Main connections

• Regularization (Lasso / Ridge / Elastic Net)
• Linear algebra & matrices
• Linear regression
• Maximum likelihood estimation
• Variance & standard deviation
• Bayes' theorem & priors

Compatible notation

Mathematical sample in LaTeX

The expression is rendered accessibly, with its LaTeX source preserved below.