Statistical Learning Theory Part 1: Hoeffding’s Inequality Derivation & Simulation

Hoeffding’s Inequality is an important concentration inequality in Mathematical Statistics and Machine Learning (ML), leveraged extensively in theoretically areas such as Statistical Learning Theory as well as applied areas such as Reinforcement Learning. I have noticed in pockets of the ML community it common to present Hoeffding’s Inequality as a “given” with only slight intuition (if any) provided as to where said inequality is derived from. I’m generally not a fan of this type of “magical thinking” with regards to understanding mathematical material. Given I will be writing future pieces that will leverage Hoeffding’s Inequality extensively, I’ve written this piece as a primer deriving the inequality step-by-step from first principles. <a href="https://anr248.medium.com/statistical-learning-theory-hoeffdings-inequality-derivation-simulation-e3a97100d147">Visit Now</a>