A Simple Introduction to Gradient Descent

Gradient descent is one of the most common optimization algorithms in machine learning. Understanding its basic implementation is fundamental to understanding all the advanced optimization algorithms built off of it.

Background

This article is supplementary to An Introduction to Machine Learning in Python: Simple Linear Regression. It should be read first or in conjunction with this article. It would also be beneficial to have a basic understanding of partial derivatives in calculus because this article also examines the partial derivatives of several variations of the Mean Squared Error (MSE).

Read More