Gradient descent is one of the most common optimization algorithms in machine learning. Understanding its basic implementation is fundamental to understanding all the advanced optimization algorithms built off of it.
Background
This article is supplementary to An Introduction to Machine L...