Deep neural networks (DNNs) have profoundly transformed the landscape of machine learning, often becoming synonymous with the broader fields of artificial intelligence and machine learning. Yet, their rise would have been unimaginable without their partner-in-crime: stochastic gradient descent (SGD)...