A Keyframe-style Learning Rate Scheduler for PyTorch

It struck me one day that defining how a learning rate should change over the course of a training run is similar to defining an animation, with a combination of: <ul> <li><a href="https://en.wikipedia.org/wiki/Key_frame" rel="noopener ugc nofollow" target="_blank">Key frames</a>: at timestep x the value should be y</li> <li>Transitions: how to interpolate values between timesteps</li> </ul> So, I’ve created an LR scheduler that uses this keyframe concept to define a schedule as a sequence of frames and transitions. I created this for personal use and have been using it for a little while. I find it quite useful and wonder if others might too, hence this article. I won’t explain the code in this post, and instead just focus on what it does and how to use it. Here’s a gist with the code if you’d like to try it out. If you use it and like it, please star the gist so I can calibrate my sense of self-worth accordingly. <h1>Creating a schedule</h1> In its simplest form, we only need to provide a single frame. For example, if we want the learning rate to be <code>0.01</code> when we’re 20% of the way through training: <a href="https://betterprogramming.pub/a-keyframe-style-learning-rate-scheduler-for-pytorch-b889110dcde8">Click Here</a>