How to use the Learning Rate Schedule Visualizer
- 1. Pick a scheduleChoose cosine, warm-up + cosine, step, exponential, linear or constant.
- 2. Set the numbersEnter the starting rate, total epochs and any schedule settings.
- 3. PlotSee the curve, values at key epochs and download the CSV.
Formulas
- Cosine annealing: lr(t) = min + (lr₀ − min) × (1 + cos(πt ÷ T)) ÷ 2.
- Step decay: lr(t) = lr₀ × γ^⌊t ÷ step⌋, for example ×0.1 every 30 epochs.
- Exponential: lr(t) = lr₀ × γᵗ, with γ close to 1 such as 0.95.
- Warm-up + cosine: rise linearly for a few epochs, then follow cosine decay.
Worked example (defaults): cosine from 0.1 to 0 over 100 epochs. At epoch 25, lr = 0.1 × (1 + cos(π/4)) ÷ 2 = 0.085355; at epoch 50 it is exactly 0.05, at epoch 75 it is 0.014645, and it reaches 0 at epoch 100.
Why schedules help
A high learning rate early on moves quickly across the loss landscape; a low one later lets the model settle into a good minimum instead of bouncing around it. Cosine annealing is a popular default for image models. Transformers almost always use a warm-up, because large early steps with randomly initialised attention layers can destabilise training.
- PyTorch: CosineAnnealingLR, StepLR, ExponentialLR, LinearLR.
- Keras: CosineDecay, ExponentialDecay and LearningRateScheduler.
- Frameworks often update per step, not per epoch. Set T to total steps to match.
See why a too-large rate fails in the gradient descent visualizer, or train a network yourself in the neural network playground.
Frequently asked questions
Which schedule should I start with?
Cosine annealing with a short warm-up is a strong default for most deep learning tasks.
Does the chart match PyTorch exactly?
The formulas match PyTorch's standard schedulers for per-epoch updates. Some schedulers have extra options, such as restarts, not shown here.
Is the Learning Rate Schedule Visualizer free?
Yes, the Learning Rate Schedule Visualizer is free. The first time you plot your first schedule, you are asked for your name and WhatsApp number once. After that, this and every other Uncodemy tool opens straight away without the form.