How to use the Gradient Descent Visualizer
- 1. Choose a functionStart with x², then try the wavy curve with many dips.
- 2. Set rate, start and stepsPick the learning rate η, the starting x and how many steps.
- 3. Run and compareWatch the path, then change η and run again.
The update rule
Gradient descent finds a minimum by repeatedly stepping downhill: x ← x − η × f′(x), where f′(x) is the slope and η is the learning rate. For f(x) = x², f′(x) = 2x, so each step multiplies x by (1 − 2η).
Worked example (the defaults): start at x = 5 with η = 0.1. Each step multiplies x by 0.8, so after 10 steps x = 5 × 0.8¹⁰ = 0.5369. Run 50 steps and x falls to about 0.00007. That is the same idea a neural network uses for millions of weights at once.
Experiments worth trying
- η = 0.5 on x² jumps straight to 0 in one step, because 1 − 2η = 0.
- η = 0.9 overshoots and zig-zags but still converges.
- η = 1.1 makes each jump bigger than the last, and the run diverges.
- The wavy function has several dips. Different starting points end in different local minima, which is why initialisation matters.
- x⁴ − 3x³ + 2 has a flat spot at x = 0 where the gradient is zero, so a start near it barely moves.
Real training uses stochastic or mini-batch gradients, momentum and adaptive optimisers like Adam, but the core rule is this one. Plan a decaying rate with the learning rate schedule visualizer, or watch a full network train in the neural network playground.
Frequently asked questions
Why does a big learning rate fail?
Each step overshoots the minimum by more than it gained, so the error grows instead of shrinking.
Is this the same as in linear regression?
Yes in spirit. Linear regression applies the same rule to the slope and intercept using the gradient of the squared error.
Is the Gradient Descent Visualizer free?
Yes, the Gradient Descent Visualizer is free. The first time you run your first descent, you are asked for your name and WhatsApp number once. After that, this and every other Uncodemy tool opens straight away without the form.