Gradient Descent Visualizer

Gradient descent is how models learn: measure which way the loss goes downhill, take a step, repeat. new x = x − η × gradient. Watch it happen, one step at a time.

Each tab starts with a short guide for beginners. Use the third tab, Paper, Python & R, for a practice problem, a pencil worksheet, and ready-to-run Python and R code. Nothing leaves your browser.

Picture a hill in thick fog. You want the lowest point but can only feel the ground under your feet. So you feel which way slopes down and take a small step that way. Repeat. That is gradient descent.

  • f(x) = the loss. It measures how wrong the model is. Lower is better.
  • Gradient f′(x) = the slope. Positive means uphill to the right, so we go left.
  • Learning rate η = how big each step is. This is the most important setting in training.

The whole algorithm in one line: new x = x − η × f′(x). The minus sign is what makes us go downhill.

Try this: press Play with the simple bowl. Then press Too small (crawls) and Too big (explodes). Finally pick Two valleys and start from x = 2, then from x = −2. Same algorithm, different valley.

Setup — f(x) = x², slope f′(x) = 2x

Step 0 / 25
-404

Descending: each step moves us downhill

3
Current x (the parameter)
9
Loss f(x)
6
Gradient f′(x) (slope)
-0.6
Next move: −η × slope

Step-by-step working

Step 1: The rule

x_new = x − η × f′(x). We repeat it 25 times, starting from x = 3 with η = 0.1.

Step 2: Measure the slope at step 0

x = 3, so f′(x) = 6. Positive slope: uphill is to the right, so we walk left.

Step 3: Take the step

x_new = 3 − 0.1 × (6) = 2.4. The step length is η times the slope, so steep ground gives big steps and flat ground gives tiny ones.

Step 4: What this learning rate does

For x², each step multiplies x by (1 − 2η) = 0.8. Between 0 and 1: smooth slide to 0. Closer to 0 means faster.

All steps so far

StepxLoss f(x)Slope f′(x)Next x
03962.4

How this calculator works

The 1D tab applies x ← x − η·f′(x) to three loss curves and animates each step. The 2D tab moves two parameters with plain gradient descent, momentum (β = 0.9) and Adam (β₁ = 0.9, β₂ = 0.999, bias-corrected), using the exact analytic gradient of each surface, and draws the paths on a heat map of the loss.

Real training estimates the gradient from mini-batches of data, so real paths are noisier than the ones shown here. Learning tool only.

Common questions