Chapter 1 // Foundations

What Machine Learning Is

You can't hand-write the rule for every problem. So write a program with knobs, and let data set them. That single move is the whole book.

Adapted from the original chapter of Dive into Deep Learning · CC BY-SA 4.0 · this chapter ships no code — the figures run this site's own verified TypeScript, checked against Python (npm run verify:introduction)

Until recently, almost every program you used was a rigid set of rules someone wrote by hand. That works right up until the problem is one no human knows how to spell out — recognizing a spoken word, reading a photo. There, we don't write the answer. We write a program that can learn it.

Act 1 · The wall

01/05

one microphone snippet · 64 samples

Sixty-four numbers off a microphone. Somewhere in this static there's a word — or there isn't. Your job: write the code that decides.

The fix isn't a cleverer rule. It's to stop writing the rule at all — and hand the job to a pile of adjustable numbers.

Act 2 · Knobs, not rules

01/05

64 knobs + 1 bias

step0

loss0.69

test acc0.53

New plan: stop writing the rule. Give the program 64 knobs — one weight per sample — then multiply, add, and read out a score. Positive means “yes.”

That loop — score, measure the loss, nudge every knob downhill, repeat — trains every model on this site. Written out, one knob's nudge is just the chapter's own sentence, "perturb the parameter and see which way the loss moves," as a formula:

w_i \leftarrow w_i - \eta\,\frac{L(w_i+\epsilon)-L(w_i-\epsilon)}{2\epsilon}

The only question left is what shape your inputs and outputs take.

Act 3 · What shape is the answer?

01/05

“How much?” and “how many?” are regression. Your contractor billed $350 for 3 hours and $250 for 2. Two bills, plotted.

These are old questions. What makes them suddenly answerable is newer than the math behind them.

Act 4 · Why now

01/04

Averaging data to estimate is five centuries old. Köbel's crowd, 1522: sixteen men's feet, summed and divided into one “foot” — 301 mm, a hair under today's 305.

The lab

Three claims from the acts you shouldn't take on faith: that random knobs can become a detector, that the likeliest label isn't the decision, and that a five-century-old trick still earns its keep.

▸ lab

Train the knobs

Run the book's loop yourself — 64 random knobs into a detector. Raise η to reach the bottom in fewer steps.

64 knobs + bias

● wake word● not

step

loss

0.69

test acc

0.53

η = 0.020

Random knobs: a coin flip. Hit run.

▸ lab

Should you eat it?

The likeliest label isn't the decision — weigh the costs and watch it flip.

∞

eat 0.20 × ∞

discard ✓0.80

expected detriment · lower wins

P(poison) = 0.20death is certain (∞ cost)cost of a wasted dinner = 1.0

Never eat — unless P(poison) is exactly 0.

→ discard it

▸ lab

Köbel's robust foot

Drag one giant into the crowd. Watch the plain mean chase it; the trimmed mean holds.

newcomer's foot = 302 mmtrim k = 1 from each end

plain mean 301.3 mm

trimmed mean 302.2 mm

one outlier moved the plain mean 0.0 mm

k = 0 is the plain average. Even k = 1 — the medieval fix — shrugs off the giant.

The code (there isn't any — yet)

This is the one chapter of the book with no code cells: it's the map, not the territory. So there's nothing to peek at here — every other page carries a </> chip that shows the book's real PyTorch. What runs above is instead this site's own port, and its heart is the chapter's optimizer described literally:

src/lib/introduction.ts · verified against Python

// Perturb each knob a hair, see which way the loss moves, step downhill.
function perturbationStep(model, train, lr, epsilon, l2) {
  const w = [...model.w];
  for (let i = 0; i < model.w.length; i++) {
    probe.w[i] = original + epsilon;  const up   = regularizedLoss(probe, train, l2);
    probe.w[i] = original - epsilon;  const down = regularizedLoss(probe, train, l2);
    probe.w[i] = original;
    w[i] = original - lr * ((up - down) / (2 * epsilon)); // a numeric gradient
  }
  // … the bias moves the same way …
  return { w, b };
}

Chapter 3 makes this cheap: instead of poking every knob twice, calculus hands you all the gradients at once. That's the leap from this page to a real trainer.

Three things to remember

A model is knobs, not rules. When you can't write the rule by hand, write a program whose behavior is a few numbers — then let data set them.
Learning is a loop. Score the guess, measure the loss, nudge every knob the way that lowers it, repeat. The same loop scales from 64 knobs to billions.
Old math, new scale. The ideas are centuries old. Data grew a hundred times faster than memory and compute chased it — and that gap, not a new equation, is why deep learning works now.

Adapted from Chapter 1 of Dive into Deep Learning by Zhang, Lipton, Li, and Smola (CC BY-SA 4.0). This chapter contains no code in the book; the figures run this site's TypeScript, verified against Python.