The 4 Traps

Lie detectors built into every test

Trap cells are automatic checks embedded in every experiment. They catch the experimenter, not the experiment.

τ — the random floor

τ is the score of our best blindfolded throw. Imagine throwing darts blindfolded: sometimes you hit the bullseye by luck. τ is the best error from $B$ random tries on $N$ held-out points. Your method only PASSes if it beats τ by a clear margin.

Hardening history

τ wasn’t always trustworthy. Here’s how it earned the right to kill rungs.

Definition: τ is the best error from B random baselines on N held-out points. A method PASSes only if its error is below τ by a relative margin.