Telling Technology · Learning in public
All episodes
EP.122 — KEPT OR REVERTED
A local model, a rulebook, a ruler

My page gets better one rule at a time. Numbers decide what stays.

I gave a free local model a 179-rule rulebook and one web page. It proposes one tiny patch, a measuring tool decides keep or revert, git remembers every keep, and the whole thing has no publish button.

A dark browser window showing a wireframe web page on the branch improve/resume-2026-10, with a yellow ruler marked 179 rules laid across it and two verdict chips: A-01 iter 2 keep 13 to 0, and A-01 iter 1 revert-gate-broke
⌁ 179 rules · 17 probes · 2 tries per rule · 0 publish buttons
why “make it better” is the wrong prompt
01 The problem

Taste is not a test

Ask any AI to “make this page better” and you get a confident rewrite: new hero, new palette, four hundred changed lines. Did contrast improve? Does it still fit a phone? Nobody can say, because nothing was measured. I had already run this loop twice with a frontier model, 29 versions of my resume page and the Rolodex polish loop. Both worked, but both ran on a frontier model and both depended on that model having good taste that night.

claude code — desktop
> Turn the three polish loops into something a small free model can run. It may only change what it can prove got better.
Then the model cannot be the judge. A rulebook says what good looks like, a tool belt measures it, and the runner keeps or reverts on the numbers. The model only ever proposes. (paraphrased)
A chat terminal where make this page better gets the answer I modernised the whole page, 412 lines changed, looks much better now, next to a checklist of contrast failures, sideways scroll, tap targets, dead links and anything that broke, each marked with a question mark
412 lines · 0 numbers
A rewrite you cannot check is a vibe, not an improvement.
02 The rulebook and the tool belt

179 rules, each one tied to something a script can measure

Every fix from those earlier loops became a rule in rules/rulebook-01.md: 14 chapters from Broken things to Print. A compiler turns it into rules.json, where 150 rules point at a probe, the gate or axe-core. The tool belt is Playwright plus axe-core: a seven-viewport gate and 17 probes (contrast, targets, focus, links, images, headings, meta, fonts, type, motion, spacing, colors, text, perf, print, hover, states), each validated on known-good and known-bad pages before a model ever sees them. The 28 rules that only pictures can judge are logged but never kept automatically.

claude code — desktop
> Compile the rulebook and show me the check behind rule A-01.
179 rules, 14 chapters. A-01 “Nothing sticks out sideways, at any of the seven widths” maps to gate: eq(verdict,'PASS') and empty(overflow). (paraphrased; the check is verbatim from rules.json)
A bar chart of the 179 rules across 14 chapters, A Broken things 11 through N Print 5, with chips reading 150 measured by a probe, gate or axe, 28 judged by pictures only, 1 read by eye
150 of 179 measured
If a rule cannot be measured, the machine is not allowed to keep it.
03 The loop, and the honest part

One patch, one measurement, one verdict

The local model (qwen3.8:27b on the Mac mini, called through my model router) gets one rule card and a page excerpt and answers with one SEARCH/REPLACE block. The runner applies it, measures again, and the judge compares numbers: better and nothing broke = keep, a git commit on the run’s own improve/<id> branch. Worse, broken gate or unsure = revert, byte identical. Two tries per rule, never a third. A control page on the tailnet takes a URL from my phone, a fetcher copies the page into a local git working copy, one worker runs one job at a time, and a STOP button ends it after the current iteration.

claude code — desktop
> Run the end-to-end rehearsal on the broken fixture page.
5 keeps, 1 revert, 1 skip, 7 iterations. A-01 try 1 widened the strip and the gate caught a 1984 px sideways scroll at 1680: reverted. Try 2 let it shrink with the column: kept, 13 → 0. Git holds v00 baseline, five keeps and a final commit. (real output, scripted model answers)
A terminal showing a real changelog: baseline gate fail, A-01 iteration 1 revert-gate-broke, A-01 iteration 2 keep 13 to 0, keeps for A-04, A-05, B-07 and C-02, a skip for C-09, a summary of 5 keeps 1 revert 7 iterations, and a git log from v00 baseline to v06 final
keep 5 · revert 1
The revert left no commit. Every keep is one.
⚠️

No live keep/revert counts yet

The run above is the rehearsal: real tools, a real git branch, a deliberately broken fixture page, but scripted model answers (fake-model). The dry run on my real resume page measured the baseline (gate PASS · 103 of 179 rules pass) and turned up five things to fix in the tool itself. The first live run with qwen3.8:27b on the Mac mini is still pending, so I am not quoting a model score here.

04 Gallery

The control page, the two tries, the tool belt, the hard parts

The first picture is a real screenshot of the control page during the dry run. The other three are typographic cards built from the repo and the rehearsal output. Tap any image to enlarge it and read the exact prompt that drew it.

BUILT 2026-10-06 TO 10-09 · FIRST LIVE RUN PENDING

The model never grades its own homework

The frontier model built the checker, the small model runs it. Numbers decide, git remembers, and the human merges. The tool cannot publish, deploy, push or send anything: its whole output is a branch and a report.

A bold typographic card on near-black: The model never grades its own homework, with numbers decide, git remembers, no publish button underneath, and the human merges, the machine never does
page-improver · 179 rules · 17 probes · qwen3.8:27b on the Mac mini · human merges only
06 Steal this

The page improver: rulebook, tool belt, runner

The 179-rule rulebook, the Playwright + axe-core probes and the keep-or-revert runner. The repo is not published under tellingtechnology yet, so the buttons point at the org for now. Everything in this episode is free and open — clone it, run it, make it yours.

skill rulebook-01.md (179 rules) cmd npm run gate | probe | shots template jobs/_template.json assets 17 probes + fixtures
run it # repo soon. then: python -m improver run jobs/<name>/job.json --model qwen3.8:27b

No GitHub? Comment KEEP on the post and the bot DMs you the link.

Next episode

The first live run: what a free model keeps on a real page

Next, qwen3.8:27b runs chapters A to G on the Mac mini against my real resume page. Real keeps, real reverts, seconds per iteration, and my own star rating next to each model.

A darkened score table of twelve local models, the candidates for the first live run
drops soon · follow so you don’t miss it