skip to content
exer//notes · local models

running a model on hardware that should not be able to run it

2 May 2026

Quantisation notes from a machine that predates the model by about a decade.

[ placeholder content ] — shipped with the site so the editorial layout can be reviewed. Not a published note.

The machine is old enough to have a spinning disk. The model was trained on hardware that costs more than the building. It works, slowly, and the way it fails is more interesting than the way it succeeds.

What quantisation actually costs you

Not accuracy in the aggregate — the benchmarks hold up better than the discourse suggests. What you lose is the tail: the long, structured, format- sensitive outputs where a single token slip cascades. Ask for prose and it is fine. Ask for a JSON schema with eleven fields and you will meet the tail.

Placeholder note shipped with the site so the editorial layout can be reviewed with realistic text. Replace before launch.