The Small-Model Trade: Reasoning In, Facts Out

A post by Walter van der Giessen, widely discussed on Hacker News, argues that the collapse in per-token compute at the frontier is not free efficiency but a deliberate trade: labs are cutting world knowledge out of weights and keeping reasoning procedures.

The evidence he assembles is the gap between benchmark families. GLM-5.2 scores 99.2% on AIME 2026 with roughly 40 billion parameters active per token; Qwen3.5 hits 91.3% with 17 billion; DeepSeek V4-Flash runs 13 billion active. On plain factual recall the picture inverts — SimpleQA's leader is Gemini 2.5 Pro at 53%, and Artificial Analysis puts Qwen3.5 4B and 9B at 80–82% hallucination rates on its knowledge benchmark.

The mechanism he offers is storage economics, citing the "Physics of Language Models" work at roughly two bits of factual knowledge per parameter. Facts are expensive to store and cheap to retrieve; reasoning is a small set of reusable procedures that distills well. Facts also rot — a training run costs months and hundreds of millions, and its beliefs about the JavaScript ecosystem are stale on arrival. Procedures don't.

For builders, the consequence is that the harness becomes load-bearing: retrieval, tool calls, docs on disk. A coding agent greps node_modules rather than recalling an API. The upside he emphasizes is auditability — a wrong fact in a knowledge base has an address and a regression test; a wrong fact in weights has neither.

The forward-looking part is explicitly the author's prediction, not a reported plan: frontier-quality reasoning on a single 24GB consumer GPU once expert layers holding facts become optional. Whether stripped-down models degrade gracefully without tools is the open question, and the post concedes retrieval does not reach zero hallucination.