Nov 9, 2026
The token cap was grading the test
In a benchmark harness, the per-item token cap was quietly part of the score. A small local model got worse by thinking, and a hosted model's hidden thinking turned cut-off replies into wrong answers.
Nov 9, 2026
In a benchmark harness, the per-item token cap was quietly part of the score. A small local model got worse by thinking, and a hosted model's hidden thinking turned cut-off replies into wrong answers.
Oct 22, 2026
Stanford's Next-Concept Modeling runs on a 2017 GPU in two gigabytes and scores green on its benchmark, then turns my paragraph about a park into a supermarket car park. The checkpoint everyone wants is a five-byte TODO.