Oct 23, 2026
ollama ps forgot to count the part that makes it fast
A speculative-decoding head cost a model 9.6 GiB that ollama ps never reported. Benchmark numbers for a stock Qwen tag and a third-party quant, on a 32 GB card.
Oct 23, 2026
A speculative-decoding head cost a model 9.6 GiB that ollama ps never reported. Benchmark numbers for a stock Qwen tag and a third-party quant, on a 32 GB card.