Benchmarks
geistlib against llama.cpp
Every model on every machine it fits, measured nightly under one frozen protocol. Each run is archived with its raw samples.
| Model | AMD Ryzen 9 9950X · CPU | RTX 2080 Ti · Vulkan | Apple M1 Max · CPU | Apple M1 Max · Metal | Raspberry Pi 5 · CPU |
|---|---|---|---|---|---|
| Qwen3.5 0.8B Q8_0 | pp -16 % · tg +15 %866 / 82.1 tok/s 2026-10-12 · 4d6e2696a | pp +30 % · tg +13 %15674 / 326.4 tok/s 2026-10-09 · 53aacaacb | — | — | pp -31 % · tg +19 %66 / 13.7 tok/s 2026-10-12 · 4d6e2696a |
| Gemma 4 E2B Q4_K_M | pp -16 % · tg +19 %528 / 51.5 tok/s 2026-10-12 · 4d6e2696a | pp +12 % · tg -4 %5767 / 158.3 tok/s 2026-10-09 · 53aacaacb | — | — | — |
| BitNet b1.58-large TQ2_0 | pp +61 % · tg +48 %2232 / 276.6 tok/s 2026-10-12 · 4d6e2696a | pp -6 % · tg -7 %13261 / 342.3 tok/s 2026-10-09 · db7f140d5 | — | — | — |
| Gemma 4 E4B Q4_K_M | pp -19 % · tg +16 %267 / 26.1 tok/s 2026-10-09 · 53aacaacb | pp -3 % · tg -6 %2941 / 96.8 tok/s 2026-10-09 · 53aacaacb | — | — | — |
| Llama 3.2 3B Q4_K_M | pp -26 % · tg +8 %379 / 37.0 tok/s 2026-10-09 · 53aacaacb | pp +4 % · tg -16 %4910 / 150.9 tok/s 2026-10-09 · 53aacaacb | — | — | — |
| Qwen3 0.6B Q8_0 | pp +18 % · tg +9 %1951 / 110.7 tok/s 2026-10-09 · 53aacaacb | pp +12 % · tg -9 %18553 / 314.9 tok/s 2026-10-09 · 53aacaacb | — | — | pp -62 % · tg +6 %70 / 14.9 tok/s 2026-10-12 · 4d6e2696a |
| Qwen3.5 4B Q4_0 | pp -18 % · tg +13 %257 / 26.0 tok/s 2026-10-09 · 53aacaacb | pp +5 % · tg -2 %3520 / 122.2 tok/s 2026-10-09 · 53aacaacb | — | — | — |
| Qwen3.8 27B Q4_0 | pp -28 % · tg +8 %39 / 4.8 tok/s 2026-10-09 · bce40a4bd | — | — | — | — |
| Ternary Bonsai 2 27B PQ2_0 | — | pp +16 % · tg +36 %636 / 40.0 tok/s 2026-10-12 · 4d6e2696a | — | — | — |
Each cell: geistlib's speed relative to llama.cpp for prefill (pp512) and decode (tg64 at depth 512), geistlib's own tokens/s, the trend over the last 30 runs, and the date and geistlib commit of the newest run. Ratios compare within a column; tokens/s across columns describe the hardware. A red mark: geistlib slowed by more than 5 % against the cell's previous run while llama.cpp did not.
Method: the frozen cross-engine protocol of geistlib (same model file by SHA-256, same shapes and threads for both engines, 4 alternating cycles × 3 samples, median), run on a quiet host by the nightly and weekly bench-matrix workflow. Speed only, not a quality comparison. Click a cell for its archive of every run; all numbers as JSON: history.json; every raw run as a git archive: geisten/bench-results.