A 2B Language Model as One File on a 4 GB Raspberry Pi

The most honest install instruction is the one with nothing to configure. As of v0.8.2, running a 2-billion-parameter language model on a Raspberry Pi 5 is three lines:

curl -L -o geist-bitnet https://github.com/geisten/geistlib/releases/latest/download/geist-bitnet-linux-arm64
chmod +x geist-bitnet
./geist-bitnet "The three largest moons of Jupiter are"

One file, 1.2 GB. It contains the engine — under 1 MB of compiled C23 — and the complete BitNet b1.58 2B-4T model. No Python, no model download step, no network access after the download. Run it with no arguments and you get a minimal REPL.

How a model fits into an executable

The weights are placed in the binary’s read-only data section at build time and loaded with geist_model_load_from_memory, which aliases them zero-copy. The kernel demand-pages the mapping exactly as it would an mmaped model file, so RAM cost does not change: peak resident memory measured 1.66 GB while generating, on a board with 4 GB. The pages are file-backed and evictable under pressure.

Two consequences worth stating plainly. First, cold start is storage-bound: the first run after boot took 14.3 s to the first token on our SD card, because 1.1 GiB of weights page in once. Every warm start after that: 0.6 s. Second, this only works with a linker that tolerates a >1 GB section — GNU ld does; Apple’s ld64 corrupted relocations and crashed (measured, reproducibly), which is why the single-file build exists for Linux/arm64 only.

Measured against Microsoft’s own runtime

Side by side on the same Pi 5: geist finishes 110 tokens in 7.1 s, bitnet.cpp needs 11.7 s

Same 4 GB board, byte-identical GGUF, both greedy at 4 threads, each take started below 57 °C with temperatures and commits visible in the frame, recorded sequentially so neither engine steals CPU from the other. This recording: 110 tokens in 7.1 s (15.5 tok/s end-to-end) against bitnet.cpp’s own timing line of 9.31 tok/s.

The frozen 10-repeat protocol says the same thing with error bars: 17.9 tok/s at short prompts, 15.0 at a 512-token context, spread under 1 %, and — new in the harness — energy from phase-separated PMIC probes: 0.44 J per decoded token, 0.20 J per prefill token. The full report is published as a gist, and make bench reproduces it on your hardware, competitor included.

What broke along the way

Findings we hit while producing this release, documented rather than hidden:

bitnet.cpp’s current main mis-decodes the canonical i2_s GGUF on ARM — it emits ???? where text belongs. Speed harnesses like llama-bench never notice; a recording does. Our comparison uses the June 2025 state (404980e), whose greedy output starts token-identical to ours.
gcc 12 crashes (internal compiler error) on bitnet.cpp’s i2_s quantizer — their clang requirement is real.
Small models confabulate fluently. Asked about bleeding a radiator, the model confidently recommended a hairdryer and a vacuum cleaner. The user guide keeps examples like this on purpose: treat output as plausible text, not knowledge.

Limitations, openly

llama.cpp’s OpenBLAS prefill still leads on the Pi. The REPL is deliberately memory-less — there is no chat template, each line is an independent completion. And the single-file build covers Linux/arm64 only; macOS and x86 build from source or use the 2 MB CLI with any GGUF.

Everything is Apache-2.0: geisten/geistlib. The comparison methodology, more recordings, and the fairness controls are in docs/DEMOS.md.