A 2B Language Model as One File on a 4 GB Raspberry Pi

The most honest install instruction is the one with nothing to configure. As of v0.8.2, running a 2-billion-parameter language model on a Raspberry Pi 5 is three lines:

curl -L -o geist-bitnet https://github.com/geisten/geistlib/releases/latest/download/geist-bitnet-linux-arm64
chmod +x geist-bitnet
./geist-bitnet "The three largest moons of Jupiter are"

One file, 1.2 GB. It contains the engine — under 1 MB of compiled C23 — and the complete BitNet b1.58 2B-4T model. No Python, no model download step, no network access after the download. Run it with no arguments and you get a minimal REPL.

How a model fits into an executable

The weights are placed in the binary’s read-only data section at build time and loaded with geist_model_load_from_memory, which aliases them zero-copy. The kernel demand-pages the mapping exactly as it would an mmaped model file, so RAM cost does not change: peak resident memory measured 1.66 GB while generating, on a board with 4 GB. The pages are file-backed and evictable under pressure.

This has two consequences:

Cold start is storage-bound. The first run after boot took 14.3 s to the first token on our SD card, because 1.1 GiB of weights page in once. Every warm start after that took 0.6 s.
The build needs a linker that tolerates a >1 GB section. GNU ld does. Apple’s ld64 corrupted relocations and crashed (measured, reproducibly). This is why the single-file build exists for Linux/arm64 only.

Measured against Microsoft’s own runtime

Side by side on the same Pi 5: geist finishes 110 tokens in 7.1 s, bitnet.cpp needs 11.7 s

Recording conditions:

same 4 GB board and byte-identical GGUF;
both engines greedy at 4 threads;
each take started below 57 °C, with temperatures and commits visible in the frame;
takes recorded sequentially, so neither engine steals CPU from the other.

This recording shows 110 tokens in 7.1 s (15.5 tok/s end-to-end) against bitnet.cpp’s own timing line of 9.31 tok/s.

The frozen 10-repeat protocol says the same thing with error bars: 17.9 tok/s at short prompts and 15.0 at a 512-token context, with a spread under 1 %. New in the harness is energy from phase-separated PMIC probes: 0.44 J per decoded token and 0.20 J per prefill token. The full report is published as a gist, and make bench reproduces it on your hardware, competitor included.

What broke along the way

We hit these problems while producing this release:

bitnet.cpp’s current main mis-decodes the canonical i2_s GGUF on ARM — it emits ???? where text belongs. Speed harnesses like llama-bench never notice; a recording does. Our comparison uses the June 2025 state (404980e), whose greedy output starts token-identical to ours.
gcc 12 crashes (internal compiler error) on bitnet.cpp’s i2_s quantizer — their clang requirement is real.
Small models confabulate fluently. Asked about bleeding a radiator, the model confidently recommended a hairdryer and a vacuum cleaner. The user guide keeps examples like this on purpose: treat output as plausible text, not knowledge.

Limitations, openly

llama.cpp’s OpenBLAS prefill still leads on the Pi. The REPL is deliberately memory-less — there is no chat template, each line is an independent completion. And the single-file build covers Linux/arm64 only; macOS and x86 build from source or use the 2 MB CLI with any GGUF.

Everything is Apache-2.0: geisten/geistlib. The comparison methodology, more recordings, and the fairness controls are in docs/DEMOS.md.