Efficient AI for small computers
We want language models to run well on cheap and edge hardware: an ordinary CPU, a few gigabytes of memory, no cloud. For that we build geistlib, an inference engine in C made for exactly these devices, and geistr, a client that picks the model that fits your machine and runs it.
# macOS (Apple Silicon) $brew install geisten/tap/geistr# Linux (x86_64, arm64) $curl -fsSL https://www.geisten.net/download/geistr/install.sh | sh# then $ geistr chat
Open source. Windows is not supported yet. For development in Python: pip install geistr
Why small computers
Most AI runs in data centres. That costs money for every request, needs a network and sends your data somewhere else. Yet many of the computers that could use AI are small: laptops without a large GPU, single-board computers, industrial PCs, devices in the field.
We think far more is possible on them than is used today. Small open models keep getting better, and quantised and ternary weights cut memory and bandwidth. What often holds them back is software built for large machines.
geisten is where we test this in the open. We build, we measure, and we publish the numbers, including the ones that go against us.
What we build
Both are open source. geistr is built on geistlib; you can also embed geistlib in your own program.
The engine · geistlib
A C23 inference engine, shipped as a static library with no runtime dependencies. It loads GGUF models and produces tokens, nothing more.
The client · geistr
A chat client for the terminal on top of geistlib: one 1.8 MB program. It suggests a model that fits your RAM, downloads and verifies it, and continues your last conversation.
geistr catalog: models ranked for your memory and speedgeistr bench: measures your own machine, not ours
geistr catalog with the speeds measured on this machine, a chat with Qwen3 0.6B on the CPU (thinking turned off with Qwen3's /no_think), /model gemma4-e2b in the same conversation, and geistr bench at the end. Apple M1 Max, geistr 0.3.3, recorded 13 October 2026. The models' answers are not fact-checked.Evidence
geistr runs on geistlib, our own inference engine. On the CPU, geistlib generates text faster than llama.cpp in every model and machine of our nightly benchmark. Reading the prompt is usually slower. Code, raw data and methods are public.
| Hardware | geistlib | bitnet.cpp | Ratio |
|---|---|---|---|
| Raspberry Pi 5, 32-token prompt | 17.9 tok/s | 9.1 tok/s | 1.96× |
| AMD Ryzen 9 9950X, 128 tokens | 103.1 tok/s | 54.3 tok/s | 1.90× |
Same weights and a cool board for both engines. At a 512-token prompt geistlib decodes 15.0 tok/s on the Pi 5; the baseline's benchmark has no like-for-like number there. Measured 1 August 2026 (Pi 5, 4 GB, geistlib 9b24b1f) and 13 July 2026 (9950X, 16 threads), against bitnet.cpp master. The bitnet.cpp commit was not recorded; a re-run against a pinned version is planned.
| Model | Hardware | Decode | Prefill |
|---|---|---|---|
| Qwen3 0.6B · Q8_0 | Ryzen 9 9950X | +9 % | +18 % |
| Qwen3.5 0.8B · Q8_0 | Ryzen 9 9950X | +15 % | −16 % |
| Gemma 4 E2B · Q4_K_M | Ryzen 9 9950X | +19 % | −16 % |
| Gemma 4 E4B · Q4_K_M | Ryzen 9 9950X | +16 % | −19 % |
| Llama 3.2 3B · Q4_K_M | Ryzen 9 9950X | +8 % | −26 % |
| Qwen3.5 4B · Q4_0 | Ryzen 9 9950X | +13 % | −18 % |
| Qwen3.8 27B · Q4_0 | Ryzen 9 9950X | +8 % | −28 % |
| BitNet b1.58-large · TQ2_0 | Ryzen 9 9950X | +48 % | +61 % |
| Qwen3 0.6B · Q8_0 | Raspberry Pi 5 | +6 % | −62 % |
| Qwen3.5 0.8B · Q8_0 | Raspberry Pi 5 | +19 % | −31 % |
Speed of geistlib relative to llama.cpp release b11429 (v0.6.0, 5 October 2026, commit d81235049), the newest release at the time of the runs: decode is text generation (tg64 at a 512-token context), prefill is reading the prompt (pp512). Newest nightly run per cell, 9 to 12 October 2026, geistlib 53aacaacb, bce40a4bd or 4d6e2696a. Same model file for both engines, same threads, median of 4 alternating cycles × 3 samples. Speed only, not a quality comparison. On the GPU (Vulkan, RTX 2080 Ti) the picture is mixed: decode is behind llama.cpp in 6 of the 7 models measured against the same commit. Apple Silicon is not in the nightly matrix; in earlier runs geistlib was behind llama.cpp there.
Live benchmark matrix: nightly runs per model and machine, with history and every run's report.
llama-cli with the libraries it loads: about 23 MB (Homebrew build of the same release b11429, Apple M1). llama-cli also contains server and multimodal code.Status
geistr and geistlib are at an early stage. This is where we stand.
Works well
Not solved yet
Next for geistr: agents you can build into your own applications, then a memory. Both are plans, not features.
Who builds this
Dr. Germar Schlegel, physical chemist and C engineer, writes geistlib and geistr in Kaarst near Düsseldorf. More about geisten
We are looking for people who run AI on small hardware, want to try geistr on their devices, or work on inference engines, quantisation and small models. Questions, ideas and numbers from your hardware are welcome. And if your company wants to run small models on its own devices, we help with that too.
Ways to join in
geistr bench on your device and share the numbers.