A chat that remembers
Close the terminal, come back tomorrow: geistr continues where you left off, and you can switch to a bigger model mid-conversation. Each conversation is a plain file on your computer.
geistr chat
Open source · Linux and macOS · CPU or GPU
Language models on the computer you already have.
geistr picks an open model that fits your memory, downloads and verifies it, and runs it on your laptop or a Raspberry Pi. No Python, no cloud, no account: your conversations stay on your computer.
# macOS (Apple Silicon) $brew install geisten/tap/geistr# Linux (x86_64, arm64) $curl -fsSL https://www.geisten.net/download/geistr/install.sh | sh# then $ geistr chat
Windows is not supported yet. For Python: pip install geistr · geistr on GitHub
/model gemma4-e2b in the same conversation. The answers are not fact-checked.- 4 GBof memory are enough for a 2B model on a Raspberry Pi 5: BitNet b1.58 2B-4T, 1.2 GB download, 15 to 18 tok/s depending on prompt length.
- 1.96×the decode speed of bitnet.cpp on a 4 GB Raspberry Pi 5, BitNet b1.58 2B-4T, same weights.
- +7 to +50 %decode speed against llama.cpp on a Ryzen 9 9950X and a Raspberry Pi 5, in all 13 CPU cells of our nightly benchmark. Where we are slower
What geistr does
A local model you can actually work with.
Models from the catalog (Qwen3, Qwen3.5, Gemma 4, BitNet, SmolLM2 and more) or any supported GGUF file.
Answers from your own files
Index a folder of notes, Markdown or PDFs. Each answer names the files it drew on. The index stays on your computer.
geistr chat --files ~/Notes
Works with the tools you use
An OpenAI- and Ollama-compatible API on your own machine. Connect Open WebUI, editor plugins or the OpenAI SDK as you would to Ollama.
geistr serve gemma4-e2b --http
A model that fits this machine
The catalog ranks models for your memory and the speed measured on your own hardware. Every download is verified by SHA-256.
geistr catalog
Under the hood · geistlib
An inference engine in C, built for small devices.
geistr runs on geistlib. You can embed it in your own program: a static library with no runtime dependencies, and a C API that Python, Rust, Go or JavaScript call through their FFI.
i2_s). Recorded one after the other from a thermally gated board and shown side by side: geistlib v0.8.2 at 15.5 tok/s, bitnet.cpp 404980e at 9.3 tok/s. 4 threads, greedy, 110 new tokens. How it was recorded- Ternary models first. BitNet with integer-only kernels: ARM SDOT on the Pi, AVX-512 VNNI on x86.
- Little memory. Weights memory-mapped and loaded on demand; KV cache in INT8, INT4 or 2 bits.
- CPU first. Metal and Vulkan on GPUs, optional.
Benchmarks
Faster than llama.cpp on x86. Faster at writing on the Pi.
geistlib against llama.cpp b11429 on the CPU, nightly: same model file, same threads. On the Ryzen 9 9950X geistlib is ahead in reading and writing for all eight models; on the Raspberry Pi 5 it writes faster in all five, but reads the prompt slower in four. Decode is writing the answer, prefill is reading the prompt. Speed only, not a quality comparison. Code, raw data and method are public.
| Model | Hardware | Decode | Prefill |
|---|---|---|---|
| Qwen3 0.6B · Q8_0 | Ryzen 9 9950X | +10 % | +22 % |
| Qwen3.5 0.8B · Q8_0 | Ryzen 9 9950X | +18 % | +52 % |
| Gemma 4 E2B · Q4_K_M | Ryzen 9 9950X | +18 % | +22 % |
| Gemma 4 E4B · Q4_K_M | Ryzen 9 9950X | +16 % | +48 % |
| Llama 3.2 3B · Q4_K_M | Ryzen 9 9950X | +7 % | +42 % |
| Qwen3.5 4B · Q4_0 | Ryzen 9 9950X | +12 % | +86 % |
| Qwen3.8 27B · Q4_0 | Ryzen 9 9950X | +8 % | +90 % |
| BitNet b1.58-large · TQ2_0 | Ryzen 9 9950X | +50 % | +28 % |
| Qwen3 0.6B · Q8_0 | Raspberry Pi 5 | +9 % | −18 % |
| Qwen3.5 0.8B · Q8_0 | Raspberry Pi 5 | +18 % | −6 % |
| Gemma 4 E2B · Q4_K_M | Raspberry Pi 5 | +13 % | −4 % |
| Qwen3.5 4B · Q4_0 | Raspberry Pi 5 | +28 % | −33 % |
| BitNet b1.58-large · TQ2_0 | Raspberry Pi 5 | +27 % | +30 % |
Live benchmark matrix: nightly runs per model and machine, with history and every run's report.
Works well
- Writing text on the CPU: faster than llama.cpp in all 13 cells, x86 and Raspberry Pi 5
- Reading the prompt on x86: +22 to +90 % against llama.cpp
- Ternary models on a Raspberry Pi 5: about twice the decode speed of bitnet.cpp
- A 1.8 MB program plus the model file: no Python, no Docker, no account
Not solved yet
- Reading the prompt on the Raspberry Pi 5: slower than llama.cpp in 4 of 5 models
- GPUs (Vulkan): decode behind llama.cpp in 6 of 7 models
- Apple M1 Max, CPU: behind llama.cpp, about −45 % reading and −19 % writing (2 models, one run each)
- Our hypothesis that many small, specialised models can beat one large one: not tested yet
Next for geistr: tools over MCP and reusable skills, then a toolkit for building your own local AI, including typed decisions instead of generated text. Plans, not features.
Method, versions and the bitnet.cpp comparison
Speed of geistlib relative to llama.cpp release b11429 (v0.6.0, 5 October 2026, commit d81235049), the newest release at the time of the runs: decode is text generation (tg64 at a 512-token context), prefill is reading the prompt (pp512). Newest nightly run per cell, 18 to 20 October 2026, geistlib c993379b0 or 44479afd2, BLAS-free release build. Several cells have only one or two runs so far under this build. Same model file for both engines, same threads, median of 4 alternating cycles × 3 samples. Ternary Bonsai 2 27B is measured against a different llama.cpp build and left out here. On the GPU (Vulkan, RTX 2080 Ti) decode is behind llama.cpp in 6 of the 7 models measured against the same commit.
| Hardware | geistlib | bitnet.cpp | Ratio |
|---|---|---|---|
| Raspberry Pi 5, 32-token prompt | 17.9 tok/s | 9.1 tok/s | 1.96× |
| AMD Ryzen 9 9950X, 128 tokens | 103.1 tok/s | 54.3 tok/s | 1.90× |
Same weights and a cool board for both engines. At a 512-token prompt geistlib decodes 15.0 tok/s on the Pi 5; the baseline's benchmark has no like-for-like number there. Measured 1 August 2026 (Pi 5, 4 GB, geistlib 9b24b1f) and 13 July 2026 (9950X, 16 threads), against bitnet.cpp master. The bitnet.cpp commit was not recorded; a re-run against a pinned version is planned.
Size: geistr 0.3.3 is one 1.8 MB binary with CPU and Metal backends, model catalog and downloader. llama.cpp's llama-cli with the libraries it loads is about 23 MB (Homebrew build of release b11429, Apple M1); llama-cli also contains server and multimodal code. Power: 5.9 W average on the Pi's internal sensors while generating (a proxy, not total draw).
Full article, raw data and limits ·Benchmark table, method and caveats on GitHub
Research
Notes from the lab.
Benchmarks, guides and engineering notes, with methods and limitations.
Release / Runtime
A 2B LLM as One File on a 4 GB Pi
curl, chmod, run — Microsoft's BitNet 2B compiled into a single 1.2 GB binary, offline, at 15–18 tok/s.
Research Series / geistlibHow much AI can a 4 GB Raspberry Pi handle?
By building a custom inference engine, I could study how a 2B BitNet model uses the Pi's CPU. It reaches about 18 tokens/s, but speed alone does not show whether a small local model is efficient or useful.
Benchmark / RuntimeA Ternary 2B Model on a Raspberry Pi
17.4 tok/s, CPU-only: a reproducible BitNet benchmark on a Raspberry Pi 5.
Try it on your hardware, and tell us.
We are looking for people who run AI on small hardware or work on inference engines, quantisation and small models. Questions, ideas and numbers from your devices are welcome. If your company wants to run small models on its own devices, we help with that too.
geistlib and geistr are written by Dr. Germar Schlegel, physical chemist and C engineer, in Kaarst near Düsseldorf. More about geisten
Ways to join in
- Run
geistr benchon your device and share the numbers. - Report what breaks in GitHub issues.
- Watch the repositories on GitHub, or ask for an email when a new version is out.