Open by default
Code, methodology and limitations are part of the result.
Local AI benchmarks / Düsseldorf, Germany
What runs
locally.
Efficiency is a feature, not a compromise. A bee doesn't carry a supercomputer in its head — it has just enough intelligence to do its job exceptionally well. We engineer AI the same way: our sub-1 MB C23 engine geist makes small models do real work on hardware you already own — whether that's ternary BitNet outpacing Microsoft's runtime on a Raspberry Pi, Gemma 4B answering questions from your own documents, or tiny models with the agent skills to drive real tools.
- 17.4 tok/s
- Raspberry Pi 5, CPU-only
- 2.1×
- faster than bitnet.cpp
- < 1 MB
- engine binary, zero deps
brew install geisten/tap/geist
geist -m bitnet-2b.gguf "What is the capital of France?"
- engine
- one binary < 1 MB, zero deps
- platforms
- macOS / Linux / Raspberry Pi
- models
- BitNet b1.58 / Gemma 4 (GGUF)
- measured
- Pi 5, CPU-only / decode
throughput17.4 tok/s2.1× bitnet.cpp
Read the benchmark
01 / WHY
Research focus
As small as possible. As smart as necessary.
A bee navigates, communicates and builds with a brain of one million neurons — no supercomputer required, just enough intelligence for the job. That is our engineering standard. While much of the industry scales models up, we scale them down. The question behind every geisten project is simple: what is the smallest model that solves this problem?
We don't believe the future is one giant model running everywhere. It's thousands of focused models, each doing one job well, running where the data lives. That's why geisten starts from the constraints — memory, compute and the task itself — and publishes the measurements to prove what works.
Use the results to choose a system. Everything needed to repeat a measurement — code, weights and method — is linked from each entry.
- Energy
- Less compute per decision.
- Latency
- No network round trip for every response.
- Privacy
- Data stays on the device or in the network.
- Cost
- Inexpensive CPUs instead of GPU clusters.
- Offline
- Systems continue to work without a cloud.
- Sustainability
- Small models age on hardware you already own.
02 / HOW
Working principles
Engineering that can be measured.
Benchmarks over claims
A number without hardware, model and method is an impression, not a result.
Local first
Local execution is the starting point, not the add-on.
Small models, fixed tasks
The smallest capable model is often the better system.
Reproducible research
A result becomes useful when it can be repeated.
Practical intelligence
A model becomes useful when it touches real sensors and real interfaces.
03 / RESEARCH
Research journal
Open experiments. Practical progress.
Benchmark / Runtime
A Ternary 2B Model on a Raspberry Pi
17.4 tok/s, CPU-only: a reproducible BitNet benchmark on a Raspberry Pi 5.
Benchmark / RuntimeMeasuring parallel LLM execution
Latency distribution instead of the average: what parallelism actually costs.
Investigation / RAGRAG with small models
When retrieval can replace model size and where it cannot.
Notes / ArchitectureSmall experts, constrained systems
Notes on compact models and their suitability for fixed tasks.
04 / BENCHMARKS
Measurement, not impression
The hardware is part of the result.
Pi 5 / BitNet b1.58 2B-4T17.4 tok/s
Faster than you can read, on an €80 computer, CPU-only. Same weights, same hardware: 2.1× bitnet.cpp. The engine is one binary under 1 MB.
| System | Measurement | geist | Reference |
|---|---|---|---|
| Raspberry Pi 5 / BitNet b1.58 2B-4T | Decode | 17.4 tok/s | 8.2 tok/s bitnet.cpp |
| AMD 9950X / BitNet b1.58 2B-4T | Decode | 103.1 tok/s | 54.3 tok/s bitnet.cpp |
| Raspberry Pi 5 / Gemma 4 E2B-it | Decode | 7.5 tok/s | 6.8 tok/s llama.cpp |
| AMD 9950X / Gemma 4 E2B-it | Decode | 48.6 tok/s | 44.1 tok/s llama.cpp |
| Model | Quantization | Size on disk | RAM required |
|---|---|---|---|
| BitNet b1.58-large | TQ2_0 | 207 MB | ≥ 1 GB |
| BitNet b1.58 2B-4T | i2_s | 1.1 GB | ≥ 4 GB |
| Gemma 4 E2B-it | Q4_K_M | 2.9 GB | ≥ 4 GB |
| Gemma 4 E4B-it | Q4_K_M | 4.6 GB | ≥ 6 GB |
CPU-only, identical GGUF weights, both engines warmed up and started from a cool baseline. References: llama.cpp d05fe1d (Raspberry Pi 5, OpenBLAS) and b9827 (AMD 9950X); bitnet.cpp is microsoft/BitNet master via its bundled llama-bench. Pi 5 measured June 2026 on a quiesced board (gcc 14.2, kernel 6.18); 9950X re-measured 13 July 2026 (16 threads). Energy and CPU-utilization columns are still missing — we are wiring them into the harness now, and they will appear here as measurements, not adjectives.
05 / OPEN SOURCE
Projects in the open
Tools that fall out of the research.
Runtime / C23 / Apache-2.0
geist
A compact inference engine for small CPU systems. Statically linkable, one binary under 1 MB, with hand-written ARM NEON and AVX-512 kernels and a clear measurement interface.
Adapter / Applied research
geist-home-assistant
A voice assistant for Home Assistant: about 2 seconds from voice command to action on a Raspberry Pi 5, fully offline. BitNet has no tool training — geist forces a valid tool call from outside the sampler, so a 2B base model routes and calls whitelisted tools anyway.
06 / ROADMAP
Direction of travel
A public working list.
- Now
Inference runtime and benchmark harness
Measurable CPU inference for small systems: one auditable C runtime, one binary, zero dependencies.
- Now
Ternary and binary models as first-class citizens
Aggressive quantization built into the runtime, not bolted on — optimised for CPUs, small GPUs and the Raspberry Pi 5.
- Now
Agents shaped for small models
Tight harness, fixed algorithms, whitelist-gated tools — evaluated against reproducible tasks.
- Next
Memory systems for small models
Memory shaped for what these models can actually hold, plus efficient vision on embedded hardware.
- Explore
Dynamic task specialization
Adaptive model behaviour: many specialised systems working together instead of one general model.
Build in public
Follow the measurements. Ask better questions.
geisten is open to developers, embedded engineers and researchers who want to make Efficient AI practical. If you measure something different on your hardware, that is not a problem — that is a data point. Discussions, bug reports and reproducible counter-measurements are welcome.