Research

Research Articles

Benchmarks, guides and engineering notes on efficient local AI, with methods and limitations.

9 publicationsLatest: 19 October 2026

Perspective 19 October 2026 24 min read

Why I Am Building Another Inference Engine

llama.cpp exists, works and is excellent. So why write geistlib, a separate inference engine in C? Because making a model fit and run well on small consumer hardware, with fewer bits per weight, a smaller cache and smart scheduling across GPU and CPU, deserves an engine built around exactly that. This article explains, level by level, what that focus changes, where it is faster, where it is not yet, and what it rules out.
Release 06 August 2026 3 min read

A 2B Language Model as One File on a 4 GB Raspberry Pi

geist v0.8.2 ships Microsoft's BitNet b1.58 2B-4T as a single 1.2 GB executable for the Raspberry Pi 5 — model compiled in, zero-copy, offline after download. Measured on the 4 GB reference board, side by side with Microsoft's own bitnet.cpp, including what broke along the way.
Perspective 25 July 2026 9 min read

AI Everywhere Requires Local AI

AI everywhere is not a chatbot in every object. It means placing the smallest useful intelligence where the work happens. Our conclusion is local-first, not local-only: keep bounded tasks close to the device, escalate deliberately and measure the complete system.
Engineering 24 July 2026 22 min read

Running a 2B BitNet Model on a Raspberry Pi 5: Speed, Power, and Limits

I built a small inference engine to learn which optimizations help a 2B BitNet model on a 4 GB Raspberry Pi 5. geistlib reaches about 18 tokens/s and outpaces the tested bitnet.cpp baseline. The remaining measurements ask how close that result is to the hardware limit, what the model can answer and what its internal power telemetry can—and cannot—tell us.
Benchmark 14 July 2026 4 min read

A Ternary 2B Model on a Raspberry Pi, Faster Than You Can Read

BitNet b1.58 2B-4T generates 17.4 tokens per second on a Raspberry Pi 5 with the geisten runtime — roughly three times faster than a person reads. The setup, the numbers, and what still goes wrong.
Engineering 25 March 2026 2 min read

From Model to Asset: A Technical Guide to EU Compliance and Deployment

Self-hosting an LLM in Europe is only the first step. A dependable system connects licensing, data protection, model provenance, runtime configuration and a signed deployment chain.
Engineering 05 March 2026 1 min read

Speculative Decoding: How AI Models Respond Faster

Speculative decoding pairs a fast draft model with a stronger verifier. The technique can reduce generation latency while preserving the output accepted by the target model.
Benchmark 05 December 2024 1 min read

Small Language Models: Why Less Can Be Better

TinyStories and Phi-3 show why compact language models matter. They make it possible to study inference, training objectives and deployment constraints without requiring frontier-scale infrastructure.
Guide 07 August 2024 1 min read

Run Llama 3 Locally in Five Minutes

Ollama provides a simple way to run Llama 3 on a local machine. This short guide covers the minimal setup and the operational questions that matter after the first prompt.