TinyStories is a collection of short, child-friendly narratives that researchers use to train and test language models in a focused environment. Its primary goal is to evaluate a model’s core language understanding and generation skills, while still encouraging the creation of smaller, efficient models.
We use TinyStories as a quick benchmark to compare our geist-1 model with a standard PyTorch-based model. Both models were trained on the same dataset using the Llama 2 setup, and although they are only partially trained, they can already complete text in ways that make them directly comparable. Each model was trained on 1,000,000 tokens and has about 110 million parameters.
Each model is running on dedicated minimal hardware: a Raspberry Pi 5 with a 64-bit quad-core Arm Cortex-A76 processor at 2.4 GHz.
Enter the first words of a story and see how both models complete it:
The geist-1 model
The standard pytorch model (llama2)
Have more questions or want to learn how our solutions can help your project? Contact us and we’ll be happy to assist.