In this blog, I’ll guide you through the process of setting up and running TinyStories on your Raspberry Pi 5.

What is TinyStories?

TinyStories is a small language model trained on a dataset of simple, child-friendly stories. Despite its modest size, it showcases impressive capabilities in generating coherent and creative text. TinyStories is designed to be computationally efficient, requiring less than 10 million parameters, which makes it ideal for low-resource devices like the Raspberry Pi.

Why Use TinyStories on a Raspberry Pi 5?

The Raspberry Pi 5 features improved processing power and support for AI workloads:

• Quad-core Cortex-A76 CPU: Fast enough to handle small language models.
• GPU Acceleration: Enables efficient use of AI frameworks.
• Low Power: Ideal for always-on applications.
• Compact Form Factor: Makes it perfect for embedded systems and personal projects.

Applications include:

• Story Generators: Create fun and educational content.
• Creative Prompts: Assist with brainstorming and writing.
• Edge AI Development: Experiment with AI directly on a local device.

Prerequisites

Before starting, ensure you have the following:

• A Raspberry Pi 5 with Raspberry Pi OS (64-bit) installed.
• An active internet connection.
• Python 3.9 or higher installed.
• At least 4 GB of RAM (8 GB recommended for smoother performance).

Step 1: Update Your System

Ensure your Raspberry Pi is up to date:

sudo apt update
sudo apt upgrade -y

Step 2: Install Python and Create the TinyStories Project

Create the project:

make tinystories
cd tinystories
sudo apt install python3-pip
python3 -m venv venv
# Activate the virtual environment
. venv/bin/activate

Step 3: Download the TinyStories Model

Download the pre-trained weights:

wget https://huggingface.co/roneneldan/tinystories-models/resolve/main/tinystories-9M.pt

Step 4: Required Libraries

TinyStories requires Python and a few dependencies:

pip3 install torch transformers

If your Raspberry Pi supports GPU acceleration, install a PyTorch version optimized for it:

pip3 install torch torchvision --extra-index-url https://download.pytorch.org/whl/raspbian

Step 5: Load and Run TinyStories

Create a Python script to load the model and generate text. Save this as run_tinystories.py:

from transformers import AutoModelForCausalLM, AutoTokenizer
import datetime

# Load the model and tokenizer
model = AutoModelForCausalLM.from_pretrained('roneneldan/TinyStories-33M')
tokenizer = AutoTokenizer.from_pretrained("EleutherAI/gpt-neo-125M")

# Check if `pad_token_id` is set; if not, assign it to `eos_token_id`
if tokenizer.pad_token_id is None:
    tokenizer.pad_token_id = tokenizer.eos_token_id  # Use `eos_token_id` as the padding token if none is defined.

# Define the prompt
prompt = "Once upon a time there was"
input_ids = tokenizer.encode(prompt, return_tensors="pt")

# Create the attention mask to ensure padding tokens are ignored
attention_mask = (input_ids != tokenizer.pad_token_id).long()

# Measure the start time
start = datetime.datetime.now()

# Generate the text
output = model.generate(input_ids, max_length=1000, num_beams=1, attention_mask=attention_mask)

# Measure the end time
stop = datetime.datetime.now()

# Calculate the number of tokens generated
output_text = tokenizer.decode(output[0], skip_special_tokens=True)
generated_tokens = len(output[0])  # Count the number of tokens in the generated output
duration = (stop - start).total_seconds()  # Calculate the time taken in seconds

# Calculate tokens per second
tokens_per_second = generated_tokens / duration if duration > 0 else 0

# Print the results
print(f"Generated text: {output_text}")
print(f"Duration: {duration:.2f} seconds")
print(f"Generated tokens: {generated_tokens}")
print(f"Tokens per second: {tokens_per_second:.2f}")

Run the script:

python3 run_tinystories.py

Conclusion

With the Raspberry Pi 5’s enhanced capabilities, running TinyStories opens up a world of possibilities for AI applications at the edge. Whether you’re building a story generator, experimenting with AI creativity, or teaching AI basics, TinyStories provides a powerful and accessible tool.

Let me know what projects you come up with or if you encounter any issues during setup. Happy tinkering!