Local AI does not always require a new workstation. We used an older OnePlus 5T to test whether a compact Qwen 2.5 model could produce useful text on a device that predates the current wave of on-device AI.
Why older devices matter
Existing hardware can offer privacy, offline availability and a longer useful life. The question is not whether it can match a server, but whether it can meet a clearly defined local task with acceptable latency and energy use.
What we observed
Quantisation does not automatically make every device faster. On older hardware, the work required to unpack lower-bit weights can offset the benefit of moving less data. In our experiment, the FP16 variant could be faster than an 8-bit variant for the tested configuration.
Lessons
- Match the model format to the actual processor and memory path.
- Test small models in the 0.5B to 1B range for constrained devices.
- Measure the complete interaction, not only a theoretical throughput number.
Conclusion
Older devices can still support useful local AI. The result depends on careful measurement and a task that respects the hardware’s limits.