Many workflows contain independent operations: translate two texts, classify several documents or prepare a summary while another component retrieves sources. Running them sequentially wastes available hardware and increases the time until the complete result is ready.
A simple comparison
We tested two small Qwen models on an Apple M1 Max. A single task completed in roughly 0.14 seconds. When two tasks ran in parallel, per-task latency rose to roughly 0.18 seconds, but both results were available after 0.18 seconds rather than after two sequential runs. The relevant metric is therefore not only individual latency; it is the latency of the completed workflow.
Memory traffic matters
For local inference, memory bandwidth can dominate. Lower-bit weights reduce traffic and can fit more effectively into cache, while dequantisation introduces its own work. The result depends on the hardware, runtime and model. It must be measured rather than assumed from a precision label.
Practical implications
Parallelism helps when tasks are independent, the runtime can schedule them efficiently and the machine has enough memory bandwidth. It is not a substitute for good batching, task design or capacity planning.
Figures
Conclusion
Small specialist models can increase throughput without making the system harder to understand. The decision should be based on end-to-end latency distributions, not only average token speed.