Stable Diffusion Runs in Only 260MB of RAM on Raspberry Pi

Stable Diffusion is a deep learning application that creates images from text prompts. Released in 2022, the product requires significantly more computing power than the Raspberry Pi.This amazing feat created by the makers and developers vita plantamura He successfully ran a 1 billion parameter stable diffusion model on a Raspberry Pi Zero 2 W using only 260MB of RAM.

This is a great feat, mainly because the Raspberry Pi Zero 2 W is known more for its compact design than its computing power. But not only are they physically smaller, but their performance is limited by hardware limitations. In this case, the Pi Zero 2 W has just 512MB of RAM. Stable Diffusion requires at least 8GB of RAM for optimal performance.

Plantamura was able to do this by developing what he calls OnnxStream. This is a bespoke inference library designed to reduce memory consumption so that images can be generated on the Raspberry Pi Zero 2 W. Is it practical? I’m not sure exactly, because the image takes quite a while to create. It may take him 3 hours to generate an image created using his VAE decoder with W8A8 accuracy. However, it works and is quite impressive in that respect. According to Plantamura, OnnxStream has 55% less memory consumption than OnnxRuntime, but only twice as fast at maximum.

OnnxStream has a lot of useful features for those who want to create with tools. Some examples include attention slicing, both dynamic and static quantization, FP16 support, and even an inference engine decoupled from WeightsProvider. A complete list of features can be found here: Official project page.