Stable Diffusion is a deep learning application that creates images from text prompts. Released in 2022, the product requires significantly more computing power than the Raspberry Pi.This amazing feat created by the makers and developers vita plantamura He successfully ran a 1 billion parameter stable diffusion model on a Raspberry Pi Zero 2 W using only 260MB of RAM.
This is a great feat, mainly because the Raspberry Pi Zero 2 W is known more for its compact design than its computing power. But not only are they physically smaller, but their performance is limited by hardware limitations. In this case, the Pi Zero 2 W has just 512MB of RAM. Stable Diffusion requires at least 8GB of RAM for optimal performance.
Plantamura was able to do this by developing what he calls OnnxStream. This is a bespoke inference library designed to reduce memory consumption so that images can be generated on the Raspberry Pi Zero 2 W. Is it practical? I’m not sure exactly, because the image takes quite a while to create. It may take him 3 hours to generate an image created using his VAE decoder with W8A8 accuracy. However, it works and is quite impressive in that respect. According to Plantamura, OnnxStream has 55% less memory consumption than OnnxRuntime, but only twice as fast at maximum.
OnnxStream has a lot of useful features for those who want to create with tools. Some examples include attention slicing, both dynamic and static quantization, FP16 support, and even an inference engine decoupled from WeightsProvider. A complete list of features can be found here: Official project page.
This software is completely open source and can be explored and modified by anyone. It can run on the Raspberry Pi Zero 2, but users can also implement it on other more powerful systems with the added benefit of better memory consumption.The project has instructions Github A page with detailed instructions on how to set it up on Windows, Mac, and Linux machines.
If you want to know more about this, raspberry pi project Or learn more to get a better idea of how they work together. Onnx Stream Be sure to follow us on GitHub and Vito Planta Mura for future updates.