Stable Diffusion is a text-to-image AI model that has gained immense popularity in recent months. It allows users to generate stunning images from text prompts, making it a powerful tool for artists, designers, and researchers. While Stable Diffusion has traditionally been used with Python and PyTorch, it is now possible to run it efficiently with ONNX (Open Neural Network Exchange).
What is ONNX?
ONNX is an open-source format for representing machine learning models. It enables models to be exported from one framework and deployed in another, ensuring interoperability and portability. By converting Stable Diffusion to ONNX, we can take advantage of its cross-platform compatibility and accelerate inference on various hardware, including CPUs, GPUs, and edge devices.
Benefits of Using ONNX for Stable Diffusion
* **Improved Performance:** ONNX optimizations reduce latency and improve throughput, making image generation faster.
* **Cross-Platform Compatibility:** ONNX allows Stable Diffusion to run on a wide range of platforms, including Windows, Linux, and macOS.
* **Hardware Acceleration:** ONNX enables the use of hardware accelerators, such as GPUs and TPUs, to further enhance performance.
* **Reduced Dependency on Python:** Running Stable Diffusion with ONNX eliminates the need for Python and PyTorch, simplifying deployment and maintenance.
How to Run Stable Diffusion with ONNX
1. **Convert the Stable Diffusion Model to ONNX:** Use the official ONNX converter to export the Stable Diffusion model to ONNX format.
2. **Install ONNX Runtime:** Download and install ONNX Runtime, a library for executing ONNX models.
3. **Load the ONNX Model:** Create an instance of ONNX Runtime and load the converted Stable Diffusion model.
4. **Generate Images:** Utilize the ONNX Runtime API to generate images from text prompts.
Example Code
The following code snippet demonstrates how to run Stable Diffusion with ONNX in C++:
“`cpp
#include
#include
int main() {
// Load the converted ONNX model
OrtSession* session = OrtCreateSessionFromFile(onnx/stable-diffusion.onnx);
// Create an input tensor for the text prompt
const char* prompt = A beautiful landscape with mountains and a lake;
OrtValue* input_tensor = OrtCreateTensorAsOrtValue(session, input,
ONNX_TENSOR_ELEMENT_TYPE_STRING,
nullptr, 1, &size_t(1));
OrtSetTensorElementAsString(input_tensor, 0, &prompt, sizeof(char*) * 1);
// Run inference
OrtValue* output_tensor = nullptr;
OrtRun(session, nullptr, &input_tensor, 1, &output_tensor, 1);
// Get the generated image
int64_t num_rows, num_cols, num_channels;
OrtGetTensorDimensions(output_tensor, &num_rows, &num_cols, &num_channels);
float* data = OrtGetTensorData
// Save the generated image
std::ofstream image_file(image.png, std::ios::binary);
image_file.write((char*)data, num_rows * num_cols * num_channels * sizeof(float));
// Cleanup
OrtReleaseValue(input_tensor);
OrtReleaseValue(output_tensor);
OrtCloseSession(session);
return 0;
}
“`
Conclusion
Running Stable Diffusion with ONNX offers numerous advantages, including improved performance, cross-platform compatibility, and reduced dependency on Python. Developers can leverage this approach to seamlessly integrate Stable Diffusion into their applications and explore its capabilities on various hardware platforms.
Kind regards,
J.O. Schneppat.
