Setting Up Stable Diffusion AI Image Generator on AMD GPUs

Table of Contents

• Introduction to Stable Diffusion AI
• Requirements for Stable Diffusion
• Installing ROCm Kernel Modules
• Setting up the Docker Container
• Cloning the Stable Diffusion Repository
• Downloading Model Checkpoints
• Creating the Conda Environment
• Installing Pytorch with ROCm
• Generating Images with Stable Diffusion
• Conclusion

Introduction to Stable Diffusion AI: An Open-Source Text-to-Image Diffusion Model

Stable Diffusion is an impressive new open-source AI system that can generate highly realistic images from text descriptions. Developed by Stability AI, Stable Diffusion leverages a novel latent diffusion model to produce striking visuals rivaling those from leading proprietary systems like DALL-E 2.

In this detailed tutorial, we will walk through getting Stable Diffusion up and running on an AMD Navi GPU from start to finish, including prerequisites like installing ROCm kernel modules and setting up a Docker container.

What is Stable Diffusion?

Stable Diffusion is a deep learning model trained on text-image pairs to generate images that match text prompts. It builds on latent diffusion models, a type of generative model which starts with random noise and gradually refines it into realistic images through an iterative denoising process. Thanks to its innovative training approach, Stable Diffusion can produce photorealistic 1024x1024 images based on short text descriptions. It also allows for image inpainting and outpainting by conditioning the model on a partial starting image.

Stable Diffusion vs DALL-E 2

While Stable Diffusion takes inspiration from DALL-E 2, there are some key differences. DALL-E is a proprietary system owned by OpenAI and access remains limited. Stable Diffusion is open source, free for non-commercial use, and places no constraints around sharing or using its AI models. In terms of image quality, DALL-E 2 still edges out Stable Diffusion slightly but the open source alternative produces impressive results nonetheless. Stable Diffusion also offers more flexible guidance during image generation through intermediate gradients.

Requirements for Stable Diffusion

There are a few key requirements to run Stable Diffusion smoothly on your system:

1. A GPU with at least 10GB VRAM - This allows fitting the full model into memory for fast diffusion.

2. At least 16GB system RAM - Helps handle the memory demands during model loading and image generation.

3. A modern Linux distro like Ubuntu 20.04 or Arch Linux - Stable Diffusion primarily targets Linux though limited Windows support exists.

4. Docker - We will setup Stable Diffusion in a Docker container to simplify dependencies.

Installing ROCm Kernel Modules for AMD GPUs

ROCm Install on Arch Linux

Since I'm using an AMD Radeon RX 6800 XT GPU, we need to install the ROCm kernel modules on my Arch Linux machine for GPU acceleration:

1. Install rocm-opencl-runtime, rocm-info, docker packages from the Arch User Repository
2. Start the docker service with 'sudo systemctl start docker'

ROCm Install on Ubuntu 20.04

For Ubuntu users, follow these steps:

1. Add the ROCK-Linux repository
2. Install rocm-dkms package
3. Install other rocm packages like hipFFT, rocBLAS, rocSOLVER
4. Reboot the system
5. Verify install with rocminfo command
6. Install docker if you haven't already

Setting up the Docker Container

With ROCm modules installed on the host, we can setup a container to run Stable Diffusion:

1. Pull the pytorch ROCm docker image matching your AMD GPU version

2. Create an alias for convenient docker runs

3. Start the stable-diffusion container using the alias

Cloning the Stable Diffusion Repository

Now from inside our docker container, we will clone the official Stable Diffusion repo from GitHub:

1. Make a directory for the repo files

2. Git clone the https://github.com/Stability-AI/stablediffusion repo

Downloading Model Checkpoints

Stable Diffusion relies on several pre-trained model files to function:

1. Download model checkpoints from https://stability.ai/models matching your VRAM

2. Unzip checkpoints into a new folder under stable-diffusion/models

3. Rename the unzipped folder to stable-diffusion-v1

Creating the Conda Environment

We need a Conda environment to manage python packages and dependencies:

1. Navigate to the stable-diffusion directory

2. Run 'conda env create -f environment.yaml' to setup the environment

This will install all necessary python packages but we still need to overwrite the PyTorch version with a ROCm compatible build.

Installing Pytorch with ROCm

To leverage our AMD GPU, we must install a PyTorch version built for ROCm:

1. Visit pytorch.org and select the ROCm nightly build for pip installation

2. Run pip install command but use --upgrade instead to overwrite system pytorch

3. Restart docker container shell to activate conda environment

Generating Images with Stable Diffusion

With everything setup, we can now use Stable Diffusion to create images!

1. Activate the conda environment in docker shell

2. Run stable-diffusion command with text prompt as argument

For example: 'A cybernetic mushroom sitting on a tree stump trending on ArtStation'

3. The first run will be slow to cache necessary files but subsequent generations will be faster

Conclusion

Getting an open-source AI system like Stable Diffusion running involves quite a few steps. But the payoff is huge - you unlock the ability to generate an endless variety of realistic images powered simply by text descriptions.

For AMD GPU owners, following this guide to install ROCm and setup a docker container makes deployment straightforward. Stable Diffusion represents an exciting milestone in democratizing access to cutting-edge deep learning models through its open source approach.

FAQ

Q: What is the difference between Stable Diffusion and DALL-E 2?
A: Stable Diffusion is an open source AI image generator while DALL-E 2 is a proprietary model by OpenAI. Both can generate images from text prompts.

Q: What hardware do I need for Stable Diffusion?
A: You need an AMD GPU like Radeon RX 6800 XT and a Linux machine. NVIDIA GPUs are currently not supported.

Q: How do I install ROCm for Ubuntu?
A: Add the ROCm repository and install the rocm-dkms package. Also install Docker.

Q: Where do I get the Stable Diffusion model checkpoints?
A: Download the checkpoints from the Stable Diffusion GitHub repo into the models/ldm/stable-diffusion-v1 folder.

Q: What's the purpose of creating a Conda environment?
A: The Conda environment installs all the Python dependencies required to run Stable Diffusion.

Q: Why install Pytorch with ROCm again?
A: To overwrite the CUDA Pytorch version with ROCm compatible Pytorch.

Q: How long does it take to generate an image?
A: The first image takes very long as it downloads additional packages. Subsequent generations are faster.

Q: Where are generated images saved?
A: Images are saved in the outputs folder within the Stable Diffusion directory.

Q: Can I run this on NVIDIA GPUs?
A: Unfortunately no. You need an AMD GPU as Stable Diffusion relies on ROCm.

Q: What Linux distros does this work on?
A: Tested on Ubuntu 20.04 and Arch Linux. Other common distros should work too.