diffusers-workflow

# Diffusers Workflow

This skill provides comprehensive guidance for working with diffusion models using the Hugging Face Diffusers library.

## When to Activate

- Generating images with Stable Diffusion
- Training diffusion models
- Implementing custom samplers
- Using ControlNet or IP-Adapter
- Video generation with diffusion
- Model distillation and optimization

## Pipeline Usage

### Basic Image Generation

```python
from diffusers import StableDiffusionXLPipeline
import torch

pipe = StableDiffusionXLPipeline.from_pretrained(
    "stabilityai/stable-diffusion-xl-base-1.0",
    torch_dtype=torch.float16,
    variant="fp16",
).to("cuda")

# Enable memory optimization
pipe.enable_model_cpu_offload()

# Generate image
image = pipe(
    prompt="A majestic lion in the savanna at sunset",
    negative_prompt="blurry, low quality",
    num_inference_steps=30,
    guidance_scale=7.5,
).images[0]

image.save("lion.png")
```

### With LoRA

```python
from diffusers import StableDiffusionXLPipeline

pipe = StableDiffusionXLPipeline.from_pretrained(
    "stabilityai/stable-diffusion-xl-base-1.0",
    torch_dtype=torch.float16,
).to("cuda")

# Load LoRA weights
pipe.load_lora_weights("path/to/lora", weight_name="lora.safetensors")

# Adjust LoRA scale
pipe.fuse_lora(lora_scale=0.8)

image = pipe("A portrait in the style of <lora_trigger>").images[0]
```

## ControlNet

### Basic ControlNet

```python
from diffusers import StableDiffusionControlNetPipeline, ControlNetModel
from diffusers.utils import load_image
import cv2
import numpy as np

# Load ControlNet
controlnet = ControlNetModel.from_pretrained(
    "lllyasviel/sd-controlnet-canny",
    torch_dtype=torch.float16,
)

pipe = StableDiffusionControlNetPipeline.from_pretrained(
    "runwayml/stable-diffusion-v1-5",
    controlnet=controlnet,
    torch_dtype=torch.float16,
).to("cuda")

# Prepare control image (Canny edges)
image = load_image("input.png")
image = np.array(image)
edges = cv2.Canny(image, 100, 200)
control