Image generation models are demanding enough on VRAM that local setup used to be genuinely fiddly. Between better tooling and more efficient model variants, it’s now a fairly smooth process even on a mid-range consumer GPU.
Hardware Reality Check
- 6-8GB VRAM — comfortably runs SD 1.5 and SDXL Turbo variants at reasonable speed
- 12GB+ VRAM — runs full SDXL and newer models like SD3/Flux without heavy compromises
- AMD GPUs — supported via ROCm on Linux, though NVIDIA’s ecosystem remains smoother for this specific workload
Installing via ComfyUI
git clone https://github.com/comfyanonymous/ComfyUI cd ComfyUI pip install -r requirements.txt # Download a model checkpoint into models/checkpoints/, then launch python main.py
Or, a Scriptable Python Approach
from diffusers import StableDiffusionXLPipeline
import torch
pipe = StableDiffusionXLPipeline.from_pretrained(
"stabilityai/stable-diffusion-xl-base-1.0",
torch_dtype=torch.float16
).to("cuda")
image = pipe(
prompt="a lighthouse on a cliff at sunset, oil painting style",
negative_prompt="blurry, low quality",
num_inference_steps=30,
guidance_scale=7.5,
).images[0]
image.save("output.png")
Key Parameters That Actually Matter
- Steps — more steps generally improve quality up to a point (20-40 is typical), with diminishing returns and longer generation time beyond that.
- Guidance scale (CFG) — how strictly the model follows your prompt; too high produces oversaturated, artifact-heavy results, too low ignores the prompt.
- Sampler — different sampling algorithms (Euler, DPM++, etc.) trade off speed against quality and produce subtly different aesthetic results for the same prompt.
Why Run It Locally at All
No per-image cost, no content policy beyond your own judgment, full control over models and LoRA fine-tunes, and no dependency on a third-party service staying available — the trade-off is entirely upfront hardware cost and setup time instead of ongoing per-use fees.