ComfyUI: Node-Based Image Generation for People Who Want Control

The first time you open ComfyUI, you will hate it. There’s no friendly prompt box waiting for your words, no big orange Generate button — just a tangle of boxes connected by coloured spaghetti, like someone wired up a modular synthesiser and walked off. I closed it the first time too. Then I went back, because the people producing the most consistent, repeatable, genuinely controllable images locally were all using it, and there’s usually a reason a difficult tool refuses to die.

ComfyUI is a node-based interface for Stable Diffusion. Instead of a form that hides the pipeline behind sensible defaults, it exposes the pipeline itself as a graph you build by hand. Every stage — loading the model, encoding the prompt, sampling, decoding — is a node, and the wires between them are the data flowing through. It is more work, and that is entirely the point.

1 Why a graph instead of a form

A traditional web UI is a wrapper around a fixed pipeline. You get the options the developers chose to surface, arranged the way they decided. That’s fine until you want to do something they didn’t anticipate: run two prompts through the same noise seed, swap the VAE only for the upscale pass, feed one image’s composition into another’s style. In a form, you’re stuck. In a graph, you just route a wire somewhere new.

The graph is also brutally honest about what’s happening. When a render looks wrong, you can see which node produced the bad latent, because you can preview the output at any point in the chain. I have debugged more bad generations in ten minutes of ComfyUI than in an hour of guessing at a checkbox in a form.

2 The anatomy of a basic workflow

The default text-to-image graph has about seven nodes, and once you understand them you understand the whole tool. A Load Checkpoint node reads the model and outputs three things: the model itself, the CLIP text encoder, and the VAE. Two CLIP Text Encode nodes turn your positive and negative prompts into conditioning. An Empty Latent Image node defines your canvas size. The KSampler is the heart — it takes the model, both conditionings, and the empty latent, and does the actual denoising. Finally a VAE Decode turns the finished latent into a picture, and Save Image writes it out.

That’s it. Every fancy workflow you’ll ever see is this skeleton with extra limbs bolted on.

3 Workflows are just files

Here’s the feature that won me over completely: a ComfyUI workflow is a JSON file. The entire graph — every node, every connection, every setting — serialises to plain text you can save, diff, and share. Even better, ComfyUI embeds the full workflow into the PNG metadata of every image it generates.

{
  "3": {
    "class_type": "KSampler",
    "inputs": {
      "seed": 875623491,
      "steps": 25,
      "cfg": 7.0,
      "sampler_name": "dpmpp_2m",
      "scheduler": "karras",
      "denoise": 1.0
    }
  }
}

This means you can drag any ComfyUI image back into the canvas and it rebuilds the exact graph that made it. No more “what settings did I use for that one good render three weeks ago” — the answer is baked into the file. For a tinkerer who never writes anything down, this is borderline miraculous.

4 Getting it running

Installation is refreshingly undramatic. Clone, install dependencies, point it at your existing model folder, and go:

git clone https://github.com/comfyanonymous/ComfyUI
cd ComfyUI
pip install -r requirements.txt
# reuse your existing checkpoints instead of re-downloading
ln -s /mnt/models/checkpoints models/checkpoints
python main.py --listen 0.0.0.0

The --listen flag exposes it on your network so you can drive it from a laptop while the GPU box does the work in another room — exactly how I run mine. There’s also ComfyUI Manager, a community extension that turns custom-node installation from a manual git-cloning chore into a search box, and it’s the first thing I install on any fresh setup.

5 The honest downsides

ComfyUI is power-user software and makes no apology for it. The learning curve is a wall, not a slope. Sharing a workflow with someone who isn’t technical means handing them a JSON file and a prayer. Custom nodes from the community are fantastic right up until one breaks after an update and takes your whole graph down with it. And for the simple case — “give me a picture of a cat” — it is objectively slower and fiddlier than just typing into a box.

6 Is it worth it?

It depends entirely on what you want from the machine. If you generate images occasionally and want them to be good, a form-based UI will serve you better and faster. If you want to understand what image generation is doing, build repeatable pipelines you can version-control, and chain together effects nobody packaged for you, ComfyUI is the best thing going and nothing else is close.

I think of it as the difference between driving an automatic and learning a manual gearbox. Most people are happier with the automatic, and that’s a perfectly reasonable choice. But if you’re the sort who self-hosts more than is sensible and enjoys knowing exactly where every photon went, you already know which one you are. Give it the second chance I nearly didn’t.