ComfyUI breaks generative AI pipelines into editable nodes, giving users control over models, prompts, masks, samplers, and other processing steps.
2
Workflows can run locally or use API nodes, and generated media stores the workflow metadata so it can be imported and shared.
3
The interface is difficult for beginners, while the team is working on easier onboarding, smoother custom-node installation, workflow subgraphs, and cloud execution.
Summary
Yedri Kosinski, known online as Comfy Anonymous, introduces ComfyUI and demonstrates how its node graph represents a diffusion pipeline. A basic image workflow separates the diffusion model, text encoder, VAE, sampler, decoder, and image-saving steps. The same structure extends to video and other media. Kosinski explains classifier-free guidance, latent-space generation, LoRAs, control nets, masking, and newer edit models such as FLUX Context. The workshop also covers API nodes for models that cannot yet run locally, community custom nodes, and the possibility of turning workflows into applications or cloud services. ComfyUI's strengths are control, local execution, open source code, and a large community ecosystem. The speakers are direct about its main weakness: node-based workflows can be hard to learn. Planned work includes better installation of custom-node dependencies, a more traditional interface layered over graphs, subgraphs, and cloud inference. The live demonstrations are limited by old models, missing model files, slow laptop hardware, and a lack of internet access.
ComfyUI is an open-source canvas for many kinds of generative AI
Kosinski describes ComfyUI as an open-source, node-based design canvas for multimodal creative applications. It supports image, video, audio, 3D, text, and other generative AI models. Local open-source models can run on Nvidia, AMD, and Intel hardware, while API-accessible closed-source models can be used remotely. Community custom nodes extend the platform when the core team has not implemented a feature. The speakers also describe workflow sharing as a major part of the product: generated images and video contain metadata that lets a user drag the media back into the canvas and recover the original workflow and its parameters.
Workflow metadata and community extensions make ComfyUI easy to share
The speakers connect ComfyUI's growth to the way workflows travel with generated media. They say a simple search produces many workflow pages, with much of that material remaining compatible with newer versions. They report 78,000 GitHub stars, 3 to 4 million active users, 20,000 daily downloads, and 22,000 custom nodes made by 3,000 public developers. The platform has also been adopted by companies including Amazon, Apple, Tencent, and Netflix. Kosinski says the appeal comes from control beyond a text prompt, including depth maps, line art, masks, and other model inputs.
The node graph exposes the parts of a diffusion pipeline
In the live demonstration, the speakers break a basic Stable Diffusion workflow into a diffusion model, a CLIP text encoder, a VAE, a sampler, a VAE decode step, and an image-saving node. This lets users inspect and change the models they have installed and understand how the image moves through the pipeline. They show an older SD 1.5 workflow running locally, then compare it with an SDXL example. A video workflow has a similar shape, with a sampler and VAE decode node, even though its settings and model requirements differ.
Custom nodes can patch a pipeline with new sampling behavior
Kosinski adds a node that changes the classifier-free guidance calculation, calling it a CFG trick. His point is that users can patch an existing pipeline by writing a node or installing one made by someone else. He explains CFG as sampling from the positive prompt and the negative prompt, then using the difference to push the result toward the positive content and away from the negative content. A CFG scale of one effectively uses only the positive prompt, which produces a less defined and more chaotic image in his demonstration. Raising the scale pushes the sampling more strongly toward the positive prompt and away from the negative one.
Latent-space generation makes diffusion more practical
The VAE matters because Stable Diffusion generates images in compressed latent space rather than directly in pixel space. Kosinski explains that, for this model, the VAE compresses both image dimensions by eight. A 512 by 512 RGB image therefore becomes a 64 by 64 representation with four channels for sampling. Earlier diffusion models that worked in pixel space were slower, while latent-space sampling reduced the amount of data the diffusion process had to handle. In ComfyUI, the VAE node handles the movement between the image representation and the latent representation.
Image quality is hard to automate because taste is subjective
Asked how to evaluate generated images automatically, Kosinski says the first problem is defining a good image. One person's preference may differ from another's. He recalls experiments at Stability AI that used models to rate workflow outputs, but says they did not work very well. He also describes a practical problem with older community fine-tunes: if a model is slightly broken, an early denoising step can create a skin-colored blob that later converges toward an unintended nude image. The speakers recommend reviewing results from such models before displaying them.
ComfyUI can turn workflows into applications, although the official path is unfinished
Kosinski says the visible interface sits on top of a backend that executes workflows. Third-party inference services already let users take a workflow and build an application around it, but Comfy does not yet offer an official route. API nodes provide another option for remote models, though they require internet access and paid credits. The team expects to build its own inference service eventually. This would support applications where a power user creates a controlled workflow and other users provide only approved parameters.
The team plans a friendlier layer over the graph and cloud execution
The speakers describe planned work around cloud execution, custom-node dependencies, and usability. They want installing and managing custom nodes to be smoother, since nodes can request different Python packages. They also plan subgraphs, which combine several nodes into one node that can be opened as a separate workflow. A more traditional interface may sit above the node graph so users can build an application without exposing every underlying control. The team does not promise a fixed roadmap because new models can change its priorities. Kosinski says the first requirement is a cloud inference service where workflows and their custom nodes run reliably.
"The thing is in this space is that things change a lot, so a new model that comes out tomorrow might mean we need to pivot a bit."49:00
Who should watch
You are building image or video generation tools and need to inspect or modify more than a prompt and a model name.
You want to run generative AI locally, share reproducible workflows, or add community nodes to a visual pipeline.
You need to decide whether a node-based interface fits your team, and want to understand its current limits around onboarding and deployment.
