Diffusion models can cut generation latency by reducing their usual 20 to 50 denoising steps to four, eight, or sometimes one through step distillation.
2
Quantization reduces memory use and can improve performance, while caching avoids recomputing latent regions that barely change between denoising steps.
3
Quantization, caching, multi-GPU techniques, and distillation can be combined, and NVIDIA used this stack to approach real-time video generation on one Blackwell B200.
Summary
Ziv Ilan explains how NVIDIA is making image and video diffusion practical for developers and enterprises. Diffusion models often need 20 to 50 denoising steps, so Ilan focuses on three ways to reduce the cost: quantization, caching, and step distillation. Dynamic post-training quantization can lower memory use and improve performance, although attention-heavy diffusion models require more care than language models. Caching skips work for latent regions that change very little between steps. Distillation trains a student with the same parameter count to produce similar results in far fewer steps. Ilan compares trajectory-based distillation, which follows the teacher's path, with distribution-based distillation, which only requires the student to reach the same output. He describes distribution-based methods as more common and higher quality. NVIDIA's FastGen repository helps manage post-training and GPU sharding for large models. Ilan says the methods can be added incrementally and produced near-real-time video on one Blackwell B200.
Diffusion latency comes from repeating denoising many times
Ilan describes diffusion generation as an iterative denoising process that commonly takes 20 to 50 steps for an image or video. Better models have created practical use cases, but developers and enterprises still need generation to be fast, mature, and scalable. Real-time image and video generation would enable uses such as robotics world models, computer games, and content creation. The difficulty grows with high-resolution output such as 720p or 1080p, where both the first result and the final quality take time. Ilan organizes his optimization discussion around quantization, caching, and distillation.
Dynamic quantization lowers memory use with limited deployment work
Ilan presents quantization as the simplest starting point. He distinguishes post-training quantization from quantization-aware training, and says diffusion models need more care when preserving image and video quality. His example comes from work with Black Forest Labs on Flux 2, using dynamic quantization. Instead of calculating fixed parameter ranges once, the dynamic approach computes some ranges during execution so they better match the data distribution. NVIDIA provides examples in its TRT-LLM VisualGen repository and offers pre-quantized checkpoints through Hugging Face. Quantization can reduce memory use enough to run on lower-end consumer or data-center GPUs, while also improving performance on newer hardware.
Caching in diffusion cannot simply copy the token-oriented KV-cache methods used for autoregressive models. During denoising, however, some areas change very little between steps. T-Cache compares those changes and avoids recomputing regions when the difference is below a threshold. Newer methods work on chunks rather than the entire pixel or latent space. Ilan uses a classroom example: a mostly still audience can remain cached while the speaker's changing area is recalculated. The threshold controls the tradeoff. Caching is available through NVIDIA's TRT-LLM VisualGen support and through serving libraries such as vLLM-Omni. Ilan warns that an aggressive or poorly chosen threshold can damage image quality.
Step distillation trades fifty denoising steps for a few
Step distillation keeps the model's parameter count but trains it to produce similar images or videos with fewer denoising steps. A student can be trained to work in four steps, eight steps, or sometimes one shot instead of the usual fifty. Ilan estimates that preserving quality while cutting the step count can produce a 10x to 200x performance improvement. He presents this as the method most likely to reach good-quality real-time generation. NVIDIA demonstrated near-real-time video generation at GTC using distillation techniques. The work matters because lower latency can support streaming and interactive applications.
Distribution-based distillation lets the student choose its own path
Ilan separates distillation into trajectory-based and distribution-based approaches. In trajectory-based training, the student learns to follow the teacher's denoising path. In distribution-based training, the student only needs to reach the same output distribution, so it can learn a different route. Ilan says distribution-based methods are currently more common and produce better quality. He also describes hybrid approaches, including one in the latest FastGen release that combines the ideas to preserve quality while making training more stable. Distillation is harder to deploy because it is a post-training process that needs data, compute, time, and experience. The result depends on the quality and relevance of the training data.
FastGen packages the difficult parts of large-model post-training
FastGen is an open-source repository from NVIDIA Research rather than a single distillation algorithm. Ilan says large video diffusion models can have 20, 30, or 40 billion parameters, with larger models expected later. Post-training such models also requires sharding across multiple GPUs. FastGen structures that process so users can focus on quality and the training recipe. Open-source data can work for general demonstrations, but a specialized use case may need its own data distribution. Ilan gives protein generation as an example where general data may not be enough. He says evaluation is necessary to understand what the chosen data changes.
Ilan's main deployment advice is incremental. A team can start with quantization because it is simpler, then add multiple-GPU execution or context parallelism, followed by caching. Distillation usually has the largest effect, but it also demands the most training work. The methods do not require choosing only one. NVIDIA has added support for open-source model families including Flux 2 and LTX 2, with more support in progress. Ilan also expects autoregressive techniques and hybrid autoregressive-diffusion designs to become more relevant, such as systems that generate a frame with diffusion and then produce later frames autoregressively.
"The more common, I would say, and better quality technique these days is distribution based."11:45
Who should watch
You are deploying image or video diffusion and need lower latency or lower GPU memory use.
Your team is deciding whether to start with quantization, caching, or step distillation and wants to understand the tradeoffs.
You are training large video diffusion models and need an open-source workflow for post-training and GPU sharding.
