Generative AI image models like Stable Diffusion refine images by iteratively denoising noise across the entire canvas, a technique called diffusion. Applying this approach to text generation at scale had long been challenging. Traditional language models generate text sequentially, one token at a time, limiting speed and flexibility. Google’s new open-source DiffusionGemma model changes this by generating 256 tokens simultaneously, refining them through multiple passes to self-correct mistakes. Built on the Gemma 4 architecture and integrated with the vLLM inference platform, DiffusionGemma can produce text up to 4 times faster on GPUs, reaching over 1,000 tokens per second on Nvidia H100 and over 1,200 on H200.
Unlike typical models, DiffusionGemma generates tokens in parallel and revises uncertain outputs with each pass, enabling bidirectional context that benefits constrained text tasks like coding or puzzle solving. It achieves this efficiency by activating a fraction of its 26 billion parameters per inference, fitting consumer GPUs like the RTX 4090. However, this speed gain comes with a trade-off in output quality compared to standard Gemma 4, making it less suitable for applications demanding the highest text fidelity.
This approach excels in scenarios with single-user or low-concurrency inference where GPUs are underutilized but is less advantageous in high-throughput cloud environments where sequential models saturate GPU resources. DiffusionGemma introduces a structural shift in text generation, offering a promising new option for enterprises needing fast, bidirectional, and flexible text generation on local or dedicated hardware setups.
