ZAYA1-8B-Diffusion-Preview: 7.7x Speedup for LLMs on AMD

Zyphra has released ZAYA1-8B-Diffusion-Preview, showcasing the conversion of an existing language model into a discrete diffusion model. This new approach maintains performance while significantly speeding up inference on AMD hardware. Here's the thing: traditional language models generate text one token at a time. This creates a bottleneck because the GPU spends too much time moving data. What's interesting is diffusion models generate multiple tokens simultaneously. This shifts the operation from being limited by memory to being limited by computation, making better use of GPU hardware. Zyphra's ZAYA1-8B-Diffusion-Preview specifically predicts unmasked tokens in a single step. The team converted their ZAYA1-8B-base model by adding 600 billion tokens of diffusion-conversion training, followed by context extension and fine-tuning. This is the first MoE diffusion model converted from an autoregressive LLM. The bottom line: this development could lead to much faster and more efficient language model performance.