DSpark Draft Model: 3.18x Faster AI Inference

Liquid AI and Hugging Face have released DSpark draft model checkpoints for the LFM2.5 series. This technology significantly improves inference throughput without changing model output quality. Tests show the DSpark draft model can increase overall throughput on GPU by up to 3.18 times, and on edge devices, it can improve performance by up to 2.87 times. For edge agent inference, the function call latency of LFM2.5-2.6B was reduced by an average of 57%. On a MacBook Pro, output speed can reach 139 tokens per second, making local agent applications more accessible. DSpark works by using a lightweight draft model to quickly generate candidate tokens, which are then validated by the target model. This helps overcome memory bandwidth limitations in traditional large language model inference. The draft model maintains the same output quality as the baseline greedy decoding, with no decrease in accuracy. This advancement means faster and more efficient AI applications for users.