NVIDIA Gated DeltaNet-2: AI Forgets Without Breaking

NVIDIA researchers have introduced a new AI model called Gated DeltaNet-2. They've submitted details to arXiv and released matching code. What's interesting is this model aims to improve how AI models handle memory. It uses a two-gate design to separate erase and write decisions within its memory. This helps reduce accidental overwrites in linear attention. NVIDIA reports stronger benchmark and retrieval results for Gated DeltaNet-2. However, independent replication is still needed. The model may also see a slight dip in throughput, from 38.0 Kt/s to 36.1 Kt/s. This development could lead to more efficient and accurate AI models, especially for processing long texts.