IBM Quantum Enhances Llama 3.1 LLM: 1.4% Perplexity Drop

Researchers have achieved a quantum enhancement of a large language model using IBM hardware. This involves a hybrid scheme with a 156-qubit Heron processor. What's interesting is this marks the first "end-to-end quantum enhancement" of an LLM on a superconducting processor for text generation. The experiment used Meta's Llama 3.1 8B. The base model was frozen, and quantum adapters were added. The hybrid version reduced the perplexity of Llama 3.1 8B by 1.4%. This was done by adding only about 6,000 parameters. During a demonstration, the quantum-enhanced Llama correctly answered questions on astronomy and biology that the base version could not. The lead author calls this a proof of concept. The quantum blocks allowed more accurate next-token prediction with minimal computational cost. This development could mean more powerful and efficient AI models in the future.