GraphEval: New Framework Detects LLM Hallucinations

A new framework called GraphEval helps identify and combat hallucinations in large language models. Hallucinations happen when a model creates responses that are factually incorrect or made up. What's interesting is that while many solutions exist to tackle these problems, methods for internally diagnosing them have been less studied. Amazon researchers developed GraphEval, which uses knowledge graphs to analyze and detect these model errors. Here's how it works: GraphEval uses a two-stage evaluation process. First, it builds a knowledge graph from the model's response. This graph contains semantic triples, like "Subject, Relationship, Object." Then, it evaluates each of these triples against a source context, which is a body of factual knowledge. A natural language inference model checks if the triple is supported by this context. If a triple contradicts the context or can't be confirmed, it's flagged as a hallucination. The bottom line is that GraphEval aims to provide insights into exactly where a hallucination occurred, offering more explainability than traditional single-score metrics.