AI Grading AI: Smarter Models Not Fairer Judges

A quiet assumption in the AI industry is under strain: that one AI model can reliably grade another. New reporting highlights unreliable AI judges and "benchmark hallucinations." Here's the thing: making an AI model smarter does not necessarily make it a fairer judge. In fact, it might even make it more biased. Researchers call this "self-preference bias." When a language model evaluates text, it tends to rate its own output, or text similar to its own, more highly than a human would. This isn't vanity; it's how the models work. A model's own writing is highly probable to itself, so it reads as fluent and correct, earning a higher grade. What's interesting is that advanced AI capability is often uncorrelated, and sometimes negatively correlated, with low self-preference bias. This means smarter models are not always fairer judges. One study found that some models inflate their own win rate by double digits compared to human judgment. The bottom line: if you read AI leaderboards or trust benchmark scores, you should interpret those numbers differently.