S. Korea AI Evaluation: Benchmark Score Manipulation Allegations

Allegations of benchmark score manipulation have surfaced ahead of South Korea's National AI Model second-round evaluation. Overseas AI companies reportedly offered to boost benchmark scores for domestic firms. Industry sources indicate U.S.-based AfterQuery proposed collaborations to provide data and post-training technologies. South Korea's Ministry of Science and ICT plans to eliminate one team from the four finalists: LG AI Research, Upstage, SK Telecom, and Motif Technologies. The evaluation includes benchmark performance and expert assessments, with discussions reportedly focusing on inflating scores. AfterQuery specializes in AI data and model optimization, known for "Benchmaxxing," which optimizes models for high benchmark scores. While all four participating companies deny allegations, some technical reports reportedly contain circumstantial evidence. However, using external training data does not violate project rules. The core issue is whether evaluation data was mixed into model training. This could allow AI to memorize test questions rather than learn problem-solving. This matters because inflated scores might not reflect true model performance in real-world applications.