Kimi K3 Scores: Portability Audit Needed for Benchmarks

Kimi K3 launched with two scores that initially showed it beating other systems like Claude Fable 5 and GPT-5.6 Sol. In Arena's July 16 Frontend Code snapshot, K3 ranked first with 1678.53, while Fable scored 1631.21 and GPT-5.6 Sol scored 1617.83. K3 also scored 34.8 in Kimi's SpreadsheetBench 2 table, slightly ahead of Fable's 34.7 and GPT-5.6 Sol's 32.4. However, these scores came from different systems with different evaluation methods. Arena's score is based on user preference in a web-development evaluation, not direct performance metrics. The SpreadsheetBench 2 result, reported by Kimi, shows a narrow 0.1-point margin between K3 and Fable. It also notes that K3 and Fable used Claude Code, while GPT-5.6 Sol used Codex, and Fable used a "fallback" option. What's important is that benchmark results like these are influenced by many factors beyond just the core model, including the agent's setup, tools used, and deployment conditions. This means directly comparing models based on these headline scores can be misleading if the underlying systems are different.