GLM-5.3: DeepSWE Coding Benchmark King at $3.99/Run
Summary
The open-weight model GLM-5.3 is the most cost-efficient performer on the DeepSWE software engineering benchmark. It delivers results comparable to or better than premium rivals at a fraction of their cost. GLM-5.3 achieved an 87.6% solve rate on pass@4 metrics for just $3.99 per rollout. By comparison, Claude Fable 5 reached 84.1% at $21.63 per run, and GPT-5.6 Sol hit 85.8% at $8.37. The DeepSWE benchmark evaluates AI coding agents on 113 original software engineering tasks. On first-attempt accuracy, GPT-5.6 Sol led with 72.7% pass@1, followed by Claude Fable 5 at 69.7% and GLM-5.3 at 69.0%. However, GLM-5.3's multi-attempt metrics, with 81.1% pass@2 and 87.6% pass@4, outpaced both competitors. GLM-5.3 excelled in concurrency and durability tasks, achieving 62% accuracy, and also in query languages and runtime internals. Claude Fable 5 showed strength in Rust programming and data serialization, while GPT-5.6 Sol led in data modeling and protocol conformance. The benchmark data suggests that combining models, such as using GLM-5.3 as a front-line model and escalating unresolved tasks, can optimize both accuracy and cost. This means development teams can achieve broad task coverage more efficiently.
This is an AI-generated audio summary. Always check the original source for complete reporting.