GLM-5.3: DeepSWE Coding Benchmark King at $3.99/Run

3h ago·0:00 listen·Source: finance.biggo.com

Summary

The open-weight model GLM-5.3 is the most cost-efficient performer on the DeepSWE software engineering benchmark. It delivers results comparable to or better than premium rivals at a fraction of their cost. GLM-5.3 achieved an 87.6% solve rate on pass@4 metrics for just $3.99 per rollout. By comparison, Claude Fable 5 reached 84.1% at $21.63 per run, and GPT-5.6 Sol hit 85.8% at $8.37. The DeepSWE benchmark evaluates AI coding agents on 113 original software engineering tasks. On first-attempt accuracy, GPT-5.6 Sol led with 72.7% pass@1, followed by Claude Fable 5 at 69.7% and GLM-5.3 at 69.0%. However, GLM-5.3's multi-attempt metrics, with 81.1% pass@2 and 87.6% pass@4, outpaced both competitors. GLM-5.3 excelled in concurrency and durability tasks, achieving 62% accuracy, and also in query languages and runtime internals. Claude Fable 5 showed strength in Rust programming and data serialization, while GPT-5.6 Sol led in data modeling and protocol conformance. The benchmark data suggests that combining models, such as using GLM-5.3 as a front-line model and escalating unresolved tasks, can optimize both accuracy and cost. This means development teams can achieve broad task coverage more efficiently.

Read the full article on finance.biggo.com

This is an AI-generated audio summary. Always check the original source for complete reporting.

Share
Keep Listening