DeepSeek V4 Flash: Struggles with Real-World AI Tasks

DeepSeek's V4 Flash AI model is struggling with real-world tasks, despite its high rankings on AI leaderboards. The model completed just 53.8% of complex agent tasks in independent testing. DeepSeek's V4 Flash was launched on July 31 and has been called a "total monster" by developers. Its pricing undercuts comparable models by about tenfold. However, testing firm Composio found that only 129 out of 240 total runs passed across 30 difficult, multi-step workflows. The tests measured V4 Flash across eight different agent harnesses, using live tools like Gmail, GitHub, Slack, and Google Sheets. Results varied widely depending on the harness used, with Pi Agent completing 20 out of 30 tasks. DeepSeek is currently offering V4 Flash in public beta, indicating it's still a work in progress. While the model is significantly cheaper than competitors, its low success rate on complex tasks can lead to wasted resources. This highlights that thoughtful integration, like using the right agent harness, can be as crucial as the model itself.