The Voratiq leaderboard evaluates the performance of various coding agents in real engineering tasks, currently featuring the gemini-3-1-pro-preview model. The agents are tested head-to-head on specific tasks, with their ratings based on the outcomes of these competitions. The current top-rated agent is gpt-5-3-codex-high, with a rating of 2036, followed by gpt-5-3-codex-xhigh and gpt-5-2-high. The leaderboard ranks 25 agents based on their effectiveness and the median time taken to complete tasks, emphasizing that the longest duration correlates with lower ratings.
Voratiq differs from standard benchmark suites like SWE-bench and HumanEval by utilizing ongoing engineering work rather than fixed tasks. Ratings are calculated using an Elo-like scale, reflecting the agents' success in applying their code to production environments. Each run consists of multiple agents competing simultaneously, and the winning agent’s implementation is integrated into the codebase, impacting its rating.
Tasks represented in the leaderboard primarily involve TypeScript and Python, covering a range of complexities from minor fixes to extensive changes. The leaderboard is updated weekly, and new models are added as they become available. Overall, Voratiq aims to enhance coding efficiency through competitive performance evaluations of AI agents.
Article timeline
Build this article’s timeline
Extract dated events, key actors, and open questions using the same timeline analysis available in Signal.
The Voratiq leaderboard evaluates the performance of various coding agents in real engineering tasks, currently featuring the gemini-3-1-pro-preview model.
🤖 AI prefill
📅When
2023
🤖 AI prefill
📍WhereWhere did this occur?
💡Why
Centers on Voratiq, gpt-5-3-codex-high, and agent leaderboard
🤖 AI prefill
Login to answer and earn XP for each gap you fill.
Comments
Login to view and post comments.