Shared benchmark scope · SWE-bench Verified

Same-benchmark agent comparison

The entries below share the same benchmark scope. Their exact scaffolds, model versions, dates, and original source values remain visible.

A higher source value only means better performance on this benchmark and configuration. It is not a cross-benchmark or production-readiness score.

Compare like with like

Decision overview

Only configurations from the same benchmark version and task scope can be placed in one comparison.

Source-reported data; missing values remain unknown.

UIUClive-SWE-agent + Gemini 3 Pro Preview (2025-11-18)

Resolved77.4%Cost per successful taskNot publishedPublished result rangeNot published

UIUClive-SWE-agent + Claude 4.5 Opus medium (20251101)

Resolved79.2%Cost per successful taskNot publishedPublished result rangeNot published

17 identical or entirely unreported fields hidden

Agent system	live-SWE-agent + Gemini 3 Pro Preview (2025-11-18)	live-SWE-agent + Claude 4.5 Opus medium (20251101)
Configuration
Base model	Gemini 3 Pro Preview (2025-11-18)	Claude 4.5 Opus medium (20251101)
Source model ID or label	gemini-3-pro-preview	claude-opus-4-5-20251101
Outcome
Resolved	77.4%	79.2%
Trust
Evaluation date	Nov 20, 2025	Dec 15, 2025
Source
Open source	Open source ↗	Open source ↗

Source-reported data; missing values remain unknown.

Public rankings create a shortlist. A small private trial decides whether the configuration fits your work.

1
Choose 10–20 representative tasks and define a clear pass condition.
2
Lock the agent, model, tools, permissions, and budget before testing.
3
Run each important task more than once; record success, cost, time, and takeovers.
4
Review failures and data-handling risks before a production decision.