Shared benchmark scope · SWE-bench Verified

Same-benchmark agent comparison

Historical evidence

The entries below share the same benchmark scope. Their exact scaffolds, model versions, dates, and original source values remain visible.

A higher source value only means better performance on this benchmark and configuration. It is not a cross-benchmark or production-readiness score.

Compare like with like

Decision overview

Only configurations from the same benchmark version and task scope can be placed in one comparison.

Source-reported data; missing values remain unknown.

SWE-bench Verified · Resolved

At a glance	live-SWE-agent + Claude 4.5 Opus medium (20251101)	live-SWE-agent + Gemini 3 Pro Preview (2025-11-18)
Strengths	Higher published result: 79.2%	—
Trade-offs & limits	—	Lower published result: 77.4%
Capabilities & cost	Base model: Claude 4.5 Opus medium (20251101)Scaffold: live-SWE-agent	Base model: Gemini 3 Pro Preview (2025-11-18)Scaffold: live-SWE-agent
	Agent details →	Agent details →

All data & sources

17 identical or entirely unreported fields hidden

Agent system	live-SWE-agent + Claude 4.5 Opus medium (20251101)	live-SWE-agent + Gemini 3 Pro Preview (2025-11-18)
Configuration
Base model	Claude 4.5 Opus medium (20251101)	Gemini 3 Pro Preview (2025-11-18)
Source model ID or label	claude-opus-4-5-20251101	gemini-3-pro-preview
Outcome
Resolved	79.2%	77.4%
Trust
Evaluation date	Dec 15, 2025	Nov 20, 2025
Source
Open source	Open source ↗	Open source ↗

Source-reported data; missing values remain unknown.

Public rankings create a shortlist. A small private trial decides whether the configuration fits your work.

1
Choose 10–20 representative tasks and define a clear pass condition.
2
Lock the agent, model, tools, permissions, and budget before testing.
3
Run each important task more than once; record success, cost, time, and takeovers.
4
Review failures and data-handling risks before a production decision.