Shared benchmark scope · SWE-bench Verified

Same-benchmark agent comparison

Historical evidence

The entries below share the same benchmark scope. Their exact scaffolds, model versions, dates, and original source values remain visible.

A higher source value only means better performance on this benchmark and configuration. It is not a cross-benchmark or production-readiness score.

Compare like with like

Decision overview

Only configurations from the same benchmark version and task scope can be placed in one comparison.

Source-reported data; missing values remain unknown.

SWE-bench Verified · Resolved

At a glance	mini-SWE-agent + MiniMax M2.5 (high reasoning)	mini-SWE-agent + Claude 4.5 Opus (high reasoning)
Strengths	—	Higher published result: 76.8%
Trade-offs & limits	Lower published result: 75.8%	—
Capabilities & cost	Base model: MiniMax M2.5 (high reasoning)Scaffold: mini-SWE-agent Reported benchmark cost: $36.64	Base model: Claude 4.5 Opus (high reasoning)Scaffold: mini-SWE-agent Reported benchmark cost: $376.95
	Agent details →	Agent details →

All data & sources

17 identical or entirely unreported fields hidden

Agent system	mini-SWE-agent + MiniMax M2.5 (high reasoning)	mini-SWE-agent + Claude 4.5 Opus (high reasoning)
Configuration
Base model	MiniMax M2.5 (high reasoning)	Claude 4.5 Opus (high reasoning)
Source model ID or label	minimax-m2.5	claude-4-5-opus
Outcome
Resolved	75.8%	76.8%
Reported benchmark cost	$36.64	$376.95
Source
Open source	Open source ↗	Open source ↗

Source-reported data; missing values remain unknown.

Public rankings create a shortlist. A small private trial decides whether the configuration fits your work.

1
Choose 10–20 representative tasks and define a clear pass condition.
2
Lock the agent, model, tools, permissions, and budget before testing.
3
Run each important task more than once; record success, cost, time, and takeovers.
4
Review failures and data-handling risks before a production decision.