Shared benchmark scope · τ³-Banking Knowledge

Same-benchmark agent comparison

Historical evidence

The entries below share the same benchmark scope. Their exact scaffolds, model versions, dates, and original source values remain visible.

A higher source value only means better performance on this benchmark and configuration. It is not a cross-benchmark or production-readiness score.

Compare like with like

Decision overview

Only configurations from the same benchmark version and task scope can be placed in one comparison.

Source-reported data; missing values remain unknown.

τ³-Banking Knowledge · Pass¹

At a glance	τ³ Banking AllTools · Qwen 3.8 Max · xhigh	τ³ Banking AllTools · Claude Opus 5 · max
Strengths	Higher published result: 55.15%	—
Trade-offs & limits	—	Lower published result: 48.71%
Capabilities & cost	Base model: Qwen 3.8 MaxScaffold: τ³ standard agent	Base model: Claude Opus 5Scaffold: τ³ standard agent
	Agent details →	Agent details →

All data & sources

18 identical or entirely unreported fields hidden

Agent system	τ³ Banking AllTools · Qwen 3.8 Max · xhigh	τ³ Banking AllTools · Claude Opus 5 · max
Configuration
Base model	Qwen 3.8 Max	Claude Opus 5
Source model ID or label	Qwen 3.8 Max · xhigh	Claude Opus 5 · max
Outcome
Pass¹	55.15%	48.71%
Source
Open source	Open source ↗	Open source ↗

Source-reported data; missing values remain unknown.

Public rankings create a shortlist. A small private trial decides whether the configuration fits your work.

1
Choose 10–20 representative tasks and define a clear pass condition.
2
Lock the agent, model, tools, permissions, and budget before testing.
3
Run each important task more than once; record success, cost, time, and takeovers.
4
Review failures and data-handling risks before a production decision.