OpenGPTEvidence-based model decisions
Start evaluation

Agent configuration · GAIA

HAL Generalist Agent · Claude-3.7 Sonnet High (February 2025)

The ranked object: model, scaffold, tools, permissions, budget, and environment together.

Open source
Accuracy64.24%
Cost per successful task$1.16
Published result rangeNot published

01

Configuration

Agent systemHAL Generalist Agent · Claude-3.7 Sonnet High (February 2025)
Base modelClaude-3.7 Sonnet High (February 2025)
Source model ID or labelClaude-3.7 Sonnet High (February 2025)
ScaffoldHAL Generalist Agent
Scaffold versionNot reported
ToolsNot reported
EnvironmentGeneral-assistant tasks requiring reasoning, multimodality, browsing, and tools
BudgetSource-specific · no normalized step or token limit published

02

Outcome

Accuracy64.24%
Partial scoreNot published
Cost per successful task$1.16
Reported benchmark cost$122.49
Median completion timeNot published
Human interventionNot published

03

Operation

Repeat-run evidence1 runs
Published result rangeNot published
Safety noteNot published
Comparability grouphal-gaia-public-validation-165
Evaluation dateNot published
Data captured

04

Trust

EvidenceComplete public run record
SourceHolistic Agent Leaderboard · GAIA
Source snapshotHAL verified snapshot · public validation set · 165 questions · Jul 30, 2026
Last checkedJul 30, 2026
Tasks165 public validation questions
Evidence levelExternal caution
Evidence note

HAL reproduced these public-validation rows and publishes the scaffold, model, score, cost, run count, and traces. Treat the snapshot as historical; no normalized tool or step budget is published.

Open audit ↗

What this result does not prove

This result applies to the exact system and source conditions shown here. It does not establish a universal best agent, production reliability, data governance, or performance on a different benchmark version.

Compare like with like

Only configurations from the same benchmark version and task scope can be placed in one comparison.

HAL Generalist Agent · Claude Sonnet 4.5 (September 2025)74.55% · Claude Sonnet 4.5 (September 2025)HAL Generalist Agent · Claude Sonnet 4.5 High (September 2025)70.91% · Claude Sonnet 4.5 High (September 2025)HAL Generalist Agent · Claude Opus 4.1 High (August 2025)68.48% · Claude Opus 4.1 High (August 2025)HAL Generalist Agent · Claude Opus 4 High (May 2025)64.85% · Claude Opus 4 High (May 2025)

Source-reported data; missing values remain unknown.

Validate before adoption

Public rankings create a shortlist. A small private trial decides whether the configuration fits your work.

  1. 1

    Choose 10–20 representative tasks and define a clear pass condition.

  2. 2

    Lock the agent, model, tools, permissions, and budget before testing.

  3. 3

    Run each important task more than once; record success, cost, time, and takeovers.

  4. 4

    Review failures and data-handling risks before a production decision.

← All AI Agent rankings