Benchmarks

8 evaluations

Public evaluation platforms and benchmarks for models and agents, grouped by field with methods and comparison scope.

General capabilities

Evaluation platform

Compares model capabilities across benchmark tasks, alongside API costs, speed and context limits.

Evaluates
Specific model versions and reasoning settings
Method
The Intelligence Index combines results across task categories; costs and runtime efficiency are reported separately.

Comparison scope

The index uses English tasks, mostly text with some image inputs. Match index versions and reasoning settings. Image and video generation quality have separate evaluations.

Evaluation platform

Ranks model responses using user preference votes, with task categories and confidence intervals.

Evaluates
Model responses in text conversations
Method
Uses pairwise response preferences, with style controls, sample counts and uncertainty estimates.

Comparison scope

Preference is not factual accuracy. Users and prompts affect results; check the task category and style-control setting.

Benchmark

Covers reasoning, data analysis and multi-step work such as coding and browser use, with results by release.

Evaluates
Models in the benchmark's tool environment
Method
Evaluates questions and tool tasks using reference answers, output checks and scoring rubrics.

Comparison scope

Each suite uses one run. Missing suites and release weights affect Overall; current weight descriptions differ, so prioritize suite-level results.

Evaluation platform

Tracks model performance in mathematics, coding and reasoning, with comparisons across models and time.

Evaluates
Model versions and their benchmark results
Method
Combines Epoch-run evaluations with externally reported results, providing sources and selected run logs.

Comparison scope

Not all results are independently measured by Epoch. Check the evaluator, benchmark version, settings and uncertainty.

Reasoning & knowledge

The original HLE tests expert knowledge and reasoning with 2,500 questions across disciplines. HLE-Rolling revises the question set over time. HLE-Diamond contains 1,000 carefully reviewed questions.

Evaluates
Model answers to academic questions with text and images
Method
Responses are graded against reference answers. Original HLE reports accuracy, confidence intervals and calibration error for models’ stated confidence.

Comparison scope

Compare results on the same question set, matching text-only or multimodal scope, tool access, graders and reasoning settings.

Coding & terminal tasks

Benchmark

Tests agents on complex tasks in isolated terminal environments, identifying both the model and agent harness.

Evaluates
Model and terminal-agent combinations
Method
Uses task verifiers to measure completion, alongside costs, resource usage and confidence intervals.

Comparison scope

The standalone leaderboard uses 4.1. The coding component of AA Intelligence Index v5.0.1 uses 4.0. Within a version, match harnesses, resource limits and trial counts.

Tool use & agents

Benchmark

Evaluates long workflows across desktop apps and browsers, distinguishing full completion from partial credit.

Evaluates
Computer-use agents and their interaction settings
Method
Checks task state after execution and reports completion, steps, elapsed time and costs.

Comparison scope

Model coverage remains limited. Compare within the same version and full or offline task set. Match interaction settings and step budgets, and distinguish completion from partial credit.

Evaluation platform

Uses feedback, correction handling and tool execution signals from real sessions to estimate gains over a baseline.

Evaluates
Orchestrator models in Arena's environment
Method
Uses causal estimation to measure relative improvement, with signal breakdowns and confidence intervals.

Comparison scope

The percentage is a net gain over a baseline, not a task success rate. It compares orchestrator models, not complete commercial agent products.

Search names, organizations, or keywords across all timelines.