Skip to content

The Aggregate — LLM benchmark aggregate

by theaggregate.ai · remote · no sign-in to list tools · v1.0.2 · published by ai.theaggregate

LLM rankings from public benchmarks, model comparisons and source-linked results. Updated daily.

Changes · 1
CompareRegistry entry

8 tools: 1 change things and 7 only read.

Scanned Oct 21, 2026

No check matched anything in the tool definitions. No finding is not a safety guarantee. How the checks work.

claude mcp add --transport http the-aggregate https://theaggregate.ai/mcp

Tools

Open a tool for its reasons and full description. Descriptions are the publisher's own words, shown as they are.

8 tools

Changes things 1Claude Code: ask
  • about_the_aggregateWhat this data is: how the IRT fusion works, current coverage counts, update cadence, and how to cite it.No findings

    Why it is in this group

    • Nothing in its name or description says what it does, so it is counted as a write.
    • The server marks it read-only (readOnlyHint: true), but an annotation never lowers a class.
    Description, as the server wrote it
    What this data is: how the IRT fusion works, current coverage counts, update cadence, and how to cite it.

    Annotations the server declares

    • title: About The Aggregate
    • readOnlyHint: true
    • destructiveHint: false
    • idempotentHint: true
    • openWorldHint: false
Only reads 7Claude Code: allow
  • compare_modelsHead-to-head between 2-4 models: aggregate ranks, rating gap with a significance note based on the standard errors, and notable benchmarks they share.No findings

    Why it is in this group

    • The name "compare_models" says "compare": a word for reading.
    Description, as the server wrote it
    Head-to-head between 2-4 models: aggregate ranks, rating gap with a significance note based on the standard errors, and notable benchmarks they share.

    Annotations the server declares

    • title: Compare models
    • readOnlyHint: true
    • destructiveHint: false
    • idempotentHint: true
    • openWorldHint: false
  • get_benchmarkOne benchmark in depth: what it measures, the original source leaderboard URL, IRT stats (difficulty, noise, model coverage), skill weights, and the current top models on it.No findings

    Why it is in this group

    • The name "get_benchmark" says "get": a word for reading.
    Description, as the server wrote it
    One benchmark in depth: what it measures, the original source leaderboard URL, IRT stats (difficulty, noise, model coverage), skill weights, and the current top models on it.

    Annotations the server declares

    • title: Benchmark detail
    • readOnlyHint: true
    • destructiveHint: false
    • idempotentHint: true
    • openWorldHint: false
  • get_leaderboardTop of the cross-benchmark aggregate ranking: every model placed on one rating scale by an IRT model fit over public benchmark leaderboards (call about_the_aggregate for the curren…No findings

    Why it is in this group

    • The name "get_leaderboard" says "get": a word for reading.
    Description, as the server wrote it
    Top of the cross-benchmark aggregate ranking: every model placed on one rating scale by an IRT model fit over public benchmark leaderboards (call about_the_aggregate for the current coverage counts). One row per model by default, fused across reasoning-effort settings. Supports paging via limit/offset.

    Annotations the server declares

    • title: Aggregate leaderboard
    • readOnlyHint: true
    • destructiveHint: false
    • idempotentHint: true
    • openWorldHint: false
  • get_modelOne model in depth: aggregate rank, Aggregate Rating with standard error, provider, what it is, cost per task where known, and its most notable benchmark results (with percentiles)…No findings

    Why it is in this group

    • The name "get_model" says "get": a word for reading.
    Description, as the server wrote it
    One model in depth: aggregate rank, Aggregate Rating with standard error, provider, what it is, cost per task where known, and its most notable benchmark results (with percentiles).

    Annotations the server declares

    • title: Model profile
    • readOnlyHint: true
    • destructiveHint: false
    • idempotentHint: true
    • openWorldHint: false
  • get_prediction_duelGuesswork — the public prediction duel: every day frontier LLMs and The Aggregate's own IRT model predict newly scraped benchmark scores before seeing them, and the errors are scor…No findings

    Why it is in this group

    • The name "get_prediction_duel" says "get": a word for reading.
    Description, as the server wrote it
    Guesswork — the public prediction duel: every day frontier LLMs and The Aggregate's own IRT model predict newly scraped benchmark scores before seeing them, and the errors are scored. Returns the current monthly standings, wins and losses included.

    Annotations the server declares

    • title: Prediction duel standings
    • readOnlyHint: true
    • destructiveHint: false
    • idempotentHint: true
    • openWorldHint: false
  • search_benchmarksFind benchmarks in the aggregate by (partial) name.No findings

    Why it is in this group

    • The name "search_benchmarks" says "search": a word for reading.
    Description, as the server wrote it
    Find benchmarks in the aggregate by (partial) name. Returns model coverage, difficulty on the rating scale, and the benchmark page URL.

    Annotations the server declares

    • title: Search benchmarks
    • readOnlyHint: true
    • destructiveHint: false
    • idempotentHint: true
    • openWorldHint: false
  • search_modelsFind ranked models by (partial) name or provider.No findings

    Why it is in this group

    • The name "search_models" says "search": a word for reading.
    Description, as the server wrote it
    Find ranked models by (partial) name or provider. Returns rank, Aggregate Rating and the model page URL. One row per model by default, fused across reasoning-effort settings.

    Annotations the server declares

    • title: Search models
    • readOnlyHint: true
    • destructiveHint: false
    • idempotentHint: true
    • openWorldHint: false

This is not a safety rating, and no finding is not a safety guarantee. The groups and findings come from fixed checks of the tool definitions the server returned on the date shown, not from its code or what it does when called. How the checks work.

Install

Pinned to version 1.0.2. Values in angle brackets are yours to fill in.

claude mcp add --transport http the-aggregate https://theaggregate.ai/mcp

Run in a terminal.

Claude Code permissions

Allow the reads, ask before a change, deny the deletes. Paste into .claude/settings.json.

.claude/settings.json
{
  "permissions": {
    "allow": [
      "mcp__the-aggregate__get_leaderboard",
      "mcp__the-aggregate__search_models",
      "mcp__the-aggregate__get_model",
      "mcp__the-aggregate__compare_models",
      "mcp__the-aggregate__search_benchmarks",
      "mcp__the-aggregate__get_benchmark",
      "mcp__the-aggregate__get_prediction_duel"
    ],
    "ask": [
      "mcp__the-aggregate__about_the_aggregate"
    ],
    "deny": []
  }
}

Tools that only read are allowed, tools that change things ask first, tools that delete or cancel are denied. It assumes the server is installed as the-aggregate. Review it before you use it.