Skip to content

MCP Selection Lab

by ChanghuLiu · remote · no sign-in to list tools · v0.5.2 · published by GitHub user ChanghuLiu

Benchmark MCP tool selection with metadata-only routing, collision, abstention, and holdout checks.

Changes · 1
CompareRegistry entry

3 tools: 1 change things and 2 only read.

Scanned Oct 21, 2026

1 check matched something in the tool definitions. No finding is not a safety guarantee. How the checks work.

claude mcp add --transport http selection-lab https://agenttrustlab.com/mcp

Findings

Matches of fixed checks, with the text they matched.

1 info

Tools

Open a tool for its reasons and full description. Descriptions are the publisher's own words, shown as they are.

3 tools

Changes things 1Claude Code: ask
  • scan_mcp_metadataCreate a free public report of MCP tool-description routing collisions.1 finding

    Why it is in this group

    • The description's first verb is "create": a word for a change.

    Findings

    • Info

      The description mentions "credentials".

      Matched text

      description: "...funds. Use a public HTTP(S) URL without credentials, query or fragment. Results use at..."

      What this check looks for
    Description, as the server wrote it
    Create a free public report of MCP tool-description routing collisions. Reads discovery metadata (initialize/tools/list) only; never calls target business tools or spends funds. Use a public HTTP(S) URL without credentials, query or fragment. Results use at most 12 tools and 24 generated cases, not a real-model evaluation. The returned report is enriched with agent_plan candidate description-only edits and a machine-readable rerun instruction. Candidate edits are not untouched-holdout proof. Reports persist and are public by link. For an existing report use get_selection_report. Set internal_test=true for owner/CI validation so it is excluded from public scans. source is a self-reported referral bucket, not identity verification.

    Annotations the server declares

    • destructiveHint: false
    • idempotentHint: false
    • openWorldHint: true
    • readOnlyHint: false
Only reads 2Claude Code: allow
  • get_selection_reportRead and agent-enrich an existing public scan report by id; no new target request or model call.No findings

    Why it is in this group

    • The name "get_selection_report" says "get": a word for reading.
    Description, as the server wrote it
    Read and agent-enrich an existing public scan report by id; no new target request or model call.

    Annotations the server declares

    • destructiveHint: false
    • idempotentHint: true
    • openWorldHint: false
    • readOnlyHint: true
  • selection_lab_infoGet free service capabilities, limits, privacy and connection details; does not scan a target.No findings

    Why it is in this group

    • The description's first verb is "get": a word for reading.
    Description, as the server wrote it
    Get free service capabilities, limits, privacy and connection details; does not scan a target.

    Annotations the server declares

    • destructiveHint: false
    • idempotentHint: true
    • openWorldHint: false
    • readOnlyHint: true

This is not a safety rating, and no finding is not a safety guarantee. The groups and findings come from fixed checks of the tool definitions the server returned on the date shown, not from its code or what it does when called. How the checks work.

Install

Pinned to version 0.5.2. Values in angle brackets are yours to fill in.

claude mcp add --transport http selection-lab https://agenttrustlab.com/mcp

Run in a terminal.

Claude Code permissions

Allow the reads, ask before a change, deny the deletes. Paste into .claude/settings.json.

.claude/settings.json
{
  "permissions": {
    "allow": [
      "mcp__selection-lab__selection_lab_info",
      "mcp__selection-lab__get_selection_report"
    ],
    "ask": [
      "mcp__selection-lab__scan_mcp_metadata"
    ],
    "deny": []
  }
}

Tools that only read are allowed, tools that change things ask first, tools that delete or cancel are denied. It assumes the server is installed as selection-lab. Review it before you use it.