← sverklo.com

MCP code-intel index

A reproducible ranking of MCP code-intelligence servers on retrieval quality, token economy, and tool-call count. Sverklo on its own board — including the slices it loses.

Smithery tells you it installs. Sverklo tells you if the code is rotting.

Latest results

baseline F1 P1 P2 P4 P5 tokens tools/task audit
sverklo us 0.58 0.63 0.27 0.84 0.83 652 1.0 B
smart-grep 0.34 0.20 0.20 0.40 0.83 714 3.2 —
gitnexus 0.30 0.35 0.00 0.27 0.83 630 1.2 F
jcodemunch 0.29 0.52 0.01 0.33 0.34 1,907 1.2 C
naive-grep 0.25 0.07 0.11 0.35 0.83 22,704 6.3 —

180 tasks across 6 datasets (express, lodash, sverklo, requests, flask, fastapi). Run 2026-05-13T18-32-20-478Z at sverklo b631c21. Published 2026-05-13 18:32 UTC. Historical Sverklo v0.20.21; version tag c97b6eb.

Audit grades are pinned repository evidence captured 2026-07-26: a73b2e5:mcp/index.html. The unauthenticated badge publish endpoint is not used as audit truth.

Inspect or rerun this pinned benchmark method 
git clone https://github.com/sverklo/sverklo.git && cd sverklo && git checkout --detach b631c21366e651ea6c97a7234066a2493241cf34 && npm ci && npm run build && npm run bench:quick

The harness writes summary.json, raw.jsonl, and report.md to benchmark/results/<timestamp>/. A rerun is comparison evidence; without the retained historical raw result artifact it is not an exact reproduction of this snapshot.

Pinned benchmark method; the historical raw result artifact is unavailable: sverklo/sverklo/tree/b631c21366e651ea6c97a7234066a2493241cf34/benchmark

What this measures

F1 (overall + per category)

Hand-verified retrieval tasks scored on F1. Per-category breakdown so wins on definition lookup don't paper over losses on reference finding. P1: definition lookup. P2: reference finding. P4: file dependencies. P5: dead-code detection.

Tokens / task

Average input tokens the agent ingests per task. The load-bearing axis for AI agents inside bounded context windows. Naive grep returns ~20K; sverklo ~500. Lower is better.

Tools / task

Average tool calls per task. A baseline that wins F1 by making 12 calls is a different product than one that wins F1 in 1 call. Lower = less round-trip latency.

What's deliberately NOT a column

No composite score. No A-F letter grade. No "verdict." Each axis stays independent so the methodology survives critique. RFC #5 documents the metric set.

How to read this

Different baselines win different categories. Smart-grep remains genuinely competitive on call-site lookups. Jcodemunch-mcp has the strongest P5 recall after the integration refresh. Sverklo leads the May 13 180-task run overall (F1 0.58), leads P1 definition lookup (0.63), and wins P4 file dependencies decisively (0.84) because the symbol graph + PageRank is what graph-shaped retrieval needs. Naive grep is the floor.

The story isn't "sverklo beats everything." It's different retrieval substrates have different strengths, and the load-bearing axis depends on what you're optimizing for. For agents inside bounded context windows, the token economy is the load-bearing axis; for human-facing search, F1 wins. The page above lets you sort either way.

Methodology

How a maintainer adds their tool

  1. Read CONTRIBUTING.md in the methodology repo.
  2. Open a PR against sverklo/sverklo adding benchmark/src/baselines/<your-tool>.ts implementing the Baseline interface.
  3. The auto-bench CI workflow runs on the PR (express dataset, ~10min), posts a results table, and uploads the raw artifact. You don't need to run anything locally.
  4. If the implementation is faithful to your tool's intended use, we merge. Then the next quarterly refresh picks it up here.

Disagreements with the methodology, the metric set, or specific task scoring: file an issue against sverklo-bench/issues. Open invitation. We've already shipped fixes to our own parser in response to bench findings (the bench-loop post documents the pattern).

Embed sverklo audit in your CI

If you maintain an MCP server (or any code-intel project), you can add the sverklo audit to your own CI in two lines. The bundled provider runs on the GitHub Actions runner by default; explicitly configuring a remote embedding provider changes that boundary:

- uses: sverklo/sverklo@main
  with:
    fail_on: ""        # or "F" to block merges on F-grade
    comment: "true"    # post idempotent PR comment

The Action posts a markdown comment with the overall grade plus a per-dimension table (dead code, circular deps, coupling, security). Idempotent — re-runs update in place. Methodology link in every comment so disagreements have a place to land. Source: sverklo/sverklo/action.yml.

Or request a one-time audit posted publicly at sverklo.com/report/<owner>/<repo>/: file the audit-request issue.

The wedge

Other surfaces in the MCP-server space score on different axes. Glama is a directory with letter grades on metadata quality. MseeP scores npm-audit-shaped security. PulseMCP curates editorially. The official Registry is neutral substrate, no opinion.

None of them measure whether the MCP server actually retrieves the right code. That's the axis above. If your team picks an MCP server based on README polish or install count, the failure mode shows up in production: the agent hallucinates symbol names because retrieval missed the relevant chunk. The bench above measures the failure mode directly.

Cite this

@misc{sverklo_mcp_index_2026,
  title  = {Sverklo MCP code-intel index — comparative evaluation of MCP retrieval servers},
  author = {Groshin, Nikita},
  year   = {2026},
  doi    = {10.5281/zenodo.19802051},
  url    = {https://sverklo.com/mcp/}
}

Sverklo is itself one of the baselines on this page. The numbers above include sverklo's losing slices — that's the point. Methodology + raw artifacts at github.com/sverklo/sverklo-bench. The reproducer command is one click in the table above.