What do public ARD registries actually return?
Same queries, same day, every public registry that speaks the ARD spec's POST /search. Zero-dependency harness, one command, one JSON per edition. Published by Desvela, which competes in this table — its row is marked, and the method below is written to survive being read by a rival.
The table — 1 Sep 2026
| Registry | Latency cold / warm | Tasks answered | Duplicates | Gibberish | Self-retrieval |
|---|---|---|---|---|---|
| Desvela (ours) | 214 ms / 86 ms | 8/8 | 0 | 0 results | n/a — own index |
| WellKnown HQ | 923 ms / 314 ms | 8/8 | 8 extra entries | 10 results, scored 51–56 | 4/5 at rank 1 |
| Hugging Face Discover | 1853 ms / 1856 ms | 8/8 | 0 | 0 results | 0/5 — does not index us |
| Official MCP Registry | Excluded: does not speak ARD — 404 to the spec's request shape, serves its own /v0.1/search. Listed so it does not look like we picked convenient rivals | ||||
Medians of 5 samples. Cold = fresh connection per request, what a one-off agent call pays. Warm = keep-alive. Both published: our first harness version measured only warm and reported 307 ms for an endpoint curl timed at 1.16 s. Neither number lies; publishing only one means picking the convenient one.
The metric that matters most: gibberish
The negative control. xkqjvwz mplfrbn tzghdkw means nothing in any language. An honest registry returns zero results. A registry that returns ten results scored 51–56 has an undocumented noise floor — and once that is true, its relevance score cannot be used to decide anything, because a score of 55 might mean "match" or might mean "noise" and nothing in the response tells you which.
What this benchmark does not measure
- Semantic ranking quality. It would require human relevance judgment — and that judgment would come from us, an interested party. Where the rival's semantic search is better than ours, and for some queries it is, this benchmark will not show it. Read their results yourself.
- Coverage. Comparing index sizes without probing for catch-all servers is the error that produces headline numbers. Our own crawler, on a path the spec introduced on 26 Aug 2026 that nobody served yet, counted 745 "publishers" in the Tranco top-100K — we sampled 12 live and none published anything: they were servers answering 200 to any URL. Coverage claims need canary probing; the monthly census does that, this page does not.
Our own limits, declared
- 4/5, not 5/5. In the rival index our entries rank first for four of our five published queries. The fifth ranks lower because our catalog entry was deliberately written not to self-preference. The original publication bar said 5/5; it was rewritten, because honesty had made it impossible.
- The 8 duplicates are ours. We publish the same catalog on
desvela.aianddesvela.dev, so the same entry reaches indexers twice — sameurl, different urn. Letting both through is the indexer's missing endpoint-dedupe, but the duplicate originates with us. - Hugging Face Discover does not index us at all, and its latency does not improve with keep-alive — its cost is compute, not connection. Both facts are in the data; we do not know why it excludes us.
Reproduce it
git clone https://github.com/mrenzo20/ard-retrieval-benchmark cd ard-retrieval-benchmark node run.mjs # Node ≥ 20, no dependencies, ~4 minutes
The query set, the registry roster with exclusions and reasons, and every edition's raw JSON live in the repo. If you re-run it and get different numbers, that is a finding — open an issue.