Every rank on this leaderboard should be reproducible, challengeable and traceable. Below we lay out the full methodology.
1. What this leaderboard is — and is not
It tracks projects with a public, verifiable GitHub repository that belong to an AI Agent platform's plugin ecosystem, ranked by the day's public metrics. It is not an official download chart, nor does it represent actual usage or a quality review.
The data has two layers: the leaderboard (the scoring policy described here, top 100 in the overall and each platform chart) and the official marketplace directory (all entries from each platform's local marketplace list, without score ranking).
The biggest limitation: platform marketplaces such as Codex and WorkBuddy contain many closed-source skills, connectors and expert packs that have no public GitHub repository. They do not take part in the overall score ranking (otherwise "neutral missing" would lift zero-data entities into the Top 100), but they are listed in full on the marketplace directory. This is a boundary of data availability, not an omission, and no estimated values are used to fill in.
norm uses the same-day percentile across the full sample (ties take the average rank) rather than dividing by the maximum, so top repositories don't crush the long tail; long-tail metrics such as stars / forks / downloads first take log1p. Four dimensions:
No zeros, no estimates. When a whole dimension is unavailable (e.g. no history snapshot yet on day one → momentum M missing), its weight is removed from the denominator, the remaining dimensions are renormalized, and the detail page marks that dimension n/a weight renormalized.
Day-one effective weights are P 0.5333 / A 0.3333 / Q 0.1333; from day two they return to 0.40 / 0.25 / 0.25 / 0.10.
npm downloads are taken only when the repository homepage explicitly points to npmjs.com/package/…; package names are never guessed from the repo name; if unavailable it is marked n/a. In addition, the curated mapping table includes only packages whose registry repository field exactly matches the GitHub repo and that were actually measured — everything else is never fabricated.
The detail page "data confidence" is computed from five signals — license / homepage / description length / topic count / downloads — and rated high (≥4), medium (≥2) or low.
4. Deduplication and platform attribution
Unique entity canonical = owner/repo (lowercase). When one repository is matched by multiple platform topics it is merged into a single entity, with platforms keeping all values — the overall chart is not spammed by the same plugin.
Platform attribution = collection query source ∪ cross-hit repository topics. For example, a repository tagged both claude-code and dsh-plugin appears on both platform charts, but counts as one in the overall chart.
The overall chart and each platform chart take the top 100 independently. When a platform has fewer than 100 samples, the actual count is shown and labeled; standards are not lowered to pad the list.
Dual relevance check: a project's own description and name must hit AI/agent semantic terms, and overall hit plugin semantic terms or platform-specific topics. A single topic tag alone is not enough — otherwise generic tools would slip into an AI plugin chart. Word-boundary regex is used to avoid substring false positives.
Official marketplace fast track: entries from the official lists in section 6 — official inclusion is itself a strong signal, so the stars ≥ 20 gate and AI semantic screening are skipped (many official plugin repos never mention AI in their description). Archived repositories, empty descriptions and resource/list repositories are still excluded, and such entries are labeled "Official".
Repositories younger than < 14 days: score ×0.85 cooldown; bulk plugins from one author (>5): ×0.97 slight down-weight; growth beyond mean +3σ is clipped. All adjustments are written below "Score breakdown" on the detail page.
6. Local official marketplace lists (discovery source)
GitHub topic search can only find "tagged repositories" and misses plugins officially listed in platform marketplaces but without a topic. To fix this, the pipeline reads the following local list files read-only as authoritative discovery sources (user configuration is never modified):
dsh --profile web --dump-config + profiles/*/package.json
npm package name (weekly downloads can be queried)
Three distinctions must be kept clear:
Official inclusion ≠ metric. Lists are used only to discover and admit "this plugin really exists and is officially listed"; stars / downloads are always re-verified through public APIs, and no number in a list is ever used as a scoring input.
Plugin ≠ repository. One marketplace repository often hosts dozens of plugins (e.g. git-subdir subdirectories). The overall chart merges by owner/repo to avoid spam; the detail page lists all official plugins hosted by that repo, while the directory page shows them at plugin granularity.
Closed-loop traceability. Each run generates a snapshot of the local lists and commits it to data/marketplace-sources/; when GitHub Actions cannot reach user directories it reads the snapshots instead, keeping the same methodology. Entries without public metrics are marked n/a and listed in the run report — never guessed or padded.
7. Data sources and evidence chain
Only public returns from api.github.com/search/repositories, api.github.com/repos/* and api.npmjs.org/downloads are used. stars / install counts from third-party leaderboards are never trusted, used at most as discovery leads and re-verified via our own API calls before inclusion.
Repositories referenced in official lists are first fetched in bulk with multiple repo:owner/name qualifiers through the Search API (separate search quota). In practice a number of real repositories are absent from the search index (HTTP 422 "resource not searchable", e.g. anthropics/claude-plugins-public); those are re-checked with GET /repos/{owner}/{repo}, and only a REST 404 marks them unverifiable. All results, including "confirmed not found", are cached in data/repo-metadata.json for 7 days — no fabrication and no daily re-checking of dead links.
Each record stores the collection query, UTC collection time, API endpoint and the original fields at that moment; the "Evidence chain" table on detail pages can be checked directly against them. Daily run reports live in state/run-YYYY-MM-DD.json, including all rejected entries and reasons.
8. A note on cross-day comparison
Scores are same-day relative percentiles; the sample set and effective weights can change every day, so absolute score values cannot be compared across days. Compare only ranks and rank changes (▲/▼). The trend curve on detail pages shows historical samples under the same methodology, for reference rather than precise auditing.
9. How to recompute it yourself
GITHUB_TOKEN=xxx python pipelines/run_daily.py python pipelines/run_daily.py --dry-run # compute only, no leaderboard artifacts python pipelines/run_daily.py --no-local-market # only run GitHub topic collection (for comparison) python pipelines/run_daily.py --no-market-npm # skip the dsh package download query
Weights, queries and inclusion word lists are in pipelines/config.py, scoring in pipelines/score.py, official-list parsing and repo verification in pipelines/local_market.py. The effective weights for the day are written to data/top100.json under meta.scoring; the official directory is in data/marketplace.json; repo/npm metadata caches are in data/repo-metadata.json and data/npm-metadata.json. When changing the methodology, update this page accordingly.
Without GITHUB_TOKEN, GitHub's unauthenticated limits are Search 10/min and core 60/hour. The first run must verify roughly 240 repositories in bulk (about 3 minutes); afterwards the cache brings requests to nearly zero. Setting a token is recommended to significantly raise the limits and shorten the first run.