Skip to content

worker/data is stale: index missing 116 repos, total_repos disagrees with itself, phantom verisim-data entry #49

Description

@hyperpolymath

The registry’s served data has drifted from the seed. No uuid is wrong — I checked, and every uuid in worker/data/repos.json matches its derivation exactly. The problem is coverage, counts, and a phantom entry.

Found while creating metadatastician/cadastra and cross-checking identities against the registry.

1. The index is missing 116 repos (~36% of the estate)

index.json  "generated"   : 2026-06-20T18:09:58.001Z     <- ~4 weeks stale
index.json  total_repos   : 203
index.json  by_name count : 204                          <- disagrees with its own total_repos
repos.json  entries       : 320
seed repos.a2ml entries   : 319

116 repos present in repos.json are absent from index.json’s by_name, e.g. a2ml-ecosystem, a2ml-estate-normalizer, a2ml_ex, a2ml_gleam, a2ml-showcase, …

worker/src/verisim.js:211 fetches repos.json at runtime, so this is what the worker actually serves.

Note worker/test/data.test.js opens with:

// so snapshot drift (e.g. a repo added to repos.json but not re-indexed) fails CI.

which is precisely this condition. Either that test is not running in CI, or it does not assert the count. Worth checking — the guard appears to be there but not catching it.

2. Phantom entry: verisim-data

worker/data/repos.json (and 5 places in index.json) carry:

{
  "name": "verisim-data",
  "uuid": "22772417-fc00-5263-a78a-7567aee04984",
  "github": "hyperpolymath/verisim-data"
}
  • hyperpolymath/verisim-data404. metadatastician/verisim-data404. The repo does not exist.
  • It is not in verisim/seed/repos.a2ml.
  • The real repo is verisimdb-data, which exists and is in the seed (line 374) and already in the JSON.

This looks like unfinished business from #46 ("dedupe verisimdb-data"): the seed was deduped, but worker/data/ was never regenerated, so the phantom survives in the served data. Its uuid derives from a URL that does not resolve.

3. export-json.sh is not what produced the committed data

This is the reason I did not just regenerate and open a PR — it is not clear which side is canonical:

committed worker/data/repos.json sync/export-json.sh output
format pretty-printed, 3646 lines compact, 321 lines
unicode escaped () literal ()
entries 320 (incl. phantom) 319

awk printf cannot produce escaping, so the committed file was not generated by export-json.sh. Either the script is stale/superseded, or the data is maintained by another path and the script is vestigial. Running it today would fix the phantom and the 116 missing repos, but reformat ~3,300 lines and assume it is canonical.

That decision is the owner’s, which is why this is an issue and not a PR.

4. Also stale (minor)

verisim/seed/repos.a2ml [meta] total-repos = 200, actual entry count 319.

Suggested order

  1. Decide which generator is canonical (or that the data is hand-maintained and export-json.sh should be removed).
  2. Fix that generator to emit the canonical format, so regenerating is a no-op except for real changes.
  3. Regenerate → drops the phantom, restores the 116 missing repos, fixes total_repos.
  4. Make worker/test/data.test.js actually fail on this drift, since it is written as though it should.

Related

🤖 Generated with Claude Code

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions