Promote web tooling (collate_entities, aggregate_results) into the package

[mmcdermott]

Background

The _web branch holds JSON manifests (entities/datasets.json, entities/tasks.json, entities/models.json) that the MEDS website fetches at runtime to render the dataset / task / model catalog pages. These files are currently manually maintained and have drifted significantly from the source code (see #186, #187, #188 for specifics).

A working prototype of the regeneration logic exists on the unmerged add_web_scripts branch:

• src/MEDS_DEV/web/collate_entities.py — walks src/MEDS_DEV/{datasets,tasks,models}/ and produces the JSON manifests
• src/MEDS_DEV/web/aggregate_results.py — improved version of the existing _web/scripts/aggregate_results.py (currently lives only on _web, not in the package, not under CI)

Both have docstring tests and are written as proper package modules.

What needs to be done

1. Rebuild on current main: the branch is stale and the source layout has changed since it was written (e.g., _results task vs. nested category readmes, MIMIC-IV dataset.yaml updates, AUMCdb addition). Re-port the scripts cleanly rather than merging the branch as-is.
2. Move under the package: place at src/MEDS_DEV/web/{collate_entities,aggregate_results}.py. Keep doctests.
3. Expose CLI entry points in pyproject.toml:
   • meds-dev-collate-entities → writes datasets.json / tasks.json / models.json
   • meds-dev-aggregate-results → consolidates per-issue result.json blobs
4. Replace the legacy script on _web: once the package version is shipped, update aggregate_benchmark_results.yaml to install MEDS-DEV from PyPI and invoke meds-dev-aggregate-results, then delete _web/scripts/aggregate_results.py.
5. Delete or archive the add_web_scripts branch once content is incorporated (currently a dangling reference).

Acceptance criteria

• meds-dev-collate-entities and meds-dev-aggregate-results are installable from PyPI as CLI commands.
• Doctests for both run as part of the fast test lane.
• aggregate_benchmark_results.yaml no longer depends on a script committed to _web.
• Resolves We need to export a code-endpoint releasing the information about supported datasets in a friendly format. #186, We need to export a code-endpoint releasing the information about supported tasks in a friendly format. #187, We need to export a code-endpoint releasing the information about supported models in a friendly format. #188 (collation), partially addresses We should expose an endpoint in the MEDS-DEV codebase to report in structured form the information necessary about current models, tasks, datasets, and results for display in the website #63.

Related

• Blocks website integration work tracked in MEDS-DEV needs a website for leaderboards and updated documentation. #148
• Sibling: workflow-side automation tracked separately (see new issue for entity-regeneration workflow)