Make result-submission form fields meaningful (dropdowns + registry-driven regen)

[mmcdermott]

Follow-up to PR #295's design discussion (thread).

Today the result-submission issue form has free-text input fields for dataset / task / model. The template's own preamble admits these are display-only — they're not validated, not used by the workflow, and not cross-checked against the JSON. So contributors can typo the dataset name and the submission still goes through; the JSON is the source of truth and the typed fields drift.

Make the typed fields actually mean something

Layer 1: dropdowns for dataset/task/model, regenerated from the registry

Replace the three input fields with dropdown fields whose options: list is generated from DATASETS / TASKS / MODELS in src/MEDS_DEV/:

- type: dropdown
  id: dataset
  attributes:
    label: "Dataset"
    options:
      - MIMIC-IV
      - AUMCdb
      # ... auto-generated
  validations:
    required: true

Drift problem: dropdown options are static in the YAML. Solution: a small workflow + CLI that regenerates .github/ISSUE_TEMPLATE/benchmark-result.yml from the registry on changes to src/MEDS_DEV/{datasets,tasks,models}/**. Mirrors how #285 regenerates _web/entities/*.json.

# .github/workflows/regenerate_issue_templates.yaml
on:
  push:
    branches: [main]
    paths:
      - "src/MEDS_DEV/datasets/**"
      - "src/MEDS_DEV/tasks/**"
      - "src/MEDS_DEV/models/**"

Plus a new CLI like meds-dev-regenerate-issue-templates (or a flag on meds-dev-collate-entities).

Layer 2: workflow-side cross-check (optional, cosmetic)

Once the dropdowns are in place, meds-dev-process-submission (PR #295) can parse the form sections out of the issue body and assert they match the JSON's dataset / task / model fields. Catches the case where someone selects "MIMIC-IV" from the dropdown but pastes a JSON with "dataset": "MIMIC-III" (wrong file).

Largely cosmetic — if Layer 1 is in place and Issue #B's client-side CLI is the preferred submission path, this just confirms the obvious. Worth landing alongside Layer 1 if it's cheap.

Layer 3 (stretch): JSON pattern validation

GitHub forms support validations.pattern (regex) on textarea. Won't fully validate JSON syntax but rejects the worst:

- type: textarea
  id: result
  validations:
    pattern: '^\s*\{[\s\S]*"result"\s*:[\s\S]*\}\s*$'

Catches empty submissions / wrong-content pastes at form-submit time, before the workflow fires.

Acceptance criteria

• meds-dev-regenerate-issue-templates (or equivalent) CLI walks the registries and rewrites the dropdown options: lists in .github/ISSUE_TEMPLATE/benchmark-result.yml.
• Workflow .github/workflows/regenerate_issue_templates.yaml runs the CLI on relevant pushes to main and commits the regenerated template.
• (Optional) meds-dev-process-submission parses the form sections and cross-checks against the JSON.
• (Optional) validations.pattern added to the result textarea.

Related

• PR Add meds-dev-process-submission CLI; collapse extract+validate #295 — discussion thread that surfaced this.
• Add meds-dev-submit-result CLI: client-side submission via gh #297 (client-side CLI submission) — alternative submission path that bypasses the form entirely. Both can land; they don't conflict.