Simplify reusable sweep ingestion with direct source dispatch / 通过直接分发源运行简化复用扫描摄取 (#2244)

* fix: select latest reused sweep artifacts

中文：修复复用扫描产物的选择逻辑，只保留每个逻辑基准测试最新上传的产物。

* refactor: dispatch reusable sweep sources directly

Remove the reusable artifact relay and send source and merge run IDs directly to InferenceX-app. Simplify manual recovery to the same two-ID dispatch and add structural workflow tests.

移除可复用工件中转流程，直接将源运行 ID 和合并运行 ID 发送到 InferenceX-app。手动恢复流程也简化为相同的双 ID 分发，并新增工作流结构测试。

* docs: update reusable ingest recovery flow

Replace stale relay-job references with the direct source/merge dispatch and app-side preparation and validation steps.

将过时的中转作业引用替换为直接的源运行/合并运行分发，以及应用侧的工件准备和验证步骤。

* docs: simplify reusable ingest guidance

Document that InferenceX-app downloads normal source artifacts, substitutes the merge changelog, and relies on existing ingestion behavior for failed rows. Remove obsolete reusable-artifact validation instructions.\n\n中文：说明 InferenceX-app 直接下载常规源运行产物、替换合并运行变更日志，并沿用现有摄取逻辑跳过失败结果。删除过时的复用产物验证说明。

Author: cquil11

.claude/commands/recover-failed-ingest.md

@@ -4,8 +4,8 @@ argument-hint: <failed-run-or-job-url | pr-number> [source-run-id]
 ---
 
 Recover the official database ingest for a failed or skipped InferenceX
-push-to-main `Run Sweep` workflow by creating a recovery PR that reuses validated
-artifacts from an earlier PR sweep. Do not add a one-off recovery workflow.
+push-to-main `Run Sweep` workflow by creating a recovery PR that reuses artifacts
+from an earlier PR sweep. Do not add a one-off recovery workflow.
 
 Inputs from `$ARGUMENTS`:
 
@@ -29,9 +29,8 @@ Run from a clean InferenceX checkout with authenticated `gh`, `git`, `jq`, and
   `.github/workflows/run-sweep.yml` on `main` whose official ingest did not
   complete.
 - Reuse only a completed `pull_request` run of `run-sweep.yml`. Unpinned reuse
-  requires success. A specifically pinned failed run is allowed only when
-  artifact validation proves its available result set is internally consistent;
-  only completed points are recovered.
+  requires success. A specifically pinned failed run is allowed; normal
+  ingestion skips failed benchmark rows, so only completed points are recovered.
 - The source run must belong to the original PR being recovered.
 - Stop if that PR changed the recovered configuration's execution semantics
   after the source SHA: image, model, recipe, runner, launcher, benchmark
@@ -111,10 +110,10 @@ TARGET_RUN_ID=<matching-run-id>
 Require event `push`, workflow path `.github/workflows/run-sweep.yml`, and branch
 `main`; confirm the target is no longer running before recovering. The
 disqualifying state is broader than `failure`/`skipped`: when `/reuse-sweep-run`
-was forgotten before merge, `reuse-ingest-artifacts` is skipped, the GPU jobs run
-(often `cancelled` to save cost), and because `collect-results`/`collect-evals`
-are not skipped, `trigger-ingest` still fires `always()` and lands a *bogus*
-ingest under the target's own `run_id`. So a target showing
+was forgotten before merge, setup leaves reuse disabled, the GPU jobs run (often
+`cancelled` to save cost), and because `collect-results`/`collect-evals` are not
+skipped, `trigger-ingest` still fires `always()` and lands a *bogus* ingest under
+the target's own `run_id`. So a target showing
 `trigger-ingest=success` (and concluding `success` or `cancelled`) can still hold
 no valid benchmark data — recovery is required. That bogus row is keyed on the
 target `run_id` and is superseded by the recovery ingest under a new `run_id`;
@@ -132,7 +131,7 @@ git diff "$ORIGINAL_BASE_SHA" "$ORIGINAL_MERGE_SHA" -- \
   perf-changelog.yaml
 ```
 
-## 2. Select and validate the source run
+## 2. Select the source run
 
 Find candidates from the original PR branch when no run ID was supplied:
 
@@ -238,7 +237,7 @@ Keep exactly one of `full-sweep-enabled`,
 `non-canary-full-sweep-enabled`, `full-sweep-fail-fast`, or
 `full-sweep-fail-fast-no-canary`.
 
-## 5. Append the recovery changelog and validate source artifacts
+## 5. Append and validate the recovery changelog
 
 Append recovery entries to the end of `perf-changelog.yaml`. Preserve the
 original entries' `config-keys`, `description`, `evals-only`, and
@@ -281,61 +280,6 @@ python3 utils/process_changelog.py \
 Confirm the generated config contains only the intended recovery scope. Its row
 counts may differ from the source run.
 
-Download only the result artifacts needed for local validation. This avoids the
-large server-log artifacts retained in the official ingest bundle. Raw per-config
-`bmk_<model>_*` artifacts are intentionally not selected — they fall through the
-`case` below; the aggregate `results_bmk` is what the validator reads:
-
-```bash
-rm -rf /tmp/source-artifacts
-ARTIFACT_ARGS=()
-while IFS= read -r name; do
-  case "$name" in
-    results_bmk|eval_results_all|run-stats|bmk_agentic_*|agentic_*)
-      ARTIFACT_ARGS+=(-n "$name")
-      ;;
-    eval_server_logs_*|eval_gpu_metrics_*)
-      ;;
-    eval_*)
-      ARTIFACT_ARGS+=(-n "$name")
-      ;;
-  esac
-done < <(
-  gh api \
-    "repos/SemiAnalysisAI/InferenceX/actions/runs/$SOURCE_RUN_ID/artifacts?per_page=100" \
-    --paginate --jq '.artifacts[] | select(.expired == false) | .name' |
-    sort -u
-)
-
-((${#ARTIFACT_ARGS[@]})) || {
-  echo "No unexpired result artifacts found" >&2
-  exit 1
-}
-gh run download "$SOURCE_RUN_ID" \
-  --repo SemiAnalysisAI/InferenceX \
-  -D /tmp/source-artifacts \
-  "${ARTIFACT_ARGS[@]}"
-```
-
-Validate the source artifacts:
-
-```bash
-python3 utils/validate_reusable_sweep_artifacts.py \
-  --artifacts-dir /tmp/source-artifacts
-```
-
-The validator first collapses reran (flaky) eval duplicates in place — keeping
-the latest result per eval identity when a retried eval left duplicate raw dirs
-/ aggregate rows — so a legitimate rerun does not fail validation. It only
-collapses identities with a clear latest result; genuinely ambiguous duplicates
-are still rejected.
-
-The validator does not compare source coverage with
-`/tmp/recovery-full-config.json`. It rejects duplicate fixed rows, missing run
-stats, inconsistent agentic artifacts, malformed eval metadata, raw/aggregate
-eval mismatches, or an empty result set. For a pinned failed batched eval run,
-only `completed_eval_concs` are recovered.
-
 ## 6. Attach the source SHA without changing the tree
 
 Make the ancestry carrier the final branch commit. `git commit-tree` guarantees
@@ -449,15 +393,15 @@ The push-to-main `Run Sweep` must:
 - run `setup` even if the merge message contains `[skip-sweep]`;
 - resolve the recovery PR and pinned source run;
 - set `reuse-enabled=true`;
-- pass `reuse-ingest-artifacts` consistency validation;
 - upload recovery changelog metadata;
-- run `trigger-ingest`.
+- dispatch `source-run-id` and `merge-run-id` from `trigger-ingest`.
 
 Then locate the resulting `repository_dispatch` run in
 `SemiAnalysisAI/InferenceX-app`. In the forgotten-`/reuse` case the target's
 bogus ingest is also a recent successful `ingest-results` run, so do not pick by
-recency — pick the run whose `Download artifacts from InferenceX run` step logs
-`RUN_ID: <RECOVERY_RUN_ID>`:
+recency — pick the run whose `Prepare artifacts from InferenceX` step logs both
+the expected source run and recovery merge run. That app workflow must download
+the source artifacts, substitute the merge changelog, and complete ingestion:
 
 ```bash
 gh run list --repo SemiAnalysisAI/InferenceX-app \
@@ -467,19 +411,22 @@ gh run list --repo SemiAnalysisAI/InferenceX-app \
 
 INGEST_RUN_ID=<candidate-run-id>
 gh run view "$INGEST_RUN_ID" --repo SemiAnalysisAI/InferenceX-app --log \
-  | grep -m1 "RUN_ID: $RECOVERY_RUN_ID"   # must match before you trust this run
+  | grep -m1 "Source run: $SOURCE_RUN_ID" # must match before you trust this run
+
+gh run view "$INGEST_RUN_ID" --repo SemiAnalysisAI/InferenceX-app --log \
+  | grep -m1 "Merge run:  $RECOVERY_RUN_ID" # must also match
 
 gh run watch "$INGEST_RUN_ID" \
   --repo SemiAnalysisAI/InferenceX-app --exit-status
 ```
 
-The ingest's first step is a `sleep 300` "wait for source run to finish", so the
-run idles ~5 minutes before doing work — that is normal, not a hang.
+Reused ingests skip the normal five-minute wait because the source run is already
+complete.
 
 Verify its logs identify `RECOVERY_RUN_ID` as the trigger and `SOURCE_RUN_ID`
 plus `SOURCE_RUN_ATTEMPT` as the reused source. Require successful artifact
-download, flattening, database ingest, run overrides, database verification,
-cache invalidation, and unmapped-entity checks.
+download, database ingest, run overrides, database verification, cache
+invalidation, and unmapped-entity checks.
 
 Post a final recovery PR comment with the original failed or skipped run/job,
 source run/attempt/SHA, recovery merge run, downstream ingest run, recovered

.github/workflows/README.md

@@ -177,21 +177,21 @@ in the PR. A run ID can pin an eligible successful or failed run:
 /reuse-sweep-run <run_id>
 ```
 
-Failed-run artifacts must still validate. The latest matching comment by an
-`OWNER`, `MEMBER`, or `COLLABORATOR` wins. Comments do not trigger or cancel
-sweeps; later commits skip a new sweep after changelog/matrix validation.
+The latest matching comment by an `OWNER`, `MEMBER`, or `COLLABORATOR` wins.
+Comments do not trigger or cancel sweeps; later commits skip a new sweep after
+changelog/matrix validation.
 Remove and re-add the sweep label to force one.
 
 `utils/merge_with_reuse.sh <pr-number>` is the supported merge path for reuse.
 It merges `main`, preserves changelog bytes, fixes an appended `XXX` PR link,
 pushes a synchronization commit, waits for checks, then merges.
 
-The main run verifies the source, validates and uploads its ingest artifacts,
-then ingests them with merge-run changelog metadata. Source coverage is
+The main run passes the selected source run ID and its own merge run ID directly
+to InferenceX-app. The app downloads source artifacts, keeps the newest upload
+for each exact artifact name, and ingests them with changelog metadata from the
+merge run. The normal ingestion code skips failed benchmark rows. Benchmark
+rows and public links retain source-run provenance. Source coverage is
 authoritative, so later matrix/eval policy changes do not invalidate reuse.
-Validation rejects duplicate fixed rows, missing run stats, inconsistent
-agentic artifacts, malformed eval metadata, and raw/aggregate eval mismatches.
-Batched evals use only `completed_eval_concs`.
 
 Reuse fails closed when authorized but ineligible or invalid; without
 authorization, `main` runs the normal full sweep.

.github/workflows/recover-reused-ingest.yml

@@ -4,158 +4,16 @@ on:
     workflow_dispatch:
         inputs:
             source-run-id:
-                description: Source PR sweep run ID
-                required: true
-                type: string
-            source-run-attempt:
-                description: Source PR sweep attempt
-                required: true
-                default: "1"
-                type: string
-            source-pr-number:
-                description: Source PR number
-                required: true
-                type: string
-            source-head-sha:
-                description: Source sweep commit SHA
+                description: Source PR sweep run ID containing benchmark artifacts
                 required: true
                 type: string
             merge-run-id:
-                description: Original failed merge run ID
+                description: Main publication run ID containing changelog metadata
                 required: true
                 type: string
-            merge-run-attempt:
-                description: Original failed merge run attempt
-                required: true
-                default: "1"
-                type: string
 
 jobs:
-    package-reused-artifacts:
-        runs-on: blacksmith-4vcpu-ubuntu-2404
-        permissions:
-            contents: read
-        steps:
-            - uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7.0.0
-
-            - name: Download source artifacts
-              env:
-                  GH_TOKEN: ${{ secrets.REPO_PAT || github.token }}
-                  SOURCE_RUN_ID: ${{ inputs.source-run-id }}
-                  ARTIFACTS_PATH: ${{ github.workspace }}/source-artifacts
-              run: |
-                  mkdir -p "$ARTIFACTS_PATH"
-
-                  download_artifact() {
-                      local name="$1"
-                      local url="$2"
-                      local zip_path="$RUNNER_TEMP/artifact.zip"
-
-                      echo "Downloading artifact: $name"
-                      for attempt in 1 2 3; do
-                          if gh api "$url" > "$zip_path"; then
-                              mkdir -p "$ARTIFACTS_PATH/$name"
-                              if unzip -oq "$zip_path" -d "$ARTIFACTS_PATH/$name"; then
-                                  rm -f "$zip_path"
-                                  return 0
-                              fi
-                              rm -rf "${ARTIFACTS_PATH:?}/$name"
-                          fi
-                          rm -f "$zip_path"
-                          echo "  Attempt $attempt/3 failed, retrying in ${attempt}s..."
-                          sleep "$attempt"
-                      done
-                      return 1
-                  }
-
-                  gh api "repos/${{ github.repository }}/actions/runs/${SOURCE_RUN_ID}/artifacts?per_page=100" --paginate --slurp \
-                    | jq -r '
-                        [.[].artifacts[]
-                          | select(.name != "changelog-metadata")
-                          | . + {logical_name: (.name | sub("_[A-Za-z][A-Za-z0-9.-]*_[0-9]+$"; ""))}
-                        ]
-                        | group_by(.logical_name)
-                        | map(sort_by(.created_at) | last)[]
-                        | "\(.name)\t\(.archive_download_url)"' \
-                    | while IFS=$'\t' read -r name url; do
-                        if ! download_artifact "$name" "$url"; then
-                            echo "::error::Failed to download source artifact: $name"
-                            exit 1
-                        fi
-                    done
-
-                  if [ -z "$(find "$ARTIFACTS_PATH" -mindepth 1 -maxdepth 1 -print -quit)" ]; then
-                      echo "::error::No source artifacts downloaded from run $SOURCE_RUN_ID"
-                      exit 1
-                  fi
-
-            - name: Record reuse provenance
-              env:
-                  ARTIFACTS_PATH: ${{ github.workspace }}/source-artifacts
-                  SOURCE_RUN_ID: ${{ inputs.source-run-id }}
-                  SOURCE_RUN_ATTEMPT: ${{ inputs.source-run-attempt }}
-                  SOURCE_PR_NUMBER: ${{ inputs.source-pr-number }}
-                  SOURCE_HEAD_SHA: ${{ inputs.source-head-sha }}
-                  MERGE_RUN_ID: ${{ inputs.merge-run-id }}
-                  MERGE_RUN_ATTEMPT: ${{ inputs.merge-run-attempt }}
-              run: |
-                  mkdir -p "$ARTIFACTS_PATH/reused-ingest-metadata"
-                  jq -n \
-                    --arg source_run_id "$SOURCE_RUN_ID" \
-                    --arg source_run_attempt "$SOURCE_RUN_ATTEMPT" \
-                    --arg source_run_url "${{ github.server_url }}/${{ github.repository }}/actions/runs/$SOURCE_RUN_ID" \
-                    --arg source_pr_number "$SOURCE_PR_NUMBER" \
-                    --arg source_head_sha "$SOURCE_HEAD_SHA" \
-                    --arg ingest_run_id "$MERGE_RUN_ID" \
-                    --arg ingest_run_attempt "$MERGE_RUN_ATTEMPT" \
-                    --arg ingest_run_url "${{ github.server_url }}/${{ github.repository }}/actions/runs/$MERGE_RUN_ID" \
-                    '{
-                      source_run_id: $source_run_id,
-                      source_run_attempt: $source_run_attempt,
-                      source_run_url: $source_run_url,
-                      source_pr_number: $source_pr_number,
-                      source_head_sha: $source_head_sha,
-                      ingest_run_id: $ingest_run_id,
-                      ingest_run_attempt: $ingest_run_attempt,
-                      ingest_run_url: $ingest_run_url
-                    }' > "$ARTIFACTS_PATH/reused-ingest-metadata/reuse_source_run.json"
-
-            - name: Validate reusable artifacts
-              run: |
-                  python3 utils/validate_reusable_sweep_artifacts.py \
-                    --artifacts-dir "${{ github.workspace }}/source-artifacts"
-
-            - name: Download original merge changelog
-              env:
-                  GH_TOKEN: ${{ secrets.REPO_PAT || github.token }}
-                  MERGE_RUN_ID: ${{ inputs.merge-run-id }}
-                  CHANGELOG_PATH: ${{ github.workspace }}/merge-changelog
-              run: |
-                  url=$(gh api "repos/${{ github.repository }}/actions/runs/${MERGE_RUN_ID}/artifacts?per_page=100" --paginate --slurp \
-                    | jq -r '[.[].artifacts[] | select(.name == "changelog-metadata")] | sort_by(.created_at) | last | .archive_download_url // empty')
-                  if [ -z "$url" ]; then
-                      echo "::error::No changelog-metadata artifact found on run $MERGE_RUN_ID"
-                      exit 1
-                  fi
-                  mkdir -p "$CHANGELOG_PATH"
-                  gh api "$url" > "$RUNNER_TEMP/changelog.zip"
-                  unzip -oq "$RUNNER_TEMP/changelog.zip" -d "$CHANGELOG_PATH"
-                  rm -f "$RUNNER_TEMP/changelog.zip"
-
-            - name: Upload reusable ingest artifacts
-              uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1
-              with:
-                  name: reused-ingest-artifacts
-                  path: source-artifacts/*
-
-            - name: Upload merge changelog
-              uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1
-              with:
-                  name: changelog-metadata
-                  path: merge-changelog/*
-
     trigger-agentic-ingest:
-        needs: package-reused-artifacts
         runs-on: ubuntu-latest
         steps:
             - name: Trigger agentic database ingest
@@ -167,8 +25,8 @@ jobs:
                     -d '{
                       "event_type": "ingest-agentic-results",
                       "client_payload": {
-                        "run-id": "${{ github.run_id }}",
-                        "run-attempt": "${{ github.run_attempt }}",
+                        "source-run-id": "${{ inputs.source-run-id }}",
+                        "merge-run-id": "${{ inputs.merge-run-id }}",
                         "database-target": "production"
                       }
                     }'