Skip to content

fix: agent-fix.py improvements (structured API, prompt fixes, pre-commit)#6065

Merged
Eren-Jeager123 merged 3 commits into
mainfrom
fix/fstring-syntax
May 8, 2026
Merged

fix: agent-fix.py improvements (structured API, prompt fixes, pre-commit)#6065
Eren-Jeager123 merged 3 commits into
mainfrom
fix/fstring-syntax

Conversation

@Eren-Jeager123
Copy link
Copy Markdown
Contributor

@Eren-Jeager123 Eren-Jeager123 commented May 8, 2026 •
edited
Loading

Improvements to the currency fix agent based on end-to-end testing.

Validated: Agent successfully fixed a CVE scan failure and correctly identified a transient HuggingFace error. See commit 070797c0 in history.

Changes:

  1. Log extraction via GitHub API — Identifies failed job via /actions/runs/{id}/jobs, downloads run zip, extracts only that job's log file (matched by name). No grep, no noise from other jobs.

  2. Only process tracked jobs — Skips check-changes, gatekeeper, etc. Only: build-image, sanity-test, security-test, telemetry-test, upstream-tests, sagemaker-test.

  3. System prompt — No angle brackets in format example. Added: "ONLY edit files provided in context."

  4. Filepath parsing — Strips LLM formatting artifacts from paths.

  5. Pre-commit clean — ruff format + lint.

  6. Workflow — Removed log zip download step. Pass --run-ids and --token to script.

@Eren-Jeager123 Eren-Jeager123 force-pushed the fix/fstring-syntax branch 2 times, most recently from 0c9b862 to 643c18f Compare May 8, 2026 07:09
@Eren-Jeager123 Eren-Jeager123 changed the title fix: f-string syntax error on Python 3.9 fix: end-to-end agent fixes + PR trigger for testing May 8, 2026
@Eren-Jeager123 Eren-Jeager123 force-pushed the fix/fstring-syntax branch 5 times, most recently from 3d1d142 to 20ba869 Compare May 8, 2026 07:32
@Eren-Jeager123 Eren-Jeager123 changed the title fix: end-to-end agent fixes + PR trigger for testing fix: end-to-end agent testing + bug fixes May 8, 2026
@aws-deep-learning-containers-ci
Copy link
Copy Markdown
Contributor

aws-deep-learning-containers-ci Bot commented May 8, 2026

🤖 Currency Fix Agent running (attempt 1/3).

Agent run: https://github.com/aws/deep-learning-containers/actions/runs/25543201372

@Eren-Jeager123 Eren-Jeager123 force-pushed the fix/fstring-syntax branch 6 times, most recently from f04b64d to dbac551 Compare May 8, 2026 08:47
@aws-deep-learning-containers-ci
Copy link
Copy Markdown
Contributor

aws-deep-learning-containers-ci Bot commented May 8, 2026

🤖 Currency Fix Agent running (attempt 1/3).

Agent run: https://github.com/aws/deep-learning-containers/actions/runs/25546329684

@aws-deep-learning-containers-ci
Copy link
Copy Markdown
Contributor

aws-deep-learning-containers-ci Bot commented May 8, 2026

🤖 Currency Fix Agent running (attempt 1/3).

Agent run: https://github.com/aws/deep-learning-containers/actions/runs/25568899224

@Eren-Jeager123 Eren-Jeager123 force-pushed the fix/fstring-syntax branch 5 times, most recently from e851e75 to 94146dd Compare May 8, 2026 18:00
@aws-deep-learning-containers-ci
Copy link
Copy Markdown
Contributor

aws-deep-learning-containers-ci Bot commented May 8, 2026

🤖 Currency Fix Agent running (attempt 1/3).

Agent run: https://github.com/aws/deep-learning-containers/actions/runs/25571185071

@aws-deep-learning-containers-ci
Copy link
Copy Markdown
Contributor

aws-deep-learning-containers-ci Bot commented May 8, 2026

🤖 Currency Fix Agent running (attempt 1/3).

Agent run: https://github.com/aws/deep-learning-containers/actions/runs/25571410235

@aws-deep-learning-containers-ci
Copy link
Copy Markdown
Contributor

aws-deep-learning-containers-ci Bot commented May 8, 2026

🤖 Currency Fix Agent running (attempt 1/3).

Agent run: https://github.com/aws/deep-learning-containers/actions/runs/25571497975

@Eren-Jeager123
fix: end-to-end agent testing on this PR
8220e05 
- Use GitHub API for structured failure info (which job/step failed + targeted log)
  instead of noisy grep across all logs
- Fix system prompt: concrete example, no angle brackets
- Fix filepath parsing edge cases
- Fix ruff: lambda → def, formatting
- Add 15-min wait for PR CI security scan to complete
- Add aiohttp==3.9.0 to Dockerfile (CVE trigger)
- Add pull_request trigger with Dockerfile path

TODO after validation: remove pull_request trigger, remove aiohttp, restore branch validation
@aws-deep-learning-containers-ci
Copy link
Copy Markdown
Contributor

aws-deep-learning-containers-ci Bot commented May 8, 2026

🤖 Currency Fix Agent running (attempt 1/3).

Agent run: https://github.com/aws/deep-learning-containers/actions/runs/25573186891

[agent-fix] Fix CVE-2024-52304, CVE-2024-23334, CVE-2025-53643, CVE-2…
070797c 
…024-30251 by upgrading aiohttp from pinned 3.9.0 to >=3.12.14
@aws-deep-learning-containers-ci
Copy link
Copy Markdown
Contributor

aws-deep-learning-containers-ci Bot commented May 8, 2026

🤖 Currency Fix Agent running (attempt 2/3).

Agent run: https://github.com/aws/deep-learning-containers/actions/runs/25574275091

@Eren-Jeager123 Eren-Jeager123 changed the title fix: end-to-end agent testing + bug fixes fix: agent-fix.py improvements (structured API, prompt fixes, pre-commit) May 8, 2026
@Eren-Jeager123 Eren-Jeager123 force-pushed the fix/fstring-syntax branch 3 times, most recently from 9633ade to 7b253ff Compare May 8, 2026 20:34
@Eren-Jeager123
chore: revert test-only changes, keep real fixes
5958a46 
Revert:
- pull_request trigger
- aiohttp==3.9.0 in Dockerfile (and agent's >=3.12.14 fix)
- Polling loop
- Branch validation skip
- Framework detection from workflow name

Keep:
- Structured API failure extraction (no grep)
- Only process tracked jobs (skip check-changes/gatekeeper)
- System prompt: no angle brackets, only edit files in context
- Filepath parsing fixes
- Pass --run-ids and --token to agent script
- Remove log zip download step (API handles it)
- Pre-commit clean (ruff format + lint)
@Eren-Jeager123 Eren-Jeager123 merged commit ae5f99c into main May 8, 2026
4 checks passed
@Eren-Jeager123 Eren-Jeager123 deleted the fix/fstring-syntax branch May 8, 2026 21:30
Yadan-Wei pushed a commit that referenced this pull request May 8, 2026
chore: sync agent-fix.py with main (structured GitHub API + prompt fi…
2a94ad3 
…xes)

origin/main #6065 refactored agent-fix.py to use the GitHub API for
structured failure extraction and tweaked the system prompt format. Pull
the upstream version verbatim so future merges from main don't conflict
with our prior local ruff fix.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

Signed-off-by: Yadan Wei <yadanwei@amazon.com>
Yadan-Wei added a commit that referenced this pull request May 12, 2026
@Yadan-Wei
feat(vllm-omni): bump to v0.20.0 and expand smoke-test matrix (#6028)
9120403 
* feat(vllm-omni): bump to 0.20.0rc1 and align with upstream vllm v0.20.0

- CUDA 12.9.1 → 13.0.2 (matches upstream vllm v0.20.0 default)
- vllm 0.18.0 → 0.20.0
- vllm-omni 0.18.0 → 0.20.0rc1 (release candidate; install with
  --prerelease=allow)
- flashinfer 0.6.6 → 0.6.8.post1
- torch_cuda_arch_list aligned with upstream:
  '7.5 8.0 8.6 8.9 9.0 10.0 11.0 12.0+PTX'
- nvcc_threads 2 → 8
- runai-model-streamer >= 0.15.3 → 0.15.7
- Add numactl / numactl-libs / numactl-devel in runtime stage
- Add VLLM_ENABLE_CUDA_COMPATIBILITY=0 env (settable to 1 at runtime
  for hosts with older NVIDIA drivers)
- Drop sox system dep (vllm-omni v0.20.0rc1 removed sox from its deps)
- Bump --stage-init-timeout in model-tests workflow 600 → 900 to
  match vllm-omni benchmark default after the stage CLI refactor
- Image config files: cuda_version cu129 → cu130, prod_image 0.18 → 0.20

Entrypoint scripts (scripts/vllm/omni_*) unchanged — 'vllm serve --omni'
still works in 0.20.0 via upstream vllm's --omni delegation.

Signed-off-by: Yadan Wei <yadanwei@amazon.com>

* fix(vllm-omni): split cuda/build requirements install for vllm v0.20.0

Upstream vllm v0.20.0 moved requirements/build.txt → requirements/build/cuda.txt
(the root build.txt no longer exists, causing 'uv pip install -r
requirements/build.txt' to fail with exit code 2).

Mirror upstream by installing cuda.txt and build/cuda.txt in two separate
RUN steps. The base vllm Dockerfile does the same split across its 'base'
and 'csrc-build' stages.

Fixes the build failure:

  ERROR: failed to solve: process "/bin/sh -c uv pip install -r
  requirements/cuda.txt -r requirements/build.txt ..." did not complete
  successfully: exit code: 2

Signed-off-by: Yadan Wei <yadanwei@amazon.com>

* refactor(vllm-omni): align Dockerfile with upstream vllm v0.20.0 and sibling DLC

Port hardening changes from the sibling docker/vllm/Dockerfile.amzn2023
and match upstream vllm v0.20.0 patterns for FlashInfer install:

- Pre-built wheel stage: add 'rm -rf dist' before recreating the dist/
  directory so stale artifacts from a previous build layer can't leak in.
- SETUPTOOLS_SCM_PRETEND_VERSION is now overrideable via --build-arg
  (e.g. 0.20.0rc1+amzn2023.abcdef12 with git SHA), falling back to
  VLLM_VERSION+amzn2023 when not provided.
- vLLM wheel install now picks the most recent vllm-*.whl by mtime and
  echoes which wheel it installs — defends against accidental glob
  matches when multiple wheels end up in the deps/ staging dir.
- FlashInfer install matches upstream vllm v0.20.0 exactly: only
  flashinfer-jit-cache is explicitly installed (flashinfer-python and
  flashinfer-cubin are already pinned in requirements/cuda.txt),
  followed by 'flashinfer show-config && flashinfer download-cubin' to
  pre-download precompiled kernels so the first inference request does
  not pay JIT compile latency.
- Serving extras: drop hf_transfer (superseded by HF_XET_HIGH_PERFORMANCE
  below) and match upstream's package set.
- Switch HF Hub acceleration from HF_HUB_ENABLE_HF_TRANSFER=1 to
  HF_XET_HIGH_PERFORMANCE=1, matching the sibling vllm Dockerfile and
  HF's direction of travel.

Signed-off-by: Yadan Wei <yadanwei@amazon.com>

* chore(vllm-omni): compute SETUPTOOLS_SCM_PRETEND_VERSION in versions.env

Pair with the Dockerfile change that made SETUPTOOLS_SCM_PRETEND_VERSION
overrideable via --build-arg. Compute the wheel version tag here so it
gets auto-forwarded to docker buildx and encodes the pinned VLLM_REF
(tag slug or commit SHA prefix) alongside VLLM_VERSION for traceability.

Mirrors the same pattern in docker/vllm/versions.env.

Example for the current pin:
  VLLM_REF=v0.20.0
  VLLM_VERSION=0.20.0
  -> SETUPTOOLS_SCM_PRETEND_VERSION=0.20.0+amzn2023.v0_20_0

Signed-off-by: Yadan Wei <yadanwei@amazon.com>

* fix(vllm-omni): explicit cuda-compat-13-0 upgrade for CVE-2025-33219

The blanket 'dnf upgrade --security' in the final stages only picks up

fixes from AL2023's security-advisory channel. The cuda-compat-13-0 fix

ships in NVIDIA's CUDA repo, which doesn't emit AL2023 advisory metadata,

so --security misses it, leaving the HIGH CVE unpatched in the scanned image.

Mirror the pattern already used in docker/base/v2/Dockerfile and

docker/pytorch/Dockerfile.cuda: add an explicit

'dnf upgrade -y --releasever latest cuda-compat-13-0' step in both

final stages (EC2 + SageMaker).

Patches: CVE-2025-33219 (cuda-compat-13-0 580.95.05 -> 1:580.126.09-1.amzn2023)
Signed-off-by: Yadan Wei <yadanwei@amazon.com>

* feat(vllm-omni): add stable-audio-open-1.0 to smoke-test matrix

Exercises the new /v1/audio/generate route introduced in upstream

vllm-omni v0.20.0 via PR vllm-project/vllm-omni#1794.

Validated on the 0.20.0 DLC image (CUDA 13, vllm 0.20.0):

  - Model loads in ~29s on 1x L4 (24GB VRAM, 3GB peak post-load)

  - /v1/audio/generate returns HTTP 200 in ~7s with a 5s audio_length

  - Response is valid WAV (PCM 16-bit stereo 44.1kHz, ~860KB for 5s)

  - Payload matches the example in the upstream PR description.

Artifact pre-staged at:

  s3://dlc-cicd-models/omni-models/stable-audio-open-1.0.tar.gz (14.3 GB)

Fleet x86-g6xl-runner (L4, compute 8.9) is used instead of g6exl because

the model only needs ~3GB VRAM at runtime.

Signed-off-by: Yadan Wei <yadanwei@amazon.com>

* feat(vllm-omni): support ref_audio_s3 in smoke-test, add voice-clone TTS models

Adds qwen3-tts-12hz-1.7b-base and cosyvoice3-0.5b to the smoke-test

matrix. Both are voice-clone TTS models that require a base64-encoded

reference audio in the request, which previously could not be expressed

in the YAML test_request field (~450KB of base64 would exceed shell

argument limits).

Changes:

- Smoke-test scripts accept '@/path/to/file' in the request argument so

  the workflow can hand in large, preprocessed payloads without hitting

  the ~128KB bash argv limit.

- Workflow grows a 'Prepare test request' step that detects ref_audio_s3

  in the YAML test_request, fetches the wav from S3 on the runner

  (same IAM path the existing download-model action uses), base64-

  encodes it, substitutes ref_audio, and docker-cps the result into

  the container for the smoke-test script to read.

- CosyVoice3 is zero-shot voice-clone only; its reference fixture is

  mirrored from upstream tests/assets/cosyvoice3/zero_shot_prompt.wav

  to s3://dlc-cicd-models/test-fixtures/audio/cosyvoice3_ref.wav.

- qwen3-tts-12hz-1.7b-base reuses the already-staged tts_ref_vivian.wav

  that the benchmark suite uses; ref_text MUST match the audio transcript

  exactly (upstream issue vllm-project/vllm-omni#3124).

Existing entries with literal test_request strings keep working

unchanged (no @ prefix => script treats argument as literal body).

Signed-off-by: Yadan Wei <yadanwei@amazon.com>

* fix(vllm-omni): pass large request body to curl via file, not argv

Previous change (00bce6f) solved the argv-limit problem between the

workflow and the smoke-test script but re-introduced it one step later:

the script still ran 'curl -d "${REQUEST}"' with REQUEST holding the

~450KB JSON, which fails with /usr/bin/curl: Argument list too long.

Keep the body in a file throughout:

- If invoked with @file, use that path directly.

- Otherwise write the literal body to a temp file.

- curl now uses '-d @${REQUEST_FILE}' for JSON and reads the urlencoded

  pairs from the file line-by-line for multipart form-data.

Fixes voice-clone TTS smoke tests (qwen3-tts-12hz-1.7b-base and

cosyvoice3-0.5b).

Signed-off-by: Yadan Wei <yadanwei@amazon.com>

* feat(vllm-omni): bump to v0.20.0 final and expand smoke-test matrix

- versions.env / Dockerfile: vllm-omni 0.20.0rc1 → 0.20.0, drop --prerelease=allow
- model-tests.yml: add wan2.1-t2v-1.3b-sync, wan2.1-vace-1.3b, ernie-image-turbo
  on x86-g6exl-runner; pre-stage commented wan2.2-i2v-a14b for follow-up

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

Signed-off-by: Yadan Wei <yadanwei@amazon.com>

* chore: fix ruff lint+format on scripts/autocurrency/agent-fix.py

Unblocks pre-commit on PRs that merge into main: ruff-format reflow plus
E731 (lambda → def) in find_match.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

Signed-off-by: Yadan Wei <yadanwei@amazon.com>

* fix: resolve agent-fix.py merge-conflict markers from main merge

The merge of main into omni-0.20.0 left 24 unresolved conflict markers
in scripts/autocurrency/agent-fix.py, breaking pre-commit. Restore our
ruff-formatted copy from da00c60 — same content as our prior fix.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

Signed-off-by: Yadan Wei <yadanwei@amazon.com>

* chore(vllm-omni): allowlist GHSA-98h9-4798-4q5v (diffusers trust_remote_code bypass)

diffusers>=0.38.0 fixes the CVE but transitively requires safetensors>=0.8.0-rc.0
which uv/pip skip by default (only safetensors 0.8.0rc0 published; 0.7.0 is
latest stable). Upstream vllm-omni#3349 tracks the bump and is waiting on
safetensors 0.8.0 final. DLC loads only pre-staged S3 models we control and
does not pass trust_remote_code=True from user code paths, so exploit surface
is narrow. Re-evaluate once safetensors 0.8.0 final ships.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

Signed-off-by: Yadan Wei <yadanwei@amazon.com>

* docs(vllm-omni): note --middleware contract in SageMaker proxy module

Document that we depend on vllm v0.20.0 still wiring args.middleware through
to FastAPI app.add_middleware in build_app. vllm-omni v0.20.0's "delegate to
upstream entrypoint" rebase (#3082, #3232) preserves this; if a future
upstream change drops the loop, this loader silently no-ops and SageMaker
/invocations returns 404 for non-default routes.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

Signed-off-by: Yadan Wei <yadanwei@amazon.com>

* test(vllm-omni): add async SageMaker endpoint test for video generation

Adds test_vllm_omni_video_async_endpoint covering the recommended production
pattern for video on SageMaker: AsyncInferenceConfig + /v1/videos/sync via the
custom-attributes routing middleware (which auto-converts JSON to
multipart/form-data for FORM_DATA_ROUTES). Uses Wan-AI/Wan2.1-VACE-1.3B-diffusers
on ml.g6e.xlarge (32 GB RAM avoids the host-RAM OOM that 16 GB instances hit
during HF model load).

Skips on SageMaker capacity errors (ResourceLimitExceeded /
InsufficientInstanceCapacity / CapacityError) so the rest of the matrix still
gets signal when newer GPU families are unavailable.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

Signed-off-by: Yadan Wei <yadanwei@amazon.com>

* chore: sync agent-fix.py with main (structured GitHub API + prompt fixes)

origin/main #6065 refactored agent-fix.py to use the GitHub API for
structured failure extraction and tweaked the system prompt format. Pull
the upstream version verbatim so future merges from main don't conflict
with our prior local ruff fix.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

Signed-off-by: Yadan Wei <yadanwei@amazon.com>

* refactor(vllm-omni): inline capacity-skip into async_endpoint fixture

Drops the duplicated _build_async_endpoint helper (added in 2393207) and
puts the SageMaker capacity-skip logic inside the async_endpoint fixture
itself. Both async tests (TTS and video) now share the same fixture body
and both gain the capacity-skip behavior. Net: 144 LOC deleted, 83 added.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

Signed-off-by: Yadan Wei <yadanwei@amazon.com>

* chore(vllm-omni): allowlist 2 go/stdlib CVEs in mooncake/libetcd_wrapper.so

CVE-2026-33811 (cgo DNS resolver double-free on long CNAME) and CVE-2026-39820
(net/mail and time.ParseDate CPU/memory exhaustion). Both are go/stdlib
issues vendored into mooncake/libetcd_wrapper.so; same unpatchable-without-
mooncake-upgrade situation as the 5 existing mooncake entries above.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

Signed-off-by: Yadan Wei <yadanwei@amazon.com>

* test(vllm-omni): switch video async endpoint test to ml.g5.2xlarge

ml.g6e.xlarge (the previous default) had quota=0 in our SageMaker accounts,
causing the capacity-skip path to permanently swallow this test. Validated
ml.g5.2xlarge end-to-end on 2026-05-11: 45 KB MP4 returned in 10s, peak GPU
memory comfortably below A10G's 24 GB ceiling, and 32 GB host RAM avoids
the OOM-during-HF-load that bit us on 16 GB instances. A10G uses PyTorch
SDPA fallback for diffusion attention (no FA3 dependency).

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

Signed-off-by: Yadan Wei <yadanwei@amazon.com>

* chore(vllm-omni): allowlist 3 more mooncake go/stdlib CVEs

CVE-2026-42499 (RFC 5322 consumePhrase DoS), CVE-2026-39836 (Windows Dial/
LookupPort NUL panic), CVE-2026-33814 (HTTP/2 SETTINGS_MAX_FRAME_SIZE=0
infinite CONTINUATION loop). All vendored into mooncake/libetcd_wrapper.so;
same unpatchable-without-mooncake-upgrade situation as the existing entries.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

Signed-off-by: Yadan Wei <yadanwei@amazon.com>

* ci(vllm-omni): rename prod_image tags + bump cosyvoice fleet to g6e.xlarge

- image configs: prod_image renamed to vllm:omni-cuda-v1 / vllm:omni-sagemaker-cuda-v1
- cosyvoice3-0.5b smoke test: x86-g6xl-runner (16 GB RAM) → x86-g6exl-runner
  (32 GB RAM). Last green run was 2026-05-07 on vllm-omni 0.20.0rc1;
  --trust-remote-code load on g6.xlarge started SIGKILL'ing the host docker
  exec under 0.20.0 final. The bigger box matches our pattern for other
  diffusion-pipeline models (wan2.1-vace, ernie-image-turbo).

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

Signed-off-by: Yadan Wei <yadanwei@amazon.com>

* chore(vllm-omni): bump DLC_MINOR_VERSION 0 → 1

Increments the dlc_minor_version Dockerfile label to mark the 0.20.0 final
release (with new smoke-test entries, async video endpoint test, and CVE
allowlist updates) as a new image revision distinct from the initial 0.20.0
build.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

Signed-off-by: Yadan Wei <yadanwei@amazon.com>

---------

Signed-off-by: Yadan Wei <yadanwei@amazon.com>
Co-authored-by: Yadan Wei <yadanwei@amazon.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@sallyseok sallyseok sallyseok approved these changes

Assignees

No one assigned

Labels

authorized

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

2 participants

@Eren-Jeager123 @sallyseok