Skip to content

Inline local rollout images in PrimeMonitor samples#2679

Open
mrmoxon wants to merge 6 commits into
mainfrom
fix/rft-rollout-image-data-urls
Open

Inline local rollout images in PrimeMonitor samples#2679
mrmoxon wants to merge 6 commits into
mainfrom
fix/rft-rollout-image-data-urls

Conversation

@mrmoxon
Copy link
Copy Markdown

@mrmoxon mrmoxon commented Jun 1, 2026

Summary

  • Convert local file:// image references in PrimeMonitor rollout samples back into data:image/...;base64,... URLs before writing the sample Parquet.
  • Make offload_images_to_disk() emit absolute file URIs via Path.as_uri(), so relative output_dir configs do not produce malformed file://outputs/... URLs.
  • Preserve the source image media type when offloading images, so JPEG/WebP/etc. inputs are not re-uploaded as data:image/png with mismatched bytes.
  • Recursively handles image URLs in prompt, completion, and trajectory payloads without mutating the original rollout objects during monitor serialization.
  • Adds a 2 MiB per-image inline cap so sample uploads do not accidentally absorb large local artifacts.
  • Keeps the PR limited to production code changes; test-file changes were removed per reviewer request.

Why

offload_images_to_disk() rewrites inline multimodal images to local paths under the orchestrator output directory, e.g. file:///data/outputs/.../assets/images/.... That is fine inside the training pod, but those paths are later stored in PrimeMonitor sample Parquet. The hosted dashboard only sees that file:// pointer and cannot load the image from a browser.

Inlining small local images during PrimeMonitor serialization keeps future uploaded samples browser-renderable without adding a new backend artifact upload/signed URL path.

Validation

  • uvx ruff check src/prime_rl/orchestrator/trajectories.py src/prime_rl/utils/monitor/prime.py
  • uvx ruff format --check src/prime_rl/orchestrator/trajectories.py src/prime_rl/utils/monitor/prime.py
  • uv run --no-sync python -m py_compile src/prime_rl/orchestrator/trajectories.py src/prime_rl/utils/monitor/prime.py
  • Confirmed git diff --name-status origin/main...HEAD only lists production files under src/.
  • Remote test machine root@95.133.252.95 at current head: ruff check/format and py_compile passed for touched production files.
  • Remote test machine root@95.133.252.95: custom offload → PrimeMonitor Parquet smoke passed for relative output dirs, monitor non-mutation, completion image inlining, and >2 MiB image skip behavior.
  • Remote test machine root@95.133.252.95: real-image MIME smoke verified PNG stays data:image/png and JPEG stays data:image/jpeg after offload → PrimeMonitor Parquet serialization.
  • Remote test machine root@95.133.252.95: live platform E2E with image sample upload passed via real PrimeMonitor registration, sample presign, R2 PUT, confirm, and final summary; uploaded Parquet bytes were asserted to contain data:image/png;base64,... and no file:// reference. Run id: yg12nqyecvzfdcyaokiscgb2.
  • Remote test machine root@95.133.252.95: realistic live platform E2E passed with run_config.max_steps=20 for 20 steps × 5 image rollouts/step. API verification showed maxSteps=20, stepsWithSamples=[0..19], total=100 samples, and sampled prompts contain data:image/png;base64,... with no file://. Run id: j66iit3cb0vlyb93v12kxhx6.
  • Remote test machine root@95.133.252.95 at current head: real renderable PNG/JPEG live platform E2E passed with run_config.max_steps=10 for 10 steps × 4 image rollouts/step. API verification showed maxSteps=10, stepsWithSamples=[0..9], step 9 has 4 decoded valid 96×64 PNG/JPEG samples with matching data:image/png / data:image/jpeg media types and no file://. Run id: e9l2830u8xfydm0rh5oyp0gg.

Note

Low Risk
Changes are limited to image offload paths and PrimeMonitor sample serialization, with a 2 MiB inline cap and no training or auth logic touched.

Overview
Rollout image offloading now writes files with the correct extension from the source data:image/... media type (not always .png), dedupes by full on-disk path, and stores absolute file:// URIs via Path.resolve().as_uri() so relative output dirs do not produce broken references.

PrimeMonitor sample Parquet serialization recursively re-embeds eligible local file:// image URLs in prompt, completion, and trajectory JSON as data:image/...;base64,... (without mutating in-memory rollouts), with per-URL caching and a 2 MiB per-image inline cap so the hosted dashboard can render images while large artifacts stay as file pointers.

Reviewed by Cursor Bugbot for commit 27ec93d. Bugbot is set up for automated code reviews on this repo. Configure here.

@mrmoxon mrmoxon marked this pull request as ready for review June 1, 2026 15:12
Comment thread src/prime_rl/utils/monitor/prime.py
@d42me d42me requested review from mikasenghaas and samsja June 3, 2026 21:25
Copy link
Copy Markdown

@cursor cursor Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cursor Bugbot has reviewed your changes and found 1 potential issue.

Fix All in Cursor

❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.

Reviewed by Cursor Bugbot for commit 3ccf1a6. Configure here.

Comment thread src/prime_rl/utils/monitor/prime.py
Comment thread tests/unit/utils/test_prime_monitor.py Outdated
Comment thread tests/unit/orchestrator/test_trajectories.py Outdated
Comment thread tests/unit/utils/test_prime_monitor.py Outdated
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants