Skip to content

fix: fail closed on unresolved DVC outputs#1488

Open
mldangelo-oai wants to merge 2 commits into
mainfrom
mdangelo/codex/fix-dvc-missing-outputs-c123
Open

fix: fail closed on unresolved DVC outputs#1488
mldangelo-oai wants to merge 2 commits into
mainfrom
mdangelo/codex/fix-dvc-missing-outputs-c123

Conversation

@mldangelo-oai
Copy link
Copy Markdown
Contributor

Summary

  • preserve structured DVC resolution status while keeping the legacy resolve_dvc_file list API
  • keep partial/unresolved .dvc pointers routed through core so missing outputs become operationally incomplete coverage
  • prevent unresolved-only .dvc pointers from being scanned as harmless pointer text

Validation

  • PYTHONPATH=/private/tmp/modelaudit-c123 PROMPTFOO_DISABLE_TELEMETRY=1 /Users/mdangelo/code/modelaudit/.venv/bin/pytest tests/utils/sources/test_dvc_integration.py tests/test_directory_file_filtering.py::TestDirectoryFileFiltering::test_hidden_dvc_pointer_expands_hidden_artifact -q: 23 passed
  • ruff check for touched files
  • ruff format --check for touched files
  • mypy for touched files
  • git diff --check

@mldangelo-oai
Copy link
Copy Markdown
Contributor Author

@codex review

@github-actions
Copy link
Copy Markdown
Contributor

github-actions Bot commented May 31, 2026

Workflow run and artifacts

Performance Benchmarks

Compared 12 shared benchmarks with a regression threshold of 15%.
Status: 0 regressions, 0 improved, 12 stable, 0 new, 0 missing.
Aggregate shared-benchmark median: 777.56ms -> 792.27ms (+1.9%).

Workload Benchmark Target Size Files Baseline Current Change Status
clean-training-checkpoint tests/benchmarks/test_picklescan_benchmarks.py::test_picklescan_clean_training_checkpoint safe_large 278.2 KiB 1 20.06ms 21.10ms +5.2% stable
suspicious-pickle-intake tests/benchmarks/test_scan_benchmarks.py::test_scan_suspicious_pickle_intake suspicious-intake 183.8 KiB 4 105.66ms 110.21ms +4.3% stable
chunked-upload-stream tests/benchmarks/test_picklescan_benchmarks.py::test_picklescan_chunked_upload_stream chunked_stream 278.2 KiB 1 23.20ms 23.79ms +2.5% stable
warm-cache-rescan tests/benchmarks/test_scan_benchmarks.py::test_scan_warm_cached_repository_rescan release-candidate 547.3 KiB 32 73.03ms 74.86ms +2.5% stable
direct-malicious-upload tests/benchmarks/test_picklescan_benchmarks.py::test_picklescan_direct_malicious_upload malicious_reduce 52 B 1 1.60ms 1.57ms -2.0% stable
mixed-model-repository tests/benchmarks/test_scan_benchmarks.py::test_scan_release_candidate_repository release-candidate 547.3 KiB 32 294.05ms 298.94ms +1.7% stable
nested-payload-review tests/benchmarks/test_picklescan_benchmarks.py::test_picklescan_nested_payload_review[nested_hex] nested_hex 130 B 1 486.1us 478.8us -1.5% stable
nested-payload-review tests/benchmarks/test_picklescan_benchmarks.py::test_picklescan_nested_payload_review[nested_raw] nested_raw 78 B 1 470.6us 464.4us -1.3% stable
padded-multi-stream-upload tests/benchmarks/test_picklescan_benchmarks.py::test_picklescan_padded_multi_stream_upload multi_stream_padded 4.1 KiB 1 1.67ms 1.65ms -1.3% stable
nested-payload-review tests/benchmarks/test_picklescan_benchmarks.py::test_picklescan_nested_payload_review[nested_base64] nested_base64 98 B 1 481.5us 475.6us -1.2% stable
single-checkpoint-preflight tests/benchmarks/test_scan_benchmarks.py::test_scan_single_checkpoint_before_load single_checkpoint.pkl 183.0 KiB 1 40.17ms 40.62ms +1.1% stable
duplicate-heavy-registry tests/benchmarks/test_scan_benchmarks.py::test_scan_duplicate_registry_snapshot registry-snapshot 915.2 KiB 13 216.68ms 218.11ms +0.7% stable

Copy link
Copy Markdown

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 083e361272

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread modelaudit/utils/sources/dvc.py
Comment thread modelaudit/cli.py
Comment thread tests/utils/sources/test_dvc_integration.py
@mldangelo-oai mldangelo-oai marked this pull request as ready for review May 31, 2026 13:36
@ianw-oai
Copy link
Copy Markdown
Contributor

ianw-oai commented Jun 1, 2026

Potential regression: the new recursive DVC directory path seems to pass the full timeout and size budget into each nested scan instead of sharing the parent budget.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants