Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 2 additions & 2 deletions .release-please-manifest.json
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
{
".": "0.2.45",
"packages/modelaudit-picklescan": "0.1.5"
".": "0.2.46",
"packages/modelaudit-picklescan": "0.1.6"
}
187 changes: 187 additions & 0 deletions CHANGELOG.md

Large diffs are not rendered by default.

27 changes: 27 additions & 0 deletions packages/modelaudit-picklescan/CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,33 @@ All notable changes to `modelaudit-picklescan` will be documented in this file.
The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/),
and this package adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).

## [0.1.6](https://github.com/promptfoo/modelaudit/compare/modelaudit-picklescan-v0.1.5...modelaudit-picklescan-v0.1.6) (2026-06-05)
Comment thread
mldangelo-oai marked this conversation as resolved.

### Bug Fixes

- avoid pickle meta-path source probing ([#1493](https://github.com/promptfoo/modelaudit/issues/1493)) ([a31df76](https://github.com/promptfoo/modelaudit/commit/a31df7630614cf35d47031c92e4d735eb049c33e))
- bound native picklescan state simulation ([#1501](https://github.com/promptfoo/modelaudit/issues/1501)) ([f4c9cdf](https://github.com/promptfoo/modelaudit/commit/f4c9cdf0f13141e31f285d6d9fd249e6af90dd4b))
- detect dynamic picklescan protocol hooks ([#1375](https://github.com/promptfoo/modelaudit/issues/1375)) ([400c132](https://github.com/promptfoo/modelaudit/commit/400c132628dd4cd31e243e4e9c46cdb5af1db46a))
- detect newline-separated picklescan calls ([#1481](https://github.com/promptfoo/modelaudit/issues/1481)) ([8dcbbb1](https://github.com/promptfoo/modelaudit/commit/8dcbbb1776e68d1317b0f8c94807ebb20bac24cc))
- fail closed on encoded nested probe cap ([6633dac](https://github.com/promptfoo/modelaudit/commit/6633dac9d284b4f3bcd994349cb2d16306e01842))
- fail closed on pickle import reference truncation ([#1449](https://github.com/promptfoo/modelaudit/issues/1449)) ([5ddac28](https://github.com/promptfoo/modelaudit/commit/5ddac28195813e8f5cb425a158b7c1f5d03caa79))
- fail closed on protocol 5 pickle buffers ([#1450](https://github.com/promptfoo/modelaudit/issues/1450)) ([e696a1f](https://github.com/promptfoo/modelaudit/commit/e696a1ff9452c9b9a7156325a1ff791f8ecd8ac6))
- flag import-only custom pickle globals ([#1499](https://github.com/promptfoo/modelaudit/issues/1499)) ([ca3a476](https://github.com/promptfoo/modelaudit/commit/ca3a4768b2dd691fb03f98da24d17a655a378162))
- flag oversized pickle frames as tampered ([#1448](https://github.com/promptfoo/modelaudit/issues/1448)) ([c4758fd](https://github.com/promptfoo/modelaudit/commit/c4758fdc66dd831e346a5445c60a02c55fe6186a))
- redact Keras evidence secrets ([#1475](https://github.com/promptfoo/modelaudit/issues/1475)) ([37eda4e](https://github.com/promptfoo/modelaudit/commit/37eda4e69404458d14aeba95f64a03de97761891))
- resolve follow-up quality findings ([#1222](https://github.com/promptfoo/modelaudit/issues/1222)) ([2968961](https://github.com/promptfoo/modelaudit/commit/2968961a40adf5c9e9333d1a2c601cc9aca7fa4e))
- **routing:** preserve Torch7 findings in Llamafile polyglots ([#1376](https://github.com/promptfoo/modelaudit/issues/1376)) ([2e95c88](https://github.com/promptfoo/modelaudit/commit/2e95c88eb37043a72551a636e30e9df54e72f486))
- scan raw nested pickles in unicode strings ([#1461](https://github.com/promptfoo/modelaudit/issues/1461)) ([4278da9](https://github.com/promptfoo/modelaudit/commit/4278da93863ae5972de9e79eddc06bd5692974b5))
- terminate call-graph alias fixpoint on oscillating rebinds ([#1247](https://github.com/promptfoo/modelaudit/issues/1247)) ([#1259](https://github.com/promptfoo/modelaudit/issues/1259)) ([89895a4](https://github.com/promptfoo/modelaudit/commit/89895a4c646feabb98888fece9cf12ef283d351e))

### Performance Improvements

- reuse call graph analysis in directory scans ([#1266](https://github.com/promptfoo/modelaudit/issues/1266)) ([2f01ddf](https://github.com/promptfoo/modelaudit/commit/2f01ddfc88d1b625687867b6745237fe25aa3bb3))

### Documentation

- align picklescan version guidance ([#1279](https://github.com/promptfoo/modelaudit/issues/1279)) ([a53eb11](https://github.com/promptfoo/modelaudit/commit/a53eb112fcbdf4d6baca1ae0124aba8129cb95e1))

## [0.1.5](https://github.com/promptfoo/modelaudit/compare/modelaudit-picklescan-v0.1.4...modelaudit-picklescan-v0.1.5) (2026-05-03)

### Bug Fixes
Expand Down
2 changes: 1 addition & 1 deletion packages/modelaudit-picklescan/Cargo.lock

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

2 changes: 1 addition & 1 deletion packages/modelaudit-picklescan/Cargo.toml
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
[package]
name = "modelaudit-picklescan-rust"
version = "0.1.5" # x-release-please-version
version = "0.1.6" # x-release-please-version
Comment thread
mldangelo-oai marked this conversation as resolved.
Comment thread
mldangelo-oai marked this conversation as resolved.
edition = "2021"
rust-version = "1.83"
description = "Native pickle security scanner engine for modelaudit-picklescan"
Expand Down
2 changes: 1 addition & 1 deletion packages/modelaudit-picklescan/pyproject.toml
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@ build-backend = "maturin"

[project]
name = "modelaudit-picklescan"
version = "0.1.5" # x-release-please-version
version = "0.1.6" # x-release-please-version
Comment thread
mldangelo-oai marked this conversation as resolved.
description = "Standalone pickle security scanner extracted from ModelAudit"
authors = [
{ name = "Ian Webster", email = "ian@promptfoo.dev" },
Expand Down
4 changes: 2 additions & 2 deletions packages/modelaudit-picklescan/tests/test_api.py
Original file line number Diff line number Diff line change
Expand Up @@ -5263,8 +5263,8 @@ def test_scan_bytes_warns_when_dunder_builtins_resolves_to_shadow_module(

def test_scan_bytes_warns_when_allowlisted_module_is_unresolved(monkeypatch: pytest.MonkeyPatch) -> None:
monkeypatch.setattr(
"modelaudit_picklescan.call_graph._find_module_spec_without_imports",
lambda _module_name: None,
"modelaudit_picklescan.call_graph._trusted_module_origin_kind",
lambda _module_name: "unresolved",
)
_clear_source_sensitive_caches()
try:
Expand Down
2 changes: 1 addition & 1 deletion packages/modelaudit-picklescan/uv.lock

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

2 changes: 1 addition & 1 deletion pyproject.toml
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@ build-backend = "hatchling.build"

[project]
name = "modelaudit"
version = "0.2.45"
version = "0.2.46"
Comment thread
mldangelo-oai marked this conversation as resolved.
Comment thread
mldangelo-oai marked this conversation as resolved.
description = "Static scanning library for detecting malicious code, potential backdoor indicators, and other security risks in ML model files"
authors = [
{ name = "Ian Webster", email = "ian@promptfoo.dev" },
Expand Down
2 changes: 1 addition & 1 deletion tests/scanners/test_jinja2_template_scanner.py
Original file line number Diff line number Diff line change
Expand Up @@ -991,7 +991,7 @@ def test_small_json_chat_template_still_analyzed(self, tmp_path: Path) -> None:
tokenizer_file = tmp_path / "tokenizer_config.json"
tokenizer_file.write_text(json.dumps({"chat_template": payload}), encoding="utf-8")

scanner = Jinja2TemplateScanner({"max_template_size": 128})
scanner = Jinja2TemplateScanner({"max_template_size": 128, "sandbox_render_timeout_seconds": 2})
result = scanner.scan(str(tokenizer_file))

assert result.has_warnings or result.has_errors
Expand Down
44 changes: 32 additions & 12 deletions tests/test_real_world_dill_joblib.py
Original file line number Diff line number Diff line change
Expand Up @@ -3,6 +3,7 @@
import os
import sys
import time
from pathlib import Path
from unittest.mock import patch

import pytest
Expand Down Expand Up @@ -253,7 +254,12 @@ def test_large_file_scanning_performance(self, tmp_path):
# Should complete within reasonable time
# CI environments may have variable performance; use generous thresholds
# to avoid flaky failures from runner contention (Linux ~2s, Windows ~4s typical)
has_runner_contention = bool(os.getenv("CI") or os.getenv("GITHUB_ACTIONS") or os.getenv("PYTEST_XDIST_WORKER"))
has_runner_contention = bool(

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P3 Badge Add typing to the modified benchmark test

This edit makes test_large_file_scanning_performance a modified pytest test, but the test still lacks tmp_path: Path and -> None; the repo test-style rule says to apply typing to all new or modified tests and explicitly calls out tmp_path: Path (AGENTS.md lines 120-121). Please update the function signature while touching this test so it stays consistent with the documented test requirements.

Useful? React with 👍 / 👎.

os.getenv("CI")
or os.getenv("GITHUB_ACTIONS")
or os.getenv("PYTEST_XDIST_WORKER")
or os.getenv("PYTEST_XDIST_WORKER_COUNT")
)
threshold = (
(15.0 if sys.platform == "win32" else 12.0)
if has_runner_contention
Expand Down Expand Up @@ -299,7 +305,7 @@ def test_multiple_files_scanning_performance(self, tmp_path):
avg_time = total_duration / len(files)
print(f"Average scan time: {avg_time:.3f}s per file")

def test_validation_performance_impact(self, tmp_path):
def test_validation_performance_impact(self, tmp_path: Path) -> None:
"""Test performance impact of file validation."""
test_file = tmp_path / "validation_test.joblib"

Expand All @@ -310,17 +316,31 @@ def test_validation_performance_impact(self, tmp_path):

from modelaudit.scanners.pickle_scanner import _is_legitimate_serialization_file

iters = 30 if (os.getenv("CI") or os.getenv("GITHUB_ACTIONS")) else 100
start_time = time.perf_counter()
for _ in range(iters): # Multiple calls to get meaningful measurement
_is_legitimate_serialization_file(str(test_file))
validation_duration = time.perf_counter() - start_time

# Validation should be very fast
avg_validation_time = validation_duration / iters
assert avg_validation_time < 0.003 # Under 3ms average on bounded opcode scan
has_runner_contention = bool(
os.getenv("CI")
or os.getenv("GITHUB_ACTIONS")
or os.getenv("PYTEST_XDIST_WORKER")
or os.getenv("PYTEST_XDIST_WORKER_COUNT")
)
iters = 30 if has_runner_contention else 60
samples: list[float] = []
_is_legitimate_serialization_file(str(test_file))
for _ in range(5):
start_time = time.process_time()
for _ in range(iters): # Multiple calls to get meaningful measurement
_is_legitimate_serialization_file(str(test_file))
samples.append((time.process_time() - start_time) / iters)

# Validation should use little CPU even when wall-clock scheduling is noisy.
avg_validation_time = min(samples)
threshold = 0.010 if sys.platform == "win32" else 0.006
if has_runner_contention:
threshold *= 4.0
assert avg_validation_time < threshold, (
f"Fastest validation batch averaged {avg_validation_time:.6f}s, expected < {threshold:.3f}s"
)

print(f"Average validation time: {avg_validation_time:.6f}s")
print(f"Average validation CPU time: {avg_validation_time:.6f}s")


class TestErrorScenarios:
Expand Down
4 changes: 2 additions & 2 deletions uv.lock

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

Loading