Skip to content

Commit 2e38e1b

Browse files
Violet2314Violet2314
committed
fix(safety): address CongkeChen review - aggregation, ContextVar, sleep boundary, audit UTC
Critical (finalized): - executor multi-block aggregation: reuse _scan_code_input/_should_block_report so NEEDS_HUMAN_REVIEW outweighs ALLOW (no silent bypass) - executor non-blocking review: append TOOL_SAFETY_NEEDS_REVIEW warning to stderr instead of running silently Warning: - _after: only attach safety_warning to dict results; never wrap non-dict returns (preserves caller return-type contract) - _before: defensively clear _REVIEW_REPORT ContextVar at entry to prevent leakage when a prior _after was skipped (handle error / is_continue=False) - _parse_risk_level: raise ValueError on invalid input (e.g. "superhigh") instead of silently falling back to default - BashTool enable_safety_guard: add integration tests covering DENY block, ALLOW pass-through, and NEEDS_HUMAN_REVIEW warning merge via real filter chain - ResourceAbuseRule: long-sleep boundary >= -> > to match "keep below N seconds" semantics (sleep(N) at threshold is now ALLOW) - _extract_host_from_bash: strip trailing '&' before host extraction so nc/telnet backgrounded lines resolve host correctly - _audit: use time.gmtime() + %SZ for UTC timestamps (container-comparable) Suggestion: - _types: expose decision_rank() so _wrapper and executor share one ranking (eliminates three drifting copies of the same map) - __init__: record ImportError in _IMPORT_ERRORS and expose import_error_for(name) so callers get actionable errors instead of TypeError: None is not callable - normalize example JSON files to LF line endings Tests: - test_filter_sleep_at_threshold_is_allowed: sleep(300) and 'sleep 300' must ALLOW when max_timeout_seconds=300 - test_filter_contextvar_does_not_leak_across_calls: stale report in ContextVar must not survive into the next _before/_after cycle - test_bash_tool_safety_guard_{blocks_deny,allows_safe,review_warning_merged}: real filter-chain coverage for BashTool opt-in guard
1 parent 35750b8 commit 2e38e1b

13 files changed

Lines changed: 274 additions & 1966 deletions

File tree

‎examples/tool_safety/tool_safety_audit.jsonl‎

Lines changed: 0 additions & 40 deletions
Original file line numberDiff line numberDiff line change
@@ -1,40 +0,0 @@
1-
{"timestamp": "1970-01-01T00:00:00+0000", "tool_name": "01_safe_python.py", "decision": "allow", "risk_level": "none", "rule_ids": [], "scan_duration_ms": 0.0, "sanitized": true, "intercepted": false, "blocked": false, "scanner_version": "1.2.0", "language": "python", "script_path": "examples/tool_safety/samples/01_safe_python.py", "findings_count": 0}
2-
{"timestamp": "1970-01-01T00:00:00+0000", "tool_name": "02_dangerous_delete.sh", "decision": "deny", "risk_level": "critical", "rule_ids": ["R001_dangerous_files"], "scan_duration_ms": 0.0, "sanitized": true, "intercepted": true, "blocked": true, "scanner_version": "1.2.0", "language": "bash", "script_path": "examples/tool_safety/samples/02_dangerous_delete.sh", "findings_count": 5}
3-
{"timestamp": "1970-01-01T00:00:00+0000", "tool_name": "03_read_credentials.py", "decision": "deny", "risk_level": "critical", "rule_ids": ["R001_dangerous_files"], "scan_duration_ms": 0.0, "sanitized": true, "intercepted": true, "blocked": true, "scanner_version": "1.2.0", "language": "python", "script_path": "examples/tool_safety/samples/03_read_credentials.py", "findings_count": 3}
4-
{"timestamp": "1970-01-01T00:00:00+0000", "tool_name": "04_network_exfil.py", "decision": "deny", "risk_level": "high", "rule_ids": ["R002_network_egress"], "scan_duration_ms": 0.0, "sanitized": true, "intercepted": true, "blocked": true, "scanner_version": "1.2.0", "language": "python", "script_path": "examples/tool_safety/samples/04_network_exfil.py", "findings_count": 3}
5-
{"timestamp": "1970-01-01T00:00:00+0000", "tool_name": "05_whitelist_network.py", "decision": "allow", "risk_level": "none", "rule_ids": [], "scan_duration_ms": 0.0, "sanitized": true, "intercepted": false, "blocked": false, "scanner_version": "1.2.0", "language": "python", "script_path": "examples/tool_safety/samples/05_whitelist_network.py", "findings_count": 0}
6-
{"timestamp": "1970-01-01T00:00:00+0000", "tool_name": "06_subprocess_call.py", "decision": "deny", "risk_level": "critical", "rule_ids": ["R003_process_system"], "scan_duration_ms": 0.0, "sanitized": true, "intercepted": true, "blocked": true, "scanner_version": "1.2.0", "language": "python", "script_path": "examples/tool_safety/samples/06_subprocess_call.py", "findings_count": 5}
7-
{"timestamp": "1970-01-01T00:00:00+0000", "tool_name": "07_shell_injection.sh", "decision": "deny", "risk_level": "critical", "rule_ids": ["R001_dangerous_files", "R002_network_egress", "R003_process_system", "R007_code_execution"], "scan_duration_ms": 0.0, "sanitized": true, "intercepted": true, "blocked": true, "scanner_version": "1.2.0", "language": "bash", "script_path": "examples/tool_safety/samples/07_shell_injection.sh", "findings_count": 8}
8-
{"timestamp": "1970-01-01T00:00:00+0000", "tool_name": "08_dependency_install.sh", "decision": "deny", "risk_level": "high", "rule_ids": ["R004_dependency_install"], "scan_duration_ms": 0.0, "sanitized": true, "intercepted": true, "blocked": true, "scanner_version": "1.2.0", "language": "bash", "script_path": "examples/tool_safety/samples/08_dependency_install.sh", "findings_count": 5}
9-
{"timestamp": "1970-01-01T00:00:00+0000", "tool_name": "09_infinite_loop.py", "decision": "deny", "risk_level": "high", "rule_ids": ["R005_resource_abuse"], "scan_duration_ms": 0.0, "sanitized": true, "intercepted": true, "blocked": true, "scanner_version": "1.2.0", "language": "python", "script_path": "examples/tool_safety/samples/09_infinite_loop.py", "findings_count": 4}
10-
{"timestamp": "1970-01-01T00:00:00+0000", "tool_name": "10_secret_leak.py", "decision": "deny", "risk_level": "critical", "rule_ids": ["R002_network_egress", "R006_secret_leak"], "scan_duration_ms": 0.0, "sanitized": true, "intercepted": true, "blocked": true, "scanner_version": "1.2.0", "language": "python", "script_path": "examples/tool_safety/samples/10_secret_leak.py", "findings_count": 7}
11-
{"timestamp": "1970-01-01T00:00:00+0000", "tool_name": "11_bash_pipeline.sh", "decision": "deny", "risk_level": "critical", "rule_ids": ["R001_dangerous_files", "R002_network_egress", "R003_process_system", "R005_resource_abuse"], "scan_duration_ms": 0.0, "sanitized": true, "intercepted": true, "blocked": true, "scanner_version": "1.2.0", "language": "bash", "script_path": "examples/tool_safety/samples/11_bash_pipeline.sh", "findings_count": 9}
12-
{"timestamp": "1970-01-01T00:00:00+0000", "tool_name": "12_human_review.py", "decision": "deny", "risk_level": "high", "rule_ids": ["R002_network_egress"], "scan_duration_ms": 0.0, "sanitized": true, "intercepted": true, "blocked": true, "scanner_version": "1.2.0", "language": "python", "script_path": "examples/tool_safety/samples/12_human_review.py", "findings_count": 1}
13-
{"timestamp": "1970-01-01T00:00:00+0000", "tool_name": "13_alias_os_system.py", "decision": "deny", "risk_level": "high", "rule_ids": ["R003_process_system"], "scan_duration_ms": 0.0, "sanitized": true, "intercepted": true, "blocked": true, "scanner_version": "1.2.0", "language": "python", "script_path": "examples/tool_safety/samples/13_alias_os_system.py", "findings_count": 1}
14-
{"timestamp": "1970-01-01T00:00:00+0000", "tool_name": "14_from_import_system.py", "decision": "deny", "risk_level": "high", "rule_ids": ["R003_process_system"], "scan_duration_ms": 0.0, "sanitized": true, "intercepted": true, "blocked": true, "scanner_version": "1.2.0", "language": "python", "script_path": "examples/tool_safety/samples/14_from_import_system.py", "findings_count": 1}
15-
{"timestamp": "1970-01-01T00:00:00+0000", "tool_name": "15_base64_pipe.sh", "decision": "deny", "risk_level": "critical", "rule_ids": ["R003_process_system", "R007_code_execution"], "scan_duration_ms": 0.0, "sanitized": true, "intercepted": true, "blocked": true, "scanner_version": "1.2.0", "language": "bash", "script_path": "examples/tool_safety/samples/15_base64_pipe.sh", "findings_count": 2}
16-
{"timestamp": "1970-01-01T00:00:00+0000", "tool_name": "16_pathlib_ssh.py", "decision": "deny", "risk_level": "critical", "rule_ids": ["R001_dangerous_files"], "scan_duration_ms": 0.0, "sanitized": true, "intercepted": true, "blocked": true, "scanner_version": "1.2.0", "language": "python", "script_path": "examples/tool_safety/samples/16_pathlib_ssh.py", "findings_count": 2}
17-
{"timestamp": "1970-01-01T00:00:00+0000", "tool_name": "17_requests_session.py", "decision": "deny", "risk_level": "high", "rule_ids": ["R002_network_egress"], "scan_duration_ms": 0.0, "sanitized": true, "intercepted": true, "blocked": true, "scanner_version": "1.2.0", "language": "python", "script_path": "examples/tool_safety/samples/17_requests_session.py", "findings_count": 1}
18-
{"timestamp": "1970-01-01T00:00:00+0000", "tool_name": "18_env_secret_print.py", "decision": "deny", "risk_level": "critical", "rule_ids": ["R006_secret_leak"], "scan_duration_ms": 0.0, "sanitized": true, "intercepted": true, "blocked": true, "scanner_version": "1.2.0", "language": "python", "script_path": "examples/tool_safety/samples/18_env_secret_print.py", "findings_count": 2}
19-
{"timestamp": "1970-01-01T00:00:00+0000", "tool_name": "19_python_c_inline.sh", "decision": "deny", "risk_level": "critical", "rule_ids": ["R001_dangerous_files", "R003_process_system"], "scan_duration_ms": 0.0, "sanitized": true, "intercepted": true, "blocked": true, "scanner_version": "1.2.0", "language": "bash", "script_path": "examples/tool_safety/samples/19_python_c_inline.sh", "findings_count": 2}
20-
{"timestamp": "1970-01-01T00:00:00+0000", "tool_name": "20_getattr_system.py", "decision": "deny", "risk_level": "critical", "rule_ids": ["R003_process_system"], "scan_duration_ms": 0.0, "sanitized": true, "intercepted": true, "blocked": true, "scanner_version": "1.2.0", "language": "python", "script_path": "examples/tool_safety/samples/20_getattr_system.py", "findings_count": 1}
21-
{"timestamp": "1970-01-01T00:00:00+0000", "tool_name": "21_find_delete.sh", "decision": "deny", "risk_level": "critical", "rule_ids": ["R001_dangerous_files", "R007_code_execution"], "scan_duration_ms": 0.0, "sanitized": true, "intercepted": true, "blocked": true, "scanner_version": "1.2.0", "language": "bash", "script_path": "examples/tool_safety/samples/21_find_delete.sh", "findings_count": 2}
22-
{"timestamp": "1970-01-01T00:00:00+0000", "tool_name": "22_xargs_rm.sh", "decision": "deny", "risk_level": "critical", "rule_ids": ["R001_dangerous_files", "R007_code_execution"], "scan_duration_ms": 0.0, "sanitized": true, "intercepted": true, "blocked": true, "scanner_version": "1.2.0", "language": "bash", "script_path": "examples/tool_safety/samples/22_xargs_rm.sh", "findings_count": 2}
23-
{"timestamp": "1970-01-01T00:00:00+0000", "tool_name": "23_curl_env_upload.sh", "decision": "deny", "risk_level": "critical", "rule_ids": ["R001_dangerous_files", "R002_network_egress", "R006_secret_leak"], "scan_duration_ms": 0.0, "sanitized": true, "intercepted": true, "blocked": true, "scanner_version": "1.2.0", "language": "bash", "script_path": "examples/tool_safety/samples/23_curl_env_upload.sh", "findings_count": 4}
24-
{"timestamp": "1970-01-01T00:00:00+0000", "tool_name": "24_dev_tcp.sh", "decision": "deny", "risk_level": "critical", "rule_ids": ["R001_dangerous_files", "R002_network_egress"], "scan_duration_ms": 0.0, "sanitized": true, "intercepted": true, "blocked": true, "scanner_version": "1.2.0", "language": "bash", "script_path": "examples/tool_safety/samples/24_dev_tcp.sh", "findings_count": 2}
25-
{"timestamp": "1970-01-01T00:00:00+0000", "tool_name": "25_private_key_literal.py", "decision": "deny", "risk_level": "critical", "rule_ids": ["R006_secret_leak"], "scan_duration_ms": 0.0, "sanitized": true, "intercepted": true, "blocked": true, "scanner_version": "1.2.0", "language": "python", "script_path": "examples/tool_safety/samples/25_private_key_literal.py", "findings_count": 2}
26-
{"timestamp": "1970-01-01T00:00:00+0000", "tool_name": "26_os_getenv_exfil.py", "decision": "deny", "risk_level": "critical", "rule_ids": ["R002_network_egress", "R006_secret_leak"], "scan_duration_ms": 0.0, "sanitized": true, "intercepted": true, "blocked": true, "scanner_version": "1.2.0", "language": "python", "script_path": "examples/tool_safety/samples/26_os_getenv_exfil.py", "findings_count": 3}
27-
{"timestamp": "1970-01-01T00:00:00+0000", "tool_name": "27_socket_access.py", "decision": "deny", "risk_level": "high", "rule_ids": ["R002_network_egress"], "scan_duration_ms": 0.0, "sanitized": true, "intercepted": true, "blocked": true, "scanner_version": "1.2.0", "language": "python", "script_path": "examples/tool_safety/samples/27_socket_access.py", "findings_count": 1}
28-
{"timestamp": "1970-01-01T00:00:00+0000", "tool_name": "28_privilege_escalation.sh", "decision": "deny", "risk_level": "critical", "rule_ids": ["R003_process_system", "R004_dependency_install"], "scan_duration_ms": 0.0, "sanitized": true, "intercepted": true, "blocked": true, "scanner_version": "1.2.0", "language": "bash", "script_path": "examples/tool_safety/samples/28_privilege_escalation.sh", "findings_count": 2}
29-
{"timestamp": "1970-01-01T00:00:00+0000", "tool_name": "29_dynamic_secret_path.py", "decision": "deny", "risk_level": "critical", "rule_ids": ["R001_dangerous_files"], "scan_duration_ms": 0.0, "sanitized": true, "intercepted": true, "blocked": true, "scanner_version": "1.2.0", "language": "python", "script_path": "examples/tool_safety/samples/29_dynamic_secret_path.py", "findings_count": 3}
30-
{"timestamp": "1970-01-01T00:00:00+0000", "tool_name": "30_safe_bash.sh", "decision": "allow", "risk_level": "none", "rule_ids": [], "scan_duration_ms": 0.0, "sanitized": true, "intercepted": false, "blocked": false, "scanner_version": "1.2.0", "language": "bash", "script_path": "examples/tool_safety/samples/30_safe_bash.sh", "findings_count": 0}
31-
{"timestamp": "1970-01-01T00:00:00+0000", "tool_name": "31_safe_find_grep.sh", "decision": "allow", "risk_level": "none", "rule_ids": [], "scan_duration_ms": 0.0, "sanitized": true, "intercepted": false, "blocked": false, "scanner_version": "1.2.0", "language": "bash", "script_path": "examples/tool_safety/samples/31_safe_find_grep.sh", "findings_count": 0}
32-
{"timestamp": "1970-01-01T00:00:00+0000", "tool_name": "32_fork_bomb.sh", "decision": "deny", "risk_level": "critical", "rule_ids": ["R005_resource_abuse"], "scan_duration_ms": 0.0, "sanitized": true, "intercepted": true, "blocked": true, "scanner_version": "1.2.0", "language": "bash", "script_path": "examples/tool_safety/samples/32_fork_bomb.sh", "findings_count": 1}
33-
{"timestamp": "1970-01-01T00:00:00+0000", "tool_name": "33_safe_git_status.sh", "decision": "allow", "risk_level": "none", "rule_ids": [], "scan_duration_ms": 0.0, "sanitized": true, "intercepted": false, "blocked": false, "scanner_version": "1.2.0", "language": "bash", "script_path": "examples/tool_safety/samples/33_safe_git_status.sh", "findings_count": 0}
34-
{"timestamp": "1970-01-01T00:00:00+0000", "tool_name": "34_safe_python_math.py", "decision": "allow", "risk_level": "none", "rule_ids": [], "scan_duration_ms": 0.0, "sanitized": true, "intercepted": false, "blocked": false, "scanner_version": "1.2.0", "language": "python", "script_path": "examples/tool_safety/samples/34_safe_python_math.py", "findings_count": 0}
35-
{"timestamp": "1970-01-01T00:00:00+0000", "tool_name": "35_long_sleep.sh", "decision": "deny", "risk_level": "high", "rule_ids": ["R005_resource_abuse"], "scan_duration_ms": 0.0, "sanitized": true, "intercepted": true, "blocked": true, "scanner_version": "1.2.0", "language": "bash", "script_path": "examples/tool_safety/samples/35_long_sleep.sh", "findings_count": 1}
36-
{"timestamp": "1970-01-01T00:00:00+0000", "tool_name": "36_httpx_client_evil.py", "decision": "deny", "risk_level": "high", "rule_ids": ["R002_network_egress"], "scan_duration_ms": 0.0, "sanitized": true, "intercepted": true, "blocked": true, "scanner_version": "1.2.0", "language": "python", "script_path": "examples/tool_safety/samples/36_httpx_client_evil.py", "findings_count": 1}
37-
{"timestamp": "1970-01-01T00:00:00+0000", "tool_name": "37_eval_exec.py", "decision": "deny", "risk_level": "critical", "rule_ids": ["R003_process_system", "R007_code_execution"], "scan_duration_ms": 0.0, "sanitized": true, "intercepted": true, "blocked": true, "scanner_version": "1.2.0", "language": "python", "script_path": "examples/tool_safety/samples/37_eval_exec.py", "findings_count": 5}
38-
{"timestamp": "1970-01-01T00:00:00+0000", "tool_name": "38_npm_install.sh", "decision": "deny", "risk_level": "high", "rule_ids": ["R004_dependency_install"], "scan_duration_ms": 0.0, "sanitized": true, "intercepted": true, "blocked": true, "scanner_version": "1.2.0", "language": "bash", "script_path": "examples/tool_safety/samples/38_npm_install.sh", "findings_count": 2}
39-
{"timestamp": "1970-01-01T00:00:00+0000", "tool_name": "39_curl_dynamic_url.sh", "decision": "deny", "risk_level": "high", "rule_ids": ["R002_network_egress"], "scan_duration_ms": 0.0, "sanitized": true, "intercepted": true, "blocked": true, "scanner_version": "1.2.0", "language": "bash", "script_path": "examples/tool_safety/samples/39_curl_dynamic_url.sh", "findings_count": 2}
40-
{"timestamp": "1970-01-01T00:00:00+0000", "tool_name": "40_background_nc.sh", "decision": "deny", "risk_level": "high", "rule_ids": ["R002_network_egress", "R003_process_system"], "scan_duration_ms": 0.0, "sanitized": true, "intercepted": true, "blocked": true, "scanner_version": "1.2.0", "language": "bash", "script_path": "examples/tool_safety/samples/40_background_nc.sh", "findings_count": 2}

0 commit comments

Comments
 (0)