Skip to content

Commit cfc973e

Browse files
committed
feat: audit logging, OTel attributes, resource rules; close issue #90 gaps
Closes the gaps flagged in the issue #90 compliance audit: - _audit.py: JSON Lines audit record (tool_name, decision, risk_level, rule_ids, scan_duration_ms, sanitized, intercepted, timestamp, recommendation) + write_audit(); emit_otel() sets tool.safety.* span attributes when OTel is active. - ToolSafetyFilter + SafetyGuardedCodeExecutor now call record_safety_decision on every scan (allowed summarized, blocked reasoned + intercepted flag). - New resource rules tool-res-large-write (huge write / dd / truncate / head -c) and tool-res-concurrent-flood (large ThreadPool/ProcessPool); python + bash detection; 2 new manifest samples (14 total). - Example deliverables: examples/tool_safety_report.json (real CLI output), examples/tool_safety_audit.jsonl (deny/review/allow records). - zh/en docs updated (20 rules, telemetry section now describes implemented audit + OTel instead of 'reserved').
1 parent e50e984 commit cfc973e

15 files changed

Lines changed: 449 additions & 13 deletions

docs/mkdocs/en/tool_safety.md

Lines changed: 5 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -65,7 +65,7 @@ result = await guarded_executor.execute_code(
6565

6666
### rule_id Domain Mapping
6767

68-
The system covers **7 major risk domains** with 18 rules (aligning with `trpc-agent-go/tool/safety` style):
68+
The system covers **7 major risk domains** with 20 rules (aligning with `trpc-agent-go/tool/safety` style):
6969

7070
| Domain Prefix | Risk Coverage | Example rule_ids |
7171
|---------------|---------------|------------------|
@@ -74,7 +74,7 @@ The system covers **7 major risk domains** with 18 rules (aligning with `trpc-ag
7474
| `tool-net-*` | Network egress | `tool-net-http`, `tool-net-socket` |
7575
| `tool-proc-*` | Process/system commands | `tool-proc-subprocess`, `tool-proc-shell-pipe`, `tool-proc-privilege-escalation` |
7676
| `tool-pkg-*` | Dependency installation | `tool-pkg-install` (pip/npm/apt) |
77-
| `tool-res-*` | Resource abuse | `tool-res-infinite-loop`, `tool-res-fork-bomb`, `tool-res-long-sleep` |
77+
| `tool-res-*` | Resource abuse | `tool-res-infinite-loop`, `tool-res-fork-bomb`, `tool-res-long-sleep`, `tool-res-large-write`, `tool-res-concurrent-flood` |
7878
| `tool-secret-*` | Sensitive information leakage | `tool-secret-logging`, `tool-secret-private-key` |
7979

8080
### Decision and RiskLevel
@@ -195,8 +195,9 @@ rule_overrides:
195195

196196
#### Relationship with Telemetry
197197

198-
- **Audit events**: Log structured events when intercepted (containing `decision`/`risk_level`/`rule_ids`/`evidence`/`recommendation`)
199-
- **OpenTelemetry**: Interface reserved (not implemented in MVP; can add span attributes in the future)
198+
- **Audit log**: Every scan appends one JSON Lines audit record to `tool_safety_audit.jsonl` (path configurable via env `TRPC_AGENT_TOOL_SAFETY_AUDIT`), containing all issue #90 mandatory fields: `tool_name`, `decision`, `risk_level`, `rule_ids`, `scan_duration_ms`, `sanitized`, `intercepted`, `timestamp`, `recommendation`. Allowed scripts are summarized; denied scripts record the block reason. See `trpc_agent_sdk/tools/safety/examples/tool_safety_audit.jsonl`.
199+
- **OpenTelemetry**: When the host has OTel enabled and a scan runs inside a span, it automatically sets span attributes `tool.safety.decision` / `tool.safety.risk_level` / `tool.safety.rule_id` / `tool.safety.scan_duration_ms` / `tool.safety.sanitized` / `tool.safety.blocked` / `tool.safety.tool_name` (no-op when OTel is absent or no span is active).
200+
- **Structured report**: See `trpc_agent_sdk/tools/safety/examples/tool_safety_report.json`; the CLI `scripts/tool_safety_check.py` prints exactly this JSON shape.
200201

201202
### Known Limitations
202203

docs/mkdocs/zh/tool_safety.md

Lines changed: 5 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -65,7 +65,7 @@ result = await guarded_executor.execute_code(
6565

6666
### rule_id 规则域映射
6767

68-
系统覆盖 **7 大风险域**,共 18 条规则(对齐 `trpc-agent-go/tool/safety` 风格):
68+
系统覆盖 **7 大风险域**,共 20 条规则(对齐 `trpc-agent-go/tool/safety` 风格):
6969

7070
| 域前缀 | 覆盖风险 | 示例 rule_id |
7171
|--------|----------|-------------|
@@ -74,7 +74,7 @@ result = await guarded_executor.execute_code(
7474
| `tool-net-*` | 网络外连 | `tool-net-http``tool-net-socket` |
7575
| `tool-proc-*` | 进程/系统命令 | `tool-proc-subprocess``tool-proc-shell-pipe``tool-proc-privilege-escalation` |
7676
| `tool-pkg-*` | 依赖安装 | `tool-pkg-install`(pip/npm/apt) |
77-
| `tool-res-*` | 资源滥用 | `tool-res-infinite-loop``tool-res-fork-bomb``tool-res-long-sleep` |
77+
| `tool-res-*` | 资源滥用 | `tool-res-infinite-loop``tool-res-fork-bomb``tool-res-long-sleep``tool-res-large-write``tool-res-concurrent-flood` |
7878
| `tool-secret-*` | 敏感信息泄漏 | `tool-secret-logging``tool-secret-private-key` |
7979

8080
### 决策与风险级别
@@ -195,8 +195,9 @@ rule_overrides:
195195

196196
#### 与 Telemetry 的关系
197197

198-
- **审计事件**:拦截时记录结构化日志(含 `decision`/`risk_level`/`rule_ids`/`evidence`/`recommendation`
199-
- **OpenTelemetry**:预留接口(MVP 不实现,未来可添加 span 属性)
198+
- **审计日志**:每次扫描写一条 JSON Lines 审计记录到 `tool_safety_audit.jsonl`(路径可用环境变量 `TRPC_AGENT_TOOL_SAFETY_AUDIT` 配置),含 issue #90 要求的全部字段:`tool_name``decision``risk_level``rule_ids``scan_duration_ms``sanitized``intercepted``timestamp``recommendation`。允许执行的脚本也记录摘要,拒绝的脚本记录拦截原因。示例见 `trpc_agent_sdk/tools/safety/examples/tool_safety_audit.jsonl`
199+
- **OpenTelemetry**:若宿主启用了 OTel 且扫描处于某个 span 内,自动注入 span 属性 `tool.safety.decision` / `tool.safety.risk_level` / `tool.safety.rule_id` / `tool.safety.scan_duration_ms` / `tool.safety.sanitized` / `tool.safety.blocked` / `tool.safety.tool_name`(未启用 OTel 或无活动 span 时为 no-op)。
200+
- **结构化报告**:扫描报告示例见 `trpc_agent_sdk/tools/safety/examples/tool_safety_report.json``scripts/tool_safety_check.py` 可直接输出该 JSON 格式。
200201

201202
### 已知限制
202203

tests/tools/safety/conftest.py

Lines changed: 20 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,20 @@
1+
# Copyright (C) 2026 Tencent. All rights reserved.
2+
# tRPC-Agent-Python is licensed under Apache-2.0.
3+
"""Shared fixtures for tool-safety tests.
4+
5+
Redirects the safety audit jsonl log to a per-test temp path so integration
6+
tests (filter / executor guard) that invoke record_safety_decision() do not
7+
write audit files into the repository working tree.
8+
"""
9+
from __future__ import annotations
10+
11+
import pytest
12+
13+
14+
@pytest.fixture(autouse=True)
15+
def _redirect_safety_audit(tmp_path, monkeypatch):
16+
"""Point TRPC_AGENT_TOOL_SAFETY_AUDIT at a temp file for every test."""
17+
monkeypatch.setenv(
18+
"TRPC_AGENT_TOOL_SAFETY_AUDIT", str(tmp_path / "test_audit.jsonl")
19+
)
20+
yield tmp_path / "test_audit.jsonl"

tests/tools/safety/samples/manifest.yaml

Lines changed: 17 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -93,4 +93,20 @@ samples:
9393
s = socket.socket()
9494
s.connect(('internal.corp', 8080))
9595
expected_decision: NEEDS_REVIEW
96-
required_rule_ids: [tool-net-socket]
96+
required_rule_ids: [tool-net-socket]
97+
98+
- name: 13_large_write
99+
language: python
100+
script: |
101+
with open("big.bin", "wb") as f:
102+
f.write(b"\x00" * 100000000)
103+
expected_decision: NEEDS_REVIEW
104+
required_rule_ids: [tool-res-large-write]
105+
106+
- name: 14_concurrent_flood
107+
language: python
108+
script: |
109+
from concurrent.futures import ThreadPoolExecutor
110+
ThreadPoolExecutor(max_workers=10000)
111+
expected_decision: NEEDS_REVIEW
112+
required_rule_ids: [tool-res-concurrent-flood]

tests/tools/safety/test_audit.py

Lines changed: 93 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,93 @@
1+
# Copyright (C) 2026 Tencent. All rights reserved.
2+
# tRPC-Agent-Python is licensed under Apache-2.0.
3+
from __future__ import annotations
4+
5+
import json
6+
7+
from trpc_agent_sdk.tools.safety._audit import AuditRecord
8+
from trpc_agent_sdk.tools.safety._audit import build_audit_record
9+
from trpc_agent_sdk.tools.safety._audit import emit_otel
10+
from trpc_agent_sdk.tools.safety._audit import record_safety_decision
11+
from trpc_agent_sdk.tools.safety._audit import write_audit
12+
from trpc_agent_sdk.tools.safety._types import Decision
13+
from trpc_agent_sdk.tools.safety._types import Finding
14+
from trpc_agent_sdk.tools.safety._types import RiskLevel
15+
from trpc_agent_sdk.tools.safety._types import SafetyReport
16+
17+
18+
def _report(decision=Decision.DENY, risk=RiskLevel.HIGH, findings=None, duration=7):
19+
return SafetyReport(
20+
decision=decision,
21+
risk_level=risk,
22+
findings=findings or [
23+
Finding(
24+
rule_id="tool-fs-recursive-delete",
25+
risk_level=RiskLevel.HIGH,
26+
rule_decision=Decision.DENY,
27+
evidence="rm -rf /",
28+
recommendation="refuse",
29+
)
30+
],
31+
recommendation="blocked",
32+
scan_duration_ms=duration,
33+
sanitized=False,
34+
)
35+
36+
37+
def test_build_audit_record_has_all_issue_required_fields():
38+
rec = build_audit_record(
39+
_report(), tool_name="weather_tool", language="python", intercepted=True
40+
)
41+
# Issue #90 mandatory audit fields:
42+
assert rec.tool_name == "weather_tool" # tool name
43+
assert rec.decision == "DENY" # decision
44+
assert rec.risk_level == "HIGH" # risk level
45+
assert rec.rule_ids == ["tool-fs-recursive-delete"] # rule id
46+
assert rec.scan_duration_ms == 7 # duration
47+
assert rec.sanitized is False # sanitized flag
48+
assert rec.intercepted is True # intercepted flag
49+
assert rec.timestamp # non-empty ISO timestamp
50+
51+
52+
def test_write_audit_appends_jsonl_lines(tmp_path):
53+
path = tmp_path / "audit.jsonl"
54+
rec = AuditRecord(
55+
timestamp="2026-07-11T00:00:00+00:00",
56+
tool_name="t",
57+
language="python",
58+
decision="DENY",
59+
risk_level="HIGH",
60+
rule_ids=["tool-fs-recursive-delete"],
61+
scan_duration_ms=1,
62+
sanitized=False,
63+
intercepted=True,
64+
)
65+
write_audit(rec, str(path))
66+
write_audit(rec, str(path))
67+
lines = path.read_text(encoding="utf-8").strip().split("\n")
68+
assert len(lines) == 2
69+
obj = json.loads(lines[0])
70+
assert obj["tool_name"] == "t"
71+
assert obj["decision"] == "DENY"
72+
assert obj["intercepted"] is True
73+
74+
75+
def test_record_safety_decision_writes_and_returns(tmp_path):
76+
path = tmp_path / "a.jsonl"
77+
rec = record_safety_decision(
78+
_report(decision=Decision.ALLOW, risk=RiskLevel.NONE, findings=[], duration=2),
79+
tool_name="safe_tool",
80+
language="bash",
81+
intercepted=False,
82+
audit_path=str(path),
83+
)
84+
assert rec.intercepted is False
85+
assert rec.decision == "ALLOW"
86+
assert path.exists()
87+
line = json.loads(path.read_text(encoding="utf-8").strip())
88+
assert line["tool_name"] == "safe_tool"
89+
90+
91+
def test_emit_otel_is_safe_without_active_span():
92+
# No active OTel span in unit tests -> must be a no-op, never raise.
93+
emit_otel(_report(), tool_name="t", intercepted=True)
Lines changed: 44 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,44 @@
1+
# Copyright (C) 2026 Tencent. All rights reserved.
2+
# tRPC-Agent-Python is licensed under Apache-2.0.
3+
"""Tests for the resource-abuse rules added to close issue #90 risk class #5."""
4+
from __future__ import annotations
5+
6+
from trpc_agent_sdk.tools.safety._bash_scanner import scan_bash
7+
from trpc_agent_sdk.tools.safety._policy import load_policy
8+
from trpc_agent_sdk.tools.safety._python_scanner import scan_python
9+
10+
11+
def _py(src):
12+
return {f.rule_id for f in scan_python(load_policy(), src)}
13+
14+
15+
def _bash(src):
16+
return {f.rule_id for f in scan_bash(load_policy(), src)}
17+
18+
19+
def test_python_large_write_huge_payload():
20+
src = 'open("big.bin", "wb").write(b"\\x00" * 100000000)'
21+
assert "tool-res-large-write" in _py(src)
22+
23+
24+
def test_python_write_small_payload_not_flagged():
25+
# Small write must not trigger the large-write rule (false-positive guard).
26+
assert "tool-res-large-write" not in _py('open("s.txt","w").write("hi")')
27+
28+
29+
def test_python_concurrent_flood_large_pool():
30+
src = "from concurrent.futures import ThreadPoolExecutor\nThreadPoolExecutor(max_workers=10000)"
31+
assert "tool-res-concurrent-flood" in _py(src)
32+
33+
34+
def test_python_small_pool_not_flagged():
35+
src = "from concurrent.futures import ThreadPoolExecutor\nThreadPoolExecutor(max_workers=4)"
36+
assert "tool-res-concurrent-flood" not in _py(src)
37+
38+
39+
def test_bash_large_write_dd_gigabytes():
40+
assert "tool-res-large-write" in _bash("dd if=/dev/zero of=big.bin bs=1G count=10")
41+
42+
43+
def test_bash_large_write_head_c_huge():
44+
assert "tool-res-large-write" in _bash("head -c 50000000 /dev/urandom > big.bin")

trpc_agent_sdk/tools/safety/__init__.py

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -3,6 +3,8 @@
33
"""Tool Script Safety Guard public API."""
44
from __future__ import annotations
55

6+
from trpc_agent_sdk.tools.safety._audit import AuditRecord
7+
from trpc_agent_sdk.tools.safety._audit import record_safety_decision
68
from trpc_agent_sdk.tools.safety._code_executor_guard import SafetyGuardedCodeExecutor
79
from trpc_agent_sdk.tools.safety._decision import aggregate
810
from trpc_agent_sdk.tools.safety._policy import Policy
@@ -27,4 +29,6 @@
2729
"aggregate",
2830
"ToolSafetyFilter",
2931
"SafetyGuardedCodeExecutor",
32+
"AuditRecord",
33+
"record_safety_decision",
3034
]

0 commit comments

Comments
 (0)