Skip to content

Commit b465361

Browse files
DNKYrDNKYr
committed
docs(tools): add design docs, test plan, and final review fixes
1 parent f0bdac0 commit b465361

6 files changed

Lines changed: 817 additions & 7 deletions

File tree

‎docs/tools_safety/README.md‎

Lines changed: 2 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -21,19 +21,15 @@ The scanner also integrates as a `BaseFilter` in the tool filter chain, so it ru
2121
### Filter Mode (automatic)
2222

2323
```python
24-
from trpc_agent_sdk.filter import register_filter, FilterType
2524
from trpc_agent_sdk.tools.safety import ToolSafetyScanner, ToolSafetyFilter, SafetyAuditLogger
2625
from trpc_agent_sdk.tools import FunctionTool
2726

2827
scanner = ToolSafetyScanner("path/to/tool_safety_policy.yaml")
2928
audit = SafetyAuditLogger("tool_safety_audit.jsonl")
3029

31-
# Register the filter globally
30+
# Create filter and attach directly to a tool
3231
filter_instance = ToolSafetyFilter(scanner=scanner, audit_logger=audit)
33-
register_filter(FilterType.TOOL, "tool_safety")(type(filter_instance))
34-
35-
# Attach to a tool
36-
tool = FunctionTool(func=my_func, filters=["tool_safety"])
32+
tool = FunctionTool(func=my_func, filters=[filter_instance])
3733
```
3834

3935
### Standalone Mode

‎docs/tools_safety/design.md‎

Lines changed: 319 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,319 @@
1+
# Tool Script Safety Guard — Design Document
2+
3+
## Overview
4+
5+
The Tool Script Safety Guard is a security mechanism that scans tool-executed scripts (Python, Bash) for risks **before** execution. It integrates into tRPC-Agent's existing filter pipeline, provides a standalone scanning API, and emits structured audit records and OpenTelemetry span attributes.
6+
7+
### Scope
8+
9+
- Scan Python scripts and Bash commands for 6 risk categories
10+
- Pluggable as a `BaseFilter` in the tool execution chain
11+
- Configurable via `tool_safety_policy.yaml` without code changes
12+
- Output structured scan reports and JSONL audit logs
13+
- Decorate existing OpenTelemetry spans with safety metadata
14+
15+
### Out of Scope
16+
17+
- Runtime sandboxing (handled by `ContainerCodeExecutor`, `CubeCodeExecutor`)
18+
- Network-level firewalling or egress control
19+
- Replacement for proper container/VM isolation
20+
21+
---
22+
23+
## Package Structure
24+
25+
```
26+
trpc_agent_sdk/tools/safety/
27+
├── __init__.py # Public exports
28+
├── _types.py # Enums, dataclasses (RiskType, Decision, ScanReport, AuditEvent)
29+
├── _policy.py # SafetyPolicy model, YAML loading/validation
30+
├── _rules.py # Rule definitions: PatternRule, AstRule (~14 rules)
31+
├── _scanner.py # ToolSafetyScanner — orchestration and scan pipeline
32+
├── _filter.py # ToolSafetyFilter(BaseFilter) — filter integration
33+
├── _audit.py # SafetyAuditLogger — JSONL audit logging
34+
├── _telemetry.py # set_safety_span_attrs() — OTel span decoration
35+
└── tool_safety_policy.yaml # Default policy file
36+
```
37+
38+
---
39+
40+
## Data Model
41+
42+
### Enums
43+
44+
| Enum | Values | Description |
45+
|------|--------|-------------|
46+
| `RiskType` | `dangerous_file_operation`, `network_access`, `system_command`, `dependency_install`, `resource_abuse`, `sensitive_info_leak` | 6 risk categories |
47+
| `Decision` | `allow`, `deny`, `needs_human_review` | Outcome of a safety scan |
48+
| `RiskLevel` | `low`, `medium`, `high`, `critical` | Severity of a finding |
49+
50+
### Dataclasses
51+
52+
**RuleFinding** — A single rule match:
53+
- `rule_id: str` — e.g. `"DANGEROUS_DELETE_001"`
54+
- `risk_type: RiskType`
55+
- `risk_level: RiskLevel`
56+
- `evidence: str` — matched script line/snippet
57+
- `message: str` — human-readable description
58+
- `recommendation: str` — suggested mitigation
59+
60+
**ScanReport** — Aggregated scan result:
61+
- `decision: Decision` — worst-case decision across all findings
62+
- `risk_level: RiskLevel | None`
63+
- `findings: list[RuleFinding]`
64+
- `scan_duration_ms: float`
65+
- `script_snippet: str | None` — first N chars for context
66+
- `scan_error: str | None`
67+
68+
**AuditEvent** — For JSONL logging:
69+
- `timestamp: str` (ISO 8601)
70+
- `tool_name: str`
71+
- `decision: str`
72+
- `risk_level: str | None`
73+
- `rule_ids: list[str]`
74+
- `scan_duration_ms: float`
75+
- `sanitized: bool`
76+
- `intercepted: bool`
77+
- `script_hash: str` (SHA-256)
78+
79+
---
80+
81+
## Policy Configuration
82+
83+
`tool_safety_policy.yaml` controls all configurable behavior:
84+
85+
```yaml
86+
version: "1.0"
87+
max_script_size_bytes: 1048576 # 1 MB
88+
max_scan_time_ms: 1000 # 1 second
89+
default_decision: deny
90+
91+
rules:
92+
- rule_id: DANGEROUS_DELETE_001
93+
enabled: true
94+
risk_type: dangerous_file_operation
95+
severity: critical
96+
decision: deny
97+
- rule_id: SENSITIVE_PATH_002
98+
enabled: true
99+
risk_type: dangerous_file_operation
100+
severity: critical
101+
decision: deny
102+
# ...
103+
104+
whitelist:
105+
domains: ["api.example.com", "trusted.internal.org"]
106+
commands: ["ls", "cat", "echo", "pwd"]
107+
paths: ["/tmp/", "/workspace/", "./"]
108+
109+
blocklist:
110+
paths: ["~/.ssh", "~/.aws", "/etc/passwd", "/etc/shadow", ".env"]
111+
commands: ["sudo", "chmod 777"]
112+
```
113+
114+
### Policy behavior rules:
115+
116+
1. Whitelist overrides blocklist — an item in both is allowed
117+
2. `default_decision` is used when no rules match (conservative: `deny`)
118+
3. Rules can be individually enabled/disabled via `enabled: false`
119+
4. Severity and decision are overridable per-rule
120+
5. `max_scan_time_ms` acts as hard timeout; timeout → `default_decision`
121+
122+
---
123+
124+
## Risk Categories and Rules
125+
126+
### 1. Dangerous File Operations (`dangerous_file_operation`)
127+
128+
| Rule ID | Triggers | Detection |
129+
|---------|----------|-----------|
130+
| `DANGEROUS_DELETE_001` | `rm -rf`, `shutil.rmtree()`, `os.remove()` on non-/tmp paths | Pattern + AST |
131+
| `SENSITIVE_PATH_002` | Access to `~/.ssh`, `/etc/passwd`, `~/.aws`, `.env`, `~/.config` | Pattern |
132+
133+
### 2. Network Access (`network_access`)
134+
135+
| Rule ID | Triggers | Detection |
136+
|---------|----------|-----------|
137+
| `NETWORK_CURL_003` | `curl`, `wget` in Bash scripts | Pattern + domain whitelist check |
138+
| `NETWORK_PYTHON_004` | `requests.get/post`, `httpx.get/post`, `urllib.request` | Pattern + AST |
139+
| `NETWORK_SOCKET_005` | `socket.connect()`, `socket.create_connection()` | Pattern + AST |
140+
141+
### 3. System and Process Commands (`system_command`)
142+
143+
| Rule ID | Triggers | Detection |
144+
|---------|----------|-----------|
145+
| `SUBPROCESS_006` | `subprocess.run/Popen/call` | Pattern + AST |
146+
| `OS_SYSTEM_007` | `os.system()`, `os.popen()`, backtick execution in Bash | Pattern |
147+
| `PRIVILEGE_ESCALA_009` | `sudo`, `su`, `chmod 777`, `chown` | Pattern |
148+
149+
### 4. Dependency Installation (`dependency_install`)
150+
151+
| Rule ID | Triggers | Detection |
152+
|---------|----------|-----------|
153+
| `DEP_INSTALL_008` | `pip install`, `npm install`, `apt-get install`, `yum install` | Pattern |
154+
155+
### 5. Resource Abuse (`resource_abuse`)
156+
157+
| Rule ID | Triggers | Detection |
158+
|---------|----------|-----------|
159+
| `FORK_BOMB_011` | `:(){ :\|:& };:`, mass `fork()` calls | Pattern + AST |
160+
| `INFINITE_LOOP_012` | `while True:`, `while (true)`, `for(;;)` | Pattern + AST |
161+
162+
### 6. Sensitive Information Leak (`sensitive_info_leak`)
163+
164+
| Rule ID | Triggers | Detection |
165+
|---------|----------|-----------|
166+
| `SENSITIVE_LOG_010` | `print(api_key)`, `write(token)`, env vars named `*KEY*`, `*TOKEN*`, `*SECRET*`, `*PASSWORD*` | Pattern + AST |
167+
168+
---
169+
170+
## Scan Pipeline
171+
172+
```
173+
script_text, tool_name, args, env_vars
174+
│
175+
▼
176+
1. PRE-CHECK —— size > max_script_size_bytes? → deny
177+
│
178+
▼
179+
2. WHITELIST FAST PATH —— all detected domains/commands/paths in whitelist?
180+
│ → allow (skip full scan)
181+
▼
182+
3. PATTERN SCAN —— apply all enabled pattern rules against text, args, env_vars
183+
│
184+
▼
185+
4. AST SCAN —— if Python detected, parse AST, apply AST rules
186+
│
187+
▼
188+
5. AGGREGATE —— combine findings, resolve highest severity → decision
189+
│
190+
▼
191+
6. AUDIT —— emit ScanReport + audit event
192+
│
193+
▼
194+
return ScanReport
195+
```
196+
197+
### Decision aggregation
198+
199+
```
200+
worst_decision = max(findings, key=severity_priority)
201+
```
202+
203+
Priority: `DENY > NEEDS_HUMAN_REVIEW > ALLOW`. A single `DENY` finding blocks execution regardless of other findings.
204+
205+
### Python detection heuristic
206+
207+
The scanner checks for: `def `, `import `, `from `, `class `, `#!python`. If none of these are present, the AST scan step is skipped and only pattern rules apply. Malformed Python that fails `ast.parse()` is caught gracefully and pattern results are still returned.
208+
209+
### Timeout
210+
211+
`asyncio.wait_for()` wraps the pattern + AST scan phases. If the timeout (`max_scan_time_ms`) is exceeded, the scan returns `default_decision` with an error note.
212+
213+
---
214+
215+
## Integration Points
216+
217+
### 1. Filter mode
218+
219+
`ToolSafetyFilter` extends `BaseFilter` and is registered for `FilterType.TOOL`. It implements `_before()` to scan tool arguments before execution and sets `rsp.is_continue = False` on deny.
220+
221+
```python
222+
scanner = ToolSafetyScanner("tool_safety_policy.yaml")
223+
register_filter(FilterType.TOOL, "tool_safety", ToolSafetyFilter(scanner, audit_logger))
224+
225+
# Attach to a tool
226+
tool = FunctionTool(func=..., filters=["tool_safety"])
227+
```
228+
229+
### 2. Standalone mode
230+
231+
```python
232+
scanner = ToolSafetyScanner("tool_safety_policy.yaml")
233+
report = await scanner.scan(
234+
script="rm -rf /home/user/data",
235+
tool_name="bash_tool",
236+
)
237+
if report.decision == Decision.DENY:
238+
raise SafetyViolationError(report)
239+
```
240+
241+
### 3. OpenTelemetry
242+
243+
When an active span exists (e.g., the tool execution span created in `ToolsProcessor._execute_tool()`), the safety filter sets these attributes:
244+
245+
| Attribute | Type | Description |
246+
|-----------|------|-------------|
247+
| `tool.safety.decision` | string | `allow`, `deny`, or `needs_human_review` |
248+
| `tool.safety.risk_level` | string | `low`, `medium`, `high`, or `critical` |
249+
| `tool.safety.rule_ids` | string[] | IDs of all triggered rules |
250+
| `tool.safety.scan_duration_ms` | float | Scan duration in milliseconds |
251+
252+
No new spans are created — existing tool spans are decorated with safety metadata.
253+
254+
---
255+
256+
## Audit Logging
257+
258+
Writes one JSON object per line to `tool_safety_audit.jsonl`:
259+
260+
```json
261+
{"timestamp":"2026-07-10T12:00:00Z","tool_name":"bash_tool","decision":"deny","risk_level":"critical","rule_ids":["DANGEROUS_DELETE_001"],"scan_duration_ms":12.5,"sanitized":false,"intercepted":true,"script_hash":"a1b2c3..."}
262+
{"timestamp":"2026-07-10T12:01:00Z","tool_name":"python_tool","decision":"allow","risk_level":null,"rule_ids":[],"scan_duration_ms":3.1,"sanitized":false,"intercepted":false,"script_hash":"d4e5f6..."}
263+
```
264+
265+
---
266+
267+
## Relationship to Other Framework Components
268+
269+
| Component | Relationship |
270+
|-----------|-------------|
271+
| **CodeExecutor** (`UnsafeLocalCodeExecutor`, `ContainerCodeExecutor`, `CubeCodeExecutor`) | Safety Guard is a **pre-execution** scanner. CodeExecutors provide **runtime** isolation. The two are complementary layers: Safety Guard blocks known-dangerous scripts; the executor sandboxes scripts that pass. Safety Guard does **not** replace sandboxing. |
272+
| **Filter system** (`BaseFilter`, `FilterRunner`, `FilterRegistry`) | `ToolSafetyFilter` is a standard filter plugged into the tool filter chain via `FilterType.TOOL`. It runs in `_before()` to inspect and potentially block execution. |
273+
| **Telemetry** (`trace`, `metrics`) | Safety Guard decorates existing tool spans with `tool.safety.*` attributes. Does not create new spans or metrics. |
274+
| **Callback system** (`ToolCallbackFilter`) | Safety Guard runs alongside callbacks in the same filter chain. Ordering: safety filter should run first (before callbacks) to block dangerous execution early. This is controlled by filter registration order. |
275+
276+
---
277+
278+
## Known Limitations
279+
280+
1. **Pattern-based detection is bypassable** — obfuscated scripts (e.g., `__import__("os").system(...)`) will evade regex rules. AST rules catch some but not all obfuscation.
281+
2. **Bash parsing is pattern-only** — no Bash AST exists. Complex Bash scripts with variable indirection may bypass rules.
282+
3. **False positives** — safe scripts that happen to mention dangerous keywords (e.g., a security tutorial) will be flagged.
283+
4. **No data flow analysis** — the scanner only checks syntax, not whether a sensitive value actually flows into a dangerous call. A script that reads `API_KEY` but never outputs it will still be flagged by `SENSITIVE_LOG_010`.
284+
5. **Not a sandbox** — this is a static analysis tool. It cannot prevent runtime exploits, memory corruption, or novel attack vectors.
285+
6. **Whitelist fast path is conservative** — if even one element is not in the whitelist, the full scan runs. This means partially-whitelisted scripts still incur scan overhead.
286+
287+
---
288+
289+
## Extending with New Rules
290+
291+
Add a new rule for a new risk type:
292+
293+
```python
294+
@pattern_rule(
295+
rule_id="NEW_RULE_013",
296+
risk_type=RiskType.SYSTEM_COMMAND,
297+
severity=RiskLevel.HIGH,
298+
pattern=r"evil_command\s+--dangerous",
299+
message="Detected use of evil_command",
300+
recommendation="Use safe_command instead",
301+
)
302+
async def check_evil_command(text: str) -> RuleFinding | None:
303+
...
304+
```
305+
306+
For AST rules, subclass `ast.NodeVisitor` and register with the scanner. All rules are automatically discovered by the scanner from the policy file and applied in order.
307+
308+
---
309+
310+
## File Index
311+
312+
| File | Purpose |
313+
|------|---------|
314+
| `design.md` | This document — architecture and design |
315+
| `design.zh_CN.md` | Chinese version of this document |
316+
| `test_plan.md` | Test cases and acceptance criteria |
317+
| `tool_safety_policy.yaml` | Default policy configuration (in `tools/safety/`) |
318+
| `tool_safety_report.json` | Example scan report output |
319+
| `tool_safety_audit.jsonl` | Example audit log output |

0 commit comments

Comments
 (0)