Skip to content

Commit b48ac21

Browse files
committed
feat: Tool Script Safety Guard for script execution (issue #90)
Static pre-execution scanner for Tool/Skill/CodeExecutor scripts. Mirrors the Go reference (trpc-agent-go/tool/safety) for rule_id prefixes and Decision/RiskLevel types, extended to issue #90's 6 risk classes. - Python scanner: ast + import-as alias tracking (prevents rename bypass) - Bash scanner: shlex + quote-aware state machine - Conservative 3-state decision (any DENY>deny; else REVIEW>review; else ALLOW) with policy risk-threshold fallback - Two zero-core-change integrations: ToolSafetyFilter (registered tool filter) + SafetyGuardedCodeExecutor (delegating wrapper) - YAML policy (editable without code), structured report, CLI - 59 tests incl. 12 acceptance manifest samples + performance (500 lines <1s) - zh/en docs
1 parent 099b571 commit b48ac21

29 files changed

Lines changed: 4516 additions & 0 deletions

docs/mkdocs/en/tool_safety.md

Lines changed: 301 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,301 @@
1+
# Tool Script Safety Guard
2+
3+
Tool Script Safety Guard is a **static script security scanning module** in the tRPC-Agent-Python SDK that performs static security analysis on scripts **before** they are executed by Agent Tools / Skills / CodeExecutor. It produces `Allow / Deny / NeedsReview` decisions and intercepts high-risk scripts at the front of the execution chain.
4+
5+
## Core Concepts
6+
7+
### What & Why
8+
9+
**Tool Script Safety Guard** is a **static pre-execution script scanner**, not a runtime sandbox. It reduces security risks by analyzing script content **before** execution through:
10+
11+
- **Static Analysis**: Detects dangerous patterns (dangerous file operations, network egress, sensitive information leakage) without running the script
12+
- **Zero Intrusion**: Integrates via Filter or Wrapper without modifying core source code, fully backward compatible
13+
- **Dual Language Support**: Supports both Python (AST + import-as alias tracking) and Bash (shlex + quote state machine)
14+
- **Conservative Policy**: Defaults to conservative decision-making, tending to block rather than allow uncertain cases
15+
16+
**Important**: This mechanism is "pre-execution static policy judgment" and **cannot replace sandbox isolation**. Runtime resource limits and environment isolation must still rely on the CodeExecutor's container or sandbox mechanisms. This is exactly why we chose the wrapper approach without modifying core source code.
17+
18+
### Quick Start
19+
20+
#### Method 1: Using `ToolSafetyFilter` to Intercept Tool/Skill Execution
21+
22+
```python
23+
from trpc_agent_sdk.tools.safety import ToolSafetyFilter
24+
25+
# Add safety filter when defining a Tool
26+
class MyTool(BaseTool):
27+
# Method 1: Attach via filters_name parameter
28+
filters_name = ["tool_safety"] # Auto-registered filter name
29+
30+
# Method 2: Dynamically attach via add_one_filter
31+
def __init__(self):
32+
super().__init__()
33+
self.add_one_filter("tool_safety")
34+
35+
async def _run_async_impl(self, **kwargs):
36+
# When kwargs contains script/code/command fields
37+
# Safety scan runs first, only executes here if decision is ALLOW
38+
code = kwargs.get("code", "")
39+
return await execute_some_script(code)
40+
```
41+
42+
#### Method 2: Using `SafetyGuardedCodeExecutor` to Wrap CodeExecutor
43+
44+
```python
45+
from trpc_agent_sdk.code_executors import UnsafeLocalCodeExecutor
46+
from trpc_agent_sdk.tools.safety import SafetyGuardedCodeExecutor
47+
48+
# Wrap any CodeExecutor, automatically scan each code block
49+
original_executor = UnsafeLocalCodeExecutor()
50+
guarded_executor = SafetyGuardedCodeExecutor(
51+
delegate=original_executor,
52+
block_on_review=True # Whether to block NEEDS_REVIEW decisions, defaults to True
53+
)
54+
55+
# Use guarded_executor, dangerous code blocks will be skipped
56+
result = await guarded_executor.execute_code(
57+
invocation_context=context,
58+
input_data=CodeExecutionInput(
59+
code="import os; os.system('rm -rf /')", # Dangerous code
60+
language="python"
61+
)
62+
)
63+
# result.stderr will contain "TOOL_SAFETY_BLOCKED [python] DENY (...)"
64+
```
65+
66+
### rule_id Domain Mapping
67+
68+
The system covers **7 major risk domains** with 18 rules (aligning with `trpc-agent-go/tool/safety` style):
69+
70+
| Domain Prefix | Risk Coverage | Example rule_ids |
71+
|---------------|---------------|------------------|
72+
| `tool-code-*` | Code execution | `tool-code-unsafe-eval`, `tool-code-unsafe-exec`, `tool-code-unsafe-import` |
73+
| `tool-fs-*` | Dangerous file operations | `tool-fs-recursive-delete`, `tool-fs-read-credentials`, `tool-fs-system-dir-write` |
74+
| `tool-net-*` | Network egress | `tool-net-http`, `tool-net-socket` |
75+
| `tool-proc-*` | Process/system commands | `tool-proc-subprocess`, `tool-proc-shell-pipe`, `tool-proc-privilege-escalation` |
76+
| `tool-pkg-*` | Dependency installation | `tool-pkg-install` (pip/npm/apt) |
77+
| `tool-res-*` | Resource abuse | `tool-res-infinite-loop`, `tool-res-fork-bomb`, `tool-res-long-sleep` |
78+
| `tool-secret-*` | Sensitive information leakage | `tool-secret-logging`, `tool-secret-private-key` |
79+
80+
### Decision and RiskLevel
81+
82+
#### Decision
83+
84+
```python
85+
class Decision(IntEnum):
86+
UNDECIDED = 0 # Undecided (rule not covered)
87+
ALLOW = 1 # Allow execution
88+
DENY = 2 # Deny execution
89+
NEEDS_REVIEW = 3 # Needs manual review
90+
```
91+
92+
#### RiskLevel
93+
94+
```python
95+
class RiskLevel(IntEnum):
96+
NONE = 0
97+
LOW = 1
98+
MEDIUM = 2
99+
HIGH = 3
100+
```
101+
102+
#### Decision Aggregation Logic (Conservative Policy)
103+
104+
The system uses **dual-track judgment**: rule-level judgment priority + policy threshold fallback.
105+
106+
1. **Any Finding with `rule_decision == DENY`**, or `risk_level >= policy.deny_risk_level``Decision.DENY`
107+
2. **Otherwise any `rule_decision == NEEDS_REVIEW`**, or `risk_level >= policy.review_risk_level``Decision.NEEDS_REVIEW`
108+
3. **Otherwise**`Decision.ALLOW`
109+
110+
Default thresholds:
111+
- `deny_risk_level: HIGH` # Risk ≥ HIGH → DENY
112+
- `review_risk_level: MEDIUM` # Risk ≥ MEDIUM → NEEDS_REVIEW
113+
114+
### Policy Configuration
115+
116+
Configure policies via YAML file. **Editing the configuration file changes behavior without modifying code** (satisfying issue acceptance #6).
117+
118+
#### Configuration File Location
119+
120+
Specify the path via environment variable `TRPC_AGENT_TOOL_SAFETY_POLICY`. If not specified, the built-in default policy is used:
121+
122+
```bash
123+
export TRPC_AGENT_TOOL_SAFETY_POLICY=/path/to/custom_policy.yaml
124+
```
125+
126+
#### YAML Field Explanations
127+
128+
```yaml
129+
name: default
130+
description: Default tool script safety policy for tRPC-Agent.
131+
132+
# Risk level thresholds (used when rule's decision is UNDECIDED)
133+
deny_risk_level: HIGH # findings >= HIGH -> DENY
134+
review_risk_level: MEDIUM # findings >= MEDIUM (and < deny) -> NEEDS_REVIEW
135+
136+
# Global whitelists
137+
whitelisted_domains: # Network whitelist domains
138+
- pypi.org
139+
- github.com
140+
- example.com
141+
allowed_commands: # Allowed commands
142+
- ls
143+
- cat
144+
- echo
145+
- python
146+
denied_paths: # Denied access paths
147+
- /etc
148+
- /root
149+
- ~/.ssh
150+
- ~/.env
151+
- ~/.aws/credentials
152+
153+
# Resource limits (informational for static scan; enforced by executor runtime)
154+
max_timeout_seconds: 30
155+
max_output_bytes: 1048576
156+
max_evidence_chars: 200 # Maximum evidence fragment length
157+
158+
# Rule-level overrides (optional)
159+
rule_overrides:
160+
# tool-net-http:
161+
# risk_level: HIGH
162+
# decision: DENY
163+
```
164+
165+
### Relationship with Other Components
166+
167+
#### Relationship with Sandbox
168+
169+
**Tool Script Safety Guard cannot replace sandbox isolation**:
170+
171+
- **This mechanism**: Static analysis, intercept **before** execution, based on pattern matching
172+
- **Sandbox**: Runtime isolation, restrict **during** execution, based on resource/permission control
173+
- **Complementarity**: Static interception reduces sandbox escape risk; sandbox prevents actual damage from statically missed code
174+
175+
**Why it cannot replace sandbox**:
176+
1. Inherent static analysis limitations: obfuscation/encoding bypasses (`base64 -d | sh`), dynamic concatenation, indirect calls can leak
177+
2. Runtime behavior is unpredictable: in-memory code injection, reflection calls cannot be statically detected
178+
3. Resource abuse requires runtime limits: infinite loops, memory exhaustion need timeout/resource quota control
179+
180+
#### Relationship with Filter
181+
182+
`ToolSafetyFilter` is a subclass of `BaseFilter`, registered as `"tool_safety"`:
183+
184+
- **Execution timing**: **Before** Tool's `_run_async_impl`
185+
- **Interception behavior**: When decision is not `ALLOW`, returns `FilterResult(is_continue=False)`, does not call `handle()`
186+
- **Applicable scenarios**: Intercept single Tool/Skill execution
187+
188+
#### Relationship with CodeExecutor
189+
190+
`SafetyGuardedCodeExecutor` is a wrapper for `BaseCodeExecutor`:
191+
192+
- **Execution timing**: **Before** CodeExecutor's `execute_code`
193+
- **Interception behavior**: Scan each CodeBlock individually, skip dangerous blocks, only execute safe blocks
194+
- **Applicable scenarios**: Protect any CodeExecutor (including `UnsafeLocalCodeExecutor`)
195+
196+
#### Relationship with Telemetry
197+
198+
- **Audit events**: Log structured events when intercepted (containing `decision`/`risk_level`/`rule_ids`/`evidence`/`recommendation`)
199+
- **OpenTelemetry**: Interface reserved (not implemented in MVP; can add span attributes in the future)
200+
201+
### Known Limitations
202+
203+
1. **Inherent Static Analysis Limitations**
204+
- Obfuscation/encoding bypasses: `base64 -d | sh`, `eval(base64.b64decode("..."))` can leak
205+
- Dynamic concatenation: `getattr(os, "system")("rm -rf /")`, indirect calls may leak
206+
- Runtime code injection: In-memory modifications, `__import__` dynamic loading cannot be statically detected
207+
208+
2. **False Positives and False Negatives**
209+
- **False positives**: Legitimate scripts match dangerous patterns (e.g., legitimate `subprocess.call` flagged)
210+
- **False negatives**: New bypass techniques not covered (e.g., new obfuscation methods)
211+
- **Tuning methods**: Adjust via `whitelisted_domains`, `allowed_commands`, `rule_overrides`
212+
213+
3. **Bash Parsing is Heuristic**
214+
- Uses `shlex` + state machine, not a complete POSIX shell parser
215+
- Complex quote/escape boundaries may be misjudged (e.g., `$'..."..."'` nesting)
216+
217+
4. **Python AST Parsing Failures**
218+
- Falls back to string heuristics when AST parsing fails (logs but does not block)
219+
- Depends on syntax correctness; syntax-error scripts may bypass detection
220+
221+
### Extending Rules
222+
223+
Adding new rules requires modifications in two places:
224+
225+
#### 1. Add Constants in `_rules.py`
226+
227+
```python
228+
# trpc_agent_sdk/tools/safety/_rules.py
229+
230+
# Add new rule ID
231+
R_MY_CUSTOM_RULE = "tool-custom-my-rule"
232+
233+
# Define default behavior in DEFAULT_RULE_POLICIES
234+
DEFAULT_RULE_POLICIES: dict[str, tuple[RiskLevel, Decision]] = {
235+
# ... existing rules ...
236+
R_MY_CUSTOM_RULE: (RiskLevel.MEDIUM, Decision.NEEDS_REVIEW),
237+
}
238+
```
239+
240+
#### 2. Add Detection Logic in Scanner
241+
242+
**Python Scanner** (`_python_scanner.py`):
243+
244+
```python
245+
# Add detection branch in scan function
246+
def _scan_python_script(policy: Policy, script: str) -> list[Finding]:
247+
findings = []
248+
# ... existing detection logic ...
249+
250+
# Add new detection
251+
if "dangerous_pattern" in script:
252+
findings.append(Finding(
253+
rule_id=R_MY_CUSTOM_RULE,
254+
risk_level=RiskLevel.MEDIUM,
255+
rule_decision=Decision.NEEDS_REVIEW,
256+
evidence="...",
257+
recommendation="...",
258+
language="python"
259+
))
260+
261+
return findings
262+
```
263+
264+
**Bash Scanner** (`_bash_scanner.py`):
265+
266+
```python
267+
# Add detection branch in Bash scan function
268+
def _scan_bash_script(policy: Policy, script: str) -> list[Finding]:
269+
findings = []
270+
# ... existing detection logic ...
271+
272+
# Add new detection
273+
if "dangerous_command" in tokens:
274+
findings.append(Finding(
275+
rule_id=R_MY_CUSTOM_RULE,
276+
risk_level=RiskLevel.MEDIUM,
277+
rule_decision=Decision.NEEDS_REVIEW,
278+
evidence="...",
279+
recommendation="...",
280+
language="bash"
281+
))
282+
283+
return findings
284+
```
285+
286+
#### 3. Override in YAML (Optional)
287+
288+
```yaml
289+
# tool_safety_policy.yaml
290+
rule_overrides:
291+
tool-custom-my-rule:
292+
risk_level: HIGH
293+
decision: DENY
294+
```
295+
296+
### References
297+
298+
- **Design Document**: `docs/superpowers/specs/2026-07-09-tool-safety-guard-design.md`
299+
- **Implementation Code**: `trpc_agent_sdk/tools/safety/`
300+
- **Test Cases**: `tests/tools/safety/samples/manifest.yaml`
301+
- **Corresponding Issue**: [trpc-group/trpc-agent-python#90](https://github.com/trpc-group/trpc-agent-python/issues/90)

0 commit comments

Comments
 (0)