|
| 1 | +# Tool Script Safety Guard — Design |
| 2 | + |
| 3 | +## Goals |
| 4 | + |
| 5 | +Provide an **opt-in** pre-execution static scanner for Tool / Skill / CodeExecutor |
| 6 | +payloads so that dangerous Python/Bash scripts can be denied or escalated to |
| 7 | +human review before they run. |
| 8 | + |
| 9 | +## Non-goals |
| 10 | + |
| 11 | +- Not a sandbox. Does not limit CPU, memory, filesystem, or network at runtime. |
| 12 | +- Not a perfect static analyzer. Dynamic construction can still evade rules. |
| 13 | +- Does not change default Tool/CodeExecutor behavior unless explicitly enabled. |
| 14 | + |
| 15 | +## Architecture |
| 16 | + |
| 17 | +``` |
| 18 | +request args (command/code/script) |
| 19 | + │ |
| 20 | + ▼ |
| 21 | + ToolSafetyFilter / wrapper / enable_safety_guard |
| 22 | + │ |
| 23 | + ▼ |
| 24 | + SafetyScanner |
| 25 | + ├─ language normalize |
| 26 | + ├─ rule pipeline (R001–R007 + custom) |
| 27 | + ├─ inline payload rescan (python -c / bash -c) |
| 28 | + └─ aggregate → Decision |
| 29 | + │ |
| 30 | + ├─ DENY → block + audit + OTel |
| 31 | + ├─ NEEDS_HUMAN_REVIEW → warn or block (policy.block_on_review) |
| 32 | + └─ ALLOW → continue execution |
| 33 | +``` |
| 34 | + |
| 35 | +## Package layout |
| 36 | + |
| 37 | +| Path | Role | |
| 38 | +|---|---| |
| 39 | +| `trpc_agent_sdk/safety/` | Implementation (light import surface) | |
| 40 | +| `trpc_agent_sdk/tools/safety/` | Official re-export entry | |
| 41 | +| `scripts/tool_safety_check.py` | CLI / CI gate | |
| 42 | +| `scripts/tool_safety_eval.py` | Detection / FP rate eval | |
| 43 | +| `examples/tool_safety/` | Policy, samples, fixtures, docs | |
| 44 | + |
| 45 | +## Risk domains |
| 46 | + |
| 47 | +| Domain | Rule IDs | |
| 48 | +|---|---| |
| 49 | +| Dangerous files | R001 | |
| 50 | +| Network egress | R002 | |
| 51 | +| Process / system | R003 | |
| 52 | +| Dependency install | R004 | |
| 53 | +| Resource abuse | R005 | |
| 54 | +| Secret leak | R006 | |
| 55 | +| Dynamic code execution | R007 | |
| 56 | + |
| 57 | +## Opt-in integration |
| 58 | + |
| 59 | +```python |
| 60 | +# Filter |
| 61 | +BashTool(enable_safety_guard=True, safety_policy_path="...") |
| 62 | + |
| 63 | +# Code executor |
| 64 | +UnsafeLocalCodeExecutor(enable_safety_guard=True) |
| 65 | + |
| 66 | +# Explicit filter |
| 67 | +tool = BashTool(filters=[ToolSafetyFilter(policy=policy)]) |
| 68 | +``` |
| 69 | + |
| 70 | +Defaults remain **disabled**. Enabling is explicit. |
| 71 | + |
| 72 | +## Policy |
| 73 | + |
| 74 | +YAML controls domains, paths, commands, thresholds, `block_on_review`, |
| 75 | +`strict_command_allowlist`, and `strict_policy`. Also loadable via |
| 76 | +`TOOL_SAFETY_POLICY_PATH` / `PolicyConfig.from_env()`. |
| 77 | + |
| 78 | +## Why this cannot replace a sandbox |
| 79 | + |
| 80 | +| | Safety Guard | Sandbox | |
| 81 | +|---|---|---| |
| 82 | +| When | Before run | During run | |
| 83 | +| Strength | Intent / pattern policy | Hard isolation | |
| 84 | +| Failure mode | Miss dynamic payload | Still contained | |
| 85 | + |
| 86 | +Use both: guard first, sandbox always for untrusted code. |
| 87 | + |
| 88 | +## Extension |
| 89 | + |
| 90 | +```python |
| 91 | +from trpc_agent_sdk.safety import register_rule, SafetyRule |
| 92 | + |
| 93 | +@register_rule |
| 94 | +class MyRule(SafetyRule): |
| 95 | + rule_id = "CUSTOM_001" |
| 96 | + ... |
| 97 | +``` |
0 commit comments