Skip to content

Commit 0db96e0

Browse files
committed
docs: add tool safety guard example and guides
1 parent fb1eb19 commit 0db96e0

14 files changed

Lines changed: 1405 additions & 0 deletions

docs/mkdocs.yml

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -33,6 +33,7 @@ nav:
3333
- Skill: skill.md
3434
- Evolution: evolution.md
3535
- Tool: tool.md
36+
- Tool Safety Guard: tool-safety-guard.md
3637
- Session:
3738
- Overview: session/index.md
3839
- Session Summary: session/summary.md
@@ -140,6 +141,7 @@ plugins:
140141
- Skill: skill.md
141142
- Evolution: evolution.md
142143
- Tool: tool.md
144+
- Tool 安全防护: tool-safety-guard.md
143145
- Session:
144146
- Overview: session/index.md
145147
- Session Summary: session/summary.md
Lines changed: 68 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,68 @@
1+
# Tool Safety Guard
2+
3+
`tool/safety` is an opt-in, fail-closed guard for model-initiated tool calls. It scans the final arguments after before-tool callbacks, blocks or requests approval before execution, emits metadata-only audit records, and recursively redacts the final result after all callbacks and hooks.
4+
5+
Existing applications keep their historical behavior until a guard is configured. Decisions always compose as `deny > ask > allow`.
6+
7+
## What it covers
8+
9+
The built-in rules cover destructive commands, sensitive paths and credential reads, non-allowlisted network access, shell bypasses, host or interactive execution, dependency changes, excessive resources, and secret exposure. Policies support strict YAML or JSON parsing, exact domains and explicit `*.example.com` wildcards, per-tool command and environment allowlists, timeouts, and hard combined stdout/stderr limits.
10+
11+
## Configure the framework
12+
13+
```go
14+
policyData, err := os.ReadFile("tool_safety_policy.yaml")
15+
if err != nil { return err }
16+
policy, err := safety.ParsePolicy(policyData, safety.PolicyFormatAuto)
17+
if err != nil { return err }
18+
sink, err := safety.NewJSONLSink("tool_safety_audit.jsonl")
19+
if err != nil { return err }
20+
defer sink.Close()
21+
guard, err := safety.NewGuard(policy, safety.WithAuditSink(sink))
22+
if err != nil { return err }
23+
24+
runOptions := agent.NewRunOptions(
25+
agent.WithToolPermissionPolicy(guard),
26+
)
27+
```
28+
29+
Framework-wide configuration protects every tool and is recommended. The built-in `workspaceexec`, `hostexec`, and `codeexec` tools also accept `WithSafetyGuard(guard)` for direct calls. Their local guard is discoverable by framework observability, so raw arguments are suppressed before the tool executes. Runtime timeout/output profiles compose by the most restrictive value; `codeexec` passes limits through `codeexecutor.ExecutionLimits`, the local executor enforces them while running, and the wrapper caps returned output and file content.
30+
31+
## Example policy
32+
33+
```yaml
34+
version: 1
35+
default_action: allow
36+
profiles:
37+
workspace_exec:
38+
allowed_commands: [go, git]
39+
allowed_domains: [api.github.com, "*.trusted.example"]
40+
allowed_env: [CI]
41+
max_timeout: 2m
42+
max_output_bytes: 1048576
43+
```
44+
45+
Policy parsing rejects unknown fields, duplicate keys, trailing documents, invalid actions, and ambiguous numeric duration units. Reloading is atomic: an invalid replacement never weakens the active policy.
46+
47+
## Reports, audit, and telemetry
48+
49+
Every `Report` has stable `decision`, `risk_level`, `rule`, `rule_ids`, `evidence`, `recommendation`, `tool_name`, `command`, `backend`, and `blocked` fields. JSONL audit events contain metadata and a request SHA-256 digest, never raw arguments or results. OpenTelemetry spans expose the decision, blocked flag, risk, sorted bounded rule IDs, backend, digest, and scan duration.
50+
51+
An audit or final-redaction failure blocks execution or suppresses the result. Output and error sanitizers run after ordinary callbacks, tool-result-message hooks, post-result hooks, and streaming final-state handling.
52+
53+
## Verify without executing dangerous samples
54+
55+
The example is scan-only:
56+
57+
```bash
58+
go test -v ./tool_safety_guard -run TestAcceptanceMetrics
59+
go run ./tool_safety_guard \
60+
-policy tool_safety_guard/tool_safety_policy.yaml \
61+
-output-dir tool_safety_guard/output
62+
```
63+
64+
The public acceptance corpus asserts high-risk detection of at least 90%, safe false positives of at most 10%, and 100% denial for credential reads, protected destructive deletion, and non-allowlisted network access.
65+
66+
## Security boundary
67+
68+
Scanning is defense in depth, not a sandbox. Use least-privilege credentials, filesystem and network isolation, non-root containers or VMs, operating-system resource controls, and human approval for `ask`. Capability-aware workspace backends are rejected when a configured profile depends on guarantees they cannot provide. Third-party CodeExecutors must honor the supplied context limits while running; the wrapper can cap their returned result but cannot force a backend that ignores cancellation to stop.
Lines changed: 68 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,68 @@
1+
# Tool 安全防护
2+
3+
`tool/safety` 是一个按需启用、失败时关闭执行通道(fail closed)的工具安全组件。它会在 before-tool 回调完成后扫描最终参数,在执行前拒绝或请求人工批准,并在所有普通回调和 Hook 完成后递归脱敏最终结果;审计只记录元数据。
4+
5+
未配置 Guard 的应用保持原有行为。多层策略固定按 `deny > ask > allow` 合并。
6+
7+
## 覆盖范围
8+
9+
内置规则覆盖危险删除、敏感路径与凭据读取、非白名单网络、Shell 绕过、宿主机或交互执行、依赖变更、资源滥用和密钥泄露。严格 YAML/JSON 策略支持精确域名、显式 `*.example.com` 通配符、工具级命令与环境变量白名单、超时及 stdout/stderr 共用的硬输出上限。
10+
11+
## 在框架中配置
12+
13+
```go
14+
policyData, err := os.ReadFile("tool_safety_policy.yaml")
15+
if err != nil { return err }
16+
policy, err := safety.ParsePolicy(policyData, safety.PolicyFormatAuto)
17+
if err != nil { return err }
18+
sink, err := safety.NewJSONLSink("tool_safety_audit.jsonl")
19+
if err != nil { return err }
20+
defer sink.Close()
21+
guard, err := safety.NewGuard(policy, safety.WithAuditSink(sink))
22+
if err != nil { return err }
23+
24+
runOptions := agent.NewRunOptions(
25+
agent.WithToolPermissionPolicy(guard),
26+
)
27+
```
28+
29+
推荐使用框架级配置来保护所有工具。内置的 `workspaceexec``hostexec``codeexec` 也支持 `WithSafetyGuard(guard)`,可用于直接调用;框架能够发现这个工具本地 Guard,并在工具执行前抑制原始参数日志和遥测。运行时超时/输出 profile 按最严格值合并;`codeexec` 通过 `codeexecutor.ExecutionLimits` 下传限制,本地执行器在运行中执行,wrapper 再限制最终输出与文件内容。
30+
31+
## 策略示例
32+
33+
```yaml
34+
version: 1
35+
default_action: allow
36+
profiles:
37+
workspace_exec:
38+
allowed_commands: [go, git]
39+
allowed_domains: [api.github.com, "*.trusted.example"]
40+
allowed_env: [CI]
41+
max_timeout: 2m
42+
max_output_bytes: 1048576
43+
```
44+
45+
解析器会拒绝未知字段、重复键、尾随文档、非法动作和没有明确单位的时长。重载是原子的:新策略无效时,旧策略继续生效。
46+
47+
## 报告、审计与遥测
48+
49+
每份 `Report` 稳定包含 `decision`、`risk_level`、`rule`、`rule_ids`、`evidence`、`recommendation`、`tool_name`、`command`、`backend` 和 `blocked`。JSONL 审计只记录元数据与请求 SHA-256 摘要,不保存原始参数或结果。OpenTelemetry Span 记录决策、是否阻断、风险、排序且有界的完整规则 ID、后端、摘要和扫描耗时。
50+
51+
审计或最终脱敏失败时会阻止执行或压制结果。输出与错误脱敏屏障位于普通回调、工具结果消息 Hook、Post-result Hook 以及流式最终状态处理之后。
52+
53+
## 安全验证(不执行危险样本)
54+
55+
示例只做扫描:
56+
57+
```bash
58+
go test -v ./tool_safety_guard -run TestAcceptanceMetrics
59+
go run ./tool_safety_guard \
60+
-policy tool_safety_guard/tool_safety_policy.yaml \
61+
-output-dir tool_safety_guard/output
62+
```
63+
64+
公开验收语料直接断言:高危检出率不低于 90%,安全样本误报率不高于 10%,凭据读取、受保护路径危险删除、非白名单网络三类拒绝率均为 100%。
65+
66+
## 安全边界
67+
68+
扫描属于纵深防御,不能替代沙箱。生产环境仍应使用最小权限凭据、文件系统与网络隔离、非 root 容器或虚拟机、操作系统资源限制,并对 `ask` 决策进行人工审批。具备能力声明的 workspace 后端无法满足策略依赖时会失败关闭;第三方 CodeExecutor 必须在运行中遵守 context 限制,wrapper 能限制最终结果,但无法强制忽略取消信号的后端停止。

docs/tool-safety-guard.md

Lines changed: 144 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,144 @@
1+
# Tool Safety Guard
2+
3+
Package `tool/safety` provides an opt-in, fail-closed guard for model-initiated tool calls. It scans the final execution input before a tool runs, returns a structured permission decision, emits metadata-only audit events, and recursively removes secrets from the final result after callbacks.
4+
5+
Existing applications are unchanged until they configure a guard.
6+
7+
## Security contract
8+
9+
The guard addresses seven execution risks: destructive commands, sensitive paths, network access, shell bypasses, host/interactive execution, dependency changes, and resource abuse. Secret exposure is an always-on input/output protection and cannot be disabled by policy.
10+
11+
The key ordering is:
12+
13+
1. before-tool callbacks finish modifying arguments;
14+
2. the permission guard scans the effective arguments;
15+
3. only `allow` reaches the executor;
16+
4. plugins and after-tool callbacks finish modifying the result;
17+
5. the final sanitizer recursively redacts the effective result;
18+
6. only the sanitized result reaches events, model messages, and tool-result telemetry.
19+
20+
`deny` outranks `ask`, which outranks `allow`. Invalid policy or audit state never silently weakens the configured contract. A failed audit write or final sanitization returns an error and suppresses the unsafe result.
21+
22+
## Create a guard
23+
24+
```go
25+
policyData, err := os.ReadFile("policy.yaml")
26+
if err != nil {
27+
return err
28+
}
29+
policy, err := safety.ParsePolicy(policyData, safety.PolicyFormatAuto)
30+
if err != nil {
31+
return err
32+
}
33+
sink, err := safety.NewJSONLSink("tool-audit.jsonl")
34+
if err != nil {
35+
return err
36+
}
37+
defer sink.Close()
38+
guard, err := safety.NewGuard(policy, safety.WithAuditSink(sink))
39+
if err != nil {
40+
return err
41+
}
42+
```
43+
44+
`NewDefaultGuard` enables all built-in detections with a default `allow` action. `Reload` parses and atomically installs a complete policy; a rejected reload leaves the previous snapshot active.
45+
46+
## Framework-wide integration
47+
48+
Pass the guard as the per-run permission policy:
49+
50+
```go
51+
runOptions := agent.NewRunOptions(
52+
agent.WithToolPermissionPolicy(guard),
53+
)
54+
```
55+
56+
This route covers standard function-call processing and Graph execution, including wrapped tools and callback-provided custom results. The guard also acts as a final-result sanitizer.
57+
58+
## Direct built-in tool integration
59+
60+
Applications that call a built-in tool directly can opt in at construction:
61+
62+
```go
63+
workspaceTool := workspaceexec.NewExecTool(exec,
64+
workspaceexec.WithSafetyGuard(guard),
65+
)
66+
67+
hostTools := hostexec.NewToolSet(
68+
hostexec.WithSafetyGuard(guard),
69+
)
70+
71+
codeTool := codeexec.NewTool(exec,
72+
codeexec.WithSafetyGuard(guard),
73+
)
74+
```
75+
76+
Use either framework-wide integration or the direct option for a call path. Configuring the same guard in both places is safe but produces two scans and two audit events.
77+
78+
For `codeexec`, scanning occurs after flexible input decoding and language validation, over the code that the executor will actually receive. The wrapper applies the most restrictive direct/invocation timeout and output profile, passes in-flight limits through `codeexecutor.ExecutionLimits`, and caps the combined returned output/file content; the local executor enforces the output budget while the child is running. For interactive write tools, an empty poll remains allowed while non-empty input or newline submission requires approval by default.
79+
80+
## Policy
81+
82+
Policies are versioned and strictly decoded. Unknown fields, duplicate keys, multiple YAML documents, trailing JSON values, invalid actions, invalid domains, and negative limits are rejected.
83+
84+
```yaml
85+
version: 1
86+
default_action: allow
87+
profiles:
88+
workspace_exec:
89+
allowed_commands: [go, git]
90+
denied_commands: [curl]
91+
allowed_domains: [api.github.com, "*.corp.example"]
92+
forbidden_paths: [/etc, ~/.ssh]
93+
allowed_env: [CI]
94+
max_timeout: 2m
95+
max_output_bytes: 1048576
96+
allow_host: false
97+
allow_background: false
98+
allow_pty: false
99+
```
100+
101+
Domain matching is exact. `*.corp.example` matches `build.corp.example`, but not `corp.example` or `evilcorp.example`. Proxying, destination remapping, external curl/SSH configuration, and SSH forwarding require stronger handling because they can change the effective destination.
102+
103+
Profile timeout and output values are ceilings, not requested defaults. Runtimes that advertise hard-limit support receive and enforce these ceilings; a configured hard limit must not be represented as enforced on an opaque backend that cannot guarantee it.
104+
105+
## Reports, audit, and telemetry
106+
107+
`Report` contains a decision, redacted findings, request digest, duration, risk level, recommendation, backend, and blocked/redacted flags. It never needs the raw request to correlate repeated activity.
108+
109+
`JSONLSink` serializes concurrent writes, creates or tightens files to owner-only mode on POSIX systems, rejects directories and symbolic links, and makes `Close` idempotent. Each line is an independent `AuditEvent` containing metadata and hashes rather than arguments or results.
110+
111+
The guard attaches bounded, low-cardinality OpenTelemetry attributes for the decision, blocked state, risk level, rule IDs, request ID, and scan duration. Applications can also drop the framework's raw tool argument/result span attributes with the telemetry span-attribute policy.
112+
113+
## Redaction
114+
115+
Final-result redaction recursively handles strings, byte slices, maps, slices, arrays, and JSON-serializable structs. Sensitive keys such as `api_key`, `password`, `private_key`, and `session_token` are redacted even when their values are short. Common access tokens, bearer credentials, cloud keys, assignments, and multi-line private keys are detected by value.
116+
117+
Do not treat redaction as authorization. Inputs containing secret material are denied; output redaction is the last containment layer for tools that unexpectedly return a secret.
118+
119+
## Verification
120+
121+
From the repository root:
122+
123+
```bash
124+
go test ./tool/safety ./tool/codeexec ./tool/hostexec ./tool/workspaceexec
125+
go test ./internal/flow/processor ./graph ./telemetry/trace
126+
go test -race ./tool/safety
127+
go test -run '^$' -bench BenchmarkGuardScan500 ./tool/safety
128+
```
129+
130+
Run the public, scan-only acceptance set from `examples`:
131+
132+
```bash
133+
go test ./tool_safety_guard
134+
go run ./tool_safety_guard -policy ./tool_safety_guard/tool_safety_policy.yaml -output-dir ./tool_safety_guard/output
135+
```
136+
137+
Platform-specific executor tests should run on their native operating systems. A Linux container or CI runner is recommended for the repository's Unix-oriented integration tests.
138+
139+
## Limitations
140+
141+
- Static scanning is a policy boundary, not a full shell interpreter or sandbox. Use isolated runtimes and least privilege as independent layers.
142+
- `ask` means “do not execute yet.” The host application owns the user-approval interaction and may retry only after approval.
143+
- Third-party code executors receive a deadline and `codeexecutor.ExecutionLimits` and must honor them while running. The wrapper still caps their returned result, but cannot retroactively make an executor that ignores context cancellation stop work.
144+
- Audit files protect integrity and local confidentiality through restrictive access, but applications remain responsible for retention, rotation, and centralized transport.

0 commit comments

Comments
 (0)