|
| 1 | +# Tool Safety Guard |
| 2 | + |
| 3 | +Package `tool/safety` provides an opt-in, fail-closed guard for model-initiated tool calls. It scans the final execution input before a tool runs, returns a structured permission decision, emits metadata-only audit events, and recursively removes secrets from the final result after callbacks. |
| 4 | + |
| 5 | +Existing applications are unchanged until they configure a guard. |
| 6 | + |
| 7 | +## Security contract |
| 8 | + |
| 9 | +The guard addresses seven execution risks: destructive commands, sensitive paths, network access, shell bypasses, host/interactive execution, dependency changes, and resource abuse. Secret exposure is an always-on input/output protection and cannot be disabled by policy. |
| 10 | + |
| 11 | +The key ordering is: |
| 12 | + |
| 13 | +1. before-tool callbacks finish modifying arguments; |
| 14 | +2. the permission guard scans the effective arguments; |
| 15 | +3. only `allow` reaches the executor; |
| 16 | +4. plugins and after-tool callbacks finish modifying the result; |
| 17 | +5. the final sanitizer recursively redacts the effective result; |
| 18 | +6. only the sanitized result reaches events, model messages, and tool-result telemetry. |
| 19 | + |
| 20 | +`deny` outranks `ask`, which outranks `allow`. Invalid policy or audit state never silently weakens the configured contract. A failed audit write or final sanitization returns an error and suppresses the unsafe result. |
| 21 | + |
| 22 | +## Create a guard |
| 23 | + |
| 24 | +```go |
| 25 | +policyData, err := os.ReadFile("policy.yaml") |
| 26 | +if err != nil { |
| 27 | + return err |
| 28 | +} |
| 29 | +policy, err := safety.ParsePolicy(policyData, safety.PolicyFormatAuto) |
| 30 | +if err != nil { |
| 31 | + return err |
| 32 | +} |
| 33 | +sink, err := safety.NewJSONLSink("tool-audit.jsonl") |
| 34 | +if err != nil { |
| 35 | + return err |
| 36 | +} |
| 37 | +defer sink.Close() |
| 38 | +guard, err := safety.NewGuard(policy, safety.WithAuditSink(sink)) |
| 39 | +if err != nil { |
| 40 | + return err |
| 41 | +} |
| 42 | +``` |
| 43 | + |
| 44 | +`NewDefaultGuard` enables all built-in detections with a default `allow` action. `Reload` parses and atomically installs a complete policy; a rejected reload leaves the previous snapshot active. |
| 45 | + |
| 46 | +## Framework-wide integration |
| 47 | + |
| 48 | +Pass the guard as the per-run permission policy: |
| 49 | + |
| 50 | +```go |
| 51 | +runOptions := agent.NewRunOptions( |
| 52 | + agent.WithToolPermissionPolicy(guard), |
| 53 | +) |
| 54 | +``` |
| 55 | + |
| 56 | +This route covers standard function-call processing and Graph execution, including wrapped tools and callback-provided custom results. The guard also acts as a final-result sanitizer. |
| 57 | + |
| 58 | +## Direct built-in tool integration |
| 59 | + |
| 60 | +Applications that call a built-in tool directly can opt in at construction: |
| 61 | + |
| 62 | +```go |
| 63 | +workspaceTool := workspaceexec.NewExecTool(exec, |
| 64 | + workspaceexec.WithSafetyGuard(guard), |
| 65 | +) |
| 66 | + |
| 67 | +hostTools := hostexec.NewToolSet( |
| 68 | + hostexec.WithSafetyGuard(guard), |
| 69 | +) |
| 70 | + |
| 71 | +codeTool := codeexec.NewTool(exec, |
| 72 | + codeexec.WithSafetyGuard(guard), |
| 73 | +) |
| 74 | +``` |
| 75 | + |
| 76 | +Use either framework-wide integration or the direct option for a call path. Configuring the same guard in both places is safe but produces two scans and two audit events. |
| 77 | + |
| 78 | +For `codeexec`, scanning occurs after flexible input decoding and language validation, over the code that the executor will actually receive. The wrapper applies the most restrictive direct/invocation timeout and output profile, passes in-flight limits through `codeexecutor.ExecutionLimits`, and caps the combined returned output/file content; the local executor enforces the output budget while the child is running. For interactive write tools, an empty poll remains allowed while non-empty input or newline submission requires approval by default. |
| 79 | + |
| 80 | +## Policy |
| 81 | + |
| 82 | +Policies are versioned and strictly decoded. Unknown fields, duplicate keys, multiple YAML documents, trailing JSON values, invalid actions, invalid domains, and negative limits are rejected. |
| 83 | + |
| 84 | +```yaml |
| 85 | +version: 1 |
| 86 | +default_action: allow |
| 87 | +profiles: |
| 88 | + workspace_exec: |
| 89 | + allowed_commands: [go, git] |
| 90 | + denied_commands: [curl] |
| 91 | + allowed_domains: [api.github.com, "*.corp.example"] |
| 92 | + forbidden_paths: [/etc, ~/.ssh] |
| 93 | + allowed_env: [CI] |
| 94 | + max_timeout: 2m |
| 95 | + max_output_bytes: 1048576 |
| 96 | + allow_host: false |
| 97 | + allow_background: false |
| 98 | + allow_pty: false |
| 99 | +``` |
| 100 | +
|
| 101 | +Domain matching is exact. `*.corp.example` matches `build.corp.example`, but not `corp.example` or `evilcorp.example`. Proxying, destination remapping, external curl/SSH configuration, and SSH forwarding require stronger handling because they can change the effective destination. |
| 102 | + |
| 103 | +Profile timeout and output values are ceilings, not requested defaults. Runtimes that advertise hard-limit support receive and enforce these ceilings; a configured hard limit must not be represented as enforced on an opaque backend that cannot guarantee it. |
| 104 | + |
| 105 | +## Reports, audit, and telemetry |
| 106 | + |
| 107 | +`Report` contains a decision, redacted findings, request digest, duration, risk level, recommendation, backend, and blocked/redacted flags. It never needs the raw request to correlate repeated activity. |
| 108 | + |
| 109 | +`JSONLSink` serializes concurrent writes, creates or tightens files to owner-only mode on POSIX systems, rejects directories and symbolic links, and makes `Close` idempotent. Each line is an independent `AuditEvent` containing metadata and hashes rather than arguments or results. |
| 110 | + |
| 111 | +The guard attaches bounded, low-cardinality OpenTelemetry attributes for the decision, blocked state, risk level, rule IDs, request ID, and scan duration. Applications can also drop the framework's raw tool argument/result span attributes with the telemetry span-attribute policy. |
| 112 | + |
| 113 | +## Redaction |
| 114 | + |
| 115 | +Final-result redaction recursively handles strings, byte slices, maps, slices, arrays, and JSON-serializable structs. Sensitive keys such as `api_key`, `password`, `private_key`, and `session_token` are redacted even when their values are short. Common access tokens, bearer credentials, cloud keys, assignments, and multi-line private keys are detected by value. |
| 116 | + |
| 117 | +Do not treat redaction as authorization. Inputs containing secret material are denied; output redaction is the last containment layer for tools that unexpectedly return a secret. |
| 118 | + |
| 119 | +## Verification |
| 120 | + |
| 121 | +From the repository root: |
| 122 | + |
| 123 | +```bash |
| 124 | +go test ./tool/safety ./tool/codeexec ./tool/hostexec ./tool/workspaceexec |
| 125 | +go test ./internal/flow/processor ./graph ./telemetry/trace |
| 126 | +go test -race ./tool/safety |
| 127 | +go test -run '^$' -bench BenchmarkGuardScan500 ./tool/safety |
| 128 | +``` |
| 129 | + |
| 130 | +Run the public, scan-only acceptance set from `examples`: |
| 131 | + |
| 132 | +```bash |
| 133 | +go test ./tool_safety_guard |
| 134 | +go run ./tool_safety_guard -policy ./tool_safety_guard/tool_safety_policy.yaml -output-dir ./tool_safety_guard/output |
| 135 | +``` |
| 136 | + |
| 137 | +Platform-specific executor tests should run on their native operating systems. A Linux container or CI runner is recommended for the repository's Unix-oriented integration tests. |
| 138 | + |
| 139 | +## Limitations |
| 140 | + |
| 141 | +- Static scanning is a policy boundary, not a full shell interpreter or sandbox. Use isolated runtimes and least privilege as independent layers. |
| 142 | +- `ask` means “do not execute yet.” The host application owns the user-approval interaction and may retry only after approval. |
| 143 | +- Third-party code executors receive a deadline and `codeexecutor.ExecutionLimits` and must honor them while running. The wrapper still caps their returned result, but cannot retroactively make an executor that ignores context cancellation stop work. |
| 144 | +- Audit files protect integrity and local confidentiality through restrictive access, but applications remain responsible for retention, rotation, and centralized transport. |
0 commit comments