Skip to content

Commit 7e7f0f5

Browse files
RoyLinRoyLin
authored andcommitted
docs(readme): document the SAE mechanistic-interpretability tier
Adds the SAE tier to the intro + a dedicated section: white-box (judges the model's residual-stream features, not obfuscable text), confidential (TEE — only features leave, via a3s-power's in-enclave tap), explainable (linear in named drivers). The sae{} ACL block, the LlmActivations routing, and the link to a3s-power's plan.
1 parent 1be9e3c commit 7e7f0f5

1 file changed

Lines changed: 36 additions & 0 deletions

File tree

README.md

Lines changed: 36 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -20,11 +20,16 @@ The three tiers trade cost for depth, so expensive judgment runs only on what ch
2020
| **L1** | deterministic regex rule engine (ACL-configurable) | µs | every event |
2121
| **L2** | a fast LLM classifier (OpenAI-compatible endpoint) | ~100s ms | events L1 escalates |
2222
| **L3** | a deep [a3s-code](https://github.com/AI45Lab/Code) agent with security skills | seconds–minutes | events L2 escalates |
23+
| **SAE** | a Sparse Autoencoder over the model's residual stream, tapped in-TEE by [a3s-power](https://github.com/A3S-Lab/Power) | ~ms | model-output `LlmActivations` events |
2324

2425
L1 catches the unambiguous cases outright and flags the rest; L2 gives a fast second opinion; L3
2526
actually investigates — reading the event in context, considering the attack chain — for the
2627
genuinely hard cases. Each tier is a `Judge`, so the set is swappable and unit-tested.
2728

29+
A fourth, parallel tier — **SAE** — judges a different signal entirely: the model's *own output*,
30+
by its internal features rather than its (obfuscable) text. See
31+
[SAE — mechanistic interpretability](#sae--mechanistic-interpretability-of-model-output).
32+
2833
## How it fits a3s-observer
2934

3035
Sentry is exactly the *"your controller"* piece observer's README leaves to you:
@@ -206,6 +211,37 @@ stronger/different model than L2 — or run without L2 at all. Validated against
206211
an SSH-private-key read → `block` with the agent reasoning *"a generic Python interpreter, not a known
207212
SSH client… key material can be transmitted outbound after being loaded into memory."*
208213

214+
## SAE — mechanistic interpretability of model output
215+
216+
The L1–L3 tiers judge what the agent *did* (observer's syscall / network events). The **SAE tier**
217+
judges what the model *said* — its output — and does it **white-box**: instead of reading the
218+
completion text (which an attacker can base64/cipher-obfuscate), it reads the model's own
219+
**residual-stream features**.
220+
221+
[a3s-power](https://github.com/A3S-Lab/Power) serves the model inside a TEE, taps the residual stream
222+
at one layer, encodes it with a Sparse Autoencoder, and emits **only** the sparse `(feature_id,
223+
activation)` pairs as an `LlmActivations` event — the prompt/completion plaintext never leaves the
224+
enclave. Sentry's `SaeJudge` scores those features against a labeled feature dictionary:
225+
226+
- **white-box** — judges the model's *internal concepts*, so an obfuscated harmful output still lights
227+
its concept feature;
228+
- **confidential** — sees only feature ids / activations, never the text (the model runs in the TEE);
229+
- **explainable** — the score is *linear in named features*, decomposed into ranked drivers
230+
(`exploit-code-synthesis (#8801) → 0.82`), not a second black box.
231+
232+
```hcl
233+
sae { dict = "features.json" escalate_at = 0.3 block_at = 0.6 } # mech-interp tier (optional)
234+
```
235+
236+
The feature dictionary (`feature_id → {concept, category, weight, severity}`) is an offline artifact:
237+
train or adopt an SAE for the served model, probe + label its safety-relevant features, and
238+
causal-validate each label (ablate the feature, confirm the score moves). Model-output events route to
239+
this tier (not the rule chain); an SAE escalation can still defer to the deep L3 agent. The `Decision`
240+
carries the explainability in `explain` (`SaeScore`: per-category scores + ranked drivers) for the
241+
dashboard. Output text has no kernel deny target, so an SAE block rides the enclosing
242+
`ToolExec`/`Egress` action event. a3s-power's side of the chain is planned in
243+
[its `docs/sae-interpretability-plan.md`](https://github.com/A3S-Lab/Power/blob/main/docs/sae-interpretability-plan.md).
244+
209245
## Config (env)
210246

211247
| var | effect |

0 commit comments

Comments
 (0)