Skip to content

Commit 80aaae3

Browse files
committed
docs(blog): rework triage diagnosis and list skills by group
False positives are now diagnosed into three documented categories (rule, approximation, engine issue) with deferred test-first fixes, and the Get started section lists all fifteen skills in semantic groups.
1 parent 979fd89 commit 80aaae3

1 file changed

Lines changed: 30 additions & 8 deletions

File tree

src/content/blog/appsec-agent.mdx

Lines changed: 30 additions & 8 deletions
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,6 @@
11
---
22
title: "Distilling Frontier Model Security Reviews into a Cheap Deterministic Scan"
3-
description: "We're releasing open-source skills that pair an LLM agent with OpenTaint's taint engine. The agent maps the attack surface, models the missing library methods, writes project-specific rules, and can confirm each finding with a real exploit — then the deterministic engine re-scans for the price of CPU, on every future commit."
3+
description: "We're releasing skills that pair an LLM agent with OpenTaint's taint engine. The agent maps the attack surface, models the missing library methods, writes project-specific rules, and can confirm each finding with a real exploit — then the deterministic engine re-scans for the price of CPU, on every future commit."
44
date: "2026-06-10"
55
keywords:
66
- "llm security agent"
@@ -18,7 +18,7 @@ import Mermaid from "@/components/astro/Mermaid.astro";
1818

1919
Frontier models can now genuinely review code for security. An LLM agent reads it like a human reviewer: it follows intent, recognizes patterns no rule anticipated, and reasons about whether a flow is actually dangerous in context. But you rent that judgment rather than own it — the agent burns tokens on every file, gives a different answer each time you run it, and can never promise it looked everywhere. A taint engine — a static analyzer that traces untrusted data through code — is the mirror image: it cannot invent anything, but whatever it is told it applies the same way every time, across the whole codebase, for the price of CPU. Treat them as alternatives and you inherit the weaknesses of whichever one you picked.
2020

21-
Today we're releasing a set of [open-source skills](https://github.com/seqra/opentaint/tree/main/skills) that run them as one loop instead. The skills plus OpenTaint — our taint-analysis engine — form a harness for the model: the agent discovers, the engine enforces. The expensive, probabilistic judgment happens once and is distilled into artifacts the engine reuses forever. A frontier-model security review becomes a deterministic scan that costs CPU, not tokens.
21+
Today we're releasing a set of [skills](https://github.com/seqra/opentaint/tree/main/skills) that run them as one loop instead. The skills plus OpenTaint — our taint-analysis engine — form a harness for the model: the agent discovers, the engine enforces. The expensive, probabilistic judgment happens once and is distilled into artifacts the engine reuses forever. A frontier-model security review becomes a deterministic scan that costs CPU, not tokens.
2222

2323
## Division of labor
2424

@@ -121,20 +121,31 @@ Once the scan completes, every finding goes to the agent for triage: it reads th
121121
triage["Triage candidates"]
122122
poc{"Generate + run PoC"}
123123
confirmed["Confirmed vulnerability"]
124-
fix["Diagnose: tune rule / fix approximation"]
124+
diag["Diagnose + document"]
125+
rule["Rule issue"]
126+
approx["Approximation issue"]
127+
engine["Engine issue"]
125128

126129
triage --> poc
127130
poc -->|"exploit succeeds"| confirmed
128-
poc -->|"no exploit found"| fix
131+
poc -->|"no exploit found"| diag
132+
diag --> rule
133+
diag --> approx
134+
diag --> engine
129135

130136
style triage fill:#fef3c7,stroke:#f59e0b,color:#92400e
131137
style poc fill:#fef3c7,stroke:#f59e0b,color:#92400e
132-
style fix fill:#fef3c7,stroke:#f59e0b,color:#92400e
138+
style diag fill:#fef3c7,stroke:#f59e0b,color:#92400e
139+
style rule fill:#fef3c7,stroke:#f59e0b,color:#92400e
140+
style approx fill:#fef3c7,stroke:#f59e0b,color:#92400e
141+
style engine fill:#dbeafe,stroke:#3b82f6,color:#1e40af
133142
style confirmed fill:#fee2e2,stroke:#ef4444,color:#991b1b`} />
134143

135-
*Stage 2 — **Triage**. A successful exploit confirms the finding, and a failed one sends the agent to diagnose what misled the engine.*
144+
*Stage 2 — **Triage**. A successful exploit confirms the finding. A failed one sends the agent to diagnose what misled the engine and document the cause.*
145+
146+
A false positive is not just discarded. The agent diagnoses *why* the engine was misled and documents the cause alongside the verdict, in one of three categories. A **rule issue**: the pattern is too broad — it treats data as untrusted that never was, or misses the check that neutralizes it. An **approximation issue**: a model propagates data the real method does not. An **engine issue**: the trace is wrong and no artifact caused it — the agent writes that up as a reproducible report for the engine maintainers.
136147

137-
When the agent can name *why* a finding is false — a neutralizing check it had overlooked, an approximation that propagates data the real method doesn't — that diagnosis is applied back into the artifacts: the overfiring rules get tuned, the approximation gets corrected. Because those artifacts are global, the fix holds on every future scan. Saturation closed the underapproximation gap in Stage 1. This triage closes the overapproximation one. They attack the two ways a model can be wrong: saturation fills in propagation that was missing, triage trims propagations and rules that were too broad.
148+
The rule and approximation diagnoses can be picked up later, and each can be expressed as a minimal reproducible test: the false positive distilled into a small example the artifact must not match. The same skills that author rules and approximations also tune them — write that example as a should-not-fire test, then adjust the rule or correct the approximation until it passes and the should-fire tests still do. Because a rule or approximation applies to the whole codebase on every scan, the fix removes the entire class of false positives, not just the finding that exposed it. Together with Stage 1's saturation, which filled in propagation the engine was missing, this closes the other direction a model can be wrong: rules and propagations that were too broad.
138149

139150
## The payoff
140151

@@ -166,7 +177,18 @@ OpenTaint is open source, Apache 2.0 licensed. It analyzes Java and Kotlin today
166177
npx skills add https://github.com/seqra/opentaint
167178
```
168179

169-
The skill offers to install the engine itself if it is missing. To run the workflow, open your coding agent in the project and ask it to find vulnerabilities. The skill asks two questions before touching anything, then works the pipeline on its own:
180+
That installs fifteen skills. One is the workflow itself, and the rest are the steps it drives — each usable on its own:
181+
182+
- **Pipeline** — `appsec-agent` runs the whole loop end to end, from build to confirmed findings.
183+
- **Build and scan** — `build-project` builds the project into the model the engine analyzes. `run-scan` runs the deterministic scan and produces the report.
184+
- **Attack surface** — `triage-dependencies` marks which dependencies could introduce sources or sinks. `discover-attack-surface` drills into a flagged package for the exact methods the project uses.
185+
- **Rules** — `create-rule` authors a detection rule test-first, and fixes one that misfires or misses. `assemble-lib-rules` merges the per-package rules with the built-ins into the project-level patterns.
186+
- **Approximations** — `analyze-external-methods` sorts the methods where traces died and decides what to approximate. `create-pass-through-approximation` models a method whose propagation is plain copying. `create-dataflow-approximation` models one a copy cannot express.
187+
- **Artifact tests** — `create-test-project` builds the annotated should-fire / should-not-fire samples that rules and approximations are verified against. `debug-rule` traces where taint is dropped when one misbehaves.
188+
- **Triage** — `analyze-findings` splits a rule's findings into distinct vulnerabilities and rules each true or false. `generate-poc` reproduces a finding against the running application.
189+
- **Engine feedback** — `report-analyzer-issue` turns a confirmed engine-side diagnosis into a reproducible report, optionally a GitHub issue.
190+
191+
`appsec-agent` offers to install the engine itself if it is missing. To run the workflow, open your coding agent in the project and ask it to find vulnerabilities. The skill asks two questions before touching anything, then works the pipeline on its own:
170192

171193
- **Scan depth** — the Lite / Normal / Deep ladder from above.
172194
- **Exploit confirmation** — whether to confirm findings with PoCs. Dynamic triage launches throwaway local instances of the application under test and tears them down at the end.

0 commit comments

Comments
 (0)