You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
docs(blog): rework triage diagnosis and list skills by group
False positives are now diagnosed into three documented categories
(rule, approximation, engine issue) with deferred test-first fixes,
and the Get started section lists all fifteen skills in semantic
groups.
Copy file name to clipboardExpand all lines: src/content/blog/appsec-agent.mdx
+30-8Lines changed: 30 additions & 8 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -1,6 +1,6 @@
1
1
---
2
2
title: "Distilling Frontier Model Security Reviews into a Cheap Deterministic Scan"
3
-
description: "We're releasing open-source skills that pair an LLM agent with OpenTaint's taint engine. The agent maps the attack surface, models the missing library methods, writes project-specific rules, and can confirm each finding with a real exploit — then the deterministic engine re-scans for the price of CPU, on every future commit."
3
+
description: "We're releasing skills that pair an LLM agent with OpenTaint's taint engine. The agent maps the attack surface, models the missing library methods, writes project-specific rules, and can confirm each finding with a real exploit — then the deterministic engine re-scans for the price of CPU, on every future commit."
4
4
date: "2026-06-10"
5
5
keywords:
6
6
- "llm security agent"
@@ -18,7 +18,7 @@ import Mermaid from "@/components/astro/Mermaid.astro";
18
18
19
19
Frontier models can now genuinely review code for security. An LLM agent reads it like a human reviewer: it follows intent, recognizes patterns no rule anticipated, and reasons about whether a flow is actually dangerous in context. But you rent that judgment rather than own it — the agent burns tokens on every file, gives a different answer each time you run it, and can never promise it looked everywhere. A taint engine — a static analyzer that traces untrusted data through code — is the mirror image: it cannot invent anything, but whatever it is told it applies the same way every time, across the whole codebase, for the price of CPU. Treat them as alternatives and you inherit the weaknesses of whichever one you picked.
20
20
21
-
Today we're releasing a set of [open-source skills](https://github.com/seqra/opentaint/tree/main/skills) that run them as one loop instead. The skills plus OpenTaint — our taint-analysis engine — form a harness for the model: the agent discovers, the engine enforces. The expensive, probabilistic judgment happens once and is distilled into artifacts the engine reuses forever. A frontier-model security review becomes a deterministic scan that costs CPU, not tokens.
21
+
Today we're releasing a set of [skills](https://github.com/seqra/opentaint/tree/main/skills) that run them as one loop instead. The skills plus OpenTaint — our taint-analysis engine — form a harness for the model: the agent discovers, the engine enforces. The expensive, probabilistic judgment happens once and is distilled into artifacts the engine reuses forever. A frontier-model security review becomes a deterministic scan that costs CPU, not tokens.
22
22
23
23
## Division of labor
24
24
@@ -121,20 +121,31 @@ Once the scan completes, every finding goes to the agent for triage: it reads th
*Stage 2 — **Triage**. A successful exploit confirms the finding, and a failed one sends the agent to diagnose what misled the engine.*
144
+
*Stage 2 — **Triage**. A successful exploit confirms the finding. A failed one sends the agent to diagnose what misled the engine and document the cause.*
145
+
146
+
A false positive is not just discarded. The agent diagnoses *why* the engine was misled and documents the cause alongside the verdict, in one of three categories. A **rule issue**: the pattern is too broad — it treats data as untrusted that never was, or misses the check that neutralizes it. An **approximation issue**: a model propagates data the real method does not. An **engine issue**: the trace is wrong and no artifact caused it — the agent writes that up as a reproducible report for the engine maintainers.
136
147
137
-
When the agent can name *why* a finding is false — a neutralizing check it had overlooked, an approximation that propagates data the real method doesn't — that diagnosis is applied back into the artifacts:the overfiring rules get tuned, the approximation gets corrected. Because those artifacts are global, the fix holds on every future scan. Saturation closed the underapproximation gap in Stage 1. This triage closes the overapproximation one. They attack the two ways a model can be wrong: saturation fills in propagation that was missing, triage trims propagations and rules that were too broad.
148
+
The rule and approximation diagnoses can be picked up later, and each can be expressed as a minimal reproducible test: the false positive distilled into a small example the artifact must not match. The same skills that author rules and approximations also tune them — write that example as a should-not-fire test, then adjust the rule or correct the approximation until it passes and the should-fire tests still do. Because a rule or approximation applies to the whole codebase on every scan, the fix removes the entire class of false positives, not just the finding that exposed it. Together with Stage 1's saturation, which filled in propagation the engine was missing, this closes the other direction a model can be wrong: rules and propagations that were too broad.
138
149
139
150
## The payoff
140
151
@@ -166,7 +177,18 @@ OpenTaint is open source, Apache 2.0 licensed. It analyzes Java and Kotlin today
166
177
npx skills add https://github.com/seqra/opentaint
167
178
```
168
179
169
-
The skill offers to install the engine itself if it is missing. To run the workflow, open your coding agent in the project and ask it to find vulnerabilities. The skill asks two questions before touching anything, then works the pipeline on its own:
180
+
That installs fifteen skills. One is the workflow itself, and the rest are the steps it drives — each usable on its own:
181
+
182
+
- **Pipeline** — `appsec-agent` runs the whole loop end to end, from build to confirmed findings.
183
+
- **Build and scan** — `build-project` builds the project into the model the engine analyzes. `run-scan` runs the deterministic scan and produces the report.
184
+
- **Attack surface** — `triage-dependencies` marks which dependencies could introduce sources or sinks. `discover-attack-surface` drills into a flagged package for the exact methods the project uses.
185
+
- **Rules** — `create-rule` authors a detection rule test-first, and fixes one that misfires or misses. `assemble-lib-rules` merges the per-package rules with the built-ins into the project-level patterns.
186
+
- **Approximations** — `analyze-external-methods` sorts the methods where traces died and decides what to approximate. `create-pass-through-approximation` models a method whose propagation is plain copying. `create-dataflow-approximation` models one a copy cannot express.
187
+
- **Artifact tests** — `create-test-project` builds the annotated should-fire / should-not-fire samples that rules and approximations are verified against. `debug-rule` traces where taint is dropped when one misbehaves.
188
+
- **Triage** — `analyze-findings` splits a rule's findings into distinct vulnerabilities and rules each true or false. `generate-poc` reproduces a finding against the running application.
189
+
- **Engine feedback** — `report-analyzer-issue` turns a confirmed engine-side diagnosis into a reproducible report, optionally a GitHub issue.
190
+
191
+
`appsec-agent` offers to install the engine itself if it is missing. To run the workflow, open your coding agent in the project and ask it to find vulnerabilities. The skill asks two questions before touching anything, then works the pipeline on its own:
170
192
171
193
- **Scan depth** — the Lite / Normal / Deep ladder from above.
172
194
- **Exploit confirmation** — whether to confirm findings with PoCs. Dynamic triage launches throwaway local instances of the application under test and tears them down at the end.
0 commit comments