Skip to content

Add 2 tools: UltraProbe (Red Teaming) + prompt-defense-audit (Guardrails)#45

Open
ppcvote wants to merge 1 commit into
ProjectRecon:mainfrom
ppcvote:add-ppcvote-tools
Open

Add 2 tools: UltraProbe (Red Teaming) + prompt-defense-audit (Guardrails)#45
ppcvote wants to merge 1 commit into
ProjectRecon:mainfrom
ppcvote:add-ppcvote-tools

Conversation

@ppcvote

@ppcvote ppcvote commented May 29, 2026

Copy link
Copy Markdown

Adds 2 tools to existing sections.

⚔️ Red Teaming & Vulnerability Scanners

  • UltraProbe — Free hosted AI prompt audit. Scans a system prompt against 17 OWASP-LLM-Top-10-aligned attack vectors (incl. 5 agent-specific ones: encoding-aware injection, function semantic immutability, memory provenance, cross-agent auth, transaction guardrails) and returns an A–F grade with per-vector evidence. Hosted at ultralab.tw/probe; CLI also published as ultraprobe on npm. MIT.

🚧 Guardrails & Compliance

  • prompt-defense-audit — Deterministic system-prompt scanner (npm). Same 17 vectors as the UltraProbe UI, plus 12 Unicode smuggling categories (Tag chars / Math alphanumeric / BiDi / Variation Selectors) and an output scanner with rules for JSON schema injection, OCR / Whisper / PDF modality marker hijack, tool-call argument injection, and SVG foreignObject HTML. Pure regex, <5ms, zero AI cost. MIT.

Both are MIT-licensed and actively maintained. Both align well with the agent-security focus of this list — the 5 agent-specific vectors above were derived from a structured analysis of six documented crypto AI agent incidents (Freysa, Lobstar Wilde, Grok×Bankrbot Morse, ElizaOS memory injection, etc.). Happy to adjust descriptions if shorter / different placement is preferred.

@secxena

secxena commented Jun 12, 2026

Copy link
Copy Markdown
Contributor

Thanks for the submission. The first link (ppcvote/ultraprobe) returns a 404, so we cannot review UltraProbe — could you update the PR with a working public repo? prompt-defense-audit does exist but is still very early in terms of adoption. We are passing for now; happy to revisit once the links work and the projects have built more traction.

@ppcvote

ppcvote commented Jun 21, 2026

Copy link
Copy Markdown
Author

@secxena thanks for the candid pass and for flagging the 404 — that one's on me. ppcvote/ultraprobe was a private staging repo at the time of your review; it went public on 2026-06-15 and is now live at https://github.com/ppcvote/ultraprobe (CLI source, MIT) — npm package is at ultraprobe@2.1.1.

On the "still very early in terms of adoption" call — fair, and worth answering with adoption signal beyond star count. Three datapoints from the last ~10 weeks that may help when you next sweep the list:

  • Cisco AI Defense mcp-scanner PR #146 — merged 2026-04-06. The PromptDefenseAnalyzer module is the same rule set the UltraProbe scanner uses.
  • Microsoft Agent Governance Toolkit PR #854 — merged 2026-04-08. PromptDefenseEvaluator upstream.
  • prompt-defense-audit is at v1.5.0 on npm with 121 tests in CI (matrix Node 20/22). Same 17 system-prompt vectors + 12 Unicode smuggling categories + 14 output-side regex rules the hosted UltraProbe UI uses.

If that adoption shape meets the bar, happy to re-open this PR or wait for your next intake window — your call entirely. Either way, thank you for the clear feedback that the link was broken, that's the kind of review pass that actually keeps the list trustworthy.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants