reporting: front-end technique & intent scan report views

AGENTS.md

@@ -23,13 +23,13 @@ gh pr list --repo nvidia/garak --state open --search "<short area keywords>"
 - If an open PR already addresses the same fix, do not open another.
 - If your approach is materially different, explain the difference in the issue.
 
-### No low-value busywork PRs
+### Issues to avoid
 
-Do not open one-off PRs for tiny edits (single typo, isolated style change, one mutable default, etc.). Mechanical cleanups are acceptable only when bundled with substantive work.
+If an issue description is short (e.g. three or fewer sentences), ask for clarification in the issue comments before proceeding.
 
-### Cohesive PRs
+Avoid issues labelled "preliminary" or "needs more information".
 
-Each PR should have a clear focus. Only update files related to the topic of that PR.
+Avoid bug issues that have no assignment or are not labelled `bug-verified`, unless you can build a working test case confirming the bug that does not conflict with project tests.
 
 ### Accountability
 
@@ -39,14 +39,32 @@ Each PR should have a clear focus. Only update files related to the topic of tha
     - Why this is not duplicating an existing PR.
     - Test commands run and results.
     - Clear statement that AI assistance was used.
+- Only add an SPDX header when directly importing code from elsewhere under license that permits this and is compatible with the project.
+
+### PR style
+
+Don't add issue numbers in PR titles; it's OK to include these in the description.
+
+### No low-value busywork PRs
+
+Do not open one-off PRs for tiny edits (single typo, isolated style change, one mutable default, etc.). Mechanical cleanups are acceptable only when bundled with substantive work.
+
+### Cohesive PRs
+
+Each PR should have a clear focus. Only update files related to the topic of that PR.
 
 ### Fail-closed behavior
 
 If work is duplicate/trivial busywork, **do not proceed**. Return a short explanation of what is missing.
 
 ### Project Guides
 
-Check the docs on "Contributing" and "Extending", in `docs/source`, and follow these.
+Follow the documentation on contributing too and extending garak. See:
+
+* For selecting contributions: `docs/source/contributing.rst`
+* For writing code: `docs/source/extending.rst`
+* When writing a probe: `docs/source/extending.probe.rst`
+* When writing a generator: `docs/source/extending.generator.rst`
 
 ### Commit messages
 
@@ -65,6 +83,7 @@ Signed-off-by: Your Name <your.email@example.com>
 ## Development requirements
 
 ### Coding guide
+- Follow the project guide docs linked above.
 - Always avoid adding new dependencies. Use the `extra_dependency_names` functionality if essential.
 - Keep documentation of garak architecture in the docs/ dir up to date - though use docstrings in the first instance if possible.
 - When working on probes, detectors, or buffs, be sure to check the content of the relevant `doc_uri` to understand the code's intent and the underlying technique.
@@ -88,6 +107,7 @@ Signed-off-by: Your Name <your.email@example.com>
 - Don't add tests for functionality already covered by tests of parent classes.
 - Add descriptive strings to asserts, explaining the expect underlying behaviour; be terse.
 - Check that tests work. If `pytest` or other project dependencies are not available, the environment has not been set up correctly; give the user this problem.
+- Re-use/parametrize tests where appropriate.
 
 ### Code primitives
 - Avoid updating `attempt` or any base classes (`probes.base.*`, `generators.base.*`, `detectors.base.*`) frivolously.

CONTRIBUTING.md

@@ -7,7 +7,7 @@ All types of contributions are encouraged and valued. See the [Table of Contents
 
 And if you like the project, but just don't have time to contribute, that's fine. There are other easy ways to support the project and show your appreciation, which we would also be very happy about:
 - Star the project
-- Post about it on LinkedIn
+- Post about it on your favorite social media platform
 - Tweet about it
 - Refer this project in your project's readme
 - Mention the project at local meetups and tell your friends/colleagues
@@ -121,26 +121,6 @@ Enhancement suggestions are tracked as [GitHub issues](https://github.com/NVIDIA
 So you'd like to send us some code? Wonderful! Check out our [guide to contributing garak code](https://reference.garak.ai/en/latest/contributing.html).
 
 Please be mindful of the risk of harm involved in publishing exploits. Only responsibly disclosed vulnerabilities are welcome in garak. OWASP maintain a great guide to [vulnerability disclosure](https://cheatsheetseries.owasp.org/cheatsheets/Vulnerability_Disclosure_Cheat_Sheet.html), which you should check out when contributing probes or data.
-<!-- TODO
-include Setup of env, IDE and typical getting started instructions?
 
--->
-
-<!-- ### Improving The Documentation -->
-<!-- TODO
-Updating, improving and correcting the documentation
-
--->
-
-<!-- ## Styleguides -->
-<!-- ### Commit Messages -->
-<!-- TODO
-
--->
-
-<!-- ## Join The Project Team -->
-<!-- TODO -->
-
-<!-- omit in toc -->
 ## Attribution
 This guide is based on the **contributing-gen**. [Make your own](https://github.com/bttger/contributing-gen)!

docs/requirements-docs.txt

@@ -1,4 +1,4 @@
 sphinx
-sphinx-rtd-theme
+sphinx-rtd-theme>=3.1.0
 sphinx-github-style
 sphinx-reredirects

docs/source/contributing.rst

@@ -26,8 +26,9 @@ If you're going to contribute, it's a really good idea to reach out, so you have
 There are a number of ways you can reach out to us:
 
 * GitHub discussions: `<https://github.com/NVIDIA/garak/discussions>`_
-* Twitter: `<https://twitter.com/garak_llm>`_
 * Discord: `<https://discord.gg/uVch4puUCs>`_
+* LinkedIn: `<https://www.linkedin.com/company/garakllm/>`_
+* X: `<https://x.com/garak_llm>`_
 
 We'd love to help, and we're always interested to hear how you're using garak.
 

docs/source/extending.probe.rst

@@ -6,6 +6,15 @@ This document covers the key points of how to develop a new probe.
 
 For information on how to contribute to garak, and how we build and manage our code, see :doc:`extending`.
 
+Scope
+*****
+
+Garak targets can be attacked in many different ways. That makes for an almost unlimited number of possible probes. However, each probe the garak maintainers accept takes time, through careful review and continuous code management; and impacts garak users in inference time and cost every time they run it. Therefore, care needs to be applied when selecting new probes.
+
+Probes that offer novel, demonstrated, substantial function are in scope.
+
+Novelty is defined by how similar a probe is to garak's existing probes. For a probe to be demonstrated, there should be a research article or experiments showing that the probe works and unlocks something somewhere that wasn't unlocked without it. For substance, the probe should generate a reasonable number of prompts; see `Substance` below.
+
 Naming
 ******
 

garak-report/src/components/IntentsView.tsx

@@ -0,0 +1,64 @@
+/**
+ * @file IntentsView.tsx
+ * @description Intent-centric T&I view. Rows are intents, columns are the
+ *              `demon:*` techniques that exercised them; each cell is the
+ *              pooled pass rate. Mirrors the DetectorsView/ProbesChart panel
+ *              structure.
+ * @module components
+ *
+ * @copyright NVIDIA Corporation 2023-2026
+ * @license Apache-2.0
+ */
+
+import { Panel, Stack, Flex, Text, Badge } from "@kui/react";
+import type { IntentsViewProps } from "../types/TechniqueIntent";
+import useTechniqueIntent from "../hooks/useTechniqueIntent";
+import useSeverityColor from "../hooks/useSeverityColor";
+import TechniqueIntentMatrix from "./TechniqueIntentMatrix";
+
+/**
+ * Panel showing the technique×intent matrix from an intent-first perspective.
+ *
+ * @param props.matrix - The `technique_intent_matrix` digest field
+ * @returns Intent analysis panel
+ */
+const IntentsView = ({ matrix }: IntentsViewProps) => {
+  const { intents, techniques } = useTechniqueIntent(matrix);
+  const { getSeverityColorByLevel } = useSeverityColor();
+
+  const techniqueNames = techniques.map((t) => t.technique_name);
+  const cells = Object.fromEntries(intents.map((i) => [i.intent_name, i.cells]));
+
+  return (
+    <Panel>
+      <Stack gap="density-xl">
+        <Stack gap="density-md">
+          <Flex gap="density-md" align="center">
+            <Text kind="title/lg">Intents</Text>
+            <Badge color="gray" kind="outline">
+              {intents.length} intent{intents.length === 1 ? "" : "s"}
+            </Badge>
+            <Badge color="gray" kind="outline">
+              {techniqueNames.length} technique{techniqueNames.length === 1 ? "" : "s"}
+            </Badge>
+          </Flex>
+          <Text kind="body/regular/md" style={{ color: "var(--color-tk-400)" }}>
+            Pass rate per intent across the techniques used to elicit it. Higher
+            is safer; blank cells were not exercised.
+          </Text>
+        </Stack>
+
+        <TechniqueIntentMatrix
+          rowLabels={intents.map((i) => i.intent_name)}
+          colLabels={techniqueNames}
+          cells={cells}
+          getColor={getSeverityColorByLevel}
+          rowAxisLabel="Intent"
+          colAxisLabel="Technique"
+        />
+      </Stack>
+    </Panel>
+  );
+};
+
+export default IntentsView;

garak-report/src/components/TechniqueIntentMatrix.tsx

@@ -0,0 +1,143 @@
+/**
+ * @file TechniqueIntentMatrix.tsx
+ * @description Heatmap-style grid rendering the technique×intent pass-rate
+ *              matrix. Rows are techniques (or intents, when transposed),
+ *              columns are the opposite axis; each cell shows the pooled pass
+ *              rate. Shared by TechniquesView and IntentsView.
+ * @module components
+ *
+ * @copyright NVIDIA Corporation 2023-2026
+ * @license Apache-2.0
+ */
+
+import { Stack, Flex, Text, Tooltip } from "@kui/react";
+import type { TechniqueIntentCell } from "../types/TechniqueIntent";
+
+/**
+ * Map a pass rate (0..1, higher = safer) to a DEFCON severity level (1 worst,
+ * 5 best) so cells reuse the existing severity palette. Mirrors the bucketing
+ * used for probe scores elsewhere in the report.
+ */
+const scoreToSeverity = (score: number | null): number => {
+  if (score == null) return 0; // unevaluated
+  if (score >= 0.9) return 5;
+  if (score >= 0.7) return 4;
+  if (score >= 0.5) return 3;
+  if (score >= 0.3) return 2;
+  return 1;
+};
+
+const formatScore = (score: number | null): string =>
+  score == null ? "—" : `${Math.round(score * 100)}%`;
+
+/** Props for the matrix grid. */
+export type TechniqueIntentMatrixProps = {
+  /** Row labels (e.g. technique tags or intent ids) */
+  rowLabels: string[];
+  /** Column labels (the opposite axis) */
+  colLabels: string[];
+  /** cell lookup: cells[rowLabel]?.[colLabel] */
+  cells: Record<string, Record<string, TechniqueIntentCell>>;
+  /** Color resolver for a severity level (from useSeverityColor) */
+  getColor: (severity: number) => string;
+  /** Human label for the row axis, shown in the corner cell */
+  rowAxisLabel: string;
+  /** Human label for the column axis */
+  colAxisLabel: string;
+};
+
+/**
+ * Render a technique×intent pass-rate heatmap. Empty (technique, intent)
+ * pairings — where no probe contributed — render as a muted dash.
+ */
+const TechniqueIntentMatrix = ({
+  rowLabels,
+  colLabels,
+  cells,
+  getColor,
+  rowAxisLabel,
+  colAxisLabel,
+}: TechniqueIntentMatrixProps) => {
+  if (rowLabels.length === 0 || colLabels.length === 0) {
+    return (
+      <Text kind="body/regular/md" style={{ color: "var(--color-tk-400)" }}>
+        No technique/intent data in this report.
+      </Text>
+    );
+  }
+
+  // CSS grid: one label column + one column per intent.
+  const templateColumns = `minmax(7rem, max-content) repeat(${colLabels.length}, minmax(3.5rem, 1fr))`;
+
+  return (
+    <Stack gap="density-sm">
+      <div
+        role="table"
+        aria-label={`${rowAxisLabel} by ${colAxisLabel} pass-rate matrix`}
+        style={{ display: "grid", gridTemplateColumns: templateColumns, gap: "2px" }}
+      >
+        {/* Header row */}
+        <div role="columnheader" style={{ padding: "4px 8px" }}>
+          <Text kind="label/sm" style={{ color: "var(--color-tk-400)" }}>
+            {rowAxisLabel} \ {colAxisLabel}
+          </Text>
+        </div>
+        {colLabels.map((col) => (
+          <div key={`h-${col}`} role="columnheader" style={{ padding: "4px 8px", textAlign: "center" }}>
+            <Text kind="label/sm">{col}</Text>
+          </div>
+        ))}
+
+        {/* Body rows */}
+        {rowLabels.map((row) => (
+          <Flex key={`r-${row}`} style={{ display: "contents" }}>
+            <div role="rowheader" style={{ padding: "4px 8px", whiteSpace: "nowrap" }}>
+              <Text kind="body/regular/sm">{row}</Text>
+            </div>
+            {colLabels.map((col) => {
+              const cell = cells[row]?.[col];
+              const score = cell ? cell.score : null;
+              const severity = scoreToSeverity(score);
+              const bg = cell ? getColor(severity) : "transparent";
+              const label = formatScore(score);
+              const cellEl = (
+                <div
+                  role="cell"
+                  data-testid={`ti-cell-${row}-${col}`}
+                  style={{
+                    padding: "6px 8px",
+                    textAlign: "center",
+                    backgroundColor: bg,
+                    borderRadius: "2px",
+                    minHeight: "1.75rem",
+                    opacity: cell ? 1 : 0.35,
+                  }}
+                >
+                  <Text kind="body/regular/sm">{label}</Text>
+                </div>
+              );
+              if (!cell) {
+                return <div key={`c-${row}-${col}`}>{cellEl}</div>;
+              }
+              return (
+                <Tooltip
+                  key={`c-${row}-${col}`}
+                  content={
+                    `${row} × ${col}: ${label} pass rate ` +
+                    `(${cell.passed}/${cell.total_evaluated} evaluations` +
+                    `${cell.nones ? `, ${cell.nones} unscoreable` : ""}, ` +
+                    `${cell.n_detectors} detector${cell.n_detectors === 1 ? "" : "s"})`
+                  }
+                >
+                  {cellEl}
+                </Tooltip>
+              );
+            })}
+          </Flex>
+        ))}
+      </div>
+    </Stack>
+  );
+};
+
+export default TechniqueIntentMatrix;