Skip to content

Commit f4f6b3c

Browse files
WilliamBerryiiiBill Berry
andauthored
feat(agents): security-planner SSSC parity (#1642)
## Summary Brings the `security-planner` agent, its supporting instructions, and the shared RAI capture-coaching guidance up to the same phase-gate, state, and question-cadence parity established by the `sssc-planner` work in PR #1497. Bundles the planner-startup, cadence Rule 5 ordering, risk-grid grammar, and state-schema test suites that guard those conventions going forward. ## Stacking - **Base:** `stack/security-state-schema` (PR #1638, "PR A") - **Upstream:** PR #1497 (sssc-planner) - **Siblings:** PR #1639 (PR C — Disclaimer/Attribution Protocol), PR #1640 (PR D) - **Do not merge** until the PR #1497 → PR #1638 cascade completes. ## Changes **Modified (10):** - `.github/agents/security/security-planner.agent.md` - `.github/instructions/security/identity.instructions.md` - `.github/instructions/security/security-model.instructions.md` - `.github/instructions/security/backlog-handoff.instructions.md` - `.github/instructions/rai-planning/rai-capture-coaching.instructions.md` - `.github/prompts/security/security-capture.prompt.md` - `.github/prompts/security/security-plan-from-prd.prompt.md` - `plugins/hve-core-all/README.md` (auto-regen) - `plugins/project-planning/README.md` (auto-regen) - `plugins/security/README.md` (auto-regen) **Added (5 Pester suites):** - `scripts/tests/linting/Test-PlannerStateSchema.Tests.ps1` — 8 tests - `scripts/tests/linting/Test-PlannerStateSchemas.Tests.ps1` — 4 tests - `scripts/tests/linting/Test-CadenceRule5Ordering.Tests.ps1` — 2 tests - `scripts/tests/linting/Test-PlannerStartupBlocks.Tests.ps1` — 6 tests - `scripts/tests/linting/Test-RiskGridGrammar.Tests.ps1` — 7 tests ## Validation | Check | Result | | --- | --- | | `npm run plugin:validate` | ✅ 13 collections | | `npm run lint:md` | ✅ 211 files, 0 errors | | `npm run lint:frontmatter` | ✅ 541 files, 0 errors | | `npm run lint:md-links` | ✅ | | `npm run lint:ps` | ✅ | | `Test-PlannerStateSchema` | ✅ 8/8 | | `Test-PlannerStateSchemas` | ✅ 4/4 | | `Test-CadenceRule5Ordering` | ✅ 2/2 | | `Test-PlannerStartupBlocks` | ✅ 6/6 | | `Test-RiskGridGrammar` | ✅ 7/7 | ## Notes for Reviewers 1. **`disclaimerShownAt: null` in identity state schema** — Adds one line to the inline state schema in `identity.instructions.md` so the canonical state shape declares the field with a default. Full Disclaimer/Attribution Protocol prose that consumes the field remains in PR C (#1639). Three-way overlap with PR #1639 on this file is expected. 2. **`Test-PlannerStateSchemas.Tests.ps1` assertion direction** — Authored draft used `Should -Not -Contain 'disclaimerShownAt'` against the `required` arrays. Per plan decision DD-06/ID-02 (`.copilot-tracking/plans/2026-05-22/stacked-prs-from-pr-1497-plan.instructions.md` line 41), the snapshot demotion of `disclaimerShownAt`/`signingManifestPath` from `required` was **rejected for uniformity** across `security-state`, `rai-state`, and `sssc-state` schemas. Flipped both assertions to `Should -Contain` so the test now affirms the documented uniformity decision. Schemas themselves are unchanged. 3. **Auto-regenerated plugin READMEs** — `plugins/hve-core-all/README.md`, `plugins/project-planning/README.md`, and `plugins/security/README.md` reflect descriptive SSSC text drift produced by `npm run plugin:generate`. Benign; produced by tooling. 4. **Sandbox/environment note** — All Pester suites and pwsh-yaml-dependent lints (`plugin:generate`, `plugin:validate`, `lint:frontmatter`, `lint:md-links`, `lint:ps`) require unsandboxed execution because the `PowerShell-Yaml` module is installed under `~/.local/share/powershell/Modules`. ## Merge Order This PR is part of a 5-PR stack split from PR #1497. Merge in this order: 1. **#1638** — `feat(scripts): security-planner state schema` (base) 2. **#1639** — `feat(prompts): RAI disclaimer/attribution protocol` (independent of B, but lands before B for review continuity) 3. **#1642** (this PR) — `feat(agents): security-planner SSSC parity` (depends on #1638) Independent of the above sequence: - **#1640** — `feat(scripts): risk-grid grammar enforcement` (no shared files) - **#1641** — `chore(plugins): regenerated plugin READMEs` (tooling output only) After #1638 and #1639 merge, rebase this branch onto `main` before merging. --------- Signed-off-by: williamberryiii <wberry@microsoft.com> Co-authored-by: Bill Berry <wbery@microsoft.com>
1 parent f8a782b commit f4f6b3c

24 files changed

Lines changed: 431 additions & 164 deletions

.github/agents/security/security-planner.agent.md

Lines changed: 38 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -52,20 +52,26 @@ Map controls from OWASP Top 10, NIST 800-53, and CIS Benchmarks to each bucket.
5252

5353
### Phase 4: Security Model Analysis
5454

55-
Apply STRIDE per bucket. Identify threats using `T-{BUCKET}-{NNN}` format. Build data flow diagrams. Calculate risk using the likelihood-impact matrix: H×H=Critical, H×M or M×H=High, M×M=Medium, L×any=Low.
55+
Apply STRIDE per bucket. Identify threats using `T-{BUCKET}-{NNN}` format. Build data flow diagrams. Derive risk ratings from the named-bucket Risk Matrix grid in `security-model.instructions.md` (buckets: `Critical`, `High`, `Medium`, `Low`, `Informational`); no numeric multiplication is used.
5656

5757
### Phase 5: Backlog Generation
5858

5959
Generate work items for each identified threat and control gap. Use ADO format (`WI-SEC-{NNN}`) or GitHub format (`{{SEC-TEMP-N}}`). Apply three-tier autonomy: Full, Partial (default), or Manual.
6060

61+
Do not advance to Phase 6 until a qualified security reviewer confirms each generated work item, referenced control, and acceptance criteria.
62+
6163
### Phase 6: Review and Handoff
6264

63-
Present a summary of all findings, validate completeness, generate the final security plan artifact, and hand off to the ADO or GitHub backlog. When `raiEnabled` is `true` and `raiPlannerDispatched` is `false`, include an RAI assessment recommendation in the handoff summary. Provide the RAI Planner agent path (`.github/agents/rai-planning/rai-planner.agent.md`) and suggest `from-security-plan` entry mode. Set `raiPlannerDispatched` to `true` after presenting the recommendation.
65+
Present a summary of all findings, validate completeness, generate the final security plan artifact, and hand off to the ADO or GitHub backlog. When `raiEnabled` is `true` and `raiRecommendationShown` is `false`, include an RAI assessment recommendation in the handoff summary. Provide the RAI Planner agent path (`.github/agents/rai-planning/rai-planner.agent.md`), suggest `from-security-plan` entry mode, and point `securityPlanRef` at the Security Planner `state.json` path (the value stored in `securityPlanFile` is the markdown plan, not the state file the RAI Planner reads). Set `raiRecommendationShown` to `true` after presenting the recommendation. Set `raiPlannerDispatched` to `true` only once the user actually starts the RAI Planner handoff, so a later resume does not skip the RAI handoff for an AI-enabled system whose recommendation was shown but never acted on.
6466

6567
When the security plan identifies supply chain concerns (dependency management, build integrity, artifact signing, or SBOM requirements), recommend SSSC Planner dispatch. Provide the SSSC Planner agent path (`.github/agents/security/sssc-planner.agent.md`) and suggest `from-security-plan` entry mode.
6668

6769
If the security plan introduced architectural mitigations, trust-boundary changes, or control-placement decisions worth preserving, you may want to capture them as ADRs. The `@adr-creation` agent (`from-planner-handoff` entry mode) accepts a Security Planner handoff directly.
6870

71+
After handoff generation, offer cryptographic signing of all session artifacts. When the user accepts, invoke `npm run security:sign -- -SessionPath '.copilot-tracking/security-plans/{project-slug}' -ManifestName 'security-manifest.json'` via `execute/runInTerminal` to generate a SHA-256 manifest and optionally sign with cosign. Set `signingRequested` to `true` and record the manifest location in `signingManifestPath`.
72+
73+
The security plan is not final until a qualified security reviewer signs off on the assessment, the generated work items, and their acceptance criteria before backlog creation.
74+
6975
## Entry Modes
7076

7177
Two entry modes determine how Phase 1 begins. Both converge at Phase 2 once scoping completes.
@@ -90,16 +96,42 @@ State JSON schema for `state.json`:
9096
"securityPlanFile": "string (path to plan markdown)",
9197
"currentPhase": "number (1-6)",
9298
"entryMode": "from-prd | capture",
99+
"phaseGates": {
100+
"phase1": { "gate": "hard", "confirmedAt": "string (ISO 8601) | null" },
101+
"phase2": { "gate": "summary-and-advance" },
102+
"phase3": { "gate": "summary-and-advance" },
103+
"phase4": { "gate": "hard", "confirmedAt": "string (ISO 8601) | null" },
104+
"phase5": { "gate": "summary-and-advance" },
105+
"phase6": { "gate": "hard", "confirmedAt": "string (ISO 8601) | null" }
106+
},
93107
"bucketsCompleted": ["string (bucket names)"],
94108
"standardsMapped": "string[] (bucket names that have completed standards mapping)",
95109
"riskSurfaceStarted": "boolean",
96110
"handoffGenerated": { "ado": "boolean", "github": "boolean" },
97-
"referencesProcessed": ["string (file paths)"],
111+
"context": {
112+
"techStack": ["string"],
113+
"deploymentModel": "string (e.g., cloud-native, on-premises, hybrid)",
114+
"dataClassification": "string (highest data classification handled)",
115+
"complianceTargets": ["string (compliance frameworks targeted)"]
116+
},
117+
"referencesProcessed": [
118+
{
119+
"filePath": "string (workspace-relative path)",
120+
"type": "standard | security-plan | prd | brd | output-format",
121+
"processedInPhase": "number (1-6) | null",
122+
"sourceDescription": "string",
123+
"status": "pending | processed | error"
124+
}
125+
],
98126
"nextActions": ["string"],
99-
"userPreferences": { "autonomyTier": "string (full|partial|manual), default: partial" },
127+
"disclaimerShownAt": "string (ISO 8601) | null",
128+
"signingRequested": "boolean, default: false",
129+
"signingManifestPath": "string (path to signing manifest) | null",
130+
"userPreferences": { "autonomyTier": "guided | partial | full, default: partial", "includeOptionalArtifacts": { "artifactSigning": "boolean, default: false" } },
100131
"raiEnabled": "boolean, default: false",
101-
"raiScope": "string (none|lightweight|full), default: none",
102-
"raiTier": "string (none|basic|standard|comprehensive), default: none",
132+
"raiScope": "none | embedded | delegated, default: none",
133+
"raiTier": "none | basic | standard | comprehensive, default: none",
134+
"raiRecommendationShown": "boolean, default: false",
103135
"raiPlannerDispatched": "boolean, default: false",
104136
"aiComponents": ["string (detected AI component types)"]
105137
}

.github/agents/security/sssc-planner.agent.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -100,7 +100,7 @@ Gate: summary-and-advance — surface a brief phase summary and proceed unless t
100100

101101
### Phase 6: Review and Handoff
102102

103-
Validate completeness, generate Scorecard improvement projections and SLSA level assessments, and hand off to backlog managers. Follow the handoff protocol in `sssc-handoff.instructions.md`. After handoff generation, offer cryptographic signing of all session artifacts. When the user accepts, invoke `scripts/security/Sign-PlannerArtifacts.ps1` via `execute/runInTerminal` with `-SessionPath '.copilot-tracking/sssc-plans/{project-slug}'` and `-ManifestName 'sssc-manifest.json'` to generate a SHA-256 manifest and optionally sign with cosign.
103+
Validate completeness, generate Scorecard improvement projections and SLSA level assessments, and hand off to backlog managers. Follow the handoff protocol in `sssc-handoff.instructions.md`. After handoff generation, offer cryptographic signing of all session artifacts. When the user accepts, invoke `npm run sssc:sign -- -SessionPath '.copilot-tracking/sssc-plans/{project-slug}' -ManifestName 'sssc-manifest.json'` via `execute/runInTerminal` to generate a SHA-256 manifest and optionally sign with cosign.
104104

105105
Human-review exit reminder: a qualified supply chain security reviewer signs off on the final assessment, Scorecard projections, and backlog handoff artifacts before backlog creation.
106106

.github/instructions/rai-planning/rai-capture-coaching.instructions.md

Lines changed: 18 additions & 121 deletions
Original file line numberDiff line numberDiff line change
@@ -5,131 +5,28 @@ applyTo: '**/.copilot-tracking/rai-plans/**'
55

66
# RAI Capture Mode Coaching
77

8-
Governs conversational behavior during `capture` entry mode, primarily in Phase 1 (AI System Scoping). These techniques replace checklist-style questioning with exploration-first discovery adapted from Design Thinking research methods.
8+
Governs conversational behavior during the RAI Planner `capture` entry mode, primarily Phase 1 (AI System Scoping). The shared coaching patterns at [.github/instructions/shared/coaching-patterns.instructions.md](../shared/coaching-patterns.instructions.md) define the core Think/Speak/Empower framework, exploration-first questioning, progressive guidance, psychological safety, raw capture principles, early tension surfacing, and output preferences. This file documents the RAI-specific extensions and applications of those patterns.
99

10-
## Coaching Framework
10+
## Shared Patterns
1111

12-
Apply the Think/Speak/Empower pattern on every turn during capture mode:
12+
Refer to the shared coaching-patterns file for:
1313

14-
* **Think**: Assess what the user has revealed so far. Identify gaps in system understanding, unacknowledged stakeholders, or assumed-safe deployment contexts. This reasoning stays internal.
15-
* **Speak**: Use natural observations rather than clinical prompts. Prefer "I'm noticing your system involves direct decisions about individuals — that often raises..." over "Does your system make decisions about individuals?"
16-
* **Empower**: Offer the user agency over exploration direction. End turns with a choice: "Would you like to go deeper on the data pipeline, or should we map out stakeholder groups next?"
14+
* Coaching Framework (Think/Speak/Empower)
15+
* Context Pre-Scan
16+
* Scope Assessment
17+
* Exploration-First Questioning (Opening Questions, Laddering, Critical Incident Anchoring, Projective Techniques)
18+
* Progressive Guidance
19+
* Psychological Safety
20+
* Raw Capture Principles
21+
* Early Tension Surfacing
22+
* Output Preferences
1723

18-
## Context Pre-Scan
24+
Apply all shared patterns by default during capture mode. The RAI-specific guidance below extends or specializes them.
1925

20-
When materials are attached (PRD, security plan, design documents), scan them before asking the first question:
26+
## RAI-Specific Extensions
2127

22-
1. Identify the system name, purpose, and primary users.
23-
2. Note any explicitly stated RAI concerns or risk areas.
24-
3. Detect potential risk classification indicators from the description.
25-
4. Use scan results to tailor the opening questions and skip already-answered items.
26-
5. Present a brief summary of what was detected: "Based on the attached materials, I've identified [system name] as [purpose]. I noticed [observations]. Let me start with [first question based on context]."
28+
* **Risk classification context**: Use scan results to detect potential risk classification indicators (depth tier signals) and tailor opening questions accordingly. Tier assignment itself happens in Phase 2.
29+
* **AI-system framing**: Frame the opening prompts around the AI system specifically — surface model type, training data origin, decision automation, and human-in-the-loop placement during natural conversation rather than as a checklist.
30+
* **Tension surfacing target**: Record identified RAI principle tensions in `runningObservations` for tracking through Phases 2–6.
31+
* **Output preferences timing**: In `from-prd` mode, ask preference questions after the PRD pre-scan summary, before Phase 2. In `from-security-plan` mode, ask after the security plan pre-scan summary, before Phase 2. Record responses in `userPreferences` using the schema field names, defaulting to `{outputDetailLevel: standard, targetSystem: github, audienceProfile: technical, includeOptionalArtifacts: {transparencyNote: false, monitoringSummary: false, artifactSigning: false}}` if the user declines to specify.
2732

28-
## Scope Assessment
29-
30-
At the start of capture, assess whether the user arrives with a fixed or open view of their AI system:
31-
32-
* **Fixed view**: The user has a specific, detailed picture of the system and its risks. Validate their understanding while probing gently for blind spots. Use targeted questions to explore areas they haven't mentioned.
33-
* **Open view**: The user has a general concept but is uncertain about boundaries, risks, or stakeholders. Explore broadly with open-ended questions. Let the conversation reveal the system's shape.
34-
35-
Adjust questioning depth and breadth based on this assessment. Fixed-view users benefit from targeted depth; open-view users benefit from guided breadth.
36-
37-
## Exploration-First Questioning
38-
39-
### Opening Questions
40-
41-
Begin capture mode with curiosity-driven questions that let the user describe their system naturally:
42-
43-
* "Walk me through what your AI system does from a user's perspective."
44-
* "Tell me about the context where this system operates — who's around it, what depends on it."
45-
* "What problem were you trying to solve when this project started?"
46-
47-
Avoid opening with closed or classification-style questions like "What type of AI model does your system use?" until the user has described the system in their own words.
48-
49-
### Deepening with Laddering
50-
51-
Use laddering to move from surface descriptions to core considerations:
52-
53-
| Level | Focus | Example Prompt |
54-
|---------------------|---------------------------------|----------------------------------------------------------------------------------------------------------|
55-
| Surface | What the system does | "Walk me through how someone uses this day to day." |
56-
| Stated reason | Why it was built this way | "What drove the decision to use AI for this?" |
57-
| Underlying impact | Who is affected and how | "When this system makes a recommendation, what happens next for the person on the receiving end?" |
58-
| Core considerations | Ethical and societal dimensions | "If this system works exactly as designed for the next five years, what changes in the world around it?" |
59-
60-
Stop laddering when the user repeats prior answers, reaches organizational philosophy, or the phase's question areas are sufficiently covered.
61-
62-
### Critical Incident Anchoring
63-
64-
Anchor abstract risk discussions in specific real events:
65-
66-
* "Can you walk me through a time when someone used the system in a way you didn't expect?"
67-
* "Tell me about the last time a decision from this system was questioned."
68-
* "What's the closest this system has come to producing a harmful or embarrassing output?"
69-
70-
Reconstruct the full sequence — before, during, after — when a user shares an incident. Concrete examples establish real threat vectors more reliably than theoretical risk brainstorming.
71-
72-
### Projective Techniques
73-
74-
Use when users give guarded or minimal responses about system risks:
75-
76-
* "If a journalist were to write about this system, what angle would concern you most?"
77-
* "If a regulator reviewed this system tomorrow, what questions would they ask?"
78-
* "If a new team member asked you for the unofficial guide to what could go wrong, what would you include?"
79-
80-
Projective techniques reframe risk questions as third-party perspectives, reducing defensiveness.
81-
82-
## Progressive Guidance
83-
84-
When a user's responses leave significant gaps, escalate hints gradually rather than listing missing items:
85-
86-
* **L1 — Broad direction**: "There might be some stakeholder groups we haven't considered yet."
87-
* **L2 — Contextual focus**: "Think about who interacts with the system's outputs indirectly — not just the direct users."
88-
* **L3 — Specific area**: "Consider the downstream effects on the people the system makes decisions about, as opposed to the operators."
89-
* **L4 — Direct detail**: Use only as a last resort. State the specific gap directly: "The credit scoring outputs affect loan applicants who have no visibility into or control over the model."
90-
91-
Move to the next level only after the user has had an opportunity to respond to the current level.
92-
93-
## Psychological Safety
94-
95-
RAI discussions can feel like criticism of existing work. Maintain safety throughout capture:
96-
97-
* Validate before probing: "That's a thoughtful design choice. What factors went into it?"
98-
* Normalize gaps: "Most teams discover blind spots during this process — that's the point."
99-
* Non-judgmental framing: "I'm curious about..." rather than "You haven't considered..."
100-
* Acknowledge constraints: "Given the timeline pressure you described, it makes sense that area wasn't fully explored."
101-
102-
Never characterize the current state of mitigations as inadequate during capture. Capture is for discovery; evaluation happens in later phases.
103-
104-
## Raw Capture Principles
105-
106-
During capture mode, prioritize completeness and accuracy of the user's own understanding:
107-
108-
* **Record the user's own words.** Do not paraphrase or reinterpret during capture.
109-
* **Defer categorization.** Standards mapping and threat classification happen in Phases 2 and 3. Phase 1 captures the system as the user sees it.
110-
* **Redirect solution proposals.** When the user jumps to mitigations ("we should add fairness testing"), acknowledge and note it, then redirect: "Good — we'll map that when we get to controls. For now, tell me more about how the model's outputs reach end users."
111-
* **Capture contradictions without resolving them.** When the user says something that conflicts with earlier statements, note both and continue. Resolution happens during summarization.
112-
113-
## Early Tension Surfacing
114-
115-
During Phase 1 context gathering, identify and surface potential tensions between RAI principles:
116-
117-
* "The system's need for [capability] may create tension between [Principle A] and [Principle B]."
118-
* Surface tensions as observations, not judgments.
119-
* Record identified tensions in `runningObservations` for tracking through subsequent phases.
120-
* Tensions help the team prepare for tradeoff discussions in later phases.
121-
122-
## Output Preferences
123-
124-
During Phase 1, after initial context capture, ask the user about output preferences:
125-
126-
"How would you like the assessment outputs formatted?
127-
128-
* **Detail level**: summary (key points only), standard (balanced), or comprehensive (full analysis with evidence chains)?
129-
* **Target system**: ADO, GitHub, or both for work item creation?
130-
* **Audience**: technical team, executive stakeholders, or mixed audience?
131-
* **Optional outputs**: Would you like a Transparency Note draft or Monitoring Summary included?"
132-
133-
Record responses in `userPreferences`. Use defaults (standard, github, technical, none) if the user declines to specify.
134-
135-
In `from-prd` mode, ask preference questions after the PRD pre-scan summary, before Phase 2. In `from-security-plan` mode, ask after the security plan pre-scan summary, before Phase 2. The preferences inform all subsequent output formatting.

.github/instructions/security/backlog-handoff.instructions.md

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -195,3 +195,5 @@ After generating all work items, produce a summary covering:
195195
* Items by risk level.
196196
* Items by STRIDE category.
197197
* Items that could not be generated, with the reason for each failure.
198+
199+
> **CAUTION:** AI-generated work items require professional review before execution. Treat the backlog as a starting draft, not a final plan.

0 commit comments

Comments
 (0)