docs(security): Spec 077 — scanner simplification (spec/plan/tasks) (#784)

* docs(security): Spec 077 scanner-simplification (spec/plan/tasks)

Planning package for making the deterministic offline detect engine the
always-on default scanner and demoting the Docker scanners + source
extraction to an opt-in deep scan that never blocks or degrades the baseline
verdict, with a single unified report.

- specs/077-scanner-simplification/spec.md: 4 user stories, 21 FRs, 8 SCs
- plan.md: constitution check (all 6 principles), structure, complexity tracking
- research.md: 8 grounded decisions (D1-D8) + alternatives
- data-model.md + contracts/: unified report + security config schema
- quickstart.md: 7 verification scenarios
- CLAUDE.md: Recent Changes entry (speckit agent-context)

Docs/spec-only; no code touched. Implementation follows via per-story PRs.

Related: Spec 077 (specs/077-scanner-simplification)

* chore(roadmap): regenerate ROADMAP.md to include Spec 077

roadmap.yaml (from main via #785) references specs/077; regenerate the
rendered view so the per-spec badge (0/42, drafted) and the epic status row
reflect the spec landing in this PR. Keeps the roadmap-up-to-date CI check
green.

Related: Spec 077 (specs/077-scanner-simplification)

Author: Dumbris

CLAUDE.md

@@ -157,4 +157,5 @@ tail -f ~/Library/Logs/mcpproxy/main.log  # main log (macOS; Linux: ~/.mcpproxy/
 - **Windows installer**: [docs/github-actions-windows-wix-research.md](docs/github-actions-windows-wix-research.md). **Prerelease** (`next` branch + `v*-rc.*` tags, opt-in, off stable channels): [docs/prerelease-builds.md](docs/prerelease-builds.md).
 
 ## Recent Changes
+- 077-scanner-simplification: Added Go 1.24 (backend/core), TypeScript 5.9 / Vue 3.5 (frontend Web UI) + Existing only — `internal/security/detect` (stdlib + `golang.org/x/text/unicode/norm`, already an indirect dep), `internal/security/scanner`, BBolt (scanner records + tool approvals), Bleve (index, untouched), zap (logging). **No new third-party dependency.**
 - 076-deterministic-tool-scanner: Added Go 1.24 + stdlib only for detection (`unicode`, `unicode/utf8`, `encoding/base64`, `encoding/hex`, `regexp`); `golang.org/x/text/unicode/norm` (already an indirect dep via x/text) for NFKC; existing `internal/security/patterns/`, `internal/security/scanner/`, `internal/runtime/tool_quarantine.go`. No new third-party dependency.

ROADMAP.md

@@ -135,7 +135,7 @@ graph TD
 
 | Epic | Status | Assignee | Priority | Progress | Spec | PR |
 | --- | --- | --- | --- | --- | --- | --- |
-| Scanner simplification (deterministic default, opt-in deep scan) | In progress | unassigned | P1 | — | [077-scanner-simplification](./specs/077-scanner-simplification/) |  |
+| Scanner simplification (deterministic default, opt-in deep scan) | In progress | unassigned | P1 | 0/42 (0%) | [077-scanner-simplification](./specs/077-scanner-simplification/) |  |
 | Windows native tray app `MCP-43` | In review | BackendEngineer | P2 | 25/60 (42%) | [002-windows-installer](./specs/002-windows-installer/) |  |
 | Web UI + macOS app UX audit | Todo | unassigned | P0 | — | [064-glass-cockpit](./specs/064-glass-cockpit/) |  |
 | Action log / transparency — info at a glance | Todo | unassigned | P0 | 63/66 (95%) | [024-expand-activity-log](./specs/024-expand-activity-log/) |  |
@@ -232,3 +232,4 @@ Legend: `shipped` ≥95% checked · `in-flight` 1–94% · `drafted` 0% · `—`
 | [074-discovery-intervals](./specs/074-discovery-intervals/) | `drafted` | 0/19 (0%) |
 | [075-macos-tcc-connect](./specs/075-macos-tcc-connect/) | `in-flight` | 11/30 (37%) |
 | [076-deterministic-tool-scanner](./specs/076-deterministic-tool-scanner/) | `in-flight` | 22/24 (92%) |
+| [077-scanner-simplification](./specs/077-scanner-simplification/) | `drafted` | 0/42 (0%) |

specs/077-scanner-simplification/checklists/requirements.md

@@ -0,0 +1,37 @@
+# Specification Quality Checklist: Scanner Simplification — Deterministic Default, Opt-In Deep Scan
+
+**Purpose**: Validate specification completeness and quality before proceeding to planning
+**Created**: 2026-06-30
+**Feature**: [spec.md](../spec.md)
+
+## Content Quality
+
+- [x] No implementation details (languages, frameworks, APIs)
+- [x] Focused on user value and business needs
+- [x] Written for non-technical stakeholders
+- [x] All mandatory sections completed
+
+## Requirement Completeness
+
+- [x] No [NEEDS CLARIFICATION] markers remain
+- [x] Requirements are testable and unambiguous
+- [x] Success criteria are measurable
+- [x] Success criteria are technology-agnostic (no implementation details)
+- [x] All acceptance scenarios are defined
+- [x] Edge cases are identified
+- [x] Scope is clearly bounded
+- [x] Dependencies and assumptions identified
+
+## Feature Readiness
+
+- [x] All functional requirements have clear acceptance criteria
+- [x] User scenarios cover primary flows
+- [x] Feature meets measurable outcomes defined in Success Criteria
+- [x] No implementation details leak into specification
+
+## Notes
+
+- The four design decisions (default-deterministic approach, curated hard-tier phrase posture, opt-in deep scan, single unified report) were resolved during brainstorming and carried into the spec, so no [NEEDS CLARIFICATION] markers were required.
+- Success criteria SC-003/SC-004 reference an "evaluation corpus" as a measurement instrument (a measurable outcome), not an implementation detail.
+- One deliberate, documented posture change exists (FR-004 / Edge Cases): some phrases that legacy rules hard-blocked may become review-only unless included in the curated hard-tier set. This is a security-posture decision, not an ambiguity.
+- Items marked incomplete require spec updates before `/speckit.clarify` or `/speckit.plan`. All items pass.

specs/077-scanner-simplification/contracts/scan-report.schema.json

@@ -0,0 +1,69 @@
+{
+  "$schema": "https://json-schema.org/draft/2020-12/schema",
+  "$id": "https://mcpproxy.app/schemas/077/scan-report.json",
+  "title": "ScanReport",
+  "description": "Unified per-server security scan report. Baseline verdict is independent of deep-scan availability (Spec 077).",
+  "type": "object",
+  "required": ["server", "status", "findings", "deep_scan"],
+  "additionalProperties": true,
+  "properties": {
+    "server": { "type": "string", "description": "Server name." },
+    "status": {
+      "type": "string",
+      "enum": ["clean", "warning", "dangerous"],
+      "description": "Verdict derived from BASELINE findings only. No 'degraded'/'failed' from deep-scan gaps (FR-014)."
+    },
+    "risk_score": { "type": "number", "minimum": 0 },
+    "scanned_at": { "type": "string", "format": "date-time" },
+    "findings": {
+      "type": "array",
+      "description": "Merged + deduplicated across all scanners that ran. No (rule_id, location) duplicates (FR-010/FR-011).",
+      "items": { "$ref": "#/$defs/finding" }
+    },
+    "deep_scan": { "$ref": "#/$defs/deepScanDescriptor" }
+  },
+  "$defs": {
+    "finding": {
+      "type": "object",
+      "required": ["rule_id", "location", "severity", "tier", "sources"],
+      "additionalProperties": true,
+      "properties": {
+        "rule_id": { "type": "string" },
+        "location": { "type": "string", "description": "server:tool; dedup key with rule_id." },
+        "severity": { "type": "string", "enum": ["info", "low", "medium", "high", "critical"] },
+        "tier": { "type": "string", "enum": ["hard", "soft"], "description": "Only hard baseline findings gate approval (FR-021)." },
+        "threat_type": { "type": "string" },
+        "confidence": { "type": "number", "minimum": 0, "description": "Higher when independent sources agree (FR-012)." },
+        "sources": {
+          "type": "array",
+          "minItems": 1,
+          "items": { "type": "string" },
+          "description": "Contributing scanner ids; all sources of a merged finding."
+        },
+        "message": { "type": "string" }
+      }
+    },
+    "deepScanDescriptor": {
+      "type": "object",
+      "required": ["enabled", "ran", "available"],
+      "additionalProperties": false,
+      "description": "Informational only; MUST NOT influence status (FR-008).",
+      "properties": {
+        "enabled": { "type": "boolean" },
+        "ran": { "type": "boolean" },
+        "available": { "type": "boolean" },
+        "scanners_failed": {
+          "type": "array",
+          "items": {
+            "type": "object",
+            "required": ["id", "reason"],
+            "properties": {
+              "id": { "type": "string" },
+              "reason": { "type": "string" }
+            }
+          }
+        }
+      }
+    }
+  }
+}

specs/077-scanner-simplification/contracts/security-config.schema.json

@@ -0,0 +1,35 @@
+{
+  "$schema": "https://json-schema.org/draft/2020-12/schema",
+  "$id": "https://mcpproxy.app/schemas/077/security-config.json",
+  "title": "SecurityConfig (Spec 077 delta)",
+  "description": "The `security` config block after Spec 077. Shows the new deep_scan group, the removed key, and back-compat aliases that migrate on load.",
+  "type": "object",
+  "additionalProperties": true,
+  "properties": {
+    "deep_scan": {
+      "type": "object",
+      "description": "Opt-in heavy-scan layer. Default OFF. When disabled, only the in-process deterministic baseline runs (FR-006).",
+      "additionalProperties": false,
+      "properties": {
+        "enabled": { "type": "boolean", "default": false, "description": "Master opt-in for Docker scanners + source extraction." },
+        "fetch_package_source": { "type": ["boolean", "null"], "default": true, "description": "Absorbs the deprecated top-level scanner_fetch_package_source." },
+        "disable_no_new_privileges": { "type": "boolean", "default": false, "description": "Absorbs deprecated scanner_disable_no_new_privileges (snap/AppArmor escape hatch)." },
+        "scanners": { "type": "array", "items": { "type": "string" }, "default": [], "description": "Optional per-scanner enable list under the umbrella." }
+      }
+    },
+    "scan_timeout_default": { "type": "string", "description": "Unchanged." },
+    "integrity_check_interval": { "type": "string", "description": "Unchanged." }
+  },
+  "x-migration": {
+    "removed": ["auto_scan_quarantined"],
+    "aliases": {
+      "scanner_fetch_package_source": "deep_scan.fetch_package_source",
+      "scanner_disable_no_new_privileges": "deep_scan.disable_no_new_privileges"
+    },
+    "note": "Old configs load unchanged: aliases map on load with identical effect; auto_scan_quarantined is ignored if present (FR-016/FR-017/SC-007)."
+  },
+  "x-unchanged": {
+    "quarantine_enabled": "global tri-state, default true — out of scope",
+    "auto_approve_tool_changes": "per-server, out of scope"
+  }
+}

specs/077-scanner-simplification/data-model.md

@@ -0,0 +1,101 @@
+# Phase 1 Data Model: Scanner Simplification
+
+Entities are described at the domain level and mapped to the existing Go types they
+extend (this is a refactor, not a greenfield schema). Field names in code may differ
+slightly; the contract JSON in `./contracts/` is authoritative for wire shapes.
+
+---
+
+## ScanReport (per server)
+
+The single consolidated result for one server's tool set. Extends the existing
+`ScanSummary` / aggregated report.
+
+| Field | Type | Notes |
+|-------|------|-------|
+| `server` | string | Server name. |
+| `status` | enum `clean` \| `warning` \| `dangerous` | **Derived from baseline findings ONLY** (FR-014). No `degraded`/`failed` from deep-scan gaps. |
+| `risk_score` | number | From `CalculateRiskScore` over the merged, deduped findings (consensus-weighted). |
+| `findings` | Finding[] | Merged + deduplicated across all scanners that ran (FR-010/FR-011). |
+| `deep_scan` | DeepScanDescriptor | Separate availability dimension (FR-008). |
+| `scanned_at` | timestamp | When the scan settled. |
+
+**Validation / rules**:
+- `status` MUST be a function of baseline findings only; deep findings never change it.
+- `findings` MUST contain no `(rule_id, location)` duplicates.
+- A `dangerous` status MUST correspond to at least one hard-tier baseline finding.
+
+**State**: A report is produced per scan and replaces the prior report for that
+server. It is emitted to clients via a single settled event (see below).
+
+---
+
+## Finding
+
+One detected issue. Extends the existing `ScanFinding`.
+
+| Field | Type | Notes |
+|-------|------|-------|
+| `rule_id` | string | Detection rule / check id (e.g. `phrase_injection`, `unicode.hidden`). |
+| `location` | string | `server:tool` (detect vocabulary), the dedup key with `rule_id`. |
+| `severity` | enum `info` \| `low` \| `medium` \| `high` \| `critical` | User-readable severity (FR-013). |
+| `tier` | enum `hard` \| `soft` | Hard → contributes to blocking verdict; soft → review-only. |
+| `threat_type` | string | Classified category; consensus match key with `location`. |
+| `confidence` | number | Raised when multiple independent sources agree (FR-012). |
+| `sources` | string[] | Contributing scanner ids (e.g. `tpa-descriptions`, `cisco-mcp-scanner`). ≥1. |
+| `signals` | Signal[] | Detect-engine signals (present for baseline findings; empty for external). |
+| `message` | string | Human-readable description. |
+
+**Validation / rules**:
+- `sources` MUST be non-empty; a merged finding lists all contributing sources.
+- `confidence` for a finding agreed by N distinct sources MUST exceed the
+  single-source confidence (monotonic in consensus).
+- Only `tier == hard` baseline findings gate approval (FR-021).
+
+---
+
+## DeepScanDescriptor
+
+Informational status of the opt-in layer. New object on the report.
+
+| Field | Type | Notes |
+|-------|------|-------|
+| `enabled` | bool | From `security.deep_scan.enabled`. |
+| `ran` | bool | Whether any deep scanner executed this scan. |
+| `available` | bool | False when Docker/source-extraction/prereqs are unavailable. |
+| `scanners_failed` | { id, reason }[] | Per-scanner best-effort failures (e.g. AppArmor/snap, extraction failure). Informational only. |
+
+**Rules**: This object MUST NOT influence `ScanReport.status`. When `enabled=false`,
+`ran=false`, `available=false`, `scanners_failed=[]`.
+
+---
+
+## SecurityConfig (config surface changes)
+
+Under `security` in `mcp_config.json`. Extends the existing `SecurityConfig`.
+
+| Field | Type | Default | Notes |
+|-------|------|---------|-------|
+| `deep_scan.enabled` | bool | `false` | Master opt-in for the heavy layer (FR-006). |
+| `deep_scan.fetch_package_source` | *bool | `true` (within deep scan) | Absorbs `scanner_fetch_package_source`. |
+| `deep_scan.disable_no_new_privileges` | bool | `false` | Absorbs `scanner_disable_no_new_privileges` (snap/AppArmor escape hatch). |
+| `deep_scan.scanners` | string[] | `[]` | Optional per-scanner enable list under the umbrella. |
+| ~~`auto_scan_quarantined`~~ | — | removed | Orphaned/never-consumed (FR-016). |
+
+**Migration (FR-017 / SC-007)**: On load, top-level `scanner_fetch_package_source`
+and `scanner_disable_no_new_privileges` map into `deep_scan.*` with identical effect;
+`auto_scan_quarantined` is ignored if present. Old configs load without edits.
+
+**Unchanged**: `quarantine_enabled` (global), `auto_approve_tool_changes` (per-server),
+and all tool-approval hashing/state (Spec 032) — out of scope (FR-019).
+
+---
+
+## BundledScanner registry defaults (FR-018)
+
+| Scanner | Default `enabled` |
+|---------|-------------------|
+| `tpa-descriptions` (in-process detect engine) | **true** |
+| `cisco-mcp-scanner`, `mcp-ai-scanner`, `mcp-scan`, `nova-proximity`, `ramparts`, `semgrep-mcp`, `trivy-mcp` | **false** |
+
+Docker scanners only run when `deep_scan.enabled=true` AND individually enabled.