Capture evidence collected so far

franklesniak · franklesniak · commit fa979d6b96ca · 2026-05-05T01:09:13.000-05:00
diff --git a/.gitignore b/.gitignore
@@ -33,6 +33,8 @@ Thumbs.db
 *.utm/
 evidence/
 Evidence/
+macoslab-preflight/
+macoslab-preflight*.zip
 .maclab/
 media-cache/
 .media-cache/
diff --git a/TODO-Phase-04-Media-Acquisition.md b/TODO-Phase-04-Media-Acquisition.md
@@ -8,6 +8,28 @@
 - **Last Updated:** 2026-05-05
 - **Scope:** Deferred verification for media discovery and download tooling before implementing Phase 4.
 
+## Confirmed Preflight Evidence
+
+Owner-supplied sanitized preflight evidence from a macOS host confirms:
+
+- Host baseline: Apple Silicon (`arm64`) running macOS 26.4.1 build 25E253.
+- `mist` is installed at `/opt/homebrew/bin/mist`.
+- `mist --help`, `mist help list`, and `mist help download` work.
+- `mist version` is not valid for the installed command surface; use `mist --version` for version capture.
+- `mist list firmware --export <path>` produced JSON with 134 firmware rows.
+- For macOS Tahoe 26.4.1 build 25E253, `mist` reported compatible firmware rows, including a signed row.
+- `mist list installer --compatible --export <path>` produced 11 compatible installer rows on this host: Tahoe 26.1 through 26.4.1 and Sequoia 15.7.2 through 15.7.5. It did not return Sonoma installer rows on this macOS 26.4.1 host.
+
+Do not commit the raw or sanitized preflight capture bundle. Convert it into durable commands, expected fields, fixtures, and redacted examples only.
+
+## Remaining Open Items
+
+- [ ] Re-run and capture `mist --version`; the first preflight captured `mist version`, which usefully proved the wrong syntax but did not capture the installed version string.
+- [ ] Capture `mist help download firmware` and verify the exact options for output directory control in the installed `mist-cli` version.
+- [ ] Let the in-progress macOS Tahoe 26.4.1 firmware download finish, then record the actual IPSW path, file size, and SHA-256 checksum.
+- [ ] Record whether the download lands under the requested cache path or the default `/Users/Shared/Mist` path.
+- [ ] Decide whether Phase 4 should treat compatible installer rows as advisory metadata only, because the current Apple-silicon VM path should prefer firmware/IPSW restore images.
+
 ## Checklist
 
-- [ ] Owner: Repository owner; Status: Open; Phase gate affected: Phase 4 media acquisition; Why it matters: `mist-cli` syntax and output can change across releases; Action: verify current `mist-cli` list/download syntax and record exact commands for discovering and downloading pinned macOS restore images or provider-appropriate install artifacts; Acceptance condition: the verified commands, version output, and expected metadata fields are documented before `Get-MacLabMedia` implementation; Source: ADR-0005 and spec Section 9.6.
+- [ ] Owner: Repository owner; Status: Partially verified; Phase gate affected: Phase 4 media acquisition; Why it matters: `mist-cli` syntax and output can change across releases; Action: use the confirmed preflight facts above, complete the remaining download/version captures, and record exact commands for discovering and downloading pinned macOS restore images or provider-appropriate install artifacts; Acceptance condition: the verified commands, version output, expected metadata fields, actual downloaded artifact path, and checksum are documented before `Get-MacLabMedia` implementation; Source: ADR-0005 and spec Section 9.6.
diff --git a/TODO-Phase-05-Parallels-Provider.md b/TODO-Phase-05-Parallels-Provider.md
@@ -8,6 +8,30 @@
 - **Last Updated:** 2026-05-05
 - **Scope:** Deferred verification for Parallels Desktop command-line automation before implementing Phase 5.
 
+## Confirmed Preflight Evidence
+
+Owner-supplied sanitized preflight evidence from a macOS host confirms:
+
+- Host baseline: Apple Silicon (`arm64`) running macOS 26.4.1 build 25E253.
+- `prlctl` is installed at `/usr/local/bin/prlctl`.
+- `prlsrvctl` is installed at `/usr/local/bin/prlsrvctl`.
+- `prlctl --version` returns Parallels CLI version 26.3.2 build 57398.
+- `prlsrvctl info` reports Parallels Desktop 26.3.2-57398, service started, default VM home under `~/Parallels`, and a valid license. Do not commit license strings from this output.
+- `prlctl list -a` returned only the table header, confirming no registered Parallels VMs existed at capture time.
+- `prlctl list --help` supports `--all`, `--full`, `--output`, `--list`, `--template`, `--json`, `--no-header`, `--stopped`, and `--name`.
+- `prlctl snapshot-list --help` supports `--tree`, `--id`, `--json`, and `--no-header`.
+
+Do not commit the raw or sanitized preflight capture bundle. Convert it into durable commands, expected fields, fixtures, and redacted examples only.
+
+## Remaining Open Items
+
+- [ ] After the pinned IPSW download finishes, create one disposable same-major macOS Tahoe 26.4.1 Parallels VM and capture the exact `prlctl create <name> -o macos --restore-image <path>` behavior.
+- [ ] Verify and document VM list/info/start/stop/delete behavior against the disposable VM.
+- [ ] Verify and document named snapshot create/list/restore behavior against the disposable VM.
+- [ ] Confirm whether this Parallels installation exposes edition as Pro through a parseable CLI field; current evidence proves Desktop 26.3.2-57398 and a valid license, but does not expose a clean "Pro" string.
+- [ ] Record the Provider Version Matrix for the disposable VM: host macOS version/build, guest macOS version/build, host/guest classification, Parallels version, IPSW path, IPSW checksum, and date tested.
+- [ ] Treat guest macOS versions higher than the host major version as rejected by default. The current macOS 26.4.1 host can verify same-major Tahoe behavior; older guest majors require explicit cross-major evidence only if they remain in scope for this host.
+
 ## Checklist
 
-- [ ] Owner: Repository owner; Status: Open; Phase gate affected: Phase 5 Parallels provider; Why it matters: provider automation must match the installed Parallels Desktop Pro command surface and must not assume unsupported host/guest macOS pairings; Action: verify current `prlctl` syntax, version output, edition detection, same-major macOS VM creation behavior, snapshot behavior, and higher-than-host guest rejection or warning behavior on the owner/demo host; Acceptance condition: exact commands, expected outputs, host macOS version/build, guest macOS version/build, Parallels version/edition, and host/guest compatibility classification are documented so provider tests can be written without guessing; Source: ADR-0005, ADR-0011, and spec Sections 8.1.1, 9.6, and 17.4.
+- [ ] Owner: Repository owner; Status: Partially verified; Phase gate affected: Phase 5 Parallels provider; Why it matters: provider automation must match the installed Parallels Desktop Pro command surface and must not assume unsupported host/guest macOS pairings; Action: use the confirmed CLI/version facts above, then verify same-major macOS VM creation behavior, snapshot behavior, and rejection/warning behavior on the owner/demo host once the pinned IPSW is available; Acceptance condition: exact commands, expected outputs, host macOS version/build, guest macOS version/build, Parallels version/edition evidence, and host/guest compatibility classification are documented so provider tests can be written without guessing; Source: ADR-0005, ADR-0011, and spec Sections 8.1.1, 9.6, and 17.4.
diff --git a/TODO-Phase-06-UTM-Provider.md b/TODO-Phase-06-UTM-Provider.md
@@ -8,6 +8,30 @@
 - **Last Updated:** 2026-05-05
 - **Scope:** Deferred verification for UTM automation and manual-step boundaries before implementing Phase 6.
 
+## Confirmed Preflight Evidence
+
+Owner-supplied sanitized preflight evidence from a macOS host confirms:
+
+- Host baseline: Apple Silicon (`arm64`) running macOS 26.4.1 build 25E253.
+- UTM is installed under `/Applications/UTM.app`.
+- `defaults read /Applications/UTM.app/Contents/Info.plist CFBundleShortVersionString` reports UTM 4.7.5.
+- Gatekeeper accepted the app, with Mac App Store signing reported.
+- `utmctl` help is available at `/Applications/UTM.app/Contents/MacOS/utmctl`.
+- Top-level `utmctl` help lists `version`, `list`, `status`, `start`, `suspend`, `stop`, `attach`, `file`, `exec`, `ip-address`, `clone`, `delete`, and `usb`.
+- Top-level `utmctl` help does not advertise create/import/snapshot primitives. Treat VM creation and snapshot management as manual-step candidates until proven otherwise.
+- `utmctl list` returned only the table header, confirming no registered UTM VMs existed at capture time. It may launch or foreground UTM because it controls the installed app.
+
+Do not commit the raw or sanitized preflight capture bundle. Convert it into durable commands, expected fields, fixtures, and redacted examples only.
+
+## Remaining Open Items
+
+- [ ] Capture `/Applications/UTM.app/Contents/MacOS/utmctl version`; the first preflight captured the app bundle version via `defaults`, not the `utmctl` subcommand.
+- [ ] After the pinned IPSW download finishes, create one disposable same-major macOS Tahoe 26.4.1 UTM VM using the GUI path and exact IPSW.
+- [ ] Verify and document which lifecycle primitives work through `utmctl` against the disposable VM: list, status, start, suspend, stop, IP address, file, exec, clone, and delete. Do not run destructive delete except against a disposable VM.
+- [ ] Verify and document which VM creation, import/export, and snapshot actions require manual UI steps.
+- [ ] Record whether macOS prompts for Terminal or shell automation permissions when `utmctl` controls UTM.
+- [ ] Record the Provider Version Matrix for the disposable VM: host macOS version/build, guest macOS version/build, host/guest classification, UTM version, IPSW path, IPSW checksum, manual-step gaps, and date tested.
+
 ## Checklist
 
-- [ ] Owner: Repository owner; Status: Open; Phase gate affected: Phase 6 UTM provider; Why it matters: UTM automation may not mirror Parallels CLI parity, manual gaps must be explicit, and macOS guests on Apple silicon must respect Apple Virtualization host/guest compatibility constraints; Action: verify current UTM and `utmctl` automation surface, including VM creation, launch, stop, snapshot, import/export, Apple Virtualization enforcement, host/guest macOS compatibility behavior, and any manual-step requirements; Acceptance condition: UTM capability matrix, manual-step gaps, host macOS version/build, guest macOS version/build, UTM version, and host/guest compatibility classification are documented before provider implementation; Source: ADR-0005, ADR-0011, and spec Sections 8.1.1, 9.6, and 17.5.
+- [ ] Owner: Repository owner; Status: Partially verified; Phase gate affected: Phase 6 UTM provider; Why it matters: UTM automation may not mirror Parallels CLI parity, manual gaps must be explicit, and macOS guests on Apple silicon must respect Apple Virtualization host/guest compatibility constraints; Action: use the confirmed app/version/command-surface facts above, then verify current UTM and `utmctl` automation against a disposable same-major VM, including VM creation, launch, stop, snapshot, import/export, Apple Virtualization enforcement, host/guest macOS compatibility behavior, and any manual-step requirements; Acceptance condition: UTM capability matrix, manual-step gaps, host macOS version/build, guest macOS version/build, UTM version, and host/guest compatibility classification are documented before provider implementation; Source: ADR-0005, ADR-0011, and spec Sections 8.1.1, 9.6, and 17.5.
diff --git a/TODO-Phase-07-Evidence-Pipeline.md b/TODO-Phase-07-Evidence-Pipeline.md
@@ -8,6 +8,28 @@
 - **Last Updated:** 2026-05-05
 - **Scope:** Deferred replacement of the temporary worked-example schema with the real macOSLab evidence-bundle schema.
 
+## Confirmed Preflight Evidence
+
+Owner-supplied sanitized host preflight evidence confirms the future evidence model needs to represent at least:
+
+- Host OS version/build and architecture.
+- Tool paths and version output for `mist`, `prlctl`, `prlsrvctl`, and UTM/`utmctl`.
+- Provider command-surface captures, such as help output and list output.
+- Media discovery JSON from `mist`.
+- Provider inventory states where no VMs are registered yet.
+- Sanitization status for hostnames, user paths, UUIDs, MAC addresses, email addresses, license material, and other local identifiers.
+- A distinction between raw local captures, sanitized review bundles, and durable evidence summaries that are safe to commit.
+
+Do not commit the raw or sanitized preflight capture bundle. The committed evidence schema should store redacted summaries and deterministic fixture data, not private host captures.
+
+## Remaining Open Items
+
+- [ ] Decide whether the real evidence schema stores raw command-output attachments, normalized parsed facts, or both.
+- [ ] Add fields for host/guest macOS compatibility classification and provider manual-step gaps.
+- [ ] Add redaction assertions that reject license strings, hardware identifiers, local usernames, VM UUIDs, MAC addresses, tenant identifiers, and device identifiers.
+- [ ] Add fixtures for "tool installed but no VMs registered" states for Parallels and UTM.
+- [ ] Add fixtures for "Defender intentionally absent on host; validate inside guest" states if Phase 8 uses host/guest evidence separation.
+
 ## Checklist
 
-- [ ] Owner: Repository owner; Status: Open; Phase gate affected: Phase 7 evidence pipeline; Why it matters: Phase 0 keeps the worked-example schema only to exercise validation tooling; Action: replace `schemas/example-config.schema.json` and its example fixtures with the real evidence-bundle schema from spec Section 25, then update `.pre-commit-config.yaml`, data-file CI, schema docs, and examples together; Acceptance condition: valid evidence examples pass schema validation, invalid examples are rejected by an explicit test or validation script, and the worked example is fully removed; Source: `docs/phase-0-bootstrap-codex-instructions.md` and spec Section 25.
+- [ ] Owner: Repository owner; Status: Open with preflight inputs captured; Phase gate affected: Phase 7 evidence pipeline; Why it matters: Phase 0 keeps the worked-example schema only to exercise validation tooling; Action: replace `schemas/example-config.schema.json` and its example fixtures with the real evidence-bundle schema from spec Section 25, then update `.pre-commit-config.yaml`, data-file CI, schema docs, and examples together; Acceptance condition: valid evidence examples pass schema validation, invalid examples are rejected by an explicit test or validation script, redaction failures are caught, and the worked example is fully removed; Source: `docs/phase-0-bootstrap-codex-instructions.md` and spec Section 25.
diff --git a/TODO-Phase-08-Validation-Loop.md b/TODO-Phase-08-Validation-Loop.md
@@ -8,6 +8,19 @@
 - **Last Updated:** 2026-05-05
 - **Scope:** Deferred verification for Defender health evidence before implementing Phase 8.
 
+## Confirmed Preflight Evidence
+
+The owner does not want Microsoft Defender for Endpoint installed on the daily-driver host Mac. Host-side `mdatp` output is therefore intentionally unavailable and should not be treated as missing evidence.
+
+Defender validation evidence should be collected inside a disposable macOS guest VM after the pinned VM build exists. This keeps the host clean and better matches the lab's purpose: validating Defender, PPPC/FDA, onboarding, and health from inside the managed guest.
+
+## Remaining Open Items
+
+- [ ] After a disposable VM exists, install or deploy Defender for Endpoint inside the guest using the intended demo path.
+- [ ] Capture `mdatp health` output before and after onboarding, after required system extension and Full Disk Access approvals, and after any intentionally broken policy state used in the demo.
+- [ ] Sanitize tenant, device, machine, user, and cloud identifiers before using the output as fixtures.
+- [ ] Decide whether Defender should be present in the `Pre-Enroll` snapshot or deployed live by Intune; do not accidentally hide the behavior the demo is meant to show.
+
 ## Checklist
 
-- [ ] Owner: Repository owner; Status: Open; Phase gate affected: Phase 8 validation loop; Why it matters: Defender `mdatp health` output can change and evidence fixtures must not leak real tenant or device data; Action: verify the current Defender `mdatp health` output shape and create sanitized fixtures for evidence tests; Acceptance condition: fixture fields needed by validation code are documented, sanitized, and covered by tests before Defender validation ships; Source: ADR-0005 and spec Section 9.6.
+- [ ] Owner: Repository owner; Status: Open, guest-only by design; Phase gate affected: Phase 8 validation loop; Why it matters: Defender `mdatp health` output can change and evidence fixtures must not leak real tenant or device data; Action: verify the current Defender `mdatp health` output shape inside a disposable macOS guest and create sanitized fixtures for evidence tests; Acceptance condition: fixture fields needed by validation code are documented, sanitized, and covered by tests before Defender validation ships; Source: ADR-0005 and spec Section 9.6.
diff --git a/docs/planning/macos-imaging-05c-recommended-software-step-by-step-revised.md b/docs/planning/macos-imaging-05c-recommended-software-step-by-step-revised.md
@@ -527,7 +527,7 @@ Do this **before** you spend time building or snapshotting VMs. Catching a broke
    pwsh --version
    code --version
    mist --help
-   mist version
+   mist --version
    mist help list
    mist help download
    prlctl --version
@@ -543,6 +543,8 @@ Do this **before** you spend time building or snapshotting VMs. Catching a broke
    tart --version
    ```
 
+   `utmctl` may launch or foreground UTM because it controls the installed app. Treat that as normal unless the command hangs, fails to return output, or prompts for an automation permission that you cannot approve.
+
 3. Launch and confirm the following open correctly:
    - Visual Studio Code
    - UTM
@@ -828,7 +830,7 @@ The concrete commands below use the **firmware / IPSW** path because current Par
 
    ```bash
    mkdir -p ~/Demo/Installers
-   mist version > ~/Demo/Installers/mist-version.txt 2>&1 || true
+   mist --version > ~/Demo/Installers/mist-version.txt 2>&1 || true
    mist --help > ~/Demo/Installers/mist-help.txt 2>&1 || true
    mist help list > ~/Demo/Installers/mist-help-list.txt 2>&1 || true
    mist help download > ~/Demo/Installers/mist-help-download.txt 2>&1 || true
@@ -841,6 +843,8 @@ The concrete commands below use the **firmware / IPSW** path because current Par
    mist list installer --compatible --export ~/Demo/Installers/mist-installers-compatible.json
    ```
 
+   Owner preflight evidence from a macOS 26.4.1 Apple-silicon host showed `mist list installer --compatible` returning Tahoe 26.1 through 26.4.1 and Sequoia 15.7.2 through 15.7.5, but no Sonoma installer rows. Treat that as host-specific discovery evidence, not as a global support matrix. Older macOS targets still need their own same-major or otherwise documented host/provider preflight.
+
 3. In `mist-firmware.json`, choose the firmware row for the pinned build. Prefer a row that is:
    - the exact macOS version and build you intend to demo
    - compatible with the host according to `mist`
