support live Scan4D coverage suggestions

[mwfarb]

We have gained some knowledge of what scanning patterns produce better splats and other models. Some of these we have codified in default settings for captures to auto edit scans in the background (60% max frustum overlap, reduce motion blur).

Some are better suited to user guidance in real-time. We should construct a list and incorporate that guidance while scanning with unobtrusive graphical or textual information. Some ideas:

• Scan 4 walls quickly to start (gives rough layout for further advice)
• Scan from the back of one wall across to the other where possible (creates quick broad context)
• Additional ideas if exterior scan...
• Additional ideas if interior scan...

Per discussion on Monday: @anthonyrowe @sagar-bharadwaj-ks

[mwfarb]

Issue #3 — Live Coverage Suggestions

Verdict: ✅ Feasible — medium effort, mostly UI/heuristics

The app already has the raw data needed for real-time scanning guidance. The infrastructure for displaying overlays during capture exists (CaptureView.swift HUD, ScanStats model, coverage overlay). The work is designing the heuristics and the unobtrusive UI.

Data already available during capture

Signal	Source	Useful for
Mesh anchor count + distribution	ARMeshAnchor positions in delegate callbacks	"Scan all 4 walls" detection
Camera pose history	FrameCaptureSession.frames array of transforms	Movement pattern analysis, directionality
Tracking status	ScanStats.trackingStatus (already displayed on HUD)	"Slow down" / "insufficient features" coaching
Capacity metrics	ScanStats.capacityScore, polygonPressure, driftEstimate	"You're approaching scan limits"
Coverage polygons	Existing convex hull projection in coverage overlay	Uncovered region detection
Classification labels	ARMeshClassification (if Issue #2 is done first)	"You haven't scanned any walls yet"
Compass heading	LocationManager.bestHeading	Cardinal direction guidance

Suggested suggestion engine architecture

┌──────────────────────────────────────────┐
│  ScanCoach (new @Observable class)       │
│                                          │
│  Inputs:                                 │
│    - ARMeshAnchor positions/normals      │
│    - Camera pose history (last N)        │
│    - ScanStats (tracking, capacity)      │
│    - Classification summary (if avail)   │
│                                          │
│  Outputs:                                │
│    - currentTip: String?                 │
│    - tipPriority: SuggestionPriority     │
│    - tipIcon: String (SF Symbol)         │
│                                          │
│  Rules engine (ordered by priority):     │
│    1. CRITICAL: Tracking lost/degraded   │
│    2. WARNING: Near capacity             │
│    3. GUIDANCE: Scan pattern hints       │
│    4. INFO: Progress encouragement       │
└──────────────────────────────────────────┘

Candidate real-time tips

Phase	Tip	Detection method
Early scan (< 30s)	"Scan all 4 walls quickly for layout context"	Timer + mesh anchor spatial extent < threshold
Early scan	"Start from one wall and sweep across to the opposite"	Camera pose history shows erratic movement
Mid-scan	"Move closer to capture fine details"	Average mesh face area is large (coarse geometry)
Mid-scan	"Slow down — frames are being dropped"	FrameCaptureSession.isBlurWarningActive or backlog
Mid-scan	"Try scanning from a lower angle"	Camera Y-position history is flat (all same height)
Late scan	"Coverage is looking good — consider finishing soon"	Coverage overlay shows < 10% uncovered
Capacity	"Approaching scan limits — save and extend"	ScanStats.isNearCapacity (already computed)
Interior	"Don't forget to scan the ceiling / under tables"	Classification (Issue #2) shows floor+walls but no ceiling
Exterior	"Capture building corners from multiple angles"	GPS + compass indicate outdoor + corners have thin coverage

UI approach

The least obtrusive approach: a single-line coach bar at the top of the capture view (below the status bar), similar to Apple's ARCoachingOverlayView pattern. It would:

• Slide in/out with animation
• Show an SF Symbol + short text
• Auto-dismiss after ~5s or when the condition resolves
• Have a "dismiss" gesture (swipe up)
• Not stack — only the highest-priority tip shows

Effort breakdown

Component	Effort
ScanCoach rules engine	Medium — mostly heuristic tuning
Coach bar UI component	Low — SwiftUI overlay with transitions
Integration with CaptureView + ScanStats	Low — wire up existing observables
Spatial analysis helpers (wall detection, coverage gaps)	Medium — requires mesh anchor normal/position analysis
Tuning & user testing	Ongoing

Dependencies

Tip

Issue #2 (classification) is not required for #3, but it significantly improves the quality of some tips. The "scan 4 walls" suggestion can use raw mesh anchor normal analysis without classification, but classification lets you give confident advice like "you've scanned 2 of 4 walls" vs a guess based on normal vectors.

---

Recommended Phasing

Phase	Scope	Effort
Phase 1	.meshWithClassification config + per-face classification in OBJ export (Issue #2 core)	~1-2 days
Phase 2	Live classification-colored wireframe (optional; nice for demo but not required for export)	~1 day
Phase 3	ScanCoach engine + coach bar UI with 3-4 basic tips (Issue #3 MVP)	~2-3 days
Phase 4	Advanced spatial tips (wall count, coverage gaps, height analysis)	~2-3 days
Phase 5	Classification-aware tips (requires Phase 1+2)	~1 day

Phase 1 is the highest-value / lowest-effort item — the downstream pipeline gets semantic labels with almost no code change. Phase 3 is self-contained and can proceed in parallel.