Skip to content

Commit 081ec83

Browse files
authored
Merge pull request #401 from opensensor/feature/uxd-01-mobile-foundation
feat(ux): mobile-first UX foundation (PRD 01 Wave 1: T1+T3+T4)
2 parents 1f9578e + 8221318 commit 081ec83

33 files changed

Lines changed: 4823 additions & 2240 deletions

docs/prd/README.md

Lines changed: 17 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,17 @@
1+
# LightNVR PRDs
2+
3+
This folder collects Product Requirement Documents that scope larger pieces of work. They are deliberately written as Markdown (rather than the historical `.docx` PRDs at the repo root) so they can be reviewed in PRs and linked from issues.
4+
5+
Each PRD is self-contained: problem, goals, requirements, phasing, acceptance, related issues.
6+
7+
## Active PRDs
8+
9+
### UX / Design
10+
11+
| PRD | Topic | Driving issues |
12+
| --- | --- | --- |
13+
| [UXD 01 — Mobile-First UX Foundation](UXD_01_MobileFirstFoundation.md) | Cross-cutting: async feedback, settings restructure, mobile chrome, theme polish, streams page layout | [#399](https://github.com/opensensor/lightNVR/issues/399) |
14+
| [UXD 02 — Live View Ergonomics](UXD_02_LiveViewErgonomics.md) | Per-stream playback transport, unified grid placement, mobile gestures, badge clarity | [#326](https://github.com/opensensor/lightNVR/issues/326), [#397](https://github.com/opensensor/lightNVR/issues/397), [#399](https://github.com/opensensor/lightNVR/issues/399) |
15+
| [UXD 03 — Recordings & Timeline UX](UXD_03_RecordingsAndTimeline.md) | Scrub continuity, ruler set-diff sync, refresh affordances, mobile timeline gestures | [#331](https://github.com/opensensor/lightNVR/issues/331), [#399](https://github.com/opensensor/lightNVR/issues/399) |
16+
17+
The three UXD PRDs share a primitive — an `<AsyncButton>` / `useAsyncAction` hook — defined in PRD 01 and consumed by 02 and 03. Land 01 P0 first.

docs/prd/UXD_01_MobileFirstFoundation.md

Lines changed: 606 additions & 0 deletions
Large diffs are not rendered by default.
Lines changed: 149 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,149 @@
1+
# PRD — Live View Ergonomics
2+
3+
**Status**: Draft
4+
**Created**: 2026-04-22
5+
**Owner**: TBD
6+
**Driving signal**: [#326 (CDx4f3kCAf3Y)](https://github.com/opensensor/lightNVR/issues/326), [#397 (AndyIsHereBoi)](https://github.com/opensensor/lightNVR/issues/397), and the Live View / mobile items from [#399](https://github.com/opensensor/lightNVR/issues/399).
7+
**Scope**: `web/js/components/preact/LiveView.jsx`, `HLSVideoCell.jsx`, `MSEVideoCell.jsx`, `GridPicker.jsx`, `FullscreenManager.jsx`, plus a new per-stream "Playback profile" model.
8+
9+
---
10+
11+
## 1. Problem
12+
13+
Live View is the page operators use most, often from a phone. Today it has three distinct ergonomic gaps:
14+
15+
1. **No control over which playback transport is offered.** A single global "WebRTC enabled" checkbox is the only knob (#397). Operators who find HLS too jittery and WebRTC too flaky want to force MSE; some want WebRTC for one camera and HLS for another. Right now they can't.
16+
2. **Grid placement isn't durable.** Reordering streams in WebRTC view doesn't stick when the user switches to HLS/MSE, and vice versa (#326). Different state stores per renderer mean the user reorders the same cameras three times.
17+
3. **Mobile gestures don't exist.** No swipe-to-cycle, no pinch-to-zoom on a tile, no long-press menu. Fullscreen requires precision-tapping a small icon. Pulling to refresh does nothing.
18+
19+
Layered on these: the recording-state badge on each tile is ambiguous (#399 — "Enabled (Detection)" hides the constant-recording case), and there's no visual cue when a tile is unhealthy vs just buffering.
20+
21+
## 2. Goals & non-goals
22+
23+
**Goals**
24+
- Per-stream playback transport preference + per-instance fallback chain.
25+
- Single source of truth for grid placement, shared across renderers.
26+
- First-class mobile gestures with discoverable affordances.
27+
- Clear, glanceable health/recording badges.
28+
29+
**Non-goals**
30+
- New playback engine. We're managing existing HLS/MSE/WebRTC, not adding LL-HLS or DASH.
31+
- Audio mixing across tiles. Continue to mute by default; audio-on is per-tile and exclusive (already today).
32+
- PTZ redesign — that's its own epic.
33+
34+
## 3. Users & top tasks
35+
36+
| Persona | Task | Today's friction |
37+
|---|---|---|
38+
| Phone user reacting to a doorbell ping | Open the app → see front-door tile in 2 taps → expand to full screen | Front door isn't first; fullscreen tap target is small |
39+
| Power user with 8 cameras on a wall display | Arrange tiles, expect arrangement to persist regardless of renderer | Arrangement diverges across HLS/MSE/WebRTC views (#326) |
40+
| Operator on flaky LTE | Force MSE for the wired indoor cams, accept HLS fallback for cellular outdoor cams | Single global toggle; no per-stream choice (#397) |
41+
| Anyone | Tell at a glance "is this tile recording? is the stream healthy?" | Badges are text-only and ambiguous |
42+
43+
## 4. Requirements
44+
45+
### 4.1 Per-stream playback transport preference
46+
47+
Add a new field to `stream_config_t` (and `db_streams`) and a UI control on the Stream Edit modal:
48+
49+
- **`playback_transport`** enum: `auto` (default), `webrtc_only`, `mse_only`, `hls_only`, `webrtc_then_mse`, `mse_then_hls`.
50+
- **`auto`** behavior matches today's: try WebRTC if globally enabled, else MSE, else HLS.
51+
- The Live View tile reads the per-stream value and only attempts the listed transports in order, surfacing a "fallback used" badge when it had to drop down.
52+
53+
UI:
54+
- Stream edit modal grows a "Playback transport" select, default Auto.
55+
- Settings → Streams Defaults gets the same control as a global default.
56+
- The existing global "WebRTC enabled" checkbox becomes a 3-checkbox group (WebRTC / MSE / HLS) per #397, gating which transports the system advertises at all. Per-stream overrides cannot select a globally-disabled transport — UI greys those out with a tooltip.
57+
58+
**Acceptance**: Disable WebRTC globally; per-stream "WebRTC only" overrides surface a clear "WebRTC disabled at server" warning instead of silently degrading.
59+
60+
### 4.2 Unified grid placement model
61+
62+
Today the placement state lives in three independent places (one per renderer). Consolidate:
63+
64+
- Single canonical `grid_layout` per user (or per-device if logged out demo mode), keyed by stream UUID, storing `(slot_index, layout_size)`.
65+
- `LiveView.jsx` reads/writes this single store regardless of active renderer.
66+
- Migrating: on first load after this lands, merge the three legacy stores using last-write-wins per stream.
67+
- Layout-size changes (e.g. 4 → 9 cells) preserve relative order; new slots fill from the top.
68+
- Touch reorder: drag-handle becomes long-press anywhere on the tile (mobile) / corner grip (desktop).
69+
70+
**Acceptance**: User reorders in HLS view, switches to MSE view, sees same arrangement. Resizing browser shrinks/grows the grid without losing user-defined slot assignments where slots remain.
71+
72+
### 4.3 Mobile gestures
73+
74+
| Gesture | Action |
75+
|---|---|
76+
| Tap on tile | Reveal tile chrome (badges, fullscreen, audio toggle) for 4 s |
77+
| Double-tap on tile | Toggle fullscreen |
78+
| Long-press on tile | Open context menu (Mute/Unmute, Snapshot, Open Recordings for this stream, Reorder mode) |
79+
| Pinch-zoom inside fullscreen tile | Digital zoom on the video element (record offset for snapshots) |
80+
| Pull-down at top of grid | Refresh streams list + reload manifests |
81+
| Swipe left/right on a fullscreen tile | Cycle to next/previous stream in current grid order |
82+
83+
Discoverability: first time each gesture would have helped (e.g. user taps fullscreen icon manually 3×), show a one-time tip toast: "Tip: double-tap the tile to fullscreen."
84+
85+
### 4.4 Recording & health badges
86+
87+
Replace the current text-only badge with a 2-glyph layout in the tile's top-right:
88+
89+
- **Recording glyph**: filled red dot (constant), red dot with motion arc (detection), red dot with clock (schedule), or stacked combinations. Tooltip / aria-label spells it out: "Constant + Detection".
90+
- **Health glyph**: green ✓ (streaming, no recent stalls), yellow • (buffering / recovering), red ⚠ (no stream / consecutive errors). Drives off existing `stream_state` from the Health API.
91+
92+
Both glyphs live in a 28×28 pill on desktop, full-size 36×36 on mobile.
93+
94+
**Acceptance**: Fixes #399 specific item ("recording indicator only says 'Enabled (Detection)' even if constant + detection are enabled"). All combinations of the three recording modes render distinctly.
95+
96+
### 4.5 Fullscreen polish
97+
98+
- Native picture-in-picture button on tiles where the browser supports it (Chrome/Edge desktop, Safari mobile iOS 14+).
99+
- "Always-fullscreen on tap" preference (off by default) for kiosk-style wall displays.
100+
- Fullscreen exit gesture: swipe-down on mobile, Esc on desktop. Today's pinch-out collision risk is mitigated by reserving pinch for digital zoom only.
101+
102+
## 5. Data model changes
103+
104+
```
105+
ALTER TABLE streams
106+
ADD COLUMN playback_transport TEXT NOT NULL DEFAULT 'auto'
107+
CHECK (playback_transport IN
108+
('auto','webrtc_only','mse_only','hls_only',
109+
'webrtc_then_mse','mse_then_hls'));
110+
```
111+
112+
`system_settings` adds:
113+
- `transport_webrtc_offered` (bool, default true)
114+
- `transport_mse_offered` (bool, default true)
115+
- `transport_hls_offered` (bool, default true)
116+
- `live_view_grid_layout_v2` (JSON, single source of truth)
117+
118+
Migration: run-once on boot (similar to T14 from the go2rtc work) — merges legacy `grid_layout_webrtc/mse/hls` keys into the new field, last-write-wins.
119+
120+
## 6. Phasing
121+
122+
| Phase | Scope | Estimate |
123+
|---|---|---|
124+
| P0 — Glyph badges + fullscreen polish | Pure-frontend; no schema work | 2 days |
125+
| P1 — Per-stream transport (#397) | Schema + migration + UI + LiveView routing | 3–5 days |
126+
| P2 — Unified grid model (#326) | Migration + LiveView refactor | 3 days |
127+
| P3 — Mobile gestures | Add gesture library, instrumentation for the tip toasts | 4–5 days |
128+
129+
## 7. Acceptance criteria
130+
131+
- A user can set `front_door` to `mse_only` and `garage` to `webrtc_then_mse` independently and Live View honors both.
132+
- Grid arrangement persists across renderer changes (issue #326 closed).
133+
- All recording-mode combinations render with distinct, accessible badges.
134+
- Mobile reachability: every interactive element on Live View is reachable with one thumb on a 6.7" phone in portrait.
135+
136+
## 8. Risks
137+
138+
| Risk | Mitigation |
139+
|---|---|
140+
| Per-stream transport explodes the test matrix | Start with the 3 single-transport modes; chained fallback in P2 |
141+
| Gesture library bloats the bundle | Hand-roll the small set we need (tap, dbltap, longpress, swipe) — no library |
142+
| Grid migration loses placements when slot count differs | Stable-sort by legacy slot then by name, fill empties top-to-bottom; show a one-time "We restored your layout" toast with an Undo |
143+
| iOS Safari fullscreen quirks | Use the existing `FullscreenManager.jsx` with a Safari-specific pseudo-fullscreen path |
144+
145+
## 9. Related issues
146+
147+
- [#326 — Live View grid placement not remembered](https://github.com/opensensor/lightNVR/issues/326)
148+
- [#397 — Per-stream-type checkboxes for dashboard](https://github.com/opensensor/lightNVR/issues/397)
149+
- [#399 — UI and styling issues](https://github.com/opensensor/lightNVR/issues/399) (specifically: logo→home, recording badge clarity, mobile reachability)
Lines changed: 115 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,115 @@
1+
# PRD — Recordings & Timeline UX
2+
3+
**Status**: Draft
4+
**Created**: 2026-04-22
5+
**Owner**: TBD
6+
**Driving signal**: [#331 (CDx4f3kCAf3Y)](https://github.com/opensensor/lightNVR/issues/331), the Recordings/Timeline items from [#399 (AndyIsHereBoi)](https://github.com/opensensor/lightNVR/issues/399).
7+
**Scope**: `web/js/components/preact/RecordingsView.jsx`, `web/js/components/preact/timeline/*` (TimelinePage, TimelineRuler, TimelineCursor, TimelineControls, TimelinePlayer).
8+
9+
---
10+
11+
## 1. Problem
12+
13+
The Recordings list and the Timeline player are the two surfaces operators touch when something *did* happen. Both have rough edges that compound during incident review:
14+
15+
1. **Scrubbing the timeline kills playback.** If you're playing a clip and drag the playhead to look earlier, playback pauses (#399). Operators expect "scrub-and-keep-playing" because every other video player works that way (YouTube, VLC, native iOS).
16+
2. **Stale recordings linger on the timeline ruler.** Background API tells the page about new recordings but never about deletions, so the ruler keeps showing colored bars for clips that are gone (#331). Hard refresh is the only fix today.
17+
3. **No refresh button on Recordings.** Operators waiting for a clip to finalize have no way to manually re-poll without a full page reload (#399).
18+
4. **No async feedback on filters and bulk actions.** Apply a filter, batch-delete, batch-download — UI doesn't show pending state, and a slow query feels broken (#399, general).
19+
5. **Mobile timeline is unusable for fine work.** Cannot pinch-zoom the ruler; cannot fling-scroll; the cursor is hard to grab with a thumb.
20+
21+
## 2. Goals & non-goals
22+
23+
**Goals**
24+
- Scrub continuity: playing state survives ruler interaction.
25+
- Live consistency: the timeline ruler reflects the current set of recordings without a full reload.
26+
- Manual refresh affordances on Recordings.
27+
- Touch-grade gestures on the timeline (pinch-zoom, fling-scroll, larger touch targets).
28+
- Universal async feedback for every filter / batch action.
29+
30+
**Non-goals**
31+
- Cross-stream timeline (multi-camera unified ruler) — separate epic.
32+
- Server-side persistence of UI filter state — a future "saved views" PRD.
33+
- Editing/trimming recordings inside lightNVR.
34+
35+
## 3. Users & top tasks
36+
37+
| Persona | Task | Today's friction |
38+
|---|---|---|
39+
| Homeowner reviewing motion clips on phone | Open Timeline, scrub to the alert time, watch | Scrub pauses; mobile cursor is a 6 px bar |
40+
| Power user auditing yesterday's events | Filter by stream + zone, bulk-download survivors | Filter UI gives no feedback; deletion artifacts persist on the ruler |
41+
| Anyone after a manual delete | See the ruler reflect the deletion immediately | Need to F5; #331 |
42+
43+
## 4. Requirements
44+
45+
### 4.1 Scrub-and-play continuity (#399)
46+
47+
- Track `wasPlaying` state when the user grabs the cursor (pointerdown/touchstart on the cursor or a click anywhere on the ruler).
48+
- During the drag, render an inline preview frame (cheap: nearest keyframe from the manifest) attached to the cursor, but do not pause/resume the player on every move.
49+
- On pointerup/touchend: if `wasPlaying`, seek and immediately call `play()`. If paused going in, stay paused.
50+
- Keyboard parity: `Space` toggles play/pause; `` / `` step ±5 s; `Shift+←` / `Shift+→` step ±60 s. None of those should change the play/pause state.
51+
52+
**Acceptance**: Manual test from #399 — playing a clip, drag the cursor backward 10 s, release, playback continues from the new position with no perceptible re-buffer pause beyond what the network requires.
53+
54+
### 4.2 Timeline ruler stays in sync (#331)
55+
56+
- The TimelinePage already re-fetches the day's recording list on a periodic timer. Today it only adds new entries. Change to a **set-diff**:
57+
- `existing = state.recordings`
58+
- `incoming = api response`
59+
- For each recording in `existing` not in `incoming`: remove from the ruler.
60+
- For each in `incoming` not in `existing`: add to the ruler.
61+
- Diff key: `id`, fall back to `(stream_name, start_ts, end_ts)` if id missing.
62+
- Removal animates briefly (100 ms fade) so the operator notices instead of "huh, did I miss something?"
63+
- Soft-deleted recordings (if/when that's a thing) get a struck-through appearance instead of vanishing.
64+
65+
**Acceptance**: With the Timeline open, run `DELETE /api/recordings/<id>` from a terminal; within the next polling tick (≤ 10 s) the bar disappears from the ruler with the fade animation. Closes #331.
66+
67+
### 4.3 Manual refresh on Recordings (#399)
68+
69+
- Header gets a refresh icon button. Tap or click triggers an immediate re-query with the current filters; uses the `<AsyncButton>` primitive from PRD 01 so the icon spins while pending.
70+
- Mobile: same button is also exposed via pull-to-refresh on the list.
71+
- The query itself doesn't change; only the trigger does.
72+
73+
### 4.4 Universal async feedback
74+
75+
Every list-mutating action goes through the `<AsyncButton>` / `useAsyncAction` primitive (defined in PRD 01):
76+
77+
- "Apply Filters" button — pending state while query runs.
78+
- Batch-Delete confirmation modal — second-step button locks during the request; results render inline ("Deleted 12 of 14; 2 failed: see log").
79+
- Batch-Download — progress bar wired to existing batch-download progress endpoint (`test_batch_delete_progress` already exists for the delete side).
80+
81+
### 4.5 Mobile timeline gestures
82+
83+
- **Pinch-zoom on the ruler** changes the time scale (1 hr → 5 min → 1 min). Cursor stays anchored at its current time.
84+
- **Two-finger pan** (or one-finger fling on the ruler track) horizontally scrolls the visible window.
85+
- **Cursor handle**: 36×36 invisible hit area centered on the visible 4 px line so thumbs can grab it.
86+
- **Snap-to-recording** when the cursor releases within 500 ms of an edge of a recording bar.
87+
88+
## 5. Phasing
89+
90+
| Phase | Scope | Estimate |
91+
|---|---|---|
92+
| P0 — Scrub continuity | TimelineCursor + TimelinePlayer; pure-frontend | 2 days |
93+
| P1 — Set-diff sync | TimelinePage data layer; closes #331 | 1–2 days |
94+
| P2 — Refresh + async feedback | Wire the AsyncButton primitive; assumes PRD 01 P0 has shipped | 1 day |
95+
| P3 — Mobile gestures | Pinch/fling/snap on the ruler | 3–4 days |
96+
97+
## 6. Acceptance criteria
98+
99+
- Closes #331: deleted recordings disappear from ruler within one polling tick.
100+
- Closes the relevant items in #399: scrub continuity, refresh button, async feedback.
101+
- Mobile time-to-cursor (median) < 1 s on a 375 px viewport.
102+
- 0 reported "ghost" recordings on the ruler in beta after P1.
103+
104+
## 7. Risks
105+
106+
| Risk | Mitigation |
107+
|---|---|
108+
| Diffing runs every poll on a busy day → CPU spike | Cap diff candidates to the visible time window; index by id |
109+
| Pinch-zoom collides with browser pinch-zoom on iOS | Use `touch-action: none` on the ruler track; reserve `pinch-zoom` only when both fingers land on the track |
110+
| Scrub-keep-playing causes excessive seeks while dragging | Throttle network seeks to commit only on pointerup; show local preview-frame during drag |
111+
112+
## 8. Related issues
113+
114+
- [#331 — Timeline doesn't auto-remove deleted recordings](https://github.com/opensensor/lightNVR/issues/331)
115+
- [#399 — UI and styling issues](https://github.com/opensensor/lightNVR/issues/399) (Recordings refresh, scrub continuity, async feedback)

web/css/main.css

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -3,6 +3,8 @@
33
/* Import base styles */
44
@import './base.css';
55
@import './components.css';
6+
/* UXD T4 — theme polish: custom scrollbar + reduce-motion rule (#399) */
7+
@import './theme/scrollbar.css';
68

79
/* Additional custom styles */
810
.page {

0 commit comments

Comments
 (0)