Skip to content

Commit c3a98a0

Browse files
Partha-dev01claude
andcommitted
v1.6.0: 10-step flow, age scoring, skip buttons, camera fixes, 30s timers
- Streamline to 10 steps: archive visual-engagement + audio stages - Add age-grouped scoring with domain-aware aggregation (neurotypical → 100%) - Add SkipStageDialog to all 5 assessment stages - Fix Stage 7 camera (render during countdown + stream re-attach) - Fix Stage 10 mobile camera (10s getUserMedia timeout) - Reduce all timed assessments to 30 seconds - Enhanced live transcript in communication stage - 31/31 Playwright tests passing Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
1 parent 1fcaa19 commit c3a98a0

17 files changed

Lines changed: 478 additions & 146 deletions

File tree

DOCS.md

Lines changed: 42 additions & 15 deletions
Original file line numberDiff line numberDiff line change
@@ -11,7 +11,7 @@
1111
- [Architecture](#architecture)
1212
- [Tech Stack](#tech-stack)
1313
- [Feature Map](#feature-map)
14-
- [Intake Flow (12 Steps)](#intake-flow-12-steps)
14+
- [Intake Flow (10 Steps)](#intake-flow-10-steps)
1515
- [AI / ML Pipeline](#ai--ml-pipeline)
1616
- [AWS Services](#aws-services)
1717
- [Authentication](#authentication)
@@ -34,7 +34,7 @@ AutiSense is a Next.js 16 web application that provides AI-powered autism screen
3434
- **Offline-first data** — IndexedDB (Dexie.js) for local storage with DynamoDB sync when online
3535
- **Adaptive therapy** — 7 post-diagnosis games with dynamic difficulty adjustment
3636

37-
The app runs a full 12-step screening flow in ~15 minutes, producing domain scores for gaze, motor, vocalization, and behavioral patterns. No video or audio ever leaves the device.
37+
The app runs a full 10-step screening flow in ~15 minutes, producing domain scores for gaze, motor, vocalization, and behavioral patterns. No video or audio ever leaves the device.
3838

3939
---
4040

@@ -43,7 +43,7 @@ The app runs a full 12-step screening flow in ~15 minutes, producing domain scor
4343
```
4444
Browser (Client)
4545
├── Main Thread (Next.js App Router)
46-
│ ├── 12-step intake flow
46+
│ ├── 10-step intake flow
4747
│ ├── Dashboard + child profiles
4848
│ ├── 7 therapy games
4949
│ ├── Community feed
@@ -96,7 +96,7 @@ Server (Amplify SSR / Lambda)
9696
| Feature | Status | Files |
9797
|---------|--------|-------|
9898
| Landing page | Done | `app/page.tsx` |
99-
| 12-step intake flow | Done | `app/intake/*/page.tsx` (10 pages) |
99+
| 10-step intake flow | Done | `app/intake/*/page.tsx` (10 pages) |
100100
| ONNX video behavioral analysis | Done | `app/intake/video-capture/page.tsx`, `app/lib/inference/*` |
101101
| Session biomarker tracking | Done | `app/lib/db/biomarker.repository.ts` |
102102
| Summary with domain scores | Done | `app/intake/summary/page.tsx` |
@@ -126,22 +126,22 @@ Server (Amplify SSR / Lambda)
126126

127127
---
128128

129-
## Intake Flow (12 Steps)
129+
## Intake Flow (10 Steps)
130130

131131
| Step | Page | What It Tests | Biomarker Output |
132132
|------|------|--------------|-----------------|
133133
| 1 | `/intake/profile` | Parental consent ||
134134
| 2 | `/intake/child-profile` | Child info (name, DOB, language) | Creates session in IndexedDB |
135135
| 3 | `/intake/device-check` | Camera + microphone permissions ||
136-
| 4 | `/intake/communication` | Speech recognition (child's voice) | `vocalizationScore` |
137-
| 5 | `/intake/visual-engagement` | Social vs non-social tap preference | `gazeScore` |
138-
| 6 | `/intake/behavioral-observation` | Free-play bubble pop reaction time | `motorScore`, `responseLatencyMs` |
139-
| 7 | `/intake/preparation` | Dynamic AI voice conversation (Bedrock + Polly + Web Speech API) | `gazeScore` (relevance), `motorScore`, `vocalizationScore`, `responseLatencyMs` |
140-
| 8 | `/intake/motor` | Tap-the-target motor coordination | `motorScore`, `responseLatencyMs` |
141-
| 9 | `/intake/audio` | Audio echo (Polly TTS + SpeechRecognition) | `vocalizationScore` |
142-
| 10 | `/intake/video-capture` | ONNX behavioral video analysis | `gazeScore`, `motorScore`, `asdRiskScore`, behavior classes |
143-
| 11 | `/intake/summary` | Aggregated domain scores from all stages ||
144-
| 12 | `/intake/report` | AI-generated clinical report (Bedrock) | PDF download |
136+
| 4 | `/intake/communication` | Word Echo — speech recognition | `vocalizationScore` |
137+
| 5 | `/intake/behavioral-observation` | Free-play bubble pop reaction time | `motorScore`, `responseLatencyMs` |
138+
| 6 | `/intake/preparation` | Action Challenge — YOLO motor verification | `motorScore`, `responseLatencyMs` |
139+
| 7 | `/intake/motor` | Tap-the-target motor coordination | `motorScore`, `responseLatencyMs` |
140+
| 8 | `/intake/video-capture` | ONNX behavioral video analysis | `gazeScore`, `motorScore`, `asdRiskScore`, behavior classes |
141+
| 9 | `/intake/summary` | Aggregated domain scores from all stages | |
142+
| 10 | `/intake/report` | AI-generated clinical report (Bedrock) | PDF download |
143+
144+
> **Archived stages** (files kept, removed from navigation): Visual Engagement (`/intake/visual-engagement`), Audio Assessment (`/intake/audio`)
145145
146146
---
147147

@@ -280,7 +280,7 @@ Difficulty engine (`app/lib/games/difficultyEngine.ts`) auto-adjusts based on re
280280

281281
| File | Tests | Coverage |
282282
|------|-------|----------|
283-
| `tests/intake-flow.spec.ts` | 15 | Full 12-step intake flow navigation, form validation, back buttons |
283+
| `tests/intake-flow.spec.ts` | 15 | Full 10-step intake flow navigation, form validation, back buttons, skip stage |
284284
| `tests/app-pages.spec.ts` | 15 | Auth, dashboard, all 7 games, feed, 4 API endpoints |
285285
| **Total** | **30** | **All passing** |
286286

@@ -569,3 +569,30 @@ npx playwright test # Run all 30 tests
569569
- Updated: `app/lib/actions/actionDetector.ts` (consecutiveHits in tracker return)
570570
- Updated: `tests/intake-flow.spec.ts` (Step 4, 7, 9 test assertions)
571571
- Updated: `tests/app-pages.spec.ts` (generate-words API test)
572+
573+
### v1.6.0 — 2026-03-05 (10-Step Flow, Age Scoring, Skip Buttons, Camera Fixes)
574+
575+
**Major Changes:**
576+
- **Streamlined to 10-step flow**: Archived Visual Engagement (emoji tap) and Audio Assessment (sentence echo) stages. Navigation rewired to skip them — files preserved for potential future use.
577+
- **Age-grouped scoring**: New `ageNormalization.ts` with 4 age brackets (12-24mo, 24-48mo, 48-72mo, 72+mo). Younger children get relaxed multipliers (e.g., 12-24mo: gaze×1.4, motor×1.5, vocal×1.6) and lower DSM-5 flag thresholds. A neurotypical child now scores ~93-100% instead of ~72%.
578+
- **Domain-aware aggregation**: Each task only contributes to domains it actually measures (e.g., communication → vocal only, motor → motor only). Hardcoded 0.5 placeholders no longer drag down unrelated domain scores. Default 0.75 for unmeasured domains.
579+
- **Skip Stage on all assessments**: New `SkipStageDialog` component added to all 5 assessment stages (communication, behavioral-observation, preparation, motor, video-capture). Shows confirmation modal before skipping. Saves default 0.5 biomarkers on skip.
580+
581+
**Fixed:**
582+
- **Stage 7 camera not showing on desktop or mobile**: Video element was only rendered during `actionPhase === "detecting"` but camera stream was assigned during countdown when element didn't exist. Now renders camera during all active phases (countdown + detecting + detected) with countdown overlay on top.
583+
- **Stage 10 mobile camera hanging**: Added 10-second timeout wrapper (`withTimeout()`) to all `getUserMedia()` calls in `cameraUtils.ts`. Prevents indefinite hanging when mobile browsers stall on camera permission.
584+
- **Stream re-attach on DOM mount**: Defensive `useEffect` in `useActionCamera.ts` re-attaches stream when video element appears in DOM (catches ref timing issues).
585+
586+
**Improved:**
587+
- **All timed assessments reduced to 30 seconds**: Behavioral observation (was 60s), motor assessment (was 45s), and video capture (was 120s) all now run for 30 seconds. Video capture criteria gate lowered to 3 samples / 15s (was 5 samples / 30s).
588+
- **Live transcript in communication stage**: Transcript display moved outside `listening` state — now visible during listening/matched/missed. Larger font (1.4rem Fredoka) with "Heard:" label and pulse animation during active listening.
589+
590+
**Files:**
591+
- Created: `app/lib/scoring/ageNormalization.ts` (age groups, multipliers, thresholds)
592+
- Created: `app/components/SkipStageDialog.tsx` (shared skip confirmation)
593+
- Modified: `app/lib/camera/cameraUtils.ts` (10s timeout wrapper)
594+
- Modified: `app/hooks/useActionCamera.ts` (stream re-attach effect)
595+
- Modified: `app/lib/db/biomarker.repository.ts` (domain-aware + age-normalized aggregation)
596+
- Modified: All 10 active intake pages (STEPS array, step counts, navigation links, skip buttons)
597+
- Updated: `tests/intake-flow.spec.ts` (10-step flow, skip button tests)
598+
- Updated: `DOCS.md` (v1.6.0 changelog, 10-step flow docs)

app/components/SkipStageDialog.tsx

Lines changed: 82 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,82 @@
1+
"use client";
2+
import { useState } from "react";
3+
4+
interface SkipStageDialogProps {
5+
onConfirm: () => void;
6+
}
7+
8+
export default function SkipStageDialog({ onConfirm }: SkipStageDialogProps) {
9+
const [showConfirm, setShowConfirm] = useState(false);
10+
11+
if (!showConfirm) {
12+
return (
13+
<button
14+
onClick={() => setShowConfirm(true)}
15+
className="btn btn-outline"
16+
style={{
17+
position: "absolute",
18+
top: 16,
19+
right: 16,
20+
minHeight: 36,
21+
padding: "6px 14px",
22+
fontSize: "0.8rem",
23+
color: "var(--text-muted)",
24+
zIndex: 10,
25+
}}
26+
>
27+
Skip Stage
28+
</button>
29+
);
30+
}
31+
32+
return (
33+
<div style={{
34+
position: "fixed",
35+
top: 0, left: 0, right: 0, bottom: 0,
36+
background: "rgba(0,0,0,0.4)",
37+
display: "flex", alignItems: "center", justifyContent: "center",
38+
zIndex: 100,
39+
padding: 20,
40+
}}>
41+
<div className="card" style={{
42+
padding: "28px 24px",
43+
maxWidth: 380,
44+
textAlign: "center",
45+
}}>
46+
<h3 style={{
47+
fontFamily: "'Fredoka',sans-serif",
48+
fontWeight: 600,
49+
fontSize: "1.1rem",
50+
marginBottom: 10,
51+
}}>
52+
Skip this stage?
53+
</h3>
54+
<p style={{
55+
fontSize: "0.88rem",
56+
color: "var(--text-secondary)",
57+
lineHeight: 1.6,
58+
marginBottom: 20,
59+
}}>
60+
Results won't be collected for this activity.
61+
The screening will continue with default values.
62+
</p>
63+
<div style={{ display: "flex", gap: 12, justifyContent: "center" }}>
64+
<button
65+
className="btn btn-outline"
66+
onClick={() => setShowConfirm(false)}
67+
style={{ minHeight: 40, padding: "8px 20px" }}
68+
>
69+
Cancel
70+
</button>
71+
<button
72+
className="btn btn-primary"
73+
onClick={onConfirm}
74+
style={{ minHeight: 40, padding: "8px 20px" }}
75+
>
76+
Skip Stage
77+
</button>
78+
</div>
79+
</div>
80+
</div>
81+
);
82+
}

app/hooks/useActionCamera.ts

Lines changed: 11 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -240,6 +240,17 @@ export function useActionCamera(): UseActionCameraReturn {
240240
detectingRef.current = false;
241241
}, []);
242242

243+
// Defensive: re-attach stream when video element appears in DOM
244+
// (no deps — runs every render to catch ref changes)
245+
useEffect(() => {
246+
const video = videoRef.current;
247+
const stream = streamRef.current;
248+
if (video && stream && !video.srcObject) {
249+
video.srcObject = stream;
250+
video.play().catch(() => {});
251+
}
252+
});
253+
243254
// Cleanup on unmount
244255
useEffect(() => {
245256
return () => {

app/intake/behavioral-observation/page.tsx

Lines changed: 24 additions & 8 deletions
Original file line numberDiff line numberDiff line change
@@ -4,12 +4,13 @@ import { useState, useEffect, useRef, useCallback } from "react";
44
import { useRouter } from "next/navigation";
55
import { addBiomarker } from "../../lib/db/biomarker.repository";
66
import { getCurrentSessionId } from "../../lib/session/currentSession";
7+
import SkipStageDialog from "../../components/SkipStageDialog";
78

89
const STEPS = [
9-
"Welcome", "Profile", "Device", "Communicate", "Visual", "Behavior",
10-
"Prepare", "Motor", "Audio", "Video", "Summary", "Report",
10+
"Welcome", "Profile", "Device", "Communicate", "Behavior",
11+
"Prepare", "Motor", "Video", "Summary", "Report",
1112
];
12-
const STEP_IDX = 5;
13+
const STEP_IDX = 4;
1314

1415
interface Bubble {
1516
id: number;
@@ -32,7 +33,7 @@ export default function BehavioralObservationPage() {
3233
const [taskComplete, setTaskComplete] = useState(false);
3334
const [bubbles, setBubbles] = useState<Bubble[]>([]);
3435
const [score, setScore] = useState(0);
35-
const [timeLeft, setTimeLeft] = useState(60);
36+
const [timeLeft, setTimeLeft] = useState(30);
3637
const [popTimes, setPopTimes] = useState<number[]>([]);
3738
const nextIdRef = useRef(0);
3839
const timerRef = useRef<ReturnType<typeof setInterval> | null>(null);
@@ -101,6 +102,20 @@ export default function BehavioralObservationPage() {
101102
setBubbles((prev) => prev.filter((b) => b.id !== id));
102103
};
103104

105+
const handleSkipStage = useCallback(async () => {
106+
if (timerRef.current) clearInterval(timerRef.current);
107+
if (spawnRef.current) clearInterval(spawnRef.current);
108+
const sid = getCurrentSessionId();
109+
if (sid) {
110+
await addBiomarker(sid, "behavioral_observation", {
111+
gazeScore: 0.5,
112+
motorScore: 0.5,
113+
vocalizationScore: 0.5,
114+
}).catch(() => {});
115+
}
116+
router.push("/intake/preparation");
117+
}, [router]);
118+
104119
const avgPopTime = popTimes.length > 1
105120
? Math.round(popTimes.slice(1).reduce((a, b) => a + b, 0) / (popTimes.length - 1))
106121
: 0;
@@ -114,7 +129,7 @@ export default function BehavioralObservationPage() {
114129
{theme === "light" ? "🌙" : "☀️"}
115130
</button>
116131
<span style={{ fontSize: "0.88rem", color: "var(--text-muted)", fontWeight: 600 }}>
117-
Step {STEP_IDX + 1} of 12
132+
Step {STEP_IDX + 1} of 10
118133
</span>
119134
</div>
120135
</nav>
@@ -132,14 +147,15 @@ export default function BehavioralObservationPage() {
132147
</div>
133148
</div>
134149

135-
<main className="main">
150+
<main className="main" style={{ position: "relative" }}>
151+
<SkipStageDialog onConfirm={handleSkipStage} />
136152
<div className="fade fade-1" style={{ textAlign: "center", marginBottom: 28 }}>
137153
<div className="breathe-orb" style={{ margin: "0 auto" }}>
138154
<div className="breathe-inner">🫧</div>
139155
</div>
140156
</div>
141157

142-
<div className="chip fade fade-1">Step 6 — Free Play</div>
158+
<div className="chip fade fade-1">Step 5 — Free Play</div>
143159
<h1 className="page-title fade fade-2">
144160
Pop the <em>bubbles!</em>
145161
</h1>
@@ -212,7 +228,7 @@ export default function BehavioralObservationPage() {
212228
)}
213229

214230
<div className="fade fade-4" style={{ display: "flex", gap: 12, marginTop: 28 }}>
215-
<Link href="/intake/visual-engagement" className="btn btn-outline" style={{ minWidth: 100 }}>
231+
<Link href="/intake/communication" className="btn btn-outline" style={{ minWidth: 100 }}>
216232
← Back
217233
</Link>
218234
<button className="btn btn-primary btn-full" disabled={!taskComplete}

app/intake/child-profile/page.tsx

Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -6,8 +6,8 @@ import { createSession } from "../../lib/db/session.repository";
66
import { setCurrentSessionId } from "../../lib/session/currentSession";
77

88
const STEPS = [
9-
"Welcome", "Profile", "Device", "Communicate", "Visual", "Behavior",
10-
"Prepare", "Motor", "Audio", "Video", "Summary", "Report",
9+
"Welcome", "Profile", "Device", "Communicate", "Behavior",
10+
"Prepare", "Motor", "Video", "Summary", "Report",
1111
];
1212

1313
const LANGUAGES = [
@@ -131,7 +131,7 @@ export default function ProfilePage() {
131131
fontWeight: 600,
132132
}}
133133
>
134-
Step 2 of 12
134+
Step 2 of 10
135135
</span>
136136
</div>
137137
</nav>

0 commit comments

Comments
 (0)