Skip to content

Commit 53896d7

Browse files
jjoonleojjoonleo
authored
docs: add Android alarm notification QA runbook (#457) (#491)
1 parent 3a42ced commit 53896d7

3 files changed

Lines changed: 265 additions & 0 deletions

File tree

‎docs/Android-Alarm-Notification-QA.md‎

Lines changed: 208 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,208 @@
1+
# Android Alarm And Notification QA
2+
3+
Use this runbook for release issue #457 after the app can be installed from a
4+
signed release AAB or release-equivalent Android build.
5+
6+
## Blockers
7+
8+
Do not mark #457 complete until these are satisfied:
9+
10+
- #452 provides a signed release AAB or release-equivalent install path.
11+
- #445 full-screen intent declaration material is reviewed or submitted by a
12+
human with Play Console access, per the parent track order.
13+
- QA has access to an Android device or emulator with Google Play services, a
14+
backend environment with alarm APIs enabled, and a test account that can create
15+
schedules.
16+
17+
## Source Context
18+
19+
- Android permissions: `docs/Android-Manifest-Permissions.md`
20+
- General native alarm checklist: `plans/native_alarm_platform_qa_checklist.md`
21+
- Native Android bridge: `android/app/src/main/kotlin/club/devkor/ontime/`
22+
- Dart reconciliation: `lib/domain/use-cases/reconcile_alarms_use_case.dart`
23+
- Alarm provider/status values: `lib/domain/entities/alarm_entities.dart`
24+
25+
## Device Matrix
26+
27+
Record every device used.
28+
29+
| Device | Android version | Install source | Backend env | Tester | Result |
30+
| --- | --- | --- | --- | --- | --- |
31+
| TBD | TBD | signed release AAB or release-equivalent install | TBD | TBD | TBD |
32+
33+
Minimum recommended coverage:
34+
35+
- Android 13 or newer for `POST_NOTIFICATIONS`.
36+
- Android 12 or newer for exact alarm special access.
37+
- One physical device for boot restore and full-screen alarm behavior.
38+
39+
## Permission Setup
40+
41+
Package name:
42+
43+
```sh
44+
PACKAGE=club.devkor.ontime
45+
```
46+
47+
Notification permission on Android 13+:
48+
49+
```sh
50+
adb shell pm grant "$PACKAGE" android.permission.POST_NOTIFICATIONS
51+
adb shell pm revoke "$PACKAGE" android.permission.POST_NOTIFICATIONS
52+
```
53+
54+
Exact alarm special access on Android 12+:
55+
56+
```sh
57+
adb shell appops set "$PACKAGE" SCHEDULE_EXACT_ALARM allow
58+
adb shell appops set "$PACKAGE" SCHEDULE_EXACT_ALARM deny
59+
adb shell appops get "$PACKAGE" SCHEDULE_EXACT_ALARM
60+
```
61+
62+
If the shell command is unavailable or ignored on the device, use Android
63+
Settings > Apps > Special app access > Alarms and reminders, then relaunch
64+
OnTime and re-run the scenario.
65+
66+
Avoid `pm clear` unless the tester intentionally wants to wipe local app data
67+
for the test account.
68+
69+
## Test Data
70+
71+
For each scenario, create or reuse a schedule whose alarm time is soon enough to
72+
observe without waiting through a long real-world route. Record:
73+
74+
| Field | Value |
75+
| --- | --- |
76+
| Schedule ID | TBD |
77+
| Schedule title | TBD |
78+
| Schedule time | TBD |
79+
| Expected preparation start/alarm time | TBD |
80+
| Account | TBD |
81+
| Backend environment | TBD |
82+
| App version/build | TBD |
83+
84+
## Scenarios
85+
86+
### 1. Notification Permission Granted
87+
88+
1. Grant `POST_NOTIFICATIONS`.
89+
2. Launch OnTime and sign in.
90+
3. Confirm notification permission copy is not blocking the app.
91+
4. Create a near-future schedule and allow alarm reconciliation to run.
92+
5. Record whether fallback or native alarm notifications can be shown.
93+
94+
Expected result: notification permission is treated as granted, and fallback
95+
notifications are available when native alarm scheduling cannot be used.
96+
97+
### 2. Notification Permission Denied
98+
99+
1. Revoke `POST_NOTIFICATIONS`.
100+
2. Launch OnTime and sign in.
101+
3. Exercise the allow-notification or My Page recovery path.
102+
4. Create a near-future schedule and allow alarm reconciliation to run.
103+
104+
Expected result: the app explains why schedule preparation reminders need
105+
notifications and does not post fallback notifications while permission is
106+
denied.
107+
108+
### 3. Exact Alarm Permission Granted
109+
110+
1. Allow `SCHEDULE_EXACT_ALARM`.
111+
2. Grant `POST_NOTIFICATIONS`.
112+
3. Create a near-future schedule.
113+
4. Background the app before the expected alarm time.
114+
5. Wait for the alarm.
115+
116+
Expected result: the native Android alarm provider is used, the alarm fires at
117+
the expected time, and the alarm path opens schedule preparation.
118+
119+
### 4. Exact Alarm Permission Denied With Notification Granted
120+
121+
1. Deny `SCHEDULE_EXACT_ALARM`.
122+
2. Grant `POST_NOTIFICATIONS`.
123+
3. Relaunch OnTime or toggle schedule alarms off and on in My Page.
124+
4. Create a near-future schedule and wait for the alarm time.
125+
126+
Expected result: the app explains precise alarm timing when recovery is needed,
127+
uses `localNotification` fallback where available, and records/reporting should
128+
show fallback behavior instead of native exact alarm scheduling.
129+
130+
### 5. Exact Alarm And Notification Denied
131+
132+
1. Deny `SCHEDULE_EXACT_ALARM`.
133+
2. Revoke `POST_NOTIFICATIONS`.
134+
3. Relaunch OnTime or toggle schedule alarms off and on in My Page.
135+
4. Create a near-future schedule.
136+
137+
Expected result: no native alarm or fallback notification is armed; the app
138+
surfaces permission recovery instead of silently claiming alarm coverage.
139+
140+
### 6. Full-Screen Alarm UI
141+
142+
1. Allow exact alarm special access and notification permission.
143+
2. Create a near-future schedule.
144+
3. Lock the device or put the app in the background.
145+
4. Wait for the alarm.
146+
5. Use the full-screen alarm action to start preparing.
147+
6. Repeat once and dismiss/cancel the ringing alarm instead of starting.
148+
149+
Expected result: `AlarmRingingActivity` appears only for the user-scheduled
150+
alarm moment, the start action routes into schedule preparation, and dismissal
151+
stops ringing/vibration and removes the alarm notification.
152+
153+
### 7. Cancellation
154+
155+
1. Create a near-future schedule with permissions granted.
156+
2. Before the alarm time, start preparation early, finish the schedule, delete
157+
the schedule, or disable schedule alarms in My Page.
158+
3. Wait past the original alarm time.
159+
160+
Expected result: no alarm or fallback notification fires after cancellation.
161+
162+
### 8. Boot Restore
163+
164+
1. Allow exact alarm special access and notification permission.
165+
2. Create a future schedule far enough out to reboot the device.
166+
3. Restart the device.
167+
4. Unlock and wait for the expected alarm time. If practical, also open OnTime
168+
once after boot to trigger app reconciliation.
169+
170+
Expected result: future native alarms are restored where Android permits boot
171+
restore, and any failure is recorded with the device model and Android version.
172+
173+
## Evidence Form
174+
175+
Attach this evidence to #457 or the release QA record.
176+
177+
| Item | Evidence |
178+
| --- | --- |
179+
| Device model | TBD |
180+
| Android version/API | TBD |
181+
| App version/build number | TBD |
182+
| Install source or artifact path | TBD |
183+
| Backend environment | TBD |
184+
| Test account | TBD |
185+
| Notification granted result | TBD |
186+
| Notification denied result | TBD |
187+
| Exact alarm granted result | TBD |
188+
| Exact alarm denied + notification granted result | TBD |
189+
| Exact alarm denied + notification denied result | TBD |
190+
| Full-screen alarm UI result | TBD |
191+
| Fallback notification result | TBD |
192+
| Cancellation result | TBD |
193+
| Boot restore result | TBD |
194+
| Failures and logs | TBD |
195+
| Screenshots/recordings | TBD |
196+
197+
## Failure Triage
198+
199+
For any failure, record:
200+
201+
- Scenario name and permission state.
202+
- Schedule ID, expected alarm time, actual observed time, and timezone.
203+
- Whether the app was foregrounded, backgrounded, terminated, locked, or after
204+
reboot.
205+
- Screenshot or screen recording.
206+
- Relevant `adb logcat` excerpt filtered to `OnTime`, `Alarm`, or
207+
`club.devkor.ontime`, without personal account data.
208+
- Whether the issue reproduces after reinstalling the same artifact.

‎docs/Release-Checklist.md‎

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -100,6 +100,8 @@ flutter build appbundle --release
100100
- Review `docs/Android-Manifest-Permissions.md` and confirm app-owned release
101101
manifest permissions are still limited to user-facing notification, exact
102102
alarm, full-screen alarm, boot restore, and vibration behavior.
103+
- Run Android alarm and notification device QA with
104+
`docs/Android-Alarm-Notification-QA.md` before closing release issue #457.
103105
- Provide release signing through one of the following secret-free paths:
104106
- `android/key.properties` with `storeFile`, `storePassword`, `keyAlias`, and
105107
`keyPassword`.

‎plans/457-android-alarm-notification-qa.md‎

Lines changed: 55 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,55 @@
1+
# Issue 457 Android Alarm And Notification QA Plan
2+
3+
Issue: #457 - [Release] QA alarm and notification flows on Android
4+
Parent track: #465 - Android permissions and alarm policy
5+
6+
## Current Decision
7+
8+
Issue #457 remains externally blocked. Do not claim closure until a human can run
9+
the Android device QA against a signed release AAB or release-equivalent install.
10+
11+
Repo-side work that can legitimately advance the issue now:
12+
13+
- Provide a scoped Android alarm and notification QA runbook.
14+
- Provide an evidence template that maps directly to #457 acceptance criteria.
15+
- Link the runbook from the release checklist so the manual release flow can find
16+
it.
17+
18+
Repo-side work that should not be done for #457 now:
19+
20+
- Do not mark #457 complete without device evidence.
21+
- Do not fake signed release, Play Console, backend, or device verification.
22+
- Do not change alarm product behavior unless QA finds a reproducible repo bug.
23+
24+
## Prerequisite State
25+
26+
- #442 is closed by PR #470. Android manifest permissions were audited.
27+
- #443 is closed by PR #469. Notification permission UX was verified and tested.
28+
- #444 is closed by PR #472. Exact alarm permission UX was improved and tested.
29+
- #452 is still open and blocked. A signed release AAB or release-equivalent
30+
install path is still required for this issue.
31+
- #445 is still open/manual in the parent track order. Play Console full-screen
32+
intent declaration work requires human account access.
33+
34+
## Plan
35+
36+
1. Add `docs/Android-Alarm-Notification-QA.md` with prerequisites, device matrix,
37+
permission-state setup commands, QA scenarios, and evidence fields.
38+
2. Link the runbook from `docs/Release-Checklist.md`.
39+
3. Verify documentation formatting and confirm the git diff is limited to #457
40+
artifacts.
41+
4. Commit the docs-only change with a Conventional Commit referencing #457.
42+
5. Open a draft PR that advances #457 but clearly states the remaining human QA
43+
and external blockers.
44+
45+
## Human Completion Criteria
46+
47+
The release owner or QA tester must attach evidence for:
48+
49+
- Notification permission granted and denied states.
50+
- Exact alarm permission granted and denied states.
51+
- Native alarm firing, full-screen alarm UI, fallback notification behavior, and
52+
cancellation.
53+
- Boot restore after device restart where practical.
54+
- Device model, Android version, install artifact, backend environment, account,
55+
schedule IDs, expected alarm times, results, and failures.

0 commit comments

Comments
 (0)