Skip to content

Commit 1b5c05d

Browse files
jjoonleojjoonleo
authored
docs: add Android release smoke test runbook (#456) (#488)
1 parent dafa016 commit 1b5c05d

4 files changed

Lines changed: 165 additions & 0 deletions

File tree

‎docs/Android-Release-Smoke-Test.md‎

Lines changed: 127 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,127 @@
1+
# Android Release Smoke Test
2+
3+
Use this runbook for the release-device smoke test tracked by #456. It must be
4+
run against a signed release build installed through Google Play Internal
5+
Testing or another release-equivalent install path after #452 has produced the
6+
installable artifact.
7+
8+
Do not close #456 from documentation alone. Close it only after a human tester
9+
records a completed pass with device, build, endpoint, and result evidence.
10+
11+
## Entry Criteria
12+
13+
- #452 is complete and the signed Android release build is available.
14+
- The tested build uses package `club.devkor.ontime`.
15+
- The build was installed from Play Internal Testing, a Play-generated APK set,
16+
or an equivalent signed release install path agreed by the release owner.
17+
- The tester has a real Android device, a test account, and access to the
18+
release API/backend signal needed to confirm endpoint and token behavior.
19+
- The release owner has confirmed which API base URL is approved for the smoke
20+
test, such as the staging URL for internal testing or the production URL for a
21+
tagged production candidate.
22+
23+
## Build And Device Evidence
24+
25+
Record this before testing:
26+
27+
```md
28+
Issue: #456
29+
Parent track: #467
30+
Tester:
31+
Date:
32+
Install source: Play Internal Testing / signed release APK / other:
33+
Artifact or Play release link:
34+
Package name:
35+
Version name:
36+
Version code:
37+
Git commit or workflow run:
38+
ENV dart define:
39+
REST_API_URL value or approved endpoint name:
40+
Firebase project:
41+
Device model:
42+
Android version:
43+
Network:
44+
Fresh install: yes / no
45+
Existing app data cleared before test: yes / no
46+
```
47+
48+
## Required Smoke Pass
49+
50+
Mark each result `pass`, `fail`, or `blocked`. Attach screenshots, screen
51+
recordings, logs, backend request IDs, or Play/Firebase evidence for failures
52+
and for any endpoint verification that cannot be inferred from the app UI.
53+
54+
| Area | Steps | Expected result | Result | Evidence |
55+
| --- | --- | --- | --- | --- |
56+
| Install | Install the signed release build from the approved release-equivalent path. | App installs without sideload, signing, package, or Play Protect errors. | | |
57+
| First launch | Launch the app from a fresh install. | App opens without a crash, blank screen, Firebase init error, or missing config error. | | |
58+
| Login | Sign in with the test account using the release build. | Login succeeds and lands on the expected authenticated screen. | | |
59+
| Endpoint | Confirm the build is using the approved release API endpoint. | Backend logs, request IDs, or app configuration evidence match the approved endpoint. | | |
60+
| FCM token | Confirm fresh install token registration reaches the backend if backend access is available. | Backend receives token registration for the tested account/device. | | |
61+
| Schedule create | Create a schedule with a place, moving time, preparation, and future start time. | The schedule is saved and visible in the app. | | |
62+
| Schedule edit | Edit the created schedule. | Updated values persist after returning to the schedule list/detail. | | |
63+
| Schedule delete | Delete the created schedule. | The schedule is removed and does not reappear after refresh or app restart. | | |
64+
| My Page | Open My Page and review visible account/settings content. | Screen loads without release-only errors and expected content is present. | | |
65+
| Privacy policy | Open the privacy policy link from the app. | The link opens the approved privacy policy URL without a broken or placeholder page. | | |
66+
| Logout | Log out, then relaunch the app. | Session is cleared and the app returns to the signed-out flow. | | |
67+
| Relaunch | Force close and reopen after the smoke flow. | App starts cleanly and remains in the expected auth state. | | |
68+
69+
## Failure Notes
70+
71+
For every failed or blocked row, record:
72+
73+
```md
74+
Area:
75+
Observed behavior:
76+
Expected behavior:
77+
Reproduction steps:
78+
Device and Android version:
79+
Version name and version code:
80+
Timestamp with timezone:
81+
Screenshots or recording:
82+
Relevant logs:
83+
Backend request ID or dashboard link:
84+
Decision: retest after fix / accept risk / block release
85+
Owner:
86+
```
87+
88+
## Completion Note Template
89+
90+
Post a completion note on #456 with this structure:
91+
92+
```md
93+
Android release smoke test result: pass / failed / blocked
94+
95+
Build:
96+
- Version name:
97+
- Version code:
98+
- Commit or workflow run:
99+
- Install source:
100+
- Artifact or Play release link:
101+
- ENV:
102+
- REST_API_URL value or approved endpoint name:
103+
104+
Device:
105+
- Model:
106+
- Android version:
107+
- Network:
108+
109+
Results:
110+
- Install:
111+
- First launch:
112+
- Login:
113+
- Endpoint verification:
114+
- FCM token registration:
115+
- Schedule create/edit/delete:
116+
- My Page:
117+
- Privacy policy link:
118+
- Logout:
119+
- Relaunch:
120+
121+
Issues found:
122+
- None, or link issue numbers with reproduction notes.
123+
124+
Evidence:
125+
- Screenshots/recordings/log links:
126+
- Backend/Play/Firebase evidence:
127+
```

‎docs/Home.md‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -14,6 +14,7 @@ Welcome to the OnTime-front project documentation! This wiki contains everything
1414
- [Android Play Signing Fingerprints](./Android-Play-Signing-Fingerprints.md) - Play App Signing, Firebase, and auth-provider fingerprint checklist
1515
- [Android Release Configuration](./Android-Release-Configuration.md) - Firebase config, signed AAB workflow, and Play internal deploy setup
1616
- [iOS Release Configuration](./iOS-Release-Configuration.md) - Required Dart defines and archive validation
17+
- [Android Release Smoke Test](./Android-Release-Smoke-Test.md) - Device smoke-test runbook and evidence template for signed Android release builds
1718
- [Privacy Policy Hosting](./Privacy-Policy-Hosting.md) - Public HTTPS privacy policy hosting checklist and evidence form
1819
- [Google Play Data Safety Worksheet](./Google-Play-Data-Safety.md) - Source-backed Data safety evidence and pending owner inputs
1920
- [Play Pre-Launch Report](./Play-Pre-Launch-Report.md) - Google Play report runbook, triage gate, and evidence template

‎docs/Release-Checklist.md‎

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -122,6 +122,9 @@ flutter build appbundle --release
122122
`club.devkor.ontime`.
123123
- Keep keystores, passwords, and `key.properties` out of git.
124124
- Verify the release build fails clearly when signing secrets are missing.
125+
- After a signed release build is available, run the device smoke test from
126+
`docs/Android-Release-Smoke-Test.md` and record the tested device, Android
127+
version, build version, approved API endpoint, and pass/fail evidence.
125128
- After the signed AAB is uploaded to internal or closed testing, run the Google
126129
Play pre-launch report and record evidence using
127130
`docs/Play-Pre-Launch-Report.md` before widening release availability.

‎plans/issue-456-android-release-smoke-test.md‎

Lines changed: 34 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,34 @@
1+
# Issue 456 Android Release Smoke Test Plan
2+
3+
## Scope
4+
5+
Issue #456 is limited to proving that a signed Android release build works on a
6+
real Android device before Play review. The required flow covers install, first
7+
launch, login, logout, schedule create/edit/delete, My Page, privacy policy
8+
link, approved release API endpoint verification, and recording device/build
9+
evidence.
10+
11+
## Current Decision
12+
13+
The issue remains externally blocked. The direct prerequisite, #452, is open and
14+
blocked until #453 locks production versioning. Without a signed release AAB or
15+
release-equivalent install path, a real Android device, release API/backend
16+
visibility, and Play/internal-testing access, this thread cannot honestly
17+
complete the smoke test.
18+
19+
## Repo-Side Action
20+
21+
Add a release smoke-test runbook and evidence template so the human tester can
22+
complete #456 as soon as #452 produces the signed build. Keep implementation to
23+
documentation; do not alter app behavior or release workflows for this issue.
24+
25+
## Human Completion Steps
26+
27+
1. Complete #453 and then #452 so a signed installable Android release build is
28+
available.
29+
2. Install the build through Play Internal Testing or another approved
30+
release-equivalent path.
31+
3. Run `docs/Android-Release-Smoke-Test.md` on a real Android device.
32+
4. Record device, Android version, version name, version code, install source,
33+
endpoint evidence, and pass/fail results on #456.
34+
5. Open follow-up bugs for any failed smoke-test row before Play review.

0 commit comments

Comments
 (0)