Skip to content

Commit d63dfff

Browse files
authored
Merge pull request #928 from synonymdev/chore/fix-ai-pr-tests-gen
2 parents fc8e50a + 0ac9ad4 commit d63dfff

4 files changed

Lines changed: 64 additions & 18 deletions

File tree

Lines changed: 58 additions & 17 deletions
Original file line numberDiff line numberDiff line change
@@ -122,31 +122,72 @@ This PR adds support for...
122122
When the user provides custom instructions after `--`:
123123
- Parse any referenced commit SHAs and read their full messages
124124
- Focus the description content on areas the user emphasizes
125-
- Structure QA Notes according to user's specific testing instructions
125+
- Structure QA Notes according to user's specific manual testing instructions and automated coverage notes
126126
- Custom instructions take priority over default generation rules for sections they address
127-
- Preserve exact testing steps provided by the user (don't summarize or omit details)
128-
129-
**QA Notes / Testing Scenarios:**
130-
- Structure with numbered headings and steps
131-
- Make steps easily referenceable
132-
- Be specific about what to test and expected outcomes
127+
- Preserve exact manual testing steps provided by the user (don't summarize or omit details)
128+
- If custom instructions include automated checks or coverage notes, place them under `#### Automated Checks`
129+
130+
**QA Notes / Validation:**
131+
- QA Notes separate actionable human QA instructions from automated verification coverage.
132+
- Always use this structure:
133+
```md
134+
### QA Notes
135+
#### Manual Tests
136+
#### Automated Checks
137+
```
138+
- Keep local verification commands, Gradle tasks, detekt, lint, unit tests, build passes, cargo test, cargo clippy, npm test, typecheck, CI coverage, or similar automated checks out of `#### Manual Tests`; summarize them under `#### Automated Checks` when they add useful context.
139+
- Use `#### Automated Checks` to summarize automated verification evidence, prioritizing coverage added, modified, or removed with file paths and a short explanation.
140+
- For removed automated coverage, state why it was removed.
141+
- Do not list standard CI or PR bot commands as checkbox items just because they run for every PR. If standard CI coverage is worth mentioning, summarize it in one sentence.
142+
- List raw commands only when they were run locally, are non-standard, use special flags or environment values, validate workflow behavior, or explain a meaningful verification gap.
143+
- For workflow behavior validation, include `(after merge)` in the automated check item because workflow changes only take effect for PRs opened after the workflow update merges.
144+
- If no actionable manual validation exists, write `N/A` under `#### Manual Tests`.
145+
- If no automated checks were run and no automated coverage changed, write `N/A` under `#### Automated Checks`.
146+
- Write manual tests using this template:
147+
```md
148+
- [ ] **{numbering}.** {optional_condition + →} {screen_action} → {next_screen_action}: expectation
149+
```
150+
- Use a list of unchecked checkboxes for each individual test.
151+
- Use a numbered prefix for each test, in bold, for example `**1.**`, `**2.**`.
152+
- Use `regression:` for regression checks, positioned after the numbering.
153+
- Use sub-lists for variations of the same test.
154+
- Use letter suffixes in numbering for each variation when a test has a sub-list, for example `**3a.**`, `**3b.**`.
155+
- Always use `` to denote navigation, for example `Send → Amount`.
156+
- Use screen names from code, formatted as separate words without the `Screen` suffix, for example `SendAmountScreen` becomes `Send Amount`.
157+
- Use short-form wording like `in-sheet` for sheet screens, `nav` for navigation, `back` for back nav, and `LN` for Lightning Network.
133158

134159
**For library repos (has `bindings/` directory or `Cargo.toml`):**
135-
Structure QA Notes around testing and integration:
160+
Structure manual QA around integration validation only. Automated checks belong under `#### Automated Checks`.
136161

137162
Example:
138163
```
139164
### QA Notes
165+
#### Manual Tests
166+
- [ ] **1.** Consumer app → exercise updated binding flow: behavior matches previous release.
167+
- [ ] **2.** `regression:` Android integration screen → trigger changed API path: no crash or stale data.
168+
#### Automated Checks
169+
- Binding tests added: cover updated Android API path in `bindings/android/...`.
170+
- CI: standard cargo and binding checks run by the PR bot.
171+
```
140172

141-
#### Testing
142-
- [ ] `cargo test` passes
143-
- [ ] `cargo clippy` clean
144-
- [ ] Android bindings: `./build_android.sh`
145-
- [ ] iOS bindings: `./build_ios.sh`
146-
147-
#### Integration
148-
- Tested in: [bitkit-android#XXX](link)
149-
- Or N/A if internal refactor with no API changes
173+
Concrete style target:
174+
```md
175+
### QA Notes
176+
#### Manual Tests
177+
- [ ] **1.** No usable channels/spending balance → scan LN invoice: error shows immediately, not after 15s.
178+
- [ ] **2.** Scanner → scan fixed amount LN invoice: Send Confirm or QuickPay opens directly.
179+
- [ ] **3a.** `regression:` Send → scanner/paste fixed amount LN invoice: in-sheet nav to Confirm or QuickPay.
180+
- [ ] **3b.** `regression:` Variable amount LN invoice/LNURL-pay: lands on Amount view.
181+
- [ ] **4a.** Activity Detail of LN transfer → tap Connection: lands on Channel Detail.
182+
- [ ] **4b.** back: returns to Activity Detail.
183+
- [ ] **5a.** Settings → Lightning Connections → tap channel: still opens Channel Detail.
184+
- [ ] **5b.** back: returns to Connections List.
185+
- [ ] **6.** `regression:` Channel Detail → tap Close Connection: works.
186+
#### Automated Checks
187+
- Unit tests added: cover invoice timeout handling in `app/src/test/.../SendInvoiceTest.kt`.
188+
- Unit tests modified: update channel navigation assertions in `app/src/test/.../ChannelDetailTest.kt`.
189+
- Test coverage removed: delete stale mock-only assertions from `app/src/test/.../OldFlowTest.kt` because the flow no longer exists.
190+
- CI: standard compile, unit test, and detekt checks run by the PR bot.
150191
```
151192

152193
**Preview Section (conditional):**
File renamed without changes.

.claude/commands

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1 @@
1+
../.agents/commands

AGENTS.md

Lines changed: 5 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,10 @@
11
# CLAUDE.md
22

3-
This file provides guidance to AI agents like Cursor/Claude Code/Codex/WARP when working with code in this repository.
3+
This file provides guidance to Codex, Claude Code, and Cursor when working with code in this repository.
4+
5+
## Agent Commands
6+
7+
Durable shared agent command specs live in `.agents/commands/`. For PR creation, follow `.agents/commands/pr.md`; `.claude/commands` is a compatibility symlink to the same files.
48

59
## Build Commands
610

0 commit comments

Comments
 (0)