Skip to content

Commit 56484c7

Browse files
anandgupta42anandgupta42
authored
fix: v0.5.18 P1 follow-ups — diff guard, prompt injection, docs, timeout (#648)
* fix: [#647] v0.5.18 P1 follow-ups — diff guard, prompt injection, docs, timeout, changelog - GitLab MR diff size guard: truncate diffs at 50 files / 200KB to avoid context overflow with clear warning message - Prompt injection mitigation: frame MR content as untrusted data in the system prompt - Rewrite `docs/usage/gitlab.md` with correct `altimate gitlab review` command, CI setup, auth docs, and known limitations - Add 15s timeout to `validateCredentials` fetch to prevent TUI hang on network partition - Fix changelog wording: `/connect` provider dialog, not `/login` * chore: restructure `/release` workflow — review first, fix before tag Key change: move multi-persona code review to before testing. Actionable P1s found by reviewers are fixed on main before the tag, not as follow-up PRs after release. Old: Test → Review → Tag → follow-up PRs New: Review → Fix → Test → Tag → done clean * fix: address CodeRabbit feedback on PR #648 - Correct truncation threshold wording in docs (more than 50 / exceeding 200 KB) - CLI warning now fires for both file-count and byte-size truncation by checking truncateDiffs() result instead of file count alone
1 parent 0d60956 commit 56484c7

File tree

5 files changed

+492
-14
lines changed

5 files changed

+492
-14
lines changed

‎.claude/commands/release.md‎

Lines changed: 385 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,385 @@
1+
---
2+
description: Release a new version of altimate-code. Reviews code, fixes issues, generates changelog, runs checks, tags, pushes, and monitors CI. Everything ships clean in one release.
3+
---
4+
5+
# Release altimate-code
6+
7+
Automate the complete release process. The core principle: **find and fix everything before tagging. No follow-up PRs.**
8+
9+
## Input
10+
11+
`$ARGUMENTS` = version bump type: `patch` (default), `minor`, or `major`. Can also be an explicit version like `0.6.0`.
12+
13+
---
14+
15+
## Step 0: Read the Release Guide
16+
17+
Read the official release documentation and current changelog style:
18+
19+
```bash
20+
cat docs/RELEASING.md
21+
head -80 CHANGELOG.md
22+
```
23+
24+
Cross-check RELEASING.md against this skill. If it has new steps or prerequisites, warn the user and adapt.
25+
26+
## Step 1: Determine Version
27+
28+
```bash
29+
npm info @altimateai/altimate-code version
30+
```
31+
32+
- `patch` or empty: bump patch (e.g., `0.5.17` → `0.5.18`)
33+
- `minor`: bump minor (e.g., `0.5.17` → `0.6.0`)
34+
- `major`: bump major (e.g., `0.5.17` → `1.0.0`)
35+
- Explicit version (matches `\d+\.\d+\.\d+`): use directly
36+
37+
Confirm with user: "Releasing **v{NEXT_VERSION}** (current: v{CURRENT_VERSION}). Proceed?"
38+
39+
## Step 2: Ensure on Main and Clean
40+
41+
```bash
42+
git branch --show-current
43+
git status --short
44+
git fetch origin main
45+
git log HEAD..origin/main --oneline
46+
```
47+
48+
- Must be on `main`. If not, stop.
49+
- Working tree must be clean. If dirty, stop.
50+
- Must be up to date with remote. If behind, stop.
51+
52+
## Step 3: Identify What Changed
53+
54+
```bash
55+
git log v{CURRENT_VERSION}..HEAD --oneline --no-merges
56+
git log v{CURRENT_VERSION}..HEAD --oneline --no-merges --grep="^feat" --grep="^fix"
57+
git diff v{CURRENT_VERSION}..HEAD --stat
58+
git diff v{CURRENT_VERSION}..HEAD --name-only -- 'packages/opencode/src/**' 'packages/dbt-tools/src/**'
59+
```
60+
61+
For each `feat:` and `fix:` commit, read its full diff (`git show {HASH} --stat`).
62+
63+
Build a list of **user-facing changes** — features added, bugs fixed, behavior modified. Ignore `ci:`, `test:`, `chore:` that don't affect UX.
64+
65+
## Step 4: Multi-Persona Code Review
66+
67+
**Purpose:** Find issues BEFORE we write tests or changelog. Everything found here gets fixed before the release ships. No follow-up PRs.
68+
69+
### 4a: Prepare the evaluation brief
70+
71+
Gather:
72+
1. **Change summary** — the user-facing changes list from Step 3
73+
2. **Source files changed** — from `git diff --name-only`
74+
3. **Diff stats** — `git diff v{CURRENT_VERSION}..HEAD --stat`
75+
4. **Version context** — what version this is, time since last release
76+
77+
### 4b: Launch the five-member evaluation team
78+
79+
Spawn **five agents in parallel**. Each agent MUST read the actual source code diffs — not just the summary. Each reviews from their unique perspective:
80+
81+
1. **CTO** — Technical risk, security exposure, breaking changes, operational readiness. "Would I deploy this on a Friday?"
82+
83+
2. **Product Manager** — Feature completeness, user value, messaging clarity, docs accuracy, changelog story. "Can I write a release announcement customers care about?"
84+
85+
3. **End User (Data Engineer)** — Day-to-day usability, error messages, discoverability, workflow friction. "Will this make my Monday easier or harder?"
86+
87+
4. **Tech Lead** — Code quality, test coverage, maintainability, marker guard, CI health. "Will we regret this in two weeks?"
88+
89+
5. **Chaos Gremlin** — Random adversarial perspective (security auditor, support engineer, new hire, compliance officer, etc.). Different each release. Asks the uncomfortable question nobody else thought of.
90+
91+
Each agent produces:
92+
93+
```markdown
94+
## {Persona} Review
95+
### Verdict: SHIP / HOLD / SHIP WITH NOTES
96+
### What looks good
97+
### Concerns
98+
- **[P0/P1/P2]** {concern} — {why}
99+
### Actionable fixes (code changes needed before release)
100+
- {specific file + what to change}
101+
### Release notes feedback
102+
```
103+
104+
**Key difference from a post-release review:** Each agent must classify concerns as either:
105+
- **Actionable** — can be fixed now in <30 min (stale docs, missing timeout, wording issues, missing guards)
106+
- **Deferred** — requires significant design work (>30 min), fine for next release
107+
108+
### 4c: Synthesize and triage
109+
110+
After all five agents complete:
111+
112+
1. **Collect all concerns**, deduplicate
113+
2. **Build the consensus table:**
114+
115+
```markdown
116+
| Persona | Verdict | P0 | P1 | P2 | Key Concern |
117+
|---------|---------|----|----|----|----|
118+
```
119+
120+
3. **Split actionable vs deferred:**
121+
122+
```markdown
123+
### Actionable (fix now, before tagging)
124+
| # | Issue | File(s) | Est. | Flagged by |
125+
|---|-------|---------|------|------------|
126+
| 1 | Missing timeout on validateCredentials | client.ts | 5 min | Tech Lead |
127+
| 2 | Stale docs reference wrong command | gitlab.md | 10 min | End User |
128+
129+
### Deferred (file issues for next release)
130+
| # | Issue | Reason for deferral |
131+
|---|-------|---------------------|
132+
| 1 | Inline per-line GitLab comments | New feature, not a fix |
133+
```
134+
135+
### 4d: Gate
136+
137+
- **Any P0** → Release blocked. Fix first, re-run from Step 3.
138+
- **3+ HOLD verdicts** → Release blocked.
139+
- **Actionable P1s exist** → Proceed to Step 5 (fix them).
140+
- **No actionable items** → Skip Step 5, proceed to Step 6.
141+
142+
## Step 5: Fix Actionable Issues
143+
144+
**Purpose:** Fix every actionable P1/P2 found by the review. This is why the review is early — so fixes ship in THIS release, not the next one.
145+
146+
### 5a: Fix each issue on main
147+
148+
For each actionable item from Step 4c:
149+
150+
1. Read the file(s) involved
151+
2. Make the fix
152+
3. Run the relevant tests
153+
4. Commit with a descriptive message:
154+
155+
```bash
156+
echo "fix: {description}" > .github/meta/commit.txt
157+
git commit -F .github/meta/commit.txt
158+
```
159+
160+
### 5b: File issues for deferred items
161+
162+
For each deferred item, create a GitHub issue:
163+
164+
```bash
165+
gh issue create --repo AltimateAI/altimate-code \
166+
--title "{type}: {description}" \
167+
--body "Found during v{NEXT_VERSION} release review. Deferred because: {reason}"
168+
```
169+
170+
### 5c: Re-verify after fixes
171+
172+
Run typecheck and marker guard to confirm fixes are clean:
173+
174+
```bash
175+
bun turbo typecheck
176+
bun run script/upstream/analyze.ts --markers --base main --strict
177+
```
178+
179+
**If fixes introduced new issues,** fix those too. Loop until clean.
180+
181+
## Step 6: Adversarial Testing
182+
183+
**Purpose:** Write and run adversarial tests covering ALL changes — including the fixes from Step 5. This ensures the fixes themselves don't introduce new problems.
184+
185+
### 6a: Write adversarial tests
186+
187+
Read the changed source files. Create test file at `packages/opencode/test/skill/release-v{NEXT_VERSION}-adversarial.test.ts`.
188+
189+
Follow established patterns (see `test/altimate/adversarial.test.ts`, `test/skill/release-v0.5.14-adversarial.test.ts`).
190+
191+
**Categories to cover per change:**
192+
1. Empty/null/undefined inputs
193+
2. Boundary values (zero, max, negative)
194+
3. Type confusion
195+
4. Injection attacks (SQL, command, prototype pollution, path traversal)
196+
5. Error propagation
197+
198+
**Test quality rules:**
199+
- Meaningful assertions, not just "doesn't throw"
200+
- Deterministic — no timing deps, no shared state
201+
- No `mock.module()` — use `Dispatcher.register()`/`reset()` or `spyOn()`
202+
203+
### 6b: Run tests
204+
205+
```bash
206+
cd packages/opencode && bun test --timeout 30000 test/skill/release-v{NEXT_VERSION}-adversarial.test.ts
207+
cd packages/opencode && bun test --timeout 30000
208+
```
209+
210+
### 6c: Gate
211+
212+
- **All pass** → Continue
213+
- **Test finds a real bug** → Fix the bug, run again. If the bug is in code from Step 5, fix it. If in original release code, commit the fix.
214+
- **Test bug** → Fix the test, continue
215+
216+
## Step 7: UX Verification
217+
218+
### 7a: Smoke test
219+
220+
```bash
221+
altimate --version
222+
altimate --help
223+
```
224+
225+
### 7b: Run feature-specific tests
226+
227+
Run all test files for the changed features:
228+
229+
```bash
230+
cd packages/opencode && bun test --timeout 30000 {relevant test files}
231+
```
232+
233+
### 7c: Verify docs match code
234+
235+
For each feature in this release, check that the corresponding doc in `docs/docs/` is accurate. Stale docs should have been caught in Step 4 and fixed in Step 5, but double-check.
236+
237+
### 7d: Gate
238+
239+
If any scenario fails with P0 severity (crash, data loss), stop and fix. P2 cosmetics can be noted for next release.
240+
241+
## Step 8: Generate Changelog
242+
243+
### 8a: Get all commits
244+
245+
```bash
246+
git log v{CURRENT_VERSION}..HEAD --oneline --no-merges
247+
```
248+
249+
**Important:** This now includes both the original changes AND the fixes from Step 5. Include everything.
250+
251+
### 8b: Write the changelog entry
252+
253+
Categorize into **Added**, **Fixed**, **Changed**. Use bold title + em-dash description matching existing style. Incorporate release notes feedback from the Step 4 persona reviews.
254+
255+
### 8c: Review with user
256+
257+
Show the changelog and ask: "Does this look correct? Edit anything?"
258+
259+
Wait for approval.
260+
261+
## Step 9: Pre-Release Checks
262+
263+
Run all mandatory checks:
264+
265+
```bash
266+
# Pre-release sanity (binary builds and starts)
267+
cd packages/opencode && bun run pre-release
268+
269+
# Marker guard
270+
bun run script/upstream/analyze.ts --markers --base main --strict
271+
```
272+
273+
**Gate: ALL CHECKS MUST PASS.** Stop on failure.
274+
275+
### Optional: Verdaccio sanity suite
276+
277+
If Docker is available:
278+
279+
```bash
280+
cd packages/dbt-tools && bun run build && cd ../..
281+
docker compose -f test/sanity/docker-compose.verdaccio.yml up \
282+
--build --abort-on-container-exit --exit-code-from sanity
283+
```
284+
285+
If Docker unavailable, skip — CI will catch it.
286+
287+
## Step 10: Commit, Tag, Push
288+
289+
```bash
290+
# Stage changelog + any adversarial tests
291+
git add CHANGELOG.md packages/opencode/test/skill/release-v{NEXT_VERSION}-adversarial.test.ts
292+
293+
# Commit
294+
echo "release: v{NEXT_VERSION}" > .github/meta/commit.txt
295+
git commit -F .github/meta/commit.txt
296+
297+
# Tag and push
298+
git tag v{NEXT_VERSION}
299+
git push origin main v{NEXT_VERSION}
300+
```
301+
302+
## Step 11: Monitor CI
303+
304+
```bash
305+
gh run list --workflow=release.yml --repo AltimateAI/altimate-code --limit 1
306+
gh run watch --repo AltimateAI/altimate-code
307+
```
308+
309+
If fails: `gh run view --repo AltimateAI/altimate-code --log-failed`
310+
311+
## Step 12: Verify and Close Issues
312+
313+
### 12a: Verify artifacts
314+
315+
```bash
316+
npm info @altimateai/altimate-code version
317+
gh release view v{NEXT_VERSION} --repo AltimateAI/altimate-code --json tagName,publishedAt,assets
318+
```
319+
320+
### 12b: Close resolved issues
321+
322+
Extract issue numbers from commits and PR closing references:
323+
324+
```bash
325+
git log v{CURRENT_VERSION}..v{NEXT_VERSION} --pretty=format:"%s %b" \
326+
| grep -oE '(#[0-9]+|([Cc]loses?|[Ff]ixes|[Rr]esolves?)\s+#[0-9]+)' \
327+
| grep -oE '[0-9]+' | sort -u
328+
```
329+
330+
For each PR, also check:
331+
```bash
332+
gh pr view {PR_NUMBER} --repo AltimateAI/altimate-code --json closingIssuesReferences --jq '.closingIssuesReferences[].number'
333+
```
334+
335+
For each open issue found, comment and close:
336+
```bash
337+
gh issue comment {N} --repo AltimateAI/altimate-code \
338+
--body "Resolved in [v{NEXT_VERSION}](https://github.com/AltimateAI/altimate-code/releases/tag/v{NEXT_VERSION})."
339+
gh issue close {N} --repo AltimateAI/altimate-code
340+
```
341+
342+
### 12c: Release summary
343+
344+
```
345+
## Release Summary: v{NEXT_VERSION}
346+
347+
| Check | Status | Details |
348+
|-------|--------|---------|
349+
| RELEASING.md | ✅ Read | |
350+
| Code review | ✅ | {N} SHIP, {N} HOLD — {N} P0, {N} P1, {N} P2 |
351+
| Issues fixed pre-release | {N} | {descriptions} |
352+
| Issues deferred | {N} | Filed as #{numbers} |
353+
| Adversarial tests | ✅ | {N}/{N} passed |
354+
| UX verification | ✅ | {N} scenarios passed |
355+
| Pre-release check | ✅ | |
356+
| Verdaccio | ✅ / ⏭️ | |
357+
| Marker guard | ✅ | |
358+
| CI workflow | ✅ | |
359+
| npm | ✅ | v{NEXT_VERSION} |
360+
| GitHub Release | ✅ | {link} |
361+
| Issues closed | ✅ | {N} issues |
362+
363+
v{NEXT_VERSION} is live! No follow-up PRs needed.
364+
```
365+
366+
---
367+
368+
## Rules
369+
370+
1. **Always read RELEASING.md first.** It is the source of truth for the process.
371+
2. **Always confirm version with user.** Never auto-release without approval.
372+
3. **Review BEFORE testing.** The multi-persona review finds design issues (stale docs, missing timeouts, naming problems). Adversarial tests find code bugs. Different tools for different problems. Review first, then test the reviewed code.
373+
4. **Fix actionable issues before tagging.** The whole point of reviewing early is to ship clean. If the review finds a stale doc or missing timeout, fix it on main before the tag. No follow-up PRs for things that could have been fixed in 10 minutes.
374+
5. **Only defer what truly can't be fixed quickly.** New features, large refactors, and design decisions get deferred. Missing timeouts, stale docs, wording fixes, and small guards get fixed now.
375+
6. **Adversarial tests cover the FINAL code.** Tests run after Step 5 fixes, so they test the code that actually ships.
376+
7. **Never skip pre-release check.** Last gate before a broken binary ships.
377+
8. **Always use `--repo AltimateAI/altimate-code`** with `gh` commands.
378+
9. **Only release from main.** Feature branches should not be tagged.
379+
10. **Changelog entries must match existing style.** Bold titles with em-dash descriptions.
380+
11. **If CI fails after push, do NOT delete the tag.** Investigate first.
381+
12. **npm is the source of truth for versions.**
382+
13. **PR template is mandatory.** PRs without exact headings get auto-closed. Create issue first, then PR with `Closes #N`.
383+
14. **No `mock.module()` in adversarial tests.** Use `Dispatcher.register()`/`reset()` or `spyOn()`.
384+
15. **Multi-persona evaluation is not optional.** The Chaos Gremlin persona must be different each release.
385+
16. **The release is done when the summary says "No follow-up PRs needed."** If it can't say that, something was missed.

‎CHANGELOG.md‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -10,7 +10,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
1010
### Added
1111

1212
- **Native GitLab MR review** — review merge requests directly from your terminal with `altimate gitlab review <MR_URL>`. Supports self-hosted GitLab instances, nested group paths, and comment deduplication (updates existing review instead of posting duplicates). Requires `GITLAB_PERSONAL_ACCESS_TOKEN` or `GITLAB_TOKEN` env var. (#622)
13-
- **Altimate LLM Gateway provider** — connect to Altimate's managed model gateway via the TUI provider dialog (`/connect` → Altimate). Credentials validated before save, stored at `~/.altimate/altimate.json` with `0600` permissions. (#606)
13+
- **Altimate LLM Gateway provider** — connect to Altimate's managed model gateway via the TUI provider dialog (select a provider → "Altimate"). Credentials validated before save, stored at `~/.altimate/altimate.json` with `0600` permissions. (#606)
1414

1515
### Fixed
1616

0 commit comments

Comments
 (0)