Skip to content

Commit 89467f5

Browse files
os-zhuangclaude
andauthored
ci: publish-artifact smoke — install the release candidate like a user would (#3100)
The 15.1.0 incident (#3091): workspace pnpm overrides kept every in-repo CI job on better-auth 1.7.0-rc.1, but overrides do not ship with published packages, so every fresh `npx create-objectstack` project resolved an untested dependency mix that 500'd all auth endpoints. #3085 added the static gate (override ↔ declaration consistency); this adds the dynamic gate — install the exact bits a user would get and drive the first-run flow. scripts/publish-smoke.sh, two modes sharing one probe set: - pack (default): pnpm-pack all 67 publishable packages (pack applies the same manifest rewrites as publish), scaffold a fresh project outside the workspace with the repo-built create-objectstack, pin @objectstack/* to the tarballs via the project's OWN pnpm-workspace.yaml overrides (nothing inherited from the repo), install, and smoke. A lockfile assertion proves no @objectstack/* leaked to the registry. - registry: npx create-objectstack@latest + npm install against the real registry — catches ^-range ecosystem drift breaking published versions. Probes: anonymous get-session 200 → sign-up 200 → sign-in 200 → session carries the user → REST CRUD round (201/200/200/200) on the scaffolded object as the seeded dev admin → zero error/fatal log lines (ANSI-stripped; catches the #3091 'Failed to register OIDC discovery routes' signature). .github/workflows/publish-smoke.yml: - workflow_run after each Release run — the changesets release PR is pushed with GITHUB_TOKEN, whose events never trigger on:push/pull_request (the release PR has no Actions checks today), so the smoke runs after Release completes, checks out the release-branch head SHA iff an open release PR exists, and reports the verdict back as a commit status on that SHA so it is visible on the release PR. Deliberately not in normal PR CI (too slow). - weekly registry canary (Mon 04:47 UTC) that opens/refreshes an issue on failure. Verified locally: pack mode green on the current tree; a negative run with plugin-auth's declarations reverted to the 15.1.0 mix (^1.6.23 + scim rc.1) goes red via the log scan — core sign-in stays up thanks to the #3087 isolation, and the smoke still catches the broken optional-plugin init. Registry mode green against published 15.1.1. better-sqlite3 stays an optionalDependency, so a failed native build falls back to WASM sqlite (#2229) instead of blocking the smoke. Co-authored-by: Jack Zhuang <277994282+os-zhuang@users.noreply.github.com> Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
1 parent 515f11a commit 89467f5

3 files changed

Lines changed: 596 additions & 0 deletions

File tree

Lines changed: 210 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,210 @@
1+
# Publish Smoke — the dynamic half of the #3091 prevention.
2+
#
3+
# 15.1.0 shipped with every fresh project's auth endpoints returning 500:
4+
# the workspace's pnpm overrides (better-auth pinned to 1.7.0-rc.1) made all
5+
# in-repo CI green, but overrides do NOT ship with published packages, so
6+
# downstream installs resolved a dependency mix that was never tested here.
7+
# The static gate is scripts/check-override-consistency.mjs (#3085 — override
8+
# targets must be reflected in published manifests). This workflow is the
9+
# dynamic gate: install the exact bits a user would get and drive the
10+
# first-run flow (auth sign-up/sign-in/get-session + REST CRUD) for real.
11+
#
12+
# Two jobs, one driver script (scripts/publish-smoke.sh):
13+
#
14+
# pack-smoke SMOKE_MODE=pack — `pnpm pack` every publishable package
15+
# (pack applies the same manifest rewrites as publish),
16+
# scaffold a fresh project OUTSIDE the workspace, pin
17+
# @objectstack/* to the tarballs via the project's own pnpm
18+
# overrides, and smoke it. This is "what 15.1.0 would have
19+
# failed": the release-candidate combination, no workspace
20+
# overrides in sight.
21+
#
22+
# registry-canary SMOKE_MODE=registry — weekly `npx create-objectstack@latest`
23+
# against the real npm registry. Catches ^-range drift in the
24+
# ecosystem (a transitive release) breaking ALREADY-published
25+
# versions after the fact. Opens/refreshes an issue on failure.
26+
#
27+
# Trigger notes: the changesets release PR (changeset-release/main) is pushed
28+
# with GITHUB_TOKEN, and GITHUB_TOKEN events never trigger other workflows —
29+
# a plain `on: pull_request` / `on: push` would silently never run (the
30+
# release PR has NO Actions checks today). So pack-smoke runs on
31+
# `workflow_run` after each Release run completes (that's the moment
32+
# changesets creates/updates the release branch), checks out the release
33+
# branch if an open release PR exists, and reports the verdict back as a
34+
# commit status on the branch head so it IS visible on the release PR.
35+
# Not wired into normal PR CI on purpose: full build + pack + clean install
36+
# is far too slow for the inner loop.
37+
38+
name: Publish Smoke
39+
40+
on:
41+
workflow_run:
42+
workflows: [Release]
43+
types: [completed]
44+
schedule:
45+
- cron: '47 4 * * 1' # weekly registry canary (Mon 04:47 UTC)
46+
workflow_dispatch:
47+
48+
permissions:
49+
contents: read
50+
51+
concurrency:
52+
group: publish-smoke-${{ github.event_name }}-${{ github.ref }}
53+
cancel-in-progress: true
54+
55+
jobs:
56+
resolve:
57+
name: Resolve target
58+
if: github.event_name != 'schedule'
59+
runs-on: ubuntu-latest
60+
timeout-minutes: 5
61+
outputs:
62+
run: ${{ steps.target.outputs.run }}
63+
ref: ${{ steps.target.outputs.ref }}
64+
report-sha: ${{ steps.target.outputs.report-sha }}
65+
steps:
66+
- name: Pick the ref to smoke
67+
id: target
68+
env:
69+
GH_TOKEN: ${{ github.token }}
70+
run: |
71+
if [ "${{ github.event_name }}" = "workflow_dispatch" ]; then
72+
# Manual run: smoke whatever ref the run was dispatched on.
73+
echo "run=true" >> "$GITHUB_OUTPUT"
74+
echo "ref=${{ github.ref }}" >> "$GITHUB_OUTPUT"
75+
echo "report-sha=" >> "$GITHUB_OUTPUT"
76+
exit 0
77+
fi
78+
# workflow_run (a Release run finished): smoke the release branch
79+
# iff an open changesets release PR exists; otherwise skip.
80+
pr=$(gh pr list --repo "$GITHUB_REPOSITORY" \
81+
--head changeset-release/main --state open \
82+
--json headRefOid -q '.[0].headRefOid // empty')
83+
if [ -n "$pr" ]; then
84+
echo "run=true" >> "$GITHUB_OUTPUT"
85+
# Checkout the exact SHA the status is reported against, so a
86+
# concurrent force-push of the release branch can't skew the two.
87+
echo "ref=$pr" >> "$GITHUB_OUTPUT"
88+
echo "report-sha=$pr" >> "$GITHUB_OUTPUT"
89+
echo "Release PR open at $pr — smoking changeset-release/main"
90+
else
91+
echo "run=false" >> "$GITHUB_OUTPUT"
92+
echo "No open release PR — nothing to smoke"
93+
fi
94+
95+
pack-smoke:
96+
name: Packed-tarball smoke (release candidate)
97+
needs: resolve
98+
if: needs.resolve.outputs.run == 'true'
99+
runs-on: ubuntu-latest
100+
timeout-minutes: 45
101+
permissions:
102+
contents: read
103+
statuses: write
104+
steps:
105+
- name: Mark pending on the release PR head
106+
if: needs.resolve.outputs.report-sha != ''
107+
env:
108+
GH_TOKEN: ${{ github.token }}
109+
run: |
110+
gh api "repos/$GITHUB_REPOSITORY/statuses/${{ needs.resolve.outputs.report-sha }}" \
111+
-f state=pending -f context='publish-smoke / packed-tarballs' \
112+
-f description='Installing the release candidate into a fresh project…' \
113+
-f target_url="$GITHUB_SERVER_URL/$GITHUB_REPOSITORY/actions/runs/$GITHUB_RUN_ID"
114+
115+
- name: Checkout repository
116+
uses: actions/checkout@v7
117+
with:
118+
ref: ${{ needs.resolve.outputs.ref }}
119+
120+
- name: Setup Node.js
121+
uses: actions/setup-node@v6
122+
with:
123+
node-version: '22'
124+
125+
- name: Enable Corepack
126+
run: corepack enable
127+
128+
- name: Get pnpm store directory
129+
shell: bash
130+
run: echo "STORE_PATH=$(pnpm store path --silent)" >> $GITHUB_ENV
131+
132+
- name: Setup pnpm cache
133+
uses: actions/cache@v6
134+
with:
135+
path: ${{ env.STORE_PATH }}
136+
key: ${{ runner.os }}-pnpm-store-v3-${{ hashFiles('**/pnpm-lock.yaml') }}
137+
restore-keys: |
138+
${{ runner.os }}-pnpm-store-v3-
139+
140+
- name: Setup turbo cache
141+
uses: actions/cache@v6
142+
with:
143+
path: .turbo/cache
144+
key: ${{ runner.os }}-turbo-${{ github.job }}-${{ github.sha }}
145+
restore-keys: |
146+
${{ runner.os }}-turbo-${{ github.job }}-
147+
${{ runner.os }}-turbo-
148+
149+
- name: Install dependencies
150+
run: pnpm install --frozen-lockfile
151+
152+
- name: Build
153+
run: pnpm run build
154+
155+
- name: Publish smoke (packed tarballs)
156+
run: bash scripts/publish-smoke.sh
157+
158+
- name: Report verdict to the release PR head
159+
if: always() && needs.resolve.outputs.report-sha != ''
160+
env:
161+
GH_TOKEN: ${{ github.token }}
162+
run: |
163+
if [ "${{ job.status }}" = "success" ]; then
164+
state=success; desc='Fresh install of the release candidate: auth + CRUD green'
165+
else
166+
state=failure; desc='Release candidate fails a fresh install — see the run log'
167+
fi
168+
gh api "repos/$GITHUB_REPOSITORY/statuses/${{ needs.resolve.outputs.report-sha }}" \
169+
-f state="$state" -f context='publish-smoke / packed-tarballs' \
170+
-f description="$desc" \
171+
-f target_url="$GITHUB_SERVER_URL/$GITHUB_REPOSITORY/actions/runs/$GITHUB_RUN_ID"
172+
173+
registry-canary:
174+
name: Registry canary (published latest)
175+
if: github.event_name == 'schedule' || github.event_name == 'workflow_dispatch'
176+
runs-on: ubuntu-latest
177+
timeout-minutes: 25
178+
permissions:
179+
contents: read
180+
issues: write
181+
steps:
182+
- name: Checkout repository
183+
uses: actions/checkout@v7
184+
185+
- name: Setup Node.js
186+
uses: actions/setup-node@v6
187+
with:
188+
node-version: '22'
189+
190+
- name: Publish smoke (npm registry)
191+
run: SMOKE_MODE=registry bash scripts/publish-smoke.sh
192+
193+
- name: Open or refresh the canary issue
194+
# Scheduled runs have no human watching — surface the breakage as an
195+
# issue (deduped: one open issue, refreshed with a comment per failure).
196+
if: failure() && github.event_name == 'schedule'
197+
env:
198+
GH_TOKEN: ${{ github.token }}
199+
run: |
200+
run_url="$GITHUB_SERVER_URL/$GITHUB_REPOSITORY/actions/runs/$GITHUB_RUN_ID"
201+
title="Registry canary failed: fresh npx create-objectstack install is broken"
202+
existing=$(gh issue list --repo "$GITHUB_REPOSITORY" --state open \
203+
--search "\"$title\" in:title" --json number -q '.[0].number // empty')
204+
body=$(printf 'The weekly publish-smoke registry canary failed: a fresh `npx create-objectstack@latest` project no longer passes first-run auth + CRUD against the npm registry.\n\nThis is the #3091 failure class hitting ALREADY-published versions (e.g. a transitive dependency released into a ^ range). See the run log for the failing probe: %s\n' "$run_url")
205+
if [ -n "$existing" ]; then
206+
gh issue comment "$existing" --repo "$GITHUB_REPOSITORY" \
207+
--body "Still failing as of $run_url"
208+
else
209+
gh issue create --repo "$GITHUB_REPOSITORY" --title "$title" --body "$body"
210+
fi

scripts/publish-smoke-pack.mjs

Lines changed: 104 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,104 @@
1+
#!/usr/bin/env node
2+
/**
3+
* Pack every publishable workspace package into <dest-dir> and write
4+
* <dest-dir>/overrides.json mapping package name → `file:` tarball spec.
5+
*
6+
* Usage: node scripts/publish-smoke-pack.mjs <dest-dir>
7+
*
8+
* Part of the publish-artifact smoke (scripts/publish-smoke.sh): `pnpm pack`
9+
* applies the SAME manifest rewrites as `pnpm publish` (workspace:* →
10+
* concrete versions, publishConfig overlay), so the tarballs are what a
11+
* downstream `npm install` would actually receive — including whatever the
12+
* published manifests declare WITHOUT this workspace's pnpm overrides.
13+
*
14+
* The whole public surface is packed (not a hand-curated closure): the CLI
15+
* alone depends on ~45 workspace packages, so any curated list would rot.
16+
* Packing everything keeps the overrides map total — a package missing from
17+
* it would make the smoke project resolve that name from the npm registry,
18+
* silently testing a published version instead of the candidate one.
19+
*/
20+
21+
import { execFile } from 'node:child_process';
22+
import { mkdirSync, writeFileSync } from 'node:fs';
23+
import { resolve } from 'node:path';
24+
import { promisify } from 'node:util';
25+
26+
const execFileP = promisify(execFile);
27+
28+
// Not consumed as npm dependencies by a scaffolded project:
29+
// create-objectstack — the scaffolder itself; the smoke runs it straight
30+
// from the repo's built bin, and no @objectstack/* manifest depends on it.
31+
// objectstack-vscode — a VS Code extension (vsce-packaged), not an npm lib.
32+
const EXCLUDE = new Set(['create-objectstack', 'objectstack-vscode']);
33+
34+
const CONCURRENCY = 8;
35+
36+
async function listPublicPackages(repoRoot) {
37+
const { stdout } = await execFileP('pnpm', ['-r', 'list', '--depth', '-1', '--json'], {
38+
cwd: repoRoot,
39+
maxBuffer: 64 * 1024 * 1024,
40+
});
41+
const all = JSON.parse(stdout);
42+
return all.filter((p) => p.name && p.private !== true && !EXCLUDE.has(p.name));
43+
}
44+
45+
async function packOne(pkg, destDir) {
46+
const { stdout } = await execFileP(
47+
'pnpm',
48+
['pack', '--json', '--pack-destination', destDir],
49+
{ cwd: pkg.path, maxBuffer: 64 * 1024 * 1024 },
50+
);
51+
let parsed;
52+
try {
53+
parsed = JSON.parse(stdout);
54+
} catch {
55+
throw new Error(`pnpm pack --json returned non-JSON for ${pkg.name}:\n${stdout}`);
56+
}
57+
if (!parsed.filename) {
58+
throw new Error(`pnpm pack reported no tarball filename for ${pkg.name}`);
59+
}
60+
return { name: pkg.name, filename: parsed.filename };
61+
}
62+
63+
async function main() {
64+
const destArg = process.argv[2];
65+
if (!destArg) {
66+
console.error('Usage: node scripts/publish-smoke-pack.mjs <dest-dir>');
67+
process.exit(1);
68+
}
69+
const repoRoot = resolve(import.meta.dirname, '..');
70+
const destDir = resolve(destArg);
71+
mkdirSync(destDir, { recursive: true });
72+
73+
const packages = await listPublicPackages(repoRoot);
74+
console.log(`Packing ${packages.length} publishable package(s) → ${destDir}`);
75+
76+
const overrides = {};
77+
const queue = [...packages];
78+
let done = 0;
79+
const workers = Array.from({ length: CONCURRENCY }, async () => {
80+
for (;;) {
81+
const pkg = queue.shift();
82+
if (!pkg) return;
83+
const { name, filename } = await packOne(pkg, destDir);
84+
overrides[name] = `file:${filename}`;
85+
done += 1;
86+
if (done % 10 === 0 || done === packages.length) {
87+
console.log(` packed ${done}/${packages.length}`);
88+
}
89+
}
90+
});
91+
await Promise.all(workers);
92+
93+
const sorted = Object.fromEntries(
94+
Object.entries(overrides).sort(([a], [b]) => a.localeCompare(b)),
95+
);
96+
const outPath = resolve(destDir, 'overrides.json');
97+
writeFileSync(outPath, JSON.stringify(sorted, null, 2) + '\n');
98+
console.log(`Wrote ${Object.keys(sorted).length} override(s) → ${outPath}`);
99+
}
100+
101+
main().catch((err) => {
102+
console.error(err.stack ?? String(err));
103+
process.exit(1);
104+
});

0 commit comments

Comments
 (0)