Skip to content

feat(dogfood): in-process runtime regression gate#2020

Merged
os-zhuang merged 1 commit into
mainfrom
feat/dogfood-regression-gate
Jun 18, 2026
Merged

feat(dogfood): in-process runtime regression gate#2020
os-zhuang merged 1 commit into
mainfrom
feat/dogfood-regression-gate

Conversation

@os-zhuang

Copy link
Copy Markdown
Contributor

Why

Static gates — build, unit tests, spec-liveness, CodeQL — verify each layer in isolation, usually against mocks. They cannot catch a break that only appears when the real engine + strategies + services + HTTP context run together.

#2018 ("org timezone drives analytics date bucketing") is the canonical case: it was green on every static gate — 900+ unit tests included — because each of its three broken seams (analytics strategy routing, in-memory count, REST execution-context) was individually correct and individually mocked. The break was only visible by booting the app and comparing a date bucket under UTC vs a non-UTC org timezone. Unit tests that mock the protocol/server (e.g. rest.test.ts) structurally cannot catch it.

What

@objectstack/dogfood (private, never published): a harness that boots real example apps in-process against in-memory SQLite — wired with the same service plugins objectstack dev loads — and exercises them through the real HTTP surface via Hono request-injection (no ports, no sockets, CI-stable). Tests act as a browser client: sign in, hit /api/v1/..., assert on real responses.

  • src/harness.tsbootDogfoodStack(config){ kernel, api, raw, signIn, apiAs, stop }. Registers ObjectQL + SQLite + HonoServer + App + Settings + Analytics + Auth + Security + REST + Dispatcher, runs kernel.bootstrap() (seed + dev-admin + route registration), and injects requests through the live Hono app.
  • test/analytics-timezone.dogfood.test.ts — golden regression for ADR-0053 Phase 2 · Slice 5: timezone-aware analytics date bucketing #1982/fix(analytics): make organization timezone drive date-dimension bucketing (#1982) #2018. Inserts leads at a tz-boundary instant (2024-03-01T03:00:00Z), sets the org timezone through the real settings route, and asserts the date bucket shifts (UTC → 2024-03-01, America/Los_Angeles → 2024-02-29) with the correct count.
  • CI — new Dogfood Regression Gate job (gated on the core paths filter); also runs under turbo run test.

Proof it's a real gate

Reverting just one of the three #2018 seams (NativeSQLStrategy routing) and rebuilding turns the LA assertion red (expected undefined to be 3 — the bucket didn't shift). Restoring it → green. A gate that stays green on the bug is not a gate; this one fails on it.

Boot-to-assert ≈ 2s. Private package — no changeset.

Notes

🤖 Generated with Claude Code

Static gates (build, unit tests, spec-liveness, CodeQL) verify each layer in
isolation, usually against mocks, and cannot catch a break that only appears
when the real engine + strategies + services + HTTP context run together. The
#2018 tz-bucketing regression is the canonical case: green on every static gate
(900+ unit tests included) because each of its three broken seams was
individually correct and individually mocked.

This adds `@objectstack/dogfood` (private), a harness that boots real example
apps in-process against in-memory SQLite — wired with the same service plugins
`objectstack dev` loads — and exercises them through the real HTTP surface via
Hono request-injection (no ports, no sockets, CI-stable). Tests act as a browser
client: sign in, hit /api/v1/..., assert on real responses.

- src/harness.ts — bootDogfoodStack(config) → { kernel, api, raw, signIn, apiAs, stop }
- test/analytics-timezone.dogfood.test.ts — golden regression for #1982/#2018:
  org timezone (set via the real settings route) must shift a boundary-crossing
  instant's date bucket (UTC 2024-03-01 → America/Los_Angeles 2024-02-29) with
  the correct count. Verified to FAIL when any #2018 seam is reverted.
- CI: new "Dogfood Regression Gate" job (also runs under turbo run test).

Boot-to-assert ~2s. Private package — no changeset.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
@vercel

vercel Bot commented Jun 18, 2026

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated (UTC)
spec Ready Ready Preview, Comment Jun 18, 2026 5:31am

Request Review

@github-actions github-actions Bot added size/m documentation Improvements or additions to documentation ci/cd dependencies Pull requests that update a dependency file tests tooling labels Jun 18, 2026
@github-actions

Copy link
Copy Markdown
Contributor

📓 Docs Drift Check

No hand-written docs reference the 1 changed package(s). ✅

@os-zhuang
os-zhuang merged commit ebc7ecb into main Jun 18, 2026
16 checks passed
@os-zhuang
os-zhuang deleted the feat/dogfood-regression-gate branch June 18, 2026 05:36
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

ci/cd dependencies Pull requests that update a dependency file documentation Improvements or additions to documentation size/m tests tooling

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant