Merge pull request #37 from tokenhost/feat/generated-tests-rollout-plan

Rollout plan: phase in generated-app tests and default behavior

Author: snissn

.github/PULL_REQUEST_TEMPLATE.md

@@ -0,0 +1,17 @@
+## Summary
+
+Describe what changed and why.
+
+## Validation
+
+- [ ] `pnpm test`
+- [ ] `pnpm test:integration`
+- [ ] Any targeted tests added/updated for this change
+
+## Generated-App Tests Rollout Checklist
+
+- [ ] If this PR changes test scaffolding or defaults, `docs/generated-app-test-rollout.md` is updated.
+- [ ] If this PR changes generated app CI behavior, `docs/generated-app-ci.md` is updated.
+- [ ] `AGENTS.md` and `SPEC.md` reflect the same generated-app testing behavior and rollout phase.
+- [ ] Canonical generated app coverage remains in CI (`integration-local`).
+- [ ] Migration/deprecation notes are included if generated defaults changed.

AGENTS.md

@@ -20,7 +20,9 @@ Legacy (kept temporarily; deprecated):
 - Build/deploy: `build.sh` compiles via `solcjs`, generates UI, then deploys via `tokenhost-web-template/contracts/deploy.js` (Foundry `forge create`).
 
 Tests:
-- `pnpm test` passes (legacy generator golden tests + THS lint tests + CRUD generator compile smoke test).
+- `pnpm test` and `pnpm test:integration` pass.
+- CI requires `static` and `integration-local` on PRs to `master`.
+- Canonical generated app coverage includes contract and UI scaffold verification.
 
 ## Phase 0 (NOW): Remediation of current snapshot (stability + security)
 
@@ -146,12 +148,46 @@ Goal: replace ad-hoc scripts with a coherent CLI that runs locally and in CI.
 - Add `th migrate` and stubs for chain migration/indexer hooks as needed.
 
 Phase 5 progress (done/partial):
-- Implemented: `th init`, `th validate`, `th import-legacy`, `th generate` (contracts only), `th build`, `th deploy`, `th verify` (stub), `th doctor`.
+- Implemented: `th init`, `th validate`, `th import-legacy`, `th generate` (contracts + UI), `th build`, `th deploy`, `th verify` (stub), `th doctor`, `th up|run|dev`.
+- `th generate --with-tests` emits generated app test scaffold and generated app CI workflow.
 
 Remaining:
-- `th generate` UI generation (SPEC 8), `th publish` (if/when in scope), real `th verify`.
+- `th publish` (if/when in scope), real `th verify`.
 - `th migrate` implementation + `migrate-chain` implementation (currently stubs).
 
+## Phase 6: Generated-app test rollout (default behavior)
+
+Goal: make generated-app tests default-on with a controlled compatibility transition.
+
+Rollout phases:
+1) scaffold emission,
+2) contract tests emission,
+3) UI tests emission,
+4) generated CI template emission,
+5) default-on switch.
+
+Phase 6 progress (done/partial):
+- Phases 1-4 are complete behind `th generate --with-tests`.
+- Generated output now includes:
+  - `tests/contract/integration.mjs`
+  - `tests/ui/smoke.mjs`
+  - `.github/workflows/generated-app-ci.yml`
+
+Remaining:
+- Phase 5 default-on switch for emitted tests.
+- Add/keep explicit opt-out (`--no-tests` or equivalent) when default-on is enabled.
+- Keep compatibility alias behavior for `--with-tests` across a deprecation window.
+
+Default-on gate (must satisfy before switching):
+- `integration-local` includes generated-app verification and is stable on `master`.
+- At least one canonical generated-app workflow verification remains in CI before and after switch.
+- No open P0/P1 generated-test-scaffold regressions.
+
+Migration/deprecation policy:
+- Existing generated app repos are not auto-migrated; teams must regenerate and merge emitted tests/workflow.
+- `--with-tests` remains accepted for at least two minor releases after default-on.
+- Remove testless generation path only after deprecation milestones are completed.
+
 ## Out of scope for this repo (unless we explicitly pull it in)
 
 - Managed platform API / Studio backend (SPEC 13).

README.md

@@ -0,0 +1,29 @@
+# Token Host Builder
+
+Token Host Builder is a schema-to-dapp generator framework for EVM CRUD apps.
+
+Canonical product behavior is defined in `SPEC.md`.  
+Execution backlog and current gap-tracking is in `AGENTS.md`.
+
+## Current generated-app testing model
+
+- `th generate <schema> --with-tests` emits generated app tests and a generated app CI workflow.
+- Generated scaffold includes:
+  - `tests/contract/integration.mjs`
+  - `tests/ui/smoke.mjs`
+  - `.github/workflows/generated-app-ci.yml`
+- Current default remains opt-in (`--with-tests`) until the default-on rollout gate is met.
+
+## Rollout and migration docs
+
+- Rollout decision doc: `docs/generated-app-test-rollout.md`
+- Generated app CI details: `docs/generated-app-ci.md`
+- Release acceptance checklist: `docs/release-checklist.md`
+
+## Builder CI baseline
+
+This repo requires:
+- `static`
+- `integration-local`
+
+`integration-local` must continue to include canonical generated-app verification before default-on is enabled for emitted tests.

SPEC.md

@@ -3,7 +3,7 @@
 Status: Draft (spec-driven design)  
 Owner: Token Host  
 Domain: `tokenhost.com`  
-Last updated: 2026-01-04  
+Last updated: 2026-02-05  
 Scope: Production system. All existing repos are legacy prototypes and are not binding.
 
 This document defines the intended production design of the Token Host platform: a managed schema-to-dapp builder that generates and deploys smart contracts, generates a hosted UI, optionally provisions indexing, and publishes apps under a dedicated hosted-app origin (default `*.apps.tokenhost.com`, plus optional custom domains).
@@ -1611,6 +1611,35 @@ Optional commands (chain management):
 
 The CLI MUST be able to export a complete bundle suitable for self-hosting.
 
+### 12.1 Generated-app test emission and default rollout
+
+Token Host MUST support generated app test scaffolding that can run in downstream app repositories without manual setup.
+
+Minimum emitted scaffold (when enabled):
+- contract tests (`tests/contract/*`),
+- UI smoke tests (`tests/ui/*`),
+- generated app CI workflow (`.github/workflows/generated-app-ci.yml`).
+
+Rollout phases MUST be:
+1) scaffold emission,
+2) contract tests emission,
+3) UI tests emission,
+4) generated CI template emission,
+5) default-on behavior.
+
+Until the default-on gate is met, test scaffold emission MAY remain opt-in (for example `th generate --with-tests`).
+After default-on, CLI MUST provide an explicit opt-out (for example `--no-tests`).
+
+Default-on gate requirements:
+- builder CI includes generated-app verification in the required integration suite,
+- canonical generated app coverage is green in CI,
+- no open P0/P1 regressions attributable to generated scaffold emission.
+
+Compatibility/deprecation requirements:
+- existing generated app repositories MUST NOT be silently mutated by builder releases,
+- migration notes for regeneration MUST be published in release notes and docs,
+- compatibility aliases for prior behavior (for example `--with-tests`) SHOULD be kept for at least two minor releases after default-on.
+
 ---
 
 ## 13. Managed Platform API (Conceptual)

docs/generated-app-test-rollout.md

@@ -0,0 +1,83 @@
+# Generated-App Tests Rollout Plan
+
+## Purpose
+
+Define the default-on rollout for generated-app tests without destabilizing existing users.
+
+## Scope
+
+This plan covers:
+- scaffold emission,
+- contract tests emission,
+- UI tests emission,
+- generated CI workflow emission,
+- default-on switch for emitted tests.
+
+## Current state (as of 2026-02-05)
+
+- Phase 1 complete: `--with-tests` emits test scaffold.
+- Phase 2 complete: contract integration test scaffold emitted.
+- Phase 3 complete: UI smoke test scaffold emitted (static + optional live mode).
+- Phase 4 complete: generated CI workflow emitted under `.github/workflows/generated-app-ci.yml`.
+- Phase 5 pending: default-on switch for emitted tests.
+
+## Rollout phases and defaults
+
+1. Phase 1: scaffold emission
+- Behavior: `th generate --with-tests` emits base `tests/` tree.
+- Default: OFF.
+
+2. Phase 2: contract tests emission
+- Behavior: emit `tests/contract/*` and add `test:contract` script.
+- Default: OFF.
+
+3. Phase 3: UI tests emission
+- Behavior: emit `tests/ui/smoke.mjs` and add `test:ui` script.
+- Default: OFF.
+
+4. Phase 4: generated CI template emission
+- Behavior: emit `.github/workflows/generated-app-ci.yml`.
+- Default: OFF.
+
+5. Phase 5: default-on switch
+- Behavior: `th generate` emits tests + generated CI workflow by default.
+- New opt-out flag: `--no-tests` (or equivalent) to disable emitted test scaffold.
+- Existing `--with-tests` remains accepted for compatibility for two minor releases after the switch, then becomes removable.
+
+## Gate for default-on switch
+
+Default-on MUST NOT be enabled until all are true:
+
+- Builder CI includes generated-app verification in `integration-local` and stays green.
+- At least two consecutive weeks of green `integration-local` runs on `master`.
+- Canonical generated app (`apps/example/job-board.schema.json`) generated-app tests pass in CI.
+- No open P0/P1 bugs attributed to generated test scaffolding or generated app CI template.
+
+## Compatibility strategy for existing generated apps
+
+Existing generated apps are not auto-migrated.
+
+Migration path for existing app repos:
+- Regenerate with `th generate --with-tests` and copy emitted `ui/tests/` + `.github/workflows/generated-app-ci.yml`.
+- Keep app-specific tests and custom workflows; merge generated workflow sections as needed.
+- Validate locally with:
+  - `pnpm run test:contract`
+  - `pnpm run test:ui`
+
+## Deprecation timeline for testless path
+
+- T0: Current (opt-in tests via `--with-tests`).
+- T1: Default-on release lands.
+  - `--with-tests` still accepted (compat alias).
+  - `--no-tests` documented as supported opt-out.
+- T2: Two minor releases after T1.
+  - Announce pending removal of `--with-tests` alias in release notes.
+- T3: Four minor releases after T1.
+  - Remove `--with-tests` alias if adoption and support metrics are stable.
+
+## Release acceptance checklist integration
+
+Before releasing default-on behavior:
+- `docs/release-checklist.md` generated-app tests section is complete.
+- PR checklist includes generated-app rollout assertions.
+- SPEC and AGENTS sections for generated tests are synchronized.

docs/release-checklist.md

@@ -0,0 +1,23 @@
+# Release Checklist
+
+Use this checklist for builder releases that affect generation, tests, deploy, or CI behavior.
+
+## Core validation
+
+- `pnpm test` passes.
+- `pnpm test:integration` passes.
+- Required GitHub checks are green (`static`, `integration-local`).
+
+## Generated-app tests rollout checks
+
+- Rollout doc is up to date: `docs/generated-app-test-rollout.md`.
+- Generated app CI doc is up to date: `docs/generated-app-ci.md`.
+- SPEC and AGENTS are synchronized for generated-app testing behavior.
+- Canonical generated app verification is present in `integration-local`.
+- Default-on switch gate is satisfied before enabling emitted tests by default.
+
+## Release notes
+
+- Breaking/behavioral changes documented.
+- Generated app migration notes included when defaults change.
+- Deprecation milestones (`--with-tests` alias / testless path) updated if applicable.