Skip to content

feat(spec): compile-check skills/ TypeScript examples (anti-drift, #3094)#3224

Merged
os-zhuang merged 1 commit into
mainfrom
ci/skill-example-drift-gate
Jul 18, 2026
Merged

feat(spec): compile-check skills/ TypeScript examples (anti-drift, #3094)#3224
os-zhuang merged 1 commit into
mainfrom
ci/skill-example-drift-gate

Conversation

@os-zhuang

Copy link
Copy Markdown
Contributor

Closes #3094.

What

Adds check:skill-examples — a CI gate that type-checks the TypeScript examples inside skills/ against the built @objectstack/spec, so a renamed export or tightened union fails here instead of in a third party's editor. This is the "示例不漂移" layer; the existing check:skill-refs / check:skill-docs only cover the "目录不漂移" (generated reference indexes).

Design

  • Opt-in by marker, not fence-meta. A self-contained, should-compile block is tagged with an inert <!-- os:check --> HTML comment on the line directly above its fence. Full extraction of every ```ts block is infeasible — most are fragments (a columns: [...] subtree) that need a hand-authored wrapper and produce false-positive noise.
    • Deliberately not a ```ts check fence-meta: that would leave the fence info-string non-standard and punch a hole in the existing check:doc-authoring scanner (which keys on ^\``(ts|typescript|tsx)$). The comment keeps the fence a bare ```ts `, so every existing scanner still sees the block.
  • Consumer-faithful resolution. Extracted blocks compile against the built dist/*.d.ts via a tsconfig paths map derived from the spec's own exports field — the exact surface import { … } from '@objectstack/spec' resolves to for a consumer, and it self-updates as subpath exports change.
  • Placement. Reads the built dist, so it runs in the required TypeScript Type Check job after the build step, next to its fellow "real consumer" gates (check:api-surface, downstream-contract, example-app typecheck) — not before the build like check:skill-refs.
  • Error mapping. tsc diagnostics are remapped from the throwaway build file back to skills/**/SKILL.md:<real line>.

Proven-red (門必先证明能红)

Failure mode Result
Type error in a tagged block ✗ (caught the 8 real drifts below)
Spec not built (no dist/*.d.ts) ✗ loud guard, never vacuous
Zero marked blocks (marker stripped/renamed) ✗ vacuous-green guard
Orphan marker (not directly above a ts fence) ✗ misplaced-marker guard

Drift the gate immediately surfaced (fixed here)

Tagging 19 self-contained examples turned up real, shipping drift:

  • objectstack-dataObjectSchema imported from the package root; it's only exported from /data (×2).
  • objectstack-platform — the removed Data namespace (import { Data } … ; const { Field } = Data) → import { Field } from '@objectstack/spec/data' (×2).
  • objectstack-ui — a defineAction example using P`…` without import { P }.

Non-self-contained fragments (local-module imports like ./apps/crm/objectstack.config, external object refs like Opportunity) are left untagged by design — they can't compile standalone and aren't meant to.

Deferred (documented, not silently dropped)

The platform feature-flags example uses a top-level featureFlags key that isn't in ObjectStackDefinitionInput — feature flags live in the kernel capability config (features: FeatureFlagSchema[], nested), and environment is a single enum (dev|staging|prod|all), not an array. Correcting it is a non-trivial rewrite of that section, so the block is left exactly as authored and untagged, to be fixed in a follow-up rather than shipping a guessed correction.

Growing coverage

New self-contained examples opt in by adding <!-- os:check --> above their fence. A misplaced marker errors (orphan guard), so under-coverage can't hide.

…stack/spec (#3094)

The TypeScript in skills/ is the first thing an AI copies when authoring
metadata, yet nothing type-checked it, so it rotted silently. A new
`check:skill-examples` gate extracts every code block tagged with an inert
`<!-- os:check -->` comment and runs it through `tsc --noEmit` against the
built spec declarations — the exact surface a consumer's import resolves to.

Opt-in by marker (not a `` ```ts check `` fence-meta) so the fence info-string
stays a bare ``` ```ts ``` and the existing `check:doc-authoring` scanner keeps
seeing the block. Module resolution is a `paths` map derived from the spec's own
`exports`, so it self-updates as subpath exports change. Runs after the build
step in the required `TypeScript Type Check` job, alongside the other
"real consumer" gates (api-surface, downstream-contract, example-app typecheck).

Four provable red modes: a type error in a tagged block, spec not built,
zero marked blocks (vacuous-green guard), a misplaced/orphan marker.

Tagging 19 self-contained examples surfaced real drift, now fixed:
- objectstack-data: `ObjectSchema` imported from the root instead of `/data` (×2)
- objectstack-platform: the removed `Data` namespace (`const { Field } = Data`)
  → `import { Field } from '@objectstack/spec/data'` (×2)
- objectstack-ui: a `defineAction` example using `P` without importing it

Non-self-contained fragments (local-module imports, external object refs) are
left untagged by design. One deeper drift is deferred: the platform feature-flags
example uses a top-level `featureFlags` key absent from `ObjectStackDefinitionInput`
(flags live in the kernel capability config) — left as-authored, untagged.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
@vercel

vercel Bot commented Jul 18, 2026

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated (UTC)
spec Ready Ready Preview, Comment Jul 18, 2026 3:08pm

Request Review

@github-actions github-actions Bot added documentation Improvements or additions to documentation ci/cd dependencies Pull requests that update a dependency file tooling labels Jul 18, 2026
@github-actions

Copy link
Copy Markdown
Contributor

📓 Docs Drift Check

This PR changes 1 package(s): @objectstack/spec.

102 hand-written doc(s) reference the affected code and may need an implementation-accuracy re-verification:

  • content/docs/ai/agents.mdx (via @objectstack/spec)
  • content/docs/ai/skills-reference.mdx (via @objectstack/spec)
  • content/docs/ai/skills.mdx (via @objectstack/spec)
  • content/docs/api/client-sdk.mdx (via @objectstack/spec)
  • content/docs/api/environment-routing.mdx (via @objectstack/spec)
  • content/docs/api/error-catalog.mdx (via @objectstack/spec)
  • content/docs/api/error-handling-client.mdx (via @objectstack/spec)
  • content/docs/api/error-handling-server.mdx (via @objectstack/spec)
  • content/docs/api/index.mdx (via @objectstack/spec)
  • content/docs/automation/approvals.mdx (via packages/spec)
  • content/docs/automation/flows.mdx (via @objectstack/spec)
  • content/docs/automation/hook-bodies.mdx (via packages/spec)
  • content/docs/automation/hooks.mdx (via @objectstack/spec)
  • content/docs/automation/index.mdx (via @objectstack/spec)
  • content/docs/automation/webhooks.mdx (via @objectstack/spec)
  • content/docs/automation/workflows.mdx (via @objectstack/spec)
  • content/docs/concepts/architecture.mdx (via @objectstack/spec)
  • content/docs/concepts/design-principles.mdx (via packages/spec)
  • content/docs/concepts/index.mdx (via @objectstack/spec)
  • content/docs/concepts/metadata-driven.mdx (via @objectstack/spec)
  • content/docs/concepts/metadata-lifecycle.mdx (via packages/spec)
  • content/docs/concepts/north-star.mdx (via packages/spec)
  • content/docs/data-modeling/analytics.mdx (via @objectstack/spec)
  • content/docs/data-modeling/drivers.mdx (via @objectstack/spec)
  • content/docs/data-modeling/external-datasources.mdx (via @objectstack/spec)
  • content/docs/data-modeling/field-types.mdx (via @objectstack/spec)
  • content/docs/data-modeling/fields.mdx (via @objectstack/spec)
  • content/docs/data-modeling/formulas.mdx (via @objectstack/spec)
  • content/docs/data-modeling/index.mdx (via @objectstack/spec)
  • content/docs/data-modeling/objects.mdx (via @objectstack/spec)
  • content/docs/data-modeling/queries.mdx (via @objectstack/spec)
  • content/docs/data-modeling/schema-design.mdx (via @objectstack/spec)
  • content/docs/data-modeling/seed-data.mdx (via @objectstack/spec)
  • content/docs/data-modeling/validation-rules.mdx (via @objectstack/spec)
  • content/docs/data-modeling/validation.mdx (via @objectstack/spec)
  • content/docs/deployment/troubleshooting.mdx (via @objectstack/spec)
  • content/docs/getting-started/build-with-claude-code.mdx (via @objectstack/spec)
  • content/docs/getting-started/cli.mdx (via @objectstack/spec)
  • content/docs/getting-started/common-patterns.mdx (via @objectstack/spec)
  • content/docs/getting-started/examples.mdx (via @objectstack/spec)
  • content/docs/getting-started/quick-reference.mdx (via @objectstack/spec)
  • content/docs/getting-started/quick-start.mdx (via @objectstack/spec)
  • content/docs/getting-started/validating-metadata.mdx (via @objectstack/spec)
  • content/docs/getting-started/your-first-project.mdx (via @objectstack/spec)
  • content/docs/kernel/cluster.mdx (via @objectstack/spec)
  • content/docs/kernel/contracts/auth-service.mdx (via packages/spec)
  • content/docs/kernel/contracts/cache-service.mdx (via packages/spec)
  • content/docs/kernel/contracts/data-engine.mdx (via @objectstack/spec)
  • content/docs/kernel/contracts/index.mdx (via @objectstack/spec)
  • content/docs/kernel/contracts/metadata-service.mdx (via packages/spec)
  • content/docs/kernel/contracts/storage-service.mdx (via packages/spec)
  • content/docs/kernel/index.mdx (via packages/spec)
  • content/docs/kernel/runtime-services/email-service.mdx (via packages/spec)
  • content/docs/kernel/runtime-services/index.mdx (via packages/spec)
  • content/docs/kernel/runtime-services/queue-service.mdx (via packages/spec)
  • content/docs/kernel/runtime-services/sharing-service.mdx (via packages/spec)
  • content/docs/kernel/runtime-services/sms-service.mdx (via packages/spec)
  • content/docs/kernel/runtime-services/storage-service.mdx (via packages/spec)
  • content/docs/kernel/services-checklist.mdx (via @objectstack/spec)
  • content/docs/permissions/authorization.mdx (via @objectstack/spec)
  • content/docs/permissions/permission-sets.mdx (via @objectstack/spec)
  • content/docs/permissions/permissions-matrix.mdx (via @objectstack/spec)
  • content/docs/permissions/positions.mdx (via @objectstack/spec)
  • content/docs/permissions/rls.mdx (via @objectstack/spec)
  • content/docs/permissions/sharing-rules.mdx (via @objectstack/spec)
  • content/docs/plugins/adding-a-metadata-type.mdx (via @objectstack/spec)
  • content/docs/plugins/development.mdx (via @objectstack/spec)
  • content/docs/plugins/index.mdx (via @objectstack/spec)
  • content/docs/plugins/packages.mdx (via @objectstack/spec)
  • content/docs/protocol/backward-compatibility.mdx (via @objectstack/spec)
  • content/docs/protocol/diagram.mdx (via packages/spec)
  • content/docs/protocol/kernel/config-resolution.mdx (via @objectstack/spec)
  • content/docs/protocol/kernel/i18n-standard.mdx (via @objectstack/spec)
  • content/docs/protocol/kernel/lifecycle.mdx (via @objectstack/spec)
  • content/docs/protocol/kernel/plugin-spec.mdx (via @objectstack/spec)
  • content/docs/protocol/kernel/runtime-capabilities.mdx (via @objectstack/spec)
  • content/docs/protocol/knowledge.mdx (via @objectstack/spec)
  • content/docs/protocol/objectql/index.mdx (via packages/spec)
  • content/docs/protocol/objectql/query-syntax.mdx (via @objectstack/spec)
  • content/docs/protocol/objectql/schema.mdx (via @objectstack/spec)
  • content/docs/protocol/objectql/security.mdx (via packages/spec)
  • content/docs/protocol/objectql/state-machine.mdx (via @objectstack/spec)
  • content/docs/protocol/objectui/actions.mdx (via @objectstack/spec)
  • content/docs/protocol/objectui/concept.mdx (via @objectstack/spec)
  • content/docs/protocol/objectui/index.mdx (via @objectstack/spec)
  • content/docs/protocol/objectui/layout-dsl.mdx (via @objectstack/spec)
  • content/docs/protocol/objectui/record-alert.mdx (via @objectstack/spec)
  • content/docs/protocol/objectui/widget-contract.mdx (via @objectstack/spec)
  • content/docs/releases/implementation-status.mdx (via @objectstack/spec)
  • content/docs/releases/index.mdx (via @objectstack/spec)
  • content/docs/releases/v12.mdx (via @objectstack/spec)
  • content/docs/releases/v13.mdx (via @objectstack/spec)
  • content/docs/releases/v9.mdx (via @objectstack/spec)
  • content/docs/ui/actions.mdx (via @objectstack/spec)
  • content/docs/ui/create-vs-edit-form.mdx (via @objectstack/spec)
  • content/docs/ui/dashboards.mdx (via @objectstack/spec)
  • content/docs/ui/forms.mdx (via @objectstack/spec)
  • content/docs/ui/index.mdx (via @objectstack/spec)
  • content/docs/ui/public-data-collection.mdx (via @objectstack/spec)
  • content/docs/ui/setup-app.mdx (via @objectstack/spec)
  • content/docs/ui/translations.mdx (via @objectstack/spec)
  • content/docs/ui/views.mdx (via @objectstack/spec)

Advisory only. To re-verify, run the docs-accuracy-audit workflow scoped to these files:
node scripts/docs-audit/affected-docs.mjs origin/main → pass the list as args.docs.

@os-zhuang os-zhuang added the skip-changeset PR has no user-facing published change; bypasses the changeset gate label Jul 18, 2026
@os-zhuang
os-zhuang merged commit dccffdb into main Jul 18, 2026
21 checks passed
@os-zhuang
os-zhuang deleted the ci/skill-example-drift-gate branch July 18, 2026 16:14
@os-zhuang

Copy link
Copy Markdown
Contributor Author

Follow-up for the deferred feature-flags drift (noted in the PR description): #3248 — the platform featureFlags example teaches a top-level key that isn't in ObjectStackDefinitionInput; needs a product/spec decision on the real authoring surface before it can be rewritten and re-tagged with <!-- os:check -->.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

ci/cd dependencies Pull requests that update a dependency file documentation Improvements or additions to documentation size/m skip-changeset PR has no user-facing published change; bypasses the changeset gate tooling

Projects

None yet

Development

Successfully merging this pull request may close these issues.

CI gate: compile-check the TypeScript examples inside skills/ (anti-drift)

1 participant