docs: audit and align API reference with source#1516
Open
johnlindquist wants to merge 15 commits intomainfrom
Open
docs: audit and align API reference with source#1516johnlindquist wants to merge 15 commits intomainfrom
johnlindquist wants to merge 15 commits intomainfrom
Conversation
|
Contributor
Contributor
📊 Benchmark Results
workflow with no steps💻 Local Development
workflow with 1 step💻 Local Development
workflow with 10 sequential steps💻 Local Development
workflow with 25 sequential steps💻 Local Development
workflow with 50 sequential steps💻 Local Development
Promise.all with 10 concurrent steps💻 Local Development
Promise.all with 25 concurrent steps💻 Local Development
Promise.all with 50 concurrent steps💻 Local Development
Promise.race with 10 concurrent steps💻 Local Development
Promise.race with 25 concurrent steps💻 Local Development
Promise.race with 50 concurrent steps💻 Local Development
workflow with 10 sequential data payload steps (10KB)💻 Local Development
workflow with 25 sequential data payload steps (10KB)💻 Local Development
workflow with 50 sequential data payload steps (10KB)💻 Local Development
workflow with 10 concurrent data payload steps (10KB)💻 Local Development
workflow with 25 concurrent data payload steps (10KB)💻 Local Development
workflow with 50 concurrent data payload steps (10KB)💻 Local Development
Stream Benchmarks (includes TTFB metrics)workflow with stream💻 Local Development
stream pipeline with 5 transform steps (1MB)💻 Local Development
10 parallel streams (1MB each)💻 Local Development
fan-out fan-in 10 streams (1MB each)💻 Local Development
SummaryFastest Framework by WorldWinner determined by most benchmark wins
Fastest World by FrameworkWinner determined by most benchmark wins
Column Definitions
Worlds:
|
Contributor
🧪 E2E Test Results❌ Some tests failed Summary
❌ Failed Tests💻 Local Development (2 failed)vite-stable (2 failed):
Details by Category❌ 💻 Local Development
✅ 📦 Local Production
✅ 🐘 Local Postgres
✅ 📋 Other
❌ Some E2E test jobs failed:
Check the workflow run for details. |
Member
VaguelySerious
left a comment
VaguelySerious
left a comment
There was a problem hiding this comment.
AI review: blocking issues found
| interface Error { | ||
| /** | ||
| ```typescript | ||
| new FatalError(message: string) |
Member
There was a problem hiding this comment.
AI Review: Blocking Note
This code block (and the one at line 53) uses ```typescript but contains a type signature, not valid TypeScript. The existing docs.test.ts typecheck suite tries to compile all ```typescript blocks and fails on 4 blocks introduced by this PR:
fatal-error.mdxlines 35, 53retryable-error.mdxlines 39, 64
Fix: either add a skip annotation or change the language tag to avoid typecheck compilation.
Contributor
Author
There was a problem hiding this comment.
Fixed — removed the type signature code blocks entirely. Other API ref pages (sleep, create-hook, etc.) use tables for constructor/parameters instead of code blocks, so now these match that pattern.
| }); | ||
| } catch (error) { | ||
| return new Response("Hook not found", { status: 404 }); | ||
| if (error instanceof Error && error.name === "HookNotFoundError") { |
Member
There was a problem hiding this comment.
AI Review: Note
HookNotFoundError is exported from workflow/errors and has a static HookNotFoundError.is(error) method (the idiomatic pattern per its own JSDoc). The string-based error.name === "HookNotFoundError" check used here (and in 3 other examples) is fragile. Consider:
import { HookNotFoundError } from "workflow/errors";
if (HookNotFoundError.is(error)) { ... }
Contributor
Author
There was a problem hiding this comment.
Fixed — all 4 occurrences now use HookNotFoundError.is(error) with import { HookNotFoundError } from "workflow/internal/errors". Each catch block also distinguishes 404 (not found) from 500 (other failures).
| return Response.json({ error: "Hook not found" }, { status: 404 }); | ||
| return Response.json({ success: true, runId: result.runId }); | ||
| } catch (error) { | ||
| return Response.json({ error: "Hook not found or invalid payload" }, { status: 400 }); |
Member
There was a problem hiding this comment.
AI Review: Note
This catch-all returns 400 for both "Hook not found" and "invalid payload". The resumeHook docs page teaches distinguishing HookNotFoundError (404) from other failures (500). Since defineHook().resume() wraps resumeHook(), the same error types apply. Consider aligning the error handling pattern here.
Contributor
Author
There was a problem hiding this comment.
Fixed — catch block now distinguishes HookNotFoundError (404) from other errors (500), matching the pattern in resume-hook.mdx.
| |---|---|---| | ||
| | `workflows.lazyDiscovery` | `boolean` | Enable lazy discovery mode. Sets the `WORKFLOW_NEXT_LAZY_DISCOVERY` flag. Deferred discovery only activates on Next.js `>= 16.2.0-canary.48`; on older versions, Workflow logs a warning and falls back to eager scanning. | | ||
| | `workflows.local.port` | `number` | Override the local development server port. Sets the `PORT` environment variable when running locally (no `VERCEL_DEPLOYMENT_ID`). | | ||
| | `workflows.local.dataDir` | `string` | Currently typed but ignored by `withWorkflow()`. In local mode, when `WORKFLOW_TARGET_WORLD` is unset, the implementation hardcodes `WORKFLOW_LOCAL_DATA_DIR` to `'.next/workflow-data'`. | |
Member
There was a problem hiding this comment.
AI Review: Note
Documenting a parameter as "typed but ignored" with the hardcoded path exposed may confuse users. Consider either omitting dataDir from the options table until it's functional, or marking it as "Reserved for future use" without exposing the hardcoded internal default.
Contributor
Author
There was a problem hiding this comment.
Fixed — removed dataDir from the options table. Also opened #1619 to remove the dead type from packages/next/src/index.ts (the option was accepted but never read).
| 'docs/content/docs/api-reference/workflow-api/resume-hook.mdx' | ||
| ); | ||
|
|
||
| expect(source).toContain('hook.metadata = await hydrateStepArguments'); |
Member
There was a problem hiding this comment.
AI Review: Note
These assertions match exact source code strings (e.g., hook.metadata = await hydrateStepArguments). If the implementation is refactored (variable renamed, extracted to helper, whitespace changed), this test breaks even when docs remain accurate. Consider testing exported types or runtime behavior rather than grepping source text.
johnlindquist
added a commit
that referenced
this pull request
Apr 6, 2026
Cherry-picked 6 still-needed fixes from #1200 that aren't covered by this audit PR or #1516: - Fix npx workflow description (observability) - Remove fetch from restricted modules list (errors) - Fix package name @workflow-worlds/postgres → @workflow/world-postgres (deploying) - Fix stream wording (foundations/starting-workflows) - Fix import path simple → simple-streaming (foundations/streaming) - Add close(), getEncryptionKeyForRun(), writeToStreamMulti() to World interface, update create() and streamer signatures (deploying/building-a-world)
7 tasks
johnlindquist
force-pushed
the
api-reference-audit
branch
from
4bf191c to
6971b71
Compare
johnlindquist
added a commit
that referenced
this pull request
Apr 6, 2026
Cherry-picked 6 still-needed fixes from #1200 that aren't covered by this audit PR or #1516: - Fix npx workflow description (observability) - Remove fetch from restricted modules list (errors) - Fix package name @workflow-worlds/postgres → @workflow/world-postgres (deploying) - Fix stream wording (foundations/starting-workflows) - Fix import path simple → simple-streaming (foundations/streaming) - Add close(), getEncryptionKeyForRun(), writeToStreamMulti() to World interface, update create() and streamer signatures (deploying/building-a-world)
pranaygp
pushed a commit
that referenced
this pull request
Apr 7, 2026
* docs: clarify Next monorepo setup Prevent confusion when Next.js apps live below the repository root and workflow code imports sibling workspace packages. This documents the output tracing root requirement at the point where users configure withWorkflow, so monorepo setups follow the same working patterns as the shipped Next.js integration instead of failing due to unresolved workspace imports. Ploop-Iter: 1 * ploop: iteration 2 checkpoint Automated checkpoint commit. Ploop-Iter: 2 * ploop: iteration 3 checkpoint Automated checkpoint commit. Ploop-Iter: 3 * docs: audit recent documentation coverage Capture recent workflow documentation updates so the public docs and package guidance stay aligned with the implementation and current docs-typecheck behavior. Ploop-Iter: 1 * docs: align docs-typecheck docs Document the current docs verification contract so contributors do not assume JavaScript examples are type-checked when only TypeScript snippets are enforced today. Add regression coverage around the README language and framework integration guidance to keep those docs aligned with the implemented Next.js and docs-typecheck behavior as future changes land. Ploop-Iter: 2 * docs: add start() troubleshooting guidance Document the most common causes of the invalid workflow function error so users can resolve start() failures from the API docs and Next.js setup flow without having to infer build-time requirements from runtime behavior. Keep the new troubleshooting page aligned with the shipped runtime message and add regression coverage so future wording or cross-link changes do not silently break that guidance. Ploop-Iter: 3 * docs: align NestJS setup docs Document both supported NestJS module formats and add a regression check so the getting-started guide stays aligned with the package README as the integration evolves. Ploop-Iter: 1 * docs: tighten NestJS CommonJS guidance Keep the NestJS getting-started guide consistent across the ESM and CommonJS paths so readers do not mix module settings or import styles mid-setup. Strengthen the docs regression coverage around the later guide sections so future edits are more likely to preserve the supported CommonJS path documented in the package README. Ploop-Iter: 2 * docs: align docs with recent workflow guidance Document the recently added troubleshooting and observability patterns so the public docs stay aligned with the behavior users now encounter in practice. This keeps the NestJS guide, workflow API reference, and docs regression coverage in sync with the runtime-facing guidance from recent changes. Ploop-Iter: 3 * docs: audit docs coverage Why: keep the docs aligned with recent API and runtime behavior changes so examples and reference pages don’t drift from the supported surface. Ploop-Iter: 1 * test: add docs audit guards Add regression coverage for doc surfaces that are easy to drift from implementation so docs audits catch mismatches early and keep published guidance aligned with the supported API surface. Ploop-Iter: 2 * docs: add docs audit guards Keep new observability and server-testing guidance anchored to machine-readable interfaces so follow-up implementation changes do not silently drift away from the documented agent and automation patterns. Ploop-Iter: 3 * docs: align observability troubleshooting guidance Keep the docs consistent so users get the same guidance when debugging hook token collisions and correlating workflow events with platform logs. This prevents the event-sourcing reference from drifting away from the observability and error docs, and adds guard tests to catch regressions. Ploop-Iter: 1 * Remove the unreferenced image file img-a-clean-minimal-technical-architecture-d-2026-02-27T14-07-52-1.png from the repo root, workbench/fastify/public/index.html (a Nitro example mistakenly placed in the fastify workbench by a ploop checkpoint), all .claude/worktrees/* submodule references, and all 15 string-presence audit guard tests in packages/docs-typecheck/src/__tests__/ (they only assert keyword presence, not semantic correctness). None of these belong in the docs audit PR. * Address all PR #1466 review feedback from VaguelySerious, pranaygp, and ijjk: 1. Remove the "Machine-Readable Surfaces" section from docs/content/docs/observability/index.mdx (reviewers say it's unnecessary and already in world docs) 2. Remove all @skip-typecheck annotations from durable-agent.mdx (8) and server-based.mdx (1) — types exist in built packages/ai/dist after pnpm build 3. In durable-agent.mdx, change "machine-readable tool activity" to "tool call details" in the stream() return description 4. In durable-agent.mdx "Aborting Long-Running Streams" section, add a warning callout that abortSignal is not yet supported (blocked by #1301), recommend timeout instead 5. In event-sourcing.mdx, update requestId description: "On Vercel, requestId is the platform request ID when available. Other worlds are not expected to provide a requestId." 6. In get-world.mdx, change "user-friendly names from the machine-readable workflowName field" to "human-readable names from the workflowName field" 7. In start-invalid-workflow-function.mdx, add "// Does NOT work" comment above the bad example line 8. In with-workflow.mdx: reframe outputFileTracingRoot as a workaround (Next.js auto-detects by default per ijjk); change options description from "control local development behavior" to "configure the Next.js integration"; scope the callout to "workflows.local options only affect local development" 9. Drop the withWorkflow() options callout from docs/content/docs/getting-started/next.mdx 10. Remove the Next.js-specific outputFileTracingRoot callout from framework-integrations.mdx 11. Add a Troubleshooting section with the start() invalid-workflow-function error to all 9 non-Next getting-started guides (astro, express, fastify, hono, nestjs, nitro, nuxt, sveltekit, vite), each with framework-appropriate config check in point 2 * docs: absorb unique accuracy fixes from PR #1200 Cherry-picked 6 still-needed fixes from #1200 that aren't covered by this audit PR or #1516: - Fix npx workflow description (observability) - Remove fetch from restricted modules list (errors) - Fix package name @workflow-worlds/postgres → @workflow/world-postgres (deploying) - Fix stream wording (foundations/starting-workflows) - Fix import path simple → simple-streaming (foundations/streaming) - Add close(), getEncryptionKeyForRun(), writeToStreamMulti() to World interface, update create() and streamer signatures (deploying/building-a-world) * docs: address review feedback on March docs audit - durable-agent.mdx: "structured tool activity" → "tool call information" per VaguelySerious's suggestion - next.mdx: drop monorepo callout from getting-started per pranaygp (too much context too early; info is in withWorkflow API ref) * docs: fix 2 typecheck failures in encryption and nestjs guides - encryption.mdx: add skip-typecheck for interface signature block (getEncryptionKeyForRun overloads are not runnable code) - nestjs.mdx: add skip-typecheck for WorkflowModule.forRoot config snippet (fragment inside callout, full import shown above) Verified: pnpm vitest run passes 300/300 in docs-typecheck.
VaguelySerious
pushed a commit
that referenced
this pull request
Apr 7, 2026
* docs: clarify Next monorepo setup Prevent confusion when Next.js apps live below the repository root and workflow code imports sibling workspace packages. This documents the output tracing root requirement at the point where users configure withWorkflow, so monorepo setups follow the same working patterns as the shipped Next.js integration instead of failing due to unresolved workspace imports. Ploop-Iter: 1 * ploop: iteration 2 checkpoint Automated checkpoint commit. Ploop-Iter: 2 * ploop: iteration 3 checkpoint Automated checkpoint commit. Ploop-Iter: 3 * docs: audit recent documentation coverage Capture recent workflow documentation updates so the public docs and package guidance stay aligned with the implementation and current docs-typecheck behavior. Ploop-Iter: 1 * docs: align docs-typecheck docs Document the current docs verification contract so contributors do not assume JavaScript examples are type-checked when only TypeScript snippets are enforced today. Add regression coverage around the README language and framework integration guidance to keep those docs aligned with the implemented Next.js and docs-typecheck behavior as future changes land. Ploop-Iter: 2 * docs: add start() troubleshooting guidance Document the most common causes of the invalid workflow function error so users can resolve start() failures from the API docs and Next.js setup flow without having to infer build-time requirements from runtime behavior. Keep the new troubleshooting page aligned with the shipped runtime message and add regression coverage so future wording or cross-link changes do not silently break that guidance. Ploop-Iter: 3 * docs: align NestJS setup docs Document both supported NestJS module formats and add a regression check so the getting-started guide stays aligned with the package README as the integration evolves. Ploop-Iter: 1 * docs: tighten NestJS CommonJS guidance Keep the NestJS getting-started guide consistent across the ESM and CommonJS paths so readers do not mix module settings or import styles mid-setup. Strengthen the docs regression coverage around the later guide sections so future edits are more likely to preserve the supported CommonJS path documented in the package README. Ploop-Iter: 2 * docs: align docs with recent workflow guidance Document the recently added troubleshooting and observability patterns so the public docs stay aligned with the behavior users now encounter in practice. This keeps the NestJS guide, workflow API reference, and docs regression coverage in sync with the runtime-facing guidance from recent changes. Ploop-Iter: 3 * docs: audit docs coverage Why: keep the docs aligned with recent API and runtime behavior changes so examples and reference pages don’t drift from the supported surface. Ploop-Iter: 1 * test: add docs audit guards Add regression coverage for doc surfaces that are easy to drift from implementation so docs audits catch mismatches early and keep published guidance aligned with the supported API surface. Ploop-Iter: 2 * docs: add docs audit guards Keep new observability and server-testing guidance anchored to machine-readable interfaces so follow-up implementation changes do not silently drift away from the documented agent and automation patterns. Ploop-Iter: 3 * docs: align observability troubleshooting guidance Keep the docs consistent so users get the same guidance when debugging hook token collisions and correlating workflow events with platform logs. This prevents the event-sourcing reference from drifting away from the observability and error docs, and adds guard tests to catch regressions. Ploop-Iter: 1 * Remove the unreferenced image file img-a-clean-minimal-technical-architecture-d-2026-02-27T14-07-52-1.png from the repo root, workbench/fastify/public/index.html (a Nitro example mistakenly placed in the fastify workbench by a ploop checkpoint), all .claude/worktrees/* submodule references, and all 15 string-presence audit guard tests in packages/docs-typecheck/src/__tests__/ (they only assert keyword presence, not semantic correctness). None of these belong in the docs audit PR. * Address all PR #1466 review feedback from VaguelySerious, pranaygp, and ijjk: 1. Remove the "Machine-Readable Surfaces" section from docs/content/docs/observability/index.mdx (reviewers say it's unnecessary and already in world docs) 2. Remove all @skip-typecheck annotations from durable-agent.mdx (8) and server-based.mdx (1) — types exist in built packages/ai/dist after pnpm build 3. In durable-agent.mdx, change "machine-readable tool activity" to "tool call details" in the stream() return description 4. In durable-agent.mdx "Aborting Long-Running Streams" section, add a warning callout that abortSignal is not yet supported (blocked by #1301), recommend timeout instead 5. In event-sourcing.mdx, update requestId description: "On Vercel, requestId is the platform request ID when available. Other worlds are not expected to provide a requestId." 6. In get-world.mdx, change "user-friendly names from the machine-readable workflowName field" to "human-readable names from the workflowName field" 7. In start-invalid-workflow-function.mdx, add "// Does NOT work" comment above the bad example line 8. In with-workflow.mdx: reframe outputFileTracingRoot as a workaround (Next.js auto-detects by default per ijjk); change options description from "control local development behavior" to "configure the Next.js integration"; scope the callout to "workflows.local options only affect local development" 9. Drop the withWorkflow() options callout from docs/content/docs/getting-started/next.mdx 10. Remove the Next.js-specific outputFileTracingRoot callout from framework-integrations.mdx 11. Add a Troubleshooting section with the start() invalid-workflow-function error to all 9 non-Next getting-started guides (astro, express, fastify, hono, nestjs, nitro, nuxt, sveltekit, vite), each with framework-appropriate config check in point 2 * docs: absorb unique accuracy fixes from PR #1200 Cherry-picked 6 still-needed fixes from #1200 that aren't covered by this audit PR or #1516: - Fix npx workflow description (observability) - Remove fetch from restricted modules list (errors) - Fix package name @workflow-worlds/postgres → @workflow/world-postgres (deploying) - Fix stream wording (foundations/starting-workflows) - Fix import path simple → simple-streaming (foundations/streaming) - Add close(), getEncryptionKeyForRun(), writeToStreamMulti() to World interface, update create() and streamer signatures (deploying/building-a-world) * docs: address review feedback on March docs audit - durable-agent.mdx: "structured tool activity" → "tool call information" per VaguelySerious's suggestion - next.mdx: drop monorepo callout from getting-started per pranaygp (too much context too early; info is in withWorkflow API ref) * docs: fix 2 typecheck failures in encryption and nestjs guides - encryption.mdx: add skip-typecheck for interface signature block (getEncryptionKeyForRun overloads are not runnable code) - nestjs.mdx: add skip-typecheck for WorkflowModule.forRoot config snippet (fragment inside callout, full import shown above) Verified: pnpm vitest run passes 300/300 in docs-typecheck.
Keep the API reference aligned with the current hook and webhook runtime behavior so readers do not rely on stale token, overload, or validation semantics. These docs changes clarify the intended runtime story for deterministic hooks versus public webhooks and document the current typed-hook contract exposed by the public API. Ploop-Iter: 1
Why: the runtime API docs were implying an incorrect import surface and incomplete resumeHook behavior, which can mislead users integrating hook resumption and world access. Recording these fixes with a regression test helps keep the docs aligned with the actual runtime entrypoints and overloads as the packages evolve. Ploop-Iter: 2
Why: the runtime API docs had drifted from current resumeHook behavior, which risks misleading users about return values and failure handling. This update keeps the reference aligned with the runtime contract so external integrations can handle hook resumption correctly and distinguish missing hooks from operational failures. Ploop-Iter: 3
Keep the oldest workflow API reference pages aligned with the current public exports so developers do not rely on stale signatures, runtime behavior, or retry semantics.\n\nThis captures the current workflow metadata surface and error/fetch/sleep behavior in the docs, reducing drift in the most stale reference pages before broader API auditing continues.\n\nPloop-Iter: 1
Keep the workflow API reference aligned with current runtime behavior so developers do not rely on stale retry semantics or error guarantees. Accurate docs here matter because these pages are used as normative guidance for step failure handling and fetch behavior inside workflows. Ploop-Iter: 2
Keep the API reference aligned with the real workflow fetch wrapper so readers do not accidentally recurse into the documented helper when writing custom step-based wrappers. Ploop-Iter: 3
Automated checkpoint commit. Ploop-Iter: 1
Align the withWorkflow API reference with the current Next.js integration so developers are not misled about environment variables, lazy discovery behavior, or unsupported options while configuring workflow apps.\n\nPloop-Iter: 2
Keep the Next.js API reference aligned with the current implementation so readers do not rely on a config option shape or environment behavior that no longer reflects runtime behavior. This preserves the API audit effort by documenting the typed-but-ignored local dataDir option accurately and by showing a supported workflow config example instead of an empty placeholder. Ploop-Iter: 3
Keep the workflow API reference aligned with the current core package so developers do not rely on stale runtime constraints or examples. Document the current workflow-only, step-only, and shared API boundaries to reduce misuse of context-sensitive helpers. Ploop-Iter: 1
Align the workflow package overview with the current hook API boundaries so readers are not misled about where hook helpers can be created or resumed. Ploop-Iter: 2
Update the hook API reference docs so they match current runtime semantics and reduce confusion around token generation and execution-context boundaries. This keeps existing guidance accurate for users integrating hook creation and resumption across workflow and runtime code. Ploop-Iter: 3
Keep the API reference aligned with the current public package surface and runtime response behavior so developers are not guided toward stale imports or incorrect webhook handling assumptions. Ploop-Iter: 1
- Remove type signature code blocks from fatal-error.mdx and retryable-error.mdx (match pattern used by other API ref pages) - Use HookNotFoundError.is() static method instead of fragile string-based error.name checks in resume-hook.mdx examples - Distinguish 404 (hook not found) from 500 (other errors) in define-hook.mdx resume example - Remove dataDir from with-workflow.mdx options table (unused option, removal PR: #1619)
johnlindquist
force-pushed
the
api-reference-audit
branch
from
e70d082 to
ecc74c3
Compare
- Replace internal error imports in docs examples with public workflow/errors - Remove brittle source-string assertions from docs runtime checks - Add API reference public contract coverage against exported declarations - Map documented workflow runtime, Next, and utils imports in docs typechecking - Align stale invalid-start error text with the runtime message Oracle-Session: pr-docs-final-review
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
workflow,workflow-api,workflow-next, andworkflow-aipackageswithWorkflowenv var behavior, hook token generation, andresumeWebhookresponse semanticsTest plan
pnpm buildin docs/)