Skip to content

Commit 4489f8e

Browse files
jumskijumski
authored
docs: split compilation docs into startup and manual compilation pages (#537)
# Split Compilation Documentation into Startup and Manual Approaches This PR reorganizes the compilation documentation by splitting it into two distinct concepts: 1. **Startup Compilation** - The primary, recommended approach where flows are automatically compiled when workers start 2. **Manual Compilation** - The HTTP-based approach for explicit compilation in CI/CD workflows Changes include: - Created new `/concepts/startup-compilation/` page focusing on the automatic compilation workflow - Created new `/concepts/manual-compilation/` page for the HTTP-based compilation approach - Added deprecation notices for manual compilation, indicating it may be removed in future releases - Updated navigation links in the sidebar to reflect the new structure - Added redirects from the old `/concepts/compilation/` path to the new pages - Updated all references throughout the documentation to point to the appropriate new pages This change better reflects the recommended workflow where most users rely on startup compilation, while still documenting the manual approach for teams with specific CI/CD needs.
1 parent be2d4c6 commit 4489f8e

File tree

13 files changed

+268
-222
lines changed

13 files changed

+268
-222
lines changed

‎pkgs/website/astro.config.mjs‎

Lines changed: 6 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -356,8 +356,12 @@ export default defineConfig({
356356
},
357357
{ label: 'Data model', link: '/concepts/data-model/' },
358358
{
359-
label: 'Compilation',
360-
link: '/concepts/compilation/',
359+
label: 'Startup Compilation',
360+
link: '/concepts/startup-compilation/',
361+
},
362+
{
363+
label: 'Manual Compilation',
364+
link: '/concepts/manual-compilation/',
361365
},
362366
{
363367
label: 'Worker lifecycle',

‎pkgs/website/redirects.config.mjs‎

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -77,6 +77,8 @@ export const redirects = {
7777

7878
'/concepts/context/': '/concepts/context-object/',
7979
'/concepts/flow-dsl/': '/concepts/understanding-flows/',
80+
'/concepts/compilation/': '/concepts/startup-compilation/',
81+
'/reference/compilation-workflow/': '/concepts/manual-compilation/',
8082

8183
// ============================================================================
8284
// MAIN BRANCH PATH MIGRATIONS (how-to split to build/deploy/reference/concepts)

‎pkgs/website/src/content/docs/build/local-development.mdx‎

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -63,8 +63,8 @@ After this initial registration, pgflow's cron keeps the worker running automati
6363
description="How workers poll and restart"
6464
/>
6565
<LinkCard
66-
title="Auto-Compilation"
67-
href="/concepts/compilation/"
66+
title="Startup Compilation"
67+
href="/concepts/startup-compilation/"
6868
description="How flows are compiled on startup"
6969
/>
7070
</CardGrid>

‎pkgs/website/src/content/docs/concepts/compilation.mdx‎

Lines changed: 0 additions & 202 deletions
This file was deleted.

‎pkgs/website/src/content/docs/concepts/index.mdx‎

Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -30,9 +30,9 @@ Steps execute through **tasks** (the actual units of work) - regular steps have
3030
description="Understanding pgflow's database schema design and table relationships"
3131
/>
3232
<LinkCard
33-
title="Compilation"
34-
href="/concepts/compilation/"
35-
description="How pgflow compiles TypeScript flows to SQL via HTTP"
33+
title="Startup Compilation"
34+
href="/concepts/startup-compilation/"
35+
description="How pgflow automatically compiles flows when workers start"
3636
/>
3737
<LinkCard
3838
title="Worker Lifecycle"

‎pkgs/website/src/content/docs/concepts/manual-compilation.mdx‎

Lines changed: 116 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,116 @@
1+
---
2+
title: Manual Compilation
3+
description: How to manually compile flows to SQL migrations via HTTP
4+
sidebar:
5+
order: 26
6+
---
7+
8+
import { Aside, CardGrid, LinkCard } from "@astrojs/starlight/components";
9+
10+
<Aside type="caution" title="Deprecation Notice">
11+
**Manual compilation via ControlPlane may be removed in a future release.** [Startup Compilation](/concepts/startup-compilation/) is the preferred approach - flows are automatically compiled when workers start, eliminating the need for manual compilation in most workflows.
12+
</Aside>
13+
14+
Some teams prefer explicit compilation for:
15+
- SQL migrations in source control
16+
- Code review of flow structure changes
17+
- CI/CD validation before deployment
18+
19+
## The Compile Command
20+
21+
To compile a flow manually:
22+
23+
```bash frame="none"
24+
npx pgflow compile myFlow
25+
```
26+
27+
This generates a migration file at `supabase/migrations/{timestamp}_create_{slug}_flow.sql`.
28+
29+
The CLI:
30+
1. Sends an HTTP GET request to the ControlPlane edge function
31+
2. Receives the compiled SQL statements
32+
3. Writes them to a timestamped migration file
33+
34+
## How Compilation Works
35+
36+
```
37+
┌─────────────┐ HTTP GET ┌─────────────────────┐
38+
│ pgflow CLI │ ─────────────────>│ ControlPlane Edge │
39+
│ │ │ Function │
40+
│ │ │ │
41+
│ │ SQL Array │ 1. Look up flow │
42+
│ │ <─────────────────│ 2. Call compileFlow│
43+
│ │ │ 3. Return SQL │
44+
│ │ └─────────────────────┘
45+
│ 4. Write │
46+
│ migration │
47+
└─────────────┘
48+
```
49+
50+
The ControlPlane edge function is created during installation:
51+
52+
```typescript title="supabase/functions/pgflow/index.ts"
53+
import { ControlPlane } from '@pgflow/edge-worker';
54+
import * as flows from '../../flows/index.ts';
55+
56+
ControlPlane.serve(flows);
57+
```
58+
59+
It registers all flows by slug and exposes the `/flows/:slug` endpoint for compilation.
60+
61+
## Why HTTP-Based Compilation?
62+
63+
**No local Deno required** - The Supabase Edge Functions runtime handles everything. Users don't need Deno installed.
64+
65+
**Same runtime as production** - Flows are compiled using the exact same Deno environment they'll run in, eliminating "works on my machine" issues.
66+
67+
**Consistent dependency resolution** - The `deno.json` import map in your edge function ensures consistent package versions.
68+
69+
**Simpler CLI** - The CLI is a lightweight Node.js package that makes HTTP requests, rather than needing to bundle the entire compilation infrastructure.
70+
71+
## Adding New Flows
72+
73+
To make a flow available for compilation:
74+
75+
1. Create the flow definition in `supabase/flows/`
76+
2. Export it from `supabase/flows/index.ts`
77+
78+
The ControlPlane automatically picks up all flows exported from your `flows/index.ts`.
79+
80+
<Aside type="note">
81+
When running `supabase functions serve`, code changes are detected automatically and the ControlPlane reloads with your new flows.
82+
</Aside>
83+
84+
## Troubleshooting
85+
86+
### "Could not connect to ControlPlane"
87+
88+
Make sure edge functions are running:
89+
90+
```bash frame="none"
91+
npx supabase functions serve --no-verify-jwt
92+
```
93+
94+
### "Flow not found"
95+
96+
Register your flow in `supabase/functions/pgflow/index.ts` and restart the functions server.
97+
98+
## Related
99+
100+
<CardGrid>
101+
<LinkCard
102+
title="Startup Compilation"
103+
href="/concepts/startup-compilation/"
104+
description="How flows auto-compile when workers start (preferred)"
105+
/>
106+
<LinkCard
107+
title="ControlPlane API"
108+
href="/reference/control-plane-api/"
109+
description="HTTP endpoint reference"
110+
/>
111+
<LinkCard
112+
title="Compile API"
113+
href="/reference/compile-api/"
114+
description="Programmatic compilation with compileFlow()"
115+
/>
116+
</CardGrid>

0 commit comments

Comments
 (0)