Skip to content

Commit c31ea60

Browse files
authored
Merge branch 'anmol/jobs-refactor' into copilot/sub-pr-404-again
2 parents cc283f9 + 8a46ad3 commit c31ea60

291 files changed

Lines changed: 3373 additions & 11486 deletions

File tree

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

.github/workflows/run-tests.yaml

Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -1,4 +1,4 @@
1-
name: Constructive matrix tests
1+
name: CI tests
22
on:
33
push:
44
branches:
@@ -148,8 +148,8 @@ jobs:
148148

149149
- name: seed app_user
150150
run: |
151-
pnpm --filter @launchql/cli exec node dist/index.js admin-users bootstrap --yes
152-
pnpm --filter @launchql/cli exec node dist/index.js admin-users add --test --yes
151+
pnpm --filter pgpm exec node dist/index.js admin-users bootstrap --yes
152+
pnpm --filter pgpm exec node dist/index.js admin-users add --test --yes
153153
154154
- name: Test ${{ matrix.package }}
155155
run: cd ./${{ matrix.package }} && pnpm test

AGENTS.md

Lines changed: 13 additions & 13 deletions
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,6 @@
1-
# LaunchQL Agent Navigation Guide
1+
# Constructive Agent Navigation Guide
22

3-
This guide helps AI agents quickly navigate and understand the LaunchQL codebase without having to read everything. LaunchQL is a comprehensive full-stack framework for building secure, role-aware GraphQL APIs backed by PostgreSQL databases.
3+
This guide helps AI agents quickly navigate and understand the Constructive codebase without having to read everything. Constructive is a comprehensive full-stack framework for building secure, role-aware GraphQL APIs backed by PostgreSQL databases.
44

55
## 🎯 Quick Start for Agents
66

@@ -12,17 +12,17 @@ This guide helps AI agents quickly navigate and understand the LaunchQL codebase
1212
5. **`packages/types`** - TypeScript type definitions
1313

1414
**Key Classes to Understand:**
15-
- **`LaunchQLPackage`** (`packages/core/src/core/class/launchql.ts`) - Workspace and module management
16-
- **`LaunchQLMigrate`** (`packages/core/src/migrate/client.ts`) - Database migration operations
15+
- **`PgpmPackage`** (`packages/core/src/core/class/pgpm.ts`) - Workspace and module management
16+
- **`PgpmMigrate`** (`packages/core/src/migrate/client.ts`) - Database migration operations
1717

1818
## 📦 Package Categories
1919

2020
### 🏗️ Core Framework
2121
| Package | Purpose | Key Files |
2222
|---------|---------|-----------|
23-
| **`core`** | Main orchestration, migrations, dependency resolution | `src/core/class/launchql.ts`, `src/migrate/client.ts` |
23+
| **`core`** | Main orchestration, migrations, dependency resolution | `src/core/class/pgpm.ts`, `src/migrate/client.ts` |
2424
| **`cli`** | Command-line interface (`lql` command) | `src/commands.ts`, `src/commands/deploy.ts` |
25-
| **`types`** | TypeScript type definitions | `src/launchql.ts` |
25+
| **`types`** | TypeScript type definitions | `src/pgpm.ts` |
2626
| **`env`** | Environment and configuration management | - |
2727

2828
### 🚀 API & Server
@@ -89,8 +89,8 @@ This guide helps AI agents quickly navigate and understand the LaunchQL codebase
8989

9090
## 🔑 Key Classes and Entry Points
9191

92-
### LaunchQLPackage Class
93-
**Location:** `packages/core/src/core/class/launchql.ts`
92+
### PgpmPackage Class
93+
**Location:** `packages/core/src/core/class/pgpm.ts`
9494

9595
**Purpose:** High-level orchestration for workspace and module management
9696

@@ -107,7 +107,7 @@ This guide helps AI agents quickly navigate and understand the LaunchQL codebase
107107
- `getContext()` - Determine if in workspace, module, or outside
108108
- `isInWorkspace()` / `isInModule()` - Context checks
109109

110-
### LaunchQLMigrate Class
110+
### PgpmMigrate Class
111111
**Location:** `packages/core/src/migrate/client.ts`
112112

113113
**Purpose:** Low-level database migration operations
@@ -122,7 +122,7 @@ This guide helps AI agents quickly navigate and understand the LaunchQL codebase
122122

123123
**Configuration:**
124124
- Supports `content` or `ast` hash methods for SQL files
125-
- Configurable via `LaunchQLMigrateOptions`
125+
- Configurable via `PgpmMigrateOptions`
126126

127127
### CLI Command Structure
128128
**Location:** `packages/cli/src/commands.ts`
@@ -150,7 +150,7 @@ export default async (argv: ParsedArgs, prompter: Inquirerer, options: CLIOption
150150
### 1. Module Development Workflow
151151
```typescript
152152
// 1. Initialize workspace
153-
const pkg = new LaunchQLPackage(cwd);
153+
const pkg = new PgpmPackage(cwd);
154154
pkg.initWorkspace();
155155

156156
// 2. Create module
@@ -177,7 +177,7 @@ await db.query('SELECT 1'); // Ready for testing
177177
### 3. Migration Workflow
178178
```typescript
179179
// Direct migration operations
180-
const migrate = new LaunchQLMigrate(pgConfig);
180+
const migrate = new PgpmMigrate(pgConfig);
181181
await migrate.deploy({ modulePath: './my-module' });
182182
await migrate.verify({ modulePath: './my-module' });
183183
```
@@ -187,7 +187,7 @@ await migrate.verify({ modulePath: './my-module' });
187187
### Module Structure
188188
```
189189
my-module/
190-
├── launchql.plan # Migration plan
190+
├── pgpm.plan # Migration plan
191191
├── my-module.control # Extension metadata
192192
├── Makefile # Build configuration
193193
├── deploy/ # Deploy scripts

DEVELOPMENT_JOBS.md

Lines changed: 135 additions & 10 deletions
Original file line numberDiff line numberDiff line change
@@ -2,9 +2,10 @@
22

33
This guide covers a local development workflow for the jobs stack:
44

5-
- Postgres + `launchql-ext-jobs`
5+
- Postgres + `launchql-database-jobs`
66
- LaunchQL API server
77
- `simple-email` function
8+
- `send-email-link` function
89
- `knative-job-service`
910

1011
It assumes:
@@ -58,17 +59,17 @@ Make sure `pgpm` is installed and up to date.
5859

5960
From the `constructive-db/` directory (with `pgenv` applied):
6061

61-
1. Bootstrap admin users:
62+
1. Create the `launchql` database (if it does not already exist):
6263

6364
```sh
64-
pgpm admin-users bootstrap --yes
65-
pgpm admin-users add --test --yes
65+
createdb launchql
6666
```
6767

68-
2. Create the `launchql` database (if it does not already exist):
68+
2. Bootstrap admin users:
6969

7070
```sh
71-
createdb launchql
71+
pgpm admin-users bootstrap --yes
72+
pgpm admin-users add --test --yes
7273
```
7374

7475
3. Deploy the main app and jobs packages into `launchql`:
@@ -102,19 +103,111 @@ This starts:
102103

103104
- `launchql-server` – GraphQL API server
104105
- `simple-email` – Knative-style HTTP function
106+
- `send-email-link` – Knative-style HTTP function
105107
- `knative-job-service` – jobs runtime (callback server + worker + scheduler)
106108

107109
---
108110

109-
## 5. Enqueue a test job (simple-email)
111+
### Switching dry run vs real Mailgun sending
112+
113+
By default, `docker-compose.jobs.yml` runs both email functions in dry-run mode (no real email is sent), and it uses placeholder Mailgun credentials unless you provide `MAILGUN_API_KEY` / `MAILGUN_KEY`.
114+
115+
Quick start commands:
116+
117+
Dry run:
118+
119+
```sh
120+
docker compose -f docker-compose.jobs.yml up -d --build --force-recreate
121+
```
122+
123+
Real sending (Mailgun):
124+
125+
```sh
126+
MAILGUN_API_KEY="your-mailgun-key" MAILGUN_KEY="your-mailgun-key" SIMPLE_EMAIL_DRY_RUN=false SEND_EMAIL_LINK_DRY_RUN=false docker compose -f docker-compose.jobs.yml up -d --build --force-recreate
127+
```
128+
129+
To use a real Mailgun key without editing `docker-compose.jobs.yml`, set these env vars before starting the stack (or put them in a local `.env` file in the `constructive/` directory). Don't commit your `.env`.
130+
131+
```sh
132+
export MAILGUN_API_KEY="your-mailgun-key"
133+
export MAILGUN_KEY="your-mailgun-key"
134+
```
135+
136+
If you're not using `mg.constructive.io`, also override `MAILGUN_DOMAIN`, `MAILGUN_FROM`, and `MAILGUN_REPLY` (for example in the override file below) to match your Mailgun setup.
137+
138+
To actually send email (instead of dry-run), set these env vars (or put them in your local `.env`):
139+
140+
```sh
141+
export SIMPLE_EMAIL_DRY_RUN=false
142+
export SEND_EMAIL_LINK_DRY_RUN=false
143+
```
144+
145+
Then recreate the stack so the new env is applied:
146+
147+
```sh
148+
docker compose -f docker-compose.jobs.yml up -d --build --force-recreate
149+
```
150+
151+
If you prefer not to export env vars, create a local override file (don't commit it) at `docker-compose.jobs.override.yml`:
152+
153+
```yml
154+
services:
155+
simple-email:
156+
environment:
157+
SIMPLE_EMAIL_DRY_RUN: "false"
158+
159+
send-email-link:
160+
environment:
161+
SEND_EMAIL_LINK_DRY_RUN: "false"
162+
```
163+
164+
Start the stack with both files:
165+
166+
```sh
167+
docker compose -f docker-compose.jobs.yml -f docker-compose.jobs.override.yml up -d --build --force-recreate
168+
```
169+
170+
To switch back to dry-run, set `SIMPLE_EMAIL_DRY_RUN=true` and `SEND_EMAIL_LINK_DRY_RUN=true` (or delete the override file) and recreate again.
171+
172+
---
173+
174+
## 5. Ensure GraphQL host routing works for `send-email-link`
175+
176+
LaunchQL selects the API by the HTTP `Host` header using rows in `meta_public.domains`.
177+
178+
For local development, `app-svc-local` seeds `admin.localhost` as the admin API domain. `docker-compose.jobs.yml` adds a Docker network alias so other containers can resolve `admin.localhost` to the `launchql-server` container, and `send-email-link` uses:
179+
180+
- `GRAPHQL_URL=http://admin.localhost:3000/graphql`
181+
182+
Quick check from your host (should return JSON, not HTML):
183+
184+
```sh
185+
curl -s -H 'Host: admin.localhost' \
186+
-H 'Content-Type: application/json' \
187+
-X POST http://localhost:3000/graphql \
188+
--data '{"query":"query { __typename }"}'
189+
```
190+
191+
If your GraphQL server requires auth, set `GRAPHQL_AUTH_TOKEN` before starting the jobs stack (it is passed through to the `send-email-link` container).
192+
193+
---
194+
195+
## 6. Enqueue a test job (simple-email)
110196

111197
With the jobs stack running, you can enqueue a test job from your host into the Postgres container:
112198

199+
First, grab a real `database_id` (required by `send-email-link`, optional for `simple-email`):
200+
201+
```sh
202+
DBID="$(docker exec -i postgres psql -U postgres -d launchql -Atc 'SELECT id FROM collections_public.database ORDER BY created_at LIMIT 1;')"
203+
echo "$DBID"
204+
```
205+
113206
```sh
114207
docker exec -it postgres \
115208
psql -U postgres -d launchql -c "
116209
SELECT app_jobs.add_job(
117-
'00000000-0000-0000-0000-000000000001'::uuid,
210+
'$DBID'::uuid,
118211
'simple-email',
119212
json_build_object(
120213
'to', 'user@example.com',
@@ -129,7 +222,39 @@ You should then see the job picked up by `knative-job-service` and the email pay
129222

130223
---
131224

132-
## 6. Inspect logs and iterate
225+
## 7. Enqueue a test job (`send-email-link`)
226+
227+
`send-email-link` queries GraphQL for site/database metadata, so it requires:
228+
229+
- The app/meta packages deployed in step 3 (`app-svc-local`, `db-meta`)
230+
- A real `database_id` (use `$DBID` above)
231+
- A GraphQL hostname that matches a seeded domain route (step 5)
232+
233+
With `SEND_EMAIL_LINK_DRY_RUN=true` (default in `docker-compose.jobs.yml`), enqueue a job:
234+
235+
```sh
236+
docker exec -it postgres \
237+
psql -U postgres -d launchql -c "
238+
SELECT app_jobs.add_job(
239+
'$DBID'::uuid,
240+
'send-email-link',
241+
json_build_object(
242+
'email_type', 'invite_email',
243+
'email', 'user@example.com',
244+
'invite_token', 'invite123',
245+
'sender_id', '00000000-0000-0000-0000-000000000000'
246+
)::json
247+
);
248+
"
249+
```
250+
251+
You should see a log like:
252+
253+
- `[send-email-link] DRY RUN email (skipping send) ...`
254+
255+
---
256+
257+
## 8. Inspect logs and iterate
133258

134259
To watch logs while you develop:
135260

@@ -153,7 +278,7 @@ docker compose -f docker-compose.jobs.yml up --build
153278

154279
---
155280

156-
## 7. Stopping services
281+
## 9. Stopping services
157282

158283
To stop only the jobs stack:
159284

FOOTER.md

Lines changed: 5 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -47,18 +47,18 @@ Common issues and solutions for pgpm, PostgreSQL, and testing.
4747

4848
### 🔁 Streaming & Uploads
4949

50+
* [etag-hash](https://github.com/constructive-io/constructive/tree/main/packages/etag-hash): **🏷️ S3-compatible ETags** created by streaming and hashing file uploads in chunks.
51+
* [etag-stream](https://github.com/constructive-io/constructive/tree/main/packages/etag-stream): **🔄 ETag computation** via Node stream transformer during upload or transfer.
52+
* [uuid-hash](https://github.com/constructive-io/constructive/tree/main/packages/uuid-hash): **🆔 Deterministic UUIDs** generated from hashed content, great for deduplication and asset referencing.
53+
* [uuid-stream](https://github.com/constructive-io/constructive/tree/main/packages/uuid-stream): **🌊 Streaming UUID generation** based on piped file content—ideal for upload pipelines.
5054
* [launchql/s3-streamer](https://github.com/constructive-io/constructive/tree/main/packages/s3-streamer): **📤 Direct S3 streaming** for large files with support for metadata injection and content validation.
51-
* [launchql/etag-hash](https://github.com/constructive-io/constructive/tree/main/packages/etag-hash): **🏷️ S3-compatible ETags** created by streaming and hashing file uploads in chunks.
52-
* [launchql/etag-stream](https://github.com/constructive-io/constructive/tree/main/packages/etag-stream): **🔄 ETag computation** via Node stream transformer during upload or transfer.
53-
* [launchql/uuid-hash](https://github.com/constructive-io/constructive/tree/main/packages/uuid-hash): **🆔 Deterministic UUIDs** generated from hashed content, great for deduplication and asset referencing.
54-
* [launchql/uuid-stream](https://github.com/constructive-io/constructive/tree/main/packages/uuid-stream): **🌊 Streaming UUID generation** based on piped file content—ideal for upload pipelines.
5555
* [launchql/upload-names](https://github.com/constructive-io/constructive/tree/main/packages/upload-names): **📂 Collision-resistant filenames** utility for structured and unique file names for uploads.
5656

5757
### 🧰 CLI & Codegen
5858

5959
* [pgpm](https://github.com/constructive-io/constructive/tree/main/packages/pgpm): **🖥️ PostgreSQL Package Manager** for modular Postgres development. Works with database workspaces, scaffolding, migrations, seeding, and installing database packages.
6060
* [@launchql/cli](https://github.com/constructive-io/constructive/tree/main/packages/cli): **🖥️ Command-line toolkit** for managing LaunchQL projects—supports database scaffolding, migrations, seeding, code generation, and automation.
61-
* [constructive-io/constructive-gen](https://github.com/constructive-io/constructive/tree/main/packages/launchql-gen): **✨ Auto-generated GraphQL** mutations and queries dynamically built from introspected schema data.
61+
* [launchql-gen](https://github.com/constructive-io/constructive/tree/main/packages/launchql-gen): **✨ Auto-generated GraphQL** mutations and queries dynamically built from introspected schema data.
6262
* [@launchql/query-builder](https://github.com/constructive-io/constructive/tree/main/packages/query-builder): **🏗️ SQL constructor** providing a robust TypeScript-based query builder for dynamic generation of `SELECT`, `INSERT`, `UPDATE`, `DELETE`, and stored procedure calls—supports advanced SQL features like `JOIN`, `GROUP BY`, and schema-qualified queries.
6363
* [@launchql/query](https://github.com/constructive-io/constructive/tree/main/packages/query): **🧩 Fluent GraphQL builder** for PostGraphile schemas. ⚡ Schema-aware via introspection, 🧩 composable and ergonomic for building deeply nested queries.
6464

0 commit comments

Comments
 (0)