Skip to content

Commit 9ec8a37

Browse files
committed
drizzle-kit SDK + CLI envelope refactor
Introduce a programmatic SDK (drizzle-kit/src/sdk/) for generate/push, unify the CLI around an envelope contract (drizzle-kit/src/cli/contract.ts), and extract shared generate/push pipelines into schema.ts. Adds SDK conformance + regression tests and reworks affected CLI/dialect/api modules accordingly.
1 parent b519800 commit 9ec8a37

108 files changed

Lines changed: 3281 additions & 3244 deletions

File tree

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

.claude/skills/tests/SKILL.md

Lines changed: 12 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -7,6 +7,18 @@ description: Anything test-related in the drizzle-orm monorepo — running, writ
77

88
This monorepo's tests fall into two buckets: **unit-style** (no DB, run anywhere) and **DB-backed** (need a real Postgres / MySQL / MariaDB / CockroachDB / MSSQL / SingleStore on a specific port). The DB-backed ones read connection-string env vars from `drizzle-kit/.env`, which itself maps to ports owned by `compose/*.yml`. If any of those three layers is missing — env file, DB container, port match — the test fails with a confusing error (most often `connect ECONNREFUSED` or `process.env.X is not set`). This skill is the fast path through the setup so the user doesn't waste time debugging environment instead of code.
99

10+
## ⛔ Hard rule — never spawn Docker from test code, test runs, or agent narration
11+
12+
**This rule has four halves. Violating any one of them reads to the user as "tests spawned Docker again".**
13+
14+
**1. Test code must not spawn containers.** Never `import Docker`/`dockerode`/`testcontainers`/`GenericContainer` and never `execSync('docker ...')` in a new test file. The legacy `drizzle-kit/tests/<dialect>/mocks.ts::createDockerDB()` and `integration-tests/tests/{pg,mysql,cockroach,mssql,bun,seeder}/*.test.ts::createDockerDB()` paths exist for historical reasons — DO NOT add new ones, and DO NOT call existing ones from new tests. Wrap new DB-backed tests in `describe.skipIf(!process.env.X)` so a missing env var skips silently.
15+
16+
**2. Latent trigger: legacy `?? createDockerDB()` fallbacks are still wired up.** Many `integration-tests/tests/{pg,mysql,cockroach,mssql,bun,seeder}/*.test.ts` files contain `process.env.X ?? await createDockerDB()`. Running `pnpm --filter integration-tests test:*` without the matching env var will silently spin up a container via `dockerode` — even if you've already brought DBs up via `compose/dockers.sh`. The same fallthrough exists in `drizzle-kit/tests/<dialect>/mocks.ts::prepareTestDatabase()`. **Setting env vars is not optional — it is what prevents Docker spawn.** Always export the connection string before invoking these scripts, even when "just running the suite to see what passes". If you discover a new test using this pattern, flag it as tech debt — do not extend it.
17+
18+
**3. Agent narration must not tempt operators into `docker run`.** When you (or a spawned subagent) summarize how to run a new DB-backed test, **always** reference `bash compose/dockers.sh up <dialect>` plus the connection-string env var. Never write "this test starts a postgres container" or suggest `docker run` in a final message — the user pattern-matches that as "tests spawned Docker again", even when the underlying code is well-behaved.
19+
20+
**4. Spawned subagents do not inherit this skill.** When delegating any test-writing or test-running work via the Agent tool, paste rules 1–3 verbatim into the subagent prompt. Subagents only see the prompt you give them, not the project's `.claude/skills/` directory. If a subagent's diff adds `import Docker` or its summary tells the operator to `docker run`, that's the failure mode this rule prevents.
21+
1022
## Workflow (always run in this order)
1123

1224
### 1. Ensure `drizzle-kit/.env` exists

drizzle-kit/.gitignore

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -10,6 +10,7 @@ tests/**/tmp/
1010
!README.md
1111
!CONTRIBUTING.md
1212
!JSON_CONTRACT.md
13+
!SDK.md
1314

1415
!.eslint
1516
!.gitignore

drizzle-kit/JSON_CONTRACT.md

Lines changed: 35 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -13,6 +13,41 @@ It is written for tools and services that call Drizzle Kit programmatically.
1313

1414
When `--json` is enabled, callers should treat `stdout` as the JSON channel. Each command invocation writes a single JSON object to `stdout`.
1515

16+
## Programmatic API
17+
18+
The same JSON contract documented in this file is available as typed root-level exports of the `drizzle-kit` package for programmatic callers — agents, build tools, custom orchestrators. The CLI and SDK share one implementation; the response shapes, status discriminator, hint vocabulary, and error codes documented below apply identically to SDK return values.
19+
20+
```typescript
21+
import { generate, push } from 'drizzle-kit';
22+
import type { GenerateJsonResponse } from 'drizzle-kit';
23+
24+
const response: GenerateJsonResponse = await generate({
25+
dialect: 'postgresql',
26+
schema: './src/db/schema.ts',
27+
out: './drizzle',
28+
});
29+
30+
if (response.status === 'missing_hints') {
31+
// Resolve unresolved items and re-invoke with `hints`. See SDK.md for the full pattern.
32+
}
33+
```
34+
35+
What the SDK gives you over the CLI:
36+
37+
- Typed inputs (`GenerateOptions`, `PushOptions`) — your editor narrows option names and types
38+
- Typed responses (`GenerateJsonResponse`, `PushJsonResponse`) — the union discriminator on `status` lets TypeScript narrow into the `ok`, `no_changes`, `missing_hints`, and `error` branches
39+
- No `--json` flag handling, no stdout parsing — the response object is what the JSON mode would have printed
40+
41+
Cross-reference map:
42+
43+
- All `status` values — see [Response statuses](#response-statuses)
44+
- All `kind` values inside `unresolved` — see [Hints flow](#hints-flow) and the kinds catalog under [status: "missing_hints"](#status-missing_hints)
45+
- All `error.code` values — see [Error codes](#error-codes)
46+
- The `--explain` envelope (SDK callers receive the same payload when `explain: true` is passed) — see [`--explain` in JSON mode](#--explain-in-json-mode)
47+
- The recommended retry-with-hints flow (the SDK equivalent) — see [Recommended automation flow](#recommended-automation-flow)
48+
49+
For a full end-to-end example including `missing_hints` handling, see [SDK.md](./SDK.md).
50+
1651
## Hints flow
1752

1853
Some schema changes are ambiguous (rename vs. create+delete) or unsafe to apply automatically (drops on non-empty data, NOT NULL on nullable columns, etc.). In interactive mode, Drizzle Kit prompts the user. In JSON mode there are no prompts: callers either supply hints up front, or receive a `status: "missing_hints"` response and reply with hints.

drizzle-kit/README.md

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -9,6 +9,8 @@ Check the full documentation on [the website](https://orm.drizzle.team/kit-docs/
99

1010
For machine-readable CLI behavior in non-interactive workflows, see [JSON mode and hints contract](./JSON_CONTRACT.md).
1111

12+
For programmatic / agent consumption, see the [SDK documentation](./SDK.md) — the same JSON contract is available as typed root-level exports (`import { generate, push } from 'drizzle-kit'`).
13+
1214

1315
### How it works
1416

drizzle-kit/SDK.md

Lines changed: 139 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,139 @@
1+
# Drizzle Kit SDK
2+
3+
The same JSON contract the `drizzle-kit` CLI emits is available as typed root-level exports for programmatic callers — agents, build tools, custom orchestrators. The CLI and SDK share one implementation; behavior, status discriminators, hint vocabulary, and error codes are identical to what is documented in [JSON_CONTRACT.md](./JSON_CONTRACT.md).
4+
5+
This document covers the public SDK surface. For the full machine-readable contract (status table, hint catalog, error code table), read [JSON_CONTRACT.md](./JSON_CONTRACT.md).
6+
7+
## Install
8+
9+
```bash
10+
npm install drizzle-kit drizzle-orm
11+
```
12+
13+
## Public surface
14+
15+
Three root-level values plus six type aliases:
16+
17+
| Export | Kind | Purpose |
18+
|---|---|---|
19+
| `defineConfig` | function | Type-checked Drizzle config builder (CLI parity) |
20+
| `generate` | async function | Programmatic equivalent of `drizzle-kit generate --json` |
21+
| `push` | async function | Programmatic equivalent of `drizzle-kit push --json` |
22+
| `Config` | type | Discriminated union of dialect configs |
23+
| `GenerateOptions` | type | Input shape for `generate` |
24+
| `PushOptions` | type | Input shape for `push` |
25+
| `GenerateJsonResponse` | type | Discriminated response union for `generate` |
26+
| `PushJsonResponse` | type | Discriminated response union for `push` |
27+
| `MissingHintsResponse` | type | The `status: 'missing_hints'` branch of both responses |
28+
29+
All other public-surface details (status discriminator values, hint kinds, error codes) live in [JSON_CONTRACT.md](./JSON_CONTRACT.md) and apply identically to SDK return values — they are not restated here.
30+
31+
```typescript
32+
import { defineConfig, generate, push } from 'drizzle-kit';
33+
import type {
34+
Config,
35+
GenerateJsonResponse,
36+
GenerateOptions,
37+
MissingHintsResponse,
38+
PushJsonResponse,
39+
PushOptions,
40+
} from 'drizzle-kit';
41+
```
42+
43+
## Minimal `generate` example
44+
45+
```typescript
46+
import { generate } from 'drizzle-kit';
47+
import type { GenerateJsonResponse } from 'drizzle-kit';
48+
49+
const response: GenerateJsonResponse = await generate({
50+
dialect: 'postgresql',
51+
schema: './src/db/schema.ts',
52+
out: './drizzle',
53+
});
54+
55+
if (response.status === 'ok') {
56+
console.log(`Generated migration at ${response.migration_path}`);
57+
} else if (response.status === 'no_changes') {
58+
console.log('Schema in sync — no migration needed');
59+
} else if (response.status === 'error') {
60+
console.error(`generate failed (${response.error.code})`);
61+
process.exit(1);
62+
}
63+
```
64+
65+
## Minimal `push` example
66+
67+
```typescript
68+
import { push } from 'drizzle-kit';
69+
import type { PushJsonResponse } from 'drizzle-kit';
70+
71+
const response: PushJsonResponse = await push({
72+
dialect: 'postgresql',
73+
schema: './src/db/schema.ts',
74+
url: process.env.DATABASE_URL!,
75+
});
76+
77+
if (response.status === 'ok') {
78+
console.log(`Applied schema changes to the live ${response.dialect} database`);
79+
} else if (response.status === 'no_changes') {
80+
console.log('Database already in sync');
81+
} else if (response.status === 'error') {
82+
console.error(`push failed (${response.error.code})`);
83+
process.exit(1);
84+
}
85+
```
86+
87+
## Handling `status: 'missing_hints'`
88+
89+
The most important agent-facing branch. When `generate` or `push` detects an ambiguity (a rename vs. drop+create, a destructive operation needing confirmation, a new schema declaration), it returns `status: 'missing_hints'` with an `unresolved` array describing each ambiguity. The caller picks a resolution for each item, builds a hint array, and re-invokes with the same options plus `hints`.
90+
91+
Hint vocabulary (which `kind` values are valid, what each `meta` shape contains) is documented in [JSON_CONTRACT.md Hints flow](./JSON_CONTRACT.md#hints-flow) — this guide shows only the SDK call pattern.
92+
93+
```typescript
94+
import { generate } from 'drizzle-kit';
95+
import type { GenerateJsonResponse, MissingHintsResponse } from 'drizzle-kit';
96+
97+
async function generateWithHints(): Promise<GenerateJsonResponse> {
98+
const baseOptions = {
99+
dialect: 'postgresql' as const,
100+
schema: './src/db/schema.ts',
101+
out: './drizzle',
102+
};
103+
104+
// First call — no hints.
105+
const first: GenerateJsonResponse = await generate(baseOptions);
106+
if (first.status !== 'missing_hints') return first;
107+
// first.status === 'missing_hints' from here on — `unresolved` is now narrowed.
108+
109+
// Build resolutions. Each unresolved item dictates which `kind` and `meta` to send.
110+
// See JSON_CONTRACT.md "Catalog: kinds in unresolved items" for the full mapping.
111+
const hints = first.unresolved.map((item) => resolveHint(item));
112+
113+
// Re-invoke with hints serialized as JSON string (matches CLI --hints flag).
114+
return generate({
115+
...baseOptions,
116+
hints: JSON.stringify(hints),
117+
});
118+
}
119+
120+
function resolveHint(item: MissingHintsResponse['unresolved'][number]): unknown {
121+
// Application-specific resolution policy — agent prompts, user prompts, or static rules.
122+
// The returned object must match the `Hint` shape documented in JSON_CONTRACT.md.
123+
throw new Error(`Provide a hint resolution for ${JSON.stringify(item)}`);
124+
}
125+
```
126+
127+
Notes:
128+
129+
- `hints` is passed as a JSON string (mirrors `--hints` on the CLI). For long hint sets, use `hintsFile` with a path to a JSON file instead.
130+
- A second `missing_hints` response after providing hints means the hint set was incomplete or invalid — see [JSON_CONTRACT.md Error codes](./JSON_CONTRACT.md#error-codes) for `invalid_hints` diagnostics.
131+
- The same flow applies to `push` — substitute `push` and `PushJsonResponse` everywhere.
132+
133+
## Cross-references
134+
135+
- [JSON_CONTRACT.md Programmatic API](./JSON_CONTRACT.md#programmatic-api) — short SDK pitch in the v1-stable contract doc
136+
- [JSON_CONTRACT.md Response statuses](./JSON_CONTRACT.md#response-statuses) — full status catalog
137+
- [JSON_CONTRACT.md Hints flow](./JSON_CONTRACT.md#hints-flow) — hint vocabulary and shapes
138+
- [JSON_CONTRACT.md Error codes](./JSON_CONTRACT.md#error-codes) — error.code table
139+
- [JSON_CONTRACT.md Recommended automation flow](./JSON_CONTRACT.md#recommended-automation-flow) — CLI-side equivalent of the SDK pattern above

drizzle-kit/scripts/build.ts

Lines changed: 19 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -89,11 +89,17 @@ async function buildCli() {
8989
console.log(' Built bin.cjs');
9090
}
9191

92+
// tsdown's `emitDtsOnly: true` still emits chunked .js files alongside the
93+
// declarations. If tsdown writes to ./dist it overwrites rolldown's clean
94+
// single-file bundles and produces a CJS entry that fails to load. Direct
95+
// tsdown's output at a temp dir and copy only .d.ts / .d.mts back into ./dist.
96+
const TSDOWN_TEMP_DIR = './dist/__dts-temp';
97+
9298
async function buildDeclarations() {
9399
// Use tsdown for declaration bundling - it handles path resolution and creates bundled .d.ts files
94100
await tsdown({
95101
entry: ['./src/index.ts'],
96-
outDir: './dist',
102+
outDir: TSDOWN_TEMP_DIR,
97103
external: [...driversPackages, /^drizzle-orm\/?/],
98104
dts: { emitDtsOnly: true },
99105
format: ['cjs', 'es'],
@@ -109,7 +115,7 @@ async function buildDeclarations() {
109115

110116
await tsdown({
111117
entry: ['./src/ext/api-postgres.ts', './src/ext/api-mysql.ts', './src/ext/api-sqlite.ts'],
112-
outDir: './dist',
118+
outDir: TSDOWN_TEMP_DIR,
113119
external: ['esbuild', 'drizzle-orm', ...driversPackages, /^drizzle-orm\/?/],
114120
dts: { emitDtsOnly: true },
115121
format: ['cjs', 'es'],
@@ -126,6 +132,16 @@ async function buildDeclarations() {
126132
console.log(' Built declarations');
127133
}
128134

135+
async function copyDeclarationsAndCleanTemp() {
136+
const entries = await fs.readdir(TSDOWN_TEMP_DIR);
137+
await Promise.all(
138+
entries
139+
.filter((f) => f.endsWith('.d.ts') || f.endsWith('.d.mts'))
140+
.map((f) => fs.copyFile(`${TSDOWN_TEMP_DIR}/${f}`, `dist/${f}`)),
141+
);
142+
rmSync(TSDOWN_TEMP_DIR, { recursive: true, force: true });
143+
}
144+
129145
async function postProcessApiFiles() {
130146
const apiFiles = ['dist/api-postgres.js', 'dist/api-mysql.js', 'dist/api-sqlite.js'];
131147
await Promise.all(
@@ -195,6 +211,7 @@ async function main() {
195211
buildDeclarations(),
196212
]);
197213

214+
await copyDeclarationsAndCleanTemp();
198215
await Promise.all([
199216
postProcessApiFiles(),
200217
fs.copyFile('package.json', 'dist/package.json'),

drizzle-kit/src/cli/commands/check.ts

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -1,7 +1,7 @@
11
import chalk from 'chalk';
22
import { readFileSync } from 'fs';
3-
import { getCommutativityDialect } from 'src/commutativity';
4-
import type { MigrationNode, NonCommutativityReport, UnifiedBranchConflict } from 'src/commutativity/types';
3+
import { getCommutativityDialect } from '../../commutativity';
4+
import type { MigrationNode, NonCommutativityReport, UnifiedBranchConflict } from '../../commutativity/types';
55
import type { Dialect } from '../../utils/schemaValidator';
66
import { prepareOutFolder, validatorForDialect } from '../../utils/utils-node';
77
import { CheckCliError } from '../errors';

drizzle-kit/src/cli/commands/generate-cockroach.ts

Lines changed: 19 additions & 23 deletions
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,3 @@
1-
import { fromDrizzleSchema, prepareFromSchemaFiles } from 'src/dialects/cockroach/drizzle';
2-
import { prepareOutFolder } from 'src/utils/utils-node';
31
import type {
42
CheckConstraint,
53
CockroachEntities,
@@ -15,17 +13,13 @@ import type {
1513
} from '../../dialects/cockroach/ddl';
1614
import { createDDL, interimToDDL } from '../../dialects/cockroach/ddl';
1715
import { ddlDiff, ddlDiffDry } from '../../dialects/cockroach/diff';
16+
import { fromDrizzleSchema, prepareFromSchemaFiles } from '../../dialects/cockroach/drizzle';
1817
import { prepareSnapshot } from '../../dialects/cockroach/serializer';
18+
import { prepareOutFolder } from '../../utils/utils-node';
1919
import { isJsonMode } from '../context';
20+
import { CommandOutputCliError } from '../errors';
2021
import { resolver } from '../prompts';
21-
import {
22-
cockroachSchemaError,
23-
cockroachSchemaWarning,
24-
explain,
25-
explainJsonOutput,
26-
humanLog,
27-
printJsonOutput,
28-
} from '../views';
22+
import { cockroachSchemaError, cockroachSchemaWarning, explain, explainJsonOutput, humanLog } from '../views';
2923
import { writeResult } from './generate-common';
3024
import type { ExportConfig, GenerateConfig } from './utils';
3125

@@ -36,7 +30,7 @@ export const handle = async (config: GenerateConfig) => {
3630
const { snapshots } = prepareOutFolder(outFolder);
3731
const { ddlCur, ddlPrev, snapshot, custom } = await prepareSnapshot(snapshots, filenames);
3832
if (config.custom) {
39-
writeResult({
33+
return writeResult({
4034
snapshot: custom,
4135
sqlStatements: [],
4236
outFolder,
@@ -47,7 +41,6 @@ export const handle = async (config: GenerateConfig) => {
4741
renames: [],
4842
snapshots,
4943
});
50-
return;
5144
}
5245

5346
const { sqlStatements, renames, groupedStatements, statements } = await ddlDiff(
@@ -68,11 +61,11 @@ export const handle = async (config: GenerateConfig) => {
6861
);
6962

7063
if (json && config.hints.hasMissingHints()) {
71-
config.hints.emitAndExit();
64+
return config.hints.toResponse();
7265
}
7366

7467
if (!config.explain) {
75-
writeResult({
68+
return writeResult({
7669
snapshot: snapshot,
7770
sqlStatements,
7871
outFolder,
@@ -82,22 +75,21 @@ export const handle = async (config: GenerateConfig) => {
8275
renames,
8376
snapshots,
8477
});
85-
return;
8678
}
8779

8880
if (json) {
8981
if (sqlStatements.length === 0) {
90-
printJsonOutput({ status: 'no_changes', dialect: 'cockroach' });
91-
return;
82+
return { status: 'no_changes' as const, dialect: 'cockroach' };
9283
}
93-
printJsonOutput(explainJsonOutput('cockroach', statements, []));
94-
return;
84+
return explainJsonOutput('cockroach', statements, []);
9585
}
9686

9787
const explainMessage = explain('cockroach', groupedStatements, []);
9888
if (explainMessage) {
9989
humanLog(explainMessage);
10090
}
91+
92+
return { status: 'ok' as const, dialect: 'cockroach' };
10193
};
10294

10395
export const handleExport = async (config: ExportConfig) => {
@@ -112,15 +104,19 @@ export const handleExport = async (config: ExportConfig) => {
112104
}
113105

114106
if (errors.length > 0) {
115-
console.log(errors.map((it) => cockroachSchemaError(it)).join('\n'));
116-
process.exit(1);
107+
throw new CommandOutputCliError('generate', errors.map((it) => cockroachSchemaError(it)).join('\n'), {
108+
stage: 'schema',
109+
dialect: 'cockroach',
110+
});
117111
}
118112

119113
const { ddl, errors: errors2 } = interimToDDL(schema);
120114

121115
if (errors2.length > 0) {
122-
console.log(errors2.map((it) => cockroachSchemaError(it)).join('\n'));
123-
process.exit(1);
116+
throw new CommandOutputCliError('generate', errors2.map((it) => cockroachSchemaError(it)).join('\n'), {
117+
stage: 'ddl',
118+
dialect: 'cockroach',
119+
});
124120
}
125121

126122
const { sqlStatements } = await ddlDiffDry(createDDL(), ddl, 'default');

0 commit comments

Comments
 (0)