Skip to content

Commit 33fd93c

Browse files
YostraYostra
committed
format
1 parent 9d82bf2 commit 33fd93c

6 files changed

Lines changed: 38 additions & 48 deletions

File tree

‎docs/pages/engine/catalog-workflow.md‎

Lines changed: 12 additions & 11 deletions
Original file line numberDiff line numberDiff line change
@@ -49,12 +49,12 @@ source.discover({ config }) → CatalogMessage
4949

5050
Each **Stream** has:
5151

52-
| Field | Description |
53-
| -------------- | ------------------------------------------------------------------ |
54-
| `name` | Table/resource name (e.g. `customers`, `invoices`) |
55-
| `primary_key` | Paths to fields that uniquely identify a record (e.g. `[["id"]]`) |
56-
| `json_schema` | JSON Schema describing the record shape (from OpenAPI or runtime) |
57-
| `metadata` | Source-specific context (e.g. `api_version`, `account_id`) |
52+
| Field | Description |
53+
| ------------- | ----------------------------------------------------------------- |
54+
| `name` | Table/resource name (e.g. `customers`, `invoices`) |
55+
| `primary_key` | Paths to fields that uniquely identify a record (e.g. `[["id"]]`) |
56+
| `json_schema` | JSON Schema describing the record shape (from OpenAPI or runtime) |
57+
| `metadata` | Source-specific context (e.g. `api_version`, `account_id`) |
5858

5959
For the Stripe source, `discover()` walks the OpenAPI spec and the resource registry to produce one stream per syncable Stripe resource.
6060

@@ -64,11 +64,11 @@ The user (or application) selects which streams to sync and how. This is where *
6464

6565
For each selected stream, the user decides:
6666

67-
| Setting | Options | Description |
68-
| ------------------------ | --------------------------------------------- | ------------------------------------------ |
69-
| `sync_mode` | `full_refresh` · `incremental` | Pull everything each time, or only changes |
70-
| `destination_sync_mode` | `append` · `overwrite` · `append_dedup` | How records land in the destination |
71-
| `cursor_field` | field path (optional) | Which field tracks changes for incremental |
67+
| Setting | Options | Description |
68+
| ----------------------- | --------------------------------------- | ------------------------------------------ |
69+
| `sync_mode` | `full_refresh` · `incremental` | Pull everything each time, or only changes |
70+
| `destination_sync_mode` | `append` · `overwrite` · `append_dedup` | How records land in the destination |
71+
| `cursor_field` | field path (optional) | Which field tracks changes for incremental |
7272

7373
This produces a **Configured Stream** — the original stream data plus the user's sync settings:
7474

@@ -109,6 +109,7 @@ destination.setup({ config, catalog }) → creates schemas, tables, indexes
109109
```
110110

111111
The destination uses the configured catalog to know:
112+
112113
- Which tables to create (from `stream.name`)
113114
- What columns each table has (from `stream.json_schema`)
114115
- What primary key to use (from `stream.primary_key`)

‎packages/openapi/index.ts‎

Lines changed: 1 addition & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -1,9 +1,5 @@
11
export type * from './types'
2-
export {
3-
SpecParser,
4-
OPENAPI_RESOURCE_TABLE_ALIASES,
5-
RUNTIME_RESOURCE_ALIASES,
6-
} from './specParser'
2+
export { SpecParser, OPENAPI_RESOURCE_TABLE_ALIASES, RUNTIME_RESOURCE_ALIASES } from './specParser'
73
export { OPENAPI_COMPATIBILITY_COLUMNS } from './runtimeMappings'
84
export { WritePathPlanner } from './writePathPlanner'
95
export { resolveOpenApiSpec } from './specFetchHelper'

‎packages/openapi/jsonSchemaConverter.ts‎

Lines changed: 1 addition & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -9,9 +9,7 @@ const SCALAR_TYPE_TO_JSON_SCHEMA: Record<ScalarType, { type: string; format?: st
99
timestamptz: { type: 'string', format: 'date-time' },
1010
}
1111

12-
export function parsedTableToJsonSchema(
13-
table: ParsedResourceTable
14-
): Record<string, unknown> {
12+
export function parsedTableToJsonSchema(table: ParsedResourceTable): Record<string, unknown> {
1513
const properties: Record<string, unknown> = {
1614
id: { type: 'string' },
1715
}

‎packages/openapi/tsconfig.json‎

Lines changed: 3 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -10,10 +10,6 @@
1010
"strict": true,
1111
"resolveJsonModule": true
1212
},
13-
"include": [
14-
"*.ts"
15-
],
16-
"exclude": [
17-
"__tests__"
18-
]
19-
}
13+
"include": ["*.ts"],
14+
"exclude": ["__tests__"]
15+
}

‎packages/source-stripe/src/types.ts‎

Lines changed: 0 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -55,7 +55,6 @@ export type ResourceConfig = BaseResourceConfig & {
5555
parentParamName?: string
5656
}
5757

58-
5958
export type RevalidateEntity = RevalidateEntityName
6059

6160
export const SUPPORTED_WEBHOOK_EVENTS: Stripe.WebhookEndpointCreateParams.EnabledEvent[] = [

‎scripts/README.md‎

Lines changed: 21 additions & 21 deletions
Original file line numberDiff line numberDiff line change
@@ -6,11 +6,11 @@ Demo and utility scripts. All `.sh` scripts require `pnpm build` first.
66

77
These invoke source/destination connector CLIs directly — no engine in between.
88

9-
| Script | Description |
10-
|--------|-------------|
11-
| `read-from-stripe.sh` | Read products from Stripe, emit NDJSON to stdout |
12-
| `write-to-sheets.sh` | Write NDJSON to Google Sheets (stdin or sample data) |
13-
| `write-to-postgres.sh` | Write NDJSON to Postgres (stdin or sample data) |
9+
| Script | Description |
10+
| ---------------------- | ---------------------------------------------------- |
11+
| `read-from-stripe.sh` | Read products from Stripe, emit NDJSON to stdout |
12+
| `write-to-sheets.sh` | Write NDJSON to Google Sheets (stdin or sample data) |
13+
| `write-to-postgres.sh` | Write NDJSON to Postgres (stdin or sample data) |
1414

1515
Pipe them together:
1616

@@ -23,28 +23,28 @@ Pipe them together:
2323

2424
These use the sync-engine CLI which handles discover → read → write in one shot.
2525

26-
| Script | Description |
27-
|--------|-------------|
28-
| `stripe-to-postgres.sh` | Sync Stripe products to Postgres |
26+
| Script | Description |
27+
| ---------------------------- | ------------------------------------- |
28+
| `stripe-to-postgres.sh` | Sync Stripe products to Postgres |
2929
| `stripe-to-google-sheets.sh` | Sync Stripe products to Google Sheets |
3030

3131
## Utilities
3232

33-
| Script | Description |
34-
|--------|-------------|
35-
| `reset-postgres.sh` | Drop all non-system schemas and tables in public |
36-
| `release-package.sh` | Create a GitHub release with a built tarball |
37-
| `d2.mjs` | Render D2 diagrams to SVG |
33+
| Script | Description |
34+
| -------------------- | ------------------------------------------------ |
35+
| `reset-postgres.sh` | Drop all non-system schemas and tables in public |
36+
| `release-package.sh` | Create a GitHub release with a built tarball |
37+
| `d2.mjs` | Render D2 diagrams to SVG |
3838

3939
## Environment variables
4040

4141
Set in `.envrc` (via direnv):
4242

43-
| Variable | Used by |
44-
|----------|---------|
45-
| `STRIPE_API_KEY` | Stripe scripts |
46-
| `DATABASE_URL` | Postgres scripts |
47-
| `GOOGLE_CLIENT_ID` | Sheets scripts |
48-
| `GOOGLE_CLIENT_SECRET` | Sheets scripts |
49-
| `GOOGLE_REFRESH_TOKEN` | Sheets scripts |
50-
| `GOOGLE_SPREADSHEET_ID` | Sheets scripts |
43+
| Variable | Used by |
44+
| ----------------------- | ---------------- |
45+
| `STRIPE_API_KEY` | Stripe scripts |
46+
| `DATABASE_URL` | Postgres scripts |
47+
| `GOOGLE_CLIENT_ID` | Sheets scripts |
48+
| `GOOGLE_CLIENT_SECRET` | Sheets scripts |
49+
| `GOOGLE_REFRESH_TOKEN` | Sheets scripts |
50+
| `GOOGLE_SPREADSHEET_ID` | Sheets scripts |

0 commit comments

Comments
 (0)