Skip to content

Dynamic resources / functions#172

Merged
Yostra merged 19 commits into
v2from
v2-straya
Mar 26, 2026
Merged

Dynamic resources / functions#172
Yostra merged 19 commits into
v2from
v2-straya

Conversation

@Yostra
Copy link
Copy Markdown
Collaborator

@Yostra Yostra commented Mar 24, 2026

Summary

Use openapi to generate resources inside resourceRegistry.ts

  • Schema
  • List functions
  • Nested functions

How to test (optional)

To see what resources it finds syncable:

node packages/source-stripe/dist/bin.js discover --config '{"api_key":"$STRIPE_API_KEY"}' | json_pp | grep resource_name

To sync all resources:

node dist/cli.js \
  --stripe-api-key "$STRIPE_API_KEY" \
  --postgres-url "$DATABASE_URL" \
  --backfill-limit 10 \
  --no-state

Copy link
Copy Markdown
Collaborator

@tonyxiao tonyxiao left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

small merge conflict btw

Copy link
Copy Markdown
Collaborator

@tonyxiao tonyxiao left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

looking great overall! added couple comments

Comment thread packages/source-stripe/src/resourceRegistry.ts
Comment thread packages/source-stripe/src/resourceRegistry.ts
}))

const config: ResourceConfig = {
order: 1,
Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Lost dependency ordering

Old resources had explicit order (1–19) and dependencies (e.g., prices depend on products). New code sets order: 1 for everything and dependencies: []. This means all resources sync in arbitrary order rather than respecting FK-like dependencies.

Comment thread packages/source-stripe/src/resourceRegistry.ts Outdated

async function fetchSpecForCommit(commitSha: string): Promise<OpenApiSpec> {
const url = `https://raw.githubusercontent.com/stripe/openapi/${commitSha}/openapi/spec3.json`
const url = `https://raw.githubusercontent.com/stripe/openapi/${commitSha}/latest/openapi.spec3.sdk.json`
Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

spec3.sdk.json vs spec3.json

The spec fetch was changed from openapi/spec3.json to latest/openapi.spec3.sdk.json. The SDK spec includes SDK-specific extensions (x-stripe-* etc.) — is this needed for anything, or would the simpler spec3.json suffice? The SDK spec is significantly larger.

Comment thread packages/openapi/specFetchHelper.ts
Comment thread packages/openapi/listFnResolver.ts Outdated
Comment thread packages/openapi/runtimeMappings.ts Outdated
timestamptz: { type: 'string', format: 'date-time' },
}

export function parsedTableToJsonSchema(table: ParsedResourceTable): Record<string, unknown> {
Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Lost x-source-schema and x-expandable-reference

The old parsedTableToJsonSchema set x-source-schema (from table.sourceSchemaName) and x-expandable-reference on output schemas. This new version doesn't include either. The x-expandable-reference annotation was used downstream — is anything still consuming it?

Comment thread packages/openapi/types.ts Outdated
Copy link
Copy Markdown
Collaborator

@tonyxiao tonyxiao left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

i think we should create a list of what we will fix in follow up PRs vs. what we think is good enough for now

Comment thread packages/openapi/runtimeMappings.ts
@Yostra Yostra force-pushed the v2-straya branch 2 times, most recently from f64d1da to 32df468 Compare March 25, 2026 04:08
@tonyxiao
Copy link
Copy Markdown
Collaborator

Rebased onto v2 and added a fixup commit (6bdfb871):

1. Rename @stripe/openapi@stripe/sync-openapi
Every package in the monorepo uses the @stripe/sync-* naming convention (sync-protocol, sync-engine, sync-source-stripe, sync-destination-postgres, etc). Renamed across all imports, package.json, and the e2e test to stay consistent.

2. Remove dead exports from resourceRegistry.ts
These were exported but never imported anywhere:

  • StripeObject, CoreSyncObject, SyncObjectName — all just type X = string with no narrowing
  • CORE_SYNC_OBJECTS — identical re-export of DEFAULT_SYNC_OBJECTS
  • SYNC_OBJECTS — unused
  • RESOURCE_TABLE_NAME_MAP — identity map ({products: 'products', ...})

3. Add missing tarball cleanup in e2e/connector-loading.test.sh
The cleanup() trap was missing rm -f for the openapi tarball, so it would leave a stale stripe-sync-openapi-*.tgz in the repo root after test runs.


Note: CI failures (bin.js spec exited with code null in apps/engine exec tests) are pre-existing on the v2 branch itself — not related to this PR.

@tonyxiao tonyxiao force-pushed the v2-straya branch 8 times, most recently from 07d4ec6 to 7999109 Compare March 25, 2026 11:52
Yostra and others added 3 commits March 25, 2026 19:20
1. Package naming convention: every package in the monorepo uses the
   @stripe/sync-* scope (sync-protocol, sync-engine, sync-source-stripe,
   etc). Rename to @stripe/sync-openapi for consistency.

2. Remove dead exports from resourceRegistry.ts that were never imported
   anywhere: StripeObject, CoreSyncObject, SyncObjectName (all just
   `string` aliases with no narrowing), CORE_SYNC_OBJECTS (redundant
   re-export of DEFAULT_SYNC_OBJECTS), SYNC_OBJECTS, and
   RESOURCE_TABLE_NAME_MAP (identity map of table→table).

3. Add missing cleanup line in e2e/connector-loading.test.sh for the
   openapi tarball — without this the cleanup() trap leaves a stale
   stripe-sync-openapi-*.tgz in the repo root.

Committed-By-Agent: claude
The exports map pointed all conditions at source ./index.ts, which
breaks Node after build — index.ts imports ./specParser.js but only
.ts source files exist. Node needs the built dist/ files.

Use the same pattern as other packages: bun→source for dev,
types/import→dist for Node. Drop the unused publishConfig.exports
since this package is private.

Fixes: bin.js spec exited with code 1: ERR_MODULE_NOT_FOUND
Committed-By-Agent: claude
…nfig

types.ts was deleted in an early commit but was still imported by
runtimeMappings.ts, specParser.ts, and re-exported via index.ts.
Also remove src/openapi from tsconfig exclude so browser.ts barrel
(the package's ./browser subpath export) gets compiled to dist.

Committed-By-Agent: claude
@Yostra Yostra merged commit 3548b45 into v2 Mar 26, 2026
4 checks passed
@Yostra Yostra deleted the v2-straya branch March 26, 2026 20:32
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants