Skip to content

Commit 75a0bca

Browse files
os-zhuangos-zhuang
committed
2 parents fb3831c + 3be7fb9 commit 75a0bca

2 files changed

Lines changed: 297 additions & 0 deletions

File tree

‎docs/adr/0023-openapi-to-connector-generator.md‎

Lines changed: 148 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,148 @@
1+
# ADR-0023: OpenAPI → Connector Generator — Bulk Connectors From Declarative API Specs
2+
3+
**Status**: Proposed (2026-06-01)
4+
**Deciders**: ObjectStack Protocol Architects
5+
**Builds on**: [ADR-0015](./0015-external-datasource-federation.md) (open mechanism / enterprise lifecycle split), [ADR-0018](./0018-unified-node-action-registry.md) (`connector_action` baseline dispatch + `engine.registerConnector()`), [ADR-0022](./0022-connectors-vs-messaging-channels.md) (Connector = transport/integration mechanism; not the messaging-semantic layer)
6+
**Consumers**: `@objectstack/spec` (`integration/connector.zod.ts`), `@objectstack/service-automation` (connector registry on the engine), `@objectstack/runtime` (the `GET /api/v1/automation/connectors` discovery route), new `@objectstack/connector-openapi`, the Studio flow palette
7+
8+
---
9+
10+
## TL;DR
11+
12+
We want to populate the connector registry with *many* third-party APIs without hand-writing each one. The honest finding from surveying ecosystems like **activepieces** (697 community pieces, MIT) is that their integrations are **code bound to their engine** (`createAction({ run: (ctx) => … })` executed inside a V8/`isolated-vm` sandbox with an injected `ActionContext`) — they are not portable into our model, which is **a serializable `Connector` definition + plain `(input, ctx) => Promise<output>` handlers** registered in-process (ADR-0018/0022). You can *port the endpoint knowledge*, but you cannot *drop in* a piece.
13+
14+
The declarative substrate that **does** map cleanly onto our model is **OpenAPI** (and Arazzo for multi-step). An OpenAPI document is vendor-neutral, fully declarative, and already describes exactly what a `Connector` needs: a base URL, an auth scheme, and a set of operations with input/output JSON Schemas.
15+
16+
**Decision:** ship `@objectstack/connector-openapi` — a generator that turns an OpenAPI 3.x document into a `Connector` definition (one operation → one action) plus a single generic HTTP handler that executes any operation. It reuses the existing `connector-rest` request machinery; it adds **no new runtime abstraction** and stays inside the ADR-0022 open-source line (static auth only).
17+
18+
---
19+
20+
## Context
21+
22+
### Why not just import activepieces / n8n / Pipedream nodes?
23+
24+
| Their model | Our model |
25+
|:---|:---|
26+
| Integration = a TS class instance (`new IAction(...)`) + closures | Connector = serializable JSON (`ConnectorSchema`) + handler functions |
27+
| `run(ctx)` receives a heavyweight `ActionContext` (auth, propsValue, store, files, connections, server callbacks, flow pause/resume) | `handler(input, ctx)` receives a plain `Record` and returns a plain `Record`; the registry lives on the engine ([engine.ts](../../packages/services/service-automation/src/engine.ts), `registerConnector`/`unregisterConnector`) |
28+
| Executed by *their* engine inside a sandbox (`packages/server/engine`, `isolated-vm`) | Executed in-process by the `connector_action` node ([connector-nodes.ts](../../packages/services/service-automation/src/builtin/connector-nodes.ts)) |
29+
| Dynamic props (server-side Dropdown), managed OAuth2 refresh | Static JSON Schema inputs; static auth (`none`/`api-key`/`basic`/`bearer`) open-source |
30+
31+
So a piece's `run()` cannot be called without their engine. What *is* reusable is the per-API knowledge (endpoints, params, error codes) — but that knowledge is far more cheaply harvested from a vendor's **OpenAPI spec** than reverse-engineered from a piece's closures. Many of the APIs those projects wrap publish OpenAPI documents directly.
32+
33+
### What a Connector needs vs. what OpenAPI provides
34+
35+
Our `Connector` (from [`connector.zod.ts`](../../packages/spec/src/integration/connector.zod.ts)) requires, at the open-source tier:
36+
37+
- `name`, `label`, `type: 'api'`, optional `icon`/`description`
38+
- `authentication` — one of the static schemes
39+
- `actions: { key, label, description?, inputSchema?, outputSchema? }[]` where the schemas are **JSON Schema**
40+
41+
OpenAPI 3.x supplies every one of these:
42+
43+
| Connector field | OpenAPI source |
44+
|:---|:---|
45+
| `label` / `description` | `info.title` / `info.description` |
46+
| base URL | `servers[0].url` |
47+
| `authentication` | `components.securitySchemes` (apiKey → `api-key`, http basic → `basic`, http bearer → `bearer`, none → `none`) |
48+
| `actions[].key` | `operationId` (fallback: `${method} ${path}` slug) |
49+
| `actions[].label` / `description` | `operation.summary` / `operation.description` |
50+
| `actions[].inputSchema` | a JSON Schema assembled from `parameters` (path/query/header) + `requestBody` |
51+
| `actions[].outputSchema` | the `200`/`2xx` `responses[*].content['application/json'].schema` |
52+
53+
The fit is near-1:1 because `ConnectorActionSchema.inputSchema/outputSchema` are already typed as free-form JSON Schema (`z.record(z.string(), z.unknown())`). No spec change is required.
54+
55+
### Why this is cheap
56+
57+
`@objectstack/connector-rest` already does the hard runtime part — build URL from base+path+query, apply static auth, JSON-encode the body, normalise the response to `{ status, ok, body }` ([rest-connector.ts](../../packages/connectors/connector-rest/src/rest-connector.ts)). The OpenAPI connector is therefore **mostly a definition generator** plus a thin handler that maps a generated action back to an HTTP call — and it can be built literally *on top of* a `createRestConnector(...)` instance.
58+
59+
---
60+
61+
## Decision
62+
63+
### 1. A new open package: `@objectstack/connector-openapi`
64+
65+
Mirror the `connector-rest` package conventions (Apache-2.0, `private: false`, peerDeps on `@objectstack/core` + `@objectstack/spec`, tsup build). It exports a pure generator and a plugin, exactly like the REST and Slack connectors.
66+
67+
```ts
68+
// packages/connectors/connector-openapi/src/openapi-connector.ts
69+
import type { Connector } from '@objectstack/spec/integration';
70+
import type { RestConnectorBundle } from '@objectstack/connector-rest';
71+
72+
export interface OpenApiConnectorOptions {
73+
/** Connector machine name (snake_case). Defaults to a slug of info.title. */
74+
name?: string;
75+
/** The parsed OpenAPI 3.x document (caller loads/derefs it). */
76+
document: OpenApiDocument;
77+
/** Override the base URL (else servers[0].url). */
78+
baseUrl?: string;
79+
/** Static auth; if omitted, inferred from securitySchemes (best-effort) and
80+
* credentials still supplied by the caller. */
81+
auth?: RestAuth;
82+
/** Only include operations whose operationId/tag matches (allowlist). */
83+
include?: (op: OperationInfo) => boolean;
84+
/** Injected for tests; defaults to global fetch. */
85+
fetchImpl?: typeof fetch;
86+
}
87+
88+
/** Returns a Connector definition + one handler per generated action. */
89+
export function createOpenApiConnector(opts: OpenApiConnectorOptions): RestConnectorBundle;
90+
```
91+
92+
It returns the **same `RestConnectorBundle` shape** (`{ def, handlers }`) that `engine.registerConnector(def, handlers)` already consumes — so the generator output is registered through the existing path with zero new engine surface.
93+
94+
### 2. One operation → one action; one generic handler
95+
96+
For each selected OpenAPI operation, emit a `ConnectorActionSchema` entry:
97+
98+
- `key` = `operationId` (deterministic slug fallback when missing)
99+
- `inputSchema` = `{ type: 'object', properties: { path, query, header, body }, required: [...] }` assembled from the operation's parameters + requestBody
100+
- `outputSchema` = the success response schema
101+
102+
The handler closes over the operation's `(method, pathTemplate)` and reuses the REST request logic: interpolate `path` params into the template, pass `query`/`header`/`body` straight through `createRestConnector`'s `request`. Concretely, the OpenAPI connector can be implemented **on top of** a `createRestConnector(...)` instance — one shared HTTP/auth implementation, per ADR-0022's "one transport" principle.
103+
104+
### 3. Generation is build-time-or-boot-time, never request-time
105+
106+
The generator runs when the plugin is constructed (boot) or offline via a CLI that writes a `*.connector.json`. The registry and the `connector_action` node see only a finished `Connector` — identical to a hand-written one. Discovery (`GET /api/v1/automation/connectors`, served by [http-dispatcher.ts](../../packages/runtime/src/http-dispatcher.ts) via the engine's `getConnectorDescriptors()`, [engine.ts](../../packages/services/service-automation/src/engine.ts)) and the Studio palette get the generated actions for free, with no awareness that they came from OpenAPI.
107+
108+
### 4. Open-source / enterprise boundary (consistent with ADR-0015 / 0022)
109+
110+
| Capability | Tier |
111+
|:---|:---|
112+
| OpenAPI 3.x → `Connector` generation, generic HTTP handler, static auth (`none`/`api-key`/`basic`/`bearer`) | **open** |
113+
| CLI to pre-generate `*.connector.json` from a spec URL/file | **open** |
114+
| Managed OAuth2 (authorize + token refresh), `service-secrets` credential vault, per-tenant connection lifecycle | **enterprise** (per ADR-0015) |
115+
| Curated/marketplace connector catalog, paid premium specs | **enterprise** |
116+
117+
A spec that declares OAuth2 is still importable open-source — we generate the actions and let the caller inject a static bearer token; the *managed* OAuth refresh is the enterprise line, exactly as ADR-0022 drew it for Slack.
118+
119+
### 5. Anti-patterns this rules out
120+
121+
- ❌ **A bespoke importer per vendor.** If the vendor publishes OpenAPI, the generic generator covers it; don't hand-roll.
122+
- ❌ **Runtime spec parsing inside the node.** The `connector_action` node stays a dumb dispatcher; all generation happens before registration.
123+
- ❌ **Inventing a new connector sub-type.** Generated connectors are ordinary `type: 'api'` connectors; no schema fork.
124+
- ❌ **Trying to auto-import activepieces/n8n nodes.** Those are engine-bound code; treat them as *reference material* for endpoints, not as importable artifacts.
125+
126+
---
127+
128+
## Consequences
129+
130+
**Positive**
131+
- Hundreds of REST APIs become connectors from their published specs, with no per-API code and no new abstraction.
132+
- Generated connectors are indistinguishable from hand-written ones to the registry, discovery endpoint, palette, and AI tooling.
133+
- One HTTP/auth implementation (`connector-rest`) backs hand-written, Slack, and OpenAPI-generated connectors alike.
134+
135+
**Negative / costs**
136+
- OpenAPI quality varies: missing `operationId`, untyped responses, `oneOf`/`allOf`, and `$ref` cycles need a deref+normalise pass. Mitigated by deterministic fallbacks and an `include` allowlist so a messy spec degrades to a usable subset rather than failing wholesale.
137+
- Large specs generate large action lists; the `include` filter and tag-based selection keep the palette manageable.
138+
- Non-REST / GraphQL / gRPC APIs are out of scope (OpenAPI only). Arazzo (multi-step workflows) is a future extension, not v1.
139+
140+
---
141+
142+
## Status & follow-ups
143+
144+
- **This ADR changes no shipped code.** It records the decision to add a generator package on top of the existing connector baseline.
145+
- Follow-up: scaffold `packages/connectors/connector-openapi` mirroring `connector-rest`, with `createOpenApiConnector` + `ConnectorOpenApiPlugin`.
146+
- Follow-up: a small `openapi-to-connector` CLI that emits a reviewable `*.connector.json`.
147+
- Follow-up: a worked example (e.g. GitHub or Stripe public OpenAPI → a handful of allowlisted actions) under `examples/`, paralleling the worked `connector_action` example from ADR-0022.
148+
- Cross-reference: see [ADR-0024](./0024-mcp-connectors.md) for the complementary path — wrapping live **MCP servers** as connectors when no OpenAPI spec exists but an MCP server does.

0 commit comments

Comments
 (0)