Skip to content

Commit f485e4a

Browse files
authored
Unify auth methods across OpenAPI, GraphQL, and MCP on a shared placements model (#943)
* Add @executor-js/http-auth: shared placements-based auth-method model One vocabulary for the HTTP protocol plugins (openapi/graphql/mcp): an apikey method carries N header/query placements, each bound to its own credential variable (or sharing one), plus literal placements for static values. Includes the renderer, required-variable extraction, slug normalization, and catalog projection helpers. Core gains the carrier-agnostic getValues seam on ResolveToolsInput (multi-input analog of getValue) and the catalog wire schema now carries placement variable/literal so multi-input methods survive the HTTP boundary. Removes the dead http-source and graphql-greenfield packages (zero consumers; they forked this exact logic). * MCP: adopt the shared placements auth model McpAuthMethod becomes none | apikey(placements) | oauth2 — one method can mix header and query placements, each rendered from its own credential input (or sharing one). buildConnectorInput renders via the shared renderer over credential.values; invokeTool fails explicitly with connection_value_missing instead of dialing unauthenticated; discovery keeps its tolerant skip. The addServer 'auth' shorthand stays as input sugar. Pre-canonical stored shapes (singular auth, retired header/query kinds, openapi-shaped v1-migration templates) are rewritten by the new one-off migrate-config module — runtime decodes only the canonical shape. Slugs and variable names are preserved verbatim (connection.template / item_ids contracts). The react codec collapses onto the new shared-auth-method-codec in packages/react, deleting the first-named-placement truncation; the edit screen diffs placements deeply. Wire tests pin the extreme case: one method = bearer header + team-id query param, asserted on what the server received. * OpenAPI: adopt the shared placements auth model The apiKey arm of Authentication becomes the shared ApiKeyAuthMethod (placements bound to per-input variables) — the variable()-templated header/query records are retired. The oauth arm keeps its core shape (type: "oauth", stored endpoints + scopes). renderAuthTemplate and requiredTemplateVariables now sit on the shared renderer; the catalog projection and react codec collapse onto the shared helpers. Stored legacy templates are rewritten by the one-off migrate-config module; equivalence tests pin that migrated templates render byte-identically to the legacy renderer (multi-variable Datadog-style, literal-only values, prefix extraction). Slugs and variable names are preserved verbatim. * Wire the one-off auth-config migration + unified-auth e2e scenarios The migration planner (and a shared SQLite runner) live in http-auth; each app routes by plugin_id to the plugins' migrate functions: local runs it at boot after the v1->v2 migration (desktop inherits), selfhost at boot beside its defensive column adds, cloud as an out-of-band script (db:migrate-auth:prod) run before deploying canonical-only code. e2e: scenarios move to canonical wire shapes and gain the extreme case — one MCP source declaring [oauth][bearer header + team-id query (two inputs)][bearer header][query token], one connection per method, every invocation made through the real MCP execute surface and asserted against what a recording MCP server received. The selfhost browser suite gains the 2-input connect modal scenario (one field per input, both rendered together on the wire). * GraphQL: adopt the shared placements auth model GraphqlAuthMethod becomes none | apikey(placements) | oauth2(header?, prefix?) — one method can mix header and query placements, each rendered from its own credential input. The local renderer is deleted: apikey renders through the shared renderer, oauth2 through oauthBearerPlacement (custom header/prefix overrides preserved). Introspection, discovery, and invoke run on credential.values; a missing input fails explicitly, naming the variable. Pre-canonical stored shapes (retired {kind:'apiKey', in, name} entries, openapi-shaped v1-migration templates, the singular auth field — incl. the v1 output that wrote BOTH auth:{oauth2} and an apiKey array) are rewritten by the one-off migrate-config module. The react codec collapses onto shared-auth-method-codec, deleting the one-template-per-placement workaround. Wire tests pin the extreme case (bearer header + team-id query on one method) against a recording GraphQL server, on both introspection and invocation. * Sweep remaining consumers onto the canonical auth shapes Examples (docs quickstart + regenerated snippet, promise-sdk, all-plugins), the kernel dynamic-worker integration test, and the local/cloud auth-tool-failure suites move off the retired variable()-templated apiKey shape. Boundary lint suppressions for the migration runner/script; changelog stub for the new http-auth package. * e2e: make the sandbox tool-result assertion shape-agnostic * Changeset + repo notes for the unified auth model * Fold http-auth into @executor-js/sdk as the /http-auth subpath Carrier-agnosticism is a composition property, not a location one: core never imports the module, so it doesn't need its own package. The vocabulary ships as @executor-js/sdk/http-auth (browser-safe subpath, own tsup entry); the standalone package, its workspace entry, deps, and fixed-group/changeset references are gone. * Accept request-shaped auth authoring on every plugin Auth inputs take two dialects, normalized to the one stored model: the canonical placements shape, and the request-shaped template that reads like the request it produces — { type: "apiKey", headers: { Authorization: ["Bearer ", variable("token")] }, queryParams: { team_id: [variable("team_id")] } } variable() is exported from each plugin. Authoring is strict where the migration is tolerant: a template value renders at most ONE variable, as the FINAL part — exotic interleavings are rejected at decode with a pointed message instead of silently flattened (only the one-off migration flattens, for data that already exists). The dialect is accepted by addServer/addIntegration/addSpec/configure on all three plugins (typed clients included — the docs quickstart and the auth-tool-failure suites author this way again), expanded before merge so stored configs stay canonical. Wire tests pin that an authored template renders identically to its canonical twin. * One apikey authoring dialect: request-shaped in, placements out Auth INPUTS accept exactly one apikey shape — the request-shaped template (headers/queryParams records with variable() slots) — expanded to canonical placements at the boundary. Stored configs and the catalog keep reading as placements; apiKeyAuthTemplateFromMethod serializes them back, so every read-modify-write surface (edit screen replace, custom- method remove, add flows) round-trips through one serializer instead of the wire accepting two dialects. Published schemas now teach a single authoring shape; canonical placement inputs are rejected with a schema error pointing at the dialect. * One discriminator: OpenAPI oauth templates re-key to kind:"oauth2" Every auth method is now kind-keyed — the core OAuthAuthentication shape drops its legacy type:"oauth" spelling, so the Authentication union narrows on .kind alone and the '"kind" in template' guards are gone. The one-off config migration rewrites stored type:"oauth" entries (slug and endpoints verbatim); oauth INPUTS are wire-typed (plain slug) and branded in the normalize pass, letting the HTTP handlers pass payloads through untouched. * Migration script: TLS for direct connections + post-migration decode audit The script connects to PlanetScale directly (no Hyperdrive), which requires TLS. The plan now also audits the SIMULATED final state of every protocol-plugin row — rewritten or untouched — against the new runtime's canonical decoders and refuses to write if any row would read as 'no usable config' after deploy. Verified against prod: 343 rows, 302 rewrite, audit clean.
1 parent 60ab842 commit f485e4a

122 files changed

Lines changed: 4629 additions & 5854 deletions

File tree

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

.changeset/unified-auth-methods.md

Lines changed: 14 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,14 @@
1+
---
2+
"executor": minor
3+
"@executor-js/react": minor
4+
---
5+
6+
**One auth model across OpenAPI, GraphQL, and MCP**
7+
8+
- Every protocol plugin now stores the same placements-based auth methods (the new `@executor-js/sdk/http-auth` vocabulary): an API-key method carries any mix of header and query placements, each rendered from its own credential input — so one source can declare OAuth, a bearer-header-plus-team-id-query method, a plain bearer, and a query token side by side, and one connection can carry several values (e.g. both Datadog keys).
9+
- MCP and GraphQL gain what only OpenAPI could do before: multi-placement methods, query-parameter credentials (servers like ui.sh's `?token=`), and multi-input connections. Rendering, catalog projection, slug normalization, and the React method editor/codec are shared instead of triplicated; the connect modal collects one value per input.
10+
- Invoking with an unresolvable credential input now fails with `connection_value_missing` (naming the missing inputs) instead of silently dialing unauthenticated.
11+
- Stored integration configs are rewritten to the canonical shape by a one-off migration: local and self-host run it automatically at startup; cloud operators run `bun run db:migrate-auth:prod` before deploying. Connection bindings and stored credential values are preserved exactly.
12+
- Authoring: apikey methods are authored in ONE request-shaped dialect on every plugin — it reads like the request it produces: `{ type: "apiKey", headers: { Authorization: ["Bearer ", variable("token")] }, queryParams: { team_id: [variable("team_id")] } }` (`variable()` is exported from each plugin; a plain-string value is a static literal). Inputs normalize to the canonical placements model, which is what stored configs and the catalog read as. Authoring is strict where the renderer is: a value carries at most one variable, as the final part.
13+
- Every method is keyed by `kind` — OpenAPI's oauth templates re-key from the retired `type: "oauth"` spelling to `kind: "oauth2"` (matching MCP/GraphQL); the one-off migration rewrites stored entries.
14+
- Breaking (wire): the retired single-placement inputs (`headerName` on MCP, `in`/`name` on GraphQL), raw canonical-placement inputs, and `type: "oauth"` oauth inputs are rejected. The `mcp.addServer` singular `auth` shorthand still works.

AGENTS.md

Lines changed: 4 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -47,7 +47,10 @@ before using it.
4747
## Package Roles
4848

4949
- `packages/core/sdk`: executor core contracts, plugin wiring, scopes, sources,
50-
secrets, policies, and test fixtures.
50+
secrets, policies, and test fixtures. The `@executor-js/sdk/http-auth`
51+
subpath carries the shared placements-based auth-method vocabulary the HTTP
52+
protocol plugins compose (core itself never imports it — composition, not
53+
location, keeps core carrier-agnostic).
5154
- `packages/core/storage-*`: storage adapters and storage test support.
5255
- `packages/plugins/*`: protocol and provider plugins. Plugin-specific
5356
runtime, React, API, and testing helpers should live with the owning plugin.

apps/cloud/package.json

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -13,6 +13,8 @@
1313
"db:studio": "drizzle-kit studio",
1414
"db:studio:prod": "op run --env-file=.env.production -- bun --bun ../../node_modules/.bun/node_modules/drizzle-kit/bin.cjs studio",
1515
"db:migrate:prod": "op run --env-file=.env.production -- bun --bun ../../node_modules/.bun/node_modules/drizzle-kit/bin.cjs migrate",
16+
"db:migrate-auth:prod": "op run --env-file=.env.production -- bun run scripts/migrate-auth-configs.ts",
17+
"db:migrate-auth:dev": "op run --env-file=.env.op -- bun run scripts/migrate-auth-configs.ts",
1618
"db:migrate:dev": "op run --env-file=.env.op -- bun --bun ../../node_modules/.bun/node_modules/drizzle-kit/bin.cjs migrate",
1719
"build": "node scripts/build.mjs",
1820
"preview": "vite preview",
Lines changed: 101 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,101 @@
1+
/* oxlint-disable executor/no-try-catch-or-throw, executor/no-json-parse -- boundary: out-of-band migration script over a raw postgres connection */
2+
// ---------------------------------------------------------------------------
3+
// One-off data migration: rewrite pre-canonical integration auth configs
4+
// into the shared placements model. Run OUT-OF-BAND against the database
5+
// BEFORE deploying code that only decodes the canonical shape (the same
6+
// rule as schema migrations — Workers never migrate at request time):
7+
//
8+
// bun run db:migrate-auth:prod # op run --env-file=.env.production
9+
// bun run db:migrate-auth:dev # against the local PGlite dev db
10+
//
11+
// Idempotent — canonical rows plan zero updates, so re-running is safe.
12+
// Pass --dry-run to print the plan without writing.
13+
// ---------------------------------------------------------------------------
14+
15+
import postgres from "postgres";
16+
import { Option } from "effect";
17+
import { planAuthConfigMigration, type AuthConfigMigrationRow } from "@executor-js/sdk/http-auth";
18+
import {
19+
decodeGraphqlIntegrationConfigOption,
20+
migrateGraphqlAuthConfig,
21+
} from "@executor-js/plugin-graphql";
22+
import { migrateMcpAuthConfig, parseMcpIntegrationConfig } from "@executor-js/plugin-mcp";
23+
import {
24+
decodeOpenApiIntegrationConfig,
25+
migrateOpenApiAuthConfig,
26+
} from "@executor-js/plugin-openapi";
27+
28+
const transforms = {
29+
mcp: migrateMcpAuthConfig,
30+
graphql: migrateGraphqlAuthConfig,
31+
openapi: migrateOpenApiAuthConfig,
32+
};
33+
34+
// Whether a config decodes under the NEW runtime (the canonical shapes).
35+
// Used by the post-migration audit: every row the new code will read must
36+
// decode, whether the migration rewrote it or left it untouched.
37+
const decodesCanonically: Record<string, (config: unknown) => boolean> = {
38+
mcp: (config) => parseMcpIntegrationConfig(config) !== null,
39+
graphql: (config) => Option.isSome(decodeGraphqlIntegrationConfigOption(config)),
40+
openapi: (config) => decodeOpenApiIntegrationConfig(config) !== null,
41+
};
42+
43+
const connectionString = process.env.DATABASE_URL;
44+
if (!connectionString) {
45+
console.error("DATABASE_URL is not set");
46+
process.exit(1);
47+
}
48+
const dryRun = process.argv.includes("--dry-run");
49+
50+
// Direct (non-Hyperdrive) connection — PlanetScale requires TLS.
51+
const sql = postgres(connectionString, { max: 1, prepare: false, ssl: "require" });
52+
53+
try {
54+
const rows = await sql<{ row_id: string; plugin_id: string; config: unknown }[]>`
55+
SELECT row_id, plugin_id, config FROM integration
56+
`;
57+
58+
const inputs: AuthConfigMigrationRow[] = rows.map((row) => ({
59+
rowId: row.row_id,
60+
pluginId: row.plugin_id,
61+
// postgres-js parses json columns; tolerate text just in case.
62+
config: typeof row.config === "string" ? JSON.parse(row.config) : row.config,
63+
}));
64+
65+
const updates = planAuthConfigMigration(inputs, transforms);
66+
console.log(`${rows.length} integration row(s), ${updates.length} to rewrite`);
67+
68+
// Post-migration audit: simulate the final state of every row this plugin
69+
// set owns and check it decodes under the new runtime. A row that neither
70+
// rewrites nor decodes would read as "no usable config" after deploy —
71+
// surface those BEFORE writing anything.
72+
const planned = new Map(updates.map((update) => [update.rowId, update.config]));
73+
const undecodable = inputs.filter((row) => {
74+
const decodes = decodesCanonically[row.pluginId];
75+
if (!decodes) return false; // not a protocol-plugin row (no auth config)
76+
return !decodes(planned.get(row.rowId) ?? row.config);
77+
});
78+
if (undecodable.length > 0) {
79+
console.error(`${undecodable.length} row(s) would NOT decode after migration:`);
80+
for (const row of undecodable) console.error(` ${row.pluginId} ${row.rowId}`);
81+
process.exit(1);
82+
}
83+
console.log("audit: every row decodes canonically after migration");
84+
85+
if (dryRun) {
86+
for (const update of updates) console.log(` would rewrite ${update.rowId}`);
87+
} else if (updates.length > 0) {
88+
await sql.begin(async (tx) => {
89+
for (const update of updates) {
90+
await tx`
91+
UPDATE integration
92+
SET config = ${tx.json(update.config as never)}
93+
WHERE row_id = ${update.rowId}
94+
`;
95+
}
96+
});
97+
console.log(`rewrote ${updates.length} row(s)`);
98+
}
99+
} finally {
100+
await sql.end();
101+
}

apps/host-selfhost/src/app.ts

Lines changed: 8 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -4,7 +4,10 @@ import { Layer } from "effect";
44

55
import { composePluginApi, ExecutorApp, textFailureStrategy } from "@executor-js/api/server";
66

7+
import { runSqliteAuthConfigMigration } from "@executor-js/sdk/http-auth";
8+
79
import { resolveAuthProviders } from "./auth";
10+
import { authConfigTransforms } from "./db/auth-config-migration";
811
import { makeSelfHostAdminApiLayer } from "./admin/handlers";
912
import { makeSelfHostSystemApiLayer } from "./system/handlers";
1013
import { selfHostAccountMiddleware } from "./account";
@@ -54,6 +57,11 @@ export const makeSelfHostApp = async (options: MakeSelfHostAppOptions = {}) => {
5457
version: SELF_HOST_SCHEMA_VERSION,
5558
});
5659

60+
// One-off data migration: rewrite pre-canonical integration auth configs
61+
// into the shared placements model. Idempotent — a no-op once every row is
62+
// canonical (same defensive-at-boot pattern as the column adds above).
63+
await runSqliteAuthConfigMigration(dbHandle.client, authConfigTransforms);
64+
5765
// ---- auth providers ---------------------------------------------------
5866
// Better Auth: cookie/bearer/api-key identity + /api/auth handler + account
5967
// API + MCP OAuth seam, all over the shared libSQL handle.
Lines changed: 17 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,17 @@
1+
// ---------------------------------------------------------------------------
2+
// The per-plugin auth-config transforms the one-off boot migration runs over
3+
// the `integration` table (see `runSqliteAuthConfigMigration`). Each plugin
4+
// owns the rewrite of its own legacy config shapes into the canonical
5+
// placements model; this map only routes by `plugin_id`.
6+
// ---------------------------------------------------------------------------
7+
8+
import type { AuthConfigTransform } from "@executor-js/sdk/http-auth";
9+
import { migrateGraphqlAuthConfig } from "@executor-js/plugin-graphql";
10+
import { migrateMcpAuthConfig } from "@executor-js/plugin-mcp";
11+
import { migrateOpenApiAuthConfig } from "@executor-js/plugin-openapi";
12+
13+
export const authConfigTransforms: Record<string, AuthConfigTransform> = {
14+
mcp: migrateMcpAuthConfig,
15+
graphql: migrateGraphqlAuthConfig,
16+
openapi: migrateOpenApiAuthConfig,
17+
};
Lines changed: 17 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,17 @@
1+
// ---------------------------------------------------------------------------
2+
// The per-plugin auth-config transforms the one-off boot migration runs over
3+
// the `integration` table (see `runSqliteAuthConfigMigration`). Each plugin
4+
// owns the rewrite of its own legacy config shapes into the canonical
5+
// placements model; this map only routes by `plugin_id`.
6+
// ---------------------------------------------------------------------------
7+
8+
import type { AuthConfigTransform } from "@executor-js/sdk/http-auth";
9+
import { migrateGraphqlAuthConfig } from "@executor-js/plugin-graphql";
10+
import { migrateMcpAuthConfig } from "@executor-js/plugin-mcp";
11+
import { migrateOpenApiAuthConfig } from "@executor-js/plugin-openapi";
12+
13+
export const authConfigTransforms: Record<string, AuthConfigTransform> = {
14+
mcp: migrateMcpAuthConfig,
15+
graphql: migrateGraphqlAuthConfig,
16+
openapi: migrateOpenApiAuthConfig,
17+
};

apps/local/src/executor.ts

Lines changed: 11 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -5,10 +5,12 @@ import { basename, join } from "node:path";
55
import { createHash } from "node:crypto";
66

77
import { Subject, Tenant, createExecutor, type AnyPlugin, type Executor } from "@executor-js/sdk";
8+
import { runSqliteAuthConfigMigration } from "@executor-js/sdk/http-auth";
89
import { collectTables } from "@executor-js/api/server";
910
import { loadPluginsFromJsonc } from "@executor-js/config";
1011

1112
import executorConfig from "../executor.config";
13+
import { authConfigTransforms } from "./db/auth-config-migration";
1214
import { createSqliteFumaDb } from "./db/sqlite-fumadb";
1315
import { migrateLocalV1ToV2IfNeeded } from "./db/v1-v2-migration";
1416

@@ -167,6 +169,15 @@ const createLocalExecutorLayer = () => {
167169
(db) => Effect.promise(() => db.close()).pipe(Effect.ignore),
168170
);
169171

172+
// One-off data migration: rewrite pre-canonical integration auth
173+
// configs (incl. v1→v2 outputs) into the shared placements model.
174+
// Idempotent — a no-op once every row is canonical.
175+
yield* Effect.tryPromise({
176+
try: () => runSqliteAuthConfigMigration(sqlite.client, authConfigTransforms),
177+
catch: (cause) =>
178+
new LocalExecutorCreateError({ message: CREATE_SQLITE_ERROR_MESSAGE, cause }),
179+
});
180+
170181
// webBaseUrl is where the executor's web UI listens — same port as the
171182
// daemon API since the daemon serves both. Mirrors serve.ts's port
172183
// resolution so a custom $PORT flows through. EXECUTOR_WEB_BASE_URL

bun.lock

Lines changed: 0 additions & 55 deletions
Some generated files are not rendered by default. Learn more about customizing how changed files appear on GitHub.

docs/snippets/sdk/quickstart/add-integration.mdx

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -4,7 +4,7 @@ Edit examples/docs-sdk-quickstart/src/main.ts between the docs:start/docs:end ma
44
_/}
55

66
```ts
7-
// An integration is the API surface. The apiKey template declares where a
7+
// An integration is the API surface. The auth template declares where a
88
// connection's credential is placed on each request — here, an `X-API-Key`
99
// header. `variable("token")` is the slot the resolved credential renders into.
1010
await executor.openapi.addSpec({

0 commit comments

Comments
 (0)