feat(appkit): supervisor API adapter for agents plugin - #400
Conversation
e86255e to
abe6d76
Compare
pkosiec
left a comment
There was a problem hiding this comment.
Thank you @hubertzub-db for your contribution! I'd like to merge that shortly after #345
to avoid breaking changes in the API. Could you please take a look at some final comments? Thanks!
PR #400 (agent/v2/sa/1-adapter-fixes) — Fixes
Inherited from PR #345 (fix on #345, picked up via rebase)
All 9 issues from PR #345 apply identically — the code is carried over. Fix on #345 first.
| # | File:Line | Issue | Fix |
|---|---|---|---|
| 1 | supervisor-api.ts:356 |
readSseEvents throws on oversized blocks but for await loop has no try/catch — unhandled rejection tears down the request |
Wrap the for await loop in try/catch. On signal?.aborted return silently, otherwise yield emitError("transport", err). Add test. |
| 2 | supervisor-api.ts:397 |
Empty stream (zero SSE events) returns without terminal status — consumer stuck in "running" | When eventCounts.size === 0, yield emitError("transport", "stream closed without events") before returning. Add test. |
| 3 | supervisor-api.ts:428 |
incomplete_details != null treated as fatal even for benign max_output_tokens truncation |
Only treat as fatal when co-occurring with status === "failed" or error != null. Yield recovered text + complete when it's just truncation. Add test. |
| 4 | supervisor-api.ts:33 |
emitError uses %O at warn level, dumping full objects to logs (CWE-532) |
Replace logger.warn("... detail=%O", ...) with logger.warn("... detail=%s", code, summariseErrorPayload(detail)). Add logger.debug for full dump. |
| 5 | supervisor-api.ts:371 |
parsed.type as string — missing field produces "undefined" string |
Replace with typeof parsed.type === "string" ? parsed.type : "" |
| 6 | supervisor-api.ts:544 |
(data.delta as string) ?? "" — fragile cast |
Replace with typeof data.delta === "string" ? data.delta : "" |
| 7 | supervisor-api.ts:542 |
`data.item_id as string | undefined` — non-strings pass through |
| 8 | sse-reader.ts:30 |
maxLineChars name says "line" but guards a "block" |
Rename to maxBlockChars (not yet public, safe to rename) |
| 9 | supervisor-api.ts:48 |
summariseErrorPayload — British spelling, codebase uses American |
Rename to summarizeErrorPayload |
PR #400-specific fixes
P1 — Must fix
| # | File:Line | Issue | Fix |
|---|---|---|---|
| 10 | supervisor-api.ts:650 |
fromSupervisorApi returns Promise<SupervisorApiAdapter> not Promise<AgentAdapter> — binds consumers to concrete class |
Narrow return type to Promise<AgentAdapter> |
| 11 | supervisor-api.ts:315 |
No timeout on streamBody or SSE stream — hangs indefinitely if upstream stalls |
Verify TimeoutInterceptor wraps run(). If not, compose AbortSignal.timeout(ms) with context signal. |
P2 — Should fix
| # | File:Line | Issue | Fix |
|---|---|---|---|
| 12 | beta.ts:23 |
SupervisorApiAdapterCtorOptions exported — internal type leaks as stable contract |
Remove from beta.ts exports |
| 13 | supervisor-api.ts:85 |
WorkspaceClientLike duplicated across adapters, not exported but used in exported type |
Export it, or use ApiClientLike & { config: ... } |
| 14 | supervisor-api.ts:654 |
Double-cast as unknown as WorkspaceClientLike signals structural mismatch |
Verify SDK's WorkspaceClient satisfies WorkspaceClientLike structurally; widen the type or narrow the cast |
| 15 | supervisor-api.ts:75 |
StreamBody type duplicated in both adapters |
Extract to connectors/serving/client.ts |
P3 — Nice to have
| # | File:Line | Issue | Fix |
|---|---|---|---|
| 16 | supervisor-api.ts:101 |
SupervisorTool closed union — new upstream tool type = breaking change |
Consider ` |
Gap — Mario's Section 6
- Frontmatter-referenced supervisor tools: The mechanism works (tool-index routing composes via
agents({ tools: { nyc: supervisorTools.genieSpace(...) } })+ frontmattertools: [nyc]), but no doc example or test demonstrates this path. Add a doc example and test.
Tests to add
fromSupervisorApidynamic-import fallback (noworkspaceClientsupplied)- Capability-mismatch warning paths (partially present already)
- Frontmatter-referenced supervisor tool resolution
Mario's Review Status
abe6d76 to
2ccfc54
Compare
|
This pull request has had no activity for 23 days and has been marked as stale. It will be closed in 7 days if there is no further activity. Add a comment, push a commit, or apply the |
Apply Mario's defensive/correctness fixes from the supervisor API adapter
review without touching the public API shape (sections 1-8 will land in a
stacked branch). Highlights:
High
- Route the three SSE error-leak sites in supervisor-api.ts (streamBody
catch, mapEvent "error", output_item.done with id="error") through a
single emitError helper that returns a stable client-facing code
(`Supervisor API error (transport|upstream_failed|upstream_tool|
upstream_unknown)`) and logs the verbose detail server-side only.
Addresses CWE-209 verbatim-upstream-error-text leak.
Medium
- Gate the terminal {status:"complete"} emission on lastCompleted.status
/ .error / .incomplete_details so a `response.completed` with a nested
failed status no longer silently succeeds; surface as upstream_failed
instead. Regression tests added.
- Skip the terminal error in the streamBody catch when signal.aborted —
consumer-initiated aborts now end with a clean stop, not a contradictory
terminal error event. Regression test added.
- Tighten the output_item.done error match: require item.type === "error"
(or pair the reserved id="error" with a non-message type) so a stray
assistant message with id="error" is not mis-classified.
- Add maxLineChars / maxBufferChars caps to readSseEvents with 1 MiB / 8
MiB defaults; throw on overflow. Addresses CWE-770. Tests added.
- Docs: add a CWE-1427 callout warning that hosted-tool `description` is a
prompt-injection sink — do not derive it from untrusted input.
- Redact the no-delta warning log: summariseErrorPayload extracts a short
`type: message` line; full payload only via DEBUG. Addresses CWE-532.
- Gate the buffer-level CRLF normalize in sse-reader on `\r` presence to
skip the regex on LF-only steady state.
Low
- mapEvent("error") fallback no longer wraps "Unknown error" with literal
JSON quotes (uses string branch).
- Drop the misleading "we await the factory at module init" comment in
dev-playground; the code never awaits.
- Fix @example imports in supervisor-api.ts JSDoc to use
@databricks/appkit/beta (the actual public re-export).
- Replace trimStart() with single-U+0020 strip in sse-reader per the SSE
spec; remove the now-dead per-line `\r$` strip after the buffer-level
CRLF normalise.
- Flag streamPath as @internal in connectors/serving/client.ts noting the
CWE-918 SSRF risk if it ever leaks to user-controlled input.
- Add JSDoc warning to workspaceClient on SupervisorApiAdapterOptions:
passing a per-request OBO client would leak identity across requests
(CWE-664).
Signed-off-by: Hubert Zub <hubert.zub@databricks.com>
Signed-off-by: Hubert Zub <hubert.zub@databricks.com>
…s#345 review Address structural feedback from Mario Cadenas' review of PR databricks#345 (sections 1-8). Stacked on top of the §9 defensive fixes commit. API changes (BETA surface only): - Add `DatabricksAdapter.fromSupervisorApi` static factory for discoverability alongside `.fromChatCompletions`. - Shrink `SupervisorApiAdapterOptions` to `{ model, workspaceClient? }`; tools no longer live on the adapter. - Hosted tools (`supervisorTools.*`) now return tagged `HostedSupervisorTool` records and accept named options instead of positional args. - Declare hosted tools on the agent's `tools` map (same place as function tools / sub-agents); the agents plugin and `runAgent` route them to the adapter via the new `AgentInput.extensions[SUPERVISOR_EXTENSION_KEY]`. - Add capability-negotiation fields to `AgentAdapter`: `acceptsExtensions?` + `consumesInputTools?`. The agents plugin and `runAgent` warn at registration when adapter capabilities don't match declared tools. Internals: - Extend `ResolvedToolEntry` / `StandaloneEntry` with a `hosted-supervisor` branch; `classifyTool` matches it before MCP hosted-tool rejection so standalone `runAgent` supports supervisor tools. - Defense-in-depth: both indexers throw if a `hosted-supervisor` entry is ever dispatched as a callable function. - `DatabricksAdapter.fromSupervisorApi` uses a dynamic import to avoid load-time cycles. Docs: - Rewrite supervisor-API section in docs/plugins/agents.md for the new shape. - Add cross-adapter sub-agent composition note (one-directional: chat parents can call supervisor children, not vice-versa, until SA's function-call events are routed back through `context.executeTool`). Playground: - Update dev-playground supervisor agent to the new shape (`DatabricksAdapter.fromSupervisorApi(...)` + tools on `createAgent`). Tests: - Rewrite supervisor-api.test.ts factory + adapter tests for the new shape. - Add `isSupervisorTool` and `DatabricksAdapter.fromSupervisorApi` tests. - New regression tests in `run-agent.test.ts` covering the hosted-supervisor extension-routing path and both capability-mismatch warnings. - New agents-plugin tests covering the same warning paths and the new `hosted-supervisor` tool-index branch. Signed-off-by: Hubert Zub <hubert.zub@databricks.com>
Signed-off-by: Hubert Zub <hubert.zub@databricks.com>
2ccfc54 to
93ee540
Compare
pkosiec
left a comment
There was a problem hiding this comment.
Tested and it works well!
We need to merge either #400 PR alone, or #345 + #400 very shortly one after another, as #400 has breaking changes in the API and we want to avoid an AppKit release between them.
Please see my last minor comments and we are good to go 👍 Thanks again for your patience and contribution!
| // nyc: supervisorTools.genieSpace({ | ||
| // id: "01ABCDEF12345678", | ||
| // description: "NYC taxi trip records and zones", | ||
| // }), | ||
| // add: supervisorTools.ucFunction({ | ||
| // name: "main.default.add", | ||
| // description: "Adds two integers and returns the sum.", | ||
| // }), |
There was a problem hiding this comment.
Can we uncomment this, add the impert, set the Genie space ID as env in dev-playground app (.env.dist, still pointing to the ID you provided here), and ensure we always have a supervisor example in dev playground?
For now, from what I can see it, it's API only example, but we could add the agent: "supervisor" as an option in agent.route.tsx (it currently hardcodes the agent name at lines 78/209). Small edit - add a selector to make it easier to test it 👍
Signed-off-by: Hubert Zub <hubert.zub@databricks.com>
Signed-off-by: Hubert Zub <hubert.zub@databricks.com>
pkosiec
left a comment
There was a problem hiding this comment.
🚀 🚀 🚀 Thanks a ton for the pull request!
Co-authored-by: Pawel Kosiec <pawel.kosiec@gmail.com>
|
Repushed the PR to verify CI here: #492 (our CI doesn't work with forks currently) All checks green: https://github.com/databricks/appkit/actions/runs/30355377147/job/90262264986?pr=492 That means the PR is ready to merge 👍 |
Adds a fourth
AgentAdaptertargeting the Databricks AI Gateway Responses API, so AppKit apps can host server-side-orchestrated agents (Genie, Knowledge Assistants, UC functions, apps, UC connections) without running the tool loop locally.Hosted tools are declared on the agent (same place as function tools), not on the adapter — the plugin routes them to the Supervisor API. Try it via the
supervisoragent in the dev-playground Agent Chat dropdown.Address structural feedback from Mario Cadenas' review of PR #345 (sections 1-8). Stacked on top of the §9 defensive fixes commit.
API changes (BETA surface only):
DatabricksAdapter.fromSupervisorApistatic factory for discoverabilityalongside
.fromChatCompletions.SupervisorApiAdapterOptionsto{ model, workspaceClient? }; tools no longer live on the adapter.supervisorTools.*) now return taggedHostedSupervisorToolrecords and accept named options instead of positional args.toolsmap (same place as function tools / sub-agents); the agents plugin andrunAgentroute them to the adapter via the newAgentInput.extensions[SUPERVISOR_EXTENSION_KEY].AgentAdapter:acceptsExtensions?+consumesInputTools?. The agents plugin andrunAgentwarn at registration when adapter capabilities don't match declared tools.Internals:
ResolvedToolEntry/StandaloneEntrywith ahosted-supervisorbranch;classifyToolmatches it before MCP hosted-tool rejection so standalonerunAgentsupports supervisor tools.hosted-supervisorentry is ever dispatched as a callable function.DatabricksAdapter.fromSupervisorApiuses a dynamic import to avoid load-time cycles.Docs:
context.executeTool).Playground:
DatabricksAdapter.fromSupervisorApi(...)+ tools oncreateAgent).Tests:
isSupervisorToolandDatabricksAdapter.fromSupervisorApitests.run-agent.test.tscovering the hosted-supervisor extension-routing path and both capability-mismatch warnings.hosted-supervisortool-index branch.