Skip to content

Commit e324ee0

Browse files
os-zhuangclaude
andcommitted
docs(audit): Dataset/Action/Flow/Agent property liveness
Cluster of security-relevant parsed-but-UNENFORCED props: agent permissions/visibility/access (who-can-chat is a no-op), flow runAs (never switches identity), action disabled-CEL (silently ignored). Plus: flow FlowNodeAction enum out of sync with executors; http vs http_request drift; agent model.provider dead (provider comes from adapter); agent autonomy surface (memory/guardrails/structuredOutput/lifecycle) aspirational except planning.maxIterations; dataset description/certified dead + Studio under-covers live props; action type:form/shortcut/bulkEnabled half-built. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
1 parent aa5e2e3 commit e324ee0

4 files changed

Lines changed: 87 additions & 0 deletions
Lines changed: 22 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,22 @@
1+
# Audit: ActionSchema property liveness & necessity
2+
3+
**Date**: 2026-06-15 · **Scope**: `packages/spec/src/ui/action.zod.ts`. **Consumers**: objectui action renderers (`components/.../action/*`, `RowActionMenu`, `record-quick-actions`), runtime dispatcher (`useConsoleActionRuntime`, `ActionRunner`), server (`runtime/http-dispatcher``engine.executeAction``body-runner`), AI bridge (`service-ai/tools/action-tools.ts`).
4+
5+
## 🔴 `disabled``enabled` naming drift (CEL form silently ignored)
6+
Spec's canonical field is `disabled` (bool | CEL), but the primary renderers (`action-button.tsx:56,116`, action-menu, action-group) read a **non-spec `schema.enabled`** and invert it. Only detail/quick-action toolbars read spec `disabled`, and only its **boolean** form. **The CEL-predicate form of `disabled` has zero consumers** — authoring `disabled: "<expr>"` is silently ignored.
7+
8+
## 🔴 Aspirational / half-built
9+
- **`type:'form'`** — in the enum, marks `target` required, documents a `/console/forms/:name` route, but **no renderer or runtime consumes it** (AI bridge classifies it unsupported). Dead action type.
10+
- **`shortcut`**`ActionEngine` registers it + exposes `handleShortcut`, but **no keydown listener** pumps events → never fires.
11+
- **`bulkEnabled`** — engine has `getBulkActions`/`executeBulk`, but no spec-driven view path calls `executeBulk`.
12+
- **`timeout`** (action-level) — DEAD; server uses `body.timeoutMs`, no UI consumer.
13+
- **`mode`** — consumed only by the AI HITL heuristic, never by UI. **`aria`** — honored by a few renderers but not the core action buttons/menus.
14+
15+
## LIVE & well-wired
16+
`name`, `label`, `objectName`, `icon`, `type`, `target` (+`${param}`/`${ctx}` interpolation), `body` (server script only), `params` (+ field/objectOverride/defaultFromRow/options/placeholder/helpText/defaultValue/required/name), `variant`, `component`, `locations`, `confirmText`, `successMessage`, `refreshAfter`, `resultDialog` (one-shot reveal), `visible` (CEL, fail-closed), `recordIdParam`/`recordIdField`, `bodyShape{wrap}`, `bodyExtra`, `method`, `opensInNewTab`/`newTabUrl`, the full `ai.*` bridge. **Action types**: api/script/flow fully wired; url thinner; **modal PARTIAL** (console maps modal→serverActionHandler, not a real modal opener); form DEAD.
17+
18+
## 🟠 Two parallel prop-readers with different coverage
19+
The **AI bridge** (`action-tools.ts`) reads almost every advanced prop; the **visual button** (`action-button.tsx`) forwards only a fixed subset and relies on the grid path (`ObjectGrid.tsx:1333 ...rest`) to carry the full def into handlers. Divergent coverage = maintenance risk.
20+
21+
## Recommendation
22+
Fix `disabled``enabled` and wire the CEL form (correctness + matches `visible`). Decide `type:'form'`/`shortcut`/`bulkEnabled`/`timeout`/`mode`/`aria` — wire or remove. Make the modal action a real modal opener or drop it. Unify the two prop-readers.
Lines changed: 21 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,21 @@
1+
# Audit: AgentSchema property liveness & necessity
2+
3+
**Date**: 2026-06-15 · **Scope**: `packages/spec/src/ai/agent.zod.ts`. **Consumers**: framework `service-ai/agent-runtime.ts` (+ agent/assistant routes, eval-runner); objectui `AgentPreview` (display) + chatbot (picker).
4+
5+
## 🔴 Model provider drift — `model.provider` is DEAD
6+
Runtime applies only `model.{model,temperature,maxTokens}` (`agent-runtime.ts:264-266`). **`model.provider` and `model.topP` are never applied** — the provider/model comes from the configured AI adapter (`plugin.ts`), so an agent setting `provider:'anthropic'` has **zero effect**. (`AgentPreview.tsx:124` displays provider, reinforcing the false impression.)
7+
8+
## 🔴 The entire "autonomy" surface is aspirational except one knob
9+
**DEAD** (no runtime reader; only authored + displayed in AgentPreview): `memory.*` (shortTerm/longTerm/reflectionInterval), `guardrails.*` (maxTokensPerInvocation/maxExecutionTimeSec/blockedTopics — real limits enforced by an unrelated quota service), `structuredOutput.*`, `lifecycle` (StateMachine), `planning.strategy`, `planning.allowReplan`. **The only LIVE planning sub-field is `planning.maxIterations`** (3 call sites).
10+
11+
## 🔴 Access control declared but UNENFORCED (security gap)
12+
`access`, `visibility`, `tenantId` — none read at runtime. `permissions` is display-only; the chat route **hardcodes `['ai:chat','ai:agents']`** (`agent-routes.ts:109`) regardless of the agent's declared `permissions`. So **"who can chat with this agent" is currently a no-op** — a latent access-control gap. (`visibility` defaults `organization` but gates nothing.)
13+
14+
## Drift
15+
**`knowledge`** — spec defines `{topics,indexes}` but the only consumer (`AgentPreview.tsx:213`) reads `knowledge.sources`/`indexes`, and **no runtime reads it at all** — RAG is wired via `service-knowledge`, not `agent.knowledge`. Chatbot picker `description` is derived from `role` (not an AgentSchema field).
16+
17+
## LIVE & necessary
18+
`name`, `label`, `role`, `avatar`, `instructions` (core), `model.{model,temperature,maxTokens}`, `skills` (→ tools+prompt), `tools` (legacy fallback), `active` (gates listing + 403 on chat), `planning.maxIterations`, `protection` (loader).
19+
20+
## Recommendation
21+
Either route `model.provider`/`topP` into the LLM call or remove them (currently misleading). **Enforce `permissions`/`visibility`/`access`** at the chat route, or stop accepting them (security). Prune the aspirational autonomy surface (memory/guardrails/structuredOutput/lifecycle/planning.{strategy,allowReplan}) or mark `experimental`. Fix the `knowledge` shape drift.
Lines changed: 20 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,20 @@
1+
# Audit: DatasetSchema property liveness & necessity
2+
3+
**Date**: 2026-06-15 · **Scope**: `packages/spec/src/ui/dataset.zod.ts`. **Consumers**: framework `service-analytics` (`dataset-compiler`/`dataset-executor`/`analytics-service`), objectui `DatasetWidget`/`DatasetReportRenderer`/`DatasetDefaultInspector`.
4+
5+
## LIVE & well-wired (the analytics query path)
6+
Dataset: `name`, `label`, `object` (FROM), `include` (join allowlist), `filter`, `dimensions[]`, `measures[]`. Dimension: `name`, `field`, `type`, `dateGranularity`. Measure: `name`, `aggregate`, `field`, `filter`, `format`, `label`, `derived{op,of}`. Evidence: `dataset-compiler.ts:137-191`, `dataset-executor.ts:184-278`, `analytics-service.ts:464-472`. Report/widget renderers bind by **name only** + a `{dimensions,measures}` selection → all sub-prop resolution happens server-side in the compiler/executor.
7+
8+
## DEAD
9+
- **`description`** (dataset) — only a Studio form field; no runtime reader.
10+
- **`certified`** (measure) — has a Studio checkbox (`DatasetDefaultInspector.tsx:160`) but **no runtime gate** (aspirational ADR-0021 governance checkpoint with zero enforcement).
11+
12+
## PARTIAL / drift
13+
- **Dimension `label`** is compiled but surfaces only via `getMeta` discovery titles; charts render the raw server-resolved dimension *value*, never the declared `label`. (Measure `label` fully flows to the renderer.)
14+
- **Dual vocabulary**: spec says `measures`/`dimensions`; presentations select via `values` (measures) and `rows`/`columns` (dimensions); the compile renames `DatasetMeasure→Metric`, `aggregate→type`. Two parallel naming layers (ADR-0021 Phase-1).
15+
16+
## 🟠 Studio under-covers the live schema (authoring gap)
17+
`DatasetDefaultInspector` edits only name/label/description/object/include + dim(name/field/type) + measure(name/aggregate/field/certified). It exposes **no editor** for the LIVE, behavior-changing props: dataset `filter`, measure `filter`, measure `format`, measure `derived`, dimension `dateGranularity`, dim/measure `label`. These must be hand-authored in `.dataset.ts`.
18+
19+
## Recommendation
20+
Remove `description`/`certified` or wire them. Surface the live-but-uneditable props in the Studio dataset designer. Reconcile the measures/dimensions ↔ values/rows/columns vocabulary in one documented place.
Lines changed: 24 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,24 @@
1+
# Audit: FlowSchema property liveness & necessity
2+
3+
**Date**: 2026-06-15 · **Scope**: `packages/spec/src/automation/flow.zod.ts`. **Consumers**: framework `service-automation` engine + node executors; objectui Studio flow designer (canvas/inspector/palette). Per ADR-0018 `node.type` is an open string validated against the live registry, not the enum.
4+
5+
## 🔴 The `FlowNodeAction` enum is significantly out of sync with reality
6+
- **Lists (but DEAD — no executor)**: `parallel_gateway`, `join_gateway`, `boundary_event` (BPMN-interop, "import/export-only"). `boundary_event` is still editable in the inspector (`flow-node-config.ts:479`) and drives the otherwise-dead node prop `boundaryConfig`.
7+
- **Omits (but LIVE — real executors)**: `loop`, `parallel`, `try_catch`, `map`, `approval` (plugin-contributed). The enum is now misleading documentation.
8+
9+
## 🔴 http vs http_request drift
10+
Engine canonical type is `http` (`HTTP_TYPE`); `http_request` is a registered **deprecated alias** (`engine.ts:486`). The Studio palette/config/type-picker author **`http_request`** and never offer `http` → new Studio flows bake in the deprecated alias.
11+
12+
## 🟠 Execution-config props that are display-only at runtime (incl. security-relevant)
13+
- **`runAs`** — engine **never switches execution identity** on it (`FlowPreview` shows it; execution always runs as-is). Security-relevant: a flow declaring `runAs: system` does not actually elevate/de-elevate.
14+
- **`status` / `active`** — engine gates on its in-memory `flowEnabled` map (`toggleFlow`), **not** on `status`/`active`. `active` is spec-flagged Deprecated and redundant with `status`.
15+
- **`errorHandling.fallbackNodeId`** — DEAD (engine uses per-node fault edges). Node **`outputSchema`** — DEAD (declared, never validated). `flow.template`, `flow.description` — no reader either layer.
16+
17+
## LIVE & well-wired
18+
Top-level: `name`, `label`, `version`, `variables[]{name,isInput,isOutput}`, `nodes[]`, `edges[]{source,target,condition(CEL),label}`, `errorHandling.{strategy,maxRetries,retryDelayMs,backoffMultiplier,...}`. Node common: `id`, `type`, `label`, `config`, `connectorConfig`, `timeoutMs`, `inputSchema` (runtime-validated), `waitEventConfig`. **Executors (LIVE)**: start/end/decision/assignment/get|create|update|delete_record/script/screen/http(+alias)/notify/connector_action/wait/subflow/map/loop/parallel/try_catch/approval.
19+
20+
## 🟠 `notify` invisible in the static designer
21+
Full executor + descriptor (`paradigms:['flow']`) but reaches the palette only via the server-driven `/automation/actions` overlay — absent from the hardcoded fallback palette + `flow-node-config.ts`. Against an older/offline backend it can't be authored and renders only generic JSON config.
22+
23+
## Recommendation
24+
Resync `FlowNodeAction` enum with the live registry (add loop/parallel/try_catch/map/approval; remove or mark import-only the 3 gateways). Make the Studio palette author `http` (canonical). **Enforce `runAs`** (or remove — a non-enforcing identity switch is a security footgun). Collapse `status`/`active`. Prune `fallbackNodeId`/`outputSchema`/`template`. Add `notify` to the static palette.

0 commit comments

Comments
 (0)