diff --git a/CODEOWNERS b/CODEOWNERS index 6dc0d3491..a932ce570 100644 --- a/CODEOWNERS +++ b/CODEOWNERS @@ -64,6 +64,10 @@ /skills/uipath-maestro-flow/references/plugins/connector-trigger/ @baishalighosh @chandusailella @bai-uipath @rockymadden /tests/tasks/uipath-maestro-flow/connector_features/ @baishalighosh +# IXP +/skills/uipath-maestro-flow/references/author/references/plugins/ixp/ @jcalero @constantinmuraru @scallaway-uipath @tommilligan @joe-prosser @vlad-crisan @richsilv @AWilcke +/tests/tasks/uipath-maestro-flow/ixp/ @jcalero @constantinmuraru @scallaway-uipath @tommilligan @joe-prosser @vlad-crisan @richsilv @AWilcke + # Review skill /skills/uipath-review/ @AlvinStanescu @gabrielavaduva @roalexandru diff --git a/skills/uipath-maestro-flow/SKILL.md b/skills/uipath-maestro-flow/SKILL.md index db0afce38..28241229d 100644 --- a/skills/uipath-maestro-flow/SKILL.md +++ b/skills/uipath-maestro-flow/SKILL.md @@ -1,6 +1,6 @@ --- name: uipath-maestro-flow -description: "Always invoke for `.flow` files. UiPath Maestro Flow (.flow) — build, edit, run, debug, fix, evaluate. Create, connect nodes; connector, approval, script, subflow; triggers, schedules; validate. Upload, publish, manage runs, instances. Diagnose errors, incidents, traces. Design eval sets, evaluators, run Studio Web evals via `uip maestro flow eval`. `uip maestro flow` CLI. For C#/XAML→uipath-rpa. For agents→uipath-agents." +description: "Always invoke for `.flow` files. UiPath Maestro Flow (.flow) — build, edit, run, debug, fix, evaluate. Create, connect nodes; connector, approval, script, subflow, ixp; triggers, schedules; validate. Upload, publish, manage runs, instances. Diagnose errors, incidents, traces. Design eval sets, evaluators, run Studio Web evals via `uip maestro flow eval`. `uip maestro flow` CLI. For C#/XAML→uipath-rpa. For agents→uipath-agents." allowed-tools: Bash, Read, Write, Edit, Glob, Grep, AskUserQuestion --- diff --git a/skills/uipath-maestro-flow/references/author/CAPABILITY.md b/skills/uipath-maestro-flow/references/author/CAPABILITY.md index b69b7a946..c2221a920 100644 --- a/skills/uipath-maestro-flow/references/author/CAPABILITY.md +++ b/skills/uipath-maestro-flow/references/author/CAPABILITY.md @@ -63,6 +63,8 @@ Capability index for building new flows (greenfield) and editing existing flows | **Wire one node's output into another node's input** | [shared/node-output-wiring.md](../shared/node-output-wiring.md) | | **Orchestrate RPA, agents, apps** | Relevant resource plugin: [rpa](references/plugins/rpa/), [agent](references/plugins/agent/), [agentic-process](references/plugins/agentic-process/), [flow](references/plugins/flow/), [api-workflow](references/plugins/api-workflow/), [hitl](references/plugins/hitl/) | | **Embed an AI agent tightly coupled to this flow** | [plugins/inline-agent/](references/plugins/inline-agent/) | +| **Extract structured fields from documents** | [plugins/ixp/](references/plugins/ixp/) — IxP extraction models for PDFs, scanned forms, receipts, invoices, contracts | +| **List IxP models / runtime projects available in flow** | [plugins/ixp/impl.md — Listing Published Models](references/plugins/ixp/impl.md#listing-published-models) — read-only registry search, no `.flow` scaffold or edits | | **Create a resource that doesn't exist yet** | Use `core.logic.mock` placeholder — see [Edit/Write: Replace a mock](references/editing-operations-json.md#replace-a-mock-with-a-real-resource-node) + relevant plugin's `impl.md` | | **Add data transform nodes** | [plugins/transform/impl.md](references/plugins/transform/impl.md) | | **Add an LLM batch transform over CSV rows** | [plugins/batch-transform/impl.md](references/plugins/batch-transform/impl.md) — `uipath.pattern.batch-transform`, gated by tenant flag `canvas.nodes.batch-transform` | @@ -131,6 +133,7 @@ Capability index for building new flows (greenfield) and editing existing flows - [hitl](references/plugins/hitl/) — human input via UiPath Apps - [agent](references/plugins/agent/) — published AI agent resources - [inline-agent](references/plugins/inline-agent/) — autonomous agent embedded in flow + - [ixp](references/plugins/ixp/) — published IxP document-extraction models (PDFs, scanned forms, receipts, invoices, contracts) - [queue](references/plugins/queue/) — Orchestrator queue item creation ### Cross-capability (shared) diff --git a/skills/uipath-maestro-flow/references/author/references/planning-arch.md b/skills/uipath-maestro-flow/references/author/references/planning-arch.md index 305eb9575..a55ceb081 100644 --- a/skills/uipath-maestro-flow/references/author/references/planning-arch.md +++ b/skills/uipath-maestro-flow/references/author/references/planning-arch.md @@ -153,6 +153,9 @@ Resource nodes invoke published UiPath automations. They are tenant-specific and | Flow | `uipath.core.flow.{key}` | [flow](plugins/flow/planning.md) | | API Workflow | `uipath.core.api-workflow.{key}` | [api-workflow](plugins/api-workflow/planning.md) | | Human Task (app-based) | `uipath.core.human-task.{key}` | [hitl](plugins/hitl/planning.md) | +| Document Extraction | `uipath.ixp.{modelName}.{fullyQualifiedName}` | [ixp](plugins/ixp/planning.md) | + +> The IxP entry uses a **two-segment tail** (`{modelName}.{fullyQualifiedName}`), unlike the other resource nodes which use a single-segment `{key}` tail. Both segments are sanitized at registry-emit time. See [plugins/ixp/planning.md](plugins/ixp/planning.md) for the sanitization rule. ### Placeholders @@ -202,6 +205,7 @@ Use this when defining edges. Every edge requires a `sourcePort` and `targetPort | `uipath.core.flow.*` | `input` | `output`, `error` | | `uipath.core.agentic-process.*` | `input` | `output`, `error` | | `uipath.core.api-workflow.*` | `input` | `output`, `error` | +| `uipath.ixp.*` | `input` | `success`, `error` | | `uipath.connector.*` (activities) | `input` | `output`, `error` | | `core.action.queue.create` | `input` | `success` | | `core.action.queue.create-and-wait` | `input` | `success` | @@ -514,6 +518,13 @@ Quick decision guide. For full details, read the linked plugin's `planning.md`. - Small ad-hoc reshaping (map/filter/groupBy) without an LLM -> [transform](plugins/transform/planning.md) - Multi-step reasoning with tool use -> [inline-agent](plugins/inline-agent/planning.md) or [agent](plugins/agent/planning.md) +### "I need to extract structured fields from documents" + +- Source is a PDF, scanned form, photo, or email attachment with **variable layout** across inputs (invoices from many vendors, receipts, contracts, forms) -> [ixp](plugins/ixp/planning.md) (`uipath.ixp.{modelName}.{fullyQualifiedName}`) +- Source is already structured (CSV, JSON, database row) -> [script](plugins/script/planning.md) or [transform](plugins/transform/planning.md) +- Need free-form summarization, classification, or open-ended reasoning -> [agent](plugins/agent/planning.md) or [inline-agent](plugins/inline-agent/planning.md) +- IxP model not yet trained -> use `core.logic.mock` and surface in Open Questions + ### "The flow needs something outside flow capabilities" 1. Add a `core.logic.mock` placeholder diff --git a/skills/uipath-maestro-flow/references/author/references/planning-impl.md b/skills/uipath-maestro-flow/references/author/references/planning-impl.md index 2d5215910..452f83502 100644 --- a/skills/uipath-maestro-flow/references/author/references/planning-impl.md +++ b/skills/uipath-maestro-flow/references/author/references/planning-impl.md @@ -57,6 +57,7 @@ uip maestro flow registry get --output json | `uipath.core.api-workflow.*` | [api-workflow/impl.md](plugins/api-workflow/impl.md) | | `uipath.core.hitl.*` | [hitl/impl.md](plugins/hitl/impl.md) | | `uipath.connector.uipath-uipath-dataservice.*` | [connector/data-fabric/impl.md](plugins/connector/data-fabric/impl.md) | +| `uipath.ixp.*` | [ixp/impl.md](plugins/ixp/impl.md) | | `uipath.connector.*` | [connector/impl.md](plugins/connector/impl.md) | | `uipath.connector.trigger.*` | [connector-trigger/impl.md](plugins/connector-trigger/impl.md) | @@ -101,6 +102,15 @@ uip maestro flow registry search "" --output json If found in neither, keep the `core.logic.mock` placeholder and note the gap. +#### IxP nodes — context-dispatched, no bindings + +IxP extraction nodes (`uipath.ixp.*`) skip binding resolution. Design-time configuration (`folderKey`, `modelName`) is emitted into the BPMN `model.context[]` array at build time (the serializer also pins `digitizationMode` to `"fileUpload"` internally), not into a separate `bindings_v2.json` file or a top-level `bindings[]` entry. Consequence for Phase 2: + +- No connection ID to bind — the node carries its tenant context inline. +- `inputs.*` is the source of truth for runtime values; validate against `registry get` `inputDefinition.properties` rather than against a binding schema. +- The node instance also carries a structured `inputs.model` blob (extraction-model metadata) that the property panel's `ixp-model-taxonomy` component reads. Copy `inputDefaults.model` verbatim — omitting it crashes the panel with `Cannot destructure property 'modelName' of 't' as it is undefined`. +- See [plugins/ixp/impl.md](plugins/ixp/impl.md) for the full JSON shape. + ### Step 4 — Replace Mock Nodes For each `core.logic.mock` node in the architectural plan: diff --git a/skills/uipath-maestro-flow/references/author/references/plugins/ixp/impl.md b/skills/uipath-maestro-flow/references/author/references/plugins/ixp/impl.md new file mode 100644 index 000000000..6a29c8b50 --- /dev/null +++ b/skills/uipath-maestro-flow/references/author/references/plugins/ixp/impl.md @@ -0,0 +1,326 @@ +# IxP Extraction Node — Implementation + +IxP Extraction nodes invoke a published UiPath Intelligent eXtraction Platform (IxP) model. They are tenant-specific resources with pattern `uipath.ixp.{sanitized-modelName}.{sanitized-fullyQualifiedName}`. + +Sanitization rule (applied to both tail segments, in this order): + +1. Lowercase. +2. Replace runs of any character outside `[a-z0-9]` with a single `-`. Slashes, spaces, underscores, and runs of dashes (e.g. `---`) all collapse to a single `-`. Dots in the FQN are NOT preserved — they also collapse to `-`. + +The dot in `uipath.ixp.{model}.{fqn}` is the segment separator the registry adds *after* sanitizing each tail segment, not part of the sanitization itself. + +Examples (verified against the live registry): + +- `"birth_certificates_oob-6252526a-ixp"` + FQN `Shared/birth_certificates_oob-6252526a-ixp` → `uipath.ixp.birth-certificates-oob-6252526a-ixp.shared-birth-certificates-oob-6252526a-ixp` (underscores and slash both → `-`). +- `"idp-benchmark---invoices-c735405a-ixp"` + FQN `Shared/idp-benchmark---invoices-c735405a-ixp` → `uipath.ixp.idp-benchmark-invoices-c735405a-ixp.shared-idp-benchmark-invoices-c735405a-ixp` (run of `---` → single `-`). + +Always prefer the `nodeType` returned by `uip maestro flow registry search` over constructing one by hand. + +## Discovery + +```bash +uip maestro flow registry pull --force +uip maestro flow registry search "uipath.ixp" --output json +``` + +Requires `uip login`. Only published IxP models from your tenant appear. Example returned `nodeType`: `uipath.ixp.idp-benchmark-invoices-c735405a-ixp.shared-idp-benchmark-invoices-c735405a-ixp` (for an `idp-benchmark---invoices-c735405a-ixp` model in the `Shared` folder). + +### Response shape + +`registry search` returns a top-level envelope; `Data` is a flat list of node entries (PascalCase keys): + +```json +{ + "Result": "Success", + "Code": "NodeSearchSuccess", + "Data": [ + { + "NodeType": "uipath.ixp.idp-benchmark-invoices-c735405a-ixp.shared-idp-benchmark-invoices-c735405a-ixp", + "Category": "document-processing", + "DisplayName": "idp-benchmark---invoices-c735405a-ixp", + "Description": "(Shared)", + "Version": "1.0.0", + "Tags": "ixp, document-understanding, extraction" + } + ] +} +``` + +Read entries as `raw["Data"][i]["NodeType"]` (not `raw["Data"]["Nodes"]`). + +### If `Data` is empty → stop and use a mock + +If `uip maestro flow registry search "uipath.ixp"` returns `Data: []`, **no IxP extraction model is published on this tenant**. Add a `core.logic.mock` placeholder node (see [If the Model Does Not Exist Yet](#if-the-model-does-not-exist-yet)) and surface the missing model in **Open Questions**. + +**Stop searching.** Do not run any of these as a fallback: + +- Domain-keyword searches: `registry search "invoice"`, `"form"`, `"document"`, `"W-9"`, `"receipt"`, `"contract"`, etc. — there is no domain-named extraction node; IxP is the only extraction primitive. +- `registry list` followed by client-side filtering for "ixp" / "extraction" — the strict `uipath.ixp` search is already authoritative. +- Variant-prefix searches: `registry search "uipath.agent.resource.tool.ixp"`, `"core.ixp"`, etc. + +The fallback is `core.logic.mock`, full stop. At most run one broader `registry search "ixp"` to confirm there are no `uipath.ixp.*` hits hidden by stricter prefix matching, then mock. + +> A `uipath.agent.resource.tool.ixp.*` hit on the broader `"ixp"` search is the *agent-tool* variant — not a flow extraction node. Treat it as "no extraction model published" and fall back to mock. + +## Listing Published Models + +When the user asks what IxP models are available — equivalently, "what runtime projects do I have access to in flow", "what document extractors can I access in flow", "list published extractors" — answer with the same registry search. Each `Data[]` entry corresponds to one published model (a.k.a. runtime project) on the tenant. + +```bash +uip login status --output json # confirm auth — without login, tenant IxP nodes are hidden +uip maestro flow registry pull --force +uip maestro flow registry search "uipath.ixp" --output json +``` + +Parse `Data[].DisplayName`, `Data[].NodeType`, and `Data[].Version` and present them as a table. Example: + +| Model (DisplayName) | NodeType | Version | +| --- | --- | --- | +| idp-benchmark---invoices-c735405a-ixp | `uipath.ixp.idp-benchmark-invoices-c735405a-ixp.shared-idp-benchmark-invoices-c735405a-ixp` | `1.0.0` | + +Rules for the listing path: + +- **Do NOT scaffold a solution, run `uip maestro flow init`, or write a `.flow` file.** Listing is read-only Q&A. +- **Do NOT mock.** If `Data: []`, answer directly: no IxP models are published on this tenant. The `core.logic.mock` fallback is for build-time planning, not for listing-time Q&A. +- **Do NOT log in for the user.** If `uip login status` shows logged-out, tell the user to run `uip login` and stop — listing without auth returns OOTB-only results and is misleading. +- **Do NOT search by `"runtime"`, `"document extractor"`, `"extractor"`, or `"IXP"` (uppercase).** These return empty results or agent-tool variants — not extraction nodes. Use `"uipath.ixp"` (lowercase) only. +- **Do NOT use `uip maestro flow process list` or any Orchestrator folder iteration.** `flow process list` enumerates *deployed flow process instances* (with `--folder-key`), not published models. Listing published IxP models always goes through `registry search "uipath.ixp"`. +- **Do NOT guess `uip maestro flow list-*` or `uip maestro ixp list-*` subcommands.** None exist. The CLI returns `unknown command 'list-...'` and there is no fallback path to pursue. + +## Registry Validation + +```bash +uip maestro flow registry get "" --output json +``` + +Confirm: + +- `category` — `document-processing` (older `document-extraction` enum was renamed; current registry serves `document-processing`) +- Input port: `input` +- Output ports: `success` and `error` (the `error` port is gated by `inputs.errorHandlingEnabled`; manifest sets `supportsErrorHandling: true`). Edges target these handle IDs in `.flow` JSON; `handleType` is `output`. +- `model.type` — `bpmn:ServiceTask`. `model.serviceType` — `IXP.Extraction`. The manifest's `model` is two fields only (`type`, `serviceType`) — no `context`, no `version`. Both are injected by the BPMN serializer at compile time. +- `form.id` — `ixp-standalone-form`. Three sections: `ixp-model` (Configuration), `ixp-file-upload` (File input), `schema-definition` (Schema definition — a single custom field `inputs.model` rendered by the `ixp-model-taxonomy` component). +- `inputDefinition.properties` — `model` (object), `modelName`, `projectName`, `projectId`, `versionTag`, `folderKey`, `folderName`, `fileRef`, `pageRange`, `attachmentConfig`, `guardrails`, `attachment`. `inputDefinition.required` — `["fileRef"]`. +- `inputDefaults` — carries the full `model` metadata blob plus flat `modelName` / `projectName` / `folderKey` / `folderName` mirrors. The blob shape is `{ modelName, fullyQualifiedName, description, folderKey, folderName }` plus DU-API extras (`kind`, `type`, `detailsUrl`, `asyncDigitizationUrl`, `asyncExtractionUrl`). +- `outputDefinition` — populated. `output` carries the full extraction-result JSON schema; `error` carries the standard error envelope. + +## Adding / Editing + +For step-by-step add, delete, and wiring procedures, see [editing-operations.md](../../editing-operations.md). Use the JSON structure below for the node-specific `inputs` and `outputs` fields. Author CAPABILITY rule #15 (no top-level `model` block on the instance) and rule #14 (`outputs` populated for nodes that produce data) both apply. + +## JSON Structure + +The IxP node instance carries `inputs` and `outputs` — and **no top-level `model` block**. The slim manifest `model` (`{ type, serviceType }`) lives only in `definitions[]`; the runtime `model.context` / `model.version` / `model.inputs` / `model.outputs` envelope is injected by the BPMN serializer at compile time. + +### Build procedure — copy from `registry get`, do not construct from memory + +The IxP node instance is **derived from the registry response**, not authored from scratch. Any IxP node built from training-data recall will hit at least one of: missing `inputs.model` (canvas crash), missing `outputs.error` (broken `$vars` resolution), legacy forbidden fields (silent schema drift). + +Run this once and source every field below from the response: + +```bash +uip maestro flow registry get "" --output json > .json +``` + +Then assemble the instance by copying these paths verbatim: + +| Instance field | Source path in `registry get` response | Required | +| --- | --- | --- | +| `inputs.model` (full object) | `Data.Node.inputDefaults.model` | **YES** — undefined → canvas crash | +| `inputs.modelName` | `Data.Node.inputDefaults.modelName` | YES | +| `inputs.projectName` | `Data.Node.inputDefaults.projectName` | YES | +| `inputs.folderKey` | `Data.Node.inputDefaults.folderKey` | YES | +| `inputs.folderName` | `Data.Node.inputDefaults.folderName` | YES | +| `inputs.versionTag` | `""` (empty string unless pinning a version) | YES | +| `inputs.pageRange` | `""` (empty string for full document) | YES | +| `inputs.fileRef` | `"=js:$vars..output."` (author this) | YES | +| `outputs.output` (full object) | `Data.Node.outputDefinition.output` | **YES** — missing → `$vars..output` fails | +| `outputs.error` (full object) | `Data.Node.outputDefinition.error` | **YES** — missing → `$vars..error` fails | + +**Forbidden in `inputs`** (legacy schema, removed from current standalone node — including any of these is a defect even if `flow validate` passes): + +- `digitizationMode` — serializer defaults to `fileUpload` internally +- `documentTaxonomy` — replaced by `inputs.model` blob +- `attachmentId` — use `inputs.attachment` for Orchestrator job attachments instead +- `fileName` — derived from `fileRef` upstream +- `mimeType` — derived from `fileRef` upstream + +If you find yourself typing any of those five field names while authoring an IxP node, stop and re-read this section. + +### Final shape + +```json +{ + "id": "extractInvoiceFields", + "type": "uipath.ixp.invoice-model.shared-invoice-model", + "typeVersion": "", + "display": { "label": "Extract Invoice Fields" }, + "inputs": { + "model": { + "modelName": "Invoice Model", + "description": "", + "kind": "Extractor", + "type": "IXP", + "fullyQualifiedName": "Shared/invoice-model", + "detailsUrl": "https://.uipath.com/.../models/invoice-model?api-version=2.0", + "asyncDigitizationUrl": "https://.uipath.com/.../models/invoice-model/digitization/start?api-version=2.0", + "asyncExtractionUrl": "https://.uipath.com/.../models/invoice-model/extraction/start?api-version=2.0", + "folderKey": "", + "folderName": "Shared" + }, + "modelName": "Invoice Model", + "description": "", + "projectName": "Invoice Model", + "versionTag": "", + "folderKey": "", + "folderName": "Shared", + "fileRef": "=js:$vars.start.output.invoice", + "pageRange": "" + }, + "outputs": { + "output": { /* copy verbatim from definition.outputDefinition.output */ }, + "error": { /* copy verbatim from definition.outputDefinition.error */ } + } +} +``` + +### Authoring rules + +1. **`inputs.model` MUST be present and MUST be the full blob from `Data.Node.inputDefaults.model`.** Copy verbatim — do not synthesize, do not abbreviate, do not omit the DU-API extras (`detailsUrl`, `asyncDigitizationUrl`, `asyncExtractionUrl`, `kind`, `type`). The `schema-definition` form section binds `inputs.model` to the `ixp-model-taxonomy` custom component, which destructures `modelName` and `folderKey` out of it. If `inputs.model` is undefined, clicking the node in Studio Web crashes the property panel with `Cannot destructure property 'modelName' of 't' as it is undefined` — and `flow validate` does not catch it. +2. **Flat mirrors stay alongside `inputs.model`.** `modelName`, `projectName`, `folderKey`, `folderName` are surfaced as disabled text fields in the `ixp-model` form section and are read directly from `inputs.*`, not from `inputs.model.*`. +3. **`fileRef` is the only schema-required input** (`inputDefinition.required: ["fileRef"]`). Use `=js:$vars..output.` per Critical Rule #13. +4. **`outputs.output` AND `outputs.error` MUST both be present**, copied verbatim from `Data.Node.outputDefinition.output` and `Data.Node.outputDefinition.error`. Omitting either breaks downstream `$vars..output` / `.error` resolution and hides the field in Studio Web's variable picker. `flow validate` does not catch the omission. +5. **No top-level `model` on the instance.** Studio Web–authored .flow files never carry one; the BPMN-format `model` envelope (with `context`, `version`, `inputs`, `outputs`) is emitted at serialize time only. +6. **`inputs` MUST NOT contain `digitizationMode`, `documentTaxonomy`, `attachmentId`, `fileName`, or `mimeType`.** These five fields were on a prior schema and have been removed from the standalone IxP node. Including them is the most common training-data-recall mistake. The serializer defaults `digitizationMode` to `fileUpload` internally — there is no scenario where you should set it on the instance. + +The `definitions[]` entry is copied verbatim from `registry get` (`Data.Node`). Critical Rule #7 applies unchanged. + +> **`uip maestro flow validate` enforces the Authoring rules above** via the `ixp-node` validator. Failures surface as `severity: "error"` issues with `path` like `nodes[].inputs.model` and a self-contained `message` describing the violation — fix the `.flow` file, not the validator. The registry's `inputDefinition.properties` is the schema of the property catalog, not a license to override the rules: `digitizationMode`, `documentTaxonomy`, `attachmentId`, `fileName`, and `mimeType` are NOT returned by `registry get` and must not be set on the instance. + +### `inputs.fileRef` vs the emitted `model.inputs[]` body + +`inputs.fileRef` is the source of truth. At BPMN serialize time, `packages/services/src/serialization/uipath-extension.ts:handleIxpExtraction` wraps the value into a `model.inputs[]` entry with target `bodyField` and body `{"downloadedFileOutput": }`. Edit `inputs.fileRef` only; never hand-edit the BPMN body. + +### Optional `attachment` input (Orchestrator job attachments) + +`inputDefinition.properties.attachment` accepts `{ ID, FullName, MimeType, Metadata }` for flows that consume Orchestrator job attachments. There is no form UI for this path on the standalone node today — set it programmatically in `inputs.attachment` if needed. `ID` is the only required field. Validate end-to-end on your tenant before relying on this path. + +## Accessing Output + +The extraction result is stored at `$vars.{nodeId}.output`. The IxP node's BPMN serializer maps the extraction service's `result` field directly to this variable (`source: '=result'`), so **the `result` wrapper is stripped** — `output` IS the extraction-result object, with no further wrapping. + +Top-level keys of `$vars.{nodeId}.output`: + +- `ExtractionResult` — `{ DocumentId, ResultsVersion, ResultsDocument }`. `ResultsDocument.Fields[]` carries the trained model's extracted values; `ResultsDocument.Tables[]` carries tabular extractions. +- `ExtractorPayloads` — provider-specific raw payloads. +- `BusinessrulesResults[]` — business-rule evaluation results, when configured. + +Each `Fields[]` element is shaped: + +```json +{ + "FieldId": "string", + "FieldName": "string", + "FieldType": "string", + "IsMissing": false, + "Values": ["string"], + "Confidence": 95 +} +``` + +Read field values via `find` against `FieldName`, then index into `Values[]`: + +```javascript +// In a Script node after the IxP node +const fields = $vars.extractInvoiceFields.output.ExtractionResult.ResultsDocument.Fields || []; +const total = fields.find(f => f.FieldName === 'invoiceTotal')?.Values?.[0]; +const vendor = fields.find(f => f.FieldName === 'vendor')?.Values?.[0]; +return { total, vendor }; +``` + +Sibling error variable: `$vars.{nodeId}.error` — populated when extraction fails *and* the `error` port is wired (`supportsErrorHandling: true`). Mapped from the service response's `Error` field (`source: '=Error'`). + +### Wrong shapes the agent tends to invent + +These all pass `flow validate` and fail silently at runtime: + +- **Wrong:** `output.result.ExtractionResult.…` — there is no `result` wrapper at runtime; `=result` strips it before the value is assigned to `output`. +- **Wrong:** `output.` flat — extracted fields are not top-level properties of `output`; they live under `output.ExtractionResult.ResultsDocument.Fields[]` and are keyed by `FieldName`. +- **Wrong:** `output.ExtractionResult.Fields` — `Fields[]` is two levels under `ExtractionResult` (`output.ExtractionResult.ResultsDocument.Fields`), not one. + +Studio Web's variable picker renders `output.ExtractionResult` as opaque and does NOT surface the nested `ResultsDocument.Fields[]` shape. The path above is the source of truth — copy it from this doc, not from picker autocomplete or `outputDefinition.output.schema` (the registry schema describes the pre-`=result` wrapper, not the runtime variable). + +### Trained-model field taxonomy + +The `FieldName` values present in `ResultsDocument.Fields[]` depend on the trained IxP model's taxonomy and are NOT exposed through `uip maestro flow registry get` (the registry's `outputDefinition.output.schema` describes the wrapper envelope shape, not the per-model trained fields). Get them from the deployment: + +```bash +uip ixp deployments get-taxonomy --folder-key "" --output json +``` + +Both args come from the `registry get` response you already fetched in [Build procedure](#build-procedure--copy-from-registry-get-do-not-construct-from-memory): `--folder-key` ← `Data.Node.inputDefaults.folderKey`; `` (positional) ← `Data.Node.inputDefaults.modelName`. No additional discovery step. Requires `uip login`; the command uses the user Bearer to call the same DU-App route that Studio Web's "Schema definition" panel uses. + +Response shape: + +```json +{ + "documentTaxonomy": { + "documentTypes": [ + { + "fields": [ + { + "fieldId": "string", + "fieldName": "string", + "type": "Text", + "components": [] + } + ] + } + ] + } +} +``` + +`type` is one of `Text`, `Date`, `Number`, `Set`, `FieldGroup`. `components[]` is populated only when `type` is `FieldGroup` and carries sub-fields with the same shape recursively. + +**camelCase → PascalCase translation.** The taxonomy response uses `fieldName` (camelCase); runtime `Fields[]` elements use `FieldName` (PascalCase). The string *contents* match — design-time `"Birth Date"` is `FieldName: "Birth Date"` at runtime — but the wrapper key changes case. Translate the key, not the value, when going from `get-taxonomy` output to runtime `Fields[].FieldName` lookups. + +Agent call sequence: + +1. `uip maestro flow registry search "uipath.ixp" --output json` — list IxP nodes. +2. `uip maestro flow registry get "" --output json` — read `Data.Node.inputDefaults.{folderKey, modelName}` (already done as part of [Build procedure](#build-procedure--copy-from-registry-get-do-not-construct-from-memory)). +3. `uip ixp deployments get-taxonomy --folder-key "" --output json` — read `documentTaxonomy.documentTypes[].fields[].fieldName`. +4. Author downstream consumers with `$vars..output.ExtractionResult.ResultsDocument.Fields.find(f => f.FieldName === '')?.Values?.[0]`. + +If the command fails (login expired, deployment not yet published, transient failure), fall back to defensive `find`-by-`FieldName` patterns with assumed field names and surface the assumptions to the user under **Open Questions**. Do NOT substitute a one-off extraction or IxP-product-UI inspection in the agent loop — `get-taxonomy` is the agent-loop path. + +## If the Model Does Not Exist Yet + +Trigger: `uip maestro flow registry search "uipath.ixp"` returns `Data: []`, OR the only matches are `uipath.agent.resource.tool.ixp.*` (agent-tool variant — not a flow extraction node). + +Action: insert a `core.logic.mock` placeholder via Direct JSON edit and stop. Do not iterate on registry searches. + +1. Fetch the definition: `uip maestro flow registry get core.logic.mock --output json`. Copy `Data.Node` verbatim into `definitions[]` if not already present. +2. Add a node to `nodes[]` with a stable id (e.g. `extractContractFieldsMock`), `type: "core.logic.mock"`, and a `display.label` whose **leading phrase** describes the work in the user's domain (e.g. `Extract Contract Fields`) rather than the underlying technology (`IxP Extraction`, `Run IxP`). The parenthetical may name IxP — e.g. `Extract Contract Fields (mock — IxP model not yet published)`. +3. Add a `layout.nodes` entry at `position: { x: 400, y: 144 }`, size `96x96`. +4. Wire edges per the parent [editing-operations.md](../../editing-operations.md) guide. `core.logic.mock` is a no-op pass-through — no `inputs`, no `outputs` block, no `bindings_v2.json` changes. +5. **Wire downstream consumers against the mock with `$vars` references, not static values.** Scripts, decisions, and end-node mappings that follow the mock MUST reference `$vars.{mockNodeId}.output` (the mock's only port) instead of hard-coded returns. Example: a script that summarises the (future) extraction writes `return { vendor: $vars.extractInvoiceFieldsMock.output.vendorName };`, not `return { ok: "OK" };`. This keeps the **node-graph** swap-ready — node IDs, edge shapes, and the `output` port name stay intact when the mock is replaced. **Field-access paths inside downstream scripts WILL need rewriting at swap time** — the real IxP `output` is shaped as `{ ExtractionResult: { ResultsDocument: { Fields: [...] } } }` (see [Accessing Output](#accessing-output)), so flat-field accessors against the mock become structured `Fields.find(f => f.FieldName === '')?.Values?.[0]` lookups against the real node. Surface the post-swap rewrite as a follow-up under **Open Questions**. +6. Run `uip maestro flow validate .flow --output json` once after all edits complete. + +Surface the missing model in the **Open Questions** section of the architectural plan: the user must train and publish the IxP extraction model via the IxP product before the flow can run. After publishing, follow the [mock replacement procedure](../../editing-operations-json.md#replace-a-mock-with-a-real-resource-node) to swap the mock for the real IxP node. + +## Classifier Variant + +IxP also exposes classifier models (type `Classifier`) that label documents rather than extracting named fields. Classifier models share the `uipath.ixp.*` node-type pattern but produce a different `output` shape (classification labels, not field values). **Classifier configuration is not covered in this file** — if the user needs classification, flag it as a prerequisite and defer to a future revision of this impl.md. + +## Debug + +| Error | Cause | Fix | +| --- | --- | --- | +| Node type not found in registry | Model not published, or registry cache stale | Run `uip login` then `uip maestro flow registry pull --force` | +| `model.context` rejected by runtime | `folderKey` or `modelName` missing from `inputs` (the context array is built from these) | Confirm `inputs.modelName` and `inputs.folderKey` are populated. | +| Empty `$vars.{nodeId}.output` | Model's taxonomy doesn't match the document, or extraction silently returned no fields | Inspect the raw API response via `$vars.{nodeId}.error` first; if no error, run the extraction against the same document on the IxP product UI to compare | +| `fileRef` not resolving | Expression references an upstream variable that isn't wired, or the upstream node didn't produce a file output | Verify the upstream node exports a file reference and that the `=js:$vars.{upstreamId}.output.` expression matches | +| Extraction failed | Underlying IxP model errored (unsupported MIME type, corrupted file, service-side failure) | Check `$vars.{nodeId}.error.detail` for the IxP service response | +| `uip maestro flow node configure` rejects with "not a connector type node" | Expected — IxP is not a connector. | Edit `inputs.*` in the `.flow` JSON directly. | +| Studio Web: "Cannot destructure property 'modelName' of 't' as it is undefined" when clicking the node | `inputs.model` blob is missing or undefined. The `schema-definition` form section binds `inputs.model` to the `ixp-model-taxonomy` component, which destructures `modelName` and `folderKey` out of it. When `inputs.model` is missing, the destructure throws. | Copy `definition.inputDefaults.model` verbatim into the node instance's `inputs.model`. The blob carries `modelName`, `fullyQualifiedName`, `description`, `folderKey`, `folderName` plus DU-API extras (`kind`, `type`, `detailsUrl`, `asyncDigitizationUrl`, `asyncExtractionUrl`). See [JSON Structure](#json-structure). | diff --git a/skills/uipath-maestro-flow/references/author/references/plugins/ixp/planning.md b/skills/uipath-maestro-flow/references/author/references/plugins/ixp/planning.md new file mode 100644 index 000000000..33344da8a --- /dev/null +++ b/skills/uipath-maestro-flow/references/author/references/plugins/ixp/planning.md @@ -0,0 +1,90 @@ +# IxP Extraction Node — Planning + +IxP Extraction nodes invoke a **trained, published** UiPath Intelligent eXtraction Platform (IxP) model to pull structured fields out of unstructured or semi-structured documents (PDFs, photos, scanned forms). They are tenant-specific resources that appear in the registry after `uip login` + `uip maestro flow registry pull`. + +In the IxP product these published models are also called **runtime projects** — the bundle published to the tenant runtime and callable from a flow. Treat "what IxP models do I have", "what runtime projects do I have access to in flow", and "list published extractors" as the same listing question — see [Listing Available Models / Runtime Projects](#listing-available-models--runtime-projects). + +## Node Type Pattern + +`uipath.ixp.{modelName}.{fullyQualifiedName}` + +Unlike other resource plugins that use the `uipath.core.*` prefix, IxP nodes use `uipath.ixp.*` and a **two-segment tail**: `{modelName}` is the model's display name, and `{fullyQualifiedName}` is the model's fully-qualified name on the tenant (e.g. `Shared/invoice-model`). Both segments are sanitized — see [impl.md](impl.md) for the rule and concrete examples. Model versioning is a separate input field (`versionTag`), not part of the node type. + +## When to Use + +Use an IxP node when the flow needs to extract **named fields** (invoice totals, contract parties, receipt line items, form values) from documents whose **layout varies across inputs** and which are not already machine-readable. + +### IxP vs Script/Connector/HTTP/Agent Decision Table + +| Use an IxP node when... | Use something else when... | +| --- | --- | +| Source is a PDF, scanned document, photo, or email attachment | Source is already structured (CSV, JSON, database row) — use [Script](../script/planning.md) | +| Fields have variable layout across documents (e.g. invoices from many vendors) | Layout is fixed and parseable by regex/XPath — use [Script](../script/planning.md) | +| A trained IxP model exists for this document type | No model exists — flag as a prerequisite and use `core.logic.mock` until the model is trained | +| Need field-level extraction with per-field confidence scores for downstream HITL review | Need free-form summarization, classification, or reasoning over text — use [Agent](../agent/planning.md) | + +### Anti-Pattern + +Don't use IxP as a generic OCR. IxP is field-oriented — it extracts named fields against a trained taxonomy. Running IxP to get raw text back and then parsing the text downstream throws away the part of IxP that's actually valuable. + +### When NOT to Use + +- **Model not yet trained or published** — use `core.logic.mock` and surface in Open Questions that a model must be trained on the IxP product before the flow can run. +- **Structured input (CSV/JSON/DB row)** — use [Script](../script/planning.md) or [Decision](../decision/planning.md). IxP adds latency and model-inference cost for no gain. +- **Free-form summarization, classification, or reasoning** — use [Agent](../agent/planning.md). An IxP model trained for field extraction will not produce useful output for those tasks. +- **Manual external API call** — use [Connector](../connector/planning.md) or [HTTP](../http/planning.md). + +## Prerequisites + +- `uip login` — IxP nodes only appear in the registry after authentication. +- `uip maestro flow registry pull --force` must be run to cache IxP model node types locally. +- A trained, published IxP extraction model must exist on the tenant. If none exists, surface it in the **Open Questions** section of the architectural plan so the user can train one while reviewing. + +## Ports + +| Input Port | Output Port(s) | +| --- | --- | +| `input` | `success`, `error` (the visible `error` port / edge target is gated by `inputs.errorHandlingEnabled`; the `outputs.error` schema entry on the node instance is always required — see [impl.md Authoring rule #4](impl.md#authoring-rules)) | + +## Output Variables + +- `$vars.{nodeId}.output` — the extraction-result object directly. The BPMN serializer maps the service's `result` field with `source: '=result'`, so **the `result` wrapper is stripped** at runtime. Top-level keys: `ExtractionResult`, `ExtractorPayloads`, `BusinessrulesResults`. Field-level values live under `output.ExtractionResult.ResultsDocument.Fields[]` / `Tables[]`; exact field names depend on the trained model's taxonomy. See [impl.md — Accessing Output](impl.md#accessing-output) for the canonical access path. +- `$vars.{nodeId}.error` — populated when the `error` port is wired and extraction fails. Mapped from the IxP service response's `Error` field (`source: '=Error'`) — see [impl.md — Accessing Output](impl.md#accessing-output). + +## Discovery + +```bash +uip maestro flow registry pull --force +uip maestro flow registry search "uipath.ixp" --output json +``` + +Requires `uip login`. Only published IxP models from your tenant appear. The returned node type uses a **two-segment tail** (`{modelName}.{fullyQualifiedName}`), unlike `uipath.core.*` siblings which use a single-segment tail. Both tail segments are sanitized: lowercase, then runs of any character outside `[a-z0-9]` → single `-`. So an FQN of `Shared/invoice-model` lands as `shared-invoice-model`. See [impl.md](impl.md) for the full rule and worked examples. + +### If `Data: []` → plan for a mock + Open Question + +If the search returns no `uipath.ixp.*` nodes, no IxP extraction model is published on this tenant. Plan the architecture around a `core.logic.mock` placeholder for the extraction step, and surface the missing model in **Open Questions** so the user can train and publish it. Do not iterate on registry searches — the mock is the planning answer until the model exists. See [impl.md — If the Model Does Not Exist Yet](impl.md#if-the-model-does-not-exist-yet) for the implementer-side procedure. + +## Listing Available Models / Runtime Projects + +Q&A use case: user asks "what IxP models do I have?", "what runtime projects do I have access to in flow?", "what document extractors can I access in flow?", "which document extractors are published?". + +Read-only listing — **do not** scaffold a solution, init a flow, or write a `.flow` file: + +1. Confirm `uip login status --output json`. Without login, only OOTB nodes are returned and tenant IxP models will not appear. +2. Refresh the cache and search: + ```bash + uip maestro flow registry pull --force + uip maestro flow registry search "uipath.ixp" --output json + ``` +3. Format `Data[]` as a table — `DisplayName`, `NodeType`, `Version`. Each entry is one published model / runtime project. See [impl.md — Listing Published Models](impl.md#listing-published-models) for parsing details. + +Search term must be `"uipath.ixp"` (lowercase). Do NOT use `"runtime"`, `"document extractor"`, `"extractor"`, or `"IXP"` — these miss extraction nodes. + +If `Data: []`, answer directly that no IxP models are published on the tenant. Do not propose a `core.logic.mock` workaround — the mock is for build-time planning, not for listing-time Q&A. + +## Planning Annotation + +In the architectural plan: + +- If the model exists: note as `resource: (ixp-extraction)` with the intended document type (e.g. "resource: vendor-invoices (ixp-extraction) — extract invoice header + line items") +- If it does not exist: note as `[CREATE NEW] ` and flag in Open Questions that an IxP model must be trained before the flow can run diff --git a/tests/tasks/uipath-maestro-flow/ixp/activation.yaml b/tests/tasks/uipath-maestro-flow/ixp/activation.yaml new file mode 100644 index 000000000..ce61c74cc --- /dev/null +++ b/tests/tasks/uipath-maestro-flow/ixp/activation.yaml @@ -0,0 +1,89 @@ +task_id: skill-flow-ixp-activation +description: > + IxP activation — 2 positive activation tests (explicit, invoice-extraction) + fanned out via `dataset.rows`. Each row carries a distinct user prompt; success + criteria are identical across all rows. + + Scope: did the skill get called, not did it work. Assertions verify the agent + invoked the uipath-maestro-flow Skill and ran the right CLI-command patterns + (registry search for uipath.ixp.*, node-add against uipath.ixp.* or + core.logic.mock) — NOT whether the resulting flow is correct or would execute. + + Failure here is signal about SKILL.md or the IxP Plugin Index entry — + do NOT relax assertions; feed findings back in. + +tags: [uipath-maestro-flow, smoke, activation, ixp] + +task_timeout: 900 + +agent: + type: claude-code + max_turns: 40 + turn_timeout: 900 + +sandbox: + driver: tempdir + python: {} + +dataset: + rows: + - id: explicit + prompt: >- + I have a UiPath Flow. Add an IXP node that reads PDF invoices from a + SharePoint folder and extracts the structured fields so the next step + can post them to SAP. + + - id: invoice-extraction + prompt: >- + Build a UiPath Flow that pulls vendor name, invoice number, invoice + date, total amount, and line items out of PDF invoices saved in a + SharePoint folder, then posts the result to our accounting system. + +initial_prompt: | + ${row.prompt} + + Important: + - The `uip` CLI is already available. + - Use `--output json` on all uip commands. + - Work inside the stated working directory — do NOT `cd` to a parent. + - Do NOT run `uip maestro flow debug` or `uip maestro flow deploy`. + - Run `uip maestro flow validate` at most ONCE. If it fails, stop. + - Do NOT iterate on validation errors or rewrite the flow to fix them. A broken flow is acceptable. + - Exploration + an attempted node-add is enough — you do not need to build a working flow. + +success_criteria: + - type: skill_triggered + expected: "yes" + skill_name: "uipath:uipath-maestro-flow" + description: "Agent invoked the uipath-maestro-flow Skill" + weight: 2.0 + pass_threshold: 1.0 + + - type: command_executed + description: "Agent searched the registry for IxP node types" + tool_name: "Bash" + command_pattern: 'uip\s+maestro\s+flow\s+registry\s+(search|list|get)\b[^\n]*ixp' + min_count: 1 + weight: 2.0 + pass_threshold: 1.0 + + # Greps the actual .flow artifact rather than the command stream — the + # command_executed checker truncates tool params at 2000 chars, which hides + # `"type": "core.logic.mock"` when the agent writes a large flow file in a + # single Write call. Per references/plugins/ixp/impl.md, the agent must land + # either an `uipath.ixp.*` node or a `core.logic.mock` placeholder. + # + # Searches `../..` rather than `.` so the assertion still passes if the agent + # `cd`s out of the stated sandbox dir before creating the solution. The cwd + # for run_command is the sandbox dir (coder_eval Sandbox.run_command), which + # for dataset-fanout tasks resolves to + # /artifacts/// + # so `../..` is the row's iteration `artifacts/` root and stays isolated + # from other rows. ASSUMES dataset fan-out (2-segment task_id); flattening + # this task to a non-dataset form would require dropping a `..`. + - type: run_command + description: "Agent added an IxP or core.logic.mock placeholder node to a .flow file (per references/plugins/ixp/impl.md)" + command: 'grep -rqE "\"type\"[[:space:]]*:[[:space:]]*\"(uipath\.ixp|core\.logic\.mock)" --include="*.flow" ../..' + expected_exit_code: 0 + weight: 2.0 + pass_threshold: 1.0 diff --git a/tests/tasks/uipath-maestro-flow/ixp/activation_listing.yaml b/tests/tasks/uipath-maestro-flow/ixp/activation_listing.yaml new file mode 100644 index 000000000..03e6ac1bd --- /dev/null +++ b/tests/tasks/uipath-maestro-flow/ixp/activation_listing.yaml @@ -0,0 +1,67 @@ +task_id: skill-flow-ixp-activation-listing +description: > + IxP listing activation — user asks what IxP models / runtime projects are + available from a Maestro flow. Read-only Q&A, not a build task. + + Asserts the agent invoked the uipath-maestro-flow Skill and ran the IxP + registry-search listing command (per references/plugins/ixp/impl.md — + Listing Published Models). Also asserts the agent did NOT take the build + path: no .flow file should be created for a listing question. + + Failure here is signal about the IxP plugin's Listing Published Models / + Listing Available Models sections — do NOT relax assertions, tighten the + copy. + +tags: [uipath-maestro-flow, smoke, activation, ixp, listing] + +# Listing is a one-shot Q&A — no need for the full activation budget. +task_timeout: 300 + +agent: + type: claude-code + max_turns: 20 + turn_timeout: 300 + +sandbox: + driver: tempdir + python: {} + +dataset: + rows: + - id: ixp-models + prompt: >- + What IxP models can I access in maestro? + +initial_prompt: | + ${row.prompt} + + Important: + - The `uip` CLI is already available. + - Use `--output json` on all uip commands. + +success_criteria: + - type: skill_triggered + expected: "yes" + skill_name: "uipath:uipath-maestro-flow" + description: "Agent invoked the uipath-maestro-flow Skill" + weight: 2.0 + pass_threshold: 1.0 + + - type: command_executed + description: "Agent searched the registry for IxP node types (listing path)" + tool_name: "Bash" + command_pattern: 'uip\s+maestro\s+flow\s+registry\s+(search|list)\b[^\n]*ixp' + min_count: 1 + weight: 2.0 + pass_threshold: 1.0 + + # Inverted check: a listing/Q&A task should NOT create a .flow file. + # Same `../..` rationale as activation.yaml — dataset-fanout cwd is + # /artifacts///. `! find ... | grep -q .` exits + # 0 when no .flow file is found (PASS) and 1 when one exists (FAIL). + - type: run_command + description: "No .flow file was created (listing is read-only Q&A, not a build task)" + command: '! find ../.. -name "*.flow" -type f | grep -q .' + expected_exit_code: 0 + weight: 2.0 + pass_threshold: 1.0 diff --git a/tests/tasks/uipath-maestro-flow/ixp/activation_negative.yaml b/tests/tasks/uipath-maestro-flow/ixp/activation_negative.yaml new file mode 100644 index 000000000..3aa8b64f4 --- /dev/null +++ b/tests/tasks/uipath-maestro-flow/ixp/activation_negative.yaml @@ -0,0 +1,70 @@ +task_id: skill-flow-ixp-activation-negative +description: > + Negative IxP routing — verifies the agent does NOT add a uipath.ixp.* + node when the user asks for a Maestro flow with no document-extraction + scope. Parent uipath-maestro-flow Skill must still fire (this is in-skill + plugin-routing coverage; "should not fire maestro-flow at all" is upstream + in coder_eval/tasks/uipath_flow/triggering/triggering.jsonl). + + Each row carries a flow-shaped prompt that does not mention documents, + PDFs, invoices, receipts, contracts, forms, or field extraction. The IxP + plugin index / planning copy / SKILL.md must be tight enough that the + agent does not over-trigger. + + Failure here means activation copy is too broad — tighten the trigger + signals, do NOT relax assertions. + +tags: [uipath-maestro-flow, smoke, activation, ixp, negative] + +# Trimmed vs the positive activation suite (40 turns / 900s) — negative rows +# do not need to drive a full registry-search → node-add → validate loop. +task_timeout: 600 + +agent: + type: claude-code + max_turns: 20 + turn_timeout: 600 + +sandbox: + driver: tempdir + python: {} + +dataset: + rows: + + - id: stripe-http + prompt: >- + Create a flow with a manual trigger and an HTTP node that calls + api.stripe.com, then a script node that logs the response. + +initial_prompt: | + ${row.prompt} + + Important: + - The `uip` CLI is already available. + - Use `--output json` on all uip commands. + - Work inside the stated working directory — do NOT `cd` to a parent. + - Do NOT run `uip maestro flow debug` or `uip maestro flow deploy`. + - Run `uip maestro flow validate` at most ONCE. If it fails, stop. + - Do NOT iterate on validation errors or rewrite the flow to fix them. A broken flow is acceptable. + - Exploration + an attempted node-add is enough — you do not need to build a working flow. + +success_criteria: + - type: skill_triggered + expected: "yes" + skill_name: "uipath:uipath-maestro-flow" + description: "Parent uipath-maestro-flow Skill fired (IxP-only routing is what this suite tests)" + weight: 1.0 + pass_threshold: 1.0 + + # Inverted twin of the positive activation.yaml grep. + # `shell=True` in coder_eval Sandbox.run_command means `!` works as a bash + # negator: grep -q exits 0 on match → ! flips to 1 (FAIL); exits 1 on + # no-match → ! flips to 0 (PASS). Same `../..` rationale as the positive + # test (dataset-fanout cwd is /artifacts///). + - type: run_command + description: "No uipath.ixp.* node landed in any .flow file (negative IxP routing)" + command: '! grep -rqE "\"type\"[[:space:]]*:[[:space:]]*\"uipath\.ixp" --include="*.flow" ../..' + expected_exit_code: 0 + weight: 3.0 + pass_threshold: 1.0 diff --git a/tests/tasks/uipath-maestro-flow/ixp/e2e_01_invoice_extraction_greenfield.yaml b/tests/tasks/uipath-maestro-flow/ixp/e2e_01_invoice_extraction_greenfield.yaml new file mode 100644 index 000000000..801d7950a --- /dev/null +++ b/tests/tasks/uipath-maestro-flow/ixp/e2e_01_invoice_extraction_greenfield.yaml @@ -0,0 +1,105 @@ +task_id: skill-flow-ixp-e2e-invoice-extraction-greenfield +description: > + E2E (greenfield): SharePoint trigger → IxP extraction → HTTP POST. Tests + end-to-end authoring of a real invoice-processing flow that monitors a + SharePoint folder, extracts structured invoice fields, and forwards the + result to a downstream system over HTTP. + + Validate-only: no `uip maestro flow debug`. IxP runtime + SharePoint + connector both require a tenant deployment which CI does not have; e2e + verifies offline structural correctness only (`uip maestro flow validate` + exit 0 on the generated flow). + +tags: [uipath-maestro-flow, e2e, lifecycle:generate, shape:multi-node, node:ixp, connector, feature:http] + +task_timeout: 900 + +agent: + type: claude-code + permission_mode: acceptEdits + max_turns: 80 + turn_timeout: 1200 + +sandbox: + driver: tempdir + python: {} + +initial_prompt: | + Build a UiPath Flow that monitors a SharePoint folder for new vendor PDF + invoices, extracts the structured data (invoice number, vendor, invoice + date, total amount, line items), and posts the result to SAP through an + HTTP request. + + - SharePoint connector: configured (use a connector trigger node) + - SharePoint folder: "New Vendor" + - IxP model: "Invoice Model" + - SAP: HTTP POST request to the configured endpoint, sending the extracted + invoice fields as the request body + + Important: + - The `uip` CLI is already available. + - Use `--output json` on all uip commands. + - Work inside the stated working directory — do NOT `cd` to a parent. + - Do NOT run `uip maestro flow debug` or `uip maestro flow deploy`. + - Run `uip maestro flow validate` at most ONCE. If it fails, stop. + - Do NOT iterate on validation errors. + +success_criteria: + - type: skill_triggered + expected: "yes" + skill_name: "uipath:uipath-maestro-flow" + description: "Agent invoked the uipath-maestro-flow Skill" + weight: 2.0 + pass_threshold: 1.0 + + - type: command_executed + description: "Agent searched the registry for IxP node types" + tool_name: "Bash" + command_pattern: 'uip\s+maestro\s+flow\s+registry\s+(search|list|get)\b[^\n]*ixp' + min_count: 1 + weight: 2.0 + pass_threshold: 1.0 + + - type: command_executed + description: "Agent searched the registry for a SharePoint connector" + tool_name: "Bash" + command_pattern: 'uip\s+maestro\s+flow\s+registry\s+(search|list|get)\b[^\n]*sharepoint' + min_count: 1 + weight: 1.5 + pass_threshold: 1.0 + + - type: run_command + description: "uip maestro flow validate passes on the generated flow file" + command: 'f=$(find . -name "*.flow" -print -quit); [ -n "$f" ] && uip maestro flow validate "$f" --output json' + timeout: 60 + expected_exit_code: 0 + weight: 3.0 + pass_threshold: 1.0 + + - type: run_command + description: "Generated flow contains an IxP extraction node or core.logic.mock fallback" + command: 'grep -rqE "\"type\"[[:space:]]*:[[:space:]]*\"(uipath\.ixp|core\.logic\.mock)" --include="*.flow" .' + expected_exit_code: 0 + weight: 2.0 + pass_threshold: 1.0 + + - type: run_command + description: "Generated flow contains a SharePoint trigger or connector node" + command: 'grep -rqiE "sharepoint" --include="*.flow" .' + expected_exit_code: 0 + weight: 2.0 + pass_threshold: 1.0 + + - type: run_command + description: "Generated flow contains an HTTP action node (core.action.http)" + command: 'grep -rqE "core\.action\.http|\"type\"[[:space:]]*:[[:space:]]*\"[^\"]*http[^\"]*\"" --include="*.flow" .' + expected_exit_code: 0 + weight: 2.0 + pass_threshold: 1.0 + + - type: run_command + description: "Generated flow references the IxP extraction output via $vars (downstream consumption)" + command: 'grep -rqE "\$vars\." --include="*.flow" .' + expected_exit_code: 0 + weight: 1.5 + pass_threshold: 1.0 diff --git a/tests/tasks/uipath-maestro-flow/ixp/e2e_02_project_selection.yaml b/tests/tasks/uipath-maestro-flow/ixp/e2e_02_project_selection.yaml new file mode 100644 index 000000000..b3a22d27b --- /dev/null +++ b/tests/tasks/uipath-maestro-flow/ixp/e2e_02_project_selection.yaml @@ -0,0 +1,154 @@ +task_id: skill-flow-ixp-e2e-project-selection +description: > + E2E (project selection): on a tenant where the demo IxP project set is + published (one model per zip in scripts/ixp-demo-projects/packages/), the + agent must pick the published model that matches the user's document + domain — not fall back to core.logic.mock and not pick a different + domain's model. + + Each row names a different document type (aviation report, birth + certificate) and lists the fields the flow should extract. The prompts + deliberately do NOT name the IxP model, NodeType, or fully-qualified + name — picking the right entry from `registry search "uipath.ixp"` is + exactly the skill behaviour under test. + + Per-row strictness comes from a run_command that switches on the row dir + in $(pwd) (dataset-fanout cwd is /artifacts//[/]/, + see comment in activation.yaml) and asserts BOTH the chosen NodeType + prefix AND the inputs.modelName match the published model for that row. + + Validate-only: no `uip maestro flow debug`. The expected NodeType prefixes + and DisplayNames assume the runner publishes the IxP demo set as named + below — the source packages live outside this repo (CI runner / external + publisher), not under `scripts/`. If a publisher renames a project, + reconcile the case statement below against + `uip maestro flow registry search "uipath.ixp" --output json` on the + runner before merge. + +tags: [uipath-maestro-flow, e2e, lifecycle:generate, shape:single-node, node:ixp] + +task_timeout: 1200 + +agent: + type: claude-code + permission_mode: acceptEdits + max_turns: 60 + turn_timeout: 1200 + +sandbox: + driver: tempdir + python: {} + +dataset: + rows: + - id: aviation + prompt: >- + Build a Maestro flow that reads NTSB aviation accident investigation + final-report PDFs and extracts the NTSB number, event date, location + (city/state), aircraft make and model, and aircraft damage. Trigger + manually; the downstream script logs the NTSB number. + + - id: birth-certificate + prompt: >- + Build a Maestro flow that reads scanned birth certificates and + extracts the child's full name, date of birth, place of birth, and + parents' names. Trigger manually; the downstream script logs the + child's full name. + +initial_prompt: | + ${row.prompt} + + Important: + - The `uip` CLI is already available and the runner is logged in. + - Use `--output json` on all uip commands. + - Work inside the stated working directory — do NOT `cd` to a parent. + - Do NOT run `uip maestro flow debug` or `uip maestro flow deploy`. + - Run `uip maestro flow validate` at most ONCE. If it fails, stop. + - Do NOT iterate on validation errors. + +success_criteria: + - type: skill_triggered + expected: "yes" + skill_name: "uipath:uipath-maestro-flow" + description: "Agent invoked the uipath-maestro-flow Skill" + weight: 2.0 + pass_threshold: 1.0 + + - type: command_executed + description: "Agent searched the registry for IxP node types" + tool_name: "Bash" + command_pattern: 'uip\s+maestro\s+flow\s+registry\s+(search|list|get)\b[^\n]*ixp' + min_count: 1 + weight: 2.0 + pass_threshold: 1.0 + + # Same `../..` cwd rationale as activation.yaml:107-112: dataset-fanout cwd + # is /artifacts//[/]/, so artifacts written + # by the agent live two levels up at most. + - type: run_command + description: "uip maestro flow validate passes on the generated flow" + command: 'f=$(find ../.. -name "*.flow" -print -quit); [ -n "$f" ] && uip maestro flow validate "$f" --output json' + timeout: 60 + expected_exit_code: 0 + weight: 3.0 + pass_threshold: 1.0 + + # Real IxP node landed — agent did NOT fall back to mock on a tenant where + # a domain-matching IxP project IS published. + - type: run_command + description: "Generated flow contains a real uipath.ixp.* node" + command: 'grep -rqE "\"type\"[[:space:]]*:[[:space:]]*\"uipath\.ixp\." --include="*.flow" ../..' + expected_exit_code: 0 + weight: 3.0 + pass_threshold: 1.0 + + # Inverted twin of the above. `! grep -q` exits 0 when no mock is found + # (PASS) and 1 when one exists (FAIL). Same `../..` rationale as above. + - type: run_command + description: "Generated flow does NOT contain a core.logic.mock fallback" + command: '! grep -rqE "\"type\"[[:space:]]*:[[:space:]]*\"core\.logic\.mock\"" --include="*.flow" ../..' + expected_exit_code: 0 + weight: 2.0 + pass_threshold: 1.0 + + # Per-row strict check: BOTH the chosen NodeType AND inputs.modelName must + # match the published model for this row's domain. Expected strings are + # pinned from `uip maestro flow registry search "uipath.ixp" --output json` + # against the CI runner's tenant. Re-pin if the publisher re-publishes — + # the hex IDs are runtime-project IDs assigned at publish time, not the + # dataset IDs in scripts/ixp-demo-projects/datasets.jsonl. + - type: run_command + description: "IxP node references the exact published model for this row's domain (strict)" + command: | + case "$(pwd)" in + *aviation*) + model='aviation-investigation-final-report-demo-a42b1d4d-ixp' + nt='uipath\.ixp\.aviation-investigation-final-report-demo-a42b1d4d-ixp\.shared-aviation-investigation-final-report-demo-a42b1d4d-ixp' + ;; + *birth*) + model='birth_certificates_oob-6252526a-ixp' + nt='uipath\.ixp\.birth-certificates-oob-6252526a-ixp\.shared-birth-certificates-oob-6252526a-ixp' + ;; + *) + echo "Unknown row in pwd: $(pwd)" >&2 + exit 2 + ;; + esac + f=$(find ../.. -name "*.flow" -print -quit) + if [ -z "$f" ]; then + echo "no .flow file under ../.." >&2 + exit 1 + fi + if ! grep -qF "\"modelName\": \"${model}\"" "$f"; then + echo "modelName mismatch — expected '${model}' in $f" >&2 + grep -E '"modelName"' "$f" >&2 || true + exit 1 + fi + if ! grep -qE "\"type\"[[:space:]]*:[[:space:]]*\"${nt}" "$f"; then + echo "NodeType mismatch — expected prefix '${nt}' in $f" >&2 + grep -E '"type"[[:space:]]*:[[:space:]]*"uipath\.ixp' "$f" >&2 || true + exit 1 + fi + expected_exit_code: 0 + weight: 5.0 + pass_threshold: 1.0 diff --git a/tests/tasks/uipath-maestro-flow/ixp/integration_handle_routing.yaml b/tests/tasks/uipath-maestro-flow/ixp/integration_handle_routing.yaml new file mode 100644 index 000000000..d3de750bc --- /dev/null +++ b/tests/tasks/uipath-maestro-flow/ixp/integration_handle_routing.yaml @@ -0,0 +1,122 @@ +task_id: skill-flow-ixp-integration-handle-routing +description: > + Integration: IxP extraction wired into a Decision node that routes on a + field from the extracted content. Exercises field-level variable access + via the canonical IxP path + (`$vars.{ixpNode}.output.ExtractionResult.ResultsDocument.Fields.find(...)`) + and two-branch Decision wiring — the multi-node Quality scenario for IxP. + + Validate-only: no `uip maestro flow debug`. Validation in CI is offline + (`uip maestro flow validate`). + +tags: [uipath-maestro-flow, integration, lifecycle:generate, shape:multi-node, node:ixp, node:decision] + +task_timeout: 900 + +agent: + type: claude-code + permission_mode: acceptEdits + max_turns: 60 + turn_timeout: 900 + +sandbox: + driver: tempdir + python: {} + +initial_prompt: | + Build a UiPath Flow project named "InvoiceRouter" that processes vendor + invoices: + + 1. Manual trigger. + 2. IxP extraction node — extracts at least vendor name and total amount + from a PDF invoice using the model "Invoice Model" with the storage + bucket named "InvoiceBucket". + 3. Decision node — branches on the extracted total amount: if greater + than 1000, route to a "HighValueReview" script; otherwise route to an + "AutoApprove" script. + 4. Both downstream scripts contain a JavaScript that logs a one-line + summary including the extracted vendor name. + + Wire: Trigger → IxP extract → Decision → (two branches: HighValueReview + / AutoApprove). + + The Decision node condition MUST reference the extracted total amount + via the canonical IxP runtime path — + `$vars..output.ExtractionResult.ResultsDocument.Fields.find(f => f.FieldName === '')?.Values?.[0]` + — per references/plugins/ixp/impl.md ("Accessing Output"). + + Important: + - The `uip` CLI is already available. + - Use `--output json` on all uip commands. + - Work inside the stated working directory — do NOT `cd` to a parent. + - Do NOT run `uip maestro flow debug` or `uip maestro flow deploy`. + - Run `uip maestro flow validate` at most ONCE. If it fails, stop. + - Do NOT iterate on validation errors. + +success_criteria: + - type: skill_triggered + expected: "yes" + skill_name: "uipath:uipath-maestro-flow" + description: "Agent invoked the uipath-maestro-flow Skill" + weight: 2.0 + pass_threshold: 1.0 + + - type: command_executed + description: "Agent searched the registry for IxP node types" + tool_name: "Bash" + command_pattern: 'uip\s+maestro\s+flow\s+registry\s+(search|list|get)\b[^\n]*ixp' + min_count: 1 + weight: 2.0 + pass_threshold: 1.0 + + - type: run_command + description: "uip maestro flow validate passes on InvoiceRouter.flow" + command: "uip maestro flow validate InvoiceRouter/InvoiceRouter/InvoiceRouter.flow --output json" + timeout: 30 + expected_exit_code: 0 + weight: 3.0 + pass_threshold: 1.0 + + - type: run_command + description: "Generated flow contains an IxP extraction node or core.logic.mock fallback" + command: 'grep -rqE "\"type\"[[:space:]]*:[[:space:]]*\"(uipath\.ixp|core\.logic\.mock)" --include="*.flow" .' + expected_exit_code: 0 + weight: 2.0 + pass_threshold: 1.0 + + - type: file_contains + description: "Flow contains a Decision node (core.logic.decision)" + path: "InvoiceRouter/InvoiceRouter/InvoiceRouter.flow" + includes: + - "core.logic.decision" + weight: 2.0 + pass_threshold: 1.0 + + - type: file_contains + description: "Decision condition references the canonical IxP output path ($vars..output.ExtractionResult.ResultsDocument.Fields)" + path: "InvoiceRouter/InvoiceRouter/InvoiceRouter.flow" + includes: + - "$vars." + - "ExtractionResult" + - "ResultsDocument" + - "Fields" + weight: 3.0 + pass_threshold: 1.0 + + - type: file_contains + description: "Both branch script nodes are present by name" + path: "InvoiceRouter/InvoiceRouter/InvoiceRouter.flow" + includes: + - "HighValueReview" + - "AutoApprove" + weight: 2.5 + pass_threshold: 1.0 + + - type: file_contains + description: "Both Decision branches wired (true and false handles)" + path: "InvoiceRouter/InvoiceRouter/InvoiceRouter.flow" + includes: + - '"true"' + - '"false"' + weight: 2.0 + pass_threshold: 1.0 diff --git a/tests/tasks/uipath-maestro-flow/ixp/smoke_01_scaffold_minimal.yaml b/tests/tasks/uipath-maestro-flow/ixp/smoke_01_scaffold_minimal.yaml new file mode 100644 index 000000000..610f1b045 --- /dev/null +++ b/tests/tasks/uipath-maestro-flow/ixp/smoke_01_scaffold_minimal.yaml @@ -0,0 +1,89 @@ +task_id: skill-flow-ixp-smoke-scaffold-minimal +description: > + Smoke: minimal scaffold — manual trigger → IxP extract → script (logs "ok") + → validate. Tests that the agent picks the IxP plugin, authors a single + extraction node via Direct JSON, and produces a flow that passes + `uip maestro flow validate`. + + Validate-only: no `uip maestro flow debug`. IxP runtime requires a tenant + deployment which CI does not have; smoke verifies offline structural + correctness only. + +tags: [uipath-maestro-flow, smoke, lifecycle:generate, shape:single-node, node:ixp] + +task_timeout: 600 + +agent: + type: claude-code + permission_mode: acceptEdits + max_turns: 40 + turn_timeout: 600 + +sandbox: + driver: tempdir + python: {} + +initial_prompt: | + Create a new Maestro Flow project called "Smoke" with a manual trigger, an + extract node, and a script node that just logs "ok". Wire the extracted + handle to the script and validate the flow. + + Document is in the storage bucket named: InvoiceBucket + Model used: Invoice Model + Script node: contains a JavaScript that logs OK when it receives the + extraction result. + + Important: + - The `uip` CLI is already available. + - Use `--output json` on all uip commands. + - Work inside the stated working directory — do NOT `cd` to a parent. + - Do NOT run `uip maestro flow debug` or `uip maestro flow deploy`. + - Run `uip maestro flow validate` at most ONCE. If it fails, stop. + - Do NOT iterate on validation errors. + +success_criteria: + - type: skill_triggered + expected: "yes" + skill_name: "uipath:uipath-maestro-flow" + description: "Agent invoked the uipath-maestro-flow Skill" + weight: 2.0 + pass_threshold: 1.0 + + - type: command_executed + description: "Agent searched the registry for IxP node types" + tool_name: "Bash" + command_pattern: 'uip\s+maestro\s+flow\s+registry\s+(search|list|get)\b[^\n]*ixp' + min_count: 1 + weight: 2.0 + pass_threshold: 1.0 + + - type: command_executed + description: "Agent validated the .flow file" + tool_name: "Bash" + command_pattern: 'uip\s+(maestro\s+)?flow\s+validate' + min_count: 1 + weight: 1.5 + pass_threshold: 1.0 + + - type: run_command + description: "uip maestro flow validate passes on Smoke.flow" + command: "uip maestro flow validate Smoke/Smoke/Smoke.flow --output json" + timeout: 30 + expected_exit_code: 0 + weight: 3.0 + pass_threshold: 1.0 + + - type: run_command + description: "Generated flow contains an IxP extraction node or core.logic.mock fallback (per references/plugins/ixp/impl.md)" + command: 'grep -rqE "\"type\"[[:space:]]*:[[:space:]]*\"(uipath\.ixp|core\.logic\.mock)" --include="*.flow" .' + expected_exit_code: 0 + weight: 2.0 + pass_threshold: 1.0 + + - type: file_contains + description: "Script node consumes the extraction output via $vars expression" + path: "Smoke/Smoke/Smoke.flow" + includes: + - "$vars." + weight: 1.5 + pass_threshold: 1.0 diff --git a/tests/tasks/uipath-maestro-flow/ixp/smoke_02_scaffold_multinode.yaml b/tests/tasks/uipath-maestro-flow/ixp/smoke_02_scaffold_multinode.yaml new file mode 100644 index 000000000..e840bdf51 --- /dev/null +++ b/tests/tasks/uipath-maestro-flow/ixp/smoke_02_scaffold_multinode.yaml @@ -0,0 +1,105 @@ +task_id: skill-flow-ixp-smoke-scaffold-multinode +description: > + Smoke: multi-script fan-out from an IxP extraction — manual trigger → + IxP extract → 3 script nodes (postReceipt, sendToValidation, logError). + Tests that the agent picks IxP, authors the extraction node, and wires + three downstream scripts that consume the extraction result. + + Note on port shape: per references/plugins/ixp/impl.md, IxP exposes a single + output port `success` plus an `error` output variable in `$vars.{nodeId}`. + The "extracted / needsValidation / error" handles map onto that shape via + fan-out from the success port and per-script use of + `$vars.{ixpNode}.output.ExtractionResult.ResultsDocument.Fields[…]` (the + canonical access path) vs `$vars.{ixpNode}.error`. Assertions accept any + valid wiring that yields three named script nodes plus a passing flow + validate. + + Validate-only: no `uip maestro flow debug`. + +tags: [uipath-maestro-flow, smoke, lifecycle:generate, shape:multi-node, node:ixp] + +task_timeout: 600 + +agent: + type: claude-code + permission_mode: acceptEdits + max_turns: 40 + turn_timeout: 600 + +sandbox: + driver: tempdir + python: {} + +initial_prompt: | + Create a flow called "ReceiptIntake" with a manual trigger, an extractor + node, and add three script nodes named `postReceipt`, `sendToValidation`, + and `logError`. Wire all three scripts to consume the extraction result — + two off the success output, one off the error output. Document is in the + storage bucket named: ReceiptBucket. Model used: Receipt Model. Each of + the three script nodes logs OK when it receives its respective extraction + result. + + Important: + - The `uip` CLI is already available. + - Use `--output json` on all uip commands. + - Work inside the stated working directory — do NOT `cd` to a parent. + - Do NOT run `uip maestro flow debug` or `uip maestro flow deploy`. + - Run `uip maestro flow validate` at most ONCE. If it fails, stop. + - Do NOT iterate on validation errors. + +success_criteria: + - type: skill_triggered + expected: "yes" + skill_name: "uipath:uipath-maestro-flow" + description: "Agent invoked the uipath-maestro-flow Skill" + weight: 2.0 + pass_threshold: 1.0 + + - type: command_executed + description: "Agent searched the registry for IxP node types" + tool_name: "Bash" + command_pattern: 'uip\s+maestro\s+flow\s+registry\s+(search|list|get)\b[^\n]*ixp' + min_count: 1 + weight: 2.0 + pass_threshold: 1.0 + + - type: command_executed + description: "Agent validated the .flow file" + tool_name: "Bash" + command_pattern: 'uip\s+(maestro\s+)?flow\s+validate' + min_count: 1 + weight: 1.5 + pass_threshold: 1.0 + + - type: run_command + description: "uip maestro flow validate passes on ReceiptIntake.flow" + command: "uip maestro flow validate ReceiptIntake/ReceiptIntake/ReceiptIntake.flow --output json" + timeout: 30 + expected_exit_code: 0 + weight: 3.0 + pass_threshold: 1.0 + + - type: run_command + description: "Generated flow contains an IxP extraction node or core.logic.mock fallback (per references/plugins/ixp/impl.md)" + command: 'grep -rqE "\"type\"[[:space:]]*:[[:space:]]*\"(uipath\.ixp|core\.logic\.mock)" --include="*.flow" .' + expected_exit_code: 0 + weight: 2.0 + pass_threshold: 1.0 + + - type: file_contains + description: "All three script nodes are present by name" + path: "ReceiptIntake/ReceiptIntake/ReceiptIntake.flow" + includes: + - "postReceipt" + - "sendToValidation" + - "logError" + weight: 2.5 + pass_threshold: 1.0 + + - type: file_contains + description: "Scripts reference the extraction output via $vars expression" + path: "ReceiptIntake/ReceiptIntake/ReceiptIntake.flow" + includes: + - "$vars." + weight: 1.5 + pass_threshold: 1.0