feat: mature the taskflow grammar (multi-model, typed outputs, conditionals, tooling, observability) by anticomputer · Pull Request #272 · GitHubSecurityLab/seclab-taskflow-agent

anticomputer · 2026-07-02T19:57:32Z

Summary

Matures the YAML taskflow grammar into a more complete declarative agent-workflow language: first-class multi-model execution, typed named object passing between tasks, GitHub-Actions-style conditional execution, offline authoring/validation tooling, and a structured run manifest with token accounting. Every change is additive; existing taskflows, personalities, toolboxes, model_configs, and prompts run unchanged.

Grammar reference is updated in doc/GRAMMAR.md.

What's new

Multi-model tasks

A task can declare models: [a, b, ...] (bare names or {model, model_settings} maps) and runs against each in parallel, with per-model labelled output streams.
model: x is exactly equivalent to models: [x]; single-model tasks are unchanged.
model_concurrency bounds how many model branches run at once.
completion selects how per-branch success reduces to task success (all/any).
The same model may be listed more than once (e.g. distinct labels mapping to one provider id) for N-sample / self-consistency runs.

Typed named outputs and neutral cross-task I/O

id names a task's output so later tasks consume it by name as outputs.<id>.
outputs declares an inline JSON Schema (Draft 2020-12) that strictly validates the value (no coercion) before it is stored, giving enums, constraints, unions, dynamic-key objects, and $ref reuse.
over selects a named iterable for repeat_prompt instead of the implicit last-tool-result.
On multi-model tasks the outputs schema is applied per branch: each branch's result is validated/coerced against it, and a branch that violates the schema is treated as a failed branch under the task's completion policy (all/any).
Replaces the previous fragile "re-encode each backend's tool result into openai's wire format" path with a neutral result representation shared by all adapters.

Multi-model x repeat_prompt

Multi-model and repeat_prompt compose as a cross product; each branch's final result is aggregated (fan-in) into outputs.<id> as a list of {"model", "item", "result"} records.
Unified capture: every task fans out into isolated per-branch sinks, and outputs.<id> is that fan-in list whenever the task fans out (repeat_prompt, multiple models, or the cross product) or the single value for a plain task. This fixes two prior asymmetries: a single-model repeat_prompt can now fan in per-item results (not just the last), and an async single-model repeat no longer races the shared store. Single-model results are still projected back into the shared store for the legacy implicit carry-over; multi-model tasks don't feed it (consume via id/over).

Conditional execution (`if`)

A task can be gated with a GitHub-Actions-style if: Jinja expression over globals / inputs / prior outputs. Falsy skips the task (recorded as skipped); referencing not-yet-existing context is treated as falsy; a malformed expression is a hard, resumable failure.
Composes with -g KEY=VALUE CLI global overrides.

Authoring and validation tooling

--lint validates a taskflow and its references offline without running it; --strict promotes unknown fields from warnings to errors.
--schema prints the JSON Schema for each grammar document type.
A corpus-validate test gate keeps every shipped example valid against the grammar.

Observability: run manifest

--manifest <session> prints a stable, machine-readable run summary: per-task status, models, timing, and token usage, plus named outputs. Written to artifacts/<session>/manifest.json on completion. Contains no endpoints or secrets.

Anthropic backend prompt caching

The anthropic_sdk backend now marks the stable prefix (tool definitions + system prompt) with a block-level cache_control breakpoint instead of the top-level request param. CAPI's native /v1/messages surface is a passthrough whose Anthropic endpoints reject a top-level cache_control for some models while accepting the block-level form on every model, so this caches where the upstream supports it and is a no-op otherwise. prompt_caching: false opts out.

Uniform token-usage reporting

All three adapters (openai_agents, copilot_sdk, anthropic_sdk) emit a neutral TokenUsage stream event; the runner logs it and stores per-task and run-level usage, including prompt-cache reads/writes, in the manifest.
Provider accounting differs and is documented: input_tokens is inclusive of cached tokens on the chat_completions surface (openai/copilot) but disjoint from cache reads/writes on the native Anthropic Messages surface.

Backwards compatibility

The model: field, existing repeat_prompt, toolboxes, personalities, and model_configs are unchanged. A one-element models: list takes the identical single-model code path.
Behavior change (local state only): the session checkpoint schema replaced last_tool_results with a result_snapshot of the neutral result store. A session that was checkpointed by an older version and resumed after upgrading loses its carry-over and, for a repeat_prompt task, fails with a clear "No last tool result available for repeat_prompt" error. In-flight sessions from older versions should be restarted, not resumed. Checkpoints are ephemeral local state, so this does not affect the grammar contract.

Validation

Full unit suite and CI-parity lint pass.
Live-verified end-to-end across all three backends (tool calls, multi-turn, repeat_prompt, multi-model fan-in, conditional flip via -g, and manifest token usage including real cache reads/writes).
New example flows backend_anthropic_sdk and backend_copilot_sdk double as functionality checks and are covered by the offline corpus-validate gate.

Planned follow-on (TODO in this PR)

Now that the outputs contract is standard JSON Schema, the schema-driven pieces are straightforward. Intended to land in this PR:

Schema-driven generation: pass the JSON Schema to each backend's native structured-output surface (OpenAI response_format, Anthropic tool input_schema; copilot_sdk falls back to validate-only) so the model is constrained to emit conforming output instead of relying on prompt prose.
Retry-to-repair: on strict-validation failure, re-prompt the model with the validation error (bounded) so strict validation is robust rather than brittle. Knob: outputs_retries.
Reusable schemas: a shared-schema mechanism (top-level schemas: / document type) so a contract can be defined once and referenced across tasks.
format policy: decide whether to enforce JSON Schema format (jsonschema does not by default) and enable a FormatChecker if so.
Multi-model fan-in observability: surface per-branch validation errors (not just result: null) and add a consensus/merge helper over per-model typed results.
Friendlier validation errors: wrap jsonschema pointer-path messages into field-oriented messages for YAML authors.

Deferred (not planned here): an opt-in coercing mode (strict validation is intentional; prefer generation + repair). No migration burden: outputs: is introduced by this PR and ships as JSON Schema from the start, so there is nothing to forward-port.

Add a `models:` field to a task so one prompt can fan out across several models concurrently, each streamed as its own labelled output block. This is the first milestone (M1) of the grammar maturity effort. Grammar (models.py): - New `ModelEntry` submodel; `models: list[str | {model, model_settings}]` on TaskDefinition, coerced from bare names or override maps. - `completion: all|any` policy and `model_concurrency` cap. - Validators: `model` xor `models`; reject multi-entry `models` with `repeat_prompt` (deferred to a later milestone); non-negative concurrency. - `effective_model_entries()` normalises single- and multi-model tasks. Engine (runner.py): - Refactor resolution into `_resolve_one_model` + `_resolve_task_models` returning `ResolvedModel` per entry (mixed backends supported). - Extract the fan-out matrix into a testable `_fan_out_deploys` helper that owns the prompt x model cross product, bounded concurrency, exception isolation, and the completion policy. - Isolate multi-model tool-result capture so concurrent models do not corrupt the shared repeat_prompt/session channel. Single-model path is unchanged. Output (render_utils.py): - `flush_async_output` takes an optional `label` so each model's buffered block is tagged with its model name. Docs/examples: - "Multiple Models" section in doc/GRAMMAR.md. - examples/taskflows/example_multi_model.yaml + model_configs/multi_model.yaml. Tests: +30 (grammar, resolution fan-out, `_fan_out_deploys` matrix/ concurrency/completion, output labelling). Full suite 321 passed, 2 skipped; CI-parity lint clean. Backwards compatible: `model:` stays valid and equals `models: [x]`. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

…tion Replaces the naive cross-task data channel and adds typed, named object passing between tasks (M2), built on a redesigned I/O foundation that also advances M0. Foundation (removes the ugly, multi-SDK-fragile channel): - results.py: neutral ToolResult; normalize_openai_tool_output handles all three openai-agents MCP serialisation shapes; a single decode_tool_result; and a per-run ResultStore (ordered results + named outputs, snapshot/restore) replacing the shared last_mcp_tool_results list. - _stream.py: copilot/anthropic tool results now record a neutral ToolResult instead of reconstructing openai-agents' faked {"text": ...} JSON envelope, so no backend fakes another's wire format. - runner.py: openai on_tool_end normalises into the store; a record_tool_result sink threads through deploy_task_agents/drive_backend_stream for the other backends; shell tasks record a ToolResult; multi-model branches stay isolated. - session.py: persists a ResultStore snapshot (result_snapshot) for resume. Typed named outputs (M2): - Grammar adds id, outputs (inline schema), and over to a task. - output_schema.py compiles an inline outputs schema into a Pydantic model (scalars/optionals/lists/nested objects/lists-of-objects), validated at load. - Producing tasks capture their final result under outputs.<id>, validated against the schema when present; downstream tasks consume it by name. - over: is an explicit, typed repeat_prompt iterable selector (evaluated to a real object), replacing the last-tool-result double-JSON heuristic. - template_utils: outputs.<id> namespace + evaluate_expression; a data-first Jinja environment so keys named items/keys/values resolve to data. Backwards compatible for external users: model:, {{ result }}, and implicit repeat_prompt are unchanged (legacy repeat_prompt verified live). Full suite 379 passed, 2 skipped; CI-parity lint clean; live typed-outputs and legacy-repeat_prompt CAPI runs verified. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

…e gate Adds robustness tooling that catches misconfiguration before any model call (M0 hardening), which also de-risks forward-porting first-party taskflows. - linting.py: lint_taskflow() validates a taskflow and every document it references (personalities, toolboxes, model configs, reusable taskflows) entirely offline. Reports unknown fields (typos; warnings, or errors under --strict), missing references, logical model names absent from the resolved model_config, and malformed Jinja in prompts / over expressions. Pure and fully unit-tested. - cli.py: --lint (with --strict) validates a taskflow and exits non-zero on errors so it can gate CI; --schema prints JSON Schema per grammar document type for editor integration. - tests/test_examples_validate.py: corpus gate that validates every bundled and example grammar document, and lints every example taskflow (no errors), on each test run - turning an accidentally broken example into an instant local failure. - README: documents --lint/--strict/--schema. Full suite 455 passed, 2 skipped; CI-parity lint clean. The corpus gate and linter tests run in CI via `hatch test`. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Replace the module-global async_output dict and lock in render_utils with an OutputRouter class that owns the buffered-output state. The active router is resolved from a ContextVar and set per run in run_main, so buffered async and multi-model output streams are isolated per run instead of sharing process-global state, and the buffering behaviour becomes directly testable. The free-function API (render_model_output / flush_async_output) is unchanged, so existing call sites are untouched and delegate to the current-context router. Output strings and buffering semantics are preserved exactly. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Allow a task to combine `models` with `repeat_prompt`, running the item x model matrix concurrently. Each cell streams as its own labelled block ("<model> [item <n>]"), and concurrency is bounded by model_concurrency * async_limit. Give a multi-model task an `id` to fan in results: each branch's final tool result is aggregated into outputs.<id> as a list of {model, item, result} records for downstream tasks to consume. Branches capture into private sinks so the shared result store stays deterministic for the next task's implicit repeat_prompt. The inline `outputs` schema is not accepted on multi-model tasks (the aggregate is a list of records); `id` provides the fan-in instead. Execution is restructured around explicit per-branch descriptors, and the fan-in aggregation is a pure, unit-tested helper. Adds unit tests (grammar combinations, aggregator), a run_main integration test that drives the full matrix through a patched deploy layer and asserts the deploy set, per-branch labels, and outputs.<id> fan-in, an example taskflow, and GRAMMAR docs. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

…fix concurrency knob - Output capture: a task that declares an `outputs` schema whose produced value violates it (or a task with no capturable result) previously raised straight out of the run loop, bypassing failure handling. Now a capture failure marks the session failed and prints the same session-saved / resume hint as other task failures before re-raising. - Linter: `over` is evaluated at runtime as an expression, so lint now checks it with compile_expression instead of from_string. A bare expression like `globals.items` (and malformed ones) are now correctly validated. - model_concurrency: a multi-model task's branch concurrency is now bounded by model_concurrency alone (previously model_concurrency * async_limit in the cross-product path, which ignored the intended per-model cap). The name now matches the behaviour, covered by a concurrency-bound test. - Docs: `completion` governs all fan-out branches (prompts x models), not only models; corrected the cross-product concurrency wording. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Add an `if` field to a task: a Jinja expression evaluated against the template context (globals / inputs / prior tasks' outputs). When it evaluates falsy the task is recorded as skipped and not run; otherwise it runs normally. This enables branching workflows, e.g. only remediate when a prior audit task produced findings. - models: `if_` field aliased to the YAML key `if` (a Python keyword). - runner: evaluate the condition before running the task and skip on falsy; a malformed expression fails fast with a clear error. - session: record skipped tasks (new `skipped` flag) so resume advances past them. - linting: `--lint` validates the `if` expression syntax offline. - docs: document `if` and, since prompts are Jinja-rendered, using {% if %} / {% for %} inside a user_prompt, including the StrictUndefined idioms (`is defined`, the `default` filter) for optional data. - Adds grammar, linter, and run_main integration tests (skipped vs run, and conditioning on a prior task's output) plus an example taskflow. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

…and status Turn the session checkpoint into a machine-readable run manifest for auditing. - session: CompletedTask gains the models a task ran against, its wall-clock duration, and the existing skipped flag; the session gains finished_at and a status property. TaskflowSession.manifest() returns a curated, token-free summary (per-task status/models/timing plus the named outputs, which include per-model fan-in records). It is written to a run-scoped artifacts/<id>/manifest.json on finish or failure. - runner: time each task and record the resolved models it ran against. - cli: `--manifest <session_id>` prints a session's manifest. - README: document the run manifest. Adds session-level, CLI, and run_main integration tests, verified with a live run. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

…pipeline example - Add CliRunner tests for command routing (--schema / --lint / --manifest and the usage/mutual-exclusivity errors). - Cover the top-level-JSON-string normalization path and the {type: X} scalar schema form. - Add an example taskflow composing a shell-produced typed output, an `if` gate on it, a multi-model fan-in task, and a skipped task; it lints clean and is exercised by the corpus gate. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

…anifest write - if-conditions now follow GitHub-Actions semantics: referencing context that does not exist yet (e.g. outputs from a skipped upstream task, including nested access like outputs.audit.findings) is treated as falsy and the task is skipped, instead of raising UndefinedError and aborting the whole run (which also left the session stuck, re-aborting on resume). A genuinely malformed if expression is still a hard failure, now routed through the same session-saved / resume messaging as other task failures. - Manifest writing is best-effort and isolated: a failure to write the audit artifact is logged and swallowed so it can never change a run's outcome or mask a task failure / suppress the resume hint. Adds tests for skipping on missing top-level and nested context (including an upstream-skip -> downstream-if chain) and for manifest-write failure isolation. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Anthropic backend: mark the stable prefix (tool definitions + system prompt) with a block-level cache_control breakpoint instead of the top-level request cache_control param. CAPI's native /v1/messages surface passes the request body through to the upstream, and its Anthropic model endpoints reject a top-level cache_control ("Extra inputs are not permitted") while accepting the block-level form on every model. This caches where the upstream supports it and is ignored otherwise, so a repeated templated prompt reuses its cached system+tools prefix. Add a neutral TokenUsage stream event emitted by all three adapters (anthropic_sdk, copilot_sdk, openai_agents). The runner logs each event and stores per-task and run-level token usage, including prompt-cache reads and writes, in the session manifest. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Add two end-to-end example taskflows (plus their model configs) that drive the native Anthropic Messages backend and the Copilot SDK backend through a tool-calling agent loop and a repeat_prompt fan-out. They double as a quick functionality check for each backend and surface per-task token usage, including prompt-cache activity, in the run manifest. Ignore the *_taskflow/ memcache state directories these flows write at runtime. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Document that TokenUsage.input_tokens is inclusive of cache_read_tokens on the chat_completions surface (openai_agents, copilot_sdk) but disjoint from cache reads/writes on the native Anthropic Messages surface, with a note at each adapter's emit site. Document the manifest's usage-accounting contract: usage is summed over completed tasks, so a failed (unrecorded, resumable) task is named in error but not itemized. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Copilot

Pull request overview

This PR substantially extends the Python taskflow runner and YAML grammar to support multi-model task execution, typed named outputs (id/outputs/over), GitHub-Actions-style conditional execution (if), offline validation tooling (--lint, --schema), and a structured run manifest including uniform token-usage accounting across backends.

Changes:

Add a neutral, backend-agnostic result store (ToolResult/ResultStore) and wire it through the runner for repeat/typed outputs and multi-model fan-in.
Implement multi-model fan-out with concurrency bounds + completion policies, plus conditional task gating (if) and per-model buffered output labeling.
Add offline tooling and observability: taskflow linter, schema dump, run manifest artifact, and adapter-neutral token usage events.

Show a summary per file

File	Description
tests/test_template_utils.py	Adds tests for `outputs.*` namespace and expression evaluation behavior.
tests/test_stream.py	Updates streaming tests for neutral `ToolResult` recording and `TokenUsage` forwarding.
tests/test_session.py	Adapts session tests to `result_snapshot` and updated `CompletedTask` fields.
tests/test_session_edge.py	Updates edge-case session persistence tests for `result_snapshot`.
tests/test_sdk_openai_adapter.py	Tests emission of neutral `TokenUsage` from OpenAI adapter.
tests/test_sdk_copilot_adapter.py	Tests translation of Copilot usage events into neutral `TokenUsage`.
tests/test_sdk_anthropic_adapter.py	Tests block-level prompt caching behavior and `TokenUsage` emission for Anthropic.
tests/test_runner.py	Expands runner unit tests for result store, over/outputs, multi-model fan-out/fan-in, schema capture.
tests/test_runner_integration.py	New integration tests for multi-model matrix execution, conditionals, and manifest writing.
tests/test_results.py	New tests for neutral result normalization/decoding, store snapshotting, and usage accumulation.
tests/test_render_utils.py	New tests for per-run output routing and labeled async/multi-model flushing.
tests/test_output_schema.py	New tests for inline `outputs` schema compilation/validation behavior.
tests/test_models.py	Adds grammar model tests for `models`, `completion`, `model_concurrency`, `id/outputs/over`, and `if`.
tests/test_manifest.py	New tests for manifest structure, status transitions, and artifact writing behavior.
tests/test_linting.py	New tests for offline linter behavior (unknown fields, refs, templates, models, strict mode).
tests/test_examples_validate.py	New “corpus gate” ensuring shipped examples validate and lint cleanly.
tests/test_cli.py	Adds tests for `--lint` and `--schema` helpers.
tests/test_cli_app.py	New Typer CLI routing tests for early-exit flags and mutual exclusivity.
src/seclab_taskflow_agent/template_utils.py	Adds data-first Jinja environment, `outputs` context, and `evaluate_expression`.
src/seclab_taskflow_agent/session.py	Adds artifacts directory, manifest generation/writing, richer `CompletedTask`, `result_snapshot`.
src/seclab_taskflow_agent/sdk/openai_agents/backend.py	Emits neutral `TokenUsage` after stream completion based on SDK usage.
src/seclab_taskflow_agent/sdk/copilot_sdk/backend.py	Emits neutral `TokenUsage` from Copilot session usage events.
src/seclab_taskflow_agent/sdk/base.py	Introduces neutral `TokenUsage` stream event type.
src/seclab_taskflow_agent/sdk/anthropic_sdk/backend.py	Switches to block-level prompt caching markers; emits neutral `TokenUsage` from Messages usage.
src/seclab_taskflow_agent/sdk/init.py	Re-exports `TokenUsage`.
src/seclab_taskflow_agent/runner.py	Core implementation of neutral store, typed outputs, `over`, conditionals, multi-model fan-out/fan-in, usage attribution.
src/seclab_taskflow_agent/results.py	New neutral result representation + store snapshot/restore + usage accumulator.
src/seclab_taskflow_agent/render_utils.py	Refactors buffered rendering into per-context `OutputRouter` with labeled flushing.
src/seclab_taskflow_agent/output_schema.py	New inline outputs schema compiler/validator (Pydantic-based).
src/seclab_taskflow_agent/models.py	Extends grammar models with multi-model entries, completion policy, `if` alias, typed outputs fields/validation.
src/seclab_taskflow_agent/linting.py	New offline linter for taskflows and references (syntax, refs, models, unknown fields).
src/seclab_taskflow_agent/cli.py	Adds `--lint`, `--strict`, `--schema`, `--manifest` flows and helpers.
src/seclab_taskflow_agent/agent.py	Persists `model` attribute on agent instance for later usage reporting.
src/seclab_taskflow_agent/_stream.py	Handles tool-end events via neutral results and forwards token usage events.
README.md	Documents manifest, linting, and schema features.
examples/taskflows/example_typed_outputs.yaml	New example demonstrating typed named outputs + `over`.
examples/taskflows/example_pipeline.yaml	New showcase pipeline combining typed outputs, conditionals, and multi-model fan-in.
examples/taskflows/example_multi_model.yaml	New example for parallel multi-model execution and completion policy.
examples/taskflows/example_multi_model_repeat.yaml	New example for multi-model × repeat cross-product execution.
examples/taskflows/example_conditional.yaml	New example demonstrating `if`-gated tasks.
examples/taskflows/backend_copilot_sdk.yaml	New end-to-end Copilot SDK backend example (tools, repeat, manifest usage).
examples/taskflows/backend_anthropic_sdk.yaml	New end-to-end Anthropic SDK backend example (tools, repeat, manifest usage).
examples/model_configs/multi_model.yaml	New example model_config for multi-model tasks.
examples/model_configs/copilot_sdk.yaml	New example model_config selecting `copilot_sdk` backend.
examples/model_configs/anthropic_sdk.yaml	New example model_config selecting `anthropic_sdk` Messages backend.
doc/GRAMMAR.md	Updates grammar reference for multi-model, typed outputs, and `if` conditionals.
.gitignore	Ignores example taskflow memcache state directories.

Review details

Tip

Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Files reviewed: 46/47 changed files
Comments generated: 3
Review effort level: Low

… del The task-level `if:` condition treats an undefined name as falsy and skips the task (the runner catches UndefinedError), so correct GRAMMAR.md which claimed it raises; also clarify that in-prompt templating still raises on undefined, unlike `if`. Remove an unnecessary `del` in a stream test helper flagged by code scanning. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Allow a typed `outputs` schema on multi-model tasks. Each branch's result is validated and coerced against the schema and stored as the `result` of its fan-in record. A branch whose result violates the schema is treated as a failed branch, so the task's `completion` policy (all/any) reduces it like any other branch failure. Removes the prior load-time restriction that rejected `outputs` on multi-model tasks. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Copilot

Review details

Files reviewed: 46/47 changed files
Comments generated: 1
Review effort level: Low

… DSL Replace the hand-rolled `outputs` type mini-language with standard JSON Schema (Draft 2020-12), validated by the already-vendored `jsonschema` library. The `outputs` field is now an inline JSON Schema, giving enums, numeric/string constraints, unions, objects with dynamic keys, `$ref` reuse, and top-level non-object contracts, and it deletes the custom schema compiler and its reserved-key parsing footguns. Validation is strict and does not coerce: a produced value whose types do not already match the contract is a failure (a violation is a hard failure for a single-model task, and a failed branch under the completion policy for a multi-model task), which surfaces malformed model output rather than silently reshaping it. The schema is checked for well-formedness at load time. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

The second task referenced `gpt_latest`, which is not defined in the taskflow's model_config (only `gpt_default` is), so it would resolve to a literal, non- existent provider id and fail at runtime. Point it at `gpt_default`. The offline linter now reports the whole example corpus clean under --strict. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Copilot

Review details

Files reviewed: 47/48 changed files
Comments generated: 8
Review effort level: Low

…, exit code - Correct stale "validated/coerced" wording in comments/docstrings/grammar docs: output-schema validation is strict and does not coerce. - Convert the remaining old-DSL `outputs` snippet in the `if` docs to JSON Schema. - Fix the `_build_prompts_to_run` docstring: a non-iterable derived value raises TypeError (from iter/list), not ValueError. - Materialize a repeat_prompt `over:` iterable before the emptiness check so a one-shot generator (e.g. Jinja map/select) is detected as empty instead of silently producing zero prompts. - Raise instead of break on a must_complete task failure so run_main propagates and the CLI exits non-zero, matching the other hard-failure paths; the session is still saved for --resume. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

…e path Every prompt task now fans out into branches (the cross product of prompts x models) that each capture into an isolated per-branch sink, regardless of which axis fanned out. After the fan-out the runner: - projects single-model branch results back into the shared store in branch order, preserving the implicit last-tool-result carry-over, run-level results, and session snapshot (now deterministic even for async repeat_prompt); multi-model tasks are excluded on purpose (no single "last" across models); - captures the named output uniformly: a plain task publishes its single validated value, and any fan-out task (repeat_prompt, multiple models, or the cross product) publishes the per-branch fan-in list [{model, item, result}]. This removes the multi_model fork in the capture layer and fixes two asymmetries: a single-model repeat_prompt can now fan in per-item results (not just the last), and an async single-model repeat no longer races the shared store. The capture model is documented in a design note above _aggregate_fanin and in doc/GRAMMAR.md. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Copilot

Review details

Files reviewed: 47/48 changed files
Comments generated: 3
Review effort level: Low

+    def getattr(self, obj: Any, attribute: str) -> Any:  # noqa: N802 - Jinja API
+        try:
+            return obj[attribute]
+        except (TypeError, LookupError):
+            pass
+        try:
+            return getattr(obj, attribute)
+        except AttributeError:
+            return self.undefined(obj=obj, name=attribute)


+        Usage accounting: token usage is reported per completed task, and the
+        run-level ``usage`` is the sum over ``completed_tasks``. A task that
+        fails is intentionally not recorded as completed (so ``--resume``
+        re-runs it); it is named in ``error`` but its partial token usage is
+        not itemized or summed here.


+                try:
+                    condition = evaluate_expression(
+                        task.if_,
+                        available_tools,
+                        globals_dict=global_variables,
+                        inputs_dict=inputs,
+                        outputs_dict=store.outputs,
+                    )
+                except jinja2.UndefinedError:
+                    condition = False
+                except jinja2.TemplateError as e:
+                    logging.error("Invalid task 'if' condition %r: %s", task.if_, e)
+                    session.mark_failed(f"Task {task_name!r} has an invalid 'if' condition: {e}")
+                    await render_model_output(
+                        f"** 🤖❗ Invalid 'if' condition: {e}\n"
+                        f"** 🤖💾 Session saved: {session.session_id}\n"
+                        f"** 🤖💡 Resume with: --resume {session.session_id}\n"
+                    )
+                    raise ValueError(f"Failed to evaluate task 'if' condition: {e}") from e
+                if not condition:


The repeat_prompt template referenced `result.url`, but the GitHub tool returns pull request objects with `html_url` (no `url`), so rendering failed under StrictUndefined. Reference `result.html_url` to match the tool output. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

anticomputer and others added 13 commits July 1, 2026 13:49

Copilot AI review requested due to automatic review settings July 2, 2026 19:57

Copilot started reviewing on behalf of anticomputer July 2, 2026 19:57 View session

github-advanced-security AI found potential problems Jul 2, 2026

View reviewed changes

Comment thread tests/test_stream.py Fixed

Copilot AI reviewed Jul 2, 2026

View reviewed changes

Comment thread src/seclab_taskflow_agent/runner.py

Comment thread src/seclab_taskflow_agent/_stream.py

Comment thread doc/GRAMMAR.md Outdated

anticomputer and others added 2 commits July 2, 2026 16:09

Copilot AI review requested due to automatic review settings July 2, 2026 21:12

Copilot started reviewing on behalf of anticomputer July 2, 2026 21:13 View session

Copilot AI reviewed Jul 2, 2026

View reviewed changes

Comment thread src/seclab_taskflow_agent/runner.py Outdated

anticomputer and others added 2 commits July 2, 2026 17:50

Copilot AI review requested due to automatic review settings July 2, 2026 21:57

Copilot started reviewing on behalf of anticomputer July 2, 2026 21:58 View session

Copilot AI reviewed Jul 2, 2026

View reviewed changes

anticomputer and others added 2 commits July 6, 2026 11:10

Copilot AI review requested due to automatic review settings July 6, 2026 16:12

Copilot started reviewing on behalf of anticomputer July 6, 2026 16:12 View session

Copilot AI reviewed Jul 6, 2026

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

feat: mature the taskflow grammar (multi-model, typed outputs, conditionals, tooling, observability)#272

feat: mature the taskflow grammar (multi-model, typed outputs, conditionals, tooling, observability)#272
anticomputer wants to merge 20 commits into
mainfrom
anticomputer/grammar-maturity

anticomputer commented Jul 2, 2026 •

edited

Loading

Uh oh!

Uh oh!

Copilot AI left a comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Copilot AI left a comment

Uh oh!

Uh oh!

Copilot AI left a comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Copilot AI left a comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Uh oh!

Conversation

anticomputer commented Jul 2, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

What's new

Multi-model tasks

Typed named outputs and neutral cross-task I/O

Multi-model x repeat_prompt

Conditional execution (if)

Authoring and validation tooling

Observability: run manifest

Anthropic backend prompt caching

Uniform token-usage reporting

Backwards compatibility

Validation

Planned follow-on (TODO in this PR)

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Review details

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Review details

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Review details

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Review details

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

anticomputer commented Jul 2, 2026 •

edited

Loading

Conditional execution (`if`)