feat: mature the taskflow grammar (multi-model, typed outputs, conditionals, tooling, observability)

.gitignore

@@ -183,3 +183,6 @@ config.yaml
 
 #database
 *.db
+
+# memcache dictionary_file state written by example taskflows at runtime
+*_taskflow/

README.md

@@ -24,7 +24,7 @@ You can find a detailed overview of the taskflow grammar [here](doc/GRAMMAR.md)
 ```
 ┌─────────────────────────────────────────────────────┐
 │                   CLI (cli.py)                      │
-│  Typer-based entry point: -p, -t, -l, -g, -m, --resume│
+│  Typer-based entry point: -p, -t, -l, -g, -m, --resume, --lint│
 └─────────────────────┬───────────────────────────────┘
                       │
 ┌─────────────────────▼───────────────────────────────┐
@@ -160,6 +160,20 @@ Failed tasks are automatically retried up to 3 times with increasing backoff
 before the session is saved. Session checkpoints are stored in the
 platform-specific application data directory.
 
+### Run Manifest
+
+Every run produces a machine-readable manifest summarising what happened:
+per-task status (ok / failed / skipped), the models each task ran against,
+timing, and the named `outputs` each task produced (including per-model fan-in
+records for multi-model tasks). It contains no endpoints or tokens.
+
+The manifest is written to a run-scoped artifacts directory when a run finishes
+or fails, and can be printed for any session by ID:
+
+```bash
+python -m seclab_taskflow_agent --manifest abc123def456
+```
+
 ### Error Output
 
 By default, errors are shown as concise one-line messages. Use `--debug` (or
@@ -174,6 +188,30 @@ Error: [BadRequestError] model 'foo' not found
 python -m seclab_taskflow_agent --debug -t examples.taskflows.echo
 ```
 
+### Linting and Schema
+
+Taskflows can be validated offline, without making any model calls, using
+`--lint`. This resolves the taskflow and every document it references
+(personalities, toolboxes, model configs, reusable taskflows), checks model
+names against the model config, validates prompt/`over` template syntax, and
+reports unknown fields (likely typos):
+
+```bash
+# Validate a taskflow and its references
+python -m seclab_taskflow_agent --lint -t examples.taskflows.echo
+
+# Treat unknown fields as errors (not just warnings)
+python -m seclab_taskflow_agent --lint --strict -t examples.taskflows.echo
+```
+
+`--lint` exits non-zero if any errors are found, so it can gate CI. The JSON
+Schema for each grammar document type can be printed with `--schema` for editor
+integration and external validation:
+
+```bash
+python -m seclab_taskflow_agent --schema
+```
+
 ### MCP Environment Denylist
 
 By default, MCP server subprocesses inherit the parent environment. To prevent

doc/GRAMMAR.md

@@ -103,6 +103,86 @@ Parameters to the model can also be specified in the task using the `model_setti
 
 If `model_settings` is absent, then the model parameters will fall back to either the default or the ones supplied in a `model_config`. However, any parameters supplied in the task will override those that are set in the `model_config`.
 
+### Multiple Models (multi-model tasks)
+
+A task can be run against several models at once using the `models` field. Each
+model runs the task in parallel and its output is streamed as its own labelled
+block, which makes it easy to compare how different models respond to the same
+prompt (for example when evaluating an audit prompt across model families).
+
+The simplest form is a list of logical model names (resolved through the
+`model_config` exactly like the singular `model` field):
+
+```yaml
+  - task:
+      models: [gpt_default, claude_native, gpt_responses]
+      agents:
+        - seclab_taskflow_agent.personalities.c_auditer
+      user_prompt: |
+        Audit this function for memory safety issues.
+```
+
+Each entry may also be a map with its own `model_settings`, so different models
+can use different parameters in the same task:
+
+```yaml
+  - task:
+      models:
+        - model: gpt_default
+          model_settings:
+            temperature: 0.2
+        - model: claude_native
+          model_settings:
+            reasoning:
+              effort: high
+      agents:
+        - seclab_taskflow_agent.personalities.c_auditer
+      user_prompt: |
+        Audit this function for memory safety issues.
+```
+
+Notes and semantics:
+
+- `model` (singular) and `models` (plural) are mutually exclusive. `model: x`
+  is exactly equivalent to `models: [x]`; existing single-model taskflows are
+  unaffected.
+- Per-entry `model_settings` support the same engine keys as `model_config`
+  (`api_type`, `endpoint`, `token`, `backend`), so different models may run on
+  different backends within one task.
+- `completion` controls when the task counts as complete across its fan-out
+  branches: `all` (default, every branch must succeed) or `any` (one branch
+  succeeding is enough). This is what `must_complete` checks against.
+- `model_concurrency` caps how many branches run at once for a multi-model
+  task (default `0` runs all models in parallel):
+
+```yaml
+  - task:
+      models: [m1, m2, m3, m4]
+      model_concurrency: 2
+      completion: any
+      agents:
+        - seclab_taskflow_agent.personalities.assistant
+      user_prompt: |
+        ...
+```
+
+- Multi-model output is buffered per model and flushed as a labelled block when
+  each model finishes, so streams from different models do not interleave.
+- `models` can be combined with `repeat_prompt`: the task runs the cross
+  product of items x models concurrently, bounded by `model_concurrency`
+  (default: all branches at once). Each branch is streamed as its own block
+  labelled `<model> [item <n>]`.
+- Multi-model tool results are not threaded into the implicit last-tool-result
+  channel that the next task's `repeat_prompt` reads (that stays deterministic).
+  To consume multi-model results downstream, give the task an `id`: each
+  branch's final result is aggregated into `outputs.<id>` as a list of records
+  `{"model": ..., "item": ..., "result": ...}` (see "Typed named outputs").
+- The inline `outputs` schema is applied per branch on multi-model tasks: each
+  branch's result is validated against the schema and stored as the
+  `result` of its fan-in record. A branch whose result violates the schema is
+  treated as a failed branch, which the `completion` policy (`all`/`any`) then
+  reduces like any other branch failure (see "Typed named outputs").
+
 ### Completion Requirement
 
 Tasks can be marked as requiring completion, if a required task fails, the taskflow will abort. This defaults to false.
@@ -118,6 +198,69 @@ Example:
         ...
 ```
 
+### Conditional execution (`if`)
+
+A task can be gated with a GitHub-Actions-style `if` condition: a Jinja
+expression evaluated against the template context (`globals`, `inputs`, and
+prior tasks' `outputs`). When it evaluates falsy the task is skipped (recorded
+as skipped and not run); otherwise it runs normally.
+
+```yaml
+  - task:
+      id: audit
+      agents: [seclab_taskflow_agent.personalities.c_auditer]
+      user_prompt: |
+        Audit this code and report findings as JSON: {"findings": [...]}
+      outputs:
+        type: object
+        properties:
+          findings:
+            type: array
+        required: [findings]
+  - task:
+      # only remediate when the audit actually found something
+      if: "outputs.audit.findings | length > 0"
+      agents: [seclab_taskflow_agent.personalities.assistant]
+      user_prompt: |
+        Propose fixes for: {{ outputs.audit.findings }}
+```
+
+Notes:
+
+- The expression may be written bare (`globals.mode == 'deep'`) or wrapped in
+  `{{ ... }}`. Standard truthiness applies (empty list/string/`0`/`false` are
+  falsy).
+- Referencing a name that does not exist (for example an output from a task that
+  has not run) is treated as falsy, so the task is skipped rather than failing,
+  matching GitHub Actions semantics. To be explicit you can still guard with
+  `is defined`, e.g. `if: "outputs.audit is defined and outputs.audit.findings"`.
+- `if` composes with everything else: a skipped task does not run its agents,
+  fan out over `models`, or capture outputs.
+
+### Conditionals and loops inside a prompt
+
+Because prompts are rendered with Jinja2, you can use `{% if %}` / `{% for %}`
+directly inside a `user_prompt`:
+
+```yaml
+  - task:
+      agents: [seclab_taskflow_agent.personalities.c_auditer]
+      user_prompt: |
+        {% if globals.mode == 'deep' %}
+        Perform a DEEP audit of {{ globals.target }}.
+        {% else %}
+        Do a quick scan of {{ globals.target }}.
+        {% endif %}
+        {% for area in globals.focus %}
+        - pay attention to {{ area }}
+        {% endfor %}
+```
+
+Unlike the task-level `if` condition (which treats undefined names as falsy and
+skips the task), undefined variables inside a prompt raise, to catch typos. Use
+`is defined` or the `default` filter for optional data:
+`{{ globals.note | default('') }}`.
+
 ### Running templated tasks in a loop
 
 Often we may want to iterate through the same tasks with different inputs. For example, we may want to fetch all the functions from a code base and then analyze each of the functions. This can be done using two consecutive tasks and with the help of the `repeat_prompt` field. 
@@ -259,6 +402,92 @@ Example:
         - seclab_taskflow_agent.toolboxes.codeql
 ```
 
+### Typed named outputs
+
+By default, data flows between tasks implicitly: `repeat_prompt` consumes the
+*last tool result* of the previous task. That is positional (whichever tool
+fired last) and untyped. A task can instead publish a **named, typed output**
+that later tasks consume by name.
+
+Three fields drive this:
+
+- `id` names a task, exposing its output to later tasks as `outputs.<id>`. The
+  shape depends on whether the task fans out:
+  - A plain task (single model, no `repeat_prompt`) publishes its single
+    produced value (its final tool result).
+  - A task that fans out (`repeat_prompt`, multiple `models`, or their cross
+    product) publishes a per-branch fan-in list of
+    `{"model": <label>, "item": <index>, "result": <value>}` records, one per
+    branch. This is uniform across the item axis and the model axis, so a
+    single-model `repeat_prompt` and a multi-model task capture the same way.
+- `outputs` declares an inline JSON Schema (Draft 2020-12). When present, the
+  task's value is validated against it before being stored. Validation is strict
+  and does not coerce, so a value whose types do not match the contract is a
+  failure. On a fan-out task the schema is applied to each branch's `result`
+  (a violation is a failed branch under the `completion` policy); on a plain
+  task it is applied to the single value (a violation is a hard failure). A
+  malformed schema is rejected when the taskflow is loaded, before any model
+  calls are made.
+- `over` is an explicit iterable selector for `repeat_prompt`: a Jinja
+  expression evaluated against the template context (so it yields a real list,
+  not a re-parsed string).
+
+Example: one task produces a typed list of functions, the next analyses each.
+
+```yaml
+  - task:
+      id: list_functions
+      agents: [seclab_taskflow_agent.personalities.assistant]
+      user_prompt: |
+        List all functions as JSON: {"functions": [{"name": ..., "body": ...}]}
+      outputs:
+        type: object
+        properties:
+          functions:
+            type: array
+            items:
+              type: object
+              properties:
+                name: {type: string}
+                body: {type: string}
+              required: [name, body]
+        required: [functions]
+  - task:
+      repeat_prompt: true
+      over: "outputs.list_functions.functions"
+      agents: [seclab_taskflow_agent.personalities.c_auditer]
+      user_prompt: |
+        Analyze function {{ result.name }}:
+        {{ result.body }}
+```
+
+The `outputs` schema is a standard JSON Schema (Draft 2020-12), authored inline
+in YAML, so the full vocabulary is available:
+
+- Types via `type` (`object`, `array`, `string`, `integer`, `number`,
+  `boolean`, `null`), with `properties`/`required` for objects and `items` for
+  arrays.
+- `enum`/`const` for fixed value sets, and constraints such as `minimum`,
+  `maximum`, `minLength`, and `pattern`.
+- Objects with dynamic keys via `additionalProperties`, and strictness via
+  `additionalProperties: false`.
+- Unions (`anyOf`/`oneOf`), and `$ref`/`$defs` to reuse a shape across fields.
+
+Because validation is strict (no coercion), have the task emit JSON whose types
+already match, e.g. the integer `7` rather than the string `"7"`.
+
+Notes:
+
+- `outputs.<id>` is a template namespace alongside `globals`, `inputs`, and
+  the per-iteration `result`. Keys named after dict methods (`items`, `keys`,
+  `values`, ...) resolve to your data, not the method.
+- Without `id`/`outputs`/`over`, the implicit last-tool-result `repeat_prompt`
+  behaviour is unchanged.
+- Implicit carry-over (the next task's `repeat_prompt` reading the previous
+  task's last tool result) is fed by single-model tasks only. A multi-model
+  task does not feed it: there is no single "last" result across models, so a
+  downstream task must consume its output by name via `id`/`over`.
+
 ### Toolboxes / MCP Servers
 
 Toolboxes are MCP server configurations. They can be defined at the Agent level or overridden at the task level. These MCP servers are started and made available to the Agents in the Agents list during a Task. The `toolboxes` field should contain a list of files for the `toolboxes` that are available for the task:

examples/model_configs/anthropic_sdk.yaml

@@ -0,0 +1,16 @@
+# SPDX-FileCopyrightText: GitHub, Inc.
+# SPDX-License-Identifier: MIT
+
+# Example model_config selecting the native Anthropic Messages backend
+# (anthropic_sdk / api_type: messages). Swap the model id for whatever
+# Claude model your endpoint exposes.
+
+seclab-taskflow-agent:
+  version: "1.0"
+  filetype: model_config
+models:
+  claude: claude-sonnet-4.5
+model_settings:
+  claude:
+    backend: anthropic_sdk
+    api_type: messages

examples/model_configs/copilot_sdk.yaml

@@ -0,0 +1,15 @@
+# SPDX-FileCopyrightText: GitHub, Inc.
+# SPDX-License-Identifier: MIT
+
+# Example model_config selecting the Copilot SDK backend (copilot_sdk),
+# which drives the Copilot CLI session protocol. Swap the model id for
+# whatever your Copilot subscription exposes.
+
+seclab-taskflow-agent:
+  version: "1.0"
+  filetype: model_config
+models:
+  cop: gpt-5.4-mini
+model_settings:
+  cop:
+    backend: copilot_sdk

examples/model_configs/multi_model.yaml

@@ -0,0 +1,16 @@
+# SPDX-FileCopyrightText: GitHub, Inc.
+# SPDX-License-Identifier: MIT
+
+# Example model_config for multi-model tasks: two logical models that a
+# single task can fan out across in parallel. Swap the provider IDs for
+# whatever your endpoint exposes.
+
+seclab-taskflow-agent:
+  version: "1.0"
+  filetype: model_config
+models:
+  gpt_fast: gpt-5.4-mini
+  gpt_alt: gpt-4.1
+model_settings:
+  gpt_fast:
+    api_type: responses

examples/taskflows/CVE-2023-2283.yaml

@@ -18,7 +18,7 @@ taskflow:
       toolboxes:
         - seclab_taskflow_agent.toolboxes.memcache
   - task:
-      model: gpt_latest
+      model: gpt_default
       must_complete: false
       agents:
         - seclab_taskflow_agent.personalities.c_auditer