Global assistant: send a full workflow YAML only for multistep changes

[hanna-paasivirta]

DO NOT MERGE

Until OpenFn/lightning#4888 is in lightning prod

Short Description

This changes the Global Assistant payload to always return a single full workflow YAML, instead of a YAML plus separate job code. This lets one conversation turn apply code changes across several steps.

This is a breaking change in the Global Assistant. It must be coordinated with Lightning: OpenFn/lightning#4888.

The PR also bundles a series of routing, tool, and prompt improvements to the Global Assistant.

Fixes #507.

Implementation Details

Payload

• Responses now carry one workflow_yaml attachment with the full workflow, with any generated job code already stitched in. There is no separate job-code artifact.
• A single source of truth lets the planner change structure and several job bodies in the same turn, then return one coherent YAML.

Planner

• Adaptor selection is delegated to the workflow subagent. The planner describes which systems to connect and no longer names adaptors. It had no adaptor list and was picking the wrong ones.
• "Add a step" requests are split by intent. A vague request adds the step only and asks what it should do. A specific request also fills in the code via the job subagent.
• Added self-correction. If a subagent reply shows the wrong tool was used (for example it cannot see job code), the planner re-routes instead of passing the message to the user.
• Returned history now uses plain string content, matching the direct routes, instead of raw tool-call blocks. This keeps all three agents consistent and removes a latent replay crash.
• Raised the max tool calls from 10 to 25 so larger workflows are not cut off mid-build.

Router

• Clarified the subagent roles. Structural edits go to the workflow agent, code goes to the job agent, and anything needing both, multiple steps, or another step as a reference goes to the planner. The job agent only ever sees one step, so requests that read or touch other steps must use the planner.
• "What does my workflow do" now routes on whether the steps contain code. It goes to the planner when they do, so the real code can be read, and to the workflow agent when they are empty, which is faster.

Docs and tests

• Updated the global_chat README and payload spec to match the new behaviour.
• Added unit tests for the planner, the router, and the empty-job fix.
• Added acceptance tests for the user scenarios involving job code changes across several steps Global chat: Applying job code changes across several steps #507

AI Usage

Please disclose how you've used AI in this work (it's cool, we just want to know!):

• Code generation (copilot but not intellisense)
• Learning or fact checking
• Strategy / design
• Optimisation / refactoring
• Translation / spellchecking / doc gen
• Other
• I have not used AI

You can read more details in our Responsible AI Policy

[hanna-paasivirta]

@josephjclark Let us know if you want @elias-ba to review this

[josephjclark]

Yes, this is a breaking change and is incompatible with older lightning versions.

But we can release this as soon as OpenFn/lightning#4888 is live