feat: workflow state + REST options.auth, idempotent contractWrite

[chrisli30]

• refactor(rest): address cosmetic review nits on partner auth

Non-blocking follow-ups from the #654 re-review (no behavior change):

• partnerPrincipal fields lowercased (partnerID/subject) — package-private type
  that never leaves the package.

• Drop the no-op WithValidMethods on the ParseUnverified preview parse; add a
  comment that alg is enforced on the real jwt.Parse verify below.

• Remove the now-unneeded key := pk loop copy (Go 1.22+ per-iteration binding).

• Test: cover RawURLEncoding in the key-decode test (prod already accepts it).

• docs: fix stale make targets in CLAUDE.md

dev-build/dev-agg/dev-op do not exist. The local gateway is run with
make gateway (logs -> gateway.log) or make dev-gateway; operators via
make dev-operator-sepolia; full local stack via make dev-stack.

• ci: move unit test workflow back to GitHub-hosted runners

The Run Unit Tests workflow's jobs (Check Changed Files, Lint, and the
13-package Unit Test matrix) mostly finish in under a minute, so
Blacksmith's faster 4-vCPU runners have little wall-clock advantage
there while still billing at the 4-vCPU rate on every push. Revert
these jobs to ubuntu-latest, undoing #505 for this workflow only.

The Docker image publish workflows (~4-5 min builds) stay on Blacksmith
where the faster hardware genuinely pays off.

Co-Authored-By: Claude Fable 5 noreply@anthropic.com
Claude-Session: https://claude.ai/code/session_01Un7opV3NNydU4EEHD4pjAV

• docs: plan on-demand single-node action backend gaps

Analysis of the EigenLayer-AVS backend gaps for chat-driven, one-time
single-node (contractWrite) execution — the preview→confirm→execute
contract over nodes:run. Verified against the code with file:line refs:

• G1: native ETH value dropped on the real path (PackExecute hardcodes 0)
• G2: no typed tx identity / return value on real executes
• G3: pending UserOp reported as failure; nodes:run is stateless
• G4: approve+swap are non-atomic (2 UserOps), partial on-chain side effects
• G5: execute is not idempotent (retried confirm re-broadcasts)
• G6: runner salt not propagated on the nodes:run real path
• G7: no execution fee on nodes:run (pricing decision)

Scope: delegated execution (no per-tx user signature); server-side spend
policy deferred; studio owns call construction + market-order helper + UX.

• feat: correct + idempotent single-node contractWrite execution (feat: correct + idempotent single-node contractWrite execution #659)

Co-authored-by: Chris Li chris@avaprotocol.org

• fix(rest): surface contractWrite per-method receipt in nodes:run response (fix(rest): surface contractWrite per-method receipt in nodes:run response #660)

Co-authored-by: Chris Li chris@avaprotocol.org

• fix: parseUnit macro implements ethers parseUnits scaling with fractional support (fix: parseUnit macro implements ethers parseUnits scaling with fractional support #657)

• docs: plan per-workflow state + native guardian-scan node

Gateway-side response to studio's guardian-monitoring hand-off. Specifies the
one missing capability as a generic primitive: durable cross-run per-WORKFLOW
state (wfstate::...), scoped to taskId not to a user, so a user can run
any number of monitoring workflows each with isolated, cascade-deleted state.
Covers the SetIfAbsent/DeleteByPrefix storage additions (reusing IncCounter's
serializable txn), the native guardian-scan node (holds GoPlus/Moralis creds,
mints the GoPlus token, diffs via wfstate, claim-once alerts), credential
config, and answers to the studio plan's five open questions. Additive storage
(new namespace + model struct), no migration.

• docs: redesign guardian plan as composable extensions (drop bespoke node)

Replace the native guardian-scan node with two generic, reusable gateway
extensions, verified feasible against the code:

1. REST node server-side auth providers (GoPlus signed-token minted from
   macros.secrets, injected into processedHeaders; secret never in client
   JSON) — precedents: options.summarize reader, balance runner reading
   macroSecrets, the /api/notify summarizer's own outbound call.
2. client-defined per-workflow state via a state.* goja binding over
   wfstate: (client defines the JSON) — precedents: console.log
   Go-func binding, r.vm.db access, apContext AddVar injection.
   The guardian becomes a pure node composition (cron→REST→REST→customCode→
   branch→notify) whose verdict/diff logic ports studio JS ~verbatim. Additive
   storage, no migration.

• docs: formalize {{state.*}} namespace for workflow custom storage

Position state as the client-defined read/write persistent namespace parallel
to {{settings.}} (config) and {{context.}} (read-only runtime): namespace
table, snake_case sub-key conventions, and the reservedSystemVarNames
registration requirement so it's re-injected fresh and collision-safe.

• docs: guardian plan §9 — resolve client-review A-D with gateway facts

Add gateway-side commitments from the studio client review: (A) no in-place
update confirmed → guardian_ruleset configVar for central rule updates + track
an in-place UpdateWorkflow; (B) read-only create-auth relaxation via partner
assertion as the Telegram-only enrollment fix, counterfactual runner accepted;
(C) serialized + at-most-once + no-retry execution makes mark-after-send
(at-least-once) the correct security bias, not a shortcut; (D) default cadence
6h, approvals-first v1.

• docs: fold studio A/B decisions into guardian plan §9

Align gateway plan with the studio-side decisions (2026-07-15): A — accept the
logic-freeze, no UpdateWorkflow ask; structural changes handled by re-create +
silent-seed first run (no wfstate read API needed). B — web hand-off is the
enrollment path (one EIP-191 sign ever, no create-auth change); partner-
assertion relaxation parked as a future lever. Net gateway scope shrinks to the
two extensions + guardian_ruleset configVar.

• docs: guardian_ruleset must always be set (unresolved source template hard-fails)

Live run against v4.2.0 showed the verdict node fails on an unset
guardian_ruleset configVar (could not resolve variable ... in source) before
the JS runs — so it is required guardian setup, not optional-with-JS-fallback.

• feat: workflow {{state.*}} store + REST options.auth GoPlus provider (feat: workflow {{state.*}} store + REST options.auth GoPlus provider #662)

Co-authored-by: Chris Li chris@avaprotocol.org

• docs: completion checklist for feat: workflow {{state.*}} store + REST options.auth GoPlus provider #662 + docs/changes record for state+auth extensions

[claude]

Review

Reviewed the substantive Go changes (idempotent nodes:run, contractWrite value passthrough, {{state.*}} binding, GoPlus REST auth provider, ParseUnit fix, partner-auth nits). Overall this is a well-tested, carefully-commented PR — good unit test coverage for each new behavior (idempotency dedup, salt propagation, state persistence/scratch, auth header injection). Two things worth addressing before/after merge:

1. GoPlus token mint holds a write-lock across a blocking network call (concurrency/perf)

goplusAuthHeader() in core/taskengine/vm_runner_rest.go:935-995 takes goplusTokenCache.Lock() (a package-level sync.RWMutex) and only releases it via defer at function return — but the HTTP POST to mint a fresh token (goplusHTTPClient.Do(req), up to the 15s client timeout) happens while the lock is held (line ~969). Every concurrent REST-node execution across the whole engine process that requests options.auth.provider: goplus will block on this same mutex — via RLock(), which a pending Lock() will still queue behind — for the full duration of that in-flight mint, not just the goroutines that raced into the cold-cache path. Under load (e.g. several guardian-monitoring workflows on cron triggers refreshing near the same expiry boundary), this serializes otherwise-unrelated workflow executions for up to the HTTP timeout.

Consider releasing the write lock before the network call and re-acquiring briefly to publish the result (accepting a handful of duplicate concurrent mints), or collapsing concurrent mints with singleflight the same way RunNodeImmediatelyRPCIdempotent already does elsewhere in this PR.

2. Idempotency cache entries are never actively evicted (unbounded storage growth)

RunNodeImmediatelyRPCIdempotent's persisted cache (core/taskengine/run_node_immediately.go:3009-3120, key prefix idem:noderun:) only removes an expired entry when readIdempotentResponse happens to be called again with that exact cache key after the 15-minute TTL has passed (see the time.Since(...) > idempotencyTTL branch). There's no background sweep of the idem:noderun: prefix. If callers generate a fresh Idempotency-Key per action (the typical/recommended usage — e.g. a UUID per chat Confirm) rather than reusing one, that key is looked up at most once or twice and its cache entry is never read again, so it's never deleted — every real execute leaves a permanent BadgerDB record behind. Over time this is unbounded growth in the primary store.

Given BadgerDB doesn't appear to be wrapped with native per-key TTL support (storage.Storage has no SetWithTTL), consider either exposing Badger's native TTL (badger.Entry.WithTTL) for this key space so expiry is enforced by the store itself, or adding a periodic sweep over the idem:noderun: prefix to reap expired entries independent of read traffic.

Everything else looked solid:

• ParseUnit fix (core/taskengine/macros/exp.go) — the prior implementation divided by the raw decimal value instead of scaling by 10^decimals, which was a significant correctness bug for any non-trivial decimals count; the new regex-validated, fraction-aware implementation plus parseunit_test.go regression coverage looks correct.
• Idempotency dedup logic itself (singleflight + persistent cache, subject-scoping, zero-address exclusion) is sound and well tested in run_node_immediately_idempotency_test.go.
• {{state.*}} scratch-vs-persistent split (vm.go / vm_runner_customcode.go) correctly guards simulation/no-task-id/no-db paths from touching real storage, with mutex-guarded scratch access.
• contractWrite native-value passthrough on the real execute path now matches the simulate path (closes the G1 fidelity gap called out in the bundled plan doc), with proper validation/error surfacing for a malformed value.
• partner.go renames (PartnerID→partnerID, Subject→subject) are correctly scoped to the package-private type only, per CLAUDE.md's guidance on reviewing rename sites individually.
• CI runner reversion and CLAUDE.md doc fixes are low-risk and consistent with their stated rationale.

Nice test coverage overall — no blocking issues, just the two perf/resource-growth notes above.

[Copilot]

Pull request overview

This PR extends the EigenLayer-AVS gateway/taskengine to support (1) durable per-workflow state accessible from CustomCode ({{state.*}}), (2) server-side REST auth providers (GoPlus) via restApi.config.options.auth, and (3) safer /nodes:run real contractWrite execution (native value passthrough, normalized receipt status, idempotency). It also includes supporting tests, documentation updates, and a CI runner change.

Changes:

• Add a durable per-workflow state store (wfstate:<taskId>:...) with a state.get/set/list goja binding and task-teardown cleanup.
• Add REST options.auth.provider parsing and GoPlus token mint/cache + Authorization header injection.
• Make /nodes:run contractWrite execution safer/more usable (forward value, normalize receipt executionStatus + userOpHash, idempotency via Idempotency-Key, and surface per-method receipts in REST metadata).

Reviewed changes

Copilot reviewed 24 out of 24 changed files in this pull request and generated 4 comments.

Show a summary per file

File	Description
PLAN_WORKFLOW_STATE_GUARDIAN_MONITORING.md	Design record for composable workflow state + REST auth providers (guardian composition).
PLAN_AGENT_ONDEMAND_ACTIONS.md	Design/verification plan for on-demand single-node contractWrite execution gaps and fixes.
go.mod	Promotes golang.org/x/sync to a direct dependency (singleflight).
docs/changes/20260717-workflow-state-and-rest-auth.md	Shipped-change record for {{state.*}} and REST options.auth GoPlus provider.
core/taskengine/vm.go	Adds VM-scoped scratch backing for non-persistent state.* mode.
core/taskengine/vm_workflow_state_test.go	Tests for workflow state persistence/simulation-noop + REST auth provider parsing/injection.
core/taskengine/vm_runner_rest.go	Implements options.auth.provider + GoPlus token mint/cache and Authorization injection.
core/taskengine/vm_runner_customcode.go	Adds state.get/set/list binding for CustomCode (DB-backed or scratch).
core/taskengine/vm_runner_contract_write.go	Forwards native value on real path; stamps normalized executionStatus/userOpHash into receipts; adds sim executionStatus.
core/taskengine/vm_runner_contract_write_ondemand_test.go	Unit tests for native value forwarding and executionStatus/userOpHash behavior.
core/taskengine/schema.go	Adds WorkflowStateKey / WorkflowStatePrefix key builders.
core/taskengine/run_node_immediately.go	Adds Idempotency-Key singleflight + persistent TTL cache; propagates runner salt into aa_salt.
core/taskengine/run_node_immediately_idempotency_test.go	Tests idempotency replay/scoping/TTL behavior.
core/taskengine/macros/parseunit_test.go	Tests corrected ParseUnit scaling + fractional handling + rejection cases.
core/taskengine/macros/exp.go	Fixes ParseUnit to match ethers parseUnits semantics (scale by 10^decimals, fractional support).
core/taskengine/engine.go	Adds singleflight group to Engine; cascades delete of wfstate keys on workflow deletion.
config/test.example.yaml	Adds GoPlus secrets + guardian_ruleset examples for tests.
config/gateway.example.yaml	Adds GoPlus secrets + guardian_ruleset examples for dev gateway config.
CLAUDE.md	Updates stale make targets / dev commands documentation.
aggregator/rest/partner.go	Refactors partner principal fields to package-private; adjusts ParseUnverified usage; relies on Go 1.22 loop binding.
aggregator/rest/partner_test.go	Extends key-decoding test coverage (RawURLEncoding) and updates principal field access.
aggregator/rest/handlers_nodes.go	Wires Idempotency-Key into nodes:run; wraps array metadata under metadata.results.
aggregator/rest/handlers_nodes_test.go	Tests array metadata wrapping and map metadata passthrough.
.github/workflows/run-test-on-pr.yml	Moves unit-test workflow jobs back to ubuntu-latest runners.

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.