Conversation
…ency Sepolia's bundler submission + mining often exceeds 60s under current network conditions. The 60s cap caused the aggregator to return 'No transaction receipt received' to the SDK while the UserOp was still in flight -- it would confirm ~30s later, but too late for the client response. 120s aligns with the SDK's TimeoutPresets.SLOW.
… to 2.17.0 Rewrite the "How the Client Interprets Execution Status" section to reflect the shipped SDK model. Replaces the pre-migration workaround notes with the final Success/Failed/Error/Pending table, the current Studio branch-skip detection (steps.length < executionNodeCount on Success), and per-component mapping of where status logic lives.
Cross-repo content (covers AVS aggregator, Studio proxy, and partner API infra) belongs in avs-infra, not in the AVS repo.
Context-memory now strictly validates that every /api/summarize request carries the aggregator's execution verdict, rejecting calls that omit status or executionError with HTTP 400. Without these fields the downstream summarizer fell through to its "error" branch and emitted "could not run" subjects even when every step succeeded, contradicting the green success badge rendered on the same email. Compute the verdict via vm.AnalyzeExecutionResult() in Summarize() before buildRequest takes vm.mu (sync.Mutex is not reentrant), map the enum to the lowercase strings "success" | "failed" | "error" expected by the API, and include them on contextMemorySummarizeRequest without omitempty so the keys are always present (executionError is "" on success).
TestRunNodeImmediately_UniswapSwap_Base and its variants, plus
TestExecuteTask_SequentialContractWrites_Base_{SelfFunded,Paymaster}, require
at least 0.01 USDC on a smart wallet on Base mainnet to run — the setup fails
with "wallet ... needs at least 0.01 USDC (10000 raw)" on any machine where
the wallet isn't pre-funded, producing unreliable local and CI runs.
Dropping the two Base test files (1840 lines combined) and pulling the four
Sepolia constants still referenced by the Sepolia approve+swap tests into a
small shared constants file.
…w warn badge Context-memory now sends four new body fields on every /api/summarize response (skippedNote, executedSteps, totalSteps, skippedSteps) and no longer suffixes the skip note onto body.summary. The aggregator was rendering a green "All steps completed successfully" badge for success-with-skipped runs because it only looked at status, and it was concatenating the skip note into the email title. Changes: - Plumb the four new fields through contextMemorySummarizeBody into Summary, and populate them in both the context-memory path and the deterministic fallback (computeSkippedStepCount mirrors the branch-skip logic so the rendering layer always has the counts). - Drop the legacy "partial_success" / "failure" status strings. The status model is now the same "success" | "failed" | "error" triad context-memory emits, with the yellow "warn" variant derived from (Status=="success" && SkippedSteps>0). Removed statusIconPartialSuccess and the unused StatusHtmlFailedTemplate / StatusHtmlSuccessTemplate constants. - Rework the status helpers (getStatusColor, getStatusDisplayText, getStatusEmoji, buildStatusHtml) to take a Summary so they can derive the warn variant. The email badge renders yellow with the skip note as its text; the Telegram formatter prepends⚠️ and surfaces the skip note on its own italic line under the subject. - Update the vm_runner_rest.go deterministic override path to match: no in-title concatenation, yellow background when branch-skipped, skipped_note emitted as its own template variable. Verified against two local simulations: the branch-skipped run renders yellow with status_text="1 node was skipped by Branch condition." and a clean title; the normal run renders green with "All steps completed successfully" and a populated executions[] array.
TaskTriggerToConfig stored cronTrigger.Config.Schedules ([]string) and
fixedTimeTrigger.Config.Epochs ([]int64) directly into the returned
map[string]interface{}. engine.go then passes that map to structpb.NewValue,
which only accepts []interface{} for list values and errors on any concrete
typed slice — so every Cron/FixedTime simulation logged
WARN engine.go:2956 Failed to convert trigger input to protobuf
{"error": "proto: invalid type: []string"}
before falling back to a json.Marshal/Unmarshal roundtrip that happens to
coerce the values. Convert both arrays to []interface{} at the source so the
direct structpb path succeeds and the fallback becomes dead (still kept as a
safety net).
Code Review — PR #525OverviewThis PR bundles six distinct changes: a UserOp receipt timeout increase (60 s → 120 s), a The motivation for each change is well-documented and the mutex sequencing insight (calling Issues1.
|
| Area | Assessment |
|---|---|
Mutex sequencing (AnalyzeExecutionResult before buildRequest) |
Correct — well-documented |
Status model migration (partial_success → warn derived from counts) |
Correct, clean |
structpb slice widening |
Correct fix |
| Duplicated skip-note logic | Should extract helper |
| Deterministic "error" state gap | Worth addressing |
| Shallow copy in fallback | Low risk now, worth documenting |
ExecutionLogs lock hygiene |
Pre-existing; document the invariant |
| Test coverage | Good — unit tests cover warn/success/failed/error paths |
No blocking issues, but items 1–2 above are worth a follow-up before this ships to production.
There was a problem hiding this comment.
Pull request overview
This PR aligns the aggregator’s execution-status model and notification rendering with the updated context-memory + Studio SDK contract, while also addressing Sepolia UserOp latency and fixing trigger config protobuf conversion.
Changes:
- Increase ERC-4337 UserOp receipt wait timeout from 60s to 120s to reduce false “no receipt” failures on Sepolia.
- Update context-memory summarization integration to send/consume the new status + skipped-step fields and render a warn (yellow) badge for success-with-skips.
- Fix cron/fixed-time trigger config conversion for
structpb.NewValue, and remove unreliable Base mainnet integration tests (extract shared Sepolia constants).
Reviewed changes
Copilot reviewed 18 out of 18 changed files in this pull request and generated 3 comments.
Show a summary per file
| File | Description |
|---|---|
| pkg/erc4337/preset/builder.go | Raises max UserOp confirmation wait to tolerate Sepolia bundler/mining delays. |
| docs/changes/0001-execution-status-redesign.md | Updates docs to the final shipped client status interpretation model (SDK 2.17.0). |
| docs/API_PROTOCOL_COMPARISON.md | Removes this doc from the repo (moved to avs-infra per PR description). |
| core/taskengine/vm_runner_rest.go | Updates email status badge rendering to support warn-on-skips and new status strings. |
| core/taskengine/vm.go | Adds mapping helper from execution verdict enum to context-memory status strings; updates comments. |
| core/taskengine/uniswap_sepolia_constants_test.go | Adds shared Sepolia Uniswap constants for remaining tests. |
| core/taskengine/summarizer_format_test.go | Updates formatter tests for new status contract and warn derivation from skipped steps. |
| core/taskengine/summarizer_format_telegram.go | Derives Telegram emoji from Summary (warn on skipped steps) and renders skippedNote. |
| core/taskengine/summarizer_format_integration_test.go | Updates integration expectations to the new status triad (warn derived via skipped steps). |
| core/taskengine/summarizer_format_email.go | Updates SendGrid dynamic data helpers to use Summary-derived warn + step counts. |
| core/taskengine/summarizer_deterministic.go | Adds skipped-step counting + note into deterministic Summary struct fields. |
| core/taskengine/summarizer_context_memory_test.go | Updates request builder tests for new required request fields. |
| core/taskengine/summarizer_context_memory_integration_test.go | Updates integration request/response models and VM→request building for new contract. |
| core/taskengine/summarizer_context_memory.go | Sends status + executionError to context-memory; plumbs skipped fields from response into Summary. |
| core/taskengine/run_node_uniswap_swap_test.go | Removes Base-dependent funded-wallet tests (unreliable locally/CI). |
| core/taskengine/node_types.go | Fixes cron/fixed-time trigger config list typing for structpb conversion. |
| core/taskengine/manual_trigger_loop_test.go | Updates comment to reflect new “failed” execution-level status wording. |
| core/taskengine/execute_sequential_contract_writes_base_test.go | Removes Base-dependent funded-wallet tests (unreliable locally/CI). |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
| // Compute execution verdict BEFORE buildRequest acquires vm.mu — AnalyzeExecutionResult | ||
| // takes the same lock and sync.Mutex is not reentrant. | ||
| executionError, _, resultStatus := vm.AnalyzeExecutionResult() | ||
| status := mapExecutionStatusToAPIString(resultStatus) |
| // Compute execution verdict BEFORE buildRequest acquires vm.mu — AnalyzeExecutionResult | ||
| // takes the same lock and sync.Mutex is not reentrant. | ||
| executionError, _, resultStatus := vm.AnalyzeExecutionResult() | ||
| status := mapExecutionStatusToAPIString(resultStatus) | ||
|
|
||
| // Build request from VM | ||
| req, err := c.buildRequest(vm, currentStepName) | ||
| req, err := c.buildRequest(vm, currentStepName, status, executionError) |
| skippedStepCount := computeSkippedStepCount(vm) | ||
| executedStepCount := len(vm.ExecutionLogs) | ||
| skippedNote := "" | ||
| if status == "success" && skippedStepCount > 0 { | ||
| noun, verb := "node", "was" | ||
| if skippedStepCount != 1 { | ||
| noun, verb = "nodes", "were" | ||
| } | ||
| skippedNote = fmt.Sprintf("%d %s %s skipped by Branch condition.", skippedStepCount, noun, verb) | ||
| } | ||
|
|
||
| return Summary{ | ||
| Subject: subject, | ||
| Body: body, | ||
| SummaryLine: summaryLine, | ||
| Status: status, | ||
| Trigger: trigger, | ||
| TriggeredAt: triggeredAt, | ||
| Executions: executions, | ||
| Errors: errors, | ||
| SmartWallet: smartWallet, | ||
| Network: resolveChainName(vm), | ||
| Annotation: annotation, | ||
| // AnalysisHtml is now generated by buildAnalysisHtmlFromStructured() in SendGridDynamicData() | ||
| Subject: subject, | ||
| Body: body, | ||
| SummaryLine: summaryLine, | ||
| Status: status, | ||
| Trigger: trigger, | ||
| TriggeredAt: triggeredAt, | ||
| Executions: executions, | ||
| Errors: errors, | ||
| SmartWallet: smartWallet, | ||
| Network: resolveChainName(vm), | ||
| Annotation: annotation, | ||
| SkippedNote: skippedNote, | ||
| ExecutedSteps: executedStepCount, | ||
| TotalSteps: executedStepCount + skippedStepCount, | ||
| SkippedSteps: skippedStepCount, |
…ncy, DRY skip note Four findings from Copilot and Claude on #525: - ContextMemorySummarizer.Summarize used AnalyzeExecutionResult() even when vm.ExecutionLogs was empty. That case, which occurs during single-node RunNodeImmediately runs where the notification node is the only step and is still executing, returned ExecutionFailed / "no execution steps found", so the aggregator was sending status=failed + a bogus executionError to context-memory. Mirror the deterministic path and treat empty logs as success so the verdict matches (nothing has failed yet). - ComposeSummary populated Summary.ExecutedSteps/TotalSteps from len(vm.ExecutionLogs) and a separate computeSkippedStepCount helper. That could overcount versus SummaryLine/subject, which exclude notification nodes via getTotalWorkflowSteps. Use the already-computed executedSteps / skippedCount / totalWorkflowSteps locals so the step fields match the rendered text, and delete the now-unused helper. - The skip-sentence pluralisation ("1 node was ... / N nodes were ...") lived in both summarizer_deterministic.go and vm_runner_rest.go. Extract buildSkippedNote(count) once so the wording can't drift. - computeExecutionStatus's docstring said "the same triad the aggregator emits" while only returning two values. Explain that "error" is reserved for VM-level failures that prevent a Summary from being built, so the deterministic fallback never emits it.
Sepolia's bundler submission + mining often exceeds 60s under current
network conditions. The 60s cap caused the aggregator to return
'No transaction receipt received' to the SDK while the UserOp was
still in flight -- it would confirm ~30s later, but too late for the
client response. 120s aligns with the SDK's TimeoutPresets.SLOW.
Rewrite the "How the Client Interprets Execution Status" section to
reflect the shipped SDK model. Replaces the pre-migration workaround
notes with the final Success/Failed/Error/Pending table, the current
Studio branch-skip detection (steps.length < executionNodeCount on
Success), and per-component mapping of where status logic lives.
Cross-repo content (covers AVS aggregator, Studio proxy, and partner
API infra) belongs in avs-infra, not in the AVS repo.
Context-memory now strictly validates that every /api/summarize request
carries the aggregator's execution verdict, rejecting calls that omit
status or executionError with HTTP 400. Without these fields the
downstream summarizer fell through to its "error" branch and emitted
"could not run" subjects even when every step succeeded, contradicting
the green success badge rendered on the same email.
Compute the verdict via vm.AnalyzeExecutionResult() in Summarize() before
buildRequest takes vm.mu (sync.Mutex is not reentrant), map the enum to
the lowercase strings "success" | "failed" | "error" expected by the API,
and include them on contextMemorySummarizeRequest without omitempty so the
keys are always present (executionError is "" on success).
TestRunNodeImmediately_UniswapSwap_Base and its variants, plus
TestExecuteTask_SequentialContractWrites_Base_{SelfFunded,Paymaster}, require
at least 0.01 USDC on a smart wallet on Base mainnet to run — the setup fails
with "wallet ... needs at least 0.01 USDC (10000 raw)" on any machine where
the wallet isn't pre-funded, producing unreliable local and CI runs.
Dropping the two Base test files (1840 lines combined) and pulling the four
Sepolia constants still referenced by the Sepolia approve+swap tests into a
small shared constants file.
Context-memory now sends four new body fields on every /api/summarize response
(skippedNote, executedSteps, totalSteps, skippedSteps) and no longer suffixes the
skip note onto body.summary. The aggregator was rendering a green "All steps
completed successfully" badge for success-with-skipped runs because it only
looked at status, and it was concatenating the skip note into the email title.
Changes:
Plumb the four new fields through contextMemorySummarizeBody into Summary,
and populate them in both the context-memory path and the deterministic
fallback (computeSkippedStepCount mirrors the branch-skip logic so the
rendering layer always has the counts).
Drop the legacy "partial_success" / "failure" status strings. The status
model is now the same "success" | "failed" | "error" triad context-memory
emits, with the yellow "warn" variant derived from (Status=="success" &&
SkippedSteps>0). Removed statusIconPartialSuccess and the unused
StatusHtmlFailedTemplate / StatusHtmlSuccessTemplate constants.
Rework the status helpers (getStatusColor, getStatusDisplayText,⚠️ and surfaces the skip note on its own
getStatusEmoji, buildStatusHtml) to take a Summary so they can derive the
warn variant. The email badge renders yellow with the skip note as its text;
the Telegram formatter prepends
italic line under the subject.
Update the vm_runner_rest.go deterministic override path to match: no
in-title concatenation, yellow background when branch-skipped, skipped_note
emitted as its own template variable.
Verified against two local simulations: the branch-skipped run renders yellow
with status_text="1 node was skipped by Branch condition." and a clean title;
the normal run renders green with "All steps completed successfully" and a
populated executions[] array.
TaskTriggerToConfig stored cronTrigger.Config.Schedules ([]string) and
fixedTimeTrigger.Config.Epochs ([]int64) directly into the returned
map[string]interface{}. engine.go then passes that map to structpb.NewValue,
which only accepts []interface{} for list values and errors on any concrete
typed slice — so every Cron/FixedTime simulation logged
before falling back to a json.Marshal/Unmarshal roundtrip that happens to
coerce the values. Convert both arrays to []interface{} at the source so the
direct structpb path succeeds and the fallback becomes dead (still kept as a
safety net).