Skip to content

Do not queue unreplayable adaptive states after action errors #2031

Description

@chubes4

Problem

Adaptive browser exploration records and queues a destination state even when a multi-step action fails partway through. The frontier path retains the entire failed action. Every later reset attempts to replay that same action from the start, fails at the same later step, rejects the queued source state, and consumes action/time budgets without exploring new product behavior.

A real Extra Chill campaign exposed this immediately. The generated repeat action clicked the search trigger twice. The first click opened the modal; the second click timed out because the modal intercepted pointer events. The explorer correctly discovered the newly visible controls, but queued that state with the full failing repeat path. Twelve subsequent reset attempts failed on the same second click and the campaign exhausted maxDurationMs with zero findings.

Evidence

  • WP Codebox PR under test: Add adaptive browser state graph exploration #2030 / commit d1f6b0f6
  • Homeboy rig run: a62c74fd-ae10-4187-bbbc-0427f45b392c
  • Seed: extrachill-main-theme-001
  • Result: 16 actions, 3 states, 16 transitions, 1 action error, 0 findings, budgetExhausted=maxDurationMs
  • Artifact: wp-codebox/browser-adaptive-exploration-artifact/v1
  • Initial transition status: error, but its newly captured destination state was added to the frontier.
  • Repeated diagnostics: browser_adaptive_reset_replay_failed and browser_adaptive_source_state_not_reproduced.

Relevant flow in packages/runtime-playground/src/browser-adaptive-explorer.ts:

  • action exceptions become actionError;
  • the post-error state is still inserted into states;
  • path = [...source.path, action] is still queued;
  • restoreAdaptivePath() can only replay the complete action, not the successfully executed prefix.

Expected

An errored action must never create a frontier path that cannot be replayed deterministically.

Valid approaches include retaining the transition as diagnostic evidence while refusing to queue its destination, or explicitly recording the successfully executed step prefix as a replayable action/path when that prefix is itself a supported contract. Do not silently force the failed step or claim the full action succeeded.

Acceptance criteria

  • Error transitions remain visible in bounded evidence.
  • Unreplayable destination states are not queued.
  • If partial-step replay is supported, evidence and contracts honestly identify the executed prefix and reproduce its digest before queueing.
  • A fixture where step one opens an overlay and step two is intercepted does not cause repeated reset failures or budget starvation.
  • The campaign continues exploring other valid actions from replayable states.
  • Reset failure diagnostics are bounded and do not consume most of the campaign duration for one known-unreplayable path.
  • Deterministic seeds retain deterministic transition ordering and summaries.
  • Add regression coverage for minimization and cancellation when a multi-step action fails partway through.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions