Skip to content

Support flow control (backpressure) visualization in backend #4321

@yunyad

Description

@yunyad

Feature Summary

During workflow execution, operators may encounter backpressure when incoming data arrives faster than it can be processed or forwarded downstream. This feature proposes adding backend support to expose flow control / backpressure status for operators and edges so that the frontend can visualize these runtime signals. With this capability, we would provide much stronger execution observability, making it easier to diagnose bottlenecks, understand runtime behavior, and reason about throughput limitations in a much more intuitive way.

Proposed Solution or Design

At a high level, the backend should expose runtime flow control information that already exists in the system and make it available through execution monitoring updates so that the frontend can present it as part of workflow visualization.

The key idea is not to introduce new execution behavior, but to surface existing runtime signals in a way that is meaningful at the workflow level. Instead of keeping flow control state internal to the runtime, the backend would translate it into a form that can be consumed by the frontend and rendered as intuitive pressure indicators on workflows.

Importantly, this feature is intended as an observability enhancement rather than a behavioral change. It should focus on exposing and organizing existing runtime information, without changing the underlying flow control mechanism itself.

Impact / Priority

(P2)Medium – useful enhancement

Affected Area

No response

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels
    No fields configured for Feature.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions