UCT: Add outstanding operation extraction support by jeynmann · Pull Request #11580 · openucx/ucx

jeynmann · 2026-06-24T18:26:36Z

What?

Add UCT-level support for failover ops extraction.
Add impl for token query.

Why?

UCP failover needs map tokens to send ops and resend failed ops.

How?

Introduces a callback-based extraction api.

evgeny-leksikov · 2026-07-01T08:33:26Z

+     * Operation type from @ref uct_ep_operation_t (e.g. UCT_EP_OP_AM_SHORT,
+     * UCT_EP_OP_PUT_ZCOPY, UCT_EP_OP_FLUSH, etc.).
+     */
+    uct_ep_operation_t operation;


UCT_EP_OP_INFO_FIELD_OPERATION is missed

evgeny-leksikov · 2026-07-01T08:34:32Z

+    UCT_EP_OP_INFO_FIELD_AM     = UCS_BIT(0), /**< am_id, am_hdr* fields */
+    UCT_EP_OP_INFO_FIELD_DATA   = UCS_BIT(1), /**< data/length (short/bcopy) */
+    UCT_EP_OP_INFO_FIELD_IOV    = UCS_BIT(2), /**< iov/iovcnt (zcopy) */
+    UCT_EP_OP_INFO_FIELD_RMA    = UCS_BIT(3), /**< remote_addr, rkey */
+    UCT_EP_OP_INFO_FIELD_ATOMIC = UCS_BIT(4), /**< atomic_op, value, compare */
+    UCT_EP_OP_INFO_FIELD_COMP   = UCS_BIT(5), /**< Original uct_completion_t */
+    UCT_EP_OP_INFO_FIELD_FLAGS  = UCS_BIT(6), /**< Original flags */


pls keep the same order as fields of struct uct_ep_op_info

evgeny-leksikov · 2026-07-01T08:37:18Z

    res_count = uct_ib_mlx5_post_send(txwq, ctrl, wqe_size, 1);
+    num_bb    = txwq->sw_pi - pi;
+
+    if (txwq->msn != NULL) {


redundant bcz check on init

evgeny-leksikov · 2026-07-01T08:41:01Z

+        if (opcode != MLX5_OPCODE_NOP) {
+            uct_rc_mlx5_txwq_set_msn(txwq, pi, num_bb);
+        } else {
+            uct_rc_mlx5_txwq_set_msn(txwq, pi, 0);
+        }


Suggested change

if (opcode != MLX5_OPCODE_NOP) {

uct_rc_mlx5_txwq_set_msn(txwq, pi, num_bb);

} else {

uct_rc_mlx5_txwq_set_msn(txwq, pi, 0);

}

num_bb = ucs_likely(opcode != MLX5_OPCODE_NOP) ?

(txwq->sw_pi - pi) : 0;

uct_rc_mlx5_txwq_set_msn(txwq, pi, num_bb);

evgeny-leksikov · 2026-07-01T08:43:32Z

+        attr.field_mask |= UCT_IB_MLX5_EXT_QP_QUERY_ATTR_FIELD_QP_NUM;
+        attr.qp_num      = ep->tx.wq.super.qp_num;
+#endif
+    }


else return INVALID_PARAM

evgeny-leksikov · 2026-07-01T08:45:03Z

+    if (attr.field_mask == 0) {
+        return UCS_OK;
+    }


could be checked in advance for ep_attr->field_mask near check at line 38

evgeny-leksikov · 2026-07-01T08:58:34Z

+            continue;
+        }
+
+        return plugin->ops.ep_outstanding_extract(ep, params);


since we query QP in plugin API then it should be plugin->ops.qp_outstanding_extract

Currently plugin needs ep ctx, not only the QP: it derives the RC ep, iface, txwq and txqp, then walks the outstanding send queue.
Passing only QP is not sufficient.

so then it should be EP in both cases

evgeny-leksikov · 2026-07-01T09:01:56Z

    uct_ib_mlx5_ext_qp_query_func_t                qp_query;                     /**< QP query callback */
    uct_ib_mlx5_ext_max_put_sgl_zcopy_count_func_t max_put_sgl_zcopy_count;      /**< Maximum PUT SGL zero-copy entry count callback */
    uct_ep_put_sgl_zcopy_func_t                    ep_put_sgl_zcopy;             /**< PUT SGL zero-copy callback */
+    uct_ep_outstanding_extract_func_t              ep_outstanding_extract;       /**< Outstanding operation extract callback */


need to align qp_query/ep_outstanding_extract APIs: QP or EP?

Problem is the 2 APIs need different scope of qp/ep info.

evgeny-leksikov · 2026-07-01T09:04:07Z

+ *       caller.
+ */
+ucs_status_t
+uct_ep_outstanding_extract(uct_ep_h ep,


how about to rename to uct_ep_outstanding_purge? (to align with uct_ep_pending_purge)

This operation is not similar as uct_ep_pending_purge: it classifies outstanding WQEs, optionally completes already delivered operations, and reports undelivered operations back to the caller for replay. Maybe "extract" describes this behavior more accurately than "purge", WDYT?

btw, why do we need to extract all the outstanding operations? in terms of IB I would expect to have a converting of failed CQE to WQE and repost it if not delivered. Should not it be enough for reposting failover protocols?

evgeny-leksikov · 2026-07-01T09:07:52Z

+uct_rc_mlx5_txwq_set_msn(uct_ib_mlx5_txwq_t *txwq, uint16_t pi, uint16_t num_bb)
+{
+    unsigned index                = pi % txwq->bb_max;
+    uct_ib_mlx5_txwq_msn_t *entry = &txwq->msn[index];


can we make msn entry part of uct_rc_iface_send_op to improve mem locality?

But not every WQE has a send_op associated with it, while extraction needs MSN information for all outstanding WQEs.

Can you provide an example? Do we need to repost WQEs (after failure) which do not have send_op

Arm RC mlx5 endpoints before peer failure so error handling leaves outstanding sends available for extraction. Suppress user completions while releasing CQ resources, and let endpoint destruction hand the failed QP to asynchronous garbage collection.

Verify that extraction reports posted operations in order and preserves the metadata needed by failover replay.

Use the failover-specific _ft suffix for the completion-backed PUT bcopy entry point and transport operation. Remove the redundant capability bit since availability follows QUERY_TOKEN support.

Organize extracted operation metadata by protocol parameter group, align field-mask ordering with the structure layout, and distinguish AM flags from flush parameters. Also align internal operation initializers touched by outstanding extraction support.

evgeny-leksikov

Please split the PR to API and impl, PR with API can include stubs if <= 500 lines.

evgeny-leksikov · 2026-07-10T15:25:58Z

+        struct {
+            uint8_t  am_id;       /**< AM handler ID */
+            unsigned flags;       /**< Flags passed to the AM operation */
+            union {
+                uint64_t   header;         /**< AM short 64-bit header word */
+                const void *header_buffer; /**< AM zcopy header buffer */
+            };
+            size_t header_length; /**< AM zcopy header length */
+        } am;


should we have such structures under union backward compatible?

evgeny-leksikov · 2026-07-10T15:33:56Z

+    num_bb    = ucs_likely(opcode != MLX5_OPCODE_NOP) ?
+                (txwq->sw_pi - pi) : 0;
+
+    if (txwq->wqe_priv != NULL) {


redundant check, done in ctor

evgeny-leksikov · 2026-07-10T15:45:15Z

+uct_rc_mlx5_txwq_set_msn(uct_ib_mlx5_txwq_t *txwq, uint16_t pi, uint16_t num_bb)
+{
+    unsigned index                = pi % txwq->bb_max;
+    uct_ib_mlx5_txwq_msn_t *entry = &txwq->msn[index];


Can you provide an example? Do we need to repost WQEs (after failure) which do not have send_op

JeynmannZ added 3 commits June 27, 2026 19:07

UCT: Add outstanding operation extraction support

1339090

UCT/MLX5: fix format

619f5c2

UCT/MLX5: add msn meta

04b656e

jeynmann force-pushed the failover_ext_uct branch from de6901e to 04b656e Compare June 27, 2026 16:27

gleon99 requested review from evgeny-leksikov and william-gallagher-nv June 30, 2026 17:14

evgeny-leksikov reviewed Jul 1, 2026

View reviewed changes

JeynmannZ added 20 commits July 2, 2026 12:16

GTEST/UCT: Cover outstanding operation extraction

0b09ad0

Verify that extraction reports posted operations in order and preserves the metadata needed by failover replay.

UCT/IB/MLX5: address comments

10ff8b2

UCT/FAILOVER: Harden mlx5 outstanding extraction

76b0726

UCT/RMA: Add completion-backed PUT bcopy

cc65b5e

UCT/RMA: Rename completion-backed PUT bcopy API

c0ffb8a

Use the failover-specific _ft suffix for the completion-backed PUT bcopy entry point and transport operation. Remove the redundant capability bit since availability follows QUERY_TOKEN support.

UCT/IB/MLX5: Guard MSN metadata updates

48fc1ef

UCT: Add endpoint failover enable state

31a51f3

UCT/IB/MLX5: Remove redundant extract wrapper

7067482

Merge remote-tracking branch 'origin/master' into failover_ext_uct

b999d83

UCT/API: fix doc build

7ef3141

UCT: fix format

226e249

UCT/RMA: Add completion-backed PUT short

383df1e

UCT: cleanup

31d4754

UCT: address comments

104bf4f

UCT/IB/MLX5: add mlx5 priv

e9b496a

TEST/UCT: add plugin test

1f729c8

Merge remote-tracking branch 'origin/master' into failover_ext_uct

3a315ae

TEST/UCT: fix CI

5b1db16

evgeny-leksikov requested changes Jul 10, 2026

View reviewed changes

Uh oh!

Conversation

jeynmann commented Jun 24, 2026

What?

Why?

How?

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

evgeny-leksikov left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants