UCP/CORE: add failover support for put bcopy/short and am bcopy#11582
Open
jeynmann wants to merge 34 commits into
Open
UCP/CORE: add failover support for put bcopy/short and am bcopy#11582jeynmann wants to merge 34 commits into
jeynmann wants to merge 34 commits into
Conversation
jeynmann
force-pushed
the
failover_ext_ucp_bcopy_short
branch
from
June 27, 2026 16:27
a65b2ea to
6305b16
Compare
Arm RC mlx5 endpoints before peer failure so error handling leaves outstanding sends available for extraction. Suppress user completions while releasing CQ resources, and let endpoint destruction hand the failed QP to asynchronous garbage collection.
Verify that extraction reports posted operations in order and preserves the metadata needed by failover replay.
jeynmann
force-pushed
the
failover_ext_ucp_bcopy_short
branch
from
July 2, 2026 10:07
6305b16 to
b5b5ca0
Compare
Use the failover-specific _ft suffix for the completion-backed PUT bcopy entry point and transport operation. Remove the redundant capability bit since availability follows QUERY_TOKEN support.
Organize extracted operation metadata by protocol parameter group, align field-mask ordering with the structure layout, and distinguish AM flags from flush parameters. Also align internal operation initializers touched by outstanding extraction support.
jeynmann
force-pushed
the
failover_ext_ucp_bcopy_short
branch
from
July 6, 2026 13:09
b5b5ca0 to
e9b8868
Compare
jeynmann
changed the title
Exchange failed-lane state with the peer, extract outstanding transport operations, and coordinate endpoint reconfiguration before replay. Introduce the failover endpoint lifecycle and baseline fault-tolerance coverage for AM zcopy recovery.
Convert owned UCT operation descriptors into internal replay requests and route them through normal UCP protocol selection. Support extracted zcopy, bcopy, and short operations without exposing replay state outside failover recovery.
Move extracted and pending operations into one ordered replay queue before destroying failed lanes. Keep requests active after successful ownership transfer, and fall back to regular discard when extraction cannot complete.
Advertise replay support for AM eager and PUT short/bcopy protocols when their candidate lanes are compatible with hardware PSN failover. Preserve protocol-specific replay metadata for single- and multi-fragment operations.
Exercise AM and PUT single- and multi-fragment protocols with a saturated send queue so failover replays both undelivered and unposted operations. Validate every received message and the final PUT payload after recovery.
Use the common RMA operation state and completion callbacks for completion-backed PUT bcopy replay. Consolidate failure cleanup and remove failover-specific RMA state.
Consume the refined outstanding-operation field masks and data layout, validate the operation field, and read AM flags from the AM parameter group.
jeynmann
force-pushed
the
failover_ext_ucp_bcopy_short
branch
from
July 7, 2026 02:26
e9b8868 to
4633700
Compare
jeynmann
changed the title
Comment on lines
+1944
to
+1955
| if (ucs_unlikely(ep->super.failover_flags & | ||
| UCT_RC_EP_FAILOVER_FLAG_ARMED)) { | ||
| /* Ownership is deferred to extract; only release HW/CQ resources | ||
| * without invoking user completions. */ | ||
| ucs_debug("ep %p: deferring tx completions for failover, hw ci %u " | ||
| "sw pi %u available %d", | ||
| ep, hw_ci, ep->tx.wq.sw_pi, | ||
| uct_rc_txqp_available(&ep->super.txqp)); | ||
| uct_rc_mlx5_iface_update_tx_res(&iface->super, ep, hw_ci); | ||
| ucs_debug("ep %p: released failover tx resources, available %d", ep, | ||
| uct_rc_txqp_available(&ep->super.txqp)); | ||
| uct_ib_mlx5_update_db_cq_ci(&iface->cq[UCT_IB_DIR_TX]); |
Contributor
There was a problem hiding this comment.
I think we need invoke a user completion with error, and it's user's (UCP) responsibility to extract or convert failed CQE to a WQE which could be reposted according to RX tocken
Comment on lines
+106
to
+108
| [UCP_OP_ID_FAILOVER_AM_BCOPY] = "failover_am_bcopy", | ||
| [UCP_OP_ID_FAILOVER_PUT_SHORT] = "failover_put_short", | ||
| [UCP_OP_ID_FAILOVER_PUT_BCOPY] = "failover_put_bcopy", |
Contributor
There was a problem hiding this comment.
do we really need new ops?
Comment on lines
+98
to
+99
| if ((op->info.field_mask & UCT_EP_OP_INFO_FIELD_COMP) && | ||
| (op->info.comp != NULL)) { |
Contributor
There was a problem hiding this comment.
I would assume that if the field flag is set, the filed is valid
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What?
Add UCP failover lane recovery and replay support for put bcopy/short and am bcopy/short .
Why?
Failover recovery needs to resend operations that were posted locally but not received by the peer before a lane failure.
How?
UCT extracted outstanding operations through
uct_ep_op_info_t.UCP owns a copy of each extracted operation and payload and selects a dedicated
failover/replayproto. Currently AM replay requiresAM_BCOPY; PUT replay requiresPUT_SHORTorPUT_BCOPYand same-MD lanes.