Skip to content

Support for irdma gen3#1640

Open
tatyana-en wants to merge 8 commits intolinux-rdma:masterfrom
tatyana-en:support-for-irdma-gen3
Open

Support for irdma gen3#1640
tatyana-en wants to merge 8 commits intolinux-rdma:masterfrom
tatyana-en:support-for-irdma-gen3

Conversation

@tatyana-en
Copy link
Copy Markdown
Contributor

[PATCH 0/9] Add RDMA support for Intel IPU E2000 (GEN 3)

This patch series introduces the user space RDMA RoCEv2 support
for the Intel Infrastructure Processing Unit (IPU) E2000 line of
products, referred to as GEN 3 in the irdma provider.

The patch series for the kernel GEN 3 irdma driver is at
https://lore.kernel.org/all/20250827152545.2056-1-tatyana.e.nikolova@intel.com/

Key highlights of this series as it pertains to GEN 3 device include:

  • Enable 64-byte Completion Queue Entry (CQE) support
  • CQE Error and Flush Handling Changes
  • Atomic Operations Support
  • Shared Receive Queue Support
  • Type2 Memory Windows and CQE Timestamp Capability

The patches have been tested on a range of hosts and platforms with a variety
of general RDMA applications which include standalone verbs (rping, perftest, etc.),
storage and HPC applications.

Changelog:

Differences from the prior RFC PR at #1486
for this series include:

  • Remove Push Mode Support for GEN 3
  • Add checks to enable SRQ and Atomic Operations if they are supported

@tatyana-en
Copy link
Copy Markdown
Contributor Author

Hi, the kernel patch has been included in linux 6.18 RC. Could this PR be merged now?

@tatyana-en
Copy link
Copy Markdown
Contributor Author

tatyana-en commented Jan 16, 2026

Hi, I followed the instructions in the docs and ran $ kernel-headers/update ~/linux.git --amend on my series. Please let me know if this is okay. If not, I can rebase my series to the current rdma-core master.

Thanks,
Tatyana

@tatyana-en tatyana-en force-pushed the support-for-irdma-gen3 branch from 12a4637 to 3f30df5 Compare March 10, 2026 22:40
mustafakismail and others added 8 commits March 23, 2026 12:00
Enable 64-byte CQE support if the feature flag IRDMA_FEATURE_64_BYTE_CQE is
set. Adjust the CQ size calculations in create_cq() and resize_cq() and use
get_cq_size() in resize CQ instead of calculating again. Make CQ size an
even number, which is a GEN 3 HW requirement.

Signed-off-by: Mustafa Ismail <mustafa.ismail@intel.com>
Signed-off-by: Tatyana Nikolova <tatyana.e.nikolova@intel.com>
Modify cq_poll_compl() to poll flushed CQEs for GEN 3

Add IBV_WC_RETRY_EXC_ERR and FLUSH_RNR_RETRY_EXC_ERR to the returned
ibv_wc_status errors

Signed-off-by: Shiraz Saleem <shiraz.saleem@intel.com>
Signed-off-by: Tatyana Nikolova <tatyana.e.nikolova@intel.com>
Add set_ib_wc_op_rq_gen_3() to set the ibv_wc opcodes for GEN 3 RQ
completions

Signed-off-by: Mustafa Ismail <mustafa.ismail@intel.com>
Signed-off-by: Tatyana Nikolova <tatyana.e.nikolova@intel.com>
Set WQE_FORMAT_V2 flag in ucontext allocation request. This prevents
driver from binding GEN3 devices to older user-space providers

Signed-off-by: Shiraz Saleem <shiraz.saleem@intel.com>
Signed-off-by: Tatyana Nikolova <tatyana.e.nikolova@intel.com>
Add atomic operations support for GEN 3 devices, namely Compare and Swap
and Fetch and Add.

Add FEATURE_ATOMIC_OPS check to allow atomic ops only when supported.

Signed-off-by: Faisal Latif <faisal.latif@intel.com>
Signed-off-by: Tatyana Nikolova <tatyana.e.nikolova@intel.com>
Implement verb API and UAPI changes to support SRQ functionality in GEN 3
devices. Add SRQ feature check and register the SRQ device ops only if SRQ
is supported.

Provide function ae_to_qp_err_code() to translate AE codes into QP errors.
The function can be leveraged for flushed CQEs and error CQEs associated
with an SRQ.

Signed-off-by: Faisal Latif <faisal.latif@intel.com>
Signed-off-by: Tatyana Nikolova <tatyana.e.nikolova@intel.com>
Remove deprecated timestamp functionality for GEN 2.

Enable FEATURE_CQE_TIMESTAMPING for GEN 3 devices. The Host Channel
Adapter (HCA) core clock frequency in the timestamp info is retrieved
from the CQE.

Signed-off-by: Shiraz Saleem <shiraz.saleem@intel.com>
Signed-off-by: Tatyana Nikolova <tatyana.e.nikolova@intel.com>
Enable Type2 MW for GEN3 devices.

Type2 MW is disabled for GEN2/1 devices via the capability flag in
query_device.

Signed-off-by: Shiraz Saleem <shiraz.saleem@intel.com>
Signed-off-by: Tatyana Nikolova <tatyana.e.nikolova@intel.com>
@tatyana-en tatyana-en force-pushed the support-for-irdma-gen3 branch 3 times, most recently from 3f30df5 to 6b2a21b Compare March 24, 2026 18:23
@tatyana-en
Copy link
Copy Markdown
Contributor Author

tatyana-en commented Mar 24, 2026

I rebased the PR. Updating the kernel headers may not be necessary. I encountered issues with Azure pipelines which don't seem related to my patches. Any help or advice on how to resolve these is highly appreciated.

@tatyana-en tatyana-en force-pushed the support-for-irdma-gen3 branch from 6b2a21b to 0e79664 Compare March 24, 2026 18:48
@jgunthorpe
Copy link
Copy Markdown
Member

/azp run

@azure-pipelines
Copy link
Copy Markdown

Azure Pipelines successfully started running 1 pipeline(s).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants