Feature: Add UDP2 interconnect protocol implementation#1357
Conversation
|
Awesome! Could you please share some information about the architecture of the new solution? I think there might be an original (acmsigmod?) article on how to build an effective interconnect layer. (I see some familiar elements in the code, but I cannot fully comprehend what it is) |
During performance testing, we identified the Motion node as a significant performance bottleneck. Further analysis revealed that the root cause stems from inefficiencies in the underlying IPC layer protocol implementation. To address this, we plan to enhance the IPC layer to improve overall system performance. Currently, the IPC layer is tightly coupled with the database kernel, which makes implementing performance optimizations challenging and limits flexibility. As an initial step, we are decoupling the IPC layer from the database kernel. This separation will not only simplify future improvements but also enable independent testing and development of the IPC layer. This commit represents foundational work—specifically, the decoupling of the existing UDP-based IPC layer from the database kernel—and lays the groundwork for subsequent performance optimization efforts. |
|
The following two files also need the license header: |
zhangyue1818
force-pushed
the
udp2
branch
2 times, most recently
from
7e7c028 to
251adf4
Compare
|
Could you please share some information about the architecture of the new solution by doc ? |
Describes the architecture and other aspects in README.md，see 94428f1 |
my-ship-it
force-pushed
the
udp2
branch
2 times, most recently
from
4b03cc2 to
ddc1f60
Compare
|
That would be better to have a comprehensive commit body to these commits, sharing the basic information on what, why and how and more, following the If we squash and merge this PR, then can paste the commit body manually. |
zhangyue1818
changed the title
UDP2 is a next-generation interconnect protocol implementation that achieves complete decoupling between the interconnect layer and the database kernel. This implementation introduces: A layered architecture with adapter layer (ic_modules.c, ic_udp2.c) and core communication library (ic_common/) Lightweight data structures (ICCdbProcess, ICExecSlice) to replace complex database kernel structures Standardized C/C++ interfaces for database-agnostic communication CMake-based build system supporting independent compilation Unified error handling mechanism with configurable error levels Key benefits: Independent development and evolution of interconnect layer End-to-end testing without database kernel dependency Clear interface boundaries for easier extension and maintenance Rapid diagnosis of interconnect vs kernel issues
UDP2 is a next-generation interconnect protocol implementation that achieves complete decoupling between the interconnect layer and the database kernel. This implementation introduces: A layered architecture with adapter layer (ic_modules.c, ic_udp2.c) and core communication library (ic_common/) Lightweight data structures (ICCdbProcess, ICExecSlice) to replace complex database kernel structures Standardized C/C++ interfaces for database-agnostic communication CMake-based build system supporting independent compilation Unified error handling mechanism with configurable error levels Key benefits: Independent development and evolution of interconnect layer End-to-end testing without database kernel dependency Clear interface boundaries for easier extension and maintenance Rapid diagnosis of interconnect vs kernel issues
UDP2 is a next-generation interconnect protocol implementation that achieves complete decoupling between the interconnect layer and the database kernel. This implementation introduces: A layered architecture with adapter layer (ic_modules.c, ic_udp2.c) and core communication library (ic_common/) Lightweight data structures (ICCdbProcess, ICExecSlice) to replace complex database kernel structures Standardized C/C++ interfaces for database-agnostic communication CMake-based build system supporting independent compilation Unified error handling mechanism with configurable error levels Key benefits: Independent development and evolution of interconnect layer End-to-end testing without database kernel dependency Clear interface boundaries for easier extension and maintenance Rapid diagnosis of interconnect vs kernel issues
UDP2 is a next-generation interconnect protocol implementation that achieves complete decoupling between the interconnect layer and the database kernel. This implementation introduces: A layered architecture with adapter layer (ic_modules.c, ic_udp2.c) and core communication library (ic_common/) Lightweight data structures (ICCdbProcess, ICExecSlice) to replace complex database kernel structures Standardized C/C++ interfaces for database-agnostic communication CMake-based build system supporting independent compilation Unified error handling mechanism with configurable error levels Key benefits: Independent development and evolution of interconnect layer End-to-end testing without database kernel dependency Clear interface boundaries for easier extension and maintenance Rapid diagnosis of interconnect vs kernel issues
UDP2 is a next-generation interconnect protocol implementation that achieves complete decoupling between the interconnect layer and the database kernel. This implementation introduces: A layered architecture with adapter layer (ic_modules.c, ic_udp2.c) and core communication library (ic_common/) Lightweight data structures (ICCdbProcess, ICExecSlice) to replace complex database kernel structures Standardized C/C++ interfaces for database-agnostic communication CMake-based build system supporting independent compilation Unified error handling mechanism with configurable error levels Key benefits: Independent development and evolution of interconnect layer End-to-end testing without database kernel dependency Clear interface boundaries for easier extension and maintenance Rapid diagnosis of interconnect vs kernel issues
5 participants
UDP2 is a next-generation interconnect protocol implementation that
achieves complete decoupling between the interconnect layer and the
database kernel.
This implementation introduces:
and core communication library (ic_common/)
complex database kernel structures
Key benefits:
Fixes #ISSUE_Number
What does this PR do?
Type of Change
Breaking Changes
Test Plan
make installcheckmake -C src/test installcheck-cbdb-parallelImpact
Performance:
User-facing changes:
Dependencies:
Checklist
Additional Context
CI Skip Instructions