fix(mergetree): use memory pool for arrow operations#2
Conversation
|
@codex review |
![gemini-code-assist[bot]](https://avatars.githubusercontent.com/in/956858?s=60&v=4){kind=small}
There was a problem hiding this comment.
Code Review
This pull request integrates custom memory management into Arrow-related operations within the Paimon core. It introduces the use of arrow::MemoryPool (derived from Paimon's MemoryPool) in KeyValueInMemoryRecordReader, SpillReader, and SpillWriter. These components now explicitly pass the memory pool to Arrow compute functions and IPC read/write operations to ensure consistent memory allocation and tracking. I have no feedback to provide.
|
@codex review |
433c0ac to
4bb4dd5
Compare
|
Codex Review: Didn't find any major issues. 🚀 ℹ️ About Codex in GitHubYour team has set up Codex to review pull requests in this repo. Reviews are triggered when you
If Codex has suggestions, it will comment; otherwise it will react with 👍. Codex can also answer questions or update the PR. Try commenting "@codex address that feedback". |
1 participant
Purpose
Linked issue: N/A
Route Arrow sort and spill IPC operations through the project
MemoryPoolso allocations are accounted consistently during in-memory key-value sorting and external spill read/write paths.Tests
cmake --build build --target paimon-core-testAPI and Format
No public API, storage format, or protocol changes.
Documentation
No new feature documentation required.
Generative AI tooling
Generated-by: OpenAI Codex