Asynchronously copy primaries to device in LocalTransporter by esseivaju · Pull Request #2406 · celeritas-project/celeritas

esseivaju · 2026-05-19T22:34:59Z

Implements double buffering for staging Geant4 primaries before GPU transport.

Primaries are still accumulated until auto_flush_ entries have been buffered. On GPU builds, the filled buffer is then moved into a staged buffer and copied to device memory on the Celeritas stream. A DeviceEvent protects the pinned host buffer lifetime until the asynchronous copy completes.

This changes the effective auto_flush_ behavior on GPU: transport is not forced when the first buffer fills. Instead, the first full buffer is staged, and transport begins when the next buffer fills or when Flush() is called at the normal event flush point.

Refactor primary insertion into a new stage_primaries API and keep the existing primary transport operator as staging followed by a step. Add focused Stepper coverage for staged primaries on host and device builds. Prompt: "PLEASE IMPLEMENT THIS PLAN: # Reviewable Async Primary Staging Plan ## Summary Implement the async primary-copy work as small, independently reviewable commits. Each chunk preserves existing behavior unless explicitly noted, and each commit should build and pass focused tests before moving on. ## Chunk 1: Add Stepper Staging API - Add `StepperInterface::stage_primaries(SpanConstPrimary)`. - Implement `Stepper<M>::stage_primaries` by moving the validation, `num_pending` counter update, and `ExtendFromPrimariesAction::insert` call out of `Stepper<M>::operator()(primaries)`. - Change `Stepper<M>::operator()(primaries)` to call `stage_primaries(primaries)` and then `(*this)()`. - No `LocalTransporter` behavior changes in this chunk. - Tests: add/update `Stepper.test.cc` coverage for `stage_primaries(...)` followed by `step()` on host and device where available. ## Chunk 2: Use Pinned LocalTransporter Buffers - Change `LocalTransporter` primary buffers from plain `std::vector<Primary>` to pinned vectors using `PinnedAllocator<Primary>`. - Keep current `Push` and `Flush` control flow unchanged: auto-flush still transports immediately. - Keep `dump_primaries_`, buffer accounting, and lost-primary accounting behavior unchanged. - Tests: compile accel target and run existing accel integration coverage available in the local build. ## Chunk 3: Add Copy Event Tracking - Introduce a small staged-primary state in `LocalTransporter` containing: - pinned host primaries, - copied buffer accounting, - optional `DeviceEvent copy_done`. - When staging to device, record `copy_done` after the async H2D copy on the state stream. - Use the event only to protect pinned host-buffer lifetime and reuse; do not use it to order stepping kernels on the same stream. - For host-only operation, keep the event null and behavior synchronous. - Tests: focused compile plus existing `DeviceEvent` and `Stepper` tests. ## Chunk 4: Defer Transport at Auto-Flush - Change device-mode `Push` so reaching `auto_flush_` stages the current pinned buffer asynchronously and keeps accepting Geant4 tracks. - If another full buffer appears while a staged batch is pending, call `Flush` first, then stage the new full buffer. - Change `Flush` so it stages any remaining current buffer, dumps from the staged host buffer if requested, then starts stepping with `(*step_)()` for staged primaries. - Clear/reuse staged host storage only after the copy event is ready or after stepping has returned. - Host-only behavior remains the current immediate transport path. ## Test Plan - After each chunk: run focused tests for touched code and confirm the target compiles. - Before each commit: run `pre-commit run`, re-add any modified files, and confirm the configured build still succeeds. - Commit each chunk separately with `scripts/dev/agent-commit.sh`. ## Assumptions - Only one staged primary batch is supported at a time. - Multi-batch device queueing is out of scope. - `DeviceEvent` is for host-buffer lifetime/progress tracking only; same-stream CUDA/HIP ordering handles copy-before-kernel execution." Assisted-by: codex (gpt-5)

Store LocalTransporter primaries in pinned host memory so existing stream-aware copies can run asynchronously on device builds. Preserve current flush timing and convert to the legacy event-writer vector type only when dumping primaries. Prompt: "PLEASE IMPLEMENT THIS PLAN: # Reviewable Async Primary Staging Plan ## Summary Implement the async primary-copy work as small, independently reviewable commits. Each chunk preserves existing behavior unless explicitly noted, and each commit should build and pass focused tests before moving on. ## Chunk 1: Add Stepper Staging API - Add `StepperInterface::stage_primaries(SpanConstPrimary)`. - Implement `Stepper<M>::stage_primaries` by moving the validation, `num_pending` counter update, and `ExtendFromPrimariesAction::insert` call out of `Stepper<M>::operator()(primaries)`. - Change `Stepper<M>::operator()(primaries)` to call `stage_primaries(primaries)` and then `(*this)()`. - No `LocalTransporter` behavior changes in this chunk. - Tests: add/update `Stepper.test.cc` coverage for `stage_primaries(...)` followed by `step()` on host and device where available. ## Chunk 2: Use Pinned LocalTransporter Buffers - Change `LocalTransporter` primary buffers from plain `std::vector<Primary>` to pinned vectors using `PinnedAllocator<Primary>`. - Keep current `Push` and `Flush` control flow unchanged: auto-flush still transports immediately. - Keep `dump_primaries_`, buffer accounting, and lost-primary accounting behavior unchanged. - Tests: compile accel target and run existing accel integration coverage available in the local build. ## Chunk 3: Add Copy Event Tracking - Introduce a small staged-primary state in `LocalTransporter` containing: - pinned host primaries, - copied buffer accounting, - optional `DeviceEvent copy_done`. - When staging to device, record `copy_done` after the async H2D copy on the state stream. - Use the event only to protect pinned host-buffer lifetime and reuse; do not use it to order stepping kernels on the same stream. - For host-only operation, keep the event null and behavior synchronous. - Tests: focused compile plus existing `DeviceEvent` and `Stepper` tests. ## Chunk 4: Defer Transport at Auto-Flush - Change device-mode `Push` so reaching `auto_flush_` stages the current pinned buffer asynchronously and keeps accepting Geant4 tracks. - If another full buffer appears while a staged batch is pending, call `Flush` first, then stage the new full buffer. - Change `Flush` so it stages any remaining current buffer, dumps from the staged host buffer if requested, then starts stepping with `(*step_)()` for staged primaries. - Clear/reuse staged host storage only after the copy event is ready or after stepping has returned. - Host-only behavior remains the current immediate transport path. ## Test Plan - After each chunk: run focused tests for touched code and confirm the target compiles. - Before each commit: run `pre-commit run`, re-add any modified files, and confirm the configured build still succeeds. - Commit each chunk separately with `scripts/dev/agent-commit.sh`. ## Assumptions - Only one staged primary batch is supported at a time. - Multi-batch device queueing is out of scope. - `DeviceEvent` is for host-buffer lifetime/progress tracking only; same-stream CUDA/HIP ordering handles copy-before-kernel execution." Assisted-by: codex (gpt-5)

Add staged primary state to LocalTransporter and route flushes through the Stepper staging API. Record a DeviceEvent after device primary copies so the pinned host buffer lifetime can be managed without changing flush timing. Prompt: "PLEASE IMPLEMENT THIS PLAN: Implement the async primary-copy work as small, independently reviewable commits. Each chunk preserves existing behavior unless explicitly noted, and each commit should build and pass focused tests before moving on. - Add `StepperInterface::stage_primaries(SpanConstPrimary)`. - Implement `Stepper<M>::stage_primaries` by moving the validation, `num_pending` counter update, and `ExtendFromPrimariesAction::insert` call out of `Stepper<M>::operator()(primaries)`. - Change `Stepper<M>::operator()(primaries)` to call `stage_primaries(primaries)` and then `(*this)()`. - No `LocalTransporter` behavior changes in this chunk. - Tests: add/update `Stepper.test.cc` coverage for `stage_primaries(...)` followed by `step()` on host and device where available. - Change `LocalTransporter` primary buffers from plain `std::vector<Primary>` to pinned vectors using `PinnedAllocator<Primary>`. - Keep current `Push` and `Flush` control flow unchanged: auto-flush still transports immediately. - Keep `dump_primaries_`, buffer accounting, and lost-primary accounting behavior unchanged. - Tests: compile accel target and run existing accel integration coverage available in the local build. - Introduce a small staged-primary state in `LocalTransporter` containing: - pinned host primaries, - copied buffer accounting, - optional `DeviceEvent copy_done`. - When staging to device, record `copy_done` after the async H2D copy on the state stream. - Use the event only to protect pinned host-buffer lifetime and reuse; do not use it to order stepping kernels on the same stream. - For host-only operation, keep the event null and behavior synchronous. - Tests: focused compile plus existing `DeviceEvent` and `Stepper` tests. - Change device-mode `Push` so reaching `auto_flush_` stages the current pinned buffer asynchronously and keeps accepting Geant4 tracks. - If another full buffer appears while a staged batch is pending, call `Flush` first, then stage the new full buffer. - Change `Flush` so it stages any remaining current buffer, dumps from the staged host buffer if requested, then starts stepping with `(*step_)()` for staged primaries. - Clear/reuse staged host storage only after the copy event is ready or after stepping has returned. - Host-only behavior remains the current immediate transport path. - After each chunk: run focused tests for touched code and confirm the target compiles. - Before each commit: run `pre-commit run`, re-add any modified files, and confirm the configured build still succeeds. - Commit each chunk separately with `scripts/dev/agent-commit.sh`. - Only one staged primary batch is supported at a time. - Multi-batch device queueing is out of scope. - `DeviceEvent` is for host-buffer lifetime/progress tracking only; same-stream CUDA/HIP ordering handles copy-before-kernel execution." Assisted-by: codex (gpt-5)

Change device-mode LocalTransporter auto-flush to stage full pinned primary buffers asynchronously and continue accepting tracks. Flush now drains staged and remaining buffered batches while preserving host-only immediate transport. Prompt: "PLEASE IMPLEMENT THIS PLAN: Implement the async primary-copy work as small, independently reviewable commits. Each chunk preserves existing behavior unless explicitly noted, and each commit should build and pass focused tests before moving on. - Add `StepperInterface::stage_primaries(SpanConstPrimary)`. - Implement `Stepper<M>::stage_primaries` by moving the validation, `num_pending` counter update, and `ExtendFromPrimariesAction::insert` call out of `Stepper<M>::operator()(primaries)`. - Change `Stepper<M>::operator()(primaries)` to call `stage_primaries(primaries)` and then `(*this)()`. - No `LocalTransporter` behavior changes in this chunk. - Tests: add/update `Stepper.test.cc` coverage for `stage_primaries(...)` followed by `step()` on host and device where available. - Change `LocalTransporter` primary buffers from plain `std::vector<Primary>` to pinned vectors using `PinnedAllocator<Primary>`. - Keep current `Push` and `Flush` control flow unchanged: auto-flush still transports immediately. - Keep `dump_primaries_`, buffer accounting, and lost-primary accounting behavior unchanged. - Tests: compile accel target and run existing accel integration coverage available in the local build. - Introduce a small staged-primary state in `LocalTransporter` containing: - pinned host primaries, - copied buffer accounting, - optional `DeviceEvent copy_done`. - When staging to device, record `copy_done` after the async H2D copy on the state stream. - Use the event only to protect pinned host-buffer lifetime and reuse; do not use it to order stepping kernels on the same stream. - For host-only operation, keep the event null and behavior synchronous. - Tests: focused compile plus existing `DeviceEvent` and `Stepper` tests. - Change device-mode `Push` so reaching `auto_flush_` stages the current pinned buffer asynchronously and keeps accepting Geant4 tracks. - If another full buffer appears while a staged batch is pending, call `Flush` first, then stage the new full buffer. - Change `Flush` so it stages any remaining current buffer, dumps from the staged host buffer if requested, then starts stepping with `(*step_)()` for staged primaries. - Clear/reuse staged host storage only after the copy event is ready or after stepping has returned. - Host-only behavior remains the current immediate transport path. - After each chunk: run focused tests for touched code and confirm the target compiles. - Before each commit: run `pre-commit run`, re-add any modified files, and confirm the configured build still succeeds. - Commit each chunk separately with `scripts/dev/agent-commit.sh`. - Only one staged primary batch is supported at a time. - Multi-batch device queueing is out of scope. - `DeviceEvent` is for host-buffer lifetime/progress tracking only; same-stream CUDA/HIP ordering handles copy-before-kernel execution." Assisted-by: codex (gpt-5)

Reserve both LocalTransporter pinned primary buffers to the auto-flush size at construction and reuse their storage with swaps. Avoid move assignment and temporary pinned buffers while draining staged work so stepping does not reallocate primary host storage. Prompt: "Allocating pinned memory is expensive. Can you pre-allocate both buffer to `auto_flush_` size, and make sure that we never need to re-allocate during the stepping" Assisted-by: codex (gpt-5)

Restore staged-batch transport semantics after the develop rebase and clear Geant track reconstruction state at the end of LocalTransporter flushes. Prompt: "I've had to rebase on develop which modified [LocalTransporter.cc](src/accel/LocalTransporter.cc) . Can you verify that the implementation is still correct?" Assisted-by: codex (gpt-5)

Clarify that Stepper primary staging does not execute transport, document the LocalTransporter copy-event lifetime contract, and mark the blocking backpressure point for future non-blocking stepper progress. Prompt: "Please add this documentation" Assisted-by: codex (gpt-5)

Keep Geant4 track reconstruction data alive after partial staged flushes until all buffered primaries with acquired reconstruction IDs have been transported. Prompt: "Please deffer clearing track_reconstruction until there are no remaining buffered primaries" Assisted-by: Codex (GPT-5)

Replace the ambiguous boolean argument to LocalTransporter::flush_impl with a scoped FlushMode enum so call sites identify whether only staged primaries or both staged and buffered primaries should be transported. Prompt: "Please use an enum as `flush_impl` argument" Assisted-by: Codex (GPT-5)

chatgpt-codex-connector

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 10631d2aad

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

Open a pull request for review
Mark a draft as ready
Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Copilot

Pull request overview

This PR introduces an explicit “primary staging” phase to decouple primary upload from stepping, enabling double-buffered (pinned) host-side accumulation in LocalTransporter so GPU H2D copies can overlap with subsequent Geant4 Push calls.

Changes:

Add StepperInterface::stage_primaries and implement it in Stepper to validate/insert primaries without executing a transport step.
Rework LocalTransporter buffering on GPU to use a staged pinned buffer plus a DeviceEvent to protect host buffer lifetime during async H2D copies, and adjust flush behavior accordingly.
Add unit tests covering the new staged-primaries path for host and device steppers.

Reviewed changes

Copilot reviewed 5 out of 5 changed files in this pull request and generated 2 comments.

Show a summary per file

File	Description
`test/celeritas/global/Stepper.test.cc`	Adds tests to verify primaries can be staged and then transported on the first subsequent step (host/device).
`src/celeritas/global/Stepper.hh`	Extends the stepper interface with `stage_primaries`.
`src/celeritas/global/Stepper.cc`	Implements `stage_primaries` and refactors `operator()(primaries)` to build on it.
`src/accel/LocalTransporter.hh`	Introduces pinned primary buffers, staged-buffer bookkeeping, and new flush modes.
`src/accel/LocalTransporter.cc`	Implements staging/clearing logic, double-buffered GPU behavior, and updated flush loop.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Handle lost-only LocalTransporter flushes so out-of-bounds primary counters are logged, accumulated, and reset even when no tracks were buffered. Validate that Stepper::stage_primaries is not called while another staged batch is pending, and cover that precondition in the Stepper tests. Prompt: "review and address the two comments in the PR" Assisted-by: Codex (GPT-5)

Copilot

Pull request overview

Copilot reviewed 5 out of 5 changed files in this pull request and generated 2 comments.

codecov · 2026-05-19T23:35:46Z

Codecov Report

❌ Patch coverage is 67.52137% with 38 lines in your changes missing coverage. Please review.
✅ Project coverage is 87.19%. Comparing base (8dd8460) to head (f405e4a).
⚠️ Report is 18 commits behind head on develop.

Files with missing lines	Patch %	Lines
src/accel/LocalTransporter.cc	63.63%	26 Missing and 10 partials ⚠️
src/accel/LocalTransporter.hh	84.61%	1 Missing and 1 partial ⚠️

Additional details and impacted files

@@             Coverage Diff             @@
##           develop    #2406      +/-   ##
===========================================
- Coverage    87.23%   87.19%   -0.04%     
===========================================
  Files         1396     1396              
  Lines        44049    44122      +73     
  Branches     13791    13343     -448     
===========================================
+ Hits         38426    38473      +47     
+ Misses        4548     4411     -137     
- Partials      1075     1238     +163

Files with missing lines	Coverage Δ
src/celeritas/global/Stepper.cc	`100.00% <100.00%> (ø)`
src/celeritas/global/Stepper.hh	`94.11% <ø> (ø)`
src/accel/LocalTransporter.hh	`90.90% <84.61%> (-4.75%)`	⬇️
src/accel/LocalTransporter.cc	`68.75% <63.63%> (-2.01%)`	⬇️

... and 112 files with indirect coverage changes

🚀 New features to boost your workflow:

❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

github-actions · 2026-05-19T23:40:18Z

Test summary

5 702 files 9 243 suites 19m 10s ⏱️
2 314 tests 2 271 ✅ 43 💤 0 ❌
32 650 runs 32 473 ✅ 177 💤 0 ❌

Results for commit f405e4a.

♻️ This comment has been updated with latest results.

sethrj

Still working through the LocalTransporter changes.

Could you please clarify (here and/or LT class docs) the nomenclature and steps for CPU and GPU:

G4 primary
-> validated and converted local Primary
-> push back onto "staged" (?)
-> asynchronous copy to "buffered" (?)
-> added to pending primaries buffer  in GenPrimaryAction (?)
-> track initializer
-> track

There's too many staging/buffering steps to just simply call it buffer/stage now 😬

esseivaju · 2026-05-20T17:17:56Z

Still working through the LocalTransporter changes.

Could you please clarify (here and/or LT class docs) the nomenclature and steps for CPU and GPU:
G4 primary
-> validated and converted local Primary
-> push back onto "staged" (?)
-> asynchronous copy to "buffered" (?)
-> added to pending primaries buffer  in GenPrimaryAction (?)
-> track initializer
-> track
There's too many staging/buffering steps to just simply call it buffer/stage now 😬

The LocalTransporter now holds two host buffers, primary and staging. Primaries are pushed by Geant4 to the former, and a the later is used for HTD memory transfer. The sequence is as follow:

G4 primary
-> validated and converted local Primary
-> push primary onto the primary buffer
-> Scenario 1: event ends without reaching auto-flush
    -> Flush staging and primary buffers
-> Scenario 2: primary buffer size >= auto_flush
    -> 2a: CPU-only
        -> Flush (transport) the primary buffer
    -> 2b: GPU
        -> Swap primary and staging buffer
        -> Stepper stages the buffer and queues HTD copy on the CUDA stream
        -> No flush / transport happens in that case, only HTD transfer
-> Scenario 3: primary buffer size >= auto_flush and stepper has a staged buffer
    -> Flush the staged buffer
    -> Scenario 2b

Rename LocalTransporter host buffers around primary and staging semantics, share the same PrimaryBuffer bookkeeping structure for both, and add a helper for interpreting flush modes. Prompt: "Review and address the open comment by implementing a helper function for flush mode, and reorganizing `LocalTransporter` data members. Try to share struc between primary and staging buffer when possible" Assisted-by: codex (gpt-5)

esseivaju added 9 commits May 19, 2026 13:56

esseivaju requested review from LSchwiebert, amandalund, Copilot and sethrj May 19, 2026 22:35

esseivaju added enhancement New feature or request core Software engineering infrastructure (corecel) ai-assisted Generated/refactored substantially with agentic/LLM AI tools labels May 19, 2026

Copilot started reviewing on behalf of esseivaju May 19, 2026 22:35 View session

chatgpt-codex-connector Bot reviewed May 19, 2026

View reviewed changes

Comment thread src/accel/LocalTransporter.cc Outdated

Copilot AI reviewed May 19, 2026

View reviewed changes

Comment thread src/celeritas/global/Stepper.cc

Comment thread src/accel/LocalTransporter.cc Outdated

esseivaju added 2 commits May 19, 2026 15:47

fix condition

0d77420

esseivaju requested a review from Copilot May 19, 2026 23:08

Copilot started reviewing on behalf of esseivaju May 19, 2026 23:09 View session

Copilot AI reviewed May 19, 2026

View reviewed changes

Comment thread src/celeritas/global/Stepper.cc

Comment thread src/accel/LocalTransporter.cc Outdated

sethrj reviewed May 20, 2026

View reviewed changes

Comment thread src/accel/LocalTransporter.hh Outdated

Comment thread src/accel/LocalTransporter.cc Outdated

Comment thread src/accel/LocalTransporter.cc Outdated

Comment thread src/accel/LocalTransporter.hh Outdated

esseivaju added 2 commits May 20, 2026 10:30

rename

2caa56e

reuse device-event

6745c80

esseivaju added 2 commits May 20, 2026 10:42

fix

f405e4a

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Asynchronously copy primaries to device in LocalTransporter#2406

Asynchronously copy primaries to device in LocalTransporter#2406
esseivaju wants to merge 15 commits into
celeritas-project:developfrom
esseivaju:async-push

esseivaju commented May 19, 2026

Uh oh!

chatgpt-codex-connector Bot left a comment

Uh oh!

Uh oh!

Copilot AI left a comment

Uh oh!

Uh oh!

Uh oh!

Copilot AI left a comment

Uh oh!

Uh oh!

Uh oh!

codecov Bot commented May 19, 2026 •

edited

Loading

Uh oh!

github-actions Bot commented May 19, 2026 •

edited

Loading

Uh oh!

sethrj left a comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

esseivaju commented May 20, 2026 •

edited

Loading

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

Uh oh!

Conversation

esseivaju commented May 19, 2026

Uh oh!

chatgpt-codex-connector Bot left a comment

Choose a reason for hiding this comment

💡 Codex Review

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Uh oh!

Uh oh!

codecov Bot commented May 19, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

github-actions Bot commented May 19, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Test summary

Uh oh!

sethrj left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

esseivaju commented May 20, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

codecov Bot commented May 19, 2026 •

edited

Loading

github-actions Bot commented May 19, 2026 •

edited

Loading

esseivaju commented May 20, 2026 •

edited

Loading