Skip to content

feat: Add server option "--allow-client-shm" (default false) controling client shared memory register/unregister - #8721

Merged
yinggeh merged 14 commits into
mainfrom
yinggeh/tri-32-lock-down-shared-memory-to-avoid-future-psirt-issues-in
Apr 15, 2026
Merged

feat: Add server option "--allow-client-shm" (default false) controling client shared memory register/unregister#8721
yinggeh merged 14 commits into
mainfrom
yinggeh/tri-32-lock-down-shared-memory-to-avoid-future-psirt-issues-in

Conversation

@yinggeh

@yinggeh yinggeh commented Apr 2, 2026

Copy link
Copy Markdown
Contributor

What does the PR do?

Introduces server option "--allow-client-shm=" (default false) to control client shm registration explicitly.
For example, if --allow-client-shm=false or unset (default)
✅ GET v2/systemsharedmemory[/region/${REGION_NAME}]/status
❌ POST v2/systemsharedmemory/region/${REGION_NAME}/register
❌ POST v2/systemsharedmemory[/region/${REGION_NAME}]/unregister
✅ GET v2/cudasharedmemory[/region/${REGION_NAME}]/status
❌ POST v2/cudasharedmemory/region/${REGION_NAME}/register
❌ POST v2/cudasharedmemory[/region/${REGION_NAME}]/unregister

if --allow-client-shm=true
✅ GET v2/systemsharedmemory[/region/${REGION_NAME}]/status
✅ POST v2/systemsharedmemory/region/${REGION_NAME}/register
✅ POST v2/systemsharedmemory[/region/${REGION_NAME}]/unregister
✅ GET v2/cudasharedmemory[/region/${REGION_NAME}]/status
✅ POST v2/cudasharedmemory/region/${REGION_NAME}/register
✅ POST v2/cudasharedmemory[/region/${REGION_NAME}]/unregister

Checklist

  • PR title reflects the change and is of format <commit_type>: <Title>
  • Changes are described in the pull request.
  • Populated github labels field
  • Related issues are referenced.
  • Added test plan and verified test passes.
  • Verified that the PR passes existing CI.
  • Verified copyright is correct on all changed files.
  • Added succinct git squash message before merging ref.
  • All template sections are filled out.
  • Optional: Additional screenshots for behavior/output changes with before/after.

Commit Type:

Check the conventional commit type
box here and add the label to the github PR.

  • feat

Related PRs:

Where should the reviewer start?

Test plan:

  • CI Pipeline ID:
    48256373

Caveats:

Background

Related Issues: (use one of the action keywords Closes / Fixes / Resolves / Relates to)

  • closes GitHub issue: #xxx

@yinggeh yinggeh self-assigned this Apr 2, 2026

@whoisj whoisj left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good to me. Just have 1 ask before approval.

Comment thread src/shared_memory_manager.h
Comment thread src/shared_memory_manager.h
@yinggeh
yinggeh requested a review from mudit-eng April 8, 2026 19:45
@yinggeh yinggeh changed the title feat: CLI option "--allow-client-shm" enable/disable client shm registration (default false) feat: Add server option "--allow-client-shm" (default false) controling client shared memory register/unregister Apr 9, 2026
@yinggeh
yinggeh requested a review from whoisj April 9, 2026 00:10

@whoisj whoisj left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Look good. Only ask is to correct the markdown to use the more modern markdown for alerts.

Also, a thought about using a constant in place of repeated literals.

Comment thread docs/protocol/extension_shared_memory.md Outdated
Comment thread src/http_server.cc Outdated
Comment thread src/shared_memory_manager.cc Outdated
@yinggeh
yinggeh requested a review from whoisj April 10, 2026 19:34
whoisj
whoisj previously approved these changes Apr 10, 2026
@yinggeh
yinggeh requested a review from Copilot April 10, 2026 23:26

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Adds a new server CLI switch (--allow-client-shm=<bool>, default false) to explicitly control whether clients may register/unregister shared-memory regions, while keeping shared-memory status queries available and leaving backend-internal shared memory unaffected.

Changes:

  • Adds --allow-client-shm parsing/plumbing and passes the value into SharedMemoryManager.
  • Rejects HTTP + gRPC shared-memory register/unregister calls when client SHM is disabled (default).
  • Updates docs and QA to cover the new default behavior and to enable SHM explicitly for SHM-dependent tests.

Reviewed changes

Copilot reviewed 15 out of 15 changed files in this pull request and generated 2 comments.

Show a summary per file
File Description
src/shared_memory_manager.h Adds constructor parameter + accessor for client-SHM enablement.
src/main.cc Instantiates SharedMemoryManager with the parsed CLI flag.
src/http_server.cc Blocks SHM register/unregister endpoints when client SHM is disabled.
src/grpc/grpc_server.cc Blocks SHM register/unregister RPCs when client SHM is disabled.
src/common.h Introduces a shared error string for the disabled-client-SHM case.
src/command_line_parser.h Adds allow_client_shm_ to server parameters + option grouping storage.
src/command_line_parser.cc Adds CLI option definition/group and parses --allow-client-shm.
qa/L0_shared_memory/test.sh Adds coverage for default-disabled behavior; enables SHM for existing tests.
qa/L0_shared_memory/shared_memory_test.py Adds test asserting register/unregister are rejected by default.
qa/L0_cuda_shared_memory/test.sh Adds coverage for default-disabled behavior; enables SHM for existing tests.
qa/L0_cuda_shared_memory/cuda_shared_memory_test.py Adds test asserting CUDA SHM register/unregister are rejected by default.
qa/L0_sequence_batcher/test.sh Enables --allow-client-shm=true when SHM-related subtests run.
qa/L0_infer/test.sh Enables --allow-client-shm=true when SHM-related subtests run.
qa/L0_batcher/test.sh Enables --allow-client-shm=true for CUDA-SHM tests; adjusts warmup + fixes typo.
docs/protocol/extension_shared_memory.md Documents the new default-disabled requirement for client SHM usage.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread qa/L0_batcher/test.sh Outdated
Comment thread qa/L0_cuda_shared_memory/test.sh Outdated
yinggeh added 2 commits April 10, 2026 16:57
…psirt-issues-in' of github.com:triton-inference-server/server into yinggeh/tri-32-lock-down-shared-memory-to-avoid-future-psirt-issues-in
@yinggeh yinggeh added the PR: feat A new feature label Apr 11, 2026
yinggeh added 2 commits April 13, 2026 18:28
… yinggeh/tri-32-lock-down-shared-memory-to-avoid-future-psirt-issues-in
@yinggeh
yinggeh requested a review from whoisj April 14, 2026 01:32

@whoisj whoisj left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@yinggeh
yinggeh merged commit 546a787 into main Apr 15, 2026
3 checks passed
@yinggeh
yinggeh deleted the yinggeh/tri-32-lock-down-shared-memory-to-avoid-future-psirt-issues-in branch April 15, 2026 00:21
yinggeh added a commit that referenced this pull request Apr 15, 2026
mc-nv pushed a commit that referenced this pull request Apr 15, 2026
mc-nv pushed a commit that referenced this pull request Apr 28, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

PR: feat A new feature

Development

Successfully merging this pull request may close these issues.

3 participants