Skip to content

fix: enforce inference restriction in SageMaker MME invoke handler - #8833

Merged
Vinya567 merged 1 commit into
mainfrom
vinyak/sagemaker-mme-inference-restriction
Jun 11, 2026
Merged

fix: enforce inference restriction in SageMaker MME invoke handler#8833
Vinya567 merged 1 commit into
mainfrom
vinyak/sagemaker-mme-inference-restriction

Conversation

@Vinya567

Copy link
Copy Markdown
Contributor

What does the PR do?

Adds the missing --http-restricted-api=inference enforcement to the SageMaker MME /invoke handler. SagemakerAPIServer::SageMakerMMEHandleInfer was not calling the RETURN_AND_RESPOND_IF_RESTRICTED macro, so the four sibling SageMaker control endpoints (LIST/GET/LOAD/UNLOAD) and the regular HTTP /v2/<m>/infer path enforced the configured header while MME /invoke silently accepted requests without it. Adds the macro call at the head of the handler and a regression block in qa/L0_sagemaker/test.sh.

Checklist

  • PR title reflects the change and is of format <type>: <description>
  • Changes are described in the pull request.
  • Related issues are referenced.
  • Populated github labels field
  • Added test plan and verified test passes.
  • Verified that the PR passes existing CI.
  • Verified copyright is correct on all changed files.
  • Added succinct git squash message before merging
  • All template sections are filled out.

Commit Type:

  • fix

Related PRs:

N/A

Where should the reviewer start?

src/sagemaker_server.cc — one new line at the head of SageMakerMMEHandleInfer:

RETURN_AND_RESPOND_IF_RESTRICTED(req, RestrictedCategory::INFERENCE);

This matches the four existing call-sites in the same file (LIST/GET/LOAD/UNLOAD, all gated on RestrictedCategory::MODEL_REPOSITORY) and HTTPAPIServer::HandleInfer (gated on RestrictedCategory::INFERENCE).

qa/L0_sagemaker/test.sh — new block that launches Triton with --http-restricted-api=inference:X-Auth=secret and exercises POST /models/<m>/invoke and POST /invocations with and without the X-Auth header. Expects 403 without and 200 with.

Test plan:

Reproduced the BEFORE behavior on nvcr.io/nvidia/tritonserver:26.05-py3: POST /models/<m>/invoke without X-Auth returned 200 OK despite --http-restricted-api=inference:X-Auth=secret. After the fix the same request returns 403 Forbidden with {"error":"This API is restricted, expecting header 'X-Auth'"}.

The new regression block emitted the expected codes on internal CI:

POST /models/sm_mme_model/invoke   (no X-Auth)   ->  403
POST /models/sm_mme_model/invoke   (X-Auth)      ->  200
POST /invocations                  (no X-Auth)   ->  403
POST /invocations                  (X-Auth)      ->  200
  • L0_sagemaker--base: passed (existing tests + new block)
  • L0_http--base: passed (existing http_restricted_api_test.py confirms the RETURN_AND_RESPOND_IF_RESTRICTED macro is unchanged on non-SageMaker paths)
  • CI Pipeline ID: 54470604

Caveats:

None.

Background

SagemakerAPIServer extends HTTPAPIServer and shares the RestrictedFeatures map via main.cc (which copies http_restricted_apis_ into the SagemakerAPIServer constructor). The four sibling SageMaker control endpoints (LIST/GET/LOAD/UNLOAD) gate on RestrictedCategory::MODEL_REPOSITORY; the regular HTTP /v2/<m>/infer path gates on RestrictedCategory::INFERENCE. The MME /invoke handler — which performs inference — was missing the gate. This PR adds the fifth call-site at the head of SageMakerMMEHandleInfer (before any model resolution or dispatch) so behavior is consistent across all HTTP inference paths.

Related Issues: (use Closes / Fixes / Resolves / Relates to)

N/A

SagemakerAPIServer::SageMakerMMEHandleInfer was missing the
RETURN_AND_RESPOND_IF_RESTRICTED gate that HTTPAPIServer::HandleInfer
and the SageMaker LIST/GET/LOAD/UNLOAD handlers use. Operators passing
--http-restricted-api=inference,<hdr>,<val> expected the MME /invoke
path to honor the restriction, but the handler silently accepted
requests without the configured header. Add the macro call at the head
of the handler so MME /invoke matches the rest of the HTTP inference
surface.

Extends qa/L0_sagemaker/test.sh with a regression block that exercises
POST /models/<m>/invoke and POST /invocations both with and without
the configured restriction header.
@Vinya567 Vinya567 self-assigned this Jun 11, 2026

@whoisj whoisj left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@Vinya567
Vinya567 merged commit b58e983 into main Jun 11, 2026
3 checks passed
@Vinya567
Vinya567 deleted the vinyak/sagemaker-mme-inference-restriction branch June 11, 2026 21:03
mc-nv pushed a commit that referenced this pull request Jun 13, 2026
…8833)

SagemakerAPIServer::SageMakerMMEHandleInfer was not invoking
RETURN_AND_RESPOND_IF_RESTRICTED, so the SageMaker MME /invoke path
silently accepted requests without the configured inference header.
Adds the macro call at the head of the handler and a regression test
in qa/L0_sagemaker/test.sh.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Development

Successfully merging this pull request may close these issues.

2 participants