Skip to content

feat: add experimental rust request logging masking extension#4030

Merged
jonpspri merged 10 commits intomainfrom
fix/rust-request-logging-sidecar
Apr 13, 2026
Merged

feat: add experimental rust request logging masking extension#4030
jonpspri merged 10 commits intomainfrom
fix/rust-request-logging-sidecar

Conversation

@lucarlig
Copy link
Copy Markdown
Collaborator

@lucarlig lucarlig commented Apr 3, 2026

Feature / Enhancement PR

Epic / Issue

#4036


Summary (1-2 sentences)

Adds an experimental opt-in PyO3-based Rust native extension for request logging sensitive-data masking in mcpgateway/middleware/request_logging_middleware.py. When experimental_rust_request_logging_masking_enabled=true, the request logging path uses the imported request_logging_masking_native_extension module to accelerate header masking, recursive payload masking, and JSON-body logging.


Implementation Details

  • Introduces tools_rust/request_logging_masking_native_extension/ as a PyO3 native extension that exposes mask_sensitive_data() and mask_sensitive_headers() to Python.
  • Adds a dedicated native JSON fast path via mask_sensitive_json_bytes() so the middleware can avoid Python orjson.loads(...) -> mask -> orjson.dumps(...) on JSON request bodies.
  • Adds a bounded thread-local LRU key-sensitivity cache in the native extension so repeated keys can be reused across calls without unbounded cache growth.
  • Adds the Python package import shim for request_logging_masking_native_extension so editable installs resolve the exported native functions correctly.
  • Adds the experimental_rust_request_logging_masking_enabled feature flag in mcpgateway/config.py.
  • Updates mcpgateway/middleware/request_logging_middleware.py so the public masking helpers lazily import and delegate to the PyO3 native extension when the flag is enabled.
  • Falls back to the Python masking path if the native extension is unavailable at runtime, and caches the failed import to avoid repeated import attempts.
  • Keeps the default path unchanged when the feature flag is off.
  • Adds parity and enabled-path tests, including malformed-Unicode cookie coverage, non-string nested keys, depth-limit parity checks, Python fallback coverage when the native extension import fails, and the native JSON fast path.
  • Keeps tests/performance/test_request_logging_masking_native_extension_benchmark.py focused on the public feature-flagged API path and direct-native comparison.

Checks

  • make lint passes
  • make test passes
  • CHANGELOG updated (if user-facing)

Notes

Latest microbenchmark run

Validated on April 13, 2026 using the current branch code, the real middleware masking source, and the built PyO3 native extension. The benchmark values below are all from this latest run.

Scenario Python median Native extension public median Direct native median Public-path speedup
nested_payload_masking 28.096 ms 0.962 ms 0.960 ms 29.21x
headers_masking 2.569 ms 0.164 ms 0.164 ms 15.67x

Cold-start and warm-path visibility from the same run:

  • nested_payload_masking: public-path overhead vs direct native 0.2%
  • headers_masking: public-path overhead vs direct native 0.0%

JSON logging fast path benchmark

The latest build also includes a dedicated native JSON-body logging path in the middleware.

Scenario Python median Native JSON-bytes median Speedup
json_logging_path 0.032 ms 0.003 ms 9.21x

Prior focused request-logging loadtest

Previously validated on April 7, 2026 in a Linux container using a focused Locust workload against a FastAPI app with RequestLoggingMiddleware enabled for detailed request logging. This run used a heavy nested JSON body and sensitive headers aligned with the request-logging benchmark payloads so the measured throughput reflects the code changed in this PR.

Scenario Path RPS Failures Avg P95
Stable (50 users, 20s) Python 24.18 0 1807.68 ms 2500 ms
Stable (50 users, 20s) Rust native extension 230.30 0 190.16 ms 280 ms
Stress (200 users, 30s) Python 21.37 0 6038.87 ms 13000 ms
Stress (200 users, 30s) Rust native extension 148.02 0 1119.42 ms 1900 ms

Focused-loadtest takeaway from the April 7 run: when the workload is dominated by detailed request logging and masking, the Rust native extension shows a large end-to-end benefit, improving throughput by 9.53x in the stable scenario and 6.93x in the stress scenario.

@lucarlig lucarlig changed the title feat: add experimental rust request logging masking sidecar feat: add experimental rust request logging masking extension Apr 3, 2026
@lucarlig lucarlig added experimental Experimental features, test proposed MCP Specification changes rust Rust programming labels Apr 3, 2026
@lucarlig lucarlig marked this pull request as ready for review April 3, 2026 15:29
@lucarlig lucarlig force-pushed the fix/rust-request-logging-sidecar branch from 9bd2d15 to de776fc Compare April 3, 2026 15:34
@lucarlig lucarlig force-pushed the fix/rust-request-logging-sidecar branch from de776fc to 413a5fd Compare April 7, 2026 09:17
@lucarlig lucarlig requested a review from gandhipratik203 April 7, 2026 09:22
@lucarlig lucarlig marked this pull request as draft April 7, 2026 10:10
@lucarlig lucarlig marked this pull request as ready for review April 7, 2026 21:10
@lucarlig lucarlig force-pushed the fix/rust-request-logging-sidecar branch 3 times, most recently from f9d014d to 7b2ca6b Compare April 8, 2026 08:51
@lucarlig lucarlig requested a review from dawid-nowak April 8, 2026 09:52
@lucarlig lucarlig added the release-fix Critical bugfix required for the release label Apr 8, 2026
@dima-zakharov
Copy link
Copy Markdown
Collaborator

Cache lifetime: The HashMap cache is created per function call and discarded. For large payloads with repeated keys, consider whether a thread-local or module-level LRU cache would be more efficient.

dima-zakharov
dima-zakharov previously approved these changes Apr 9, 2026
@lucarlig
Copy link
Copy Markdown
Collaborator Author

@dima-zakharov I addressed the cache-lifetime point in the native extension.

The key-sensitivity lookup now uses a bounded thread-local LRU cache, while keeping the existing per-request cache as the first lookup. That keeps repeated keys hot across calls without letting the cache grow unbounded.

I also added the missing Python package shim so the extension loads through the intended request_logging_masking_native_extension import path during editable installs.

Verification from this run:

  • cargo test --manifest-path tools_rust/request_logging_masking_native_extension/Cargo.toml
  • uv run python tests/performance/test_request_logging_masking_native_extension_benchmark.py
  • nested_payload_masking: Python 28.096 ms, public native 0.962 ms, direct native 0.960 ms
  • headers_masking: Python 2.569 ms, public native 0.164 ms, direct native 0.164 ms

Signed-off-by: lucarlig <luca.carlig@ibm.com>
Signed-off-by: lucarlig <luca.carlig@ibm.com>
Signed-off-by: lucarlig <luca.carlig@ibm.com>
Signed-off-by: lucarlig <luca.carlig@ibm.com>
Signed-off-by: lucarlig <luca.carlig@ibm.com>
Signed-off-by: lucarlig <luca.carlig@ibm.com>
Signed-off-by: lucarlig <luca.carlig@ibm.com>
Signed-off-by: lucarlig <luca.carlig@ibm.com>
@lucarlig lucarlig force-pushed the fix/rust-request-logging-sidecar branch from 1229d76 to 0a1cd59 Compare April 13, 2026 09:41
Signed-off-by: lucarlig <luca.carlig@ibm.com>
@lucarlig lucarlig requested a review from dima-zakharov April 13, 2026 10:01
dima-zakharov
dima-zakharov previously approved these changes Apr 13, 2026
@jonpspri jonpspri merged commit 086f81a into main Apr 13, 2026
28 checks passed
@jonpspri jonpspri deleted the fix/rust-request-logging-sidecar branch April 13, 2026 12:17
claudia-gray pushed a commit that referenced this pull request Apr 13, 2026
* feat: add rust request logging masking sidecar

Signed-off-by: lucarlig <luca.carlig@ibm.com>

* perf: cache key sensitivity in rust masking sidecar

Signed-off-by: lucarlig <luca.carlig@ibm.com>

* feat: restore pyo3 request logging path

Signed-off-by: lucarlig <luca.carlig@ibm.com>

* refactor: rename request logging native extension

Signed-off-by: lucarlig <luca.carlig@ibm.com>

* perf: add native json request logging fast path

Signed-off-by: lucarlig <luca.carlig@ibm.com>

* test: cover request logging native fallback paths

Signed-off-by: lucarlig <luca.carlig@ibm.com>

* feat: generate stubs for request logging native extension

Signed-off-by: lucarlig <luca.carlig@ibm.com>

* perf: reuse request logging key sensitivity cache

Signed-off-by: lucarlig <luca.carlig@ibm.com>

* chore: normalize detect-secrets baseline after rebase

Signed-off-by: lucarlig <luca.carlig@ibm.com>

---------

Signed-off-by: lucarlig <luca.carlig@ibm.com>
gcgoncalves pushed a commit that referenced this pull request Apr 23, 2026
* feat: add rust request logging masking sidecar

Signed-off-by: lucarlig <luca.carlig@ibm.com>

* perf: cache key sensitivity in rust masking sidecar

Signed-off-by: lucarlig <luca.carlig@ibm.com>

* feat: restore pyo3 request logging path

Signed-off-by: lucarlig <luca.carlig@ibm.com>

* refactor: rename request logging native extension

Signed-off-by: lucarlig <luca.carlig@ibm.com>

* perf: add native json request logging fast path

Signed-off-by: lucarlig <luca.carlig@ibm.com>

* test: cover request logging native fallback paths

Signed-off-by: lucarlig <luca.carlig@ibm.com>

* feat: generate stubs for request logging native extension

Signed-off-by: lucarlig <luca.carlig@ibm.com>

* perf: reuse request logging key sensitivity cache

Signed-off-by: lucarlig <luca.carlig@ibm.com>

* chore: normalize detect-secrets baseline after rebase

Signed-off-by: lucarlig <luca.carlig@ibm.com>

---------

Signed-off-by: lucarlig <luca.carlig@ibm.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

experimental Experimental features, test proposed MCP Specification changes release-fix Critical bugfix required for the release rust Rust programming

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants