Move appsec communication to sidecar

[cataphract]

Move AppSec communication to the sidecar

Summary

The AppSec helper already runs as a shared library loaded inside the sidecar
process (the sidecar dlopens it via ddog_sidecar_enable_appsec). But until
now the extension did not talk to it through the sidecar: the helper opened its
own dedicated Unix domain socket, and the extension connected to that socket
directly, side-channeling past the sidecar's own transport. A lock file was used
to single-instance the helper. This PR removes that side-channel socket and lock
file.

The extension now reaches the helper by sending messages through the existing
sidecar transport — the same channel already used for traces, telemetry, etc.
The AppSec request/response payloads (the dds-framed msgpack messages:
client_init, request_init, request_shutdown, …) are themselves unchanged,
but they are no longer exchanged over a raw socket: they are now wrapped as a
new message type inside the existing sidecar communication protocol, and the
sidecar unwraps them and forwards them to the helper (and routes the reply
back).

As a consequence the standalone C++ helper is deleted in its entirety, along
with its build system and third-party dependencies (see "Secondary changes").

How communication works now

The broker API (libdatadog + components-rs)

The actual broker logic lives in libdatadog: this PR bumps the libdatadog
submodule, which adds the AppSec message-brokering API to the sidecar. To call
that API from this repo, components-rs/ carries the matching generated C
headers (sidecar.h, common.h) and the thin sidecar.rs wrapper, and the
new symbols are exported in datadog.sym. The new entry points are:

• ddog_sidecar_send_appsec_message(transport, session_id, client_id, data) —
  called by the extension side. It ships data to the sidecar, which
  routes it to the helper and returns the helper's reply as a
  ddog_AppsecCResponse { ptr, len, capacity, disconnect }. The disconnect
  flag tells the extension the session was torn down on the other end.
• ddog_sidecar_appsec_response_drop(response) — frees the response buffer
  returned by send_appsec_message (a libdatadog-allocated Vec in the
  extension's own process, holding the bytes that came back over the sidecar
  transport). This is distinct from free_response below, which frees the
  helper's separate buffer on the sidecar side.
• ddog_sidecar_appsec_register_message_handler(on_message, on_disconnect, free_response)
  — called by the helper when it is loaded into the sidecar, to register
  the three C callbacks the sidecar invokes.

Extension side

The socket transport in network.c/helper_process.c is gone (~600 lines
removed): no more connect()/retry loops, writev/recv framing,
SO_PEERCRED credential checks, send/recv timeouts, or socket/lock path
computation. dd_conn shrinks to { bool connected; uint64_t client_id; }.

Sending a command is now a single round-trip:

• dd_conn_roundtripv() (replacing the old send + recv pair) serializes the
  outgoing msgpack into one buffer and hands it to a new
  dd_trace_send_appsec_message() wrapper in ddtrace.c.
• That wrapper grabs the per-thread sidecar transport via the new
  ddtrace_get_sidecar_transport() export (added to datadog.sym and made
  ZTS-aware in ext/sidecar.c — it resolves the right thread's
  DATADOG_G(sidecar) from a passed-in TSRM cache) and calls
  ddog_sidecar_send_appsec_message(...).
• The reply comes back as a ddog_AppsecCResponse, is parsed as msgpack, and
  freed with ddog_sidecar_appsec_response_drop().

Two new identifiers carry the routing:

• session_id — the extension's formatted sidecar session id (the same one
  already used for the sidecar); it scopes a client to a PHP process tree.
• client_id — a uint64 assigned by the helper. The extension's first
  message (client_init) is sent with client_id == 0, meaning "create a new
  client"; the helper allocates an id and returns it in the client_init
  response, and the extension stores it in conn->client_id for all
  subsequent messages.

New explicit teardown: commands/client_shutdown.{c,h} introduces a
client_shutdown message ({ clean: bool, error?: string }) sent from
GSHUTDOWN/request shutdown so the helper can drop the client promptly instead
of relying on a socket EOF. GSHUTDOWN was also made thread-safe (registers a
thread-local destructor and redirects TSRMLS_CACHE to the worker's cache).

test_mock_transport.{c,h} is new: in phpt tests there is no real sidecar, so
the mock transport intercepts dd_trace_send_appsec_message() and does raw
framed socket I/O against mock_helper over an fd handed in from PHP
(set_mock_helper_fd()).

Helper side

The helper no longer binds a UnixListener, no longer acquires a flock-based
single-instance lock (lock.rs, –178, deleted), and no longer polls the
sidecar for readiness. config.rs loses the socket and lock-path fields.

Instead, at startup it calls ddog_sidecar_appsec_register_message_handler and
serves traffic entirely through the three callbacks (new client/sidecar_msg.rs,
+718). Client tasks are still async Tokio tasks, but they are now fed by an
mpsc channel instead of a socket stream:

• on_message(session_id, client_id, data, len) -> AppsecCResponse — the
  hot path. It looks up the client task for (session_id, client_id) in a
  CLIENTS map; if client_id == 0 it spawns a new client task and assigns a
  fresh id, writing it back through the *client_id out-parameter. It forwards
  the bytes to the task over an mpsc::Sender<HelperRequest> and block_ons
  the oneshot reply, then returns it as an AppsecCResponse. Errors/timeouts
  produce a FatalError reply with disconnect: true.
• on_disconnect(session_id, client_id) — removes the client (or all
  clients of a session when client_id == 0) from the map; dropping the sender
  closes the task's channel and the task exits.
• free_response(ptr, len, capacity) — reconstitutes and drops the Vec
  that backed a response.

The client message loop (client.rs) was reworked to decode Commands from
the mpsc stream (via the existing protocol::CommandCodec) and answer over a
oneshot sender rather than writing back to a socket; server.rs gains a
ClientTaskSet that owns the spawned tasks and drains them on shutdown.

So the end-to-end path for one command is:

extension: dd_conn_roundtripv → dd_trace_send_appsec_message
        → ddog_sidecar_send_appsec_message(session_id, client_id, data)
sidecar:  routes to helper → on_message(session_id, client_id, data)
helper:   mpsc → client task → oneshot reply → AppsecCResponse
sidecar:  returns bytes to extension; later free_response(...)
extension: parse msgpack; ddog_sidecar_appsec_response_drop

Secondary changes

• C++ helper deleted: all of appsec/src/helper/** (client, engine, WAF
  glue, msgpack/network/proto layers) and its tests/benches/fuzzers.
• Third-party deps dropped: appsec/third_party/{libddwaf,msgpack-c,cpp-base64}
  submodules and their .gitmodules/CMake wiring; cmake/helper.cmake,
  check_fslib.cpp and related clang-tidy/clang-format machinery.
• Build/packaging: the separate helper build is gone —
  .gitlab/build-appsec-helper.sh deleted, the Rust helper now builds to the
  unified libddappsec-helper.so, generate-appsec.php/generate-package.php
  drop the C++ helper compile/asan/coverage jobs, and the final/SSI/debug
  artifact scripts stop shipping a second helper binary.
• Config removed: DD_APPSEC_HELPER_RUNTIME_PATH and the
  helper_rust_redirection toggle (and their supported-configurations.json
  entries) — there is no socket/lock path to configure and no C++/Rust choice
  to make. client_init now only accepts the rust helper runtime.
• Windows: config.w32 excludes the three new unix-only sidecar appsec
  symbols from the DLL exports (AppSec is not supported on Windows).
• New integration tests: ConnectivityTests.groovy (clients exit promptly
  on sidecar disconnect, and clean shutdown via client_shutdown on GSHUTDOWN)
  and CrashDetectionTests.groovy (coredump on sidecar SIGSEGV; clean sidecar
  exit after Apache restart); TelemetryTests simplified now that the runtime
  is always Rust.

[datadog-datadog-prod-us1]

 

✨ Unblock PR with BitsAI

⚠️ Warnings

🚦 13 Pipeline jobs failed

DataDog/apm-reliability/dd-trace-php | ASAN test_c with multiple observers: [8.2]     

DataDog/apm-reliability/dd-trace-php | test_extension_ci: [7.0]     

DataDog/apm-reliability/dd-trace-php | test_extension_ci: [7.1]     

View all 13 failed jobs.

ℹ️ Info

No other issues found (see more)

🧪 All tests passed
❄️ No new flaky tests detected

🔄 Datadog auto-retried 1 job - 0 passed on retry

🎯 Code Coverage (details)
• Patch Coverage: 54.49%
• Overall Coverage: 16.71% (-43.97%)

Useful? React with 👍 / 👎

_{This comment will be updated automatically if new data arrives.
🔗 Commit SHA: ca267b0 | Docs | Datadog PR Page | Give us feedback!}

[pr-commenter]

Benchmarks [ tracer ]

Benchmark execution time: 2026-08-02 01:27:21

Comparing candidate commit ca267b0 in PR branch glopes/sidecar-comm with baseline commit af463f7 in branch master.

Found 4 performance improvements and 2 performance regressions! Performance is the same for 188 metrics, 0 unstable metrics.

Explanation

This is an A/B test comparing a candidate commit's performance against that of a baseline commit. Performance changes are noted in the tables below as:

• 🟩 = significantly better candidate vs. baseline
• 🟥 = significantly worse candidate vs. baseline

We compute a confidence interval (CI) over the relative difference of means between metrics from the candidate and baseline commits, considering the baseline as the reference.

If the CI is entirely outside the configured SIGNIFICANT_IMPACT_THRESHOLD (or the deprecated UNCONFIDENCE_THRESHOLD), the change is considered significant.

Feel free to reach out to #apm-benchmarking-platform on Slack if you have any questions.

More details about the CI and significant changes

You can imagine this CI as a range of values that is likely to contain the true difference of means between the candidate and baseline commits.

CIs of the difference of means are often centered around 0%, because often changes are not that big:

---------------------------------(------|---^--------)-------------------------------->
                              -0.6%    0%  0.3%     +1.2%
                                 |          |        |
         lower bound of the CI --'          |        |
sample mean (center of the CI) -------------'        |
         upper bound of the CI ----------------------'

As described above, a change is considered significant if the CI is entirely outside the configured SIGNIFICANT_IMPACT_THRESHOLD (or the deprecated UNCONFIDENCE_THRESHOLD).

For instance, for an execution time metric, this confidence interval indicates a significantly worse performance:

----------------------------------------|---------|---(---------^---------)---------->
                                       0%        1%  1.3%      2.2%      3.1%
                                                  |   |         |         |
       significant impact threshold --------------'   |         |         |
                      lower bound of CI --------------'         |         |
       sample mean (center of the CI) --------------------------'         |
                      upper bound of CI ----------------------------------'

scenario:HookBench/benchHookOverheadTraceFunction-opcache

• 🟩 execution_time [-7.320µs; -3.731µs] or [-4.327%; -2.205%]

scenario:MessagePackSerializationBench/benchMessagePackSerialization

• 🟩 execution_time [-4.971µs; -2.729µs] or [-4.432%; -2.433%]

scenario:MessagePackSerializationBench/benchMessagePackSerialization-opcache

• 🟩 execution_time [-5.231µs; -3.509µs] or [-4.546%; -3.049%]

scenario:SamplingRuleMatchingBench/benchRegexMatching2-opcache

• 🟩 execution_time [-10.850µs; -10.676µs] or [-86.774%; -85.382%]

scenario:SamplingRuleMatchingBench/benchRegexMatching3-opcache

• 🟥 execution_time [+304.274ns; +609.326ns] or [+2.443%; +4.893%]

scenario:SamplingRuleMatchingBench/benchRegexMatching4-opcache

• 🟥 execution_time [+472.051ns; +720.949ns] or [+3.804%; +5.810%]

[codecov-commenter]

Codecov Report

❌ Patch coverage is 59.67213% with 492 lines in your changes missing coverage. Please review.
✅ Project coverage is 75.84%. Comparing base (a1bb038) to head (366de49).

Files with missing lines	Patch %	Lines
appsec/helper-rust/src/client.rs	44.40%	144 Missing ⚠️
appsec/helper-rust/src/client/sidecar_msg.rs	81.47%	88 Missing ⚠️
appsec/helper-rust/src/server.rs	0.00%	77 Missing ⚠️
appsec/src/extension/test_mock_transport.c	58.76%	30 Missing and 10 partials ⚠️
appsec/src/extension/commands_helpers.c	66.96%	30 Missing and 7 partials ⚠️
appsec/src/extension/ddtrace.c	45.45%	30 Missing and 6 partials ⚠️
appsec/src/extension/network.c	58.82%	20 Missing and 8 partials ⚠️
appsec/helper-rust/src/lib.rs	0.00%	26 Missing ⚠️
appsec/src/extension/logging.h	0.00%	12 Missing ⚠️
appsec/src/extension/helper_process.c	66.66%	2 Missing ⚠️
... and 1 more

Additional details and impacted files

@@            Coverage Diff             @@
##           master    #3725      +/-   ##
==========================================
+ Coverage   68.81%   75.84%   +7.02%     
==========================================
  Files         166       66     -100     
  Lines       19030    13366    -5664     
  Branches     1797     1189     -608     
==========================================
- Hits        13095    10137    -2958     
+ Misses       5121     2662    -2459     
+ Partials      814      567     -247     

Flag	Coverage Δ
helper-rust-unit	52.54% <60.16%> (+3.17%)	⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

Files with missing lines	Coverage Δ
appsec/helper-rust/src/client/protocol.rs	95.07% <100.00%> (+0.01%)	⬆️
appsec/helper-rust/src/config.rs	97.77% <ø> (+0.21%)	⬆️
appsec/helper-rust/src/ffi.rs	74.48% <ø> (ø)
appsec/helper-rust/src/service/updateable_waf.rs	96.37% <100.00%> (+0.03%)	⬆️
appsec/helper-rust/src/service/waf_ruleset.rs	66.23% <ø> (ø)
appsec/helper-rust/src/telemetry.rs	74.07% <ø> (ø)
appsec/helper-rust/src/telemetry/sidecar.rs	82.60% <ø> (+3.50%)	⬆️
appsec/src/extension/commands/client_init.c	79.51% <100.00%> (+1.46%)	⬆️
appsec/src/extension/commands/request_exec.c	81.81% <100.00%> (ø)
appsec/src/extension/configuration.h	100.00% <ø> (ø)
... and 14 more

... and 34 files with indirect coverage changes

---

Continue to review full report in Codecov by Sentry.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update a1bb038...366de49. Read the comment docs.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
• 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

[pr-commenter]

Benchmarks [ appsec ]

Benchmark execution time: 2026-08-02 00:41:01

Comparing candidate commit ca267b0 in PR branch glopes/sidecar-comm with baseline commit af463f7 in branch master.

Found 3 performance improvements and 0 performance regressions! Performance is the same for 9 metrics, 0 unstable metrics.

Explanation

This is an A/B test comparing a candidate commit's performance against that of a baseline commit. Performance changes are noted in the tables below as:

• 🟩 = significantly better candidate vs. baseline
• 🟥 = significantly worse candidate vs. baseline

We compute a confidence interval (CI) over the relative difference of means between metrics from the candidate and baseline commits, considering the baseline as the reference.

If the CI is entirely outside the configured SIGNIFICANT_IMPACT_THRESHOLD (or the deprecated UNCONFIDENCE_THRESHOLD), the change is considered significant.

Feel free to reach out to #apm-benchmarking-platform on Slack if you have any questions.

More details about the CI and significant changes

You can imagine this CI as a range of values that is likely to contain the true difference of means between the candidate and baseline commits.

CIs of the difference of means are often centered around 0%, because often changes are not that big:

---------------------------------(------|---^--------)-------------------------------->
                              -0.6%    0%  0.3%     +1.2%
                                 |          |        |
         lower bound of the CI --'          |        |
sample mean (center of the CI) -------------'        |
         upper bound of the CI ----------------------'

As described above, a change is considered significant if the CI is entirely outside the configured SIGNIFICANT_IMPACT_THRESHOLD (or the deprecated UNCONFIDENCE_THRESHOLD).

For instance, for an execution time metric, this confidence interval indicates a significantly worse performance:

----------------------------------------|---------|---(---------^---------)---------->
                                       0%        1%  1.3%      2.2%      3.1%
                                                  |   |         |         |
       significant impact threshold --------------'   |         |         |
                      lower bound of CI --------------'         |         |
       sample mean (center of the CI) --------------------------'         |
                      upper bound of CI ----------------------------------'

scenario:LaravelBench/benchLaravelOverhead-appsec

• 🟩 execution_time [-2.617ms; -2.441ms] or [-17.663%; -16.474%]

scenario:SymfonyBench/benchSymfonyOverhead-appsec

• 🟩 execution_time [-2.859ms; -2.612ms] or [-25.026%; -22.864%]

scenario:WordPressBench/benchWordPressOverhead-appsec

• 🟩 execution_time [-11.898ms; -11.770ms] or [-27.592%; -27.294%]

[estringana]

Should this comment go?