Move appsec communication to sidecar

.claude/ci/appsec-gradle-integration.md

@@ -8,9 +8,8 @@ file.
 
 | CI Job | Image | What it does |
 |--------|-------|-------------|
-| `appsec integration tests: [{target}]` | `docker:29.4.0-noble` | Gradle integration tests with Rust helper by default (release/zts/musl variants) |
-| `appsec integration tests (helper-cpp): [{target}]` | same | Same tests with Rust helper redirection disabled (`-PuseHelperCpp`); PHP 8.3 release and release-zts |
-| `appsec integration tests (ssi): [{target}]` | same | SSI mode (PHP 8.3 only), Rust helper by default |
+| `appsec integration tests: [{target}]` | `docker:29.4.0-noble` | Gradle integration tests with embedded AppSec (release/zts/musl variants) |
+| `appsec integration tests (ssi): [{target}]` | same | SSI mode (ordinary CI covers PHP 8.3) |
 | `helper-rust build and test` | same | `cargo fmt --check` + build + unit tests |
 | `helper-rust code coverage` | same | Unit test coverage via `cargo-llvm-cov` |
 | `helper-rust integration coverage` | same | Integration coverage collection (not needed locally) |
@@ -60,25 +59,16 @@ The `--tests` filter accepts:
 - Class only: `"*Apache2FpmTests*"` or `"com.datadog.appsec.php.integration.Apache2FpmTests"`
 - Wildcard: `"*FpmTests*"`
 
-### Rust helper (default)
+### Embedded Rust helper
 
 ```bash
 ./gradlew test8.3-debug --info \
     --tests "com.datadog.appsec.php.integration.Apache2FpmTests.Pool environment"
 ```
 
-This builds the Rust helper via the `buildHelperRust` task (musl build, works on both glibc and musl targets), stores the binary in the `php-helper-rust` Docker volume, and mounts it alongside the C++ helper so `DD_APPSEC_HELPER_RUST_REDIRECTION` can select it.
-
-### C++ helper opt-out
-
-Add `-PuseHelperCpp` to disable Rust helper redirection:
-
-```bash
-./gradlew test8.3-debug -PuseHelperCpp --info \
-    --tests "com.datadog.appsec.php.integration.Apache2FpmTests.Pool environment"
-```
-
-This sets `DD_APPSEC_HELPER_RUST_REDIRECTION=false` in the test container. The Rust helper may still be built and mounted, but the extension uses the C++ helper from the `buildAppsec-*` task.
+The tracer build embeds `appsec/helper-rust` into its Rust sidecar component.
+SSI targets use `buildPortableLibdatadogPhp` to build a musl-based
+`libdatadog_php.so` that can load on glibc and musl targets.
 
 ## Image Tags
 
@@ -100,27 +90,26 @@ Pattern: `test{version}-{variant}`
 | `debug` | Debug build (assertions enabled) |
 | `release-zts` | Thread-safe build |
 | `release-musl` | Alpine/musl (only `8.5-release-musl`) |
-| `release-ssi` / `debug-ssi` | SSI mode (only PHP 8.3) |
+| `release-ssi` / `debug-ssi` | SSI mode; ordinary CI covers PHP 8.3 and coverage also runs PHP 8.4 ZTS |
 
 Full list: `./gradlew tasks --group=Verification`
 
 ### Helper-rust tasks
 
 | Task | Description |
 |---|---|
-| `buildHelperRust` | Build helper-rust with musl (universal binary). Output in `php-helper-rust` volume. |
-| `testHelperRust` | `cargo fmt --check` + `cargo build --release` + `cargo test --release` (runs inside `php-deps` image) |
+| `testHelperRust` | Workspace-scoped `cargo fmt`, Clippy, build, and unit tests for `helper-rust` |
 | `coverageHelperRust` | Unit test coverage via `cargo-llvm-cov`. Output: `php-helper-rust-coverage` volume. |
-| `buildHelperRustWithCoverage` | Build with `-C instrument-coverage` for integration coverage collection. |
-| `generateHelperRustIntegrationCoverage` | Merge `.profraw` files into lcov after integration run. |
+| `buildPortableLibdatadogPhp` | Build the portable sidecar library, with helper coverage when `-PuseHelperRustCoverage` is set |
+| `generateHelperRustIntegrationCoverage-{id}` | Merge sidecar `.profraw` files into lcov after a coverage run |
 
 ### Build tasks
 
 | Task | Description |
 |---|---|
 | `buildTracer-{v}-{var}` | Build ddtrace.so for given PHP version/variant |
-| `buildAppsec-{v}-{var}` | Build ddappsec.so (C++ extension + helper) |
-| `buildHelperRust` | Build Rust helper (musl, universal) |
+| `buildAppsec-{v}-{var}` | Build ddappsec.so |
+| `buildPortableLibdatadogPhp` | Build portable `libdatadog_php.so` for SSI |
 | `buildLibddwaf` | Build libddwaf shared library |
 
 ### Other tasks
@@ -142,7 +131,7 @@ Start a test container without running tests (for manual debugging):
 ./gradlew runMain8.3-release -PtestClass=com.datadog.appsec.php.integration.Apache2FpmTests
 ```
 
-The `-PtestClass` property is required (the task is not created without it). Add `-PuseHelperCpp` to opt out of Rust helper redirection, or `-PhelperBinary=...` to bind-mount an explicit helper binary.
+The `-PtestClass` property is required (the task is not created without it).
 
 SSI variant:
 
@@ -165,16 +154,12 @@ build/test-logs/com.datadog.appsec.php.integration.Apache2FpmTests-8.3-debug/
 ├── access.log
 ├── appsec.log         # PHP extension appsec log
 ├── error.log          # Apache error log
-├── helper.log         # Helper process log (C++ or Rust)
+├── helper.log         # Embedded AppSec helper log
 ├── php_error.log
 ├── php_fpm_error.log
 └── sidecar.log
 ```
 
-To distinguish which helper ran, check `helper.log`:
-- Rust: starts with `[INFO] AppSec helper starting`
-- C++: starts with `[info]` lines like `Started listening on abstract socket`
-
 ## Musl/Alpine Target
 
 The `test8.5-release-musl` target uses an Alpine-based nginx+fpm image. Tests tagged with `@Tag("musl")` are included; untagged tests are excluded.
@@ -183,19 +168,21 @@ The `test8.5-release-musl` target uses an Alpine-based nginx+fpm image. Tests ta
 ./gradlew test8.5-release-musl --info
 ```
 
-The `buildHelperRust` task already produces a musl-linked binary (built on Alpine with `cargo +nightly`, using LLVM libunwind). The `patchelf --remove-needed libc.musl-*` step makes it load on both musl and glibc systems.
+`buildPortableLibdatadogPhp` builds the library with the `glibc-compat`
+Cargo feature against musl (also rebuilding std with LLVM libunwind for
+backtrace support). `compile_rust.sh` then uses `patchelf` to strip musl's
+`DT_NEEDED` entries and add the glibc sonames, so the resulting library
+loads on both musl and glibc systems.
 
 ## CI Job Mapping
 
 | CI Job | Gradle Command |
 |---|---|
 | `appsec integration tests: [test8.3-release]` | `./gradlew test8.3-release` |
-| `appsec integration tests (helper-cpp): [test8.3-release]` | `./gradlew test8.3-release -PuseHelperCpp` |
-| `appsec integration tests (helper-cpp): [test8.3-release-zts]` | `./gradlew test8.3-release-zts -PuseHelperCpp` |
 | `appsec integration tests (ssi): [test8.3-release-ssi]` | `./gradlew test8.3-release-ssi` |
 | `helper-rust build and test` | `./gradlew testHelperRust` |
 | `helper-rust code coverage` | `./gradlew coverageHelperRust` |
-| `helper-rust integration coverage` | `./gradlew buildHelperRustWithCoverage` then integration test with `-PuseHelperRustCoverage` |
+| `helper-rust integration coverage` | `./gradlew buildPortableLibdatadogPhp -PuseHelperRustCoverage` then SSI tests with the same property |
 
 CI also passes `--scan -Pbuildscan` for Gradle build scans, which is optional locally.
 
@@ -205,18 +192,18 @@ Gradle uses named Docker volumes for build artifacts and caches. Key volumes:
 
 | Volume | Contents |
 |---|---|
-| `php-helper-rust` | `libddappsec-helper.so` (Rust helper binary) |
+| `php-portable-libdatadog-php` | Portable `libdatadog_php.so` used by SSI tests |
 | `php-tracer-{v}-{var}` | Built `ddtrace.so` |
-| `php-appsec-{v}-{var}` | Built `ddappsec.so` + C++ helper |
+| `php-appsec-{v}-{var}` | Built `ddappsec.so` |
 | `php-tracer-cargo-cache` | Cargo registry cache |
 | `php-tracer-cargo-cache-git` | Cargo git cache |
 | `php-appsec-boost-cache` | Boost build cache |
-| `php-helper-rust-coverage` | Coverage-instrumented binary + profraw files |
+| `php-helper-rust-coverage` | Helper unit-test coverage output |
 
 To force a rebuild, remove the relevant volume:
 
 ```bash
-docker volume rm php-helper-rust
+docker volume rm php-portable-libdatadog-php
 ```
 
 To clean everything:
@@ -285,7 +272,9 @@ If you need to inspect sidecar/helper or PHP issues:
 
 - The `test` task itself is disabled (`tasks['test'].enabled = false`). Use versioned tasks like `test8.3-debug`.
 - Docker images are pulled from `docker.io/datadog/dd-appsec-php-ci`. Without `-PfloatingImageTags`, images are resolved by SHA256 digest from `gradle/tag_mappings.gradle`. If a digest is not locally available, Docker will pull it.
-- The `buildHelperRust` task uses the `nginx-fpm-php-8.5-release-musl` image (Alpine with Rust nightly). This image must be available locally or pullable.
+- `buildPortableLibdatadogPhp` uses the
+  `nginx-fpm-php-8.5-release-musl` image with nightly Rust. The image must
+  be available locally or pullable.
 - On first run, Gradle downloads its wrapper, dependencies, and Docker images. Expect 5-10 minutes. Subsequent runs with warm caches take ~20-50 seconds for a single test.
 - **c-ares DNS failure in Alpine containers.** Alpine's `curl` and `git` use
   c-ares for DNS, which fails to resolve hosts when the DNS server includes

.claude/ci/appsec-native-tests.md

@@ -11,7 +11,6 @@ file.
 | `test appsec extension: [{ver}, {arch}, debug]` | `datadog/dd-trace-ci:php-{ver}_bookworm-6` | Builds appsec PHP extension + runs phpunit `.phpt` tests |
 | `test appsec extension: [{ver}, {arch}, debug-zts]` | same | ZTS variant |
 | `test appsec extension: [{ver}, {arch}, debug-zts-asan]` | same | ASAN variant (PHP 7.4+) |
-| `test appsec helper asan` | `datadog/dd-trace-ci:bookworm-6` | Builds C++ helper with ASAN, runs gtest suite |
 | `appsec lint` | `datadog/dd-trace-ci:php-8.3_bookworm-6` | clang-format + clang-tidy |
 | `appsec code coverage` | `datadog/dd-trace-ci:php-8.3_bookworm-6` | Coverage instrumented build (not needed locally) |
 
@@ -52,7 +51,7 @@ sudo apt-get update -qq && sudo apt-get install -y -qq \
 # (cmake's xtest target builds ddtrace.so automatically as a dependency)
 mkdir -p appsec/build
 cd appsec/build
-cmake .. -DCMAKE_BUILD_TYPE=Debug -DDD_APPSEC_BUILD_HELPER=OFF \
+cmake .. -DCMAKE_BUILD_TYPE=Debug \
   -DCMAKE_CXX_FLAGS="-stdlib=libc++" -DCMAKE_CXX_LINK_FLAGS="-stdlib=libc++" \
   -DDD_APPSEC_TESTING=ON
 ASAN_OPTIONS=malloc_context_size=0 make -j$(nproc) xtest
@@ -75,7 +74,7 @@ cache name (e.g. `appsec-ext-8.3-debug-asan`) since the cmake cache is
 incompatible:
 
 ```bash
-cmake .. -DCMAKE_BUILD_TYPE=Debug -DDD_APPSEC_BUILD_HELPER=OFF \
+cmake .. -DCMAKE_BUILD_TYPE=Debug \
   -DCMAKE_CXX_FLAGS="-stdlib=libc++" -DCMAKE_CXX_LINK_FLAGS="-stdlib=libc++" \
   -DDD_APPSEC_TESTING=ON -DENABLE_ASAN=ON
 ASAN_OPTIONS=malloc_context_size=0 make -j4 xtest
@@ -143,77 +142,6 @@ reuse it. To force a full rebuild:
 docker volume rm php-appsec-8.3-debug php-tracer-8.3-debug
 ```
 
-## Helper tests (C++ ASAN)
-
-The C++ helper tests use the `bookworm-6` image (no PHP needed). The
-binary is a gtest executable. With `--overlayfs --root`, all writes
-(including `appsec/build-helper`) persist in the Docker volume
-automatically — no manual bind mount needed.
-
-### Full suite
-
-```bash
-.claude/ci/dockerh --cache appsec-helper --overlayfs --root \
-  datadog/dd-trace-ci:bookworm-6 -- bash -c '
-set -e
-apt-get update -qq && apt-get install -y -qq \
-  libc++-17-dev libc++abi-17-dev > /dev/null 2>&1
-# Required: libddwaf submodule dir may trigger git's safe.directory check.
-git config --global --add safe.directory \
-  /project/dd-trace-php/appsec/third_party/libddwaf
-mkdir -p appsec/build-helper
-cd appsec/build-helper
-cmake .. -DCMAKE_BUILD_TYPE=Debug -DDD_APPSEC_BUILD_EXTENSION=OFF \
-  -DDD_APPSEC_ENABLE_COVERAGE=OFF -DDD_APPSEC_TESTING=ON \
-  -DCMAKE_CXX_FLAGS="-stdlib=libc++ -fsanitize=address -fsanitize=leak -DASAN_BUILD" \
-  -DCMAKE_C_FLAGS="-fsanitize=address -fsanitize=leak -DASAN_BUILD" \
-  -DCMAKE_EXE_LINKER_FLAGS="-fsanitize=address -fsanitize=leak" \
-  -DCMAKE_MODULE_LINKER_FLAGS="-fsanitize=address -fsanitize=leak"
-make -j$(nproc) ddappsec_helper_test
-cd /project/dd-trace-php
-./appsec/build-helper/tests/helper/ddappsec_helper_test
-'
-```
-
-`clang-tidy-17` is not needed here — omitting it saves ~5–8 seconds of
-apt time. CI installs it via a shared `before_script` template.
-
-CI also passes `-DBOOST_CACHE_PREFIX=$CI_PROJECT_DIR/boost-cache` and
-`-DCLANG_TIDY=/usr/bin/run-clang-tidy-17` to cmake. The first only
-affects where the Boost cache lands (not a correctness issue locally).
-The second means CI silently runs clang-tidy checks during the helper
-build that are skipped locally.
-
-### Single test
-
-Once built, the pre-built binary can be re-run without rebuilding, but
-the `libc++` runtime must still be installed (container rootfs is not
-persisted by `--overlayfs`):
-
-```bash
-.claude/ci/dockerh --cache appsec-helper --overlayfs --root \
-  datadog/dd-trace-ci:bookworm-6 -- bash -c '
-apt-get update -qq && apt-get install -y -qq libc++1-17 libc++abi1-17 > /dev/null 2>&1
-./appsec/build-helper/tests/helper/ddappsec_helper_test \
-  --gtest_filter="WafTest.TraceAttributesAreSent"
-'
-```
-
-The filter supports wildcards: `--gtest_filter="WafTest.*"` runs all
-tests in the `WafTest` suite. Use `--gtest_list_tests` to see available
-tests.
-
-### Without ASAN
-
-For a faster non-ASAN build (useful when debugging test logic rather than
-memory issues), drop the sanitizer flags:
-
-```bash
-cmake .. -DCMAKE_BUILD_TYPE=Debug -DDD_APPSEC_BUILD_EXTENSION=OFF \
-  -DDD_APPSEC_TESTING=ON \
-  -DCMAKE_CXX_FLAGS="-stdlib=libc++" -DCMAKE_CXX_LINK_FLAGS="-stdlib=libc++"
-```
-
 ## Appsec lint
 
 This can easily run locally:
@@ -240,12 +168,6 @@ make tidy_fix    # fix clang-tidy violations
   `~/.cache/dd-ci/<NAME>/appsec/build/boost_cache/` (extension tests) or
   in the `php-appsec-boost-cache` Docker volume (Gradle).
 
-- `appsec/build-helper` (helper tests) is **not** a `dockerh` cache
-  overlay. Pass it explicitly as `-v ~/.cache/dd-ci/appsec-helper/appsec/build-helper:...`
-  (see the Helper tests section). Files are owned by root because helper
-  tests run with `--user root`; clean up with
-  `docker run --rm -v ~/.cache/dd-ci/appsec-helper/appsec:/w alpine rm -rf /w/build-helper`.
-
 - The `libc++-17-dev` and `libc++abi-17-dev` packages must be installed
   in every new container — the cmake cache references libc++ headers and
   will fail to compile without them.

.claude/ci/building-locally.md

@@ -28,13 +28,7 @@ Before any build, ensure the relevant submodules are initialised
 # Tracer extension (ddtrace.so) — needs libdatadog
 git submodule update --init libdatadog
 
-# Appsec extension or helper — needs these additionally
-git submodule update --init \
-  appsec/third_party/libddwaf \
-  appsec/third_party/msgpack-c \
-  appsec/third_party/cpp-base64
-
-# Appsec helper rust — needs libddwaf-rust
+# Appsec extension/helper — needs libddwaf-rust
 git submodule update --init --recursive \
   appsec/third_party/libddwaf-rust
 ```
@@ -186,7 +180,7 @@ from the CI release build above (builds test targets, uses libc++):
 
 ```bash
 mkdir -p appsec/build && cd appsec/build
-cmake .. -DCMAKE_BUILD_TYPE=Debug -DDD_APPSEC_BUILD_HELPER=OFF \
+cmake .. -DCMAKE_BUILD_TYPE=Debug \
   -DCMAKE_CXX_FLAGS="-stdlib=libc++" \
   -DCMAKE_CXX_LINK_FLAGS="-stdlib=libc++" \
   -DDD_APPSEC_TESTING=ON
@@ -196,47 +190,23 @@ make -j$(nproc) xtest
 For ASAN, add `-DENABLE_ASAN=ON` to cmake. See
 [appsec-native-tests.md](appsec-native-tests.md) for full details.
 
-## Appsec Helpers
-
-The tarball needs two helper binaries in `appsec_$(uname -m)/`:
-`libddappsec-helper.so` (C++) and `libddappsec-helper-rust.so`
-(Rust), plus `appsec/recommended.json`.
-
-### Rust helper
+## Embedded AppSec Helper
 
-Image is on Docker Hub. Output: `appsec_$(uname -m)/libddappsec-helper-rust.so`.
+The Rust AppSec helper is a workspace crate embedded in the tracer's sidecar
+component. Run its checks through the integration Gradle project:
 
 ```bash
 git submodule update --init --recursive \
   appsec/third_party/libddwaf-rust
 
-.claude/ci/dockerh --cache compile-appsec-helper-rust --overlayfs \
-    datadog/dd-appsec-php-ci:nginx-fpm-php-8.5-release-musl \
-    -e CI_COMMIT_SHA=$(git rev-parse HEAD) \
-    -e CI_COMMIT_BRANCH=$(git rev-parse --abbrev-ref HEAD) \
-    -- bash .gitlab/build-appsec-helper-rust.sh
+cd appsec/tests/integration
+./gradlew testHelperRust --info
+./gradlew buildPortableLibdatadogPhp --info
 ```
 
-### C++ helper
-
-The CI image (`nginx_musl_toolchain`) lives at
-`registry.ddbuild.io/images/mirror/b1o7r7e0/nginx_musl_toolchain`
-(not on Docker Hub). Transfer it via
-`docker save ... | ssh HOST docker load` if needed. Output:
-`appsec_$(uname -m)/libddappsec-helper.so` + `recommended.json`.
-
-```bash
-git submodule update --init \
-  appsec/third_party/libddwaf \
-  appsec/third_party/msgpack-c \
-  appsec/third_party/cpp-base64
-
-.claude/ci/dockerh --cache compile-appsec-helper-cpp --overlayfs \
-    registry.ddbuild.io/images/mirror/b1o7r7e0/nginx_musl_toolchain \
-    -e CI_COMMIT_SHA=$(git rev-parse HEAD) \
-    -e CI_COMMIT_BRANCH=$(git rev-parse --abbrev-ref HEAD) \
-    -- bash .gitlab/build-appsec-helper.sh
-```
+The release package needs the AppSec extensions in
+`appsec_$(uname -m)/` and `appsec/recommended.json`; there is no standalone
+helper artifact.
 
 ## Profiler Extension
 
@@ -327,9 +297,8 @@ platforms and fails if artifacts are missing.
 compiled `.so` files:
 - `extensions_$(uname -m)/` — ddtrace extensions
   (`ddtrace-{API}[-zts|-debug|-debug-zts].so`)
-- `appsec_$(uname -m)/` — appsec extensions (`ddappsec-{API}[-zts].so`) +
-  helpers (`libddappsec-helper.so` and `libddappsec-helper-rust.so`) +
-  `recommended.json`
+- `appsec_$(uname -m)/` — appsec extensions (`ddappsec-{API}[-zts].so`)
+- `appsec/recommended.json` — bundled AppSec rules
 - `datadog-profiling/{triplet}/lib/php/{API}/` — profiler
   extensions
 
@@ -466,7 +435,7 @@ troubleshooting but not for performance testing.
 # Tracer only (gnu, x86_64, PHP 8.2, NTS)
 tooling/bin/build-debug-artifact gnu-x86_64-8.2-nts
 
-# Tracer + appsec (extension + both helpers) + profiler
+# Tracer + appsec (extension + embedded helper) + profiler
 tooling/bin/build-debug-artifact gnu-x86_64-8.2-nts --appsec --profiler
 
 # Musl/arm64 variant, custom output directory (preferred if the location is somewhere else)