Skip to content

Commit c9a2b8e

Browse files
cataphractclaude
cataphract
and
claude
committed
fix(ssi): Handle Podman cgroupns=host cgroup path for container origin detection
Update libdatadog submodule to fix container ID extraction when running under Podman with cgroupns=host. The container cgroup path includes a /container subdirectory after the .scope suffix (e.g. 0::/machine.slice/libpod-HEXID.scope/container), which the previous regex did not handle. This caused origin detection to fail: no entity ID was sent to the agent, so container tags were missing from APM traces. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
1 parent 3442a78 commit c9a2b8e

10 files changed

Lines changed: 2164 additions & 1 deletion

‎.claude/ci/appsec-gradle-integration.md‎

Lines changed: 264 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,264 @@
1+
# AppSec Gradle Integration Tests
2+
3+
## CI Jobs
4+
5+
**Source:** `.gitlab/generate-appsec.php` — generates the appsec-trigger
6+
child pipeline; all job `script:` sections are defined inline in this
7+
file.
8+
9+
| CI Job | Image | What it does |
10+
|--------|-------|-------------|
11+
| `appsec integration tests: [{target}]` | `docker:24.0.4-gbi-focal` | Gradle integration tests with C++ helper (release/zts variants) |
12+
| `appsec integration tests (helper-rust): [{target}]` | same | Same tests with Rust helper (`-PuseHelperRust`); includes `debug` variant |
13+
| `appsec integration tests (ssi): [{target}]` | same | SSI mode (PHP 8.3 only) |
14+
| `helper-rust build and test` | same | `cargo fmt --check` + build + unit tests |
15+
| `helper-rust code coverage` | same | Unit test coverage via `cargo-llvm-cov` |
16+
| `helper-rust integration coverage` | same | Integration coverage collection (not needed locally) |
17+
18+
Runner: `docker-in-docker:amd64`
19+
Matrix: PHP 7.0--8.5 × release/debug/zts/musl/ssi (varies by job group)
20+
21+
**Important:** `testX.Y-debug` are not gradle targets that are run on CI. They
22+
may, however, be useful for debugging.
23+
24+
CI passes `TERM=dumb` and `--scan -Pbuildscan` to Gradle. `TERM=dumb`
25+
suppresses progress animations in CI logs; both flags are optional
26+
locally.
27+
28+
## Prerequisites
29+
30+
- JDK 17+
31+
- Docker daemon running
32+
33+
## Working Directory
34+
35+
All `./gradlew` commands run from:
36+
37+
```
38+
appsec/tests/integration/
39+
```
40+
41+
## Running Tests
42+
43+
### Full suite for one PHP target
44+
45+
```bash
46+
./gradlew test8.3-debug --info
47+
```
48+
49+
### Single test (fastest feedback loop)
50+
51+
```bash
52+
./gradlew test8.3-debug --info \
53+
--tests "com.datadog.appsec.php.integration.Apache2FpmTests.Pool environment"
54+
```
55+
56+
The `--tests` filter accepts:
57+
- Full method: `"com.datadog.appsec.php.integration.Apache2FpmTests.Pool environment"`
58+
- Class only: `"*Apache2FpmTests*"` or `"com.datadog.appsec.php.integration.Apache2FpmTests"`
59+
- Wildcard: `"*FpmTests*"`
60+
61+
### With helper-rust (instead of C++ helper)
62+
63+
```bash
64+
./gradlew test8.3-debug -PuseHelperRust --info \
65+
--tests "com.datadog.appsec.php.integration.Apache2FpmTests.Pool environment"
66+
```
67+
68+
This builds the Rust helper via the `buildHelperRust` task (musl build, works on both glibc and musl targets) and stores the binary in the `php-helper-rust` Docker volume.
69+
70+
### C++ helper (default)
71+
72+
Omit `-PuseHelperRust` and `-PhelperBinary`. The C++ helper is built via the `buildAppsec-*` task.
73+
74+
## Image Tags
75+
76+
By default, Gradle resolves Docker images via pinned SHA256 digests in `gradle/tag_mappings.gradle`. To use floating tags (locally-built images or latest from Docker Hub):
77+
78+
```bash
79+
./gradlew test8.3-debug -PfloatingImageTags --info
80+
```
81+
82+
## Available Gradle Tasks
83+
84+
### Test tasks
85+
86+
Pattern: `test{version}-{variant}`
87+
88+
| Variant | Notes |
89+
|---|---|
90+
| `release` | Standard build |
91+
| `debug` | Debug build (assertions enabled) |
92+
| `release-zts` | Thread-safe build |
93+
| `release-musl` | Alpine/musl (only `8.5-release-musl`) |
94+
| `release-ssi` / `debug-ssi` | SSI mode (only PHP 8.3) |
95+
96+
Full list: `./gradlew tasks --group=Verification`
97+
98+
### Helper-rust tasks
99+
100+
| Task | Description |
101+
|---|---|
102+
| `buildHelperRust` | Build helper-rust with musl (universal binary). Output in `php-helper-rust` volume. |
103+
| `testHelperRust` | `cargo fmt --check` + `cargo build --release` + `cargo test --release` (runs inside `php-deps` image) |
104+
| `coverageHelperRust` | Unit test coverage via `cargo-llvm-cov`. Output: `php-helper-rust-coverage` volume. |
105+
| `buildHelperRustWithCoverage` | Build with `-C instrument-coverage` for integration coverage collection. |
106+
| `generateHelperRustIntegrationCoverage` | Merge `.profraw` files into lcov after integration run. |
107+
108+
### Build tasks
109+
110+
| Task | Description |
111+
|---|---|
112+
| `buildTracer-{v}-{var}` | Build ddtrace.so for given PHP version/variant |
113+
| `buildAppsec-{v}-{var}` | Build ddappsec.so (C++ extension + helper) |
114+
| `buildHelperRust` | Build Rust helper (musl, universal) |
115+
| `buildLibddwaf` | Build libddwaf shared library |
116+
117+
### Other tasks
118+
119+
| Task | Description |
120+
|---|---|
121+
| `loadCaches` | Restore Docker volume caches from tarball |
122+
| `saveCaches` | Save Docker volume caches to tarball |
123+
| `clean` | Delete build directory and clean Docker volumes |
124+
| `check` | Run all test tasks |
125+
126+
All tasks: `./gradlew tasks --all`
127+
128+
## Interactive Container (runMain)
129+
130+
Start a test container without running tests (for manual debugging):
131+
132+
```bash
133+
./gradlew runMain8.3-release -PtestClass=com.datadog.appsec.php.integration.Apache2FpmTests
134+
```
135+
136+
The `-PtestClass` property is required (the task is not created without it). Add `-PuseHelperRust` or `-PhelperBinary=...` as needed.
137+
138+
SSI variant:
139+
140+
```bash
141+
./gradlew runMain8.3-release-ssi -PtestClass=com.datadog.appsec.php.integration.Apache2FpmTests
142+
```
143+
144+
## Logs
145+
146+
After a test run, logs are in:
147+
148+
```
149+
build/test-logs/<TestClassName>-<version>-<variant>/
150+
```
151+
152+
For example:
153+
154+
```
155+
build/test-logs/com.datadog.appsec.php.integration.Apache2FpmTests-8.3-debug/
156+
├── access.log
157+
├── appsec.log # PHP extension appsec log
158+
├── error.log # Apache error log
159+
├── helper.log # Helper process log (C++ or Rust)
160+
├── php_error.log
161+
├── php_fpm_error.log
162+
└── sidecar.log
163+
```
164+
165+
To distinguish which helper ran, check `helper.log`:
166+
- Rust: starts with `[INFO] AppSec helper starting`
167+
- C++: starts with `[info]` lines like `Started listening on abstract socket`
168+
169+
## Musl/Alpine Target
170+
171+
The `test8.5-release-musl` target uses an Alpine-based nginx+fpm image. Tests tagged with `@Tag("musl")` are included; untagged tests are excluded.
172+
173+
```bash
174+
./gradlew test8.5-release-musl -PuseHelperRust --info
175+
```
176+
177+
The `buildHelperRust` task already produces a musl-linked binary (built on Alpine with `cargo +nightly`, using LLVM libunwind). The `patchelf --remove-needed libc.musl-*` step makes it load on both musl and glibc systems.
178+
179+
## CI Job Mapping
180+
181+
| CI Job | Gradle Command |
182+
|---|---|
183+
| `appsec integration tests: [test8.3-release]` | `./gradlew test8.3-release` |
184+
| `appsec integration tests (helper-rust): [test8.3-debug]` | `./gradlew test8.3-debug -PuseHelperRust` |
185+
| `appsec integration tests (ssi): [test8.3-release-ssi]` | `./gradlew test8.3-release-ssi` |
186+
| `helper-rust build and test` | `./gradlew testHelperRust` |
187+
| `helper-rust code coverage` | `./gradlew coverageHelperRust` |
188+
| `helper-rust integration coverage` | `./gradlew buildHelperRustWithCoverage` then integration test with `-PuseHelperRustCoverage` |
189+
190+
CI also passes `--scan -Pbuildscan` for Gradle build scans, which is optional locally.
191+
192+
## Docker Volumes
193+
194+
Gradle uses named Docker volumes for build artifacts and caches. Key volumes:
195+
196+
| Volume | Contents |
197+
|---|---|
198+
| `php-helper-rust` | `libddappsec-helper.so` (Rust helper binary) |
199+
| `php-tracer-{v}-{var}` | Built `ddtrace.so` |
200+
| `php-appsec-{v}-{var}` | Built `ddappsec.so` + C++ helper |
201+
| `php-tracer-cargo-cache` | Cargo registry cache |
202+
| `php-tracer-cargo-cache-git` | Cargo git cache |
203+
| `php-appsec-boost-cache` | Boost build cache |
204+
| `php-helper-rust-coverage` | Coverage-instrumented binary + profraw files |
205+
206+
To force a rebuild, remove the relevant volume:
207+
208+
```bash
209+
docker volume rm php-helper-rust
210+
```
211+
212+
To clean everything:
213+
214+
```bash
215+
./gradlew clean
216+
```
217+
218+
## Debugging
219+
220+
Attach a Java debugger to the test runner:
221+
222+
```bash
223+
./gradlew test8.3-debug --tests "*Apache2FpmTests*" --debug-jvm
224+
```
225+
226+
Enable PHP Xdebug in the test container:
227+
228+
```bash
229+
./gradlew test8.3-debug --tests "*Apache2FpmTests*" -PXDEBUG=1
230+
```
231+
232+
## Expected skips
233+
234+
A significant number of tests are skipped on any given target — this is
235+
normal. Skip conditions are `@EnabledIf` guards in the test classes:
236+
237+
| Test class | Skips on | Reason |
238+
|---|---|---|
239+
| `FrankenphpClassicTests`, `FrankenphpWorkerTests` | anything except `8.4-zts` | Requires ZTS + PHP 8.4 |
240+
| `Laravel8xTests` | anything except `7.4` (NTS) | Requires PHP 7.4 non-ZTS |
241+
| `Symfony62Tests` | anything except `8.1` (NTS) | Requires PHP 8.1 non-ZTS |
242+
| `RaspSqliTests` | no MySQL service | Requires a running MySQL |
243+
| `SsiStableConfigTests` | non-SSI variants | Requires `-DSSI=true` |
244+
245+
On `test8.3-debug` expect ~67 skips out of ~300 tests; all are expected.
246+
247+
## Test report
248+
249+
After a run, the HTML report is at:
250+
251+
```
252+
appsec/tests/integration/build/reports/tests/test8.3-debug/index.html
253+
```
254+
255+
(Replace `test8.3-debug` with your target.) Open in a browser for a
256+
structured pass/fail/skip breakdown.
257+
258+
## Gotchas
259+
260+
- The `test` task itself is disabled (`tasks['test'].enabled = false`). Use versioned tasks like `test8.3-debug`.
261+
- Docker images are pulled from `docker.io/datadog/dd-appsec-php-ci`. Without `-PfloatingImageTags`, images are resolved by SHA256 digest from `gradle/tag_mappings.gradle`. If a digest is not locally available, Docker will pull it.
262+
- The `buildHelperRust` task uses the `nginx-fpm-php-8.5-release-musl` image (Alpine with Rust nightly). This image must be available locally or pullable.
263+
- On first run, Gradle downloads its wrapper, dependencies, and Docker images. Expect 5-10 minutes. Subsequent runs with warm caches take ~20-50 seconds for a single test.
264+
- `--info` is recommended for seeing test output in the console. Without it, output goes only to the HTML report in `build/reports/tests/`.

0 commit comments

Comments
 (0)