Add huge-label lookup scale coverage

Author: ktsaou

.agents/sow/current/SOW-0021-20260613-netipc-at-scale.md

@@ -881,6 +881,18 @@ Recorded user decisions:
   - C Windows on `/tmp/plugin-ipc-sow0021-20260614052244`: `cmake --build build-windows-focused --target test_win_service_extra -j4` passed and `NIPC_TEST_FILTER=malformed_first_response timeout 600 build-windows-focused/bin/test_win_service_extra.exe` passed with `84 passed, 0 failed`.
   - Go Windows on `/tmp/plugin-ipc-sow0021-20260614052244`: `cd src/go && "/c/Program Files/Go/bin/go.exe" test -count=1 -timeout=180s ./pkg/netipc/service/raw -run '^TestWin(Apps|Cgroups)LookupRejectsMalformedTypedResponses$'` passed.
   - Rust Windows on `/tmp/plugin-ipc-sow0021-20260614052244`: `/c/Users/costa/.cargo/bin/cargo.exe test --manifest-path src/crates/netipc/Cargo.toml test_lookup_rejects_malformed_typed_responses_windows -- --nocapture` passed.
+- Closed the huge-valid-metadata oversized-item gap:
+  - Existing Level 2 transparent overflow tests covered a huge APPS_LOOKUP cgroup path and a huge CGROUPS_LOOKUP name, but not a huge valid label.
+  - Extended C, Rust, and Go POSIX/Windows transparent `PAYLOAD_EXCEEDED` retry tests so each logical request contains a normal item, an oversized path/name item, an oversized label item, and a trailing normal item.
+  - Expected result in every language/platform: both huge items return `OVERSIZED_ITEM`, the trailing item still returns `KNOWN`, and the logical call hides intermediate `PAYLOAD_EXCEEDED` outcomes from Level 2 consumers.
+  - The cgroups test response budget is `256` bytes. This remains far below the 512-byte huge name/label payloads but leaves enough room for compact `PAYLOAD_EXCEEDED` and `OVERSIZED_ITEM` control records, so the test exercises scale handling rather than an impossible control-response buffer.
+- Validated huge-valid-metadata oversized-item isolation:
+  - C POSIX: `cmake --build build-coverage --target test_service -j12 && /usr/bin/ctest --test-dir build-coverage --output-on-failure -R '^test_service$'` passed.
+  - Go POSIX: `cd src/go && go test -count=1 -timeout=180s ./pkg/netipc/service/raw -run 'Test(Cgroups|Apps)LookupTransparentPayloadExceededRetry'` passed.
+  - Rust POSIX: `cargo test --manifest-path src/crates/netipc/Cargo.toml transparent_payload_exceeded -- --nocapture` passed.
+  - C Windows on `/tmp/plugin-ipc-sow0021-20260614052244`: `cmake --build build-windows-focused --target test_win_service_extra -j4` passed and `NIPC_TEST_FILTER=test_lookup_payload_exceeded_retry timeout 600 build-windows-focused/bin/test_win_service_extra.exe` passed with `36 passed, 0 failed`.
+  - Go Windows on `/tmp/plugin-ipc-sow0021-20260614052244`: `cd src/go && "/c/Program Files/Go/bin/go.exe" test -count=1 -timeout=180s ./pkg/netipc/service/raw -run "^TestWin(Cgroups|Apps)LookupTransparentPayloadExceededRetry$"` passed.
+  - Rust Windows on `/tmp/plugin-ipc-sow0021-20260614052244`: `/c/Users/costa/.cargo/bin/cargo.exe test --manifest-path src/crates/netipc/Cargo.toml transparent_payload_exceeded -- --nocapture` passed.
 
 ## Validation
 
@@ -986,6 +998,11 @@ Tests or equivalent validation:
   - C, Rust, and Go POSIX tests validate APPS_LOOKUP and CGROUPS_LOOKUP fail the whole logical call when a response item advertises more labels than its encoded item contains.
   - C, Rust, and Go Windows tests validate the same APPS_LOOKUP and CGROUPS_LOOKUP malformed status/table cases.
   - Latest focused evidence: C POSIX `ctest` `test_service` passed; C Windows focused filter `84 passed, 0 failed`; Go/Rust POSIX and Windows focused malformed-response tests passed.
+- Oversized valid metadata isolation validation:
+  - C, Rust, and Go POSIX tests validate APPS_LOOKUP and CGROUPS_LOOKUP transparent `PAYLOAD_EXCEEDED` retry when a logical response contains two different oversized valid items: one huge name/path item and one huge label item.
+  - C, Rust, and Go Windows tests validate the same APPS_LOOKUP and CGROUPS_LOOKUP huge valid label isolation case.
+  - The final logical response keeps both oversized items as explicit `OVERSIZED_ITEM` outcomes and still returns the trailing normal item as `KNOWN`.
+  - Latest focused evidence: C POSIX `ctest` `test_service` passed; C Windows focused filter `36 passed, 0 failed`; Go/Rust POSIX and Windows focused transparent-overflow tests passed.
 - Endpoint-disappears-after-partial-progress validation:
   - C, Rust, and Go POSIX tests validate APPS_LOOKUP and CGROUPS_LOOKUP fail the whole logical call when a valid first partial response is followed by endpoint disappearance before follow-up completion.
   - C, Rust, and Go Windows tests validate the same APPS_LOOKUP and CGROUPS_LOOKUP endpoint-disappears-after-partial-progress case.
@@ -1035,7 +1052,7 @@ Reviewer findings:
 - External reviewer finding: local oversized request-key synthesis could run before proving the client is connected. Handled by enforcing `READY` before logical lookup work in C, Go, and Rust.
 - External reviewer finding: cgroups all-local oversized handling could skip the zero-item probe path. Handled by continuing after a local oversized item only when more request items remain; otherwise the client sends the zero-item request and validates endpoint/generation behavior.
 - External reviewer concern: hidden fixed payload ceilings would contradict initialization-tunable budgets. Reviewed against code and docs. `NIPC_MAX_PAYLOAD_CAP` / `MaxPayloadCap` remains a named default growth ceiling; request/response payload budgets are exposed through initialization config. Server learned-capacity growth now also honors those configured ceilings.
-- External reviewer finding: coverage scripts and broader adversarial matrix still need updates before SOW completion. Coverage script expansion is now implemented; C/Rust/Go coverage gates pass; representative `8192` and `32768` logical-call tests now pass in C/Rust/Go on POSIX and Windows; POSIX and Windows baseline/SHM lookup-scale interop now pass across all C/Rust/Go directed pairs, including mixed-status lookup cases and heavier `65536` stress-only runs; lookup status codec interop now proves `PAYLOAD_EXCEEDED` and `OVERSIZED_ITEM` wire parity across C/Rust/Go; Rust malformed typed lookup response parity is now covered on POSIX and Windows; malformed follow-up responses after partial progress are now covered in C/Rust/Go on POSIX and Windows; reordered and duplicate response-item corruption is now covered in C/Rust/Go on POSIX and Windows; invalid status enum, invalid status-dependent field, and invalid label-table corruption are now covered in C/Rust/Go on POSIX and Windows; endpoint absence before call, endpoint disappearance after partial progress, and endpoint disappearance before the first subcall are now covered in C/Rust/Go on POSIX and Windows; zero-item typed lookup calls are now covered in C/Rust/Go on POSIX and Windows; duplicate and unsorted request keys under request splitting are now covered in C/Rust/Go on POSIX and Windows; full POSIX and Windows benchmark regenerations now pass; downstream topology-containers post-vendor validation now passes. Broader adversarial matrix review remains open.
+- External reviewer finding: coverage scripts and broader adversarial matrix still need updates before SOW completion. Coverage script expansion is now implemented; C/Rust/Go coverage gates pass; representative `8192` and `32768` logical-call tests now pass in C/Rust/Go on POSIX and Windows; POSIX and Windows baseline/SHM lookup-scale interop now pass across all C/Rust/Go directed pairs, including mixed-status lookup cases and heavier `65536` stress-only runs; lookup status codec interop now proves `PAYLOAD_EXCEEDED` and `OVERSIZED_ITEM` wire parity across C/Rust/Go; Rust malformed typed lookup response parity is now covered on POSIX and Windows; malformed follow-up responses after partial progress are now covered in C/Rust/Go on POSIX and Windows; reordered and duplicate response-item corruption is now covered in C/Rust/Go on POSIX and Windows; invalid status enum, invalid status-dependent field, and invalid label-table corruption are now covered in C/Rust/Go on POSIX and Windows; huge valid label isolation is now covered in C/Rust/Go on POSIX and Windows; endpoint absence before call, endpoint disappearance after partial progress, and endpoint disappearance before the first subcall are now covered in C/Rust/Go on POSIX and Windows; zero-item typed lookup calls are now covered in C/Rust/Go on POSIX and Windows; duplicate and unsorted request keys under request splitting are now covered in C/Rust/Go on POSIX and Windows; full POSIX and Windows benchmark regenerations now pass; downstream topology-containers post-vendor validation now passes. Broader adversarial matrix review remains open.
 
 Same-failure scan:
 
@@ -1095,7 +1112,7 @@ Lessons:
 Follow-up mapping:
 
 - Still open inside this active SOW:
-  - add any remaining broader adversarial tests from the planned matrix beyond the now-covered representative `8192`/`32768` logical-call cases, now-covered mid-logical timeout/abort cases, now-covered malformed follow-up responses after partial progress, now-covered reordered/duplicate response-item corruption, now-covered invalid status/status-dependent/label-table response corruption, now-covered endpoint absence before call, now-covered endpoint disappearance after partial progress, now-covered endpoint disappearance before the first subcall, now-covered zero-item typed lookup calls, now-covered stale request-capacity reconnect cases, now-covered duplicate/unsorted request keys under request splitting, now-covered request cap-minus-one/exact/plus-one boundaries, now-covered exact response-fit plus/minus-one boundaries, now-covered no-progress overflow cases, now-covered raw no-growth overflow cases, now-covered logical response-byte ceilings, now-covered mixed-status cross-language interop cases, and now-covered lookup status codec interop cases;
+  - add any remaining broader adversarial tests from the planned matrix beyond the now-covered representative `8192`/`32768` logical-call cases, now-covered mid-logical timeout/abort cases, now-covered malformed follow-up responses after partial progress, now-covered reordered/duplicate response-item corruption, now-covered invalid status/status-dependent/label-table response corruption, now-covered huge valid label isolation cases, now-covered endpoint absence before call, now-covered endpoint disappearance after partial progress, now-covered endpoint disappearance before the first subcall, now-covered zero-item typed lookup calls, now-covered stale request-capacity reconnect cases, now-covered duplicate/unsorted request keys under request splitting, now-covered request cap-minus-one/exact/plus-one boundaries, now-covered exact response-fit plus/minus-one boundaries, now-covered no-progress overflow cases, now-covered raw no-growth overflow cases, now-covered logical response-byte ceilings, now-covered mixed-status cross-language interop cases, and now-covered lookup status codec interop cases;
   - keep lookup-scale interop green across POSIX baseline, POSIX SHM, Windows Named Pipe, and Windows SHM; all four profiles now cover both all-known scale and mixed-status C/Rust/Go directed tests.
 
 ## Downstream Vendoring Plan

src/crates/netipc/src/service/raw_unix_tests.rs

@@ -1182,7 +1182,7 @@ fn test_lookup_zero_item_calls() {
 fn test_cgroups_lookup_transparent_payload_exceeded_retry() {
     let svc = unique_service("rs_svc_cgroups_lookup_scale");
     let mut cfg = server_config();
-    cfg.max_response_payload_bytes = 160;
+    cfg.max_response_payload_bytes = 256;
     let calls = Arc::new(std::sync::atomic::AtomicU32::new(0));
     let handler_calls = calls.clone();
     let handler = cgroups_lookup_dispatch(Arc::new(move |req, builder| {
@@ -1200,13 +1200,22 @@ fn test_cgroups_lookup_transparent_payload_exceeded_retry() {
             } else {
                 b"ok"
             };
+            let label_value;
+            let labels;
+            let labels_ref: &[(&[u8], &[u8])] = if item.as_bytes() == b"/huge-label" {
+                label_value = vec![b'l'; 512];
+                labels = [(b"huge".as_slice(), label_value.as_slice())];
+                &labels
+            } else {
+                &[]
+            };
             if builder
                 .add(
                     CGROUP_LOOKUP_KNOWN,
                     ORCHESTRATOR_K8S,
                     item.as_bytes(),
                     name_ref,
-                    &[],
+                    labels_ref,
                 )
                 .is_err()
             {
@@ -1221,13 +1230,18 @@ fn test_cgroups_lookup_transparent_payload_exceeded_retry() {
     connect_ready(&mut client);
 
     let view = client
-        .call_cgroups_lookup(&[b"/a".as_slice(), b"/huge".as_slice(), b"/b".as_slice()])
+        .call_cgroups_lookup(&[
+            b"/a".as_slice(),
+            b"/huge".as_slice(),
+            b"/huge-label".as_slice(),
+            b"/b".as_slice(),
+        ])
         .expect("cgroups lookup scale call");
     assert!(
         calls.load(Ordering::SeqCst) >= 2,
         "handler should be called for at least two subrequests"
     );
-    assert_eq!(view.item_count, 3);
+    assert_eq!(view.item_count, 4);
     assert_eq!(view.generation, 7);
     let item0 = view.item(0).expect("item 0");
     assert_eq!(item0.status, CGROUP_LOOKUP_KNOWN);
@@ -1236,9 +1250,12 @@ fn test_cgroups_lookup_transparent_payload_exceeded_retry() {
     assert_eq!(item1.status, CGROUP_LOOKUP_OVERSIZED_ITEM);
     assert_eq!(item1.path.as_bytes(), b"/huge");
     let item2 = view.item(2).expect("item 2");
-    assert_eq!(item2.status, CGROUP_LOOKUP_KNOWN);
-    assert_eq!(item2.path.as_bytes(), b"/b");
-    assert_eq!(item2.name.as_bytes(), b"ok");
+    assert_eq!(item2.status, CGROUP_LOOKUP_OVERSIZED_ITEM);
+    assert_eq!(item2.path.as_bytes(), b"/huge-label");
+    let item3 = view.item(3).expect("item 3");
+    assert_eq!(item3.status, CGROUP_LOOKUP_KNOWN);
+    assert_eq!(item3.path.as_bytes(), b"/b");
+    assert_eq!(item3.name.as_bytes(), b"ok");
 
     client.close();
     server.stop();
@@ -1267,6 +1284,15 @@ fn test_apps_lookup_transparent_payload_exceeded_retry() {
             } else {
                 b"/ok"
             };
+            let label_value;
+            let labels;
+            let labels_ref: &[(&[u8], &[u8])] = if pid == 44 {
+                label_value = vec![b'l'; 512];
+                labels = [(b"huge".as_slice(), label_value.as_slice())];
+                &labels
+            } else {
+                &[]
+            };
             if builder
                 .add(
                     PID_LOOKUP_KNOWN,
@@ -1279,7 +1305,7 @@ fn test_apps_lookup_transparent_payload_exceeded_retry() {
                     b"ok",
                     cgroup_path_ref,
                     b"name",
-                    &[],
+                    labels_ref,
                 )
                 .is_err()
             {
@@ -1294,13 +1320,13 @@ fn test_apps_lookup_transparent_payload_exceeded_retry() {
     connect_ready(&mut client);
 
     let view = client
-        .call_apps_lookup(&[11, 22, 33])
+        .call_apps_lookup(&[11, 22, 44, 33])
         .expect("apps lookup scale call");
     assert!(
         calls.load(Ordering::SeqCst) >= 2,
         "handler should be called for at least two subrequests"
     );
-    assert_eq!(view.item_count, 3);
+    assert_eq!(view.item_count, 4);
     assert_eq!(view.generation, 9);
     let item0 = view.item(0).expect("item 0");
     assert_eq!(item0.status, PID_LOOKUP_KNOWN);
@@ -1310,9 +1336,12 @@ fn test_apps_lookup_transparent_payload_exceeded_retry() {
     assert_eq!(item1.status, PID_LOOKUP_OVERSIZED_ITEM);
     assert_eq!(item1.pid, 22);
     let item2 = view.item(2).expect("item 2");
-    assert_eq!(item2.status, PID_LOOKUP_KNOWN);
-    assert_eq!(item2.pid, 33);
-    assert_eq!(item2.comm.as_bytes(), b"ok");
+    assert_eq!(item2.status, PID_LOOKUP_OVERSIZED_ITEM);
+    assert_eq!(item2.pid, 44);
+    let item3 = view.item(3).expect("item 3");
+    assert_eq!(item3.status, PID_LOOKUP_KNOWN);
+    assert_eq!(item3.pid, 33);
+    assert_eq!(item3.comm.as_bytes(), b"ok");
 
     client.close();
     server.stop();