New env-gated background loop that runs the same PxL shape AE's
anomaly-gated path uses, but with an empty Target (no ns/pod predicate)
and over a configurable rolling window. Writes via the existing sink so
the byte-shape of forensic_db rows is comparable between the
PASSTHROUGH=1 phase (EVERYTHING) and the PASSTHROUGH=0 phase
(AE-FILTER). One-shot A/B that yields the per-table capture fraction of
the adaptive write path.

- internal/passthrough/passthrough.go — Loop + Config; defaults to
  30s window / 30s refresh / clickhouse.PixieTables() table list.
- internal/passthrough/passthrough_test.go — 6 tests; the load-bearing
  one is TestLoop_EmitsEmptyTargetPxL (asserts neither df.namespace nor
  df.pod predicates appear in the emitted PxL).
- cmd/main.go: ADAPTIVE_PASSTHROUGH + _WINDOW_SEC + _REFRESH_SEC env
  knobs. Adapter is constructed unconditionally when passthrough is on
  (joins the existing PushPixie / streaming construction path so the
  same pxapi grpc stream is reused). Loop is registered with the
  shutdown WaitGroup so SIGTERM waits for the in-flight tick.
- cmd/BUILD.bazel: drop @px// load (other AE BUILD.bazel files use
  //bazel — sticking out as the only one with @px is a leftover from a
  prior gazelle run; align). Add passthrough dep.

Signed-off-by: entlein <einentlein@gmail.com>

Stand-alone workflow that builds entlein/dx (private Active-Diagnosis
Framework) into ghcr.io/k8sstormcenter/dx-daemon. Separates the dx image
publish from the bazel-based vizier_release pipeline; the dx repo ships
its own Dockerfile.dxd (Go cross-compile + distroless final stage) so it
doesn't need to live as a submodule inside src/vizier/services/dx.

Triggers:
- tag push 'release/dx/v*' on this repo cuts a release build, image tag
  derived from the tag suffix (release/dx/v0.1.0 -> image tag 0.1.0).
- workflow_dispatch lets us build any dx ref on demand with a custom tag
  (default: short sha of the resolved dx commit).

Pulls dx via DX_ENTLEIN_PAT (already configured on the repo). Multi-arch
build (linux/amd64, linux/arm64); Dockerfile.dxd cross-compiles in the
native BUILDPLATFORM stage and the final stage is COPY-only, so target
emulation isn't required.

Signed-off-by: entlein <einentlein@gmail.com>

The dx image build pipeline lives in entlein/dx itself (PR #53,
branch feat/bazel-release): bazel-based with @px external pin to
pixie's ae-prod tip, pushes to docker.io/entlein/dx-daemon on a
release/dx/v* tag in the dx repo. The pixie-side buildx workflow
this reverts duplicated that intent in the wrong repo + the wrong
build system (docker buildx instead of bazel + pl_go_image macros)
+ the wrong registry (ghcr.io/k8sstormcenter instead of
docker.io/entlein).

Signed-off-by: entlein <einentlein@gmail.com>

…affordances

Fixes the silent-halt bug: the trigger gated on a RAW event_time high-water-mark,
so a single anomaly in a larger unit (ms/ns) drove the watermark past all real
seconds rows and AE stopped processing forever (data still on Pixie). Normalize
event_time to canonical nanoseconds in the poll SELECT filter+order and in the
in-memory/persisted cursor, boundary-dedup, and maxSeen (normalizeEventTimeNanos
+ chNormEventTimeNanos). Validated at the data layer: vs a poisoned watermark the
raw filter returns 0 rows, the normalized filter recovers all 60.

Also adds ADAPTIVE_PUSH_REFRESH_SEC (negative = single-shot pull) for the
reproducible load-test harness, an in-package trigger unit test, and an e2e
hermetic load test (mock PixieQuerier, exact rows+bytes).

Signed-off-by: entlein <einentlein@gmail.com>

…ness

Consolidate the adaptive_export load-test harness under src/e2e_test/ (matching
vzconn_loadtest / px_cluster conventions). Control-plane experiments (E1-E4, E6,
E8 sustained) are proven exactly-reproducible on a live rig; the data-plane
experiments (E5, E8 data-mode) are authored and pending live validation on a
vizier-registered rig (status documented in README + FINDINGS_AND_BACKLOG).

- harness/: shell + python (inject, exp_control, exp_e8, stats, ...) + lib helpers
- fixtures/EXPERIMENTS.md: curated kubescape_logs data-set catalog + expected outputs
- k8s/: isolated sinks + per-rep generator pod (no probes)
- tools/loadgen/: cleanloadgen + httpsink (docker-built test tool; .bazelignore'd
  pending a bazel target — lib/pq is already vendored in the module)
- FINDINGS_AND_BACKLOG.md: F8 watermark-poison bug + the fix + AE backlog

The AE Go unit/e2e tests live with the service (internal/{trigger,e2e}).

Signed-off-by: entlein <einentlein@gmail.com>

…C14) + diagrams

Signed-off-by: entlein <einentlein@gmail.com>

…iagram; gen sustained-DNS mode

C15 = AE must keep re-pulling+writing an active pod until t_end or DX stop (the
contract DX steers on; last week's 'wrote then stopped' is its violation). Add
DX-steering sequence diagram. Generator gains SUSTAIN_SEC (distinct-DNS trickle,
a Pixie-traced protocol) + configurable SETTLE_PRE_MS warm-up for fresh-pod
capture; harness wires GEN_SUSTAIN_SEC/GEN_SETTLE_MS.

Signed-off-by: entlein <einentlein@gmail.com>

…lisation

The 700821d trigger unit-normalisation wrapped event_time in a
multiIf(...) inside both the WHERE filter and the ORDER BY. Three
existing tests in watermark_test.go + one in clickhouse_test.go pinned
the raw 'event_time >= N' substring and broke at HEAD.

Update each test's expected substring to match the new normalized form
(') >= <ns-scaled N>' — the closing paren of multiIf, then the value
in canonical nanoseconds). Per-test ns-scaling:

  watermark_test.go:94  1744000000000000000  already ns -> unchanged
  watermark_test.go:125 InitialWatermark=42  < 1e10 sec -> * 1e9
  watermark_test.go:156 InitialWatermark=7   < 1e10 sec -> * 1e9
  watermark_test.go:297 event_time='5000'    < 1e10 sec -> * 1e9
  clickhouse_test.go:82 ref.T=1744..303e9 ns already ns -> unchanged

go test ./src/vizier/services/adaptive_export/... all green.

Signed-off-by: entlein <einentlein@gmail.com>

…future-stamp watermark poison)

Signed-off-by: entlein <einentlein@gmail.com>

Records one forensic_db.ae_reconcile row per data-plane pull (read_count vs
wrote_count, window, ns/pod) across ALL three write paths — controller fan-out
(filter), passthrough firehose, and streaming scanner — so a reconcile run
localizes loss to query (R5: read<PEM) vs sink (R6: wrote<read) and quantifies
re-pull dup (C8). Counts alone (write >= read) were proven insufficient.

- new internal/reconcile leaf package (Row, Recorder, Nop) — no import cycle
- sink.Record: CH-backed recorder (INSERT INTO forensic_db.ae_reconcile)
- ae_reconcile table: schema.sql + KnownTables + OperatorOwnedTables (synced);
  not a pixie table (absent from PixieTables, so VerifyPixieSchema ignores it)
- wired: passthrough.tick, controller.pushPixieRows (deferred, all return
  paths), streaming.scanner.Run; gated by ADAPTIVE_RECONCILE=true, else Nop
- unit test proves read/wrote capture incl. the sink-drop read>wrote shape
- fixed apply_test trailing-tables guard for the new operator table
- harness: exp_row_reconcile.sh (row-level PEM<->CH), ae_vs_all.sh, exp_datavolume_extreme.sh

Signed-off-by: entlein <einentlein@gmail.com>

px -o json empty result previously printed one blank line → counted as a phantom
LOSS=1. Guard: empty set → 0-byte keys file; drop all-empty-field keys. Confirmed
against the controlled log4j run (backend http 14/14 exact, conn 66>=12, no loss).

Signed-off-by: entlein <einentlein@gmail.com>

…log4shell

Reliable BY CONSTRUCTION against bob#140 (stateful/unreliable exploit on re-fire):
fresh-JVM backend (delete pod) + attacker-before-backend + the WORKING resolvable FQDN
attacker.<ns>.svc.cluster.local:1389 (NOT the bare attacker-ns.svc which NXDOMAINs and
gets dropped), then VERIFY the actual backend->:1389 LDAP egress in forensic_db.conn_stats
and RETRY until confirmed (the validity gate). Never assumes the exploit fired. Node-side.

Signed-off-by: entlein <einentlein@gmail.com>

…tion)

Reword from offensive 'exploit' to detection-signal-generation language: this validates
the kubescape->DX->AE detection chain. No logic change.

Signed-off-by: entlein <einentlein@gmail.com>

… http2

- pxl.CompilePassthrough/Render: precompile per-table PxL once (fixed
  window => constant relative start_time), only the two time_ bounds are
  stamped per tick. Rendered output is byte-identical to QueryFor with an
  empty Target (TestCompilePassthrough_MatchesQueryFor), so this is a
  structural change, not a capture change. upid->pod/ns stays in PxL.
- passthrough: tickConcurrent fans every table out at once (was a serial
  loop); shared pull() helper. Sink/recorder are pool/HTTP-backed and
  already called concurrently elsewhere.
- drop http2_messages.beta from the firehose set (not materialised on
  every cluster => ""Table not found"" every tick); shared PixieTables/DDL
  lists untouched.
- toggle ADAPTIVE_PASSTHROUGH_COMPILED (default on; =false reverts to the
  legacy serial QueryFor path).

Signed-off-by: entlein <einentlein@gmail.com>

…e.go

Fixes "missing strict dependencies: import of .../internal/reconcile" that
broke the AE image build for passthrough, sink, streaming, controller, cmd
(pre-existing since the ADAPTIVE_RECONCILE commit added the package + imports
without bazel deps; never CI-built). Also wires the new pxl/compile.go srcs
+ passthrough/pxl test srcs (compiled_test.go, reconcile_test.go, compile_test.go).

- new internal/reconcile/BUILD.bazel (go_library, stdlib-only)
- +//internal/reconcile dep: passthrough, sink, streaming, controller, cmd
- pxl go_library +compile.go; pxl_test +compile_test.go; passthrough_test +compiled_test.go,reconcile_test.go (+reconcile dep)

Signed-off-by: entlein <einentlein@gmail.com>

F1 RCA: Pixie caps every px.display at max_output_rows_per_table (default
10000, query_flags.go) — the planner add_limit_to_batch_result_sink_rule
silently truncates wide firehose windows / busy pods at the READ (write path
is clean: ae_reconcile shows read==wrote). Fix uses Pixie own native knob —
prepend `#px:set max_output_rows_per_table=1000000` to every generated PxL
(QueryFor + CompilePassthrough) so all pull paths are uncapped. Validated on
rig: a 14208-row window returned 10000 (capped) vs 14298 (with flag). No
pagination loop, no extra round-trips. See memory project-ae-passthrough-10k-cap.

Signed-off-by: entlein <einentlein@gmail.com>

Consolidates the recurring content_type silent-drop incident class
into one default-suite test gate (6 tests, ~15ms):

  I1 TestContract_ContentTypeIsInt64InSchema
  I2 TestContract_FastEncodeContentTypeAsInt
  I3 TestContract_SilentDropDetected
  I3.b TestContract_SilentDropNotTriggeredOnSuccess
  I3.c TestContract_SilentDropToleratesMissingSummaryHeader
  I4 TestContract_HTTPEventsRoundTrip

Top-of-file docstring chronicles the incident timeline so future
operators can grep their way to the contract.

Signed-off-by: entlein <einentlein@gmail.com>

Measures the AdaptiveExport value prop: datavolume REDUCTION of DX-steered AE
(rev-3 streaming, AE writes only DX-steered activeSet pods over the control
surface) vs saving ALL data (passthrough firehose). Two arms, same fixed load,
forensic_db active-part deltas (rows+bytes) per table; reduction = 1 - DX/ALL.
Uses the canonical resolvable JNDI FQDN (attacker.attacker-ns.svc.cluster.local)
so the chain fires + DX can classify (a malformed host → NXDOMAIN → no steer).
Successor to ae_vs_all.sh, whose AE arm used the rev-2 controller gate + stale JNDI.

Signed-off-by: entlein <einentlein@gmail.com>

Measures all AE non-functional requirements under steady load on the rig:
throughput (rows+bytes/sec), capture completeness (AE read vs broker count =
F1 cap proof), write fidelity (read==wrote + write-error count), end-to-end
freshness latency (now - max(time_) in CH), resource footprint (AE pod cpu/mem
idle vs loaded), per-cycle cadence. Emits a structured report; companion to
exp_dx_steering_reduction.sh. Real-data only.

Signed-off-by: entlein <einentlein@gmail.com>

…ring + live-pod guard)

Run-1 reported false 100% reduction because stale adaptive_attribution windows
rehydrated DEAD pods (deleted loaders) into the activeSet → AE streamed dead pods
→ 0 rows. Clear adaptive_attribution before the DX arm so the activeSet only gets
freshly-steered LIVE pods; add a guard that prints the steered pods + marshalsec
fire count + live log4j-poc pods so a dead-arm result is caught, not reported as
a reduction.

Signed-off-by: entlein <einentlein@gmail.com>

lag query used now()-DateTime64 (type error -> na); use dateDiff(second,...).
Capture-completeness vs broker was window-misaligned (623% artifact) -> drop it;
report tot_read vs tot_wrote (read==wrote) + errs instead. The 10k-cap/completeness
proof is the dedicated F1 test (max_read>10000 vs broker for the SAME window).

Signed-off-by: entlein <einentlein@gmail.com>

…ary)

Run-2 byte-delta reduction came out negative because system.parts byte delta is
compaction-noisy (merges land mid-window). Report rows reduction as primary
(actual captured-row count, noise-free); keep bytes as secondary context.

Signed-off-by: entlein <einentlein@gmail.com>

Eviction-RCA finding (PR #63 NFR campaign): AE had NO memory limit (only cpu
300m) and was CPU-pinned at 300m under concurrent passthrough. AE measured tiny
(16-38Mi steady), but the raised 1M-row passthrough cap can spike, so cap at 1Gi
so AE can never memory-pressure a node; raise cpu limit 300m->1 core (was
throttling). NOTE node evictions were NOT AE/OOM — node-01 went NotReady
(network/heartbeat); the memory consumer is PEM (1365Mi, OOMs at the 2Gi default).

Signed-off-by: entlein <einentlein@gmail.com>

Root cause of the recurring "AE unauthenticated / writes 0 / crashloop" reverts:
kustomization.yaml bundled adaptive_export_secrets.yaml (placeholder pixie-api-key)
with the role+deployment, so EVERY infra re-apply (make log4j) clobbered the real
key that ae-auth had written. Separation of concerns: remove the secret from the
kustomization — infra (role+deployment) stays re-appliable; the secret holds real
creds and is owned solely by `make ae-auth`, created once, never touched by infra
re-applies. Secret manifest kept as a hand-applied seed-only template (documented).

Signed-off-by: entlein <einentlein@gmail.com>

The DX-steered arm was failing because the harness only fired stage-1 (JNDI/LDAP).
That generates ldap-egress but NO kubescape R0001 → backend never flagged → DX no
case → indeterminate → AE steers wrong/no pods. R0001 comes from stage-2 (post-
exploitation exec). fire() now does stage-1 (JNDI) + stage-2 (whoami/shadow/token/
getent in the backend) → kubescape R0001+R0006 → DX rules backend MALIGNANT →
backend enters AE activeSet → reduction is measurable. Verified live: DX evidence
unexpected-spawn+sensitive-file-read → verdict ruled_in generic=MALIGNANT.

Signed-off-by: entlein <einentlein@gmail.com>

…dd DX-steering diagnostics

Standing terminology rule: allowlist/blocklist, never whitelist/blacklist.
Pure rename (no behavior change) of the rev-3 streaming filter:
  FilterModeWhitelist  → FilterModeAllowlist
  MaxWhitelistSize     → MaxAllowlistSize
  ADAPTIVE_STREAM_MAX_WHITELIST → ADAPTIVE_STREAM_MAX_ALLOWLIST  (env)
  mode=whitelist log string → mode=allowlist
plus all comments/identifiers/tests in streaming, activeset, cmd/main.

DX-steering diagnostics (the reason DX-arm-writes-0 has been hard to RCA —
we could not tell "empty ActiveSet" from "broker returned 0 rows"):
  - scanner: log the empty-allowlist short-circuit (was silent) so an
    empty ActiveSet is visible in logs, distinct from "query completed rows=0".
  - FilterUpdater: emitted-filter log Debug→Info so the steered pod count
    per ActiveSet change is visible without debug logging.

NOTE: ADAPTIVE_STREAM_MAX_WHITELIST env renamed → tooling that sets the old
name must switch to ADAPTIVE_STREAM_MAX_ALLOWLIST.

Signed-off-by: entlein <einentlein@gmail.com>

The Pixie dx_evidence_graph UI reads dx_attack_graph via px.DataFrame
clickhouse_dsn, whose query template hardcodes event_time + hostname and
ORDER BY event_time. A table without those columns fails 'Unknown
identifier event_time'; a table created by hand (local, not via the
operator) isn't globally registered. Fix: make AE own it like the other
forensic tables.

- schema.sql: dx_attack_graph DDL with event_time(UInt64 nanos) + hostname,
  edge columns, fromUnixTimestamp64Nano partition/TTL (nanos-correct).
- KnownTables + OperatorOwnedTables: register it so Apply creates it at boot.
- apply_test: assert last-applied DDL == last OperatorOwnedTables entry
  (robust to appended operator tables) instead of hardcoding trigger_watermark.

go test ./.../clickhouse green.

Signed-off-by: entlein <einentlein@gmail.com>

Pixie's clickhouse_dsn type mapper reads UInt8 as BOOLEAN and does not
handle UInt16/UInt32/Float32 -> px fails with 'Column[N] given incorrect
type' rendering the dx_evidence_graph. weight/max_severity/num_findings
-> Int64, confidence -> Float64 (map cleanly to INT64/FLOAT64). Verified
live: px run returns all 6 edges with every column. event_time stays
UInt64 (matches kubescape_logs, which px reads).

Signed-off-by: entlein <einentlein@gmail.com>

…canner buildPxL

The DX/streaming arm silently capped each per-table pull at Pixie's default
10000-row limit while the passthrough/ALL arm (pxl.CompilePassthrough /
QueryFor) already raises it to 1,000,000 via the broker's #px:set query flag.
Validated live on 6a33dac0: a single streaming http_events pull returned
exactly rows=10000 (the cap). Left unfixed this UNDER-counts the DX arm and
OVERSTATES the DX-vs-ALL volume reduction. Prepend the same #px:set directive
to the streaming scanner's PxL so both arms are uncapped and comparable.

Signed-off-by: entlein <einentlein@gmail.com>

dx POSTs a JSON array of edges to /dx/attack_graph; AE writes them to
forensic_db.dx_attack_graph via JSONEachRow (Applier.WriteAttackGraph). Wired in
main.go when CONTROL_ADDR is set; 501 if the sink is unset. This is the AE half
of the dx->AE->CH attack-graph write path.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Signed-off-by: entlein <einentlein@gmail.com>

…e, fix stale tests

PR-53 review follow-ups (see review summary in conversation):

1. License headers added to 17 src/e2e_test/adaptive_export_loadtest/
   harness/*.{sh,py} scripts. Matches the convention every other
   src/e2e_test/*/sh in pixie already follows.

2. Dead code removed (was unreachable per golang.org/x/tools/cmd/deadcode):
   - internal/script/script.go: IsClickHouseScript, IsScriptForCluster,
     GetActions, getScriptName, getInterval, templateScript, plus the
     ScriptConfig and ScriptActions types. The cron-script sync flow they
     served was replaced by the streaming model; only the Script and
     ScriptDefinition types remain.
   - internal/pixie/pixie.go: Client.GetPresetScripts (replaced by
     builtinPresetScripts() inline in cmd/main.go).
   - internal/streaming/{supervisor,writer}.go: SupervisorStats,
     TableStats, Stats types + Supervisor.Stats and BatchWriter.Stats
     methods (no production reader). Atomic counters dropped; the
     existing flush log preserves the per-flush summary.

3. Stale passthrough tests fixed.
   TestLoop_DefaultsTablesToPixieTables and TestNew_AppliesDefaults
   asserted len(clickhouse.PixieTables()) == 13 but passthrough.New
   strips excludedTables (http2_messages.beta), yielding 12. Tests now
   compare against filterExcluded(clickhouse.PixieTables()) and add an
   extra assertion that excluded tables were not written.

4. ClickHouse HTTP client consolidation: new internal/chhttp/ package
   collapses three near-identical HTTP CH clients
   (clickhouse.Applier, sink.ClickHouseHTTP, trigger.ClickHouseWatermarkStore)
   into one. Centralises endpoint validation, basic-auth header,
   30s default timeout, fail-loud INSERT settings (4 CH input_format
   knobs), and the X-ClickHouse-Summary read path. The 4 INSERT
   call sites in the three callers all route through chhttp.Client.Insert
   now; SELECT through Query or QueryStream (the latter preserves the
   QueryActive streaming behaviour). Net code: -200 LOC across the
   three callers plus a 200-LOC chhttp package with its own tests.

5. Pixie service scaffold wired into cmd/main.go:
   services.SetupService("adaptive-export", 50900) +
   services.SetupSSLClientFlags() + services.PostFlagSetupAndParse() +
   services.SetupServiceLogging(). Matches the pattern every other
   pixie Go service uses. CheckServiceFlags() is deliberately skipped
   (AE does not run a TLS gRPC server). Existing AE env-var reads
   (ADAPTIVE_*) are untouched and still authoritative for tuning knobs.

All 14 adaptive_export internal packages pass go test (1 new chhttp,
13 unchanged), including the 3 differential oracle tests in
pxl/compile_test.go. arc lint OKAY for everything except 3 pre-existing
findings unrelated to this commit (loadgen has its own go.mod; one
QF1001 De Morgan's-law nit in reconcile_test.go from c9f19b6).

Signed-off-by: entlein <einentlein@gmail.com>

…der)

Signed-off-by: entlein <einentlein@gmail.com>

User flagged on review 4536971862:
- cmd/BUILD.bazel:18 dropped the '@px' external-workspace prefix on
  pl_build_system.bzl load. Restored — the standalone AE build pulls
  pixie as @px and needs the qualifier.

Lint cleanup (PR-53 scope, no production code touched outside renames):
- chhttp/chhttp_test.go: errcheck on two w.Write calls + gofumpt on
  the gotSettings declaration.
- passthrough/reconcile_test.go: staticcheck QF1001 (De Morgan's law)
  on the conn_stats sink-drop assertion.
- script/script.go: rename ScriptId -> ScriptID (ST1003). Propagated
  to pixie/pixie.go and cmd/main.go callsites.
- internal/e2e/BUILD.bazel and internal/trigger/BUILD.bazel: gazelle
  drift — adding loadtest_test.go and clickhouse_internal_test.go to
  srcs.
- k8s/00-sinks.yaml + gen-pod.tmpl.yaml: yamllint compliance —
  document-start marker, dedent sequence items per .yamllint
  indent-sequences=false, tighten flow-mapping spaces, collapse
  multi-space after commas. YAML semantics unchanged.
- harness/stats.py: flake8 E501 — split a long line into two.

Final arc lint state on PR-53 file scope: 0 Errors, 14 Warnings
(SHELLCHECK SC2155/SC2086 in pre-existing harness scripts from
ae7b86f, not introduced or modified by this commit). Pre-existing
loadgen typecheck failures (separate go.mod) are unaffected.

Signed-off-by: entlein <einentlein@gmail.com>

…bit items

User review 4536971862:

#2 passthrough/reconcile_test.go — strengthened with two new tests that
   exercise the FULL chain (loop → real sink.ClickHouseHTTP → httptest
   CH endpoint → reconcile recorder). TestTick_ReconcileCatchesCHSilentDrop
   mimics CH's X-ClickHouse-Summary silent-drop shape (200 OK,
   written_rows=0) and asserts the loop records WroteCount=0 with a
   silent-drop attribution — the exact R6 (sink-layer loss) regression
   the instrument exists to detect. TestTick_ReconcileAttributesCHFailureCorrectly
   covers the 500-response branch. The pre-existing in-process-fake test
   stays as the wiring check.

#4 pixie/pixie.go — added a long comment justifying the API-key auth
   choice. pixie.Client targets the Pixie CLOUD (cloudpb's PluginService),
   whose auth interceptor accepts pixie-api-key and rejects JWT service
   tokens (those are for INSIDE-cluster vizier services). The
   pixieapi.Adapter that talks to vizier directly already uses JWT via
   jwtutils.GenerateJWTForService — same pattern as
   cloud_connector/vizhealth/checker.go:111. So the split is intentional;
   flipping pixie.go to JWT would break cloud auth, not improve it.

#6 pxl/queryfor — hardened escapePxL so a raw \n, \r, \t or NUL byte
   in Target.Pod/Target.Namespace can't terminate the PxL string literal
   and inject a new statement. Added TestQueryFor_RejectsInjectionInTargetFields
   driving QueryFor with 7 adversarial pod/namespace shapes (newline,
   single-quote-only, CR, backslash-escape-of-escape, NUL, tab) plus
   the regex_match fallback path, asserting the output line count and
   every statement's leading token. Extends existing
   TestEscapePxL_TableDriven with the new escape mappings.

CodeRabbit oversights (verified each against current code):

CR r3379377432 (main.go) — wrapped the control-surface listener in
   http.Server with Read/ReadHeader/Write/Idle timeouts so a slow
   client can't pin a goroutine indefinitely.

CR r3379377607 (pixieapi.go) — switched direct-mode dial from
   pxapi.WithDisableTLSVerification (env + addr-gated, brittle) to
   pxapi.WithDirectTLSSkipVerify (added in PR #49 b523ce3 for the
   same node-IP-dial scenario). Removed the cluster.local + PX_DISABLE_TLS
   precondition in NewDirect; the always-skip semantics match the AE
   deployment shape. Refactored the obsolete env-gate test.

CR r3379377645 (streaming/filter.go) — the deltaCh-close path returned
   without calling disarm(), leaking the timer's goroutine on shutdown.
   Now calls disarm() before return on chan-close.

CR r3426923299 (sink/clickhouse.go Record) — capped Record at a 2s
   per-call context timeout. The 30s chhttp default was too long for
   the scanner/passthrough/controller hot paths that call Record inline;
   reconcile is best-effort by contract, so a stalled CH must not pin
   the pull loop.

All 14 AE packages pass go test. arc lint: 0 Errors on PR-53 file
scope.

Signed-off-by: entlein <einentlein@gmail.com>

Finish the harness consolidation #53 started: adopt exp_matrix.sh (canonical
ALL-vs-DX reduction matrix) + nfr.sh (throughput/mem/verdict-latency) as the
single runners; cut the overlapping variants (ae_vs_all, exp_datavolume_extreme,
exp_dx_steering_reduction, exp_ae_nfr_benchmark, exp_pipeline_reconcile) and the
superseded standalone setup (deploy_ae, build_gen_image). README rewritten as the
single 'how to run' source: two families — fixture-isolation (run.sh + E-series)
and live-attack e2e (log4shell_fire → exp_matrix → nfr → exp_row_reconcile).

Add e2e_log4shell_soc.yaml: k3s + full SOC stack (Pixie/kubescape/ClickHouse/AE/dx
via k8sstormcenter/soc) on the oracle runner; asserts every canonical harness
script runs + dx rules in; profiles dx in real life (pprof CPU/heap + verdict
latency, uploaded). Uses existing repo secrets.

Signed-off-by: entlein <einentlein@gmail.com>

…ntainer_images BUILD

This file is unrelated to adaptive_export — the only change was buildifier
alphabetizing container_type/bazel_sdk_versions (no functional change). Restore
main's version to keep #53's diff to real AE changes.

Signed-off-by: entlein <einentlein@gmail.com>

run-genfiles: the PR had reordered the kwargs in stirling/.../container_images/
BUILD.bazel alphabetically (bazel_sdk_versions before container_type) — a
local buildifier or gazelle drift from when the file was first committed.
CI gazelle wants the original order (container_type first), so the diff
loop fails. Restored to origin/main's ordering. Local gazelle disagrees
with CI's expected output (tooling-version drift); CI is authoritative.

run-container-lint: two findings.

  1. staticcheck QF1001 (De Morgan's law) in
     pxl/queryfor_test.go:318. Rewrote the !(a || b || c || d) negated
     disjunction as the equivalent !a && !b && !c && !d. Test behaviour
     unchanged; verified locally with TestQueryFor_RejectsInjection.

  2. golangci-lint typechecking failed on
     src/e2e_test/adaptive_export_loadtest/tools/loadgen/cmd/{cleanloadgen,
     httpsink}/main.go because that subtree carries its own go.mod and
     does not resolve as a package under the root px.dev/pixie module.
     Added the loadgen subtree to .arclint's exclude list. Same fix the
     existing entries for other-module subtrees apply.

Local 'arc lint' on the PR-53 file scope: 0 Errors, 14 SHELLCHECK
Warnings + 26 SHELLCHECK Advice (all pre-existing in ae7b86f
harness scripts; not introduced by this PR).

Signed-off-by: entlein <einentlein@gmail.com>

run-genfiles CI re-failed after f244ffc reverted this file to main's
ordering — turns out gazelle on this repo IS alphabetizing the kwargs
(bazel_sdk_versions before container_type), and CI runs 'gazelle fix'
then 'git diff' to catch any drift. So main's ordering is no longer
gazelle-stable; the file has to match gazelle's preference, not main's.

Verified locally with 'bazel run //:gazelle -- fix'; the file is now
idempotent (a second gazelle run produces no diff).

Signed-off-by: entlein <einentlein@gmail.com>

run-container-lint re-failed after the run-genfiles fix because two
files added on this branch (a03aa15) had unfixed lint errors:

.github/workflows/e2e_log4shell_soc.yaml — 5 yamllint Errors:
- 1 indentation: list items under steps: must be parent-aligned per
  the repo's .yamllint config (indent-sequences: false), not 2-indented.
  Dedented every step item + its run: block by 2 spaces.
- 4 line-length (>120 chars): split the long kubectl-set-image, the
  long grep-detection gate, the curl pprof URL, and the verdict-latency
  grep across continuation lines. Semantics unchanged.

src/e2e_test/adaptive_export_loadtest/harness/{exp_matrix.sh, nfr.sh}
— missing Apache headers. Applied via arc lint --apply-patches; exec
bits restored.

Local arc lint on PR-53 file scope: 0 Errors, 14 SHELLCHECK Warnings
+ 26 Advice (all pre-existing in harness scripts, unchanged).

Signed-off-by: entlein <einentlein@gmail.com>

arc lint --apply-patches exits non-zero on Warning level too. Resolved
each: replaced unused 'for i/t in ...' loop vars with '_', split a
SC2155 declare-and-assign, dropped two never-referenced hip/pip
assignments.

Signed-off-by: entlein <einentlein@gmail.com>

…owup)

AE control endpoints had no auth. Add it using the SAME shared lib the vizier
broker/PEM use (px.dev/pixie/src/shared/services/utils): SetAuth verifies a
bearer JWT via jwtutils.ParseToken (signature + audience), middleware on
Handler() with /healthz exempt. dx already mints this exact service JWT
(GenerateJWTForService, PL_JWT_SIGNING_KEY) — it just attaches it. No new
secret/crypto. Flag-gated (CONTROL_REQUIRE_AUTH + PL_JWT_SIGNING_KEY), default
OFF so it merges before dx sends the bearer; flip on after dx is updated.

Also: reject invalid t_end (<=0) and query windows (start>=end, non-positive).
Test mints a real JWT via the shared lib; asserts 401 on missing/bad token,
pass on valid, /healthz open.