Skip to content

Commit 301e0d3

Browse files
hyperpolymathclaude
andcommitted
perf(d-3): dedicated trust-header rewrite + mTLS handshake scenarios
Phase D-2 (PR #14) left two cost surfaces folded into the "exact route allow (proxy 200)" scenario: the Phase A contract-header construction in Proxy.build_backend_headers/1, and the transport-level mTLS handshake. A baseline collected from D-2 would have attributed both to the proxy-200 number, so a regression in either would only show up as an aggregate slowdown -- the comparator couldn't tell us which leg moved. Phase D-3 (this change, hyperpolymath/standards#99 sub-deliverable) pulls each into its own Benchee scenario so D-4 baseline collection attributes them independently: * "trust-header rewrite (Proxy.build_backend_headers)" - direct call to a new @doc false Proxy.__benchmark_build_backend_headers__/1 seam over the existing private build_backend_headers/1. No policy lookup, no Gateway.call/2 pipeline, no network I/O. Isolates the cost surface the Phase A contract invariant lives on: the gateway sets X-Trust-Level / X-Request-ID as authoritative values that shadow any client-supplied header, and the cost of building that final header map is what this scenario measures. * "mTLS handshake (test CA)" - raw :ssl acceptor + :ssl.connect/4 + immediate :ssl.close/1, using a test CA chain generated in-memory at bench startup via :public_key.pkix_test_data/1. Each iteration is one fresh handshake (verify_peer + cert chain validation + key exchange) closed before any application bytes are exchanged. Bounds the connection-spike SLO Phase E rollout has to budget for. Why in-memory rather than reusing the committed Phase B fixture in test/fixtures/mtls/: the fixture ships only *.crt files -- *.key is gitignored at the repo root. Reusing the committed certs would require committing the matching keys (or carving a fixture exception in .gitignore), which the estate security posture prefers we don't do for a bench fixture. The chain shape (test CA -> server peer + test CA -> client peer, verify_peer enforced) is identical to Phase B's fixture; only the key material differs, and it never touches disk. bench/baseline.json gains the two new scenario keys (both TODO; the _status stays "scaffold-placeholder" because D-3 still doesn't collect real numbers -- D-4 does, via the `perf: rebaseline` ritual in docs/perf-contract.md). _schema_version bumps 0.1.0 -> 0.2.0 to signal the shape change to anyone diffing the file or running compare.exs against an older results.json. docs/perf-contract.md moves both items from "Out of scope" to in scope, adds them to the scenarios table and the targets table, and re-anchors the remaining out-of-scope list on D-4 (baseline collection), Phase E (historical dashboard), and a possible follow-up amortised-handshake scenario if D-4 numbers show the proxy-200 + handshake bracket is loose. Phase D-3 is the last harness-shape change before D-4 collection: scenario keys are now stable, the comparator's per-scenario diff stays correctly keyed across the flip from scaffold to active. Unblocks: standards#99 D-4 (real baseline collection + gate arming) and, downstream, the remaining items in section 1.1 of boj-server's docs/integration/hcg-tier2-rollout-runbook.md. Refs hyperpolymath/standards#91 Refs hyperpolymath/standards#99 Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
1 parent c227629 commit 301e0d3

4 files changed

Lines changed: 276 additions & 55 deletions

File tree

bench/baseline.json

Lines changed: 17 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,6 @@
11
{
2-
"_comment": "Phase D scaffold-placeholder baseline (standards#99). Numbers are TODO. Phase D-1 (PR #12) wired the Benchee harness + this file + the CI gate; Phase D-2 (this PR) added a real in-process loopback backend so the 'exact route allow (proxy 200)' scenario measures real dial-and-read cost instead of port-1 connection-refused. Phase D-4 collects real numbers via the `perf: rebaseline` ritual in docs/perf-contract.md and flips `_status` to 'active' to arm the CI gate. DO NOT trust these numbers; the perf-regression CI gate is intentionally non-blocking until that flip.",
3-
"_schema_version": "0.1.0-scaffold",
2+
"_comment": "Phase D scaffold-placeholder baseline (standards#99). Numbers are TODO. Phase D-1 (PR #12) wired the Benchee harness + this file + the CI gate; Phase D-2 (PR #14) added a real in-process loopback backend so the 'exact route allow (proxy 200)' scenario measures real dial-and-read cost instead of port-1 connection-refused; Phase D-3 (this update) added two scenarios -- 'trust-header rewrite (Proxy.build_backend_headers)' isolating the Phase A contract-header construction cost, and 'mTLS handshake (test CA)' isolating per-handshake cost using the Phase B real-CA fixture. Phase D-4 collects real numbers via the `perf: rebaseline` ritual in docs/perf-contract.md and flips `_status` to 'active' to arm the CI gate. DO NOT trust these numbers; the perf-regression CI gate is intentionally non-blocking until that flip.",
3+
"_schema_version": "0.2.0-scaffold",
44
"_generated_at": "TODO(Phase D-4): replace with ISO8601 timestamp from collection run",
55
"_generated_by": "TODO(Phase D-4): replace with `bench/gateway_latency.exs` git SHA + machine fingerprint",
66
"_status": "scaffold-placeholder",
@@ -22,10 +22,24 @@
2222
"p95_us": "TODO",
2323
"p99_us": "TODO",
2424
"ips": "TODO"
25+
},
26+
"trust-header rewrite (Proxy.build_backend_headers)": {
27+
"_comment_d3": "Phase D-3 scenario. Isolates the Phase A contract-header construction cost (X-Trust-Level / X-Request-ID / X-Forwarded-*). Direct call to Proxy.__benchmark_build_backend_headers__/1 on a pre-built conn with assigns[:trust_level] = :authenticated and assigns[:request_id] set. No policy lookup, no network. Expected to be the fastest of the five scenarios.",
28+
"p50_us": "TODO",
29+
"p95_us": "TODO",
30+
"p99_us": "TODO",
31+
"ips": "TODO"
32+
},
33+
"mTLS handshake (test CA)": {
34+
"_comment_d3": "Phase D-3 scenario. Per-handshake cost on a raw :ssl acceptor; cert/key chain generated in-memory via :public_key.pkix_test_data/1 (the committed Phase B fixture in test/fixtures/mtls ships only *.crt -- *.key is gitignored, so the chain is rebuilt at bench startup with the same verify_peer + test-CA shape). Each iteration :ssl.connect/4 with the client-side config + immediate :ssl.close/1. Bounds the connection-spike SLO for Phase E rollout; amortised per-request cost is bracketed by this + the 'exact route allow (proxy 200)' scenario.",
35+
"p50_us": "TODO",
36+
"p95_us": "TODO",
37+
"p99_us": "TODO",
38+
"ips": "TODO"
2539
}
2640
},
2741
"tolerance": {
28-
"_comment": "Multiplicative regression tolerance applied to each percentile. 1.20 = a PR is allowed to be up to 20% slower than baseline before the gate fails. Phase D-4 will revisit these once we have signal on intra-run variance.",
42+
"_comment": "Multiplicative regression tolerance applied to each percentile. 1.20 = a PR is allowed to be up to 20% slower than baseline before the gate fails. Phase D-4 will revisit these once we have signal on intra-run variance. D-3 scenarios use the same tolerances pending D-4 calibration; the mTLS handshake scenario is likely the noisiest (TLS RNG draws + cert chain validation timing) and may need a looser p99 cap once we see the spread.",
2943
"p50_max_ratio": 1.20,
3044
"p95_max_ratio": 1.30,
3145
"p99_max_ratio": 1.50

bench/gateway_latency.exs

Lines changed: 188 additions & 21 deletions
Original file line numberDiff line numberDiff line change
@@ -10,13 +10,35 @@
1010
# immediately) so the timing only captured the in-gateway cost up to the
1111
# proxy dial, not the dial-and-read cost.
1212
#
13-
# Phase D-2 (this file): adds an in-process Plug.Cowboy loopback backend
14-
# that responds 200 cheaply, so the allow scenario measures the real
15-
# end-to-end cost the gateway pays in production — header rewrite +
16-
# Req-based proxy + response forwarding. This is the cost surface the
17-
# Phase E rollout SLA (p99 latency at production endpoints) is measured
18-
# against; without it the baseline numbers a Phase D-4 collection would
19-
# produce are not comparable to production reality.
13+
# Phase D-2 (http-capability-gateway#14): adds an in-process Plug.Cowboy
14+
# loopback backend that responds 200 cheaply, so the allow scenario
15+
# measures the real end-to-end cost the gateway pays in production —
16+
# header rewrite + Req-based proxy + response forwarding. This is the
17+
# cost surface the Phase E rollout SLA (p99 latency at production
18+
# endpoints) is measured against; without it the baseline numbers a
19+
# Phase D-4 collection would produce are not comparable to production
20+
# reality.
21+
#
22+
# Phase D-3 (this file): pulls two cost surfaces out of the proxy-200
23+
# scenario into dedicated scenarios so D-4 baseline collection can
24+
# attribute them independently:
25+
#
26+
# • trust-header rewrite (Proxy.build_backend_headers) — the gateway's
27+
# contract-critical header surface (X-Trust-Level / X-Request-ID /
28+
# X-Forwarded-*) measured in isolation, with no policy lookup and no
29+
# network I/O. Phase A's "the trust class the backend sees is the
30+
# value the gateway resolved" invariant lives here, so if a change
31+
# ever inflates this cost, D-4 will catch it.
32+
#
33+
# • mTLS handshake (test CA) — per-handshake cost on a raw :ssl
34+
# listener using the Phase B real-CA fixtures in test/fixtures/mtls.
35+
# Each iteration opens a fresh TLS connection with client-internal.crt
36+
# and immediately closes it; no policy lookup, no proxy. This is the
37+
# connection-spike cost the Phase E rollout has to budget for —
38+
# amortised cost across many requests is bounded by this number plus
39+
# the proxy-200 scenario, and isolating it lets us track CA chain /
40+
# curve / Cowboy-TLS-opt changes that would shift it independent of
41+
# the rest of the pipeline.
2042
#
2143
# Run locally:
2244
#
@@ -28,27 +50,38 @@
2850
#
2951
# CI invocation lives in .github/workflows/perf-regression.yml.
3052
#
31-
# ── Scenarios (Phase D-2 scope) ──────────────────────────────────────────────
53+
# ── Scenarios (Phase D-3 scope) ──────────────────────────────────────────────
3254
#
33-
# 1. health endpoint — fastest path, no policy lookup, no proxy
34-
# 2. policy deny (405 fast-path) — unknown verb, policy table hit, no proxy
35-
# 3. exact route allow (proxy 200) — O(1) policy hit; proxies to the
36-
# in-process loopback backend which
37-
# responds 200 with a tiny JSON body
55+
# 1. health endpoint — fastest path, no policy, no proxy
56+
# 2. policy deny (405 fast-path) — unknown verb, policy table hit, no proxy
57+
# 3. exact route allow (proxy 200) — O(1) policy hit; proxies to the
58+
# in-process loopback backend which
59+
# responds 200 with a tiny JSON body
60+
# 4. trust-header rewrite (Proxy.build_backend_headers)
61+
# — D-3 — direct call to the proxy
62+
# header-rewrite seam; isolates the
63+
# Phase A contract-header construction
64+
# from policy lookup and network I/O
65+
# 5. mTLS handshake (test CA) — D-3 — :ssl.connect/4 with the Phase B
66+
# client-internal cert against a raw
67+
# :ssl acceptor; isolates per-handshake
68+
# cost from the proxy hot-path
3869
#
3970
# Each scenario is reported under p50 / p95 / p99 latency, matching the SLO
4071
# names in docs/perf-contract.md. Throughput (req/s) is reported as a
4172
# secondary signal; the SLO is latency, not throughput.
4273
#
4374
# ── What this harness still deliberately does NOT do ─────────────────────────
4475
#
45-
# • Exercise the mTLS handshake path (Phase D-3; reuses Phase B fixture)
46-
# • Capture the X-Trust-Level rewrite cost as a dedicated scenario
47-
# (Phase D-3 — the cost is currently folded into the proxy-200 scenario)
48-
# • Compare against a populated baseline.json (Phase D-4 collects baseline)
76+
# • Compare against a populated baseline.json (Phase D-4 collects baseline
77+
# numbers on a CI-equivalent target and flips `_status` to `active`)
4978
# • Run distributed (single-node only; estate concurrency-pool guard in #122)
79+
# • Capture amortised cost of N requests over one kept-alive TLS connection
80+
# as a dedicated scenario (the per-handshake cost in scenario 5 plus the
81+
# proxy-200 cost in scenario 3 already bracket it; if D-4 numbers show
82+
# the bracket is loose, a follow-up adds an explicit amortised scenario)
5083
#
51-
# Those are sequenced post-D-2 tasks under the single-lane HCG channel
84+
# Those are sequenced post-D-3 tasks under the single-lane HCG channel
5285
# (standards#91).
5386

5487
# `mix run` loads all deps onto the code path, including Benchee and
@@ -58,6 +91,11 @@
5891
# initialised — Gateway.call/2 depends on those processes existing.
5992
{:ok, _} = Application.ensure_all_started(:http_capability_gateway)
6093

94+
# Phase D-3 scenario 5 uses :ssl directly (raw acceptor + client connect)
95+
# without going through Plug.Cowboy's TLS startup, so the application
96+
# needs to be started explicitly. It's part of OTP, no extra dep needed.
97+
{:ok, _} = Application.ensure_all_started(:ssl)
98+
6199
alias HttpCapabilityGateway.{Gateway, PolicyCompiler}
62100

63101
# ── Loopback backend fixture (Phase D-2) ────────────────────────────────────
@@ -131,6 +169,121 @@ Application.put_env(:http_capability_gateway, :rate_limits, %{
131169
require Plug.Test
132170
import Plug.Test, only: [conn: 2]
133171

172+
alias HttpCapabilityGateway.Proxy
173+
174+
# ── Trust-header rewrite seam fixture (Phase D-3, scenario 4) ───────────────
175+
#
176+
# Pre-build a conn that already has the assigns the Phase C trust-header
177+
# rewrite reads (trust_level + request_id). The bench function then calls
178+
# the Proxy benchmark hook directly so the measurement is the rewrite
179+
# cost in isolation -- no policy lookup, no Gateway.call/2 pipeline, no
180+
# network. This is the cost surface the Phase A contract invariant lives
181+
# on: build_backend_headers/1 is what produces X-Trust-Level / X-Request-ID
182+
# as the authoritative gateway-resolved values.
183+
trust_header_conn =
184+
conn(:get, "/api/bench")
185+
|> Plug.Conn.put_req_header("authorization", "Bearer test-token")
186+
|> Plug.Conn.put_req_header("user-agent", "bench/d-3")
187+
|> Plug.Conn.put_req_header("accept", "application/json")
188+
|> Plug.Conn.assign(:trust_level, :authenticated)
189+
|> Plug.Conn.assign(:request_id, "bench-d3-rewrite")
190+
191+
# ── mTLS acceptor fixture (Phase D-3, scenario 5) ────────────────────────────
192+
#
193+
# Stand up a raw :ssl listener using a test CA chain generated in-memory
194+
# via :public_key.pkix_test_data/1. Each Benchee iteration in scenario 5
195+
# dials this listener with the matching client cert+key and immediately
196+
# closes the connection, so what's measured is the per-handshake cost
197+
# (verify_peer + cert chain validation against the test CA + key exchange
198+
# + finished).
199+
#
200+
# Why in-memory and not test/fixtures/mtls/*.{crt,key}: the committed
201+
# Phase B fixture (used by test/mtls_test.exs) only ships the .crt files
202+
# — *.key is gitignored at the repo root. Reusing the committed *.crt
203+
# files would require also committing the matching *.key files (or
204+
# carving a fixture exception in .gitignore), which the estate security
205+
# posture prefers we don't do for a bench fixture. The chain shape and
206+
# verify_peer behaviour are identical; this scenario tests the same TLS
207+
# transport guarantee, just with key material that never touches disk.
208+
# If D-4 baseline collection shows the handshake cost is sensitive to
209+
# RSA key size or cert chain length, a follow-up moves to a fixture
210+
# shape closer to the production CA's parameters.
211+
#
212+
# Deliberately raw :ssl instead of Plug.Cowboy.https: this scenario must
213+
# isolate the handshake from any HTTP-level pipeline, so we accept and
214+
# discard at the TLS layer. The HTTP-over-TLS cost is bracketed by
215+
# scenario 3 (proxy-200) + scenario 5 (handshake) for D-4 baseline.
216+
#
217+
# Port choice: 19_878 — one above the D-2 loopback (19_877) so the two
218+
# fixtures can coexist when CI runs them back-to-back without socket
219+
# reuse jitter.
220+
# :public_key.pkix_test_data/1 returns a map with :server_config and
221+
# :client_config keys, each already shaped as a property list ready to
222+
# concat into :ssl.listen / :ssl.connect options. Empty cert-opts lists
223+
# tell OTP to use the default RSA key parameters; if D-4 reveals the
224+
# default size shifts handshake cost in a misleading direction we'll
225+
# pin {rsa, Size, Exp} explicitly. Map keys here are Erlang atoms, so
226+
# Elixir's %{atom: value} shorthand maps 1:1.
227+
%{server_config: tls_listen_base, client_config: tls_client_base} =
228+
:public_key.pkix_test_data(%{
229+
server_chain: %{root: [], intermediates: [], peer: []},
230+
client_chain: %{root: [], intermediates: [], peer: []}
231+
})
232+
233+
tls_port = 19_878
234+
235+
tls_listen_opts =
236+
tls_listen_base ++
237+
[
238+
verify: :verify_peer,
239+
fail_if_no_peer_cert: true,
240+
reuseaddr: true,
241+
active: false
242+
]
243+
244+
tls_client_opts =
245+
tls_client_base ++
246+
[
247+
verify: :verify_peer,
248+
# The synthesised peer cert has an arbitrary CN -- skip hostname
249+
# verify since we dial 127.0.0.1. Cert-chain trust is still
250+
# enforced via verify_peer against the in-memory CA the client
251+
# was given via :public_key.pkix_test_data.
252+
server_name_indication: :disable,
253+
active: false
254+
]
255+
256+
# Acceptor loop: a tiny process that accepts handshakes and closes them.
257+
# Spawned as part of fixture setup; lives for the duration of the bench
258+
# run and is killed in the after clause.
259+
{:ok, listen_socket} = :ssl.listen(tls_port, tls_listen_opts)
260+
261+
acceptor_pid =
262+
spawn_link(fn ->
263+
accept_loop = fn loop ->
264+
case :ssl.transport_accept(listen_socket, 5_000) do
265+
{:ok, transport_socket} ->
266+
# Drive the handshake to completion on the server side; the
267+
# cost the *client* sees is what scenario 5 measures, but the
268+
# server has to play its half or the client times out.
269+
case :ssl.handshake(transport_socket, 5_000) do
270+
{:ok, tls_socket} -> :ssl.close(tls_socket)
271+
{:error, _reason} -> :ok
272+
end
273+
274+
loop.(loop)
275+
276+
{:error, :timeout} ->
277+
loop.(loop)
278+
279+
{:error, :closed} ->
280+
:ok
281+
end
282+
end
283+
284+
accept_loop.(accept_loop)
285+
end)
286+
134287
# ── Benchee scenarios ────────────────────────────────────────────────────────
135288

136289
try do
@@ -145,11 +298,18 @@ try do
145298
end,
146299
"exact route allow (proxy 200)" => fn ->
147300
conn(:get, "/api/bench") |> Gateway.call([])
301+
end,
302+
"trust-header rewrite (Proxy.build_backend_headers)" => fn ->
303+
Proxy.__benchmark_build_backend_headers__(trust_header_conn)
304+
end,
305+
"mTLS handshake (test CA)" => fn ->
306+
{:ok, sock} = :ssl.connect(~c"127.0.0.1", tls_port, tls_client_opts, 5_000)
307+
:ssl.close(sock)
148308
end
149309
},
150-
# Phase D-1/D-2: keep timings short so the workflow stays under the
151-
# 6-minute estate CI budget. Phase D-4 may widen warmup/time once the
152-
# rebaseline ritual is in place.
310+
# Phase D-1/D-2/D-3: keep timings short so the workflow stays under
311+
# the 6-minute estate CI budget. Phase D-4 may widen warmup/time
312+
# once the rebaseline ritual is in place.
153313
warmup: 1,
154314
time: 2,
155315
memory_time: 0,
@@ -166,4 +326,11 @@ after
166326
# Stop the loopback backend so the script exits cleanly. Plug.Cowboy
167327
# registers the listener under the plug module's name by default.
168328
_ = Plug.Cowboy.shutdown(HttpCapabilityGateway.Bench.LoopbackBackend.HTTP)
329+
330+
# Tear down the mTLS acceptor fixture (Phase D-3): closing the listen
331+
# socket causes the transport_accept loop to exit on :closed; the
332+
# linked acceptor process then terminates normally.
333+
Process.unlink(acceptor_pid)
334+
:ssl.close(listen_socket)
335+
Process.exit(acceptor_pid, :shutdown)
169336
end

0 commit comments

Comments
 (0)