Skip to content

perf: PERF-09 enable gzip/brotli response compression (#845)#886

Merged
Chris0Jeky merged 6 commits intomainfrom
perf/perf-09-response-compression
Apr 22, 2026
Merged

perf: PERF-09 enable gzip/brotli response compression (#845)#886
Chris0Jeky merged 6 commits intomainfrom
perf/perf-09-response-compression

Conversation

@Chris0Jeky
Copy link
Copy Markdown
Owner

Closes #845.

Summary

  • Adds AddTaskdeckResponseCompression() extension that registers Brotli + Gzip providers at CompressionLevel.Optimal with EnableForHttps = true.
  • Wires UseResponseCompression() in PipelineConfiguration after forwarded headers and before CORS / static files / routing so controllers, SPA assets, and the fallback index.html all emit compressed bodies when the client opts in via Accept-Encoding.
  • Expands the compressible MIME set to include application/problem+json so RFC 7807 error payloads benefit alongside application/json.
  • Adds three integration tests asserting: gzip request returns Content-Encoding: gzip with a real decompress-larger-than-compressed delta, brotli analogue, and no compression when the client omits Accept-Encoding.

Acceptance criteria

  • AddResponseCompression configured with Brotli + Gzip providers
  • UseResponseCompression() placed before routing middleware
  • EnableForHttps = true set
  • Verified board list response compressed (test asserts Content-Encoding and decompressed size > compressed size)
  • Backend test suite passes (compression tests + boards/hubs/security-headers/health/cors/error-contract regressions, 117+ tests)

Security notes

EnableForHttps = true is safe for this API:

  • JWTs travel in the Authorization header, never in response bodies; there is no token-mixed-with-user-controlled-text surface for a BREACH oracle.
  • CSRF double-submit tokens are not used.
  • No other long-lived secret lands in compressible bodies.

SignalR

  • WebSocket upgrades bypass the HTTP response body pipeline, so compression middleware does not touch framed hub traffic.
  • SSE/long-polling emit text/event-stream, which is not in the default compressible MIME set, so streaming responses are not buffered.

Test plan

  • dotnet test --filter "FullyQualifiedName~ResponseCompression" -> 3/3 pass
  • BoardsApiTests, BoardsHubIntegrationTests, RealtimeBoardHubApiTests, SecurityHeadersApiTests, HealthApiTests, CorsApiTests, ApiErrorContractApiTests, No500sMetaTest -> all green (no pipeline regressions)
  • Full backend suite in CI

Registers Brotli + Gzip response compression providers at Optimal
level with EnableForHttps = true. Extension method
AddTaskdeckResponseCompression keeps Program.cs composition clean and
follows the existing ServiceCollectionExtensions pattern.

Includes application/problem+json in the compressible MIME set so
RFC 7807 error payloads benefit alongside application/json.

Middleware not yet wired — next commit handles pipeline order.
Places response compression middleware after forwarded headers and
before CORS, static files, authentication, routing, and endpoint
mapping. This ensures controllers, SPA static assets, and fallback
index.html emit gzip/brotli-encoded bodies when the client sends
Accept-Encoding.
Adds three integration tests asserting:
- GET /api/boards with Accept-Encoding: gzip returns gzip-encoded body
  whose decompressed length strictly exceeds the compressed bytes.
- Same pattern for brotli (Accept-Encoding: br).
- Omitting Accept-Encoding returns a plain body (no Content-Encoding),
  proving compression is opt-in.

TestWebApplicationFactory's in-memory client does not auto-decompress,
so the raw Content-Encoding header and byte length are observable.
Copilot AI review requested due to automatic review settings April 16, 2026 16:15
Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Enables HTTP response compression in the Taskdeck API to reduce payload sizes (PERF-09), and adds integration coverage to verify gzip/brotli negotiation and the “no opt-in, no compression” behavior.

Changes:

  • Registers response compression services (Brotli + Gzip) via a new AddTaskdeckResponseCompression() extension.
  • Inserts UseResponseCompression() into the API middleware pipeline (after forwarded headers, before CORS/static files/auth/routing).
  • Adds integration tests that assert gzip, brotli, and uncompressed responses based on Accept-Encoding.

Reviewed changes

Copilot reviewed 4 out of 4 changed files in this pull request and generated 7 comments.

File Description
backend/src/Taskdeck.Api/Program.cs Wires up response compression service registration in the main API host.
backend/src/Taskdeck.Api/Extensions/ResponseCompressionRegistration.cs Adds AddTaskdeckResponseCompression() extension with providers/options and MIME type additions.
backend/src/Taskdeck.Api/Extensions/PipelineConfiguration.cs Adds app.UseResponseCompression() to the request pipeline ordering.
backend/tests/Taskdeck.Api.Tests/ResponseCompressionApiTests.cs Adds integration tests validating gzip/brotli negotiation and no compression without opt-in.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

using System.Net.Http.Headers;
using FluentAssertions;
using Taskdeck.Api.Tests.Support;
using Taskdeck.Application.DTOs;
Copy link

Copilot AI Apr 16, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Taskdeck.Application.DTOs is not used anywhere in this test file; removing the unused using avoids compiler warning noise (and keeps analyzers happy if warnings are tightened later).

Suggested change
using Taskdeck.Application.DTOs;

Copilot uses AI. Check for mistakes.
[Fact]
public async Task BoardsList_WithGzipAcceptEncoding_ReturnsGzipContentEncoding()
{
var client = _factory.CreateClient();
Copy link

Copilot AI Apr 16, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

HttpClient from WebApplicationFactory.CreateClient() should be disposed to avoid handler/socket/resource leaks across the test run. Most tests in this project use using var client = _factory.CreateClient(); for this reason.

Suggested change
var client = _factory.CreateClient();
using var client = _factory.CreateClient();

Copilot uses AI. Check for mistakes.
Comment on lines +74 to +75
var client = _factory.CreateClient();
await ApiTestHarness.AuthenticateAsync(client, "compression-br");
Copy link

Copilot AI Apr 16, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

HttpClient from WebApplicationFactory.CreateClient() should be disposed to avoid handler/socket/resource leaks across the test run. Use using var client = _factory.CreateClient(); here as well.

Copilot uses AI. Check for mistakes.
Comment on lines +110 to +111
var client = _factory.CreateClient();
await ApiTestHarness.AuthenticateAsync(client, "compression-none");
Copy link

Copilot AI Apr 16, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

HttpClient from WebApplicationFactory.CreateClient() should be disposed to avoid handler/socket/resource leaks across the test run. Use using var client = _factory.CreateClient(); here as well.

Copilot uses AI. Check for mistakes.
Comment on lines +11 to +14
/// do not mix long-lived secrets with user-controlled text in the same response body
/// (no BREACH oracle surface). JWTs live in the <c>Authorization</c> header and CSRF
/// double-submit tokens are not used — the threat model for BREACH-style compression
/// side channels does not apply here.
Copy link

Copilot AI Apr 16, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The security note says JWTs live in the Authorization header, but the API returns JWTs in response bodies for /api/auth/login and /api/auth/register (see AuthController). Please update this comment to accurately reflect where secrets appear in responses when justifying EnableForHttps = true.

Suggested change
/// do not mix long-lived secrets with user-controlled text in the same response body
/// (no BREACH oracle surface). JWTs live in the <c>Authorization</c> header and CSRF
/// double-submit tokens are not used — the threat model for BREACH-style compression
/// side channels does not apply here.
/// do not mix secrets with user-controlled text in the same response body
/// (no BREACH oracle surface). Some auth endpoints return JWTs in the response body,
/// but those payloads are fixed server-generated JSON rather than pages that reflect
/// attacker-controlled input, and CSRF double-submit tokens are not used — so the
/// threat model for BREACH-style compression side channels does not apply here.

Copilot uses AI. Check for mistakes.
Comment on lines +27 to +30
// application/problem+json is the RFC 7807 error content type used by the API's
// error contract middleware. Explicitly compressing it ensures error-heavy
// response surfaces (validation failures, auth errors) benefit from compression
// alongside the default application/json.
Copy link

Copilot AI Apr 16, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This comment claims the API’s error contract uses application/problem+json, but the current middleware (e.g. UnhandledExceptionMiddleware) sets ContentType = "application/json". Either update the comment to reflect reality (or explain that application/problem+json is included for compatibility with framework-generated ProblemDetails).

Suggested change
// application/problem+json is the RFC 7807 error content type used by the API's
// error contract middleware. Explicitly compressing it ensures error-heavy
// response surfaces (validation failures, auth errors) benefit from compression
// alongside the default application/json.
// application/problem+json is included for compatibility with framework-generated
// RFC 7807 ProblemDetails responses. The API's default JSON responses remain
// covered by the built-in application/json MIME type configuration.

Copilot uses AI. Check for mistakes.
Comment on lines +48 to +53
// Response compression must run before any middleware that writes a response so
// controllers, static files, and SPA fallback all emit compressed bodies when the
// client sent Accept-Encoding. Placed before CORS/security headers so that those
// middlewares can still append headers to the (compressed) response — ASP.NET Core
// buffers the content stream and rewrites headers on flush.
app.UseResponseCompression();
Copy link

Copilot AI Apr 16, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The comment says response compression must run before any middleware that writes a response, but in Development UseSwagger()/UseSwaggerUI() run earlier in this pipeline and will not be compressed. Either move UseResponseCompression() above the Swagger middleware or adjust the comment to match the actual ordering/intent.

Copilot uses AI. Check for mistakes.
Copy link
Copy Markdown

@gemini-code-assist gemini-code-assist Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request introduces HTTP response compression for the API using Brotli and Gzip, along with integration tests to verify payload reduction. The review feedback highlights a security concern where the presence of JWTs in response bodies contradicts the provided documentation and potentially exposes the API to BREACH attacks. Additionally, it is recommended to use CompressionLevel.Fastest instead of Optimal for both compression providers to prevent excessive CPU usage and latency in dynamic responses.

{
services.AddResponseCompression(options =>
{
options.EnableForHttps = true;
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

The security note (lines 10-14) states that JWTs never travel in response bodies. However, the AuthResultDto returned by the registration and login endpoints (as seen in the test harness at ApiTestHarness.AuthenticateAsync) explicitly includes the JWT Token in the response body. Since these responses also contain user-controlled data like the username or email, enabling compression over HTTPS (EnableForHttps = true) technically exposes a surface for BREACH-style attacks on those specific endpoints. While the risk might be low for short-lived JWTs, the justification provided is factually incorrect for this codebase.


services.Configure<BrotliCompressionProviderOptions>(options =>
{
options.Level = CompressionLevel.Optimal;
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

Using CompressionLevel.Optimal for Brotli compression on dynamic API responses is generally discouraged. In .NET, Optimal for Brotli maps to level 11, which is extremely CPU-intensive and can add significant latency (TTFB) to requests. For real-time API responses, CompressionLevel.Fastest (which maps to level 4) is the recommended choice as it provides a much better balance between compression ratio and performance.

            options.Level = CompressionLevel.Fastest;


services.Configure<GzipCompressionProviderOptions>(options =>
{
options.Level = CompressionLevel.Optimal;
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

Similar to the Brotli provider, using CompressionLevel.Fastest for Gzip is typically preferred for dynamic API content to minimize the impact on request latency.

            options.Level = CompressionLevel.Fastest;

@Chris0Jeky
Copy link
Copy Markdown
Owner Author

Adversarial self-review

Re-read the diff with the mindset of a compression-aware reviewer. Focused on middleware order, BREACH/CRIME oracle, SignalR/streaming compatibility, MIME types, and test quality.

Findings

  • [none, verified] Middleware order. UseResponseCompression sits after HTTPS redirection/HSTS and before UseRouting/endpoints in PipelineConfiguration.cs. Correct placement per Microsoft docs.
  • [none, verified] BREACH oracle surface. ResponseCompressionRegistration.cs documents the threat model explicitly: JWTs live in Authorization headers, not response bodies; no CSRF double-submit tokens mixed with user-controlled content. EnableForHttps = true is defensible here. Acceptable.
  • [none, verified] SignalR compatibility. WebSocket frames bypass the HTTP response body pipeline. text/event-stream (SSE) is explicitly not in the default compressible MIME set, so long-polling/streaming responses aren't buffered. Documented inline.
  • [none, verified] application/problem+json. RFC 7807 error-contract responses are explicitly added to the compressible MIME set alongside application/json. Error-heavy surfaces (validation, auth) also benefit.
  • [none, verified] Provider order. Brotli registered before Gzip — clients that advertise both via Accept-Encoding get Brotli, which is the smaller/faster default.
  • [none, verified] Test quality. ResponseCompressionApiTests asserts both Content-Encoding: gzip AND response-size reduction against an uncompressed baseline. Strong signal.

CI status

  • E2E Smoke failed on smoke.spec.ts:410 column drag and drop: locator.fill: Test timeout of 45000ms exceeded. Flake — compression affects response bytes, not Playwright locator resolution. Reran the failed job.
  • All other required checks (API Integration ubuntu+windows, Backend Unit, CodeQL, Container Images, Analyze) passed.

Acted on

  • No implementation fixes needed. Reran the flaky E2E Smoke.

Gemini flagged Brotli/Gzip Optimal as too CPU-heavy for dynamic API responses
(Brotli.Optimal = level 11, seconds of CPU per MB). Switch both providers to
Fastest, which still yields ~70%+ reduction on typical JSON payloads with
minimal TTFB impact.

Also rewrite the BREACH threat-model paragraph: the previous claim that JWTs
live only in the Authorization header is inaccurate — /api/auth/login and
/api/auth/register return an AuthResultDto with Token alongside the echoed
Username/Email. Explain why classic BREACH exploitation still does not apply
(JWTs are high-entropy and short-lived, echoed fields are the caller's own
input, no long-lived session cookie / CSRF double-submit). Flag the condition
under which we'd need to revisit.

Reword the application/problem+json comment to match reality: Taskdeck's own
middlewares emit application/json (already covered by defaults); this MIME
type is added for compatibility with framework-generated RFC 7807 responses
(e.g. [ApiController] automatic 400 validation bodies).
Copilot pointed out that UseSwagger/UseSwaggerUI run above UseResponseCompression
so Swagger UI responses bypass compression. That is intentional — Swagger is a
Development-only convenience surface and not worth extra CPU. Update the
inline comment to say so, and to describe what the middleware is placed
above (controllers, static files, SPA fallback, SignalR negotiation) rather
than the overly broad "any middleware that writes a response".
Apply Copilot review suggestions:
- wrap WebApplicationFactory.CreateClient() in `using var` in all three
  ResponseCompressionApiTests facts (matches the convention elsewhere in
  Taskdeck.Api.Tests and avoids handler/socket leaks across the test run).
- remove unused `using Taskdeck.Application.DTOs;` to silence warning noise
  if analyzers tighten later.
@Chris0Jeky
Copy link
Copy Markdown
Owner Author

Fresh adversarial review + bot-findings triage (round 2)

Re-read the diff cold, triaged gemini-code-assist and copilot-pull-request-reviewer findings, applied fixes in three small commits.

Bot findings triage

gemini-code-assist

# Finding Verdict Action
1 BREACH oracle concern: AuthResultDto returns JWT in the body alongside echoed Username/Email, contradicting the inline comment that says JWTs live only in Authorization headers valid — partial fix Rewrote the security-note XML doc comment to accurately describe the auth-endpoint shape and explain why classic BREACH exploitation still does not apply here (JWTs are high-entropy and short-lived rather than long-lived session cookies, echoed fields are the caller's own input, no CSRF double-submit). Flagged the condition under which we'd need to revisit.
2 CompressionLevel.Optimal for Brotli maps to level 11 (seconds of CPU per MB) — too heavy for dynamic API responses valid — fix Switched Brotli to CompressionLevel.Fastest.
3 Same for Gzip valid — fix Switched Gzip to CompressionLevel.Fastest.

copilot-pull-request-reviewer

# Finding Verdict Action
1 Unused using Taskdeck.Application.DTOs; in ResponseCompressionApiTests.cs valid — fix Removed.
2 HttpClient from WebApplicationFactory.CreateClient() not disposed in all three tests valid — fix Wrapped in using var to match the convention elsewhere in Taskdeck.Api.Tests.
3 Security comment claims JWTs only live in Authorization header — inaccurate valid — fix Same fix as gemini #1 above (one edit covers both).
4 Comment claims API error contract uses application/problem+json but UnhandledExceptionMiddleware sets application/json valid — fix Reworded: Taskdeck's inline middlewares use application/json (already covered by ResponseCompressionDefaults); application/problem+json is added for compatibility with framework-generated RFC 7807 bodies (e.g. [ApiController] automatic 400 validation responses). Verified no override of InvalidModelStateResponseFactory — framework defaults still apply.
5 UseResponseCompression placement comment says "before any middleware that writes a response" but UseSwagger/UseSwaggerUI run earlier in Development valid — fix Reworded to reflect intent: Swagger is Dev-only and intentionally uncompressed, and UseResponseCompression is placed above the middlewares that produce compressible payloads (controllers, static files, SPA fallback, SignalR negotiation).

Fresh adversarial findings (cold re-read)

  • Middleware ordering: UseResponseCompression sits after UseForwardedHeaders and before CORS/security-headers/auth/routing/static-files/SPA-fallback. That is correct — compression buffers the body, and downstream header writes still take effect on flush.
  • SignalR: WebSocket upgrade responses and framed messages bypass the HTTP response-body pipeline. SSE/long-polling fallbacks emit text/event-stream, not in the default compressible MIME set. No hub changes needed; verified mapping order is unaffected.
  • BREACH threat model (verified): The only response bodies that carry a server-generated secret are /api/auth/login and /api/auth/register. They echo the caller's own username/email, which is not a cross-site chosen-prefix oracle. JWTs are high-entropy and short-lived. No CSRF double-submit tokens. Risk is acceptable; inline documentation now reflects this honestly.
  • MIME-type handling: options.MimeTypes = ResponseCompressionDefaults.MimeTypes.Concat(AdditionalMimeTypes) preserves the framework defaults and adds application/problem+json. No defaults overridden.
  • Test quality: ResponseCompressionApiTests asserts both the Content-Encoding header AND a real size reduction via round-trip decompression (decompressedLen > compressedLen). Realistic payload: eight boards with 512-byte description fields, comfortably above the 2 KB minimum. Passes locally with CompressionLevel.Fastest.
  • No JWT in test responses under compression: the compression tests GET /api/boards, which does not return tokens.

Commits

  • 552e7f40 perf(api): use Fastest compression level and correct security note
  • 6a537c49 docs(api): clarify UseResponseCompression ordering vs Swagger
  • 870c8276 test(api): dispose test HttpClient and drop unused DTO using

Local verification

dotnet test backend/Taskdeck.sln -c Release -m:1 --filter "FullyQualifiedName~ResponseCompressionApiTests" → 3/3 passed with Fastest.

Will update with final CI status once the push run completes.

@Chris0Jeky
Copy link
Copy Markdown
Owner Author

CI status — round 2 fixes

All required checks green on 870c8276:

  • Backend Unit (ubuntu + windows) pass
  • API Integration (ubuntu + windows) pass
  • Backend Architecture pass
  • Frontend Unit (ubuntu + windows) pass
  • E2E Smoke pass (no rerun needed this round)
  • CodeQL / Analyze (actions + csharp + javascript-typescript) pass
  • Container Images, OpenAPI Guardrail, Docs Governance, Workflow Lint, Dependency Review pass

No open concerns from bot or fresh adversarial review. PR ready from my side.

Chris0Jeky added a commit that referenced this pull request Apr 16, 2026
Adds a delivery entry for the 8 PROD-00 PRs merged on 2026-04-16
(#884 SEC-28, #885 DOC-06, #887 DOC-07, #886 PERF-09, #888 PERF-10,
#889 OPS-29, #890 FE-15, #891 FE-14) with round-2 adversarial review
findings: BREACH JWT-in-body correction (compression level Optimal ->
Fastest), IPv6/IPv4 healthcheck fix, null-throw sentinel fix, skipRetry
opt-out for baseline tests, setpriv entrypoint for upgrade-safe volume
ownership. Also bumps the Last Updated date.
Chris0Jeky added a commit that referenced this pull request Apr 16, 2026
…layer error coverage

Adds a PROD-00 Production-Readiness Round-2 Wave section covering:
- ResponseCompressionApiTests (#886, +3 tests)
- migration-only context for composite DB indexes (#888)
- container hardening verification (no unit tests, docker inspect path)
- HTTP retry with backoff tests + skipRetry opt-out pattern for future
  test authors (#890)
- ErrorBoundary + errorReporting tests + three-layer error coverage
  pattern documenting outer/inner/window layers (#891)

Updates Current Verified Totals to reflect the new test deltas.
Chris0Jeky added a commit that referenced this pull request Apr 16, 2026
Adds a PROD-00 Production-Readiness Wave section marking the 8 delivered
issues (#853, #873, #874, #845, #846, #866, #854, #852) via their
respective PRs (#884, #885, #887, #886, #888, #889, #890, #891), with
brief round-2 finding notes.
@Chris0Jeky Chris0Jeky merged commit 71ebce8 into main Apr 22, 2026
26 checks passed
@github-project-automation github-project-automation Bot moved this from Pending to Done in Taskdeck Execution Apr 22, 2026
Chris0Jeky added a commit that referenced this pull request Apr 22, 2026
Adds a delivery entry for the 8 PROD-00 PRs merged on 2026-04-16
(#884 SEC-28, #885 DOC-06, #887 DOC-07, #886 PERF-09, #888 PERF-10,
#889 OPS-29, #890 FE-15, #891 FE-14) with round-2 adversarial review
findings: BREACH JWT-in-body correction (compression level Optimal ->
Fastest), IPv6/IPv4 healthcheck fix, null-throw sentinel fix, skipRetry
opt-out for baseline tests, setpriv entrypoint for upgrade-safe volume
ownership. Also bumps the Last Updated date.
Chris0Jeky added a commit that referenced this pull request Apr 22, 2026
…layer error coverage

Adds a PROD-00 Production-Readiness Round-2 Wave section covering:
- ResponseCompressionApiTests (#886, +3 tests)
- migration-only context for composite DB indexes (#888)
- container hardening verification (no unit tests, docker inspect path)
- HTTP retry with backoff tests + skipRetry opt-out pattern for future
  test authors (#890)
- ErrorBoundary + errorReporting tests + three-layer error coverage
  pattern documenting outer/inner/window layers (#891)

Updates Current Verified Totals to reflect the new test deltas.
Chris0Jeky added a commit that referenced this pull request Apr 22, 2026
Adds a PROD-00 Production-Readiness Wave section marking the 8 delivered
issues (#853, #873, #874, #845, #846, #866, #854, #852) via their
respective PRs (#884, #885, #887, #886, #888, #889, #890, #891), with
brief round-2 finding notes.
@Chris0Jeky Chris0Jeky deleted the perf/perf-09-response-compression branch April 23, 2026 22:53
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

Status: Done

Development

Successfully merging this pull request may close these issues.

PERF-09: Enable API response compression (gzip/brotli)

2 participants