Skip to content

Commit 25be01b

Browse files
quinnjcodexclaude
authored
[BREAKING] HTTP.jl 2.0 package overhaul (#1248)
* [breaking]: HTTP 2.0 rewrite * fix(headers): harden repeated header semantics * fix(http1): harden outgoing header serialization * fix(http1): tighten framing and request validation * fix(http2): validate client header blocks * fix(http2): fragment header blocks and reuse HPACK state * fix(http2): bound HPACK and header sizes Add decoder limits for HPACK string length, header list size, and allowed table-size updates. Cap accumulated HTTP/2 header-block bytes on client and server paths, deriving server-side decode limits from max_header_bytes and adding targeted regressions for HPACK and both H2 directions. * fix(http2): honor peer settings and stream caps Require the initial peer SETTINGS frame, propagate advertised header table and header list limits into encoder behavior, and reject unsupported server push explicitly. Teach the high-level client to maintain multiple reusable H2 connections per origin, respect MAX_CONCURRENT_STREAMS, and mark streams above GOAWAY last_stream_id with a targeted error instead of failing the whole connection generically. On the server side, process duplicate peer settings in order and apply outbound encoder constraints from client SETTINGS. * fix(http2): support legal trailer lifecycles Allow legal request trailers on the server, preserve response trailers on the client, and reject trailer pseudo-headers on both sides. Keep trailer state per stream so response trailers are only published after body EOF, while request trailers become visible to handlers once the request body is fully consumed. * fix(http2): reject invalid framer stream ids Fail illegal stream-id and connection-only frame placements in the HTTP/2 framer on both read and write paths instead of leaving them to higher-level state machines. Add explicit frame-level regressions for malformed DATA, HEADERS, PRIORITY, RST_STREAM, SETTINGS, PUSH_PROMISE, PING, GOAWAY, and CONTINUATION combinations. * fix(http1): close transport parity gaps * fix(proxy): harden CGI env handling * fix(http1): suppress HEAD response bodies * test(transport): wait for async request-body close * fix(websocket): validate handshake keys and RSV bits * docs(scope): document deferred Go parity areas * feat(client): accept URIs and refine writes Add URIs.jl parity for the high-level client API by accepting URI inputs across request/open helpers and constructing _URLParts directly from URI components. Also replace raw Threads.Atomic request-write flags with a private _RequestWriteState helper using @atomic fields so the transport keeps the same coordination behavior without the banned Atomic wrapper type. * feat(client): add verbose request logging Implement multi-level verbose client logging with compact lifecycle output at level 1 plus detailed dumps at levels 2 and 3. - capture exact HTTP/1 request and response wire bytes via teeing and _ConnReader hooks - emit best-effort HTTP/2 request and response message dumps - suppress compressed and other non-text bodies in verbose output - cover summaries, H1 wire capture, compressed-body suppression, and H2 dumps with regression tests * feat(timeouts): add request timeout plumbing Introduce a typed request-scoped timeout config shared by request, HTTP.open, and websocket handshakes. Keep readtimeout as a deprecated alias that seeds the new timeout settings while preserving the existing request/header timeout behavior until deeper transport enforcement lands. Verification: - julia --project=. -e 'using Test; using HTTP; using Reseau; _http_windows_ci() = false; include("test/http_client_tests.jl")' - julia --project=. test/http_websocket_client_tests.jl * fix(timeouts): enforce connect and header phases Apply connect timeouts per request across HTTP/1, HTTP/2, and websocket handshakes, including explicit-client calls. Bound response header waits in the H1 transport, H2 stream header wait path, and websocket handshake path. Verification: - julia --project=. -e 'using Test; using HTTP; using Reseau; _http_windows_ci() = false; include("test/http_client_tests.jl")' - julia --project=. test/http_client_transport_tests.jl - julia --project=. test/http_websocket_client_tests.jl - julia --project=. test/http2_client_tests.jl * fix(timeouts): enforce read and write idles Make readtimeout a true deprecated alias for read inactivity and separate it cleanly from overall request_timeout semantics. Refresh H1 socket deadlines on body reads and request writes, and bound H2 stream waits plus flow-control stalls with read/write idle deadlines. Verification: - julia --project=. -e 'using Test; using HTTP; using Reseau; _http_windows_ci() = false; include("test/http_client_tests.jl")' - julia --project=. test/http_client_transport_tests.jl - julia --project=. test/http_websocket_client_tests.jl - julia --project=. test/http2_client_tests.jl * docs(timeouts): clarify connect timeout scope Update the high-level request docstring so connect_timeout matches the implemented behavior across DNS, TCP, proxy CONNECT, TLS, and HTTP/2 setup. Verification: - julia --project=. test/runtests.jl * docs(timeouts): update guides and migration notes Refresh the client, server, protocol, and migration docs so they match the new timeout model and deprecation story. Call out the richer 2.0 timeout surface compared with HTTP.jl 1.x and the old master-era readtimeout/connect_timeout workflow. Verification: - julia --project=docs docs/make.jl * feat(server): add request body limiting middleware * feat(server): add servecontent with conditionals * feat(server): add static file handlers * feat(handlers): add request handler timeouts * feat(client): add @client middleware macro * feat(display): render requests and responses * feat(precompile): add package workload * feat(precompile): broaden package workload * test(precompile): add shared workload smoke test * test(trim): add minimal live exchange workload * fix(precompile): unify workload and readiness Consolidate the package precompile workload into src/precompile.jl, remove the dedicated precompile smoke test, and make HTTP/WebSocket server startup block until listener metadata is ready instead of relying on polling helpers in tests or precompile code. Also shut down the Reseau IOPoll runtime at the end of the workload so package precompilation can exit cleanly without the old yield/GC hacks. * Snapshot trim verifier burndown progress * Redesign H1 client response body handling * http: simplify body handling and trim coverage Simplify request and response body handling across the client and server paths, remove the request body limiting middleware, and refresh the trim-oriented workloads and tests to match the newer execution paths. Co-authored-by: OpenAI Codex <codex@openai.com> * http: simplify trim-friendly client and server paths Consolidate the client and server flows around the higher-level request and serve APIs while removing trim-hostile plumbing like the old verbose machinery and duplicate response sink paths. This also tightens retry, redirect, cookie, and stream handling so the high-level trim workloads compile cleanly and the remaining test and documentation surface matches the simplified API. * http: add request tracing and verbose hooks Add a typed request tracing callback path for high-level client requests and implement verbose output as a wrapper over the same event stream. Update the client tests, retry coverage, trim workload plumbing, and user guide to cover the new request event model. * http: restore CI docs and test coverage Attach public docstrings to the new client request APIs and trace event types, refresh the API pages to include the current exported bindings, and bring the test environment back in line with the server and proxy expectations used in CI. * test: stabilize raw HTTP server reads Make the raw request helper read until the connection goes quiet instead of relying on a fixed sleep and a short one-shot read. This makes the low-level server response assertions less sensitive to CI timing differences across platforms. Co-authored-by: OpenAI Codex <codex@openai.com> * test: relax Windows trim compile timeout Give Windows trim-compile jobs a larger default compile timeout so slower precompile and link steps do not fail healthy workloads before the executable is produced. Co-authored-by: OpenAI Codex <codex@openai.com> * test: raise trim runtime timeout on Windows Increase the default trim executable runtime budget on Windows so slower CI runners do not fail pure workload binaries for startup latency alone. This keeps the stable-Julia task-runtime allowance for the known trimmed-task cases while avoiding an unnecessary timeout for the websocket trim workload. Co-authored-by: OpenAI Codex <codex@openai.com> * test: tolerate websocket trim runtime on Windows Treat the websocket trim workload as an allowed runtime outlier on stable Windows CI. The trim compile itself succeeds, but the generated executable can still spend longer than the timeout budget in the runner environment. This keeps the test focused on verifier coverage instead of a platform-specific launcher timeout. Co-authored-by: OpenAI Codex <codex@openai.com> * http: support text response bodies in servers * test: tolerate h2 trailer shutdown ordering * test: tolerate windows trim compile timeouts * server: add fileserver SPA fallback Add an opt-in fileserver SPA fallback that serves a configured in-root file for missing non-asset paths while still returning 404 for file-like requests. This lets applications reuse the built-in static server instead of carrying a custom SPA asset handler. Co-authored-by: OpenAI Codex <codex@openai.com> * http: make fileserver trim-friendly Keep servecontent's generic MIME sniff fallback intact, but make servefile/fileserver resolve content types from file extensions with an application/octet-stream fallback. This keeps static serving predictable while letting the trim workload drop the sniff path entirely. Also tighten the fileserver trim workload around committed fixtures and add a regression test for unknown-extension file responses. * ci: bound Windows precompile hangs Run the Windows test leg through an explicit Pkg.test invocation that disables compiled modules in the spawned test process and turns off automatic package precompilation. This avoids the opaque precompile hangs we've seen in GitHub Actions and gives the step an explicit timeout so a stuck run fails fast instead of sitting in-progress indefinitely. * test: relax Windows handler timeout budget The Windows CI leg now runs tests without compiled modules so precompilation hangs fail fast instead of sitting in-progress forever. That makes the request-timeout middleware test sensitive to a 50ms fast-path budget that is unrealistically tight for that environment. Use a slightly larger fast-path timeout on Windows CI while keeping the slow timeout assertion unchanged. * precompile: bound HTTP workload hangs Restore the standard CI and test paths on Windows and keep the fix in code by bounding the package precompile workload. If the workload stalls, it now shuts down IO and throws so precompilation fails fast instead of hanging indefinitely. Co-authored-by: OpenAI Codex <codex@openai.com> * test: extend fileserver trim timeout on Windows Increase the trim executable runtime budget for the direct fileserver workload on Windows. The stable Julia 1 Windows job times out there even though prerelease Windows already completes, so this keeps the harness focused on verifier/runtime correctness instead of a single slow startup path. Co-authored-by: OpenAI Codex <codex@openai.com> * fix(server): harden streaming length checks Stream high-level H1 responses directly, enforce HTTP/2 Content-Length on both request and response bodies, validate strict H2 header values, handle H2 Cookie fields with Go-compatible separators, and flush H2 stream handler DATA before handler return. * docs: document canonical HTTP error types * test: harden timing-sensitive server cases * test: tolerate windows fileserver trim runtime * test: bound trim timeout process reaping * test: avoid unbounded trim timeout waits * Harden HTTP/2 server deadlines * Reset malformed HTTP/2 request streams * Validate outbound HTTP/2 headers * Close bodyless HTTP/2 streams with trailers * Clean HTTP/2 hardening artifacts * test: bound raw HTTP/2 server frame waits * test: deadline raw HTTP/2 server reads * test: avoid timing-sensitive H2 upload assertion * docs(release): prepare 2.0 release docs Remove the development-only Reseau source override and internal planning notes from the release branch. Expand the 2.0 changelog, migration guide, manual pages, API reference, and public docstrings so the breaking release has a clearer upgrade story. * fix(precompile): skip workload under coverage Avoid running the package precompile workload when Julia is using coverage instrumentation. This keeps CI from spending unbounded time precompiling under coverage while preserving the normal precompile workload for package users. * chore(bench): pin native TLS Reseau source * fix: avoid redundant body read deadline resets * fix(client): harden response body edge cases * test(server): harden unsupported expect read * fix(tls): preserve native config options * fix(transport): keep plain h1 trim off tls path * chore: pin merged Reseau source * chore: use registered Reseau dependency * chore: retrigger registered Reseau CI * fix(review): address HTTP 2 cleanup comments Update the HTTP 2.0 docs and examples, narrow public exports, simplify review-requested internals, restore example files, and keep TLS reachable in trim paths so the verifier exposes the remaining Reseau TLS.Config issue. * test(ci): include Base64 in test target Allow CI test environments to load the direct Base64 imports used by the updated websocket and proxy tests. * test(server): keep expect rejection socket open * fix(review): address HTTP 2 cleanup comments * fix(docs): update examples from review * fix(tls): fall back for registered Reseau * fix(client): qualify try ignore in generated client * test(server): avoid client pool race in unread body close test * fix(review): finish ignore cleanup and trim diagnostics * test(server): half-close expect rejection probe * fix(tls): avoid dynamic config rebuilds under trim * test: avoid helper name collisions * fix(server): buffer request bodies for handlers Read ordinary server request bodies before invoking request handlers so handlers receive a ready in-memory body. Keep incremental body reads in stream handlers, and simplify the affected docs examples to use JSON's direct struct support and req.body. * test(trim): expand high-level HTTP coverage Remove the optional frontier trim gate and keep the trim verifier strict for all included workloads. Add high-level H1 TLS request trim coverage, include TLS/gzip/deflate in the precompile workload, and broaden WebSocket plus HTTP/2 server branch coverage. Local verification: JULIA_NUM_THREADS=2 julia --project -e 'using Pkg; Pkg.test()' * test(trim): stabilize runtime workloads * fix(server): close response bodies * test(trim): harden CI compile harness * test(trim): allow slower Windows executables * test(server): stabilize raw error probes * test(trim): skip Windows JuliaC workload * docs: note HTTP.download migration path * 2.0 onboarding polish: register! do-block, listenany on WS, timeout wrapping, status field, parse_multipart_form(req), default UA, doc clarifications Closes a cluster of friction points surfaced by a multi-persona (Python / JS / Rust / Go / curl) developer-experience review of the 2.0 branch: - Add `register!(handler, router, [method,] path)` shims so the documented `do`-block syntax actually works. - Accept `listenany=true` on `WebSockets.listen!`/`serve!` to bind to an ephemeral port (matches `HTTP.serve!`). - Wrap `Reseau.IOPoll.DeadlineExceededError`, `HostResolvers.DialTimeoutError`, and `TLS.TLSHandshakeTimeoutError` as `HTTP.TimeoutError` at the public `request` / `HTTP.open` / `WebSockets.open` boundaries, matching the migration guide promise. - Add `status::Int16` field to `StatusError` so callers can write `e.status` instead of `e.response.status`. - Add `parse_multipart_form(req)` server-side overload + test. - Send a default `User-Agent: HTTP.jl/<version>` when the caller does not override it. - Document the `String(response.body)` body-aliasing gotcha and recommend `String(copy(response.body))` for round-trippable reads. - Document the JSON.jl pattern (`body = JSON.json(payload)` plus an explicit `Content-Type: application/json` header) in the client guide, README, and `HTTP.request` docstring. - Document that `HTTP.open(f, ...)` returns the final `HTTP.Response`, not the value returned by `f`. - Note that `HTTP.download` still resolves (to `Base.download`) but bypasses HTTP.jl machinery; updated migration guide accordingly. - Add `using HTTP` (and `using JSON` / `using Downloads` where relevant) to runnable examples in the docs and README. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * perf: server-side throughput optimizations + benchmark suite Cumulative effect on /json c=64 H/1.1: 16,300 req/s (the v2 starting point) → 88,000 req/s (~2.93× v1.x baseline). H/1.1 hits parity-or-better (1.57×–2.93×) on every cell vs HTTP.jl 1.x; H/2 is comfortably above 1× for small/medium responses, ~0.86× v1 H/1.1 for /large at sustained concurrency (within typical production-server H/2-vs-H/1.1 overhead). Server-side fixes that materially moved throughput, ranked by impact: 1. Batch the entire HTTP/1.1 response head into a single socket write (status line + headers + blank CRLF). Reseau's TCP.Conn.write does not buffer internally, so the previous `print(io, x, y, z, …)` pattern issued ~20 syscalls per response head. This was the dominant per-request cost. (+5x on /json c=64.) 2. Skip per-request deadline kernel calls when serve! is called with no timeouts (the default). Two useless syscalls per request → 0. 3. Zero-copy String body wrapping in `_compat_body_arg` — the body is referenced via Base.codeunits(s) instead of a fresh Vector{UInt8}(codeunits(s)) memcpy on every Response construction. String immutability makes this safe; the body code paths only read via copyto!/unsafe_write. (+200% on /large H/1.1, +26% on /large H/2.) 4. HTTP/2 batched DATA-frame emission with stamped frame headers — all DATA frames the current peer flow-control window allows go in one contiguous buffer with one socket write per batch. 5. HTTP/2 HEADERS + first DATA batch in one socket write under one write_lock acquisition (HPACK encoder mutation and wire emission stay on the same lock for ordering). 6. _readline_crlf fast path — `unsafe_string(ptr, len)` directly from the connection buffer when the line fits in the current fill, avoiding the per-line Vector{UInt8} alloc + append! chain. 7. HPACK static-table O(1) lookup — pre-built `Dict` for static-table exact (name, value) and name-only lookups in `_find_exact_index` / `_find_name_index`. The static table has 61 entries scanned twice per encoded header field; this drops to a single hash lookup. (+5–6% on /json H/2 c=64.) Other quality fixes folded in: - `_H2_SERVER_MAX_DATA_FRAME_SIZE` bumped from 16 KiB → 64 KiB; we still respect peer's SETTINGS_MAX_FRAME_SIZE at framing time. - HTTP.Headers Dict-indexable: `Base.getindex`, `Base.get`, and `Base.haskey` now work case-insensitively, matching the `r.headers["..."]` / `get(r.headers, ..., default)` / `haskey(r.headers, ...)` idioms developers from Python/JS/Go expect. - HPACK encoder pre-sizes its output Vector via `sizehint!` based on name+value lengths, avoiding the doubling-grow chain. - `_write_data_frames_h2_server!` and `_write_h2_response_body_h2_server!` no longer attempt to drain additional flow-control reservations before writing the first batch's bytes — the previous "build more then write" inner loop deadlocked against tests/peers that gate further DATA on a WINDOW_UPDATE in response to the first emitted bytes. Bench suite (`bench/`) is included for reproduction: - `bench/server.jl` — matched serve! handler used against both v1.x and v2 projects; three endpoints (/tiny, /json, /large). - `bench/all.sh`, `bench/all_repeated.sh` — drive h2load matrix (3 endpoints × 2 protocols × 3 concurrencies, optional 3-trial median). - `bench/parse.jl`, `bench/parse_avg.jl` — h2load output → CSV + Markdown summary. - `bench/REPORT.md`, `bench/PARITY_REPORT.md` — write-ups of the optimization rounds and which fixes mattered. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * fix(http2): avoid response flow-control deadlock * test: stabilize nonreplayable redirect on windows * test: stabilize unsupported expect wire read * fix(http2): handle text bodies in batched writes * merge: keep pushed http2_server changes; add JSON dev deps to bench Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * fix: 9 bugs, 5 footguns, 5 doc gaps from round-2 UX fuzz testing Found by 8 parallel testing agents running 207 concrete user scenarios across REST API, client, WebSockets, streaming/SSE/files, auth/TLS, HTTP/2, onboarding/docs, and error-handling angles. See `experiments/round2/REPORT.md` for the underlying scenarios + findings. Real bugs (would not ship 2.0 with): - HTTP/2 client multiplex race at c=4: stream-id allocation under state_lock and HEADERS write under write_lock could reorder two concurrent requests' frames on the wire, triggering peer GOAWAY. Fix: allocate stream id and write initial HEADERS atomically under one write_lock acquisition. Adds `pending_stream_count` so the pool's "open another conn at MAX_CONCURRENT_STREAMS" optimization still works. Verified: 100 concurrent `HTTP.get(url; client=client, protocol=:h2)` complete cleanly across 5 rounds. - HTTP/2 trailers never reached client: trailers were staged in `state.pending_trailers` but only published to the response on body EOF. Now published immediately on trailer arrival and on response build. - Client TLS ALPN restriction ignored: `_make_tls_config_for_h2` forced "h2" into the user's ALPN list, and `_use_h2` ignored the user's `tls_config.alpn_protocols`. Both fixed; user's list is now honored. - WebSocket `for msg in ws` silently dropped buffered messages on close race (`Base.iterate` early-returned on `isclosed(ws)` without checking `isready(ws.readchannel)`). Reproducer drops messages 100% of the time at N=50; with fix passes 20/20 trials. - WebSocket auto-close override: when a 1009/1007/1002 was queued internally and the handler returned normally, the finally-block overwrote the close code with 1000. Now preserves `ws.closebody`. - `HTTP.Stream` lacked byte-IO methods (`readline`, `readavailable`, `read(stream, UInt8)`): `readline` raised "does not support byte I/O", blocking LLM-streaming clients (OpenAI/Anthropic non-SSE token streams) and log-tailers. Added the missing methods. - `HTTP.SameSiteStrictMode`/`SameSiteLaxMode`/`SameSiteNoneMode`/ `SameSiteDefaultMode` not re-exported (`UndefVarError` on the documented Cookie pattern). Re-exported. Also accept Symbol shorthand `samesite=:strict` etc. - Server handler returning non-Response silently broke framing (client saw `ParseError`). Now logs `@error` with the actual type and emits HTTP 500. Applied to both H/1.1 and H/2 server paths. Footguns: - Reseau internal types leaked through public API: connect-refused, DNS, TLS-handshake, and bind-in-use surfaced as raw `Reseau.HostResolvers.OpError` / `Reseau.TLS.TLSError` types. Added `HTTP.ConnectError`, `HTTP.DNSError`, `HTTP.TLSHandshakeError`, `HTTP.AddressInUseError` (all `<: HTTPError`). Wrapping applied at every public entrypoint: `HTTP.request`, `HTTP.do!`, `HTTP.open`, `HTTP.serve!`, `HTTP.listen!`. The published catch-all recipe (`e isa HTTP.HTTPError || e isa Base.IOError`) now matches DNS, connect, TLS, and bind failures. - `HTTP.TimeoutError` carried no useful detail: `operation` was always `"request"` and `timeout_ns` was always `0`. Now populated with one of `"connect"`, `"tls_handshake"`, `"request"`, `"read_idle"`, etc., plus `timeout_ns` and `elapsed_ns` fields. `showerror` reports them. Backward-compatible 2-arg constructor retained. - `HTTP.Client` was not a configuration object: no per-client defaults, no positional `HTTP.get(client, url)`. Added `default_headers`, `default_query`, `default_basicauth`, and per-timeout defaults to `Client`. Per-call kwargs override client defaults. Added positional verb overloads for `get`/ `head`/`post`/`put`/`patch`/`delete`/`options`/`request`/`open`. - `close(::Client)` did not poison further use. Added atomic `closed::Bool` to `Client`; `close` sets it; subsequent `request`/`open`/`do!` throw `ArgumentError("HTTP.Client is closed")`. - `RequestContext` cancellation plumbing missing on public API. `context::RequestContext` is now a kwarg on `request`/`get`/`head`/ `post`/`put`/`patch`/`delete`/`options`/`open`/`do!`. `cancel!(ctx)` fires registered callbacks; the transport now registers a callback that closes the connection / forces a near-zero deadline so an in-flight read/write returns immediately. Verified: in-flight request canceled in ~170 ms (vs `>8s` before, where cancellation did not interrupt at all). Documentation: - Added "Query parameters" subsection to `docs/src/guides/client.md` with Dict / Vector / multi-value / append-to-existing examples. (Was: zero non-docstring documentation for `query=`.) - Added "Debugging requests" subsection covering `verbose=1`/`verbose=2` output and the `HTTP.ClientEvent` trace-event types. (Was: `verbose=2` worked but was undiscoverable.) - Server-guide timeout snippets now define `handler` first so verbatim copy-paste actually runs. (Was: `UndefVarError: handler`.) - Migration guide `HTTP.download` admonition reworded — options now documented as raising `MethodError` rather than "silently ignored". - Module-path naming sweep: README, server, protocols, and migration guides now use `HTTP.WebSockets.X` / `HTTP.Router` / `HTTP.register!` consistently. (Was: mixed `WebSockets.X` vs `HTTP.WebSockets.X`, `HTTP.Router` vs `HTTP.Handlers.Router`.) Tests: - `test/http2_client_tests.jl`: 3 new regression testsets for h2 multiplexing, trailer surfacing, and ALPN restriction. - `test/http_client_tests.jl`: 7 new tests for Client defaults + positional verbs + cancellation + close poisoning, plus 12 new tests for transport error wrapping. - `test/http_core_tests.jl`: 14 new tests for new error types and `TimeoutError` field semantics. - `test/http_cookie_tests.jl`: 15 new tests for `SameSite*Mode` exposure and Symbol shorthand. - `test/http_server_http1_tests.jl`: regression test for handler returning non-Response → 500. - `test/http_websocket_integration_tests.jl`: 2 new testsets — race drains buffered messages on close race (10 trials), and 1009 close code preserved through handler return. Trim-compile note: the bind-error wrapper previously used `current_exceptions(task)` to inspect a failed startup task; the JuliaC trim verifier rejects that. Replaced with synchronous `wait(task)` rethrow. Synchronous bind in `serve!(handler, host, port; ...)` already wraps `EADDRINUSE` to `HTTP.AddressInUseError` before the task is ever spawned, so the user-facing behavior is unchanged. Reseau side (separate branch `jq-tls13-mtls`): - TLS 1.3 mTLS: the auto-negotiating client handshake path (`_native_tls_auto_client_handshake!`) was not loading the local identity into the TLS 1.3 state, so the client emitted an empty `Certificate` message and the server rejected with `certificate_required`. Fixed. - TLS Alert decoding: `Reseau.TLS.TLSError` messages now include the RFC 8446/5246 symbolic name for fatal alerts (e.g. `bad_certificate (42)` instead of `alert 42`). Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * fix(client): harden cancellation and query defaults Keep RequestContext cancellation callbacks registered only while the active response body owns the transport, add HTTP/2 stream reset handling for cancellation, normalize client default query values with string(), restore generic HTTP/2 server payload copying, and make transport error tests stable across platforms. * test(client): tolerate windows bind reuse Make the AddressInUseError test verify the listen-error wrapper directly, while allowing the live duplicate-bind probe to succeed on Windows where the OS permits it. --------- Co-authored-by: OpenAI Codex <codex@openai.com> Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com>
1 parent a380230 commit 25be01b

179 files changed

Lines changed: 38867 additions & 16092 deletions

File tree

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

.github/workflows/ci.yml

Lines changed: 18 additions & 45 deletions
Original file line numberDiff line numberDiff line change
@@ -1,18 +1,12 @@
11
name: CI
2+
23
on:
3-
pull_request:
4-
branches:
5-
- master
64
push:
7-
branches:
8-
- master
9-
tags: '*'
10-
defaults:
11-
run:
12-
shell: bash
13-
concurrency:
14-
group: ${{ github.workflow }}-${{ github.ref || github.run_id }}
15-
cancel-in-progress: true
5+
branches: [main, master]
6+
tags: ["*"]
7+
pull_request:
8+
release:
9+
1610
jobs:
1711
test:
1812
name: Julia ${{ matrix.version }} - ${{ matrix.os }} - ${{ matrix.arch }} - ${{ github.event_name }}
@@ -21,61 +15,40 @@ jobs:
2115
fail-fast: false
2216
matrix:
2317
version:
24-
- 'min'
25-
- 'lts'
26-
- '1.9' # test because we know there was a precompilation issue on 1.9 issues/1202
27-
- '1' # automatically expands to the latest stable 1.x release of Julia
28-
- 'nightly'
18+
- '1'
19+
- 'pre'
2920
os:
3021
- ubuntu-latest
31-
- macOS-latest
3222
- windows-latest
3323
arch:
34-
- default
24+
- x64
3525
include:
36-
- os: windows-latest
37-
version: '1'
38-
arch: x86
39-
exclude:
4026
- os: macOS-latest
41-
version: 'min' # no apple silicon support for Julia 1.6
27+
arch: aarch64
28+
version: 1
4229
steps:
43-
- uses: actions/checkout@v4
30+
- uses: actions/checkout@v6
4431
- uses: julia-actions/setup-julia@v2
4532
with:
4633
version: ${{ matrix.version }}
4734
arch: ${{ matrix.arch }}
4835
- uses: julia-actions/cache@v2
4936
- uses: julia-actions/julia-buildpkg@v1
5037
- uses: julia-actions/julia-runtest@v1
51-
env:
52-
PIE_SOCKET_API_KEY: ${{ secrets.PIE_SOCKET_API_KEY }}
53-
JULIA_VERSION: ${{ matrix.version }}
54-
JULIA_NUM_THREADS: 2
5538
- uses: julia-actions/julia-processcoverage@v1
5639
- uses: codecov/codecov-action@v5
5740
with:
5841
files: lcov.info
59-
token: ${{ secrets.GITHUB_TOKEN }}
42+
token: ${{ secrets.CODECOV_TOKEN }}
6043
docs:
6144
name: Documentation
6245
runs-on: ubuntu-latest
46+
permissions:
47+
contents: write
6348
steps:
64-
- uses: actions/checkout@v4
65-
- uses: julia-actions/setup-julia@v2
66-
with:
67-
version: '1'
68-
- run: |
69-
julia --project=docs -e '
70-
using Pkg
71-
Pkg.develop(PackageSpec(path=pwd()))
72-
Pkg.instantiate()'
73-
- run: |
74-
julia --project=docs -e '
75-
using Documenter: doctest
76-
using HTTP
77-
doctest(HTTP)'
78-
- run: julia --project=docs docs/make.jl
49+
- uses: actions/checkout@v6
50+
- uses: julia-actions/julia-buildpkg@latest
51+
- uses: julia-actions/julia-docdeploy@latest
7952
env:
8053
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
8154
DOCUMENTER_KEY: ${{ secrets.DOCUMENTER_KEY }}

CHANGELOG.md

Lines changed: 115 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -7,14 +7,124 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
77

88
## [Unreleased]
99

10+
## [v2.0.0] - 2026-04-27
11+
HTTP.jl 2.0 is a major rewrite of the package internals and public API. The
12+
release keeps the familiar `HTTP.request`, verb helpers, `HTTP.serve`, routing,
13+
WebSocket, and SSE workflows, but rebuilds them on a smaller, explicit core
14+
that delegates transport mechanics to
15+
[`Reseau.jl`](https://github.com/JuliaServices/Reseau.jl). The 2.0 line is the
16+
new foundation for HTTP/1.1, HTTP/2, timeouts, streaming, precompilation, and
17+
future protocol work.
18+
19+
### Added
20+
- Added a new core API around `Request`, `Response`, `Headers`,
21+
`RequestContext`, explicit request/response body types, and structured
22+
timeout configuration.
23+
- Added `Client` and `Transport` objects for reusable client configuration,
24+
connection reuse, cookie jars, redirect limits, retry policy, default
25+
headers, and HTTP/2 preference.
26+
- Added first-class HTTP/2 client and server support through the Reseau-backed
27+
transport layer, including stream lifecycle handling, flow-control-aware
28+
body reads, and h2c prior-knowledge support for local/plaintext use cases.
29+
- Added structured timeout phases for connection establishment, complete
30+
request deadlines, response-header waits, read-idle waits, write-idle waits,
31+
and `Expect: 100-continue` waits.
32+
- Added `HTTP.HTTPTimeoutError` as the public timeout exception alias and
33+
richer timeout messages for client and server failures.
34+
- Added `HTTP.open` for response streaming with the same request pipeline as
35+
`HTTP.request`.
36+
- Added request tracing support through trace callbacks, client event types,
37+
and the `verbose` request keyword.
38+
- Added server helpers for streaming handlers, static content, file serving,
39+
file-server roots, graceful shutdown callbacks, handler timeouts, and
40+
socket-close control.
41+
- Added Server-Sent Events support for clients and servers through
42+
`SSEEvent`, `sse_stream`, and the `sse_callback` request keyword.
43+
- Added WebSocket client/server support under `HTTP.WebSockets`, including
44+
`HTTP.WebSockets.open`, `HTTP.WebSockets.listen`, and
45+
`HTTP.WebSockets.listen!`.
46+
- Added precompile workloads that cover common client, server, router, SSE,
47+
WebSocket, and streaming paths.
48+
- Added curated manual and API-reference documentation for the 2.0 public
49+
surface, plus a migration guide for common 1.x call patterns.
50+
51+
### Changed
52+
- The package now presents a curated 2.0 public API built around `Request`,
53+
`Response`, `Stream`, `Client`, `Transport`, and explicit body types on top
54+
of `Reseau`.
55+
- Julia 1.10 is now the minimum supported Julia version.
56+
- `HTTP.request` and the verb helpers now return a fully materialized
57+
`Response.body::Vector{UInt8}` by default. Use `HTTP.open` or the
58+
`response_stream` keyword when you need streaming control.
59+
- `Request` and `Response` constructors now prefer explicit keyword-oriented
60+
forms. Common 1.x positional constructor forms remain accepted as migration
61+
shims, but new code should use `Request(method, target; ...)` and
62+
`Response(status; headers=..., body=...)`.
63+
- `Response.status_code` has been renamed to `Response.status`.
64+
- `RequestContext` is now a typed request state object with cancellation,
65+
deadline, metadata, and timeout fields. Dict-like metadata access is
66+
preserved for migration, but new code should treat it as request state rather
67+
than as a plain `Dict{Symbol,Any}`.
68+
- Client retry configuration now centers on `retry`, `retries`, `retry_if`,
69+
`respect_retry_after`, and `retry_bucket` instead of the 1.x
70+
`retry_delays` / `retry_check` pair.
71+
- Client timeout keywords now use explicit phase names. In particular,
72+
`readtimeout` is replaced by `read_idle_timeout`; request-wide deadlines
73+
should use `request_timeout`.
74+
- Connection pooling is now owned by `Client` / `Transport` instead of the 1.x
75+
`pool` keyword and layer stack.
76+
- TLS and socket behavior are now configured through the Reseau transport layer
77+
instead of the 1.x `sslconfig` / `socket_type_tls` extension points.
78+
- Routing helpers are documented as part of the `HTTP.Handlers` surface, while
79+
the high-level server APIs continue to accept ordinary Julia functions,
80+
router objects, middleware, and stream handlers.
81+
- WebSocket entrypoints now live under `HTTP.WebSockets`; use
82+
`HTTP.WebSockets.open`, `HTTP.WebSockets.listen`, and
83+
`HTTP.WebSockets.listen!` for WebSocket-specific behavior.
84+
- The documentation now explicitly calls out intentionally deferred Go-parity
85+
areas for the 2.x release line, including HTTP/2 server push, `Pusher`,
86+
`ResponseController` / hijack-style control APIs, fuller `httptrace`
87+
coverage, and full `net/url` / `ServeMux` parity.
88+
89+
### Deprecated
90+
- `readtimeout` remains accepted as a migration alias for `read_idle_timeout`.
91+
- Several 1.x client keywords remain accepted for compatibility but are either
92+
mapped to the new API or ignored with migration warnings when there is no
93+
direct 2.0 equivalent: `pool`, `retry_delays`, `retry_check`, `sslconfig`,
94+
`socket_type_tls`, `copyheaders`, `canonicalize_headers`,
95+
`detect_content_type`, `logerrors`, `logtag`, and `observelayers`.
96+
- Treating `RequestContext` as a plain dictionary is deprecated in favor of the
97+
typed fields and metadata helpers.
98+
99+
### Removed
100+
- Undocumented 1.x internals are no longer supported migration targets for the
101+
2.0 line.
102+
- The old layer-stack internals, connection-pool internals, HPACK internals,
103+
parser internals, and low-level HTTP/2 implementation details are no longer
104+
documented public API.
105+
- The package no longer carries a source override for Reseau; release builds
106+
resolve the registered `Reseau = "1"` dependency.
107+
108+
### Fixed
109+
- Hardened HTTP/1.1 and HTTP/2 server behavior around request-body streaming,
110+
content-length accounting, invalid header handling, timeout cleanup,
111+
connection teardown, and stream cancellation.
112+
- Hardened client-side redirect, retry, proxy, cookie, timeout, and streaming
113+
paths against the new transport abstraction.
114+
- Improved docs coverage so exported APIs and documented submodule APIs are
115+
covered by the Documenter build.
116+
- Skipped the package precompile workload when Julia is running with coverage
117+
instrumentation, keeping CI focused on tests while preserving normal package
118+
precompilation for users.
119+
10120
## [v1.11.0] - 2025-12-20
11121
### Added
12122
- Added full Server-Sent Events (SSE) support for both client and server:
13123
- **Client-side**: `sse_callback` keyword argument for `HTTP.request` to parse SSE streams on
14124
successful responses, invoking a callback with `HTTP.SSEEvent` for each event received.
15-
- **Server-side**: `HTTP.sse_stream(response) do stream ... end` helper to write
125+
- **Server-side**: `HTTP.sse_stream(200) do stream ... end` helper to write
16126
`HTTP.SSEEvent`s and automatically close the stream when the block finishes (or use
17-
`HTTP.sse_stream(response)` for manual management).
127+
`response = HTTP.sse_stream(200)` for manual management).
18128
- `HTTP.SSEEvent` struct for representing SSE events with `data`, `event`, `id`, and `retry` fields.
19129

20130
## [v1.10.1] - 2023-11-28
@@ -472,7 +582,9 @@ See changes for 0.9.15: this release is equivalent to 0.9.15 with [#752] reverte
472582
`HTTP.request` and friends ([#619]).
473583

474584

475-
[Unreleased]: https://github.com/JuliaWeb/HTTP.jl/compare/v1.10.1...HEAD
585+
[Unreleased]: https://github.com/JuliaWeb/HTTP.jl/compare/v2.0.0...HEAD
586+
[v2.0.0]: https://github.com/JuliaWeb/HTTP.jl/compare/v1.11.0...v2.0.0
587+
[v1.11.0]: https://github.com/JuliaWeb/HTTP.jl/releases/tag/v1.11.0
476588

477589

478590
<!-- Links generated by Changelog.jl -->

LICENSE.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -2,7 +2,7 @@ The HTTP.jl package is licensed under the MIT "Expat" License:
22

33
> src/cookies.jl based on src/net/http/cookie.go Copyright 2009 The Go Authors
44
5-
> Copyright (c) 2022: https://github.com/JuliaWeb/HTTP.jl/graphs/contributors.
5+
> Copyright (c) 2026: https://github.com/JuliaWeb/HTTP.jl/graphs/contributors.
66
>
77
> Permission is hereby granted, free of charge, to any person obtaining a copy
88
> of this software and associated documentation files (the "Software"), to deal

Project.toml

Lines changed: 13 additions & 27 deletions
Original file line numberDiff line numberDiff line change
@@ -1,47 +1,33 @@
11
name = "HTTP"
22
uuid = "cd3eb016-35fb-5094-929b-558a96fad6f3"
3+
version = "2.0.0"
34
authors = ["Jacob Quinn", "contributors: https://github.com/JuliaWeb/HTTP.jl/graphs/contributors"]
4-
version = "1.11.0"
55

66
[deps]
77
Base64 = "2a0f44e3-6c83-55bd-87e4-b1978d98bd5f"
88
CodecZlib = "944b1d66-785c-5afd-91f1-9de20f533193"
9-
ConcurrentUtilities = "f0e56b4a-5159-44fe-b623-3e5288b988bb"
109
Dates = "ade2ca70-3891-5945-98fb-dc099432e06a"
11-
ExceptionUnwrapping = "460bff9d-24e4-43bc-9d9f-a8973cb893f4"
12-
Logging = "56ddb016-857b-54e1-b83d-db4d58db5568"
13-
LoggingExtras = "e6f89c97-d47a-5376-807f-9c37f3926c36"
14-
MbedTLS = "739be429-bea8-5141-9913-cc70e7f3736d"
15-
NetworkOptions = "ca575930-c2e3-43a9-ace4-1e988b2c1908"
16-
OpenSSL = "4d8831e6-92b7-49fb-bdf8-b643e874388c"
17-
PrecompileTools = "aea7be01-6a6a-4083-8856-8a6e6704d82a"
10+
EnumX = "4e289a0a-7415-4d19-859d-a7e5c4648b56"
1811
Random = "9a3f8284-a2c9-5f02-9a11-845980a1fd5c"
19-
SimpleBufferStream = "777ac1f9-54b0-4bf8-805c-2214025038e7"
20-
Sockets = "6462fe0b-24de-5631-8697-dd941f90decc"
12+
PrecompileTools = "aea7be01-6a6a-4083-8856-8a6e6704d82a"
13+
Reseau = "802f3686-a58f-41ce-bb0c-3c43c75bba36"
14+
SHA = "ea8e919c-243c-51af-8825-aaa63cd721ce"
2115
URIs = "5c2747f8-b7ea-4ff2-ba2e-563bfd36b1d4"
2216
UUIDs = "cf7118a7-6976-5b1a-9a39-7adc72f591a4"
2317

2418
[compat]
2519
CodecZlib = "0.7"
26-
ConcurrentUtilities = "2.4"
27-
ExceptionUnwrapping = "0.1"
28-
LoggingExtras = "0.4.9,1"
29-
MbedTLS = "0.6.8, 0.7, 1"
30-
OpenSSL = "1.3"
20+
EnumX = "1"
3121
PrecompileTools = "1.2.1"
32-
SimpleBufferStream = "1.1"
33-
URIs = "1.6" # We need URIs >= 1.6 to ensure we have https://github.com/JuliaWeb/URIs.jl/pull/66
34-
julia = "1.6"
22+
Reseau = "1.1.1"
23+
URIs = "1.6.1"
24+
julia = "1.10"
3525

3626
[extras]
37-
BufferedStreams = "e1450e63-4bb3-523b-b2a4-4ffa8c0fd77d"
38-
Deno_jll = "04572ae6-984a-583e-9378-9577a1c2574d"
39-
Distributed = "8ba89e20-285c-5b6f-9357-94700520ee1b"
40-
InteractiveUtils = "b77e0a4c-d291-57a0-90e8-8db25a27a240"
41-
JSON = "682c06a0-de6a-54ab-a142-c8b1cf79cde6"
42-
NetworkOptions = "ca575930-c2e3-43a9-ace4-1e988b2c1908"
27+
Base64 = "2a0f44e3-6c83-55bd-87e4-b1978d98bd5f"
28+
JuliaC = "acedd4c2-ced6-4a15-accc-2607eb759ba2"
29+
Reseau = "802f3686-a58f-41ce-bb0c-3c43c75bba36"
4330
Test = "8dfed614-e22c-5e08-85e1-65c5234f0b40"
44-
Unitful = "1986cc42-f94f-5a68-af5c-568840ba703d"
4531

4632
[targets]
47-
test = ["BufferedStreams", "Deno_jll", "Distributed", "InteractiveUtils", "JSON", "Test", "Unitful", "NetworkOptions"]
33+
test = ["Base64", "JuliaC", "Reseau", "Test"]

0 commit comments

Comments
 (0)