specs for transport

Author: evgeny-simplex

spec/TOPICS.md

@@ -7,3 +7,9 @@
 - **Padding schemes**: Three different padding formats across the codebase — Crypto.hs uses 2-byte Word16 length prefix (max ~65KB), Crypto/Lazy.hs uses 8-byte Int64 prefix (file-sized), and both use '#' fill character. Ratchet header padding uses fixed sizes (88 or 2310 bytes). All use `pad`/`unPad` but with incompatible formats. The relationship between padding, encryption, and message size limits spans Crypto, Lazy, Ratchet, and the protocol layer.
 
 - **NaCl construction variants**: crypto_box, secret_box, and KEM hybrid secret all use the same XSalsa20+Poly1305 core (Crypto.hs `xSalsa20`), but with different key sources (DH, symmetric, SHA3_256(DH||KEM)). The lazy streaming variant (Lazy.hs) adds prepend-tag vs tail-tag placement. File.hs wraps lazy streaming with handle-based I/O. Full picture requires reading Crypto.hs, Lazy.hs, File.hs, and SNTRUP761.hs together.
+
+- **Transport encryption layering**: Three encryption layers overlap — TLS (Transport.hs), optional block encryption via sbcHkdf chains (Transport.hs tPutBlock/tGetBlock), and SMP protocol-level encryption. Block encryption is disabled for proxy connections (already encrypted), and absent for NTF protocol. The interaction of these layers with proxy version downgrade logic spans Transport.hs, Client.hs, and the SMP proxy module.
+
+- **Certificate chain trust model**: ChainCertificates (Shared.hs) defines 0–4 cert chain semantics, used by both Client.hs (validateCertificateChain) and Server.hs (validateClientCertificate, SNI credential switching). The 4-length case skipping index 2 (operator cert) and the FQHN-disabled x509validate are decisions that span the entire transport security model.
+
+- **Handshake protocol family**: SMP (Transport.hs), NTF (Notifications/Transport.hs), and XFTP (FileTransfer/Transport.hs) all have handshake protocols with the same structure (version negotiation + session binding + key exchange) but different feature sets. NTF is a strict subset. XFTP doesn't use the TLS handshake at all (HTTP2 layer). The shared types (THandle, THandleParams, THandleAuth) mean changes to the handshake infrastructure affect all three protocols.

spec/modules/Simplex/FileTransfer/Transport.md

@@ -0,0 +1,23 @@
+# Simplex.FileTransfer.Transport
+
+> XFTP protocol types, version negotiation, and encrypted file streaming with integrity verification.
+
+**Source**: [`FileTransfer/Transport.hs`](../../../../src/Simplex/FileTransfer/Transport.hs)
+
+## xftpClientHandshakeStub — XFTP doesn't use TLS handshake
+
+`xftpClientHandshakeStub` always fails with `throwE TEVersion`. The source comment states: "XFTP protocol does not use this handshake method." The XFTP handshake is performed at the HTTP/2 layer — `XFTPServerHandshake` and `XFTPClientHandshake` are sent as HTTP/2 request/response bodies (see `FileTransfer/Client.hs` and `FileTransfer/Server.hs`).
+
+## receiveSbFile — constant-time auth tag verification
+
+`receiveSbFile` validates the authentication tag using `BA.constEq` (constant-time byte comparison). The auth tag is collected from the stream after all file data — if the file data ends mid-chunk, the remaining bytes of that chunk are used first, and a follow-up read provides the rest of the tag if needed.
+
+## receiveFile_ — two-phase integrity verification
+
+File reception has two verification phases:
+1. **During receive**: either size checking (plaintext via `hReceiveFile`) or auth tag validation (encrypted via `receiveSbFile`)
+2. **After receive**: `LC.sha256Hash` of the entire received file is compared to `chunkDigest`
+
+## sendEncFile — auth tag appended after all chunks
+
+`sendEncFile` streams encrypted chunks via `LC.sbEncryptChunk`, then sends `LC.sbAuth sbState` (the authentication tag) as a final frame when the remaining size reaches zero.

spec/modules/Simplex/Messaging/Notifications/Transport.md

@@ -0,0 +1,36 @@
+# Simplex.Messaging.Notifications.Transport
+
+> Notification Router Protocol transport: manages push notification subscriptions between client and NTF Router.
+
+**Source**: [`Notifications/Transport.hs`](../../../../../src/Simplex/Messaging/Notifications/Transport.hs)
+
+**Protocol spec**: [`protocol/push-notifications.md`](../../../../../protocol/push-notifications.md) — SimpleX Notification Router protocol.
+
+## Overview
+
+This module implements the transport layer for the **Notification Router Protocol**. Per the protocol spec: "To manage notification subscriptions to SMP routers, SimpleX Notification Router provides an RPC protocol with a similar design to SimpleX Messaging Protocol router."
+
+The protocol spec diagram shows three separate protocols in the notification flow:
+1. **Notification Router Protocol** (this module): client ↔ SimpleX Notification Router — subscription management
+2. **SMP protocol**: SMP Router → SimpleX Notifications Subscriber — notification signals
+3. **Push provider** (e.g., APN): SimpleX Push Router → device — per the spec: "the notifications are e2e encrypted between SimpleX Notification Router and the user's device"
+
+## Differences from SMP transport
+
+The NTF protocol reuses SMP's transport infrastructure but with reduced parameters:
+
+| Property | SMP | NTF |
+|----------|-----|-----|
+| Block size | 16384 | 512 |
+| Block encryption | Yes (v11+) | No (`encryptBlock = Nothing`) |
+| Service certificates | Yes (v16+) | No (`serviceAuth = False`) |
+| Version range | 6–19 | 1–3 |
+| Handshake messages | 2–3 | 2 |
+
+## Same ALPN/legacy fallback pattern as SMP
+
+`ntfServerHandshake` uses the same pattern as `smpServerHandshake`: if ALPN is not negotiated (`getSessionALPN` returns `Nothing`), the server offers only `legacyServerNTFVRange` (v1 only).
+
+## NTF handshake uses SMP shared types
+
+The handshake reuses SMP's `THandle`, `THandleParams`, `THandleAuth` types. The `encodeAuthEncryptCmds` and `authEncryptCmdsP` helper functions are defined locally in this module (with NTF-specific version thresholds). NTF never sets `sessSecret` / `sessSecret'`, `peerClientService`, or `clientService` — these are always `Nothing`.

spec/modules/Simplex/Messaging/Transport.md

@@ -0,0 +1,114 @@
+# Simplex.Messaging.Transport
+
+> SMP transport layer: TLS connection management, SMP handshake protocol, block encryption, version negotiation.
+
+**Source**: [`Transport.hs`](../../../../src/Simplex/Messaging/Transport.hs)
+
+**Protocol spec**: [`protocol/simplex-messaging.md` — Transport connection](../../../../protocol/simplex-messaging.md#transport-connection-with-the-smp-router) — SMP encrypted transport, handshake syntax, certificate chain requirements.
+
+## Overview
+
+This is the core transport module. It defines:
+- The `Transport` typeclass abstracting over TLS and WebSocket connections
+- The SMP handshake protocol (server and client sides)
+- Optional block encryption using HKDF-derived symmetric key chains (v11+)
+- Version negotiation with backward-compatible extensions
+
+Per the protocol spec: "Each transport block has a fixed size of 16384 bytes for traffic uniformity." The `sessionIdentifier` field uses tls-unique channel binding (RFC 5929) — "it should be included in authorized part of all SMP transmissions sent in this transport connection."
+
+## SMP version 13 is missing
+
+The version history jumps from 12 (`blockedEntitySMPVersion`) to 14 (`proxyServerHandshakeSMPVersion`). Version 13 was skipped.
+
+## proxiedSMPRelayVersion — anti-fingerprinting cap
+
+`proxiedSMPRelayVersion = 18`, one below `currentClientSMPRelayVersion = 19`. The code comment states: "SMP proxy sets it to lower than its current version to prevent client version fingerprinting by the destination relays when clients upgrade at different times."
+
+In practice (Server.hs), the SMP proxy uses `proxiedSMPRelayVRange` to cap the destination relay's version range in the `PKEY` response sent to the client, so the client sees a capped version range rather than the relay's actual range.
+
+## withTlsUnique — different API calls yield same value
+
+`withTlsUnique` extracts the tls-unique channel binding (RFC 5929) using a type-level dispatch:
+- **Server** (`STServer`): `T.getPeerFinished` — the peer's (client's) Finished message
+- **Client** (`STClient`): `T.getFinished` — own (client's) Finished message
+
+Both calls yield the client's Finished message. If the result is `Nothing`, the connection is closed immediately (`closeTLS cxt >> ioe_EOF`).
+
+## defaultSupportedParams vs defaultSupportedParamsHTTPS
+
+Two TLS parameter sets:
+
+- **`defaultSupportedParams`**: ChaCha20-Poly1305 ciphers only, Ed448/Ed25519 signatures only, X448/X25519 groups. Per the protocol spec: "TLS_CHACHA20_POLY1305_SHA256 cipher suite, ed25519 EdDSA algorithms for signatures, x25519 ECDHE groups for key exchange."
+- **`defaultSupportedParamsHTTPS`**: extends `defaultSupportedParams` with `ciphersuite_strong`, additional groups, and additional hash/signature combinations. The source comment says: "A selection of extra parameters to accomodate browser chains."
+
+In the SMP server (Server.hs), when HTTP credentials are configured, `defaultSupportedParamsHTTPS` is used for all connections on that port (not selected per-connection). When no HTTP credentials are configured, `defaultSupportedParams` is used.
+
+## SMP handshake flow
+
+Per the [protocol spec](../../../../protocol/simplex-messaging.md#transport-handshake), the handshake is a two-message exchange (three if service certs are used):
+
+1. **Server → Client**: `paddedRouterHello` containing `smpVersionRange`, `sessionIdentifier` (tls-unique), and `routerCertKey` (certificate chain + X25519 key signed by the server's certificate)
+2. **Client → Server**: `paddedClientHello` containing agreed `smpVersion`, `keyHash` (router identity — CA certificate fingerprint), optional `clientKey`, `proxyRouter` flag, and optional `clientService`
+3. **Server → Client** (service only): `paddedRouterHandshakeResponse` containing assigned `serviceId` or `handshakeError`
+
+The client verifies `sessionIdentifier` matches its own tls-unique (`when (sessionId /= sessId) $ throwE TEBadSession`). The server verifies `keyHash` matches its CA fingerprint (`when (keyHash /= kh) $ throwE $ TEHandshake IDENTITY`).
+
+Per the protocol spec: "For TLS transport client should assert that sessionIdentifier is equal to tls-unique channel binding defined in RFC 5929."
+
+### legacyServerSMPRelayVRange when no ALPN
+
+If ALPN is not negotiated (`getSessionALPN c` returns `Nothing`), the server offers `legacyServerSMPRelayVRange` (v6 only) instead of the full version range. Per the protocol spec: "If the client does not confirm this protocol name, the router would fall back to v6 of SMP protocol." The spec notes: "This is added to allow support of older clients without breaking backward compatibility and to extend or modify handshake syntax."
+
+### Service certificate handshake extension
+
+When `clientService` is present in the client handshake, the server performs additional verification:
+- The TLS client certificate chain must exactly match the certificate chain in the handshake message (`getPeerCertChain c == cc`)
+- The signed X25519 public key is verified against the leaf certificate's key (`getCertVerifyKey leafCert` then `C.verifyX509`)
+- On success, the server sends `SMPServerHandshakeResponse` with a `serviceId`
+- On failure, the server sends `SMPServerHandshakeError` before raising the error
+
+Per the protocol spec (v16+): "`clientService` provides long-term service client certificate for high-volume services using SMP router (chat relays, notification routers, high traffic bots). The router responds with a third handshake message containing the assigned service ID."
+
+The client only includes service credentials when `v >= serviceCertsSMPVersion && certificateSent c` (the TLS client certificate was actually sent).
+
+## tPutBlock / tGetBlock — optional block encryption
+
+When `encryptBlock` is set, transport blocks are encrypted before being sent over TLS:
+
+- **Send**: `atomically $ stateTVar sndKey C.sbcHkdf` advances the chain key and returns `(SbKey, CbNonce)`; the block is encrypted with `C.sbEncrypt`
+- **Receive**: same pattern with `rcvKey` and `C.sbDecrypt`
+
+The chain keys are initialized from `C.sbcInit sessionId secret` where `sessionId` is the tls-unique value and `secret` is the session DH shared secret.
+
+The code comment on `proxyServer` flag states: "This property, if True, disables additional transport encrytion inside TLS. (Proxy server connection already has additional encryption, so this layer is not needed there)." The protocol spec confirms: "`proxyRouter` flag (v14+) disables additional transport encryption inside TLS for proxy connections, since proxy router connection already has additional encryption."
+
+The protocol spec version history (v11) describes this as "additional encryption of transport blocks with forward secrecy."
+
+## smpTHandleClient — chain key swap
+
+`smpTHandleClient` applies `swap` to the chain key pair before creating `TSbChainKeys`. The code comment states: "swap is needed to use client's sndKey as server's rcvKey and vice versa."
+
+## Proxy version downgrade logic
+
+When the proxy connects to a destination relay older than v14 (`proxyServerHandshakeSMPVersion`), the client-side handshake caps the version range:
+
+```
+if proxyServer && maxVersion smpVersionRange < proxyServerHandshakeSMPVersion
+  then vRange {maxVersion = max (minVersion vRange) deletedEventSMPVersion}
+```
+
+The code comment explains: "Transport encryption between proxy and destination breaks clients with version 10 or earlier, because of a larger message size (see maxMessageLength)." The cap at `deletedEventSMPVersion` (v10) ensures transport encryption (v11+) is not negotiated with older relays.
+
+The comment also notes: "Prior to version v6.3 the version between proxy and destination was capped at 8, by mistake, which also disables transport encryption and the latest features."
+
+## forceCertChain
+
+`forceCertChain` forces evaluation of the certificate chain and signed key via `length (show cc) `seq` show signedKey `seq` cert`. Introduced in commit 9e7e0d10 ("smp-server: conserve resources"), sub-bullet "transport: force auth params, remove async wrapper" — part of a commit that adds strictness annotations throughout (`bang more thunks`, `strict`).
+
+## smpTHandle — version 0 bootstrap
+
+`smpTHandle` creates a `THandle` with version 0, no auth, and no block encryption. This handle is used for the handshake exchange itself (`sendHandshake`/`getHandshake`). After the handshake completes, `smpTHandle_` creates the real handle with the negotiated version, auth, and encryption parameters.
+
+## getHandshake — forward-compatible parsing
+
+The code comment states: "ignores tail bytes to allow future extensions." The protocol spec confirms: "`ignoredPart` in handshake allows to add additional parameters in handshake without changing protocol version — the client and routers must ignore any extra bytes within the original block length."

spec/modules/Simplex/Messaging/Transport/Buffer.md

@@ -0,0 +1,17 @@
+# Simplex.Messaging.Transport.Buffer
+
+> Buffered TLS reading with TMVar-based concurrency lock.
+
+**Source**: [`Transport/Buffer.hs`](../../../../../src/Simplex/Messaging/Transport/Buffer.hs)
+
+## TBuffer — concurrent read safety via getLock
+
+`TBuffer` uses a `TMVar ()` as a mutex (`getLock`). `getBuffered` acquires the lock via `withBufferLock`, then loops and accumulates bytes until the requested count is reached.
+
+## getBuffered — first chunk has no timeout
+
+`getBuffered` reads the first chunk via `getChunk` (no timeout), but applies `withTimedErr t_` (the transport timeout) to subsequent chunks.
+
+## getLnBuffered — test only
+
+The source comment states: "This function is only used in test and needs to be improved before it can be used in production, it will never complete if TLS connection is closed before there is newline."

spec/modules/Simplex/Messaging/Transport/Client.md

@@ -0,0 +1,23 @@
+# Simplex.Messaging.Transport.Client
+
+> TLS client connection setup: TCP/SOCKS5 connection, TLS handshake, certificate validation, host types.
+
+**Source**: [`Transport/Client.hs`](../../../../../src/Simplex/Messaging/Transport/Client.hs)
+
+## ConnectionHandle — three-stage cleanup
+
+`ConnectionHandle` has three constructors: `CHSocket` (raw socket), `CHContext` (TLS context), `CHTransport` (transport connection). An `IORef` holds the current handle, updated by `set` on each successful transition. The `E.bracket` cleanup function tears down the connection at whatever stage it reached.
+
+## SocksIsolateByAuth
+
+`SocksIsolateByAuth` is the default SOCKS authentication mode. When active, [Simplex.Messaging.Client](../Client.md) generates SOCKS credentials (`SocksCredentials sessionUsername ""`) where `sessionUsername` is `B64.encode $ C.sha256Hash $ bshow userId <> ...` with additional components based on `sessionMode` (`TSMUser`, `TSMSession`, `TSMServer`, `TSMEntity`).
+
+The three modes defined here: `SocksAuthUsername` (explicit credentials), `SocksAuthNull` (no auth, `@` prefix), `SocksIsolateByAuth` (empty string — credentials generated by the caller).
+
+## validateCertificateChain
+
+Validation checks the SHA-256 fingerprint of the identity certificate (extracted via `chainIdCaCerts` — see [Shared.md](./Shared.md#chainidcacerts--certificate-chain-semantics)) against the key hash. If the fingerprint doesn't match, the chain is rejected with `UnknownCA`. If the fingerprint matches, standard X.509 validation is performed using the CA certificate as trust anchor.
+
+## No TLS timeout for client connections
+
+The code comment states: "No TLS timeout to avoid failing connections via SOCKS." `transportTimeout` is set to `Nothing` for all client connections via `clientTransportConfig`.

spec/modules/Simplex/Messaging/Transport/Credentials.md

@@ -0,0 +1,13 @@
+# Simplex.Messaging.Transport.Credentials
+
+> Certificate generation for transport layer: Ed25519 key pairs, X.509 signing, TLS credential extraction.
+
+**Source**: [`Transport/Credentials.hs`](../../../../../src/Simplex/Messaging/Transport/Credentials.hs)
+
+## genCredentials — nanosecond stripping
+
+`genCredentials` zeroes out nanoseconds from the current time before creating the certificate validity period: `todNSec = 0`. The source comment explains: "remove nanoseconds from time - certificate encoding/decoding removes them."
+
+## tlsCredentials — root fingerprint from last credential
+
+`tlsCredentials` extracts the SHA-256 fingerprint from `L.last credentials` (the root/CA certificate), and the private key from `L.head credentials` (the leaf). The returned `KeyHash` wraps this root fingerprint.

spec/modules/Simplex/Messaging/Transport/HTTP2.md

@@ -0,0 +1,13 @@
+# Simplex.Messaging.Transport.HTTP2
+
+> Bridges TLS transport to HTTP/2 configuration, buffer management, and body reading.
+
+**Source**: [`Transport/HTTP2.hs`](../../../../../src/Simplex/Messaging/Transport/HTTP2.hs)
+
+## allocHTTP2Config — manual buffer allocation
+
+`allocHTTP2Config` uses `mallocBytes` to allocate a write buffer (`Ptr Word8`) for the `http2` package's `Config`. The config bridges TLS to HTTP/2 by passing `cPut c` and `cGet c` from the `Transport` typeclass into the HTTP/2 config's `confSendAll` and `confReadN`.
+
+## http2TLSParams
+
+`http2TLSParams` uses `ciphersuite_strong_det` (from `Network.TLS.Extra`), distinct from the `ciphersuite_strong` used in `defaultSupportedParamsHTTPS`. This is the default `suportedTLSParams` in the HTTP/2 client configuration.