add section: buffer and pad msgs to protect against traffic analysis

Author: SomberNight

docs/protocol-basics.rst

@@ -32,6 +32,9 @@ within strings, so no confusion is possible there.  However it does permit newli
 extraneous whitespace between elements; client and server MUST NOT use newlines in such a
 way.
 
+Messages SHOULD be :ref:`padded <padding_messages>` to bucketed lengths,
+and buffered (to introduce delays) to protect against traffic analysis.
+
 If using JSON RPC 2.0's feature of parameter passing by name, the
 names shown in the description of the method or notification in
 question MUST be used.
@@ -258,3 +261,117 @@ and confirm the returned roots match.
   implementation would require hashing approximately 88MB of data to
   provide a single merkle proof.  ElectrumX implements an optimization
   such that it hashes only approximately 180KB of data per proof.
+
+
+.. _padding_messages:
+
+Traffic analysis
+----------------
+
+The goal is to defend against a passive network Man-in-the-Middle, such as an ISP
+or a Tor exit node, observing the encrypted TLS stream, and making educated guesses
+of the message contents based on TCP packet flow: timing, direction, and sizes of TCP packets.
+
+.. note:: When using raw cleartext TCP as transport for the JSON-RPC payloads, without encryption,
+  a passive network observer can see all the plaintext messages.
+
+As a generic mitigation, implementations (both client and server) SHOULD
+
+- pad messages to bucketed lengths (e.g. powers of 2, with a min size), and
+
+- introduce small timing delays, ideally by buffering messages.
+
+We can fully backwards-compatibly add padding to the JSON-RPC messages by adding extra
+whitespaces inside the JSON objects in a way that parsers ignore.
+This can be done at any protocol version.
+
+For example, instead of sending::
+
+  {"jsonrpc":"2.0","method":"server.version","id":0,"params":["electrum/4.5.8","1.4"]}\n
+
+
+the client could send::
+
+  {"jsonrpc":"2.0","method":"server.version","id":0,"params":["electrum/4.5.8","1.4"]           }\n
+
+
+For better results, both the client and the server SHOULD implement logic to pad the messages
+that they send. So that requests and responses (and notifications) SHOULD all be padded.
+This does not have to be rolled out simultaneously: it is ok for only a client to pad what
+they send and not the server (or the other way around),
+that just limits the effectiveness of the defense against traffic analysis.
+
+Note when the JSON-RPC messages are sent in the TLS stream, they are sometimes batched together.
+That is, a single TCP packet might contain multiple small JSON-RPC messages,
+e.g. if the client tries to send multiple messages in a short burst.
+Also, many protocol requests are <100 bytes, so it would be wasteful to pad all to e.g. 1 kbyte.
+
+To save bandwidth, instead of padding individual JSON-RPC messages,
+participants (the client and the server) COULD implement an application-level buffer,
+write the messages into that buffer, periodically empty the buffer into the TLS stream
+and only add the padding into e.g. the last JSON-RPC message when emptying the buffer.
+
+.. note:: Example implementation
+  in the `Electrum client <https://github.com/spesmilo/electrum/pull/9875>`_
+  and in the `electrumx server <https://github.com/spesmilo/electrumx/pull/301>`_:
+
+  Both the client and the server writes raw JSON-RPC protocol messages into a buffer,
+  which is then occasionally flushed to the wire. When it is flushed, padding is added
+  to round up the total length to 1 KB, or to the next power of 2.
+  The buffer is flushed if it reaches 1 KB, plus there is extra logic that periodically polls
+  if the oldest message in the buffer is older than 1 second, in which case it is also flushed.
+  The worst-case 1 second delay is a performance hit that might in cases be felt by the user.
+  This is a tradeoff to make the flow of packets harder to analyse for an observer.
+
+  Implementations COULD make the buffer size and the max time delay configurable.
+
+.. note:: Many protocol requests are <100 bytes. Contrast that with broadcasting a transaction,
+  which could potentially be several megabytes of data.
+  (max consensus-valid tx is 4 MB, times 2 for hex-encoding)
+  Hence padding to a constant size is not practical.
+  Instead it is recommended to pad to bucketed lengths, e.g. to powers of 2.
+
+The specific details of the size of the buffer, how often it is flushed, and how the padding
+is done is not specified by the protocol at the moment. Consequently, neither the client nor
+the server can enforce the other to pad and much less to delay messages.
+
+.. note:: Some server implementations do not deal with TLS at all,
+  they only implement the raw cleartext TCP protocol, and just recommend operators
+  to put a reverse proxy in front that does TLS termination.
+  Such protocol implementations (both client and server) nevertheless still
+  SHOULD implement all mentioned traffic analysis protections.
+  That way, if the operator tunnels the traffic over TLS externally,
+  the resulting stream meaningfully receives the protections.
+
+.. note:: An alternative approach to the buffer-based delaying and padding
+  could be to more aggressively use JSON-RPC batching to batch messages, and to use the
+  optional Record Padding of
+  `TLS 1.3 <https://www.rfc-editor.org/rfc/rfc8446#section-5.4>`_.
+  See `SSL_CONF_cmd RecordPadding` in openssl. However this only allows
+  padding to multiples of a constant, while above the recommendation was to pad to powers of 2.
+  The implementation might still want to delay messages a bit
+  to accumulate them into a larger JSON-RPC batch. Also note if users
+  (e.g. server operators) are expected to do their own TLS termination
+  (e.g. using a reverse proxy), configuring TLS RecordPadding would become an externality
+  for them to do, which they might forget.
+
+.. note:: Buffering the messages to introduce timing delays
+  and padding to ~bucketed sizes is a good baseline.
+  However even approximate timing and direction of TCP packets
+  can leak too much information in some scenarios.
+
+  To combat timing analysis, both the client and the server
+  COULD send dummy RPCs with a random timer, but more importantly at strategically selected events.
+  For example, when the client receives a new block header notification,
+  it COULD probabilistically send a random number of "server.ping" messages
+  with small random sleeps in-between.
+
+  Protocol version 1.7 extends "server.ping" so that either party can send it, and
+  that it can be sent either as a JSON-RPC "Request" or as a JSON-RPC "Notification".
+  If sent as a notification, the receiver is expected not to respond.
+
+  When the server sends a block header notification to the client,
+  it COULD also probabilistically send noise ("server.ping") notifications to the client,
+  perhaps conditioned on whether it will also send
+  :func:`blockchain.scriptpubkey.subscribe` notifications.
+  (so server could send noise if there are no status notifications to be sent)