MAJOR: mux_quic: support stream elasticity during connection lifetime

qcc_release_remote_stream() is called each time a remote stream is
closed. Flow control accounting is updated and when necessary, a
MAX_STREAMS_BIDI frame is prepared to allow the peer to initiate new
streams.

This patch extends stream elasticity features with the QUIC bidirection
stream flow control mechanism. The announced value can now be possibly
reduced depending on conn_calc_max_streams().

The first step is to decrement closed streams from the global committed
extra streams total. This must be performed conn_calc_max_streams() to
ensure the calculation will be valid.

Then, there is two cases depending on conn_calc_max_streams() result. If
the value is less than the peer still remaining stream window, nothing
more is performed. If the opposite case, flow control must be increased
and a MAX_STREAMS_BIDI frame is prepared, with the value adjusted to not
exceed the stream elasticity limit. Global extra streams total is then
finally incremented.

This calcul also ensures that when all streams are closed, global extra
streams accounting operations are decremented by 1, as a connection
always has access to one stream which is excluded from the global total.

Note that if stream elasticity is not active, flow control increases
principle is unchanged and remains statically performed.

This patch is labelled as major as it complexifies bidirectional stream
flow control mechanisme. This is a sensitive operation as there is a
risk of connection freeze if flow control updates are inadvertently
skipped.

Author: a-denoyelle

doc/configuration.txt

@@ -5727,9 +5727,15 @@ tune.streams-elasticity <number>
   use lower values (120 to 200) to support 1.2 to 2 streams per connection on
   average at full load.
 
-  There is a limitation for QUIC listeners with enabled 0-RTT. In this case,
-  the initial value advertised to the peer will ignore stream elasticity and
-  instead rely solely on the "tune.quic.stream.max-concurrent" setting.
+  Contrary to HTTP/2, QUIC is capable to dynamically adjust the number of
+  concurrent streams during the connection lifetime. However, QUIC flow control
+  is stricter than HTTP/2, thus it is preferable when using it to specify
+  values big enough to prevent extra latency on the connection. There is also a
+  limitation for QUIC listeners with enabled 0-RTT. In this case, the initial
+  value advertised to the peer will ignore stream elasticity and instead rely
+  solely on the "tune.quic.stream.max-concurrent" setting. However, the stream
+  elasticity principle will still be effective past this initial annoucement
+  during the connection lifetime.
 
   Monitoring the total number of active streams on backends, including queues,
   provides a practical indicator of a sustainable target load and helps avoid

src/mux_quic.c

@@ -2467,6 +2467,7 @@ int qcc_recv_stop_sending(struct qcc *qcc, uint64_t id, uint64_t err)
 static int qcc_release_remote_stream(struct qcc *qcc, uint64_t id)
 {
 	struct quic_frame *frm;
+	uint64_t conn_max, rem, non_extra, inc;
 
 	TRACE_ENTER(QMUX_EV_QCS_END, qcc->conn);
 
@@ -2485,6 +2486,40 @@ static int qcc_release_remote_stream(struct qcc *qcc, uint64_t id)
 		 */
 		if (qcc->lfctl.cl_bidi_r > qcc->lfctl.ms_bidi_rel / 2 ||
 		    qcc->lfctl.cl_bidi_r + qcc->lfctl.ms_bidi == max) {
+
+			BUG_ON(qcc->lfctl.ms_bidi_rel < qcc->lfctl.cl_bidi_r);
+			rem = qcc->lfctl.ms_bidi_rel - qcc->lfctl.cl_bidi_r;
+			/* if every streams are closed, decrement extra stream accounting by 1 */
+			non_extra = !rem ? 1 : 0;
+
+			if (!(qcc->flags & QC_CF_IS_BACK) && global.tune.streams_elasticity) {
+				/* If stream elasticity is active, first decrement closed from extra streams. */
+				if (qcc->lfctl.ms_bidi_rel > 1) {
+					_HA_ATOMIC_SUB(&tg_ctx->committed_extra_streams,
+					               qcc->lfctl.cl_bidi_r - non_extra);
+				}
+
+				/* Now calculate the available streams. */
+				conn_max = conn_calc_max_streams(qcc->lfctl.ms_bidi_init);
+				if (conn_max <= rem) {
+					/* More streams already consumed than currently allowed,
+					 * keep the current flow control limit.
+					 */
+					qcc->lfctl.ms_bidi_rel = rem;
+					qcc->lfctl.cl_bidi_r = 0;
+					goto out;
+				}
+
+				/* Update flow control limit up to the allowed elasticity limit. */
+				inc = conn_max - rem;
+				_HA_ATOMIC_ADD(&tg_ctx->committed_extra_streams, inc - non_extra);
+				qcc->lfctl.ms_bidi_rel = rem + inc;
+			}
+			else {
+				/* Stream elasticity not active, flow control increase remains static. */
+				inc = qcc->lfctl.cl_bidi_r;
+			}
+
 			TRACE_DATA("increase max stream limit with MAX_STREAMS_BIDI", QMUX_EV_QCC_SEND, qcc->conn);
 			frm = qc_frm_alloc(QUIC_FT_MAX_STREAMS_BIDI);
 			if (!frm) {
@@ -2493,12 +2528,11 @@ static int qcc_release_remote_stream(struct qcc *qcc, uint64_t id)
 				goto err;
 			}
 
-			frm->max_streams_bidi.max_streams = qcc->lfctl.ms_bidi +
-			                                    qcc->lfctl.cl_bidi_r;
+			frm->max_streams_bidi.max_streams = qcc->lfctl.ms_bidi + inc;
 			LIST_APPEND(&qcc->lfctl.frms, &frm->list);
 			tasklet_wakeup(qcc->wait_event.tasklet);
 
-			qcc->lfctl.ms_bidi += qcc->lfctl.cl_bidi_r;
+			qcc->lfctl.ms_bidi += inc;
 			qcc->lfctl.cl_bidi_r = 0;
 		}
 	}