Stop daemon response flushing from defeating mod_ratelimit.

When proxying a response body back from a daemon process, mod_wsgi forced
a flush both whenever the daemon socket momentarily ran dry and after each
response-buffer-size worth of data. Both forced the mod_ratelimit
RATE_LIMIT filter to emit short writes, so depending on the Apache version
a configured rate limit was either largely ineffective or applied far too
aggressively, never honoured accurately for a daemon mode response.

mod_wsgi requires Apache 2.4, whose core output filter already bounds how
much response data is buffered in the child processes, so these forced
flushes are not needed to limit memory use and have been removed. Response
data is now passed straight on, so a downstream pacing or batching filter
receives full-size writes and works correctly.

A new response-flush-delay option (default 5 milliseconds) controls how
long mod_wsgi waits for more data before flushing when the daemon socket
runs dry, so a transient stall during an active transfer no longer
triggers a flush while a genuinely idle application still has its partial
output flushed within the delay. Setting it to 0 flushes on any stall and
is not recommended when a pacing filter is in use.

The response-buffer-size option is repurposed as a coarse runaway guard
that forces a flush only once that many bytes have been passed downstream
without one, bounding a downstream filter that buffers without draining.
Its default is raised from 65536 bytes to 8388608 bytes accordingly.

Author: GrahamDumpleton

docs/configuration-directives/WSGIDaemonProcess.rst

@@ -769,20 +769,69 @@ Options which can be supplied to the ``WSGIDaemonProcess`` directive are:
 .. _response-buffer-size:
 
 **response-buffer-size=nnn**
-    Defines the maximum number of bytes that will be buffered for a
-    response in the Apache child processes when proxying the response body
-    from the WSGI application. The default size is 65536 bytes. Be careful
-    increasing this to provide extra buffering of responses as it
-    contributes to the runtime memory size of the Apache child processes.
+    Acts as a coarse upper bound, in bytes, on how much response content is
+    passed down the output filter chain without a flush when proxying a
+    response body from the WSGI application. If this many bytes are passed
+    without a flush occurring for another reason, a flush is forced. The
+    default is 8388608 bytes (8 MB), and if specified the value must be at
+    least 65536 bytes.
+
+    Its only purpose is to bound a downstream output filter that buffers
+    response data without draining it, capping such buffering to roughly
+    this many bytes per request. The Apache core output filter already
+    bounds normal in-memory buffering of the response in the Apache child
+    processes, so the default is set high and the guard does not affect
+    normal operation. Setting it low forces frequent flushes that can
+    interfere with a downstream pacing or batching output filter such as
+    ``mod_ratelimit`` (see ``response-flush-delay``), so it should only be
+    reduced if you specifically need to cap such a filter.
 
 .. _response-socket-timeout:
 
 **response-socket-timeout=nnn**
     Defines the maximum number of seconds allowed to pass before timing out
-    on a write operation back to the HTTP client when the response buffer
-    has filled and data is being forcibly flushed. Defaults to 0 seconds
-    indicating that it will default to the value of the ``socket-timeout``
-    option.
+    on a write operation back to the HTTP client when transferring the
+    response body. Defaults to 0 seconds indicating that it will default to
+    the value of the ``socket-timeout`` option.
+
+.. _response-flush-delay:
+
+**response-flush-delay=nnn**
+    Defines, in **milliseconds**, how long mod_wsgi will wait for further
+    response data to arrive from the daemon process before flushing the
+    data already read out to the HTTP client. Defaults to 5 milliseconds.
+
+    When proxying a response back from a daemon process, mod_wsgi reads the
+    data as it becomes available and would otherwise flush to the client
+    every time the daemon socket momentarily ran dry. During a bulk
+    transfer that emptying is usually just an artefact of timing, and
+    flushing on it defeats a downstream output filter that paces or batches
+    data, most notably ``mod_ratelimit`` (whose ``RATE_LIMIT`` filter is
+    forced to emit a short write on every flush and so throttles the
+    response far below the configured rate). Waiting a brief period for
+    more data lets mod_wsgi coalesce it into larger writes so such a filter
+    can do its job, while a genuinely idle application still has its partial
+    output flushed within the delay so streaming responses remain
+    responsive.
+
+    The default of 5 milliseconds is comfortably longer than the time it
+    takes the local daemon process to produce more data during an active
+    transfer, so it does not affect throughput, while bounding any extra
+    latency added to a paused streaming response to that same small value.
+
+    Setting the value to 0 disables the wait, so data is flushed on any
+    momentary stall. This is generally not recommended. In particular, do
+    not set it to 0 when a downstream pacing or batching output filter such
+    as ``mod_ratelimit`` is in use: doing so reintroduces a flush on every
+    momentary stall, which prevents such a filter from accumulating
+    full-size writes and can throttle responses far below the configured
+    rate. A value of 0 is only appropriate for latency-sensitive streaming
+    where no such filter is present and even a few milliseconds of flush
+    delay must be avoided.
+
+    Note that, unlike the various ``*-timeout`` options which are expressed
+    in seconds, this option is expressed in milliseconds because the useful
+    values are far smaller than one second.
 
 .. _server-metrics:
 

docs/release-notes/version-6.0.2.rst

@@ -2,6 +2,52 @@
 Version 6.0.2
 =============
 
+Features Changed
+----------------
+
+* The way mod_wsgi proxies a response body back from a daemon process to
+  the HTTP client has been reworked so that it no longer interferes with
+  downstream output filters that pace or batch data, most notably
+  ``mod_ratelimit``. Previously mod_wsgi forced a flush both every time the
+  daemon socket momentarily ran dry and after each ``response-buffer-size``
+  worth of data had been passed on. Both kinds of flush forced the
+  ``mod_ratelimit`` ``RATE_LIMIT`` filter to emit a short write and so
+  throttled daemon mode responses far below the configured rate. As
+  mod_wsgi requires Apache 2.4, whose core output filter already bounds how
+  much response data is buffered in the Apache child processes, these
+  forced flushes are no longer needed to limit memory use and have been
+  removed. mod_wsgi now passes response data straight on, so a pacing
+  filter receives full-size writes and works correctly, while a genuinely
+  idle application still has its partial output flushed promptly so
+  streaming responses remain responsive.
+
+  The visible effect of the previous behaviour varied with the version of
+  Apache in use. With some versions of mod_ratelimit the forced flushes
+  effectively let the response through at close to full speed, so a
+  configured rate limit had little effect; with others they caused the
+  rate limit to be applied far too aggressively, throttling the response
+  well below the configured rate. In neither case could the rate limit be
+  honoured accurately for a daemon mode response, which it now is.
+
+  Two options on the :doc:`../configuration-directives/WSGIDaemonProcess`
+  directive control this behaviour:
+
+  - The new ``response-flush-delay`` option (default 5 milliseconds) sets
+    how long mod_wsgi waits for more response data before flushing when the
+    daemon socket runs dry, so a transient stall during an active transfer
+    does not trigger a flush while a genuinely paused application still has
+    its output flushed within that delay. Setting it to 0 flushes on any
+    stall and is not recommended when a pacing filter such as
+    ``mod_ratelimit`` is in use, as it can throttle responses far below the
+    configured rate.
+
+  - The ``response-buffer-size`` option has changed meaning. It is now a
+    coarse runaway guard that forces a flush only once this many bytes have
+    been passed downstream without one, in order to bound a downstream
+    filter that buffers without draining. Its default has been raised from
+    65536 bytes to 8388608 bytes (8 MB) accordingly; normal bounding of
+    response memory use is handled by the Apache core output filter.
+
 Bugs Fixed
 ----------
 

src/express/apache.py

@@ -243,6 +243,7 @@
    header-buffer-size=%(header_buffer_size)s \\
    response-buffer-size=%(response_buffer_size)s \\
    response-socket-timeout=%(response_socket_timeout)s \\
+   response-flush-delay=%(response_flush_delay)s \\
    server-metrics=%(server_metrics_flag)s%(daemon_switch_interval_option)s
 </IfDefine>
 <IfDefine !MOD_WSGI_MULTIPROCESS>
@@ -273,6 +274,7 @@
    receive-buffer-size=%(receive_buffer_size)s \\
    response-buffer-size=%(response_buffer_size)s \\
    response-socket-timeout=%(response_socket_timeout)s \\
+   response-flush-delay=%(response_flush_delay)s \\
    server-metrics=%(server_metrics_flag)s%(daemon_switch_interval_option)s
 </IfDefine>
 </IfDefine>

src/express/options.py

@@ -395,18 +395,32 @@ def add_option(platforms, *args, **kwargs):
         'indicating internal default of 32768 bytes is used.')
 
 add_option('unix', '--response-buffer-size', type='int', default=0,
-        metavar='NUMBER', help='Maximum amount of response content '
-        'that will be allowed to be buffered in the Apache child '
-        'worker process when proxying the response from a daemon '
-        'process. Defaults to 0, indicating internal default of '
-        '65536 bytes is used.')
+        metavar='NUMBER', help='Coarse upper bound, in bytes, on how '
+        'much response content is passed down the output filter chain '
+        'without a flush when proxying the response from a daemon '
+        'process. Only bounds a downstream filter that buffers without '
+        'draining; the Apache core output filter handles normal memory '
+        'bounding. Defaults to 0, indicating the internal default of '
+        '8388608 bytes (8 MB) is used. Setting it low can interfere with '
+        'a downstream pacing filter such as mod_ratelimit.')
 
 add_option('unix', '--response-socket-timeout', type='int', default=0,
         metavar='SECONDS', help='Maximum number of seconds allowed '
         'to pass before timing out on a write operation back to the '
-        'HTTP client when the response buffer has filled and data is '
-        'being forcibly flushed. Defaults to 0 seconds indicating that '
-        'it will default to the value of the \'socket-timeout\' option.')
+        'HTTP client when transferring the response body. Defaults to 0 '
+        'seconds indicating that it will default to the value of the '
+        '\'socket-timeout\' option.')
+
+add_option('unix', '--response-flush-delay', type='int', default=5,
+        metavar='MILLISECONDS', help='Number of milliseconds mod_wsgi '
+        'will wait for further response data from a daemon process before '
+        'flushing what it has to the HTTP client. Waiting briefly lets a '
+        'downstream output filter such as mod_ratelimit pace or batch the '
+        'response correctly instead of being forced to emit a short write '
+        'on every momentary stall. Defaults to 5 milliseconds. Setting it '
+        'to 0 flushes on any stall and is not recommended when a pacing '
+        'filter such as mod_ratelimit is in use, as it can throttle '
+        'responses far below the configured rate.')
 
 add_option('all', '--enable-sendfile', action='store_true',
         default=False, help='Flag indicating whether sendfile() support '

src/server/wsgi_daemon.c

@@ -149,6 +149,8 @@ const char *wsgi_add_daemon_process(cmd_parms *cmd, void *mconfig,
 
     int response_socket_timeout = 0;
 
+    int response_flush_delay = -1;
+
     const char *script_user = NULL;
     const char *script_group = NULL;
 
@@ -510,6 +512,17 @@ const char *wsgi_add_daemon_process(cmd_parms *cmd, void *mconfig,
             if (response_socket_timeout < 0)
                 return "Invalid response socket timeout for WSGI daemon process.";
         }
+        else if (!strcmp(option, "response-flush-delay"))
+        {
+            if (!*value)
+                return "Invalid response flush delay for WSGI daemon process.";
+
+            /* Value is in milliseconds; 0 disables the delay. */
+
+            response_flush_delay = atoi(value);
+            if (response_flush_delay < 0)
+                return "Invalid response flush delay for WSGI daemon process.";
+        }
         else if (!strcmp(option, "socket-user"))
         {
             uid_t socket_uid;
@@ -741,6 +754,13 @@ const char *wsgi_add_daemon_process(cmd_parms *cmd, void *mconfig,
 
     entry->response_socket_timeout = apr_time_from_sec(response_socket_timeout);
 
+    /* Default the response flush delay to 5 milliseconds. */
+
+    if (response_flush_delay < 0)
+        response_flush_delay = 5;
+
+    entry->response_flush_delay = apr_time_from_msec(response_flush_delay);
+
     entry->script_user = script_user;
     entry->script_group = script_group;
 

src/server/wsgi_daemon.h

@@ -127,6 +127,7 @@ typedef struct
     int header_buffer_size;
     int response_buffer_size;
     apr_time_t response_socket_timeout;
+    apr_time_t response_flush_delay;
     const char *script_user;
     const char *script_group;
     int cpu_time_limit;