HTTP/2 proxy has slower speeds compared to HTTP/1.1

[vanchaxy]

I'm exposing LibreSpeed using an Envoy gateway, and speed test results are slower with HTTP/2 and RTT ~55ms. Both gateways are without any settings changes except for externalTrafficPolicy: Cluster.

LibreSpeed is exposed via two gateways: one local (RTT ~2ms) and one remote via tunnel (RTT ~55ms). These are the results (download/upload in mbps):

• Local Envoy gateway, HTTP/1.1: 1000/1000
• Local Envoy gateway, HTTP/2: 950/950
• Remote load balancer (direct to service without proxy), HTTP/1.1: 700/700 - the maximum possible due to the tunnel bandwidth
• Remote Envoy gateway, HTTP/1.1: 700/700
• Remote Envoy gateway, HTTP/2: 380/20 - very slow upload, may be related to this issue: Slow proxy when request has large payloads envoy#35732
• Remote Envoy gateway with ClientTrafficPolicy, HTTP/2: 380/550

The last test was with ClientTrafficPolicy using the following specs:

http2:
  initialConnectionWindowSize: 64Mi
  initialStreamWindowSize: 64Mi

HTTP/2 speed is consistently slower than HTTP/1.1 with default settings (especially upload), and even with tuning, it's slower than HTTP/1.1, which works at maximum speed.

Is that expected for the gateway to be much slower using HTTP/2? Is there some tuning needed that I'm missing?

[arkodg]

you'll need to also tune http in BackendTrafficPolicy to a value closer to the bandwidth delay product linked to backend

[vanchaxy]

Thanks for the answer! I assume you mean the http2 spec field. What if the backend is HTTP/1.1? Is there anything to tune in that case, or is the slower speed a limitation of proxying HTTP/2 to HTTP/1.1?

[arkodg]

for all http, you could get better performance if you tune connection buffers (CTP and BTP) as well, but bump them wisely, they multiple with the number of connections, and will increase proxy memory usage

[arkodg]

yes its http2*

[vigen-b]

Why do we need to configure http2 in the BackendTrafficPolicy if the backend is HTTP/1.1?