Bug report
General Information
- Cilium CLI version: v0.16.24 (the code below is unchanged on
main and in v0.19.4)
- Kubernetes: kubectl v1.31+, any apiserver version
- Platform: environment-independent — reproduces whenever an HTTP proxy / gateway /
load balancer that does not support SPDY sits in front of the kube-apiserver
Problem
cilium port-forward (e.g. cilium hubble port-forward) fails with
error upgrading connection: ... Upgrade request required when an HTTP proxy that
does not support SPDY is in front of the kube-apiserver.
The dialer in pkg/k8s/portforward/portforward.go (vendored from cilium/cilium)
is SPDY-only, with no WebSocket attempt and no fallback:
roundTripper, upgrader, err := spdy.RoundTripperFor(pf.config)
dialer := spdy.NewDialer(upgrader, &http.Client{Transport: roundTripper}, http.MethodPost, req.URL())
SPDY has been deprecated since 2015 (KEP-4006) and is no longer supported by many
proxies/gateways/load balancers. kubectl made WebSocket the default for port-forward
in v1.31, with SPDY as a fallback.
Why exec is unaffected
The exec path already uses the apimachinery WebSocket/SPDY fallback dialer
(createDialer, added in #37538). port-forward was never migrated.
#38988 later set exec to SPDY-primary with WebSocket fallback, due to hardcoded
idle-timeout issues in the upstream remotecommand WebSocket executor. That is
specific to remotecommand and does not apply to port-forward, which uses a separate
path (portforward.NewSPDYOverWebsocketDialer / the tunneling dialer).
Fix
Use the apimachinery fallback dialer in portforward.go with WebSocket as primary
and SPDY as fallback, matching kubectl's port-forward behaviour since v1.31:
spdyRoundTripper, upgrader, err := spdy.RoundTripperFor(pf.config)
if err != nil {
return nil, err
}
spdyDialer := spdy.NewDialer(upgrader, &http.Client{Transport: spdyRoundTripper}, http.MethodPost, req.URL())
tunnelingDialer, err := portforward.NewSPDYOverWebsocketDialer(req.URL(), pf.config)
if err != nil {
return nil, err
}
dialer := portforward.NewFallbackDialer(tunnelingDialer, spdyDialer, func(err error) bool {
return httpstream.IsUpgradeFailure(err) || httpstream.IsHTTPSProxyError(err)
})
SPDY remains an automatic fallback for older API servers, so the change is
backward-compatible.
How to reproduce
- Place an HTTP proxy that does not support SPDY in front of the kube-apiserver.
- Run
cilium hubble port-forward (or any cilium command that port-forwards).
- The SPDY upgrade is rejected and the command fails.
Bug report
General Information
mainand in v0.19.4)load balancer that does not support SPDY sits in front of the kube-apiserver
Problem
ciliumport-forward (e.g.cilium hubble port-forward) fails witherror upgrading connection: ... Upgrade request requiredwhen an HTTP proxy thatdoes not support SPDY is in front of the kube-apiserver.
The dialer in
pkg/k8s/portforward/portforward.go(vendored fromcilium/cilium)is SPDY-only, with no WebSocket attempt and no fallback:
SPDY has been deprecated since 2015 (KEP-4006) and is no longer supported by many
proxies/gateways/load balancers. kubectl made WebSocket the default for port-forward
in v1.31, with SPDY as a fallback.
Why exec is unaffected
The exec path already uses the apimachinery WebSocket/SPDY fallback dialer
(
createDialer, added in #37538). port-forward was never migrated.#38988 later set exec to SPDY-primary with WebSocket fallback, due to hardcoded
idle-timeout issues in the upstream remotecommand WebSocket executor. That is
specific to remotecommand and does not apply to port-forward, which uses a separate
path (
portforward.NewSPDYOverWebsocketDialer/ the tunneling dialer).Fix
Use the apimachinery fallback dialer in
portforward.gowith WebSocket as primaryand SPDY as fallback, matching kubectl's port-forward behaviour since v1.31:
SPDY remains an automatic fallback for older API servers, so the change is
backward-compatible.
How to reproduce
cilium hubble port-forward(or anyciliumcommand that port-forwards).