network: propagate peer full close as RemoteClose for user-space io_handle when half-close is enabled#45198
Open
PetrMc wants to merge 1 commit into
Open
Conversation
…andle when half-close is enabled Signed-off-by: Petr McAllister <petr.mcallister@gmail.com>
|
Hi @PetrMc, welcome and thank you for your contribution. We will try to review your Pull Request as quickly as possible. In the meantime, please take a look at the contribution guidelines if you have not done so already. |
|
1.36 branch also needs to be fixed. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Commit Message
network: propagate peer full close as RemoteClose for user-space io_handle when half-close is enabled
Description
Bug
In
tcp_proxytunneling mode (HTTP CONNECT) routed through aninternal_listener, upstream tunneled streams are not reset when the downstream side fully closes. They remain pinned until the upstream idle timeout fires (envoy default 1 hour), accumulating across requests. This topology is used by Istio ambient mesh on the gateway side; the bug was originally reported and reproduced there (see istio/istio#60074).The downstream upstream side correctly detects the close at the user-space
IoHandlelevel (the peer io_handle is destroyed,peer_handle_is nulled,setEof()andonPeerDestroy()both fire). Buttcp_proxyenables half-close semantics on its downstream connection viaenableHalfClose(true), andConnectionImplis configured to ignore theClosedfile event in that mode (source/common/network/connection_impl.cc:215). A subsequent read returning EOF becomes a half-close, not aRemoteClose. The tcp_proxy filter keeps the upstream CONNECT stream open waiting for response data while downstream is fully gone.For real OS sockets this is fine: kernel signals like
EPIPEon write or RST detection indoReadproduce the properRemoteCloseevent. For user-spaceIoHandlepairs there is no kernel; the only signal of "peer fully gone" isonPeerDestroy()settingpeer_handle_to nullptr.Fix
Add a new
Network::IoHandle::wasPeerFullyClosed()method that:falseby default. Real OS socket implementations inherit this; behavior unchanged.IoHandleImploverrides to return a newpeer_fully_closed_bool, set insideonPeerDestroy().In
ConnectionImpl::onReadReady(), after the existing half-close translation, check whetherenable_half_close_ && result.end_stream_read_ && ioHandle().wasPeerFullyClosed(). If so, treat the event asRemoteCloseand setDetectedCloseType::RemoteReset, just like the RST handling above. tcp_proxy'sonDownstreamEventthen propagates this toHttpUpstream::onDownstreamEvent → resetEncoder → resetStream, producing the expectedRST_STREAMframe upstream.A peer
shutdown(WR)(legitimate half-close, peer keeps its io_handle alive) only triggerssetEof(), notonPeerDestroy().peer_fully_closed_stays false. Half-close semantics are preserved.Risk and behavior change
wasPeerFullyClosed()returns false; the new branch never fires.IoHandleusers with half-close disabled: zero change. Existing full-close path (!enable_half_close_ && end_stream_read_) already converts EOF to close.IoHandleusers with half-close enabled AND a legitimate half-close pattern: half-close still works. Only full peer destroy triggers the new path.IoHandleusers with half-close enabled AND full peer close (tcp_proxy withtunneling_configover internal_listener, possibly other patterns): the connection now firesRemoteCloseimmediately, releasing upstream tunneled streams. This is the fix.Guarded behind
envoy.reloadable_features.internal_listener_peer_destroyed_propagation, default on. Set to false to restore legacy behavior.Reproducing the bug
Set up a
tcp_proxylistener withtunneling_configwhose upstream is reached via aninternal_listener, with half-close enabled on the tcp_proxy downstream (the default). Open multiple downstream connections that get the upstream tunnel established, then close the downstream side. Observe upstream tunneled streams pin until the upstream idle timeout (default 1 hour) clears them.The unit tests in this PR exercise the connection-level half-close translation directly. A real-world reproduction using Istio ambient is documented at istio/istio#60074.
Risk Level
Low. New behavior is gated behind a runtime feature flag. Affects only the user-space
IoHandlehalf-close-enabled path. Real OS sockets unchanged. Legitimate half-close clients through internal_listener unchanged.Testing
Unit tests (
test/extensions/io_socket/user_space/io_handle_impl_test.cc): verifywasPeerFullyClosed()returns false by default, stays false after peershutdown(WR), becomes true after peerclose().Connection-level tests (
test/extensions/io_socket/user_space/connection_compatbility_test.cc):HalfCloseEnabledPeerFullClosePropagatesRemoteClose: with feature on, full peer close firesRemoteClose.HalfCloseEnabledPeerFullCloseLegacyBehavior: with feature off, legacy behavior preserved.HalfCloseEnabledPeerShutdownWritePreservesHalfClose: with feature on, peershutdown(WR)is still observed as half-close.Docs Changes
Release note in
changelogs/current/bug_fixes/tcp_proxy__fix-stream-leak-on-internal-listener-peer-full-close.rst.Release Notes
Added.
Platform Specific Features
N/A. Pure user-space path.
Runtime guard
envoy.reloadable_features.internal_listener_peer_destroyed_propagation