[Detail Bug] HTTP/2 client: cancelled request during header send leaks stream state and prevents connection from going idle

[detail-app]

Detail Bug Report

https://app.detail.dev/org_89d327b3-b883-4365-b6a3-46b6701342a9/bugs/bug_508ac475-94e1-4b91-8a92-a427e7b209f5

Introduced in #1 by @quettabit on Apr 7, 2026

Summary

• Context: This project requires Python >=3.11 (pyproject.toml:8), where asyncio.CancelledError is a subclass of BaseException, not Exception. The cleanup code in Connection.send_headers uses except Exception which does NOT catch CancelledError.
• Bug: When a request task is cancelled during send_headers, the CancelledError propagates out of the exception handler, skipping cleanup of _streams. The caller's finally block sees stream_id=None, so release_stream can't clean up the leaked state.
• Actual vs. expected: When cancellation occurs during send_headers, state should be cleaned up from _streams, but currently the state leaks permanently.
• Impact: Leaked stream state permanently inflates open_stream_count. The connection never becomes idle, the reaper won't close it, and the pool creates new connections to compensate. This causes connection pool bloat and eventual resource exhaustion.

Code with Bug

async def send_headers(
    self,
    state: _StreamState,
    headers: list[tuple[str, str]],
    end_stream: bool = False,
) -> int:
    assert self._h2 is not None
    async with self._write_lock:
        if state.error is not None:
            raise state.error
        pending_state = self._pending_streams.pop(id(state), None)  # Remove from pending
        if pending_state is None:
            raise ProtocolError("Stream reservation missing")
        stream_id = self._h2.get_next_available_stream_id()
        self._streams[stream_id] = state
        try:
            self._h2.send_headers(stream_id, headers, end_stream=end_stream)
            await self._flush_h2_data_and_drain()                  # can be cancelled
        except Exception:                                          # <-- BUG 🔴 CancelledError not caught; cleanup skipped
            self._streams.pop(stream_id, None)
            raise
        return stream_id

Explanation

• send_headers() moves a stream from _pending_streams to _streams before awaiting _flush_h2_data_and_drain().
• If the task is cancelled while awaiting _flush_h2_data_and_drain() (e.g., asyncio.timeout, TaskGroup cancellation, user cancellation), Python raises asyncio.CancelledError.
• In Python 3.11+, CancelledError inherits from BaseException, so except Exception: does not run and the rollback (_streams.pop(stream_id, None)) is skipped.
• The caller never receives a stream_id (it remains None), so the caller’s release_stream(None, state) only removes the state from _pending_streams and cannot remove the already-leaked entry from _streams.
• Closing the connection does not remove entries from _streams; _fail_all_streams() only signals state futures/events, so the leaked dict entry persists.

Codebase Inconsistency

Other cleanup paths explicitly include cancellation in their handlers, suggesting this was an oversight in send_headers:

• except (asyncio.CancelledError, Exception): in streaming request cleanup
• except (asyncio.CancelledError, Exception): in close() recv_task cleanup
• except (asyncio.CancelledError, Exception): in _drain_body

Recommended Fix

Change the handler in Connection.send_headers to catch cancellation so rollback always runs, e.g.:

except BaseException:
    self._streams.pop(stream_id, None)
    raise

History

This bug was introduced in commit 3dc9795.