transport: optimize heap allocations in ready reader and update syscall connection handling

[arjan-bal]

This PR eliminates per-call heap allocations in nonBlockingReader.ReadOnReady().

Previously, calling RawConn.Read with an inline closure caused captured variables (and the closure itself) to escape to the heap. To resolve this, we moved the closure's required state and return values into fields on the nonBlockingReader struct. The state is set before execution, and the results are read afterward. Because the closure now only relies on the receiver's fields, we instantiate it exactly once during nonBlockingReader construction and reuse it, completely avoiding allocations on the hot path.

Additionally, this change updates the types for which non-blocking reads are performed to the following:

• Unwrapped types created by net.Dial. This avoid reading encrypted data from credentials.syscallConns.
• Types that already implement the ReadyReader interface themselves to support encrypted connections.

These changes are a prerequisite for #9032.

Benchmarks

#9032 uses nonBlockingReader instead of standard net.Conn.Read and confirms zero increase in heap allocations for streaming RPCs.

RELEASE NOTES: N/A

[codecov]

Codecov Report

❌ Patch coverage is 51.11111% with 22 lines in your changes missing coverage. Please review.
✅ Project coverage is 82.78%. Comparing base (a21508e) to head (138ddfd).
⚠️ Report is 13 commits behind head on master.

Files with missing lines	Patch %	Lines
internal/transport/readyreader/ready_reader.go	51.11%	11 Missing and 11 partials ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##           master    #9035      +/-   ##
==========================================
- Coverage   83.06%   82.78%   -0.28%     
==========================================
  Files         413      413              
  Lines       33135    33404     +269     
==========================================
+ Hits        27523    27655     +132     
- Misses       4199     4258      +59     
- Partials     1413     1491      +78     

Files with missing lines	Coverage Δ
internal/transport/readyreader/ready_reader.go	52.45% <51.11%> (-24.69%)	⬇️

... and 34 files with indirect coverage changes

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

[easwars]

@arjan-bal : Could the failing test in CI be related to changes here? I haven't seen that test flake anytime recently.

[arjan-bal]

@arjan-bal : Could the failing test in CI be related to changes here? I haven't seen that test flake anytime recently.

The test was reported in #8993 and a fix was made. I suspect the fix was incomplete so I've re-opened the issue and kept is assigned to the test author.

I don't think the failure is related to the changes in this PR, as the ReadyReader code is currently only used with ALTS. The changes to integrate it into the HTTP/2 framer will happen in multiple follow-up PRs.

[arjan-bal]

/gemini review

[gemini-code-assist]

Code Review

This pull request optimizes the nonBlockingReader by moving per-call state into the struct to avoid heap allocations and refactors the New function to use a specialized newNonBlockingReader constructor. The review feedback highlights that these optimizations introduce thread-safety concerns that need to be documented or addressed, and suggests resetting the internal state after each read to ensure that references can be properly garbage collected.

[gemini-code-assist]

The nonBlockingReader struct is no longer thread-safe because it uses shared fields (state) to store per-call request and response parameters to avoid heap allocations. While this is an effective optimization for gRPC's typical single-threaded reader loop, it is a regression in safety compared to the previous implementation. If ReadOnReady is called concurrently from multiple goroutines, it will result in data races and corrupted state. Please document this limitation or ensure that concurrent access is not possible.

[arjan-bal]

By default, most readers and writers are not thread-safe. They maintain internal state without mutex protection, requiring callers to handle synchronization themselves. Because lack of thread safety is the standard expectation, only implementations that are thread-safe need to document their behavior. For example, none of the exported types in the bufio package mention thread safety: https://pkg.go.dev/bufio#Reader

[Pranjali-2501]

The test was reported in #8993 and a fix was made. I suspect the fix was incomplete so I've re-opened the issue and kept is assigned to the test author.

Test reported in #8993 is fixed and not flaking any more.

Test Test/ClientStreaming_ClientCallRecvMsgTwice flaking on your PR is different and is Flaky too.

[easwars]

Nit: Handle the error case to reduce the indent in the non-error case.

[arjan-bal]

Done.