ltc: fix NodeRPC::Send UAF (thread-race on shared HTTP client) — .157 crash-loop ship-blocker - #270
Merged
Merged
Conversation
NodeRPC holds a single shared m_http_request + m_stream and Send() drove them with no synchronization. The asio thread_pool worker path (clean_tracker -> ShareTracker::think -> score -> score-callback -> NodeRPC::getblockheader -> Send) issues a blocking RPC concurrently with the main thread's own RPC traffic on the same client. ASan (firewall-isolated sibling, BuildId 9a12b6bc) caught the resulting heap-use-after-free: while one thread is mid http::write serializing the request fields (read_iovec), the other enters Send -> prepare_payload -> set_content_length_impl -> delete_element and frees the Content-Length basic_fields element the first thread is still reading. In the non-ASan build this surfaces as the .157 SEGV-inside-malloc crash-loop during reorg/clean churn (the heap corruption was the downstream symptom). Guard the full write+read cycle with a per-client mutex. A single HTTP connection cannot interleave two request/response pairs anyway, so this adds no stall beyond what correctness already requires. Follow-up (separate change): keep think()/score() off the synchronous RPC-on-pool-thread path entirely (cache header height instead of a live getblockheader in the score callback).
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Ship-blocker: .157 v36-crossing crash-loop, root-caused
The .157 soak SEGV-loop (~every 25-50 min during reorg/clean churn, inside malloc) is not a sharechain reorg-prune bug as earlier hypothesized. A firewall-isolated ASan sibling (BuildId
9a12b6bc) captured a definitive heap-use-after-free:Root cause
ltc::coin::NodeRPCholds a single sharedm_http_request+m_streamandSend()had no mutex. The asio pool worker (clean_tracker -> think -> score fires a blocking getblockheader) and the main thread both driveSend()on the same client. One thread mid-http::writewhile the other re-runsprepare_payload()frees/reallocs the header element under it -> UAF. In the non-ASan build this is the SEGV-inside-malloc crash-loop (heap corruption is the downstream symptom). Crossing churn just raises clean_tracker frequency, widening the window — which is why it looked crossing-specific.Fix
Serialize the full write+read cycle in
Send()under a per-clientstd::mutex. A single HTTP connection cannot interleave two request/response pairs anyway, so the lock adds no stall beyond what correctness already requires. +7 lines, no behavior change on the single-threaded path.Verification
c2pooltarget, exit 0).Follow-up (separate PR, non-blocking on cutover)
Get
think()/score()off the synchronous-RPC-on-pool-thread path: cache header height instead of a livegetblockheaderin the score callback. Also consider guarding the reconnect-timer path againstSend()(rarer race onm_stream).