HBASE-30092 Avoid stringifying protobuf RPC parameters in task snapshots#8489
Open
Sigma-Ma wants to merge 1 commit into
Open
HBASE-30092 Avoid stringifying protobuf RPC parameters in task snapshots#8489Sigma-Ma wants to merge 1 commit into
Sigma-Ma wants to merge 1 commit into
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
JIRA: https://issues.apache.org/jira/browse/HBASE-30092
What changes were proposed in this pull request?
This change avoids calling
Message.toString()whenMonitoredRPCHandlerImplgenerates RPC monitoring information.For protobuf parameters, HBase now records only the protobuf message type, such as
ScanRequest. The existingpacketlengthfield continues to report the serialized request size.Why are the changes needed?
HBASE-30092 reports that
MonitoredRPCHandlerImpl.generateCallInfoMap()can cause an out-of-memory error while generating RegionServer heartbeat information for a large in-flight RPC request.Calling
toString()on a protobuf recursively materializes the complete TextFormat representation. For large RPC requests, this creates a significant additional allocation in the monitoring path and can exhaust the RegionServer heap.Recording only the protobuf message type keeps monitoring allocation independent of the request payload size while preserving the RPC method and serialized packet length.
How was this patch tested?
Added a regression test to
TestTaskMonitor, covering:ScanRequest.The focused unit test can be run with: