Add Sofa RPC instrumentation (Bolt, H2C, REST, Triple)#11135
Add Sofa RPC instrumentation (Bolt, H2C, REST, Triple)#11135gh-worker-dd-mergequeue-cf854d[bot] merged 10 commits intomasterfrom
Conversation
BenchmarksStartupParameters
See matching parameters
SummaryFound 0 performance improvements and 0 performance regressions! Performance is the same for 60 metrics, 11 unstable metrics. Startup time reports for petclinicgantt
title petclinic - global startup overhead: candidate=1.62.0-SNAPSHOT~17ebcd0a24, baseline=1.62.0-SNAPSHOT~95de525c87
dateFormat X
axisFormat %s
section tracing
Agent [baseline] (1.076 s) : 0, 1075865
Total [baseline] (11.105 s) : 0, 11104682
Agent [candidate] (1.063 s) : 0, 1063086
Total [candidate] (11.078 s) : 0, 11077775
section appsec
Agent [baseline] (1.249 s) : 0, 1249264
Total [baseline] (11.016 s) : 0, 11016234
Agent [candidate] (1.261 s) : 0, 1261220
Total [candidate] (11.142 s) : 0, 11141579
section iast
Agent [baseline] (1.235 s) : 0, 1234725
Total [baseline] (11.356 s) : 0, 11355815
Agent [candidate] (1.223 s) : 0, 1223385
Total [candidate] (11.363 s) : 0, 11363001
section profiling
Agent [baseline] (1.186 s) : 0, 1185869
Total [baseline] (11.116 s) : 0, 11115524
Agent [candidate] (1.185 s) : 0, 1184668
Total [candidate] (11.058 s) : 0, 11058045
gantt
title petclinic - break down per module: candidate=1.62.0-SNAPSHOT~17ebcd0a24, baseline=1.62.0-SNAPSHOT~95de525c87
dateFormat X
axisFormat %s
section tracing
crashtracking [baseline] (1.26 ms) : 0, 1260
crashtracking [candidate] (1.223 ms) : 0, 1223
BytebuddyAgent [baseline] (645.735 ms) : 0, 645735
BytebuddyAgent [candidate] (637.925 ms) : 0, 637925
AgentMeter [baseline] (29.825 ms) : 0, 29825
AgentMeter [candidate] (29.467 ms) : 0, 29467
GlobalTracer [baseline] (250.663 ms) : 0, 250663
GlobalTracer [candidate] (250.463 ms) : 0, 250463
AppSec [baseline] (32.791 ms) : 0, 32791
AppSec [candidate] (32.599 ms) : 0, 32599
Debugger [baseline] (60.609 ms) : 0, 60609
Debugger [candidate] (59.792 ms) : 0, 59792
Remote Config [baseline] (602.344 µs) : 0, 602
Remote Config [candidate] (590.203 µs) : 0, 590
Telemetry [baseline] (8.161 ms) : 0, 8161
Telemetry [candidate] (8.032 ms) : 0, 8032
Flare Poller [baseline] (9.819 ms) : 0, 9819
Flare Poller [candidate] (6.716 ms) : 0, 6716
section appsec
crashtracking [baseline] (1.231 ms) : 0, 1231
crashtracking [candidate] (1.236 ms) : 0, 1236
BytebuddyAgent [baseline] (662.909 ms) : 0, 662909
BytebuddyAgent [candidate] (671.444 ms) : 0, 671444
AgentMeter [baseline] (12.054 ms) : 0, 12054
AgentMeter [candidate] (12.125 ms) : 0, 12125
GlobalTracer [baseline] (248.928 ms) : 0, 248928
GlobalTracer [candidate] (250.128 ms) : 0, 250128
IAST [baseline] (24.421 ms) : 0, 24421
IAST [candidate] (24.689 ms) : 0, 24689
AppSec [baseline] (184.866 ms) : 0, 184866
AppSec [candidate] (186.036 ms) : 0, 186036
Debugger [baseline] (65.914 ms) : 0, 65914
Debugger [candidate] (66.351 ms) : 0, 66351
Remote Config [baseline] (613.821 µs) : 0, 614
Remote Config [candidate] (616.572 µs) : 0, 617
Telemetry [baseline] (8.365 ms) : 0, 8365
Telemetry [candidate] (8.47 ms) : 0, 8470
Flare Poller [baseline] (3.521 ms) : 0, 3521
Flare Poller [candidate] (3.529 ms) : 0, 3529
section iast
crashtracking [baseline] (1.231 ms) : 0, 1231
crashtracking [candidate] (1.214 ms) : 0, 1214
BytebuddyAgent [baseline] (808.463 ms) : 0, 808463
BytebuddyAgent [candidate] (801.207 ms) : 0, 801207
AgentMeter [baseline] (11.473 ms) : 0, 11473
AgentMeter [candidate] (11.487 ms) : 0, 11487
GlobalTracer [baseline] (240.411 ms) : 0, 240411
GlobalTracer [candidate] (238.286 ms) : 0, 238286
IAST [baseline] (26.118 ms) : 0, 26118
IAST [candidate] (25.665 ms) : 0, 25665
AppSec [baseline] (33.301 ms) : 0, 33301
AppSec [candidate] (30.973 ms) : 0, 30973
Debugger [baseline] (60.849 ms) : 0, 60849
Debugger [candidate] (65.181 ms) : 0, 65181
Remote Config [baseline] (562.246 µs) : 0, 562
Remote Config [candidate] (547.498 µs) : 0, 547
Telemetry [baseline] (12.561 ms) : 0, 12561
Telemetry [candidate] (9.341 ms) : 0, 9341
Flare Poller [baseline] (3.472 ms) : 0, 3472
Flare Poller [candidate] (3.459 ms) : 0, 3459
section profiling
crashtracking [baseline] (1.172 ms) : 0, 1172
crashtracking [candidate] (1.187 ms) : 0, 1187
BytebuddyAgent [baseline] (692.644 ms) : 0, 692644
BytebuddyAgent [candidate] (692.136 ms) : 0, 692136
AgentMeter [baseline] (9.161 ms) : 0, 9161
AgentMeter [candidate] (8.958 ms) : 0, 8958
GlobalTracer [baseline] (207.193 ms) : 0, 207193
GlobalTracer [candidate] (206.351 ms) : 0, 206351
AppSec [baseline] (32.791 ms) : 0, 32791
AppSec [candidate] (32.743 ms) : 0, 32743
Debugger [baseline] (65.853 ms) : 0, 65853
Debugger [candidate] (65.417 ms) : 0, 65417
Remote Config [baseline] (578.107 µs) : 0, 578
Remote Config [candidate] (567.718 µs) : 0, 568
Telemetry [baseline] (7.805 ms) : 0, 7805
Telemetry [candidate] (7.783 ms) : 0, 7783
Flare Poller [baseline] (3.556 ms) : 0, 3556
Flare Poller [candidate] (3.536 ms) : 0, 3536
ProfilingAgent [baseline] (93.932 ms) : 0, 93932
ProfilingAgent [candidate] (94.561 ms) : 0, 94561
Profiling [baseline] (94.512 ms) : 0, 94512
Profiling [candidate] (95.126 ms) : 0, 95126
Startup time reports for insecure-bankgantt
title insecure-bank - global startup overhead: candidate=1.62.0-SNAPSHOT~17ebcd0a24, baseline=1.62.0-SNAPSHOT~95de525c87
dateFormat X
axisFormat %s
section tracing
Agent [baseline] (1.06 s) : 0, 1060468
Total [baseline] (8.869 s) : 0, 8868880
Agent [candidate] (1.059 s) : 0, 1059166
Total [candidate] (8.859 s) : 0, 8859134
section iast
Agent [baseline] (1.223 s) : 0, 1223247
Total [baseline] (9.558 s) : 0, 9558334
Agent [candidate] (1.22 s) : 0, 1219750
Total [candidate] (9.587 s) : 0, 9586591
gantt
title insecure-bank - break down per module: candidate=1.62.0-SNAPSHOT~17ebcd0a24, baseline=1.62.0-SNAPSHOT~95de525c87
dateFormat X
axisFormat %s
section tracing
crashtracking [baseline] (1.233 ms) : 0, 1233
crashtracking [candidate] (1.248 ms) : 0, 1248
BytebuddyAgent [baseline] (633.781 ms) : 0, 633781
BytebuddyAgent [candidate] (636.972 ms) : 0, 636972
AgentMeter [baseline] (29.635 ms) : 0, 29635
AgentMeter [candidate] (29.508 ms) : 0, 29508
GlobalTracer [baseline] (250.066 ms) : 0, 250066
GlobalTracer [candidate] (249.74 ms) : 0, 249740
AppSec [baseline] (32.715 ms) : 0, 32715
AppSec [candidate] (32.446 ms) : 0, 32446
Debugger [baseline] (59.804 ms) : 0, 59804
Debugger [candidate] (59.189 ms) : 0, 59189
Remote Config [baseline] (608.599 µs) : 0, 609
Remote Config [candidate] (600.769 µs) : 0, 601
Telemetry [baseline] (8.156 ms) : 0, 8156
Telemetry [candidate] (9.706 ms) : 0, 9706
Flare Poller [baseline] (8.328 ms) : 0, 8328
Flare Poller [candidate] (3.538 ms) : 0, 3538
section iast
crashtracking [baseline] (1.224 ms) : 0, 1224
crashtracking [candidate] (1.219 ms) : 0, 1219
BytebuddyAgent [baseline] (800.98 ms) : 0, 800980
BytebuddyAgent [candidate] (799.129 ms) : 0, 799129
AgentMeter [baseline] (11.411 ms) : 0, 11411
AgentMeter [candidate] (11.381 ms) : 0, 11381
GlobalTracer [baseline] (238.935 ms) : 0, 238935
GlobalTracer [candidate] (237.744 ms) : 0, 237744
AppSec [baseline] (33.76 ms) : 0, 33760
AppSec [candidate] (29.614 ms) : 0, 29614
Debugger [baseline] (59.405 ms) : 0, 59405
Debugger [candidate] (65.223 ms) : 0, 65223
Remote Config [baseline] (542.653 µs) : 0, 543
Remote Config [candidate] (1.152 ms) : 0, 1152
Telemetry [baseline] (11.736 ms) : 0, 11736
Telemetry [candidate] (9.261 ms) : 0, 9261
Flare Poller [baseline] (3.39 ms) : 0, 3390
Flare Poller [candidate] (3.359 ms) : 0, 3359
IAST [baseline] (25.737 ms) : 0, 25737
IAST [candidate] (25.669 ms) : 0, 25669
LoadParameters
See matching parameters
SummaryFound 0 performance improvements and 2 performance regressions! Performance is the same for 19 metrics, 15 unstable metrics.
Request duration reports for petclinicgantt
title petclinic - request duration [CI 0.99] : candidate=1.62.0-SNAPSHOT~17ebcd0a24, baseline=1.62.0-SNAPSHOT~95de525c87
dateFormat X
axisFormat %s
section baseline
no_agent (17.3 ms) : 17129, 17471
. : milestone, 17300,
appsec (18.958 ms) : 18766, 19150
. : milestone, 18958,
code_origins (17.952 ms) : 17778, 18126
. : milestone, 17952,
iast (18.146 ms) : 17965, 18328
. : milestone, 18146,
profiling (18.425 ms) : 18240, 18609
. : milestone, 18425,
tracing (17.761 ms) : 17589, 17934
. : milestone, 17761,
section candidate
no_agent (18.272 ms) : 18086, 18459
. : milestone, 18272,
appsec (18.858 ms) : 18669, 19048
. : milestone, 18858,
code_origins (17.978 ms) : 17800, 18157
. : milestone, 17978,
iast (17.809 ms) : 17632, 17985
. : milestone, 17809,
profiling (19.357 ms) : 19165, 19549
. : milestone, 19357,
tracing (17.635 ms) : 17461, 17810
. : milestone, 17635,
Request duration reports for insecure-bankgantt
title insecure-bank - request duration [CI 0.99] : candidate=1.62.0-SNAPSHOT~17ebcd0a24, baseline=1.62.0-SNAPSHOT~95de525c87
dateFormat X
axisFormat %s
section baseline
no_agent (1.25 ms) : 1237, 1263
. : milestone, 1250,
iast (3.295 ms) : 3250, 3339
. : milestone, 3295,
iast_FULL (5.917 ms) : 5857, 5977
. : milestone, 5917,
iast_GLOBAL (3.723 ms) : 3661, 3785
. : milestone, 3723,
profiling (2.048 ms) : 2030, 2066
. : milestone, 2048,
tracing (1.874 ms) : 1858, 1890
. : milestone, 1874,
section candidate
no_agent (1.247 ms) : 1235, 1260
. : milestone, 1247,
iast (3.229 ms) : 3183, 3275
. : milestone, 3229,
iast_FULL (5.859 ms) : 5800, 5917
. : milestone, 5859,
iast_GLOBAL (3.726 ms) : 3669, 3782
. : milestone, 3726,
profiling (2.173 ms) : 2152, 2194
. : milestone, 2173,
tracing (1.826 ms) : 1811, 1841
. : milestone, 1826,
DacapoParameters
See matching parameters
SummaryFound 1 performance improvements and 0 performance regressions! Performance is the same for 11 metrics, 0 unstable metrics.
Execution time for biojavagantt
title biojava - execution time [CI 0.99] : candidate=1.62.0-SNAPSHOT~17ebcd0a24, baseline=1.62.0-SNAPSHOT~95de525c87
dateFormat X
axisFormat %s
section baseline
no_agent (15.548 s) : 15548000, 15548000
. : milestone, 15548000,
appsec (14.499 s) : 14499000, 14499000
. : milestone, 14499000,
iast (18.065 s) : 18065000, 18065000
. : milestone, 18065000,
iast_GLOBAL (17.9 s) : 17900000, 17900000
. : milestone, 17900000,
profiling (14.763 s) : 14763000, 14763000
. : milestone, 14763000,
tracing (14.935 s) : 14935000, 14935000
. : milestone, 14935000,
section candidate
no_agent (15.598 s) : 15598000, 15598000
. : milestone, 15598000,
appsec (14.728 s) : 14728000, 14728000
. : milestone, 14728000,
iast (18.059 s) : 18059000, 18059000
. : milestone, 18059000,
iast_GLOBAL (17.852 s) : 17852000, 17852000
. : milestone, 17852000,
profiling (15.26 s) : 15260000, 15260000
. : milestone, 15260000,
tracing (14.845 s) : 14845000, 14845000
. : milestone, 14845000,
Execution time for tomcatgantt
title tomcat - execution time [CI 0.99] : candidate=1.62.0-SNAPSHOT~17ebcd0a24, baseline=1.62.0-SNAPSHOT~95de525c87
dateFormat X
axisFormat %s
section baseline
no_agent (1.497 ms) : 1485, 1509
. : milestone, 1497,
appsec (3.872 ms) : 3647, 4096
. : milestone, 3872,
iast (2.291 ms) : 2220, 2361
. : milestone, 2291,
iast_GLOBAL (2.331 ms) : 2260, 2402
. : milestone, 2331,
profiling (2.123 ms) : 2067, 2179
. : milestone, 2123,
tracing (2.105 ms) : 2050, 2160
. : milestone, 2105,
section candidate
no_agent (1.492 ms) : 1481, 1504
. : milestone, 1492,
appsec (2.568 ms) : 2512, 2623
. : milestone, 2568,
iast (2.282 ms) : 2212, 2353
. : milestone, 2282,
iast_GLOBAL (2.337 ms) : 2266, 2408
. : milestone, 2337,
profiling (2.098 ms) : 2042, 2154
. : milestone, 2098,
tracing (2.103 ms) : 2048, 2158
. : milestone, 2103,
|
96c7d44 to
ea98882
Compare
|
For info, I clean up the duplicate "inst:sofa rpc" (without the space after |
b3f3860 to
0555301
Compare
| @@ -0,0 +1 @@ | |||
| apply from: "$rootDir/gradle/java.gradle" | |||
There was a problem hiding this comment.
Note you can delete dd-java-agent/instrumentation/sofarpc/build.gradle and omit the ":dd-java-agent:instrumentation:sofarpc", entry from settings.gradle if this parent directory doesn't contain any source / resources.
mcculls
left a comment
There was a problem hiding this comment.
LGTM, thanks for the comprehensive tests
| class SofaRpcRestTest extends InstrumentationSpecification { | ||
|
|
||
| @Shared | ||
| int port = 12205 | ||
|
|
||
| @Shared | ||
| ProviderBootstrap restProviderBootstrap | ||
|
|
There was a problem hiding this comment.
Just curious if tests can be implemented with JUnit?
Recently we added a lot of functionality and base classes to simplify conversion from Groovy to JUnit.
Assuming this is new code it would benefit to have JUnit tests if possible
…n, scope/span ordering
…span hierarchy, add REST and Triple tests
…er spans, rpc.method tag
614e75b to
17ebcd0
Compare
|
/merge |
|
View all feedbacks in Devflow UI.
The expected merge time in
|
93780c1
into
master
What Does This Do
Adds auto-instrumentation for Sofa RPC, a high-perfomance RPC framework. Produces
sofarpc.requestclient and server spans with distributed trace propagation for all four supported transports: Bolt, H2C, REST and Triple(gRPC)Motivation
Sofa RPC is widely used in Ant Financial's ecosystem and it's not covered by any existing dd-trace-java integration. WIthout this instrumentaiton, traces break at the service boundary - downstream PRC calls are unlinked from traces
Additional Notes
Module structure
A single module
sofarpc-5.0covers all protocols and all public version of sofa-rpc. The framework was open-sources at 5.0.0; no earlier versions exists in public repositoriesHow server spans are coordinated
ProviderProxyInvoker.invoke()is a single entry point for all protocols on the server side - but it has noo info about the transport delivered the request. Transport specific instrumentations write the protocol name into ThreadLocal (SofaRpcProtocolContext) beforeinvoke()is called.ProviderProxuInvokerInstrumentaitonreads it to created the span and tag with the protocol. If the ThreadLocal is empty (uninstrumented transport), no span is created.Trace propagation by protocol
Bolt and H2C - context travels inside SofaRequest.requestProps (the framework's own key-value bag)

REST -

SofaRequest.requestPropsis not serialized as HTTP headers by the REST transport. Context instead travels as standard HTTP headers, injected and extracted by HTTP layer (Apache HttpClient on the client side, Netty on the server side). TheRestServerHandlerInstumentationsets the protocol context so thatProviderProxyInvoketInstrumentationactivated and creates asofarpc.request[server] span that naturally attaches to the activenetty.request` spanTriple - uses gRPC unses the hood. The grpc-1.5 instrumentation handles trace propagation via gRPC Metadata. The

TripleServerInstrumentationsetsprotocol="tri"so thatProviderProxyInvokerInstrumentaitoncreates asofarpc.request[server]span with child of the activegrpc.serverspanSpan attributes
Resource name format:
{rpc.service}/{rpc.method}(e.g.com.example.HelloService:1.0/sayHello)Known limitations
SofaRequest.getTaargetServiceUniqueName()returns null on the server side - the service name is not propagated thought the JAX-RS layer.Contributor Checklist
type:and (comp:orinst:) labels in addition to any other useful labelsclose,fix, or any linking keywords when referencing an issueUse
solvesinstead, and assign the PR milestone to the issueJira ticket: [PROJ-IDENT]
Note: Once your PR is ready to merge, add it to the merge queue by commenting
/merge./merge -ccancels the queue request./merge -f --reason "reason"skips all merge queue checks; please use this judiciously, as some checks do not run at the PR-level. For more information, see this doc.