Commit 5fc0969
authored
Keep Netty 4.1 server spans open until streaming responses complete (#11941)
test(netty-4.1): Add pipelining test
fix(netty-4.1): pipelined response context tracking
Disable Netty server tracing on context queue overflow
Fix Netty 4.1 response context handling
Track server request contexts per pipelined response, snapshot only the
Accept header, and keep AppSec response-block state channel-level so
late streaming chunks are dropped until close.
Add Netty chunked response span timing tests
Fix Netty 4.1 server span completion for streaming responses
Keep server spans open until terminal response writes, finish incomplete
responses on channel close, and use queued request contexts to avoid
keep-alive/pipelining misattribution. Cover chunked, bodyless, HEAD,
CONNECT, interim, and WebSocket upgrade response cases.
Fix Netty server span lifecycle for streaming responses
Keep Netty 4.1 server spans open until terminal response writes while
preserving request-ended callback timing for AppSec/IAST. Track one-shot
beforeFinish execution separately from span finish so streaming spans
get
correct duration without dropping request-end security tags.
Fix Netty 4.1 server span edge cases from PR review
- Defer beforeFinish until the response actually completes.
- Finish close-delimited responses without error.
- Guard Content-Length parsing.
- Skip AppSec response analysis for all interim 1xx responses in
MaybeBlockResponseHandler, not just 100 Continue.
Add a regression test for close-delimited responses.
Restore request-ended callbacks for terminal Netty responses
Resolved: terminal responses no longer call `beforeFinish(...)`
immediately after `ctx.write(...)`. The write-promise listener is now
the only normal terminal completion path, so `finishSpan(...)` runs
after the write completes and invokes `beforeFinish(...)` there.
The exception path is also guarded with `writePromise.isDone()`: if the
terminal write promise has already completed, the listener has already
handled finishing, so the catch block just rethrows and avoids a second
finish/decorator pass.
Changed HttpServerResponseTracingHandler.java so unknown-length
responses are marked close-delimited before `LastHttpContent` can make
them terminal. Close-delimited contexts now wait for `channelInactive`.
Added a regression in NettyChunkedResponseSpanTest.java for
`DefaultFullHttpResponse` with no `Content-Length` and no chunked
encoding.
Restrict close-delimited response handling to HTTP/1.x only.
The Netty HTTP/2 server path now caches when
`Http2StreamFrameToHttpObjectCodec` is present and avoids marking
no-length responses as close-delimited there. This keeps HTTP/2
responses dependent on `LastHttpContent`/END_STREAM, so an early stream
close is still reported as an incomplete-response error.
The Netty 4.1 websocket upgrade check is now value-based for status
`101`, and the regression test for a distinct `HttpResponseStatus(101,
...)` passes.
It now treats known-length responses as complete once the declared body
bytes are successfully written, so closing the connection afterward no
longer marks the span as an incomplete-response error.
Chunked/incomplete known-length closes still error as expected.
Tests passed for `NettyChunkedResponseSpanTest`; changes are not
committed.
Fix Netty IAST reporting on async response completion
Reactivate the stored request context while finishing Netty server spans
so IAST request-ended callbacks run with the request span active. This
lets sink reports attach to the request span when response write
promises complete asynchronously or on channel close.
Add Netty regressions covering IAST request-ended callbacks for
known-length full responses and delayed write-promise completion.
Preserve Netty HTTP/2 stream contexts
Fallback to the channel context mirror when no queued
ServerRequestContext exists, so HTTP/2 multiplex child-channel responses
still decorate and finish propagated server spans. Add regression
coverage for the no-queue context-mirror path and document why
CONTEXT_ATTRIBUTE_KEY is maintained.
Fix deferred AppSec block responses for Netty pipelining
Avoid logging dropped Netty outbound messages
Use computed Netty 4.1 response status code
Use single-thread Netty event loop in tests
Merge branch 'ygree/fix-netty41-http11-pipelining-context-queue' into ygree/fix-netty41-chunked-tracing
Avoid synthetic accessors in Netty blocking response handler
Fix Netty block function pipelining deferral
Fix Netty pipelined block response ordering
Defer blocked responses until the active HTTP/1.1 response is complete,
including chunked/streaming responses that finish on LastHttpContent.
Handle header-only responses such as HEAD, 204, 205, 304, and explicit
Content-Length: 0 without requiring an application LastHttpContent.
Also keep interim 1xx responses from completing the server span and add
coverage for chunked, header-only, HEAD, interim, and malformed
Content-Length cases.
Fix Netty HTTP/2 response AppSec context handling
Fall back to the mirrored channel context when HTTP/2 stream channels
have no request context queue. Ignore informational responses until the
final response and add regression coverage for response blocking.
Fix Netty AppSec blocked response handling
Merge branch 'ygree/fix-netty41-http11-pipelining-context-queue' into ygree/fix-netty41-chunked-tracing
Merge branch 'master' into ygree/fix-netty41-http11-pipelining-context-queue
Merge branch 'ygree/fix-netty41-http11-pipelining-context-queue' into ygree/fix-netty41-chunked-tracing
Align request-context completion with Netty’s message framing contract.
This removes HEAD-specific state, header parsing, and synthetic outbound
messages while making pipelined response attribution follow Netty’s
actual response boundaries.
Reuse Netty blocking handler for deferred pipelined requests
Mark channels when a request block is committed so later pipelined
requests are forwarded to the existing blocking handler and discarded.
Serialize response-function handler installation on the event loop and
clean up partially installed handlers on failure.
Fix Netty blocking response scheduling races
Snapshot request metadata before scheduling blocking responses instead
of
retaining and replaying reference-counted HTTP requests.
Reject scheduled blocking responses when their server request context
has
already completed, preventing stale commits from blocking keep-alive
connections.
Reuse Netty request context queues for keep-alive traffic
Reuse Netty request context queues for keep-alive traffic
Merge branch 'ygree/fix-netty41-http11-pipelining-context-queue' into ygree/fix-netty41-chunked-tracing
Align Netty response completion with LastHttpContent
Finish server spans and dequeue pipelined request contexts only when
Netty writes LastHttpContent. Use the terminal write promise so
asynchronous failures retain error reporting and request context.
Remove Content-Length, bodyless-response, request-method, and
close-delimited completion inference, along with synthetic terminators
and the HTTP/2 marker used by that model. Preserve channel-close cleanup
for responses that never reach terminal content.
Update pipelining, AppSec, HTTP/2, framing, and asynchronous write tests
for the terminal-content contract.
Merge branch 'master' into ygree/fix-netty41-http11-pipelining-context-queue
Merge branch 'ygree/fix-netty41-http11-pipelining-context-queue' into ygree/fix-netty41-chunked-tracing
Raise agent jar size budget
The agent jar exceeds the previous budget by about 1 KiB. Raise the
budget to the next 16 KiB boundary to pass the verification check.
Merge branch 'master' into ygree/fix-netty41-chunked-tracing
Wait for Netty server spans before closing test sockets
Merge branch 'master' into ygree/fix-netty41-chunked-tracing
Co-authored-by: yury.gribkov <yury.gribkov@datadoghq.com>1 parent a92b3a6 commit 5fc0969
8 files changed
Lines changed: 1362 additions & 225 deletions
File tree
- dd-java-agent/instrumentation/netty
- netty-4.1/src
- main/java/datadog/trace/instrumentation/netty41/server
- test/java/datadog/trace/instrumentation/netty41/server
- netty-common/src/main/java/datadog/trace/instrumentation/netty41
Lines changed: 36 additions & 1 deletion
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
15 | 15 | | |
16 | 16 | | |
17 | 17 | | |
| 18 | + | |
18 | 19 | | |
19 | 20 | | |
20 | 21 | | |
| |||
95 | 96 | | |
96 | 97 | | |
97 | 98 | | |
98 | | - | |
| 99 | + | |
| 100 | + | |
| 101 | + | |
| 102 | + | |
| 103 | + | |
| 104 | + | |
| 105 | + | |
| 106 | + | |
| 107 | + | |
| 108 | + | |
| 109 | + | |
| 110 | + | |
99 | 111 | | |
100 | 112 | | |
101 | 113 | | |
102 | 114 | | |
| 115 | + | |
| 116 | + | |
| 117 | + | |
| 118 | + | |
| 119 | + | |
| 120 | + | |
| 121 | + | |
| 122 | + | |
| 123 | + | |
| 124 | + | |
| 125 | + | |
| 126 | + | |
| 127 | + | |
| 128 | + | |
| 129 | + | |
| 130 | + | |
| 131 | + | |
| 132 | + | |
| 133 | + | |
| 134 | + | |
| 135 | + | |
| 136 | + | |
| 137 | + | |
103 | 138 | | |
Lines changed: 90 additions & 36 deletions
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
14 | 14 | | |
15 | 15 | | |
16 | 16 | | |
| 17 | + | |
17 | 18 | | |
18 | 19 | | |
19 | 20 | | |
| 21 | + | |
20 | 22 | | |
21 | 23 | | |
22 | 24 | | |
| |||
25 | 27 | | |
26 | 28 | | |
27 | 29 | | |
28 | | - | |
| 30 | + | |
| 31 | + | |
29 | 32 | | |
30 | 33 | | |
31 | 34 | | |
| |||
50 | 53 | | |
51 | 54 | | |
52 | 55 | | |
53 | | - | |
| 56 | + | |
54 | 57 | | |
55 | | - | |
56 | | - | |
| 58 | + | |
| 59 | + | |
| 60 | + | |
| 61 | + | |
| 62 | + | |
| 63 | + | |
57 | 64 | | |
58 | | - | |
59 | | - | |
60 | | - | |
61 | | - | |
62 | | - | |
63 | | - | |
64 | | - | |
65 | | - | |
66 | | - | |
67 | | - | |
68 | | - | |
69 | | - | |
70 | | - | |
71 | | - | |
72 | | - | |
73 | | - | |
| 65 | + | |
| 66 | + | |
74 | 67 | | |
75 | | - | |
76 | | - | |
| 68 | + | |
| 69 | + | |
| 70 | + | |
| 71 | + | |
77 | 72 | | |
78 | | - | |
79 | | - | |
| 73 | + | |
| 74 | + | |
| 75 | + | |
| 76 | + | |
| 77 | + | |
80 | 78 | | |
81 | | - | |
| 79 | + | |
| 80 | + | |
| 81 | + | |
| 82 | + | |
| 83 | + | |
| 84 | + | |
| 85 | + | |
| 86 | + | |
| 87 | + | |
| 88 | + | |
82 | 89 | | |
83 | | - | |
84 | | - | |
85 | | - | |
86 | | - | |
87 | | - | |
88 | | - | |
| 90 | + | |
| 91 | + | |
| 92 | + | |
| 93 | + | |
| 94 | + | |
| 95 | + | |
| 96 | + | |
| 97 | + | |
| 98 | + | |
| 99 | + | |
| 100 | + | |
| 101 | + | |
| 102 | + | |
| 103 | + | |
| 104 | + | |
| 105 | + | |
| 106 | + | |
| 107 | + | |
| 108 | + | |
| 109 | + | |
| 110 | + | |
| 111 | + | |
| 112 | + | |
| 113 | + | |
| 114 | + | |
| 115 | + | |
| 116 | + | |
| 117 | + | |
| 118 | + | |
| 119 | + | |
| 120 | + | |
| 121 | + | |
| 122 | + | |
| 123 | + | |
| 124 | + | |
| 125 | + | |
| 126 | + | |
| 127 | + | |
| 128 | + | |
| 129 | + | |
| 130 | + | |
| 131 | + | |
| 132 | + | |
| 133 | + | |
| 134 | + | |
| 135 | + | |
| 136 | + | |
| 137 | + | |
| 138 | + | |
| 139 | + | |
| 140 | + | |
| 141 | + | |
| 142 | + | |
| 143 | + | |
| 144 | + | |
| 145 | + | |
| 146 | + | |
89 | 147 | | |
| 148 | + | |
90 | 149 | | |
91 | 150 | | |
92 | 151 | | |
| |||
98 | 157 | | |
99 | 158 | | |
100 | 159 | | |
101 | | - | |
102 | | - | |
103 | | - | |
104 | | - | |
105 | | - | |
106 | 160 | | |
Lines changed: 4 additions & 0 deletions
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
70 | 70 | | |
71 | 71 | | |
72 | 72 | | |
| 73 | + | |
| 74 | + | |
| 75 | + | |
| 76 | + | |
73 | 77 | | |
74 | 78 | | |
75 | 79 | | |
| |||
Lines changed: 60 additions & 8 deletions
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
13 | 13 | | |
14 | 14 | | |
15 | 15 | | |
| 16 | + | |
16 | 17 | | |
17 | 18 | | |
18 | 19 | | |
| |||
39 | 40 | | |
40 | 41 | | |
41 | 42 | | |
42 | | - | |
| 43 | + | |
43 | 44 | | |
44 | 45 | | |
45 | 46 | | |
46 | 47 | | |
47 | | - | |
| 48 | + | |
| 49 | + | |
48 | 50 | | |
49 | | - | |
50 | | - | |
51 | | - | |
| 51 | + | |
| 52 | + | |
| 53 | + | |
| 54 | + | |
| 55 | + | |
| 56 | + | |
| 57 | + | |
| 58 | + | |
52 | 59 | | |
53 | 60 | | |
54 | 61 | | |
| |||
70 | 77 | | |
71 | 78 | | |
72 | 79 | | |
73 | | - | |
| 80 | + | |
74 | 81 | | |
75 | 82 | | |
76 | 83 | | |
| |||
84 | 91 | | |
85 | 92 | | |
86 | 93 | | |
| 94 | + | |
87 | 95 | | |
88 | 96 | | |
89 | 97 | | |
| |||
95 | 103 | | |
96 | 104 | | |
97 | 105 | | |
98 | | - | |
| 106 | + | |
99 | 107 | | |
100 | 108 | | |
101 | 109 | | |
| 110 | + | |
| 111 | + | |
102 | 112 | | |
103 | 113 | | |
104 | 114 | | |
| |||
108 | 118 | | |
109 | 119 | | |
110 | 120 | | |
| 121 | + | |
111 | 122 | | |
112 | 123 | | |
113 | 124 | | |
114 | | - | |
| 125 | + | |
115 | 126 | | |
| 127 | + | |
116 | 128 | | |
117 | 129 | | |
118 | 130 | | |
119 | 131 | | |
| 132 | + | |
| 133 | + | |
| 134 | + | |
| 135 | + | |
| 136 | + | |
| 137 | + | |
| 138 | + | |
| 139 | + | |
| 140 | + | |
| 141 | + | |
| 142 | + | |
| 143 | + | |
| 144 | + | |
| 145 | + | |
| 146 | + | |
| 147 | + | |
| 148 | + | |
120 | 149 | | |
121 | 150 | | |
122 | 151 | | |
| |||
159 | 188 | | |
160 | 189 | | |
161 | 190 | | |
| 191 | + | |
| 192 | + | |
| 193 | + | |
| 194 | + | |
| 195 | + | |
| 196 | + | |
| 197 | + | |
| 198 | + | |
| 199 | + | |
| 200 | + | |
| 201 | + | |
| 202 | + | |
| 203 | + | |
| 204 | + | |
| 205 | + | |
| 206 | + | |
| 207 | + | |
| 208 | + | |
| 209 | + | |
| 210 | + | |
| 211 | + | |
| 212 | + | |
| 213 | + | |
162 | 214 | | |
0 commit comments