feat: queue time capture for Rack#2838
Conversation
|
sentry-ruby/lib/sentry/span.rb
Outdated
| MESSAGING_MESSAGE_RETRY_COUNT = "messaging.message.retry.count" | ||
|
|
||
| # Time in ms the request spent in the server queue before processing began. | ||
| HTTP_QUEUE_TIME_MS = "http.queue_time_ms" |
There was a problem hiding this comment.
@AbhiPrasad @lcian is there an attribute convention for time spent in queue waiting for the http request to be picked up in an HTTP server?
The closest I could find is http.client.request.time_in_queue but that's for clients.
There was a problem hiding this comment.
Hmmm I cannot find anything like that. Maybe we introduce our own http.server.request.time_in_queue counterpart?
There was a problem hiding this comment.
sounds good; shall I add to sentry-conventions too?
sl0thentr0py
left a comment
There was a problem hiding this comment.
some small stuff, and mainly want to wait for feedback on the attribute convention name
Co-authored-by: Neel Shah <neel.shah@sentry.io>
solnic
left a comment
There was a problem hiding this comment.
✅ but I left a bunch of minor 💅🏻 suggestions 🙃
Co-authored-by: Peter Solnica <peter@solnica.online>
Co-authored-by: Peter Solnica <peter@solnica.online>
Non-numeric t= values like "t=invalid" or "t=" were silently converted to 0.0 by Ruby's String#to_f, resulting in a Unix epoch timestamp and a queue time of ~56 years instead of nil. Apply the same numeric regex guard already used for raw timestamps to the t= branch before calling to_f. Co-Authored-By: Claude <noreply@anthropic.com>
Co-authored-by: Peter Solnica <peter@solnica.online>
Co-authored-by: Peter Solnica <peter@solnica.online>
Co-authored-by: Peter Solnica <peter@solnica.online>
Co-authored-by: Peter Solnica <peter@solnica.online>
Co-authored-by: Peter Solnica <peter@solnica.online>
Co-authored-by: Peter Solnica <peter@solnica.online>
parse_request_start_header returned nil for headers that arrived with leading/trailing whitespace or as comma-separated values (multiple header occurrences collapsed by a proxy). Strip whitespace from the token and split on commas, taking the first (earliest) timestamp. Co-Authored-By: Claude <noreply@anthropic.com>
When env["puma.request_body_wait"] is a String (e.g. serialized by
middleware), the `> 0` comparison raised ArgumentError, which was
silently swallowed by the broad rescue, causing queue time to be
dropped even with a valid X-Request-Start header.
Call to_f on String values before the comparison. Numeric strings
("40") convert correctly; non-numeric strings ("N/A") become 0.0
and are treated as absent, so total_time_ms is returned unchanged.
Co-Authored-By: Claude <noreply@anthropic.com>
…-ruby into puma-queue-time
There was a problem hiding this comment.
Cursor Bugbot has reviewed your changes and found 1 potential issue.
Bugbot Autofix is OFF. To automatically fix reported issues with Cloud Agents, enable Autofix in the Cursor dashboard.
| queue_time_ms >= 0 ? queue_time_ms : 0.0 # more sanity check | ||
| else | ||
| total_time_ms | ||
| end |
There was a problem hiding this comment.
Queue time spoofable via client header
Medium Severity
extract_queue_time trusts env["HTTP_X_REQUEST_START"] without verifying it was set/overwritten by a trusted proxy and without any upper-bound sanity check, so a client-supplied X-Request-Start can inject arbitrarily large http.server.request.time_in_queue values and skew transaction data.
Adds the `http.server.request.time_in_queue` semantic convention attribute to capture the time (in milliseconds) a request spent waiting in the server queue before processing began. This attribute is populated from the `X-Request-Start` header set by reverse proxies (Nginx, HAProxy, Heroku router, etc.) and first introduced in sentry-ruby: getsentry/sentry-ruby#2838 Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
* feat(http): add http.server.request.time_in_queue attribute Adds the `http.server.request.time_in_queue` semantic convention attribute to capture the time (in milliseconds) a request spent waiting in the server queue before processing began. This attribute is populated from the `X-Request-Start` header set by reverse proxies (Nginx, HAProxy, Heroku router, etc.) and first introduced in sentry-ruby: getsentry/sentry-ruby#2838 Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * feat(http): add generated attribute definitions for http.server.request.time_in_queue Generate TypeScript and Python attribute definitions including type exports, metadata entries, and TypedDict annotations. Co-Authored-By: Claude <noreply@anthropic.com> --------- Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
The Rails CaptureExceptions middleware subclass overrode start_transaction without calling super, so the http.server.request.time_in_queue attachment added in #2838 was silently skipped for all Rails apps. Extract queue time attachment into a private attach_queue_time method on the Rack base class so the Rails subclass can invoke it explicitly while keeping its own options hash (preserving the correct SPAN_ORIGIN constant "auto.http.rails" via lexical scoping). Fixes #2873 Co-Authored-By: Claude <noreply@anthropic.com>
The Rails CaptureExceptions middleware subclass overrode start_transaction without calling super, so the http.server.request.time_in_queue attachment added in #2838 was silently skipped for all Rails apps. Extract queue time attachment into a private attach_queue_time method on the Rack base class so the Rails subclass can invoke it explicitly while keeping its own options hash (preserving the correct SPAN_ORIGIN constant "auto.http.rails" via lexical scoping). Fixes #2873 Co-Authored-By: Claude <noreply@anthropic.com>
The Rails CaptureExceptions middleware subclass overrode start_transaction without calling super, so the http.server.request.time_in_queue attachment added in #2838 was silently skipped for all Rails apps. Extract queue time attachment into a private attach_queue_time method on the Rack base class so the Rails subclass can invoke it explicitly while keeping its own options hash (preserving the correct SPAN_ORIGIN constant "auto.http.rails" via lexical scoping). Fixes #2873 Co-authored-by: Claude <noreply@anthropic.com>
<!-- Use this checklist to make sure your PR is ready for merge. You may delete any sections you don't need. --> ## DESCRIBE YOUR PR docs for Ruby queue-time capture getsentry/sentry-ruby#2838 ## IS YOUR CHANGE URGENT? Help us prioritize incoming PRs by letting us know when the change needs to go live. - [ ] Urgent deadline (GA date, etc.): <!-- ENTER DATE HERE --> - [ ] Other deadline: <!-- ENTER DATE HERE --> - [x] None: Not urgent, can wait up to 1 week+ ## SLA - Teamwork makes the dream work, so please add a reviewer to your PRs. - Please give the docs team up to 1 week to review your PR unless you've added an urgent due date to it. Thanks in advance for your help! ## PRE-MERGE CHECKLIST *Make sure you've checked the following before merging your changes:* - [ ] Checked Vercel preview for correctness, including links - [ ] PR was reviewed and approved by any necessary SMEs (subject matter experts) - [ ] PR was reviewed and approved by a member of the [Sentry docs team](https://github.com/orgs/getsentry/teams/docs) ## LEGAL BOILERPLATE <!-- Sentry employees and contractors can delete or ignore this section. --> Look, I get it. The entity doing business as "Sentry" was incorporated in the State of Delaware in 2015 as Functional Software, Inc. and is gonna need some rights from me in order to utilize my contributions in this here PR. So here's the deal: I retain all rights, title and interest in and to my contributions, and by keeping this boilerplate intact I confirm that Sentry can use, modify, copy, and redistribute my contributions, under Sentry's choice of terms. ## EXTRA RESOURCES - [Sentry Docs contributor guide](https://docs.sentry.io/contributing/) --------- Co-authored-by: getsantry[bot] <66042841+getsantry[bot]@users.noreply.github.com> Co-authored-by: Alex Krawiec <alex.krawiec@sentry.io> Co-authored-by: Claude <noreply@anthropic.com>


Captures how long requests wait for a e.g. Puma thread from
X-Request-Startheaders and attaches it to transactions ashttp.server.request.time_in_queue. Sentry only measures after Puma picks up the request. Under load, requests might wait long in the queue but only take a fraction to process. Similar to what judoscale and scout do.works with all major reverse proxies:
X-Request-Start: t=1234567890.123(seconds)X-Request-Start: t=1234567890123456(microseconds)X-Request-Start: t=1234567890123456(microseconds)X-Request-Start: t=1234567890(seconds)subtracts
puma.request_body_waitfor accuracy (excludes slow client uploads from queue time).Usage
works 🤞 if your reverse proxy sets the header:
opt-out