Skip to content

Commit 99a17dc

Browse files
feat(telemetry-processor): Size limits (#16066)
Add recommendations of size limits for metric, spans and logs.
1 parent 38cdc12 commit 99a17dc

File tree

2 files changed

+15
-6
lines changed

2 files changed

+15
-6
lines changed

develop-docs/sdk/telemetry/telemetry-processor/backend-telemetry-processor.mdx

Lines changed: 1 addition & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -81,11 +81,7 @@ The telemetry buffer on the backend must follow the common [telemetry buffer req
8181

8282
1. The telemetry buffer **SHOULD** drop older items as the overflow policy. It **MAY** also drop newer items to preserve what's already buffered.
8383

84-
Here are the size limit recommendations for the backend. Each SDK MAY choose a different value, but it **MUST** respect the [envelope max sizes](/sdk/data-model/envelopes/#size-limits):
85-
- 100 items for logs
86-
- 100 items for metrics
87-
- 1000 items for spans
88-
84+
On the backend, use the same size limits as the [common requirements](/sdk/telemetry/telemetry-processor/#telemetry-buffer), except for spans, where we recommend **1000** because span volume is higher.
8985

9086
##### Span Buffer
9187

develop-docs/sdk/telemetry/telemetry-processor/index.mdx

Lines changed: 14 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -92,9 +92,22 @@ The telemetry buffer batches high-volume data and forwards it to the telemetry s
9292
2. When the telemetry buffer overflows and it drops data, it **MUST** record client reports.
9393
3. The telemetry buffer **MUST** forward low-volume data, such as normal events, session replays, or user feedback, directly to the telemetry scheduler.
9494
4. The telemetry buffer **MUST** start a timeout of 5 seconds when the first item is added. When the timeout expires, the telemetry buffer **MUST** forward all items to the telemetry scheduler.
95-
5. The telemetry buffer **MUST** define a size limit of x items. Each SDK MAY choose its own x based on its needs, but SDKs **MUST** ensure that the serialized envelope doesn't exceed the [envelope max sizes](/sdk/data-model/envelopes/#size-limits).
95+
5. The telemetry buffer **MUST** define a size limit of `x` items. See [Size Limit Recommendations](#size-limit-recommendations) below for more details.
9696
6. When the size limit is reached, the telemetry buffer **MUST** forward all items to the telemetry scheduler. The buffer **MAY** forward items in batches.
9797

98+
99+
## Size Limit Recommendations
100+
101+
We recommend starting with 100 items for logs, metrics, and spans for `x`, which is the maximum number of items the buffer can hold. SDKs **MAY** choose different values based on their needs, but they **SHOULD** respect the max suggested `x` values:
102+
103+
| Data type | Max suggested x |
104+
| --- | --- |
105+
| Logs | ~1150 |
106+
| Metrics | ~1800 (Each metric **MUST** be `<= 2 KiB` or Relay drops the entire envelope) |
107+
| Spans | ~1300 |
108+
109+
Here is the reasoning behind the max numbers. Using the example payloads in [Logs](/sdk/telemetry/logs/#appendix-a-example-log-envelope), [Metrics](/sdk/telemetry/metrics/#appendix-a-example-trace_metric-envelope), and the [Span Protocol](/sdk/telemetry/spans/span-protocol/#span-v2-envelope-item-payload), the average serialized item sizes are ~730 B (log), ~460 B (metric), and ~630 B (span). Applying a 20% safety buffer against the Relay [envelope size limits](/sdk/data-model/envelopes/#size-limits) yields the maximum suggested x values.
110+
98111
## Data Forwarding Scenarios
99112

100113
The telemetry buffer **MUST** forward all data in memory to the telemetry scheduler to avoid data loss in the following scenarios:

0 commit comments

Comments
 (0)