Skip to content

Commit edbeaa8

Browse files
docs(debug-files): Summarize compression findings
Co-authored-by: daniel.szoke <daniel.szoke@sentry.io>
1 parent da54bd2 commit edbeaa8

1 file changed

Lines changed: 46 additions & 0 deletions

File tree

docs/snapshots/2026-01-29-debug-files-upload-compression.md

Lines changed: 46 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,27 @@
11
# Debug files upload compression findings
22

3+
## Conversation summary
4+
- `debug-files upload` does not compress whole debug files; it uploads raw bytes
5+
in chunks and only applies per-chunk transport compression (gzip or
6+
uncompressed).
7+
- There is no CLI option to compress entire debug files before upload.
8+
- The symbolic crates (v12.16.3) do not expose a whole-file DIF compression API.
9+
They only decompress compressed sections or embedded payloads when reading,
10+
and provide ZIP creation for source bundles.
11+
- `symsorter` in the symbolicator repo performs whole-file compression itself
12+
using zstd, with `-z/-zz/-zzz` mapping to zstd levels. Symbolic is used there
13+
only for parsing/iterating objects.
14+
- A symsorter-compressed file passed to `sentry-cli debug-files upload` would be
15+
skipped because format detection relies on object-file magic bytes; zstd
16+
wrappers do not match any supported DIF format. There is no whole-file
17+
decompression step in the upload path. If the format check were bypassed, the
18+
raw compressed bytes would be uploaded (still subject to per-chunk transport
19+
compression).
20+
- Extracting the symsorter compression logic into symbolic would likely require
21+
a small public helper in `symbolic-debuginfo` (or similar) that wraps zstd
22+
encoding, and then updating symsorter to call it while keeping the CLI's
23+
compression-level mapping in symsorter.
24+
325
## Question
426
Does `debug-files upload` ever compress the entire debug file before uploading
527
(not just transport-level chunk compression)?
@@ -58,3 +80,27 @@ Does `debug-files upload` ever compress the entire debug file before uploading
5880
- `symbolic` is used for parsing and iterating over objects (`Archive`, `Object`,
5981
`ObjectKind`), but compression is performed in symsorter, not via symbolic
6082
APIs. (See `symbolicator/crates/symsorter/src/app.rs` imports and usage.)
83+
84+
## CLI compression option
85+
- `debug-files upload` does not expose a flag to compress whole debug files
86+
before upload. The upload path only applies per-chunk transport compression
87+
negotiated with the server. (See `src/commands/debug_files/upload.rs` and
88+
`src/api/mod.rs:L354-L392`.)
89+
90+
## Symsorter-compressed file uploaded via sentry-cli
91+
- DIF discovery uses `Archive::peek` to identify object formats. If the format
92+
is `Unknown`, the file is skipped during collection. (See
93+
`src/utils/dif_upload/mod.rs:L733-L754`.)
94+
- `Archive::peek` looks for known object magic bytes (ELF, PE, Mach-O, PDB,
95+
etc.) and does not recognize zstd wrappers. (See
96+
`symbolic-debuginfo-12.16.3/src/object.rs:L127-L172`.)
97+
- There is no whole-file decompression step in the upload path, so a symsorter
98+
zstd-wrapped file would not be decompressed by sentry-cli. If the format check
99+
were bypassed, the raw compressed bytes would be uploaded.
100+
101+
## Extracting symsorter compression into symbolic (discussion)
102+
- This would require adding a public helper (likely in `symbolic-debuginfo`) to
103+
zstd-compress raw bytes or write a compressed stream, then updating symsorter
104+
to call that helper.
105+
- The policy mapping of `-z/-zz/-zzz` to zstd levels is CLI-specific and should
106+
remain in symsorter, while symbolic would accept a raw zstd level.

0 commit comments

Comments
 (0)