Add mermaid diagrams to Xet protocol spec docs#2299
Closed
rajatarya wants to merge 5 commits into
Closed
Conversation
- index.md: block diagram showing overall Xet architecture (file → chunks → xorbs → shard → CAS) - xorb.md: packet diagram for chunk header wire layout (replaces ASCII art) - shard.md: packet diagrams for all binary structures — header, FileDataSequenceHeader/Entry, FileVerificationEntry, FileMetadataExt, CASChunkSequenceHeader/Entry, footer (replaces ASCII art) - chunking.md: flowchart for the CDC boundary decision algorithm - hashing.md: flowchart showing the four hash computation paths (chunk, xorb, file, verification) - file-id.md: sequence diagram for the resolve URL → X-Xet-Hash flow Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
|
The docs for this PR live here. All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update. |
All packet diagrams now use 1 unit = 1 byte instead of 1 unit = 1 bit. This prevents 32-byte hash fields from spanning 8 rows with repeated labels, making the diagrams much more compact and readable. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
The 8-byte chunk header is too small for byte-level units (cells are unreadably tiny on the 32-unit row). Bit-level gives 2 well-proportioned rows of 32 bits each with readable labels. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Contributor
Author
|
The Mermaid rendering doesn't look great, adds unnecessary tags in the renderings. Let me know if you'd rather I abandon these changes because they don't look good. |
Some markdown-to-HTML converters split on blank lines within fenced code blocks and inject <p> tags before mermaid processes the content. Removing all blank lines inside mermaid blocks fixes this. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Unquoted square brackets in mermaid flowchart nodes (e.g. A[Text]) can be interpreted as markdown link references by some parsers, causing <p></p> tags to wrap each label. Using quoted strings (A["Text"]) fixes this. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
assafvayner
requested changes
Apr 14, 2026
assafvayner
left a comment
assafvayner
left a comment
Contributor
There was a problem hiding this comment.
some visuals not rendering right, but it's worth fixing imo
Comment on lines
57
to
63
| ```mermaid | ||
| graph TD | ||
| A[File Input] --> B[Content-Defined Chunking] | ||
| B --> C[Hash Computation] | ||
| C --> D[Chunk Creation] | ||
| D --> E[Deduplication Query] | ||
| A["File Input"] --> B["Content-Defined Chunking"] | ||
| B --> C["Hash Computation"] | ||
| C --> D["Chunk Creation"] | ||
| D --> E["Deduplication Query"] | ||
| ``` |
Contributor
There was a problem hiding this comment.
doing this change doesn't render correctly, we get a bunch of
| │(8 bytes)│ (8 bytes)│ │(8 bytes)│ | ||
| └─────────┴──────────┴─────────────────────────────────────────────────────────────────────────────┴─────────┘ | ||
| 104 112 120 192 200 | ||
| ```mermaid |
Contributor
There was a problem hiding this comment.
need to make these not scroll, stretch them vertically.
| ChunkHash --> CH | ||
| CH -->|"Merkle Tree\n+ INTERNAL_NODE_KEY"| XorbHash["Xorb Hash"] | ||
| CH -->|"Merkle Tree\n+ INTERNAL_NODE_KEY\nthen blake3(root, zeros)"| FileHash["File Hash"] | ||
| CH -->|"blake3(concat hashes,\nVERIFICATION_KEY)"| VerifHash["Term Verification Hash"] |
Contributor
There was a problem hiding this comment.
bunch of
in this one too, needs to probably drop the quote
Contributor
Author
|
Closing — went stale (last update Apr 14) and the protocol docs have evolved enough that these mermaid diagrams need a rewrite rather than a rebase. Will re-open a fresh PR once the s3-gateway + bucket-listing perf work settles and the protocol docs stabilize. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
4 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Existing sequence diagrams in upload-protocol.md, download-protocol.md, and auth.md were already good and left unchanged.
Test plan
🤖 Generated with Claude Code
Note
Low Risk
Low risk documentation-only change; primary risk is incorrect mermaid syntax or mislabeled offsets/bit ranges causing reader confusion.
Overview
Adds mermaid diagrams across the Xet protocol spec docs to replace/augment text-only explanations.
This introduces: a CDC decision flowchart in
chunking.md, a hashing flowchart inhashing.md, a file-id resolve sequence diagram infile-id.md, and an overall architecture block diagram inindex.md. It also replaces several ASCII-art binary layout tables with mermaidpacketdiagrams inshard.mdandxorb.md, and tweaks the deduplication flow diagram labels indeduplication.md.Written by Cursor Bugbot for commit 0f4a765. This will update automatically on new commits. Configure here.