Skip to content

Commit 9d90683

Browse files
committed
update parallel
1 parent ec1ca64 commit 9d90683

1 file changed

Lines changed: 1 addition & 1 deletion

File tree

_articles/Parallel.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -79,7 +79,7 @@ LightX2V has specially designed the Ulysses-4090 variant for the RTX 4090 GPU. I
7979

8080
Ring Attention, proposed by the University of California, Berkeley, draws on the block-wise computation principle of FlashAttention. It treats the long sequence as a series of "building blocks" (K/V blocks) that need to be assembled sequentially, and distributes them to each GPU. Then, it exchanges the blocks along a ring-shaped communication structure (Ring) like a pipeline until the attention computation for the entire sequence is completed.
8181

82-
![Ring-Attention workflow diagram]({{ site.baseurl }}/assets/parallel-blog/ring-attention-workflow.png)
82+
![Ring-Attention workflow diagram]({{ site.baseurl }}/assets/Parallel-blog/img3.png)
8383

8484
#### Workflow
8585

0 commit comments

Comments
 (0)