Skip to content

(feat): add dask documentation#216

Merged
ilan-gold merged 20 commits into
mainfrom
ig/dask-docs
May 15, 2025
Merged

(feat): add dask documentation#216
ilan-gold merged 20 commits into
mainfrom
ig/dask-docs

Conversation

@ilan-gold
Copy link
Copy Markdown
Contributor

Fixes #3346

Comment thread docs/how-to-dask.md Outdated
Comment on lines +81 to +84
In the {doc}`multi-gpu showcase notebook for rapids-singlecell <rapids-singlecell:notebooks/06-multi_gpu_show>`, {meth}`dask.array.Array.persist` appears across the notebook.
This loads the entire dataset into memory while keeping the representation as a dask array.
Thus, lazy computation still works but only necessitates a single read into memory.
The catch is that you have enough memory to use `persist`.
Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@Intron7 can you review this section specifically? feel free to do more, but this one would be great if you only have a bit of time

@ilan-gold ilan-gold requested a review from flying-sheep April 29, 2025 12:36
Copy link
Copy Markdown
Member

@flying-sheep flying-sheep left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks great! Maybe link 1–2 of Dask’s guides/tutorials, especially in the “Custom block-wise array operations” section.

I just directly committed other things I saw, please check that everything I did is OK with you:

  • I reverted the addition of hatch.toml, as we already have the docs env defined in pyproject.toml. I moved your additions over there instead.
  • I replaced the bold paragraphs acting as headers with actual headers.
  • I fixed a link to our code (I’m pretty sure I got the line number you meant right).
  • I fixed some typos.

Comment thread docs/how-to-dask.md Outdated
Comment thread docs/how-to-dask.md Outdated
ilan-gold and others added 2 commits May 14, 2025 12:30
Co-authored-by: Severin Dicks <37635888+Intron7@users.noreply.github.com>
Co-authored-by: Severin Dicks <37635888+Intron7@users.noreply.github.com>
@ilan-gold ilan-gold merged commit 8c6c9bc into main May 15, 2025
2 checks passed
@ilan-gold ilan-gold deleted the ig/dask-docs branch May 15, 2025 13:37
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Dask docs

3 participants