You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: docs/how-to-dask.md
+4-3Lines changed: 4 additions & 3 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -9,9 +9,9 @@ Here we will go through some common questions and answers about `dask`, with a s
9
9
If you are in a jupyter notebook, when you render the `repr` of your `client`, you will see a link, usually something like `http://localhost:8787/status`.
10
10
If you are working locally, this link alone should suffice.
11
11
12
-
If you are working on some sort of remote notebook from a web browser, you will need to replace `http://localhost:8787` by the root url of the notebook.
12
+
If you are working on some sort of remote notebook from a web browser, you will need to replace `http://localhost` by the root url of the notebook.
13
13
14
-
If you are in vscode, there is an[`dask` extension] which will allow you to monitor there.
14
+
If you are in vscode, there is a[`dask` extension] which will allow you to monitor there.
15
15
16
16
**How do I know how to allocate resources?**
17
17
@@ -68,7 +68,8 @@ These are the summed together to produce a single `(n_vars,n_vars)` matrix, whic
68
68
Because `dask` does not implement matrix multiplication for sparse-in-dask, we do it ourselves.
69
69
We use `map_blocks` over a CSR sparse-in-dask array where the chunking looks something like `(chunk_size, n_vars)`.
70
70
When we compute the invdividual block's gram matrix, we add an axis via `[None, ...]` so that we can sum over that axis i.e., the `da.map_blocks` call produces a `(n_obs // chunk_size, n_vars, n_vars)` sized-matrix which is summed over the first dimension.
71
-
However, to make this work, we need to be very specific about how `da.map_blocks` expects its result to look like, done via `new_axis` and `chunks``new_axis` indicates that we are adding a single new axis at the front.
71
+
However, to make this work, we need to be very specific about how `da.map_blocks` expects its result to look like, done via `new_axis` and `chunks`.
72
+
`new_axis` indicates that we are adding a single new axis at the front.
72
73
The `chunks` argument specifices that the output of `da.map_blocks` should have `x.blocks.size` number of `(1, n_vars, n_vars)` matrixes.
73
74
This `chunks` argument thus allows the inferral of the shape of the output.
0 commit comments