You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: docs/user_guide/examples/explanation_performance.md
+5-5Lines changed: 5 additions & 5 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -6,7 +6,7 @@ In this tutorial, we will show how to squeeze performance in Parcels by using a
6
6
7
7
## Option 1: explicitly load the full FieldSet into memory
8
8
9
-
**Works well for: small Datasets (less than a few GB)**
9
+
**Best for: small Datasets (less than a few GB)**
10
10
11
11
For relatively small Datasets (less than a few GB), it is possible to load the entire FieldSet into memory. This can be done by calling the `load()` method on the `xarray.Dataset` object:
12
12
@@ -24,7 +24,7 @@ This will make Parcels use `numpy` functions in the interpolation routines, whic
24
24
25
25
## Option 2: use cached zarr files
26
26
27
-
**Works well for: large Datasets (more than a few GB) and particles distributed over a small part of the domain**
27
+
**Best for: large Datasets (more than a few GB) and particles distributed over a small part of the domain**
28
28
29
29
If your Dataset is too large to fit into memory, but your particles are only distributed over a small part of the domain, it could be efficient to use cached zarr files. This can be done by using the (experimental) `zarr.CacheStore` in combination with the `parcels.open_raw_zarr()` function. This will make Parcels only load the chunks that are needed for the particles, and cache these chunks in memory for future use.
30
30
@@ -51,7 +51,7 @@ In our performance testing, we have found that using zarr files saved without an
51
51
52
52
## Option 3: use `fieldset.to_windowed_arrays()`
53
53
54
-
**Works well for: large Datasets (more than a few GB) and particles distributed over the entire domain**
54
+
**Best for: large Datasets (more than a few GB) and particles distributed over the entire domain**
55
55
56
56
If your Dataset is so large that it doesn't fit into memory, you can use the `fieldset.to_windowed_arrays()` method to make Parcels only hold two timeslices in memory. Note that this only works if the two timeslices still fit into memory.
57
57
@@ -70,7 +70,7 @@ fieldset.to_windowed_arrays()
70
70
71
71
## Option 4: use Dask
72
72
73
-
**Works well for: large Datasets (more than a few GB) and small ParticleSets (less than a few hundred particles)**
73
+
**Best for: large Datasets (more than a few GB) and small ParticleSets (less than a few hundred particles)**
74
74
75
75
If your Dataset is so large that it doesn't fit into memory, and you have very few particles, you can use Dask to perform the interpolation operations. In this case, you don't have to do any special setup, as Parcels will automatically use Dask if the `xarray.Dataset` is a Dask array.
76
76
@@ -81,5 +81,5 @@ If your Dataset is so large that it doesn't fit into memory, and you have very f
81
81
| Works out-of-the-box | Only performs well for very small ParticleSets |
82
82
83
83
```{note}
84
-
The long-term plan for Parcles development is to make this Option 4 work well for all cases. However, this will require significant work on Dask indexing.
84
+
The long-term plan for Parcels development is to make this Option 4 work well for all cases. However, this will require significant work on Dask indexing.
0 commit comments