Skip to content

Commit 12f92fc

Browse files
State the loader memory behavior precisely in the intro
1 parent d77a391 commit 12f92fc

2 files changed

Lines changed: 6 additions & 6 deletions

File tree

examples/variational_inference/streaming_dataset.ipynb

Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -28,9 +28,9 @@
2828
"the same structure as PyTorch's `torch.utils.data`:\n",
2929
"\n",
3030
"* a {class}`~pymc.variational.streaming.DataLoader` batches (and optionally\n",
31-
" shuffles) an out-of-core source into fixed-size minibatches, holding only one\n",
32-
" batch in memory at a time. Here the source is a directory of Parquet shards read\n",
33-
" by {func}`~pymc.variational.streaming.parquet_source`.\n",
31+
" shuffles) an out-of-core source into fixed-size minibatches without loading\n",
32+
" the whole dataset into memory. Here the source is a directory of Parquet shards\n",
33+
" read by {func}`~pymc.variational.streaming.parquet_source`.\n",
3434
"* the model observes a `pm.Data` placeholder of one batch, not the whole array.\n",
3535
"* a {class}`~pymc.variational.streaming.Trainer` drives ADVI, writing each\n",
3636
" minibatch into that placeholder with `set_data` every step. There are no callbacks.\n",

examples/variational_inference/streaming_dataset.myst.md

Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -30,9 +30,9 @@ This notebook uses the streaming API in `pymc.variational.streaming`, which foll
3030
the same structure as PyTorch's `torch.utils.data`:
3131

3232
* a {class}`~pymc.variational.streaming.DataLoader` batches (and optionally
33-
shuffles) an out-of-core source into fixed-size minibatches, holding only one
34-
batch in memory at a time. Here the source is a directory of Parquet shards read
35-
by {func}`~pymc.variational.streaming.parquet_source`.
33+
shuffles) an out-of-core source into fixed-size minibatches without loading
34+
the whole dataset into memory. Here the source is a directory of Parquet shards
35+
read by {func}`~pymc.variational.streaming.parquet_source`.
3636
* the model observes a `pm.Data` placeholder of one batch, not the whole array.
3737
* a {class}`~pymc.variational.streaming.Trainer` drives ADVI, writing each
3838
minibatch into that placeholder with `set_data` every step. There are no callbacks.

0 commit comments

Comments
 (0)