Skip to content

Commit 5d13f6e

Browse files
Fix figure layout warnings and tighten the scale-test prose
Set layout=tight at figure creation instead of calling tight_layout on a constrained-layout figure, which emitted a warning into the committed outputs. Declare pyarrow via extra_dependencies and the standard install note. State that the Criteo numbers come from a run outside the notebook, fix the out-of-memory extrapolation to follow from the stated measurements, and correct two sentences that overstated what stays in memory.
1 parent 2d9771e commit 5d13f6e

2 files changed

Lines changed: 112 additions & 112 deletions

File tree

examples/variational_inference/streaming_dataset.ipynb

Lines changed: 86 additions & 91 deletions
Large diffs are not rendered by default.

examples/variational_inference/streaming_dataset.myst.md

Lines changed: 26 additions & 21 deletions
Original file line numberDiff line numberDiff line change
@@ -9,6 +9,9 @@ kernelspec:
99
display_name: Python 3 (ipykernel)
1010
language: python
1111
name: python3
12+
myst:
13+
substitutions:
14+
extra_dependencies: pyarrow
1215
---
1316

1417
(streaming_dataset)=
@@ -47,6 +50,11 @@ bounded buffer.
4750
`N` is small here so the notebook runs in seconds. The streaming code is the same at
4851
any size, and the last section shows what changes at scale.
4952

53+
+++
54+
55+
:::{include} ../extra_installs.md
56+
:::
57+
5058
```{code-cell} ipython3
5159
import glob
5260
import tempfile
@@ -68,9 +76,9 @@ az.style.use("arviz-variat")
6876
## Put a dataset on disk and forget the array
6977

7078
We build a logistic-regression dataset, write it to Parquet shards, and delete the
71-
in-memory table. From here the features only exist on disk. We keep `X` and `y`
72-
around only to build the in-RAM `pm.Minibatch` baseline later; the streaming fit
73-
never reads them.
79+
in-memory table. From here the streaming path reads only the disk copy. We keep `X`
80+
and `y` around only to build the in-RAM `pm.Minibatch` baseline later; the streaming
81+
fit never reads them.
7482

7583
```{code-cell} ipython3
7684
N = 30_000
@@ -130,8 +138,7 @@ with pm.Model() as model:
130138
idata_stream = approx.sample(1000)
131139
```
132140

133-
The negative-ELBO trace shows the fit converging while only ever holding a
134-
`batch_size` buffer in memory:
141+
The negative-ELBO trace shows the fit converging on minibatches read off disk:
135142

136143
```{code-cell} ipython3
137144
fig, ax = plt.subplots(figsize=(9, 3))
@@ -164,15 +171,14 @@ bs_stream = idata_stream.posterior["b"].values.reshape(-1, 4)
164171
bs_inram = idata_inram.posterior["b"].values.reshape(-1, 4)
165172
names = ["intercept", "slope x1", "slope x2", "slope x3"]
166173
167-
fig, axes = plt.subplots(1, 4, figsize=(13, 3))
174+
fig, axes = plt.subplots(1, 4, figsize=(13, 3), layout="tight")
168175
for k, ax in enumerate(axes):
169176
ax.hist(bs_stream[:, k], bins=40, density=True, alpha=0.5, label="streaming")
170177
ax.hist(bs_inram[:, k], bins=40, density=True, alpha=0.5, label="in-RAM")
171178
ax.axvline(b_true[k], color="k", ls="--", lw=1)
172179
ax.set(title=names[k], yticks=[])
173180
axes[0].legend(fontsize=8)
174-
fig.suptitle("Posterior of b: streaming vs in-RAM (dashed = ground truth)", y=1.04)
175-
fig.tight_layout();
181+
fig.suptitle("Posterior of b: streaming vs in-RAM (dashed = ground truth)", y=1.04);
176182
```
177183

178184
## Memory usage
@@ -186,31 +192,30 @@ cost. The line below is its lower bound (`N * ncols * 8` bytes), not a measureme
186192
ncols = 4 # 3 features + observed
187193
n_grid = np.logspace(5, 9, 50)
188194
inram_gb = n_grid * ncols * 8 / 1e9 # whole dataset resident (array lower bound)
189-
stream_gb = np.full_like(n_grid, batch_size * ncols * 8 / 1e9) # just the buffer
195+
stream_gb = np.full_like(n_grid, batch_size * ncols * 8 / 1e9) # one resident batch
190196
191-
fig, ax = plt.subplots(figsize=(8, 5))
197+
fig, ax = plt.subplots(figsize=(8, 5), layout="tight")
192198
ax.loglog(n_grid, inram_gb, lw=2.5, label="in-RAM pm.Minibatch (O(N))")
193199
ax.loglog(n_grid, stream_gb, lw=2.5, label="streaming DataLoader (O(batch))")
194200
ax.axhline(26, color="0.5", ls="--", lw=1)
195201
ax.text(n_grid[-1], 30, "26 GB RAM", color="0.5", ha="right", va="bottom")
196202
ax.set_xlabel("dataset size N")
197203
ax.set_ylabel("array footprint (GB, lower bound)")
198204
ax.set_title("Memory is flat in N when streaming")
199-
ax.legend(loc="lower right", framealpha=0.95)
200-
fig.tight_layout();
205+
ax.legend(loc="lower right", framealpha=0.95);
201206
```
202207

203208
That line is only the bare array. Actual peak RSS is higher, because of the
204-
framework and PyTensor's resident copy, and it hits the RAM ceiling sooner. To get
205-
the real number on public data, we measured peak memory on the
209+
framework and PyTensor's resident copy, and it hits the RAM ceiling sooner. As a
210+
real-data check, outside this notebook, we ran the same logistic model (13 numeric
211+
features plus the click label) on the
206212
[Criteo 1TB Click Logs](https://huggingface.co/datasets/criteo/CriteoClickLogs), a
207-
standard out-of-core learning benchmark, with the same logistic model (13 numeric
208-
features plus the click label). Streaming through the `DataLoader` stayed flat at
209-
about 0.7 GB across a sweep from 1M to 150M rows. The in-RAM `pm.Minibatch` baseline
210-
rose linearly to 15.7 GB at 150M rows, about 21 times more, and extrapolates to
211-
out-of-memory near 238M rows on a 26 GB machine. The streaming and in-RAM posteriors agree coefficient for coefficient; the
212-
largest gap is about 0.1, on the intercept. Criteo is
213-
public, so anyone can rerun this.
213+
standard, publicly available out-of-core learning benchmark. Peak memory for the
214+
streaming `DataLoader` stayed flat at about 0.7 GB across a sweep from 1M to 150M
215+
rows, while the in-RAM `pm.Minibatch` baseline rose linearly to 15.7 GB at 150M
216+
rows, which extrapolates to out-of-memory around 250M rows on the same 26 GB
217+
machine. The streaming and in-RAM posteriors agreed coefficient for coefficient;
218+
the largest gap was about 0.1, on the intercept.
214219

215220
## When to use it
216221

0 commit comments

Comments
 (0)