[WIP] LanceDB dataloader#100

Open

AyushExel wants to merge 43 commits into

NVIDIA:mainfrom

AyushExel:lancedb-dataloader

Commits on Jun 23, 2026

Add LanceDB-powered dataloaders for the 3 Cosmos training loaders

AyushExel
and
claude
committed
docs: add action-loader disk-footprint table to README

AyushExel
and
claude
committed
experiments: filtered-sampling (predicate pushdown) benchmark + docs

AyushExel
and
claude
committed
validate: train-equivalence test (Lance vs base loader, same model)

AyushExel
and
claude
committed
validate: multi-GPU per-epoch timing (training is compute-bound here)

AyushExel
and
claude
committed
validate: data-bound regime demo (S3 + tiny compute) shows Lance 1.74x faster epoch

AyushExel
and
claude
committed
validate: decent-epochs real training, base vs lance outputs identical

AyushExel
and
claude
committed
experiments: borrow episode-shuffle + lance-optimal knobs (audit)

AyushExel
and
claude
committed
merge working branch into experiments (bring train_* demos)
AyushExel
committed
experiments: add lance-episode mode to data-bound demo + honest re-measure

AyushExel
and
claude
committed
perf: batch take_blobs across a batch's episodes (was per-episode loop)

AyushExel
and
claude
committed
fix: honest faithful action numbers (base-episode baseline, local vs S3)

AyushExel
and
claude
committed

Commits on Jun 24, 2026

Commits on Jul 1, 2026

Commits on Jul 2, 2026

Commits on Jul 9, 2026