lightning

Imagenette Dataset with Lightning

Dependencies

This example requires the following packages, which are not installed by the main plugin but are already present in the Docker images:

• lightning
• torch
• torchvision

Starting and filling the DB

Once the Docker container is set up, it is possible to start the database and populate it with images from the imagenette dataset using the provided script within the Docker container:

./fill-db.sh  # might take a few minutes

In this example we will import the Imagenette2-320 dataset (a subset of ImageNet) as a Cassandra dataset and then read the data into NVIDIA DALI.

The raw files are already present in the /tmp directory of the provided Docker container, from which the following commands can be run.

Multi-GPU training

Run the training of the Imagenette dataset with the lightning application with:

$ python3 train_model.py --num-gpu NUM_GPUS \
  -a resnet50 --b 64 --workers 4 --lr=1.0e-3 \
  --train-data-table imagenette.data_train --train-rows-fn train.rows \
  --val-data-table imagenette.data_val --val-rows-fn val.rows