Container image is very large - can we reduce size?

[tamuri]

In an effort to improve the wait time of simulations starting on Azure Batch run, I have noticed that the image size has ballooned from 1.7GB to 10.6GB. I need to dig further into how the Batch accounts allocate space for VM images, but want to get ahead of any out-of-space errors.

The increase in container size is caused by the pytorch, Nvidia CUDA (and related) packages etc. which have come in via the SDV dependency (#1758).

@marghe-molaro - I'm wondering whether we need all these libraries when running the simulation, as opposed to training the emulator.

[tamuri]

nvidia-cublas-cu12: 527.32 MB
nvidia-cuda-cupti-cu12: 42.92 MB
nvidia-cuda-nvrtc-cu12: 62.81 MB
nvidia-cuda-runtime-cu12: 4.35 MB
nvidia-cudnn-cu12: 975.93 MB
nvidia-cufft-cu12: 280.36 MB
nvidia-curand-cu12: 94.12 MB
nvidia-cusolver-cu12: 193.44 MB
nvidia-cusparse-cu12: 268.63 MB
nvidia-nccl-cu12: 240.01 MB
nvidia-nvjitlink-cu12: 51.19 MB
nvidia-nvtx-cu12: 0.40 MB

torch: 1709.16 MB

triton: 554.44 MB

[marghe-molaro]

Hi @tamuri, thanks for pointing this out, and agreed that a lot of these packages are only relevant to training rather than using the emulator.

Proposed path forward:

• Short term solution: keep a copy of the working tlo+sdv environment that was brought in Add SDV as dependency and update requirements file #1758, but reverse this in master to reduce the image size on Azure immediately.
• I will look into extracting the subset of packages within the sdv env actually required to load and sample the library of emulators; eventually these will be the only packages to be added to the tlo environment. This can be brought in with PR Include option to use emulator in RTI module #1756 ?
• I am currently working on automating the whole data generation/training for emulators of different disease modules, so we can revisit the idea of a joint tlo+sdv environment and how that would link to master/tlo's Azure account at a later stage;

Does this sound ok?

[tamuri]

Thanks, that sounds good. I will revert the changes for the SDV deps. but you can use the requirements file from that PR to setup your local environment.

[tamuri]

Fixed by #1796 and #1797