Add ephemeral OS disk support to Azure Batch pools

[adamrtalbot]

Why

Azure Batch supports ephemeral OS disks, which place the node OS disk on local/cache storage instead of remote Azure storage. This improves node performance and lowers cost. nf-azure had no way to enable it.

What

• Add azure.batch.pools.<name>.ephemeralOsDisk config option (boolean, default false)
• poolVmConfig() attaches an OSDisk with ephemeral settings on the cache disk (DiffDiskPlacement.CACHE_DISK) when enabled
• Include the flag in the pool cache funnel, so autopool pool IDs account for it (toggling provisions a new pool rather than reusing an existing one)
• Add unit tests for both config parsing and VM config generation
• Document the new option in config.mdx

Notes

• The selected VM size must support an ephemeral OS disk with a cache large enough for the OS image, otherwise pool creation fails.
• Adding the flag to the funnel changes autopool pool IDs; on upgrade, existing autopools are not reused.

🤖 Generated with Claude Code

[netlify]

✅ Deploy Preview for nextflow-docs ready!

Name	Link
🔨 Latest commit	8bb45f3
🔍 Latest deploy log	https://app.netlify.com/projects/nextflow-docs/deploys/6a3d0567ec79060008c20a37
😎 Deploy Preview	https://deploy-preview-7259--nextflow-docs.netlify.app
📱 Preview on mobile	Toggle QR Code... Use your smartphone camera to open QR code link.

---

To edit notification comments on pull requests, go to your Netlify project configuration.

[adamrtalbot]

This behaviour is worse than normal.

With ephemeral disks, we carve out a virtual 64GB of the data disk which is used for the OS. This can't be resized, so this is effectively worse than having a separate OS disk to store docker containers which we can bump to 1023GB.

# without ephemeral OS disk:

NAME    MAJ:MIN RM  SIZE RO TYPE MOUNTPOINTS
sda       8:0    0   64G  0 disk
|-sda1    8:1    0   63G  0 part /etc/ssl/certs
|                                /etc/rdma/dat.conf
|                                /etc/hosts
|                                /etc/hostname
|                                /etc/resolv.conf
|                                /etc/pki
|                                /etc/sudoers
|                                /etc/group
|                                /etc/passwd
|                                /etc/dat.conf
|-sda14   8:14   0    4M  0 part
|-sda15   8:15   0  106M  0 part
`-sda16 259:0    0  913M  0 part
sdb       8:16   0  150G  0 disk
`-sdb1    8:17   0  150G  0 part /mnt/batch/tasks
sr0      11:0    1  630K  0 rom
Filesystem      Size  Used Avail Use% Mounted on
overlay          61G   43G   19G  70% /
tmpfs            64M     0   64M   0% /dev
shm              64M     0   64M   0% /dev/shm
/dev/root        61G   43G   19G  70% /etc/pki
/dev/sdb1       147G   41M  140G   1% /mnt/batch/tasks
tmpfs            16G     0   16G   0% /proc/acpi
tmpfs            16G     0   16G   0% /proc/scsi
tmpfs            16G     0   16G   0% /sys/firmware
# with ephemeral OS disk:

NAME    MAJ:MIN RM  SIZE RO TYPE MOUNTPOINTS
sda       8:0    0   64G  0 disk
|-sda1    8:1    0   63G  0 part /etc/rdma/dat.conf
|                                /etc/ssl/certs
|                                /etc/hosts
|                                /etc/hostname
|                                /etc/resolv.conf
|                                /etc/sudoers
|                                /etc/group
|                                /etc/passwd
|                                /etc/dat.conf
|                                /etc/pki
|-sda14   8:14   0    4M  0 part
|-sda15   8:15   0  106M  0 part
`-sda16 259:0    0  913M  0 part
sdb       8:16   0   86G  0 disk
`-sdb1    8:17   0   86G  0 part /mnt/batch/tasks
sr0      11:0    1  630K  0 rom
Filesystem      Size  Used Avail Use% Mounted on
overlay          61G   43G   19G  70% /
tmpfs            64M     0   64M   0% /dev
shm              64M     0   64M   0% /dev/shm
/dev/root        61G   43G   19G  70% /etc/pki
/dev/sdb1        85G   41M   80G   1% /mnt/batch/tasks <- note this is smaller!
tmpfs            16G     0   16G   0% /proc/acpi
tmpfs            16G     0   16G   0% /proc/scsi
tmpfs            16G     0   16G   0% /sys/firmware

Classic Microsoft 🙄

[bentsherman]

do you still want to add the config option? seems like a simple addition, but I'd rather not add another lever if it is strictly worse

[adamrtalbot]

Let's not merge. Handling local storage is already an absolute nightmare on Azure and an extra variable for people to get wrong isn't going to help