MIP-3D finetune env: rebase to openmpi5.0-cuda12.6-ubuntu24.04 base (#5032)

* Environment files for MIP3D FT

* Updated azure-mlflow version; added some manual upgrades

* changing azureml-mlflow

* MIP-3D finetune env: rebase to openmpi5.0-cuda12.6-ubuntu24.04 base

Builds on chingiz-kb/mip3d-ft-env. Replaces the ACPT-based image with the
openmpi-base curated AzureML image (same family olympus uses). Two reasons:

1. ACPT ships on a biweekly tag cadence and we cannot patch CVEs against it
   on our side -- the v7 attempt (single-pip-install with explicit
   pip/wheel/setuptools/cryptography/PyJWT/aiohttp/requests/python-dotenv
   upgrades in both PTCA and base envs) still left scanner findings against
   ffmpeg / libcjson / libzvbi / libmbedtls / .NET runtime that ACPT itself
   has to fix in a base-image rebuild. We do not own that cadence.

2. mcr.microsoft.com/azureml/openmpi5.0-cuda12.6-ubuntu24.04 is on a real OS
   patch cadence and has the openmpi-base curated-env layout AML training
   already supports. After dropping ffmpeg (no video codepaths in MIP3D --
   NIFTI/DICOM only) and libaio-dev (deepspeed dropped), the v2 scan went
   from 15 findings -> 1 (a known marshmallow MED 5.3 we are pinning around
   for azure-ai-ml 1.16.1 _T import compat).

Dockerfile changes:
- FROM mcr.microsoft.com/azureml/openmpi5.0-cuda12.6-ubuntu24.04:{{latest-image-tag}}
- AZUREML_CONDA_ENVIRONMENT_PATH=/azureml-envs/acft-medimageparse-3d-finetune,
  matching the sibling openmpi curated envs (sklearn-1.5, automl forecasting-gpu).
- Three-step install: conda create py3.10 + pip 26.1; pip install torch==2.8.0
  cu126 wheels from the pytorch index; pip install -r requirements.txt.
- Final RUN sanity-checks the fragile import chains (transformers/peft and
  marshmallow/azure-ai-ml MLClient) so import breakage fails the build, not
  the AML run.
- Apt set narrowed to the libgl/libsm/libxext/libxrender/libfontconfig set
  that opencv-headless + matplotlib + pillow need at runtime.

requirements.txt changes:
- Drop deepspeed==0.15.1: not used. Active Lightning strategy in
  olympus_biomed_parse_3d/configs/trainer/biomedparse_trainer.yaml is
  ddp_find_unused_parameters_true; deepspeed strategy / optimizer config
  blocks are commented out. Saves ~200MB and avoids JIT-compile risk.
- Drop ffmpeg==1.4 (python wrapper): unused. The OS ffmpeg apt tree was
  responsible for 11/12 findings on the v1 openmpi scan.
- Bump transformers to >=5.0.0,<6.0.0 to fix CVE-2026-1839 (torch.load RCE
  in _load_rng_state). Per Qualys advisory, only 5.0.0rc3+ has the fix.
- Bump peft to >=0.19.1,<0.20.0: 0.17.1 hard-imports HybridCache at module
  load (removed in transformers 5.x); 0.19.1 moved that inside a function.
- Pin marshmallow==3.23.2 (azure-ai-ml 1.16.1 imports `_T` from
  marshmallow.fields, removed in 3.24.0). The CVE-2025-68480 risk
  (DoS in Schema.load(many=True)) does not apply to a training entrypoint
  that does not accept untrusted input.
- Drop the explicit base-env conda upgrade RUN step (no separate base env on
  the openmpi image -- the conda env we create is the only env).

Validated: built and pushed openmpi-test-20260513-184353 to ACR, scan shows
1 finding (the marshmallow pin documented above), and submitted an AML
training job on hls-4xA100 that pulled the image, activated the env,
imported the wheel, converted the safetensors checkpoint, and entered
training.

---------

Co-authored-by: chingiz-kb <ckabytayev+github@microsoft.com>

Author: sidkiblawi

assets/training/finetune_acft_image/environments/acft_image_medimageparse_3d_finetune/asset.yaml

@@ -0,0 +1,6 @@
+name: acft-medimageparse-3d-finetune
+version: auto
+type: environment
+spec: spec.yaml
+extra_config: environment.yaml
+categories: ["PyTorch", "Training"]

assets/training/finetune_acft_image/environments/acft_image_medimageparse_3d_finetune/context/Dockerfile

@@ -0,0 +1,72 @@
+# MIP-3D finetune image based on a maintained AzureML base image instead of ACPT.
+#
+# Base: mcr.microsoft.com/azureml/openmpi5.0-cuda12.6-ubuntu24.04
+#   - Same family olympus uses
+#   - On a real OS-patch cadence (vs ACPT biweekly tags shipping unpatched USNs)
+#
+# Pattern follows the existing azureml-assets openmpi-base curated envs (e.g.
+# training/general/environments/sklearn-1.5, training/automl/.../forecasting-gpu):
+#   FROM openmpi-base
+#   conda create -p $AZUREML_CONDA_ENVIRONMENT_PATH python=3.10
+#   conda run -p ... pip install <deps>
+#
+# AML training infra reads $AZUREML_CONDA_ENVIRONMENT_PATH at startup, so this
+# env-path layout is effectively required for curated envs.
+
+FROM mcr.microsoft.com/azureml/openmpi5.0-cuda12.6-ubuntu24.04:{{latest-image-tag}}
+
+USER root
+
+ENV AZUREML_CONDA_ENVIRONMENT_PATH=/azureml-envs/acft-medimageparse-3d-finetune
+ENV CONDA_PREFIX=$AZUREML_CONDA_ENVIRONMENT_PATH
+ENV CONDA_DEFAULT_ENV=$AZUREML_CONDA_ENVIRONMENT_PATH
+ENV PATH=$AZUREML_CONDA_ENVIRONMENT_PATH/bin:$PATH
+# mpi needs to locate libpython from the conda env
+ENV LD_LIBRARY_PATH=$AZUREML_CONDA_ENVIRONMENT_PATH/lib:$LD_LIBRARY_PATH
+
+# OS deps + general housekeeping. Base is on a real cadence so apt upgrade should
+# usually be a no-op (vs ACPT where we had to manually patch openssl/curl/.NET).
+# NOTE: ffmpeg & its libav* / cjson / zvbi / mbedtls deps were responsible for
+# 11/12 findings on the v1 openmpi scan; MIP3D is NIFTI/DICOM (no video decode),
+# so we drop them entirely. libaio-dev was for deepspeed (also dropped). The
+# remaining libgl/libsm/libxext/libxrender/libfontconfig set is what opencv-headless
+# + matplotlib + pillow runtime need.
+RUN apt-get update && \
+    DEBIAN_FRONTEND=noninteractive apt-get -y upgrade && \
+    apt-get install -y --no-install-recommends \
+        ca-certificates \
+        curl \
+        git \
+        libfontconfig1 \
+        libgl1 \
+        libice6 \
+        libsm6 \
+        libxext6 \
+        libxrender1 \
+    && apt-get clean && \
+    rm -rf /var/lib/apt/lists/*
+
+# Step 1: minimal conda env with python 3.10 (matches azureml-acft-image-components compat).
+# Keep this step pure conda-channel — the SAT solver OOMs on ACR Standard build
+# agents when conda + pip resolution run together. Sibling envs split for the
+# same reason.
+RUN conda create -p $AZUREML_CONDA_ENVIRONMENT_PATH -q -y \
+        python=3.10 pip=26.1 \
+        -c conda-forge && \
+    conda clean -a -y
+
+# Step 2: torch from the pytorch CUDA index (split out so pip doesn't pull CPU-only wheels).
+RUN conda run -p $AZUREML_CONDA_ENVIRONMENT_PATH pip install --no-cache-dir \
+        --index-url https://download.pytorch.org/whl/cu126 \
+        torch==2.8.0 torchvision==0.23.0 torchaudio==2.8.0
+
+# Step 3: everything else from requirements.txt.
+WORKDIR /workspace
+COPY requirements.txt .
+RUN conda run -p $AZUREML_CONDA_ENVIRONMENT_PATH pip install --no-cache-dir -r requirements.txt
+
+# Sanity-check the import chains we know are fragile (transformers/peft, marshmallow/azure-ai-ml).
+# If these break, fail the image build instead of failing at AML runtime.
+# Single-line RUN (no backslash continuations) so ACR Tasks' dependency scanner
+# does not try to parse the continuation lines as Dockerfile instructions.
+RUN conda run -p $AZUREML_CONDA_ENVIRONMENT_PATH python -c "import torch, transformers, peft; from marshmallow.fields import _T; from azure.ai.ml import MLClient; print('torch', torch.__version__, 'transformers', transformers.__version__, 'peft', peft.__version__, 'azure-ai-ml import ok')"

assets/training/finetune_acft_image/environments/acft_image_medimageparse_3d_finetune/context/requirements.txt

@@ -0,0 +1,67 @@
+# MIP-3D finetune env requirements.
+# torch / torchvision / torchaudio are installed in the Dockerfile from the
+# pytorch cu126 index (so they are not listed here).
+
+# Core scientific stack
+numpy==1.26.4
+pandas==2.2.2
+hydra-core==1.3.2
+marshmallow==3.23.2  # pinned for azure-ai-ml 1.16.1 _T import compat
+
+# Azure ML SDK
+azure-ai-ml==1.16.1
+azure-identity==1.16.1
+azureml-mlflow==1.62.0.post2
+azureml-dataprep==5.4.1
+
+# Tracking / logging
+mlflow-skinny==3.9.0
+
+# DL training stack
+timm==0.9.16
+opencv-python-headless==4.11.0.86
+# deepspeed: not used by MIP-3D training path. Active Lightning strategy in
+# olympus_biomed_parse_3d/configs/trainer/biomedparse_trainer.yaml is
+# "ddp_find_unused_parameters_true"; deepspeed strategy + optimizer configs are
+# all commented out. Dropping saves ~200MB and avoids JIT-compile risk.
+transformers>=5.0.0,<6.0.0
+open-clip-torch==2.26.1
+sentencepiece==0.2.1
+peft>=0.19.1,<0.20.0
+ninja==1.11.1.1
+kornia==0.7.3
+lightning==2.5.5
+
+# Domain / IO
+pydicom==2.4.5
+SimpleITK==2.4.0
+scikit-image==0.24.0
+nltk==3.9.4
+h5py==3.14.0
+
+# Utilities
+python-dotenv==1.2.2
+einops==0.8.0
+mup==1.0.0
+arrow==1.3.0
+dotmap==1.3.30
+appdirs==1.4.4
+# ffmpeg python wrapper dropped: no video codepaths in MIP3D (NIFTI/DICOM only)
+# and the OS ffmpeg pkg accounts for 11/12 scan findings.
+filelock>=3.20.1
+
+# Build tooling pins (CVE / compatibility)
+wheel==0.46.2
+setuptools>=82.0.1
+protobuf==6.33.5
+cryptography==46.0.7
+pillow==12.2.0
+
+# Transitive CVE pins
+aiohttp==3.13.4
+onnx==1.21.0
+requests==2.33.0
+pytest==9.0.3
+
+# MIP-3D component (built from AzureMlCli, includes olympus_biomed_parse_3d)
+azureml-acft-image-components[mip3d]=={{latest-pypi-version}}

assets/training/finetune_acft_image/environments/acft_image_medimageparse_3d_finetune/environment.yaml

@@ -0,0 +1,12 @@
+image:
+  name: azureml/curated/acft-medimageparse-3d-finetune
+  os: linux
+  context:
+    dir: context
+    dockerfile: Dockerfile
+    template_files:
+    - Dockerfile
+    - requirements.txt
+  publish:
+    location: mcr
+    visibility: public

assets/training/finetune_acft_image/environments/acft_image_medimageparse_3d_finetune/spec.yaml

@@ -0,0 +1,16 @@
+$schema: https://azuremlschemas.azureedge.net/latest/environment.schema.json
+
+description: AzureML ACFT MedImageParse 3d Image for Training
+
+
+name: "{{asset.name}}"
+version: "{{asset.version}}"
+
+build:
+  path: "{{image.context.path}}"
+  dockerfile_path: "{{image.dockerfile.path}}"
+
+os_type: linux
+
+tags:
+  Preview: ""