[fix]: proj 24 25 and training images vulnerability fixes (#4977)

* [feat]: Fix image vulnerabilities

* Update Dockerfile to fix vulnerabilities and upgrade packages

Removed onnx installation and upgraded several packages including onnx to version 1.21.0 to address vulnerabilities.

* fix: add CVE justifications and missing security pins for acpt-rft and acpt-pytorch-2.8

acpt-rft Dockerfile:
- Add per-package CVE/GHSA justification comments for all security overrides
- Document transitive dep chains explaining why parent upgrades don't resolve each CVE

acpt-pytorch-2.8-cuda12.6 Dockerfile:
- cryptography >=46.0.5 -> >=46.0.7 (CVE-2026-41727)
- Add python-dotenv>=1.2.2, requests>=2.33.0, urllib3>=2.6.3 to base conda env
- Add per-package CVE justification comments with dep chain analysis

acpt-pytorch-2.8-cuda12.6 requirements.txt:
- requests: unpinned -> >=2.33.0 (GHSA-gc5v-m9x4-r6x2)
- Add starlette>=0.49.1, wheel>=0.46.2, protobuf>=6.33.5
- Add onnx>=1.21.0 (5 GHSAs, parent onnxruntime uses >=1.16.0)
- Add python-dotenv>=1.2.2 (CVE-2026-28684, pydantic-settings only requires >=0.21.0)
- Add PyJWT>=2.12.0 (CVE-2026-32597, msal/azureml-core use loose floors)

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* chore: align ai-ml-automl-dnn-text-gpu-ptca Dockerfile with main

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* fix: add python-dotenv CVE justification and pin across all conda envs in automl-dnn-vision-gpu

python-dotenv>=1.2.2: CVE-2026-28684 (GHSA-mf9w-mj56-hr94); transitive dep chain:
  azureml-defaults -> azureml-inference-server-http -> pydantic-settings
  -> python-dotenv>=0.21.0. pydantic-settings (all versions through 2.14.0)
  only requires >=0.21.0, no parent upgrade resolves this.

- Added to base conda env (was present, added justification comment)
- Added to ptca conda env (was missing)
- Added to active conda env (was missing)

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* [fix]: Update vulnerability fixes

* chore: revert ai-ml-automl-dnn-text-gpu Dockerfile to match main

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* fix: upgrade transformers to >=5.0.0rc3 in acpt-rft for GHSA-69w3-r845-3855

CVE-2026-1839: arbitrary code execution in Trainer._load_rng_state() via
torch.load() without weights_only=True. Fixed in transformers 5.0.0rc3.

- requirements.txt: 4.57.6 -> >=5.0.0rc3 (direct dep, major version bump)
- Dockerfile: added transformers>=5.0.0rc3 to security override RUN
- Removed stale comment claiming 5.0.0rc3 not on PyPI (latest is 5.6.2)

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

---------

Co-authored-by: Yeshwanth Nagaraj <ynagaraj@microsoft.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: HarshaVardhanBabu <hnamburi@microsoft.com>

Author: 4 people

assets/training/finetune_acft_hf_nlp/environments/acpt-rft/context/Dockerfile

@@ -1,8 +1,10 @@
 FROM mcr.microsoft.com/aifx/acpt/stable-ubuntu2204-cu126-py310-torch280:{{latest-image-tag:biweekly\.\d{6}\.\d{1}.*}}
 USER root
-# Security: upgrade all OS packages and remove stale kernel headers to patch USN vulnerabilities
+# Security: upgrade all OS packages and patch USN-8176-1 (.NET 8.0 vulnerabilities)
 RUN apt-get update && \
     DEBIAN_FRONTEND=noninteractive apt-get -y upgrade && \
+    DEBIAN_FRONTEND=noninteractive apt-get install -y --only-upgrade \
+        dotnet-host-8.0 dotnet-hostfxr-8.0 dotnet-runtime-8.0 2>/dev/null || true && \
     apt-get autoremove -y && \
     apt-get autoclean && \
     apt-get clean && \
@@ -42,16 +44,31 @@ RUN pip install --no-cache-dir 'xgrammar>=0.1.32'
 RUN pip install openai==2.14.0
 RUN pip install --force-reinstall --no-cache-dir --no-build-isolation git+https://github.com/deepseek-ai/DeepGEMM.git@c9f8b34dcdacc20aa746b786f983492c51072870
 RUN pip install https://github.com/yeshsurya/flash-attention/releases/download/v2.8.3-linux-1/flash_attn-2.8.3-cp310-cp310-linux_x86_64.whl
-# aiohttp: vllm/sglang use loose floors, pip can't force 3.13.4 transitively
-# protobuf: vllm uses loose floors (>=6.30.0), pip can't force 6.33.5 transitively
-# onnx: onnxruntime/azureml-acft-accelerator require onnx>=1.16.0; override needed (GHSA-p433-9wv8-28xj etc.)
-# fastmcp: GHSA-rww4-4w9c-7733, GHSA-m8x7-r2rg-vh5g, GHSA-vv7q-7jx5-f767; >=3.2.0 required
-# requests: transitive dep of azure-core/mlflow/transformers; parents use loose floors (GHSA-gc5v-m9x4-r6x2)
-RUN pip install --upgrade aiohttp==3.13.4 protobuf==6.33.5 setuptools==82.0.0 pip==26.0 wheel==0.46.2 cryptography==46.0.7 'requests>=2.33.0' 'onnx>=1.21.0' 'fastmcp>=3.2.0' 'Mako>=1.3.11' 'pytest>=8.4.2'
-# Fix vulnerabilities in base conda env (python3.13) from ACPT base image (biweekly.202603.1)
-# Still vulnerable in base: cryptography(44.0.1), pip(25.3), setuptools(80.9.0), wheel(0.45.1)
+# Fix security vulnerabilities in ptca conda env (active environment)
+# aiohttp==3.13.4: GHSA-63hf-3vf5-4wqf et al. (8 CVEs); transitive dep of vllm/sglang/azure SDK,
+#   parents use loose floors — pip can't force >=3.13.4 transitively
+# protobuf==6.33.5: CVE-2026-40186; transitive dep of vllm (requires >=6.30.0), parent uses loose floor
+# setuptools==82.0.0: GHSA-58pv-8j8x-9vj2 (vendored jaraco.context), GHSA-8rrh-rw8j-w5fx (vendored wheel)
+# pip==26.0: GHSA-4xh5-x5gv-qwph, GHSA-6vgw-5pg2-w6jp
+# wheel==0.46.2: CVE-2026-24049
+# cryptography==46.0.7: CVE-2026-41727; azureml-mlflow pins <46.0.0 but upgrading anyway for CVE fix
+# requests>=2.33.0: GHSA-gc5v-m9x4-r6x2; transitive dep of azure-core/mlflow/transformers,
+#   msal requires >=2.0.0,<3, azureml-core requires >=2.19.1,<3 — no parent pins safe floor
+# onnx>=1.21.0: GHSA-p433-9wv8-28xj, GHSA-538c-55jv-c5g9 et al.; transitive dep of onnxruntime/
+#   azureml-acft-accelerator (require >=1.16.0), no parent upgrade resolves this
+# fastmcp>=3.2.0: GHSA-rww4-4w9c-7733, GHSA-m8x7-r2rg-vh5g, GHSA-vv7q-7jx5-f767
+# Mako>=1.3.11: CVE-2025-46803; transitive dep of alembic, parent uses loose floor
+# pytest>=9.0.3: GHSA-6w46-j5rx-g56g; pre-installed in ptca env from base image, no parent to upgrade
+# lxml>=6.1.0: GHSA-vfmq-68hx-4jfw; transitive dep of multiple packages, parent uses loose floor
+# transformers>=5.0.0rc3: GHSA-69w3-r845-3855 (CVE-2026-1839); arbitrary code execution via torch.load()
+#   in Trainer._load_rng_state(). Direct dep — upgraded from 4.57.6 to patched 5.x
+RUN pip install --upgrade aiohttp==3.13.4 protobuf==6.33.5 setuptools==82.0.0 pip==26.0 wheel==0.46.2 cryptography==46.0.7 'requests>=2.33.0' 'onnx>=1.21.0' 'fastmcp>=3.2.0' 'Mako>=1.3.11' 'pytest>=9.0.3' 'lxml>=6.1.0' 'transformers>=5.0.0rc3'
+# Fix vulnerabilities in base conda env (python 3.13) from ACPT base image (biweekly.202603.1)
+# Base env ships: cryptography(44.0.1), pip(25.3), setuptools(80.9.0), wheel(0.45.1), aiohttp(3.12.x)
+# Same CVEs as ptca env above; base env is separate and must be patched independently
 RUN conda run -n base python -m pip install --no-cache-dir --upgrade pip==26.0 wheel==0.46.2 setuptools==82.0.0 cryptography==46.0.7 aiohttp==3.13.4
-# ray vendors aiohttp for runtime_env agent; use find to patch all copies regardless of path naming.
+# ray vendors its own copy of aiohttp inside thirdparty_files/ for runtime_env agent;
+# the vendored copy is not upgraded by pip install above. Patching all copies in-place.
 RUN find /opt/conda/envs/ptca/lib/python3.10/site-packages/ray -type d -name 'thirdparty_files' | while read dir; do \
       rm -rf "$dir"/aiohttp*; \
       pip install --no-cache-dir --target "$dir" 'aiohttp==3.13.4'; \

assets/training/finetune_acft_hf_nlp/environments/acpt-rft/context/requirements.txt

@@ -26,7 +26,7 @@ tensorboard==2.20.0
 tensordict==0.9.1
 torchdata==0.11.0
 torchvision==0.23.0
-transformers==4.57.6
+transformers>=5.0.0rc3
 uvicorn==0.35.0
 zmq==0.0.0
 filelock>=3.20.1

assets/training/finetune_acft_image/environments/acft_image_medimageparse_finetune/context/requirements.txt

@@ -16,7 +16,7 @@ sentencepiece==0.2.1
 peft==0.17.1
 ninja==1.11.1.1
 kornia==0.7.3
-python-dotenv==1.0.1
+python-dotenv==1.2.2
 einops==0.8.0 
 mup==1.0.0 
 pydicom>=2.4.5

assets/training/general/environments/acpt-pytorch-2.2-cuda12.1/context/Dockerfile

@@ -5,7 +5,8 @@ RUN conda install -n base -c conda-forge pip=26.0 -y
 # These are from base image adding till we get new base image tag
 # PyJWT 2.10.1 (CVE-2026-32597) is installed in the base conda env (python3.13) from ACPT base image; manually upgrading since base image hasn't been patched yet
 # aiohttp 3.12.14 (8 CVEs) and urllib3 2.5.0 (3 CVEs) are also in base env from base image
-RUN conda run -n base python -m pip install --upgrade 'wheel>=0.46.2' 'cryptography>=46.0.7' 'setuptools>=82.0.1' 'PyJWT>=2.12.0' 'aiohttp>=3.13.4' 'urllib3>=2.6.3' 'requests>=2.33.0'
+# python-dotenv 1.1.0 (GHSA-mf9w-mj56-hr94) is in base env from base image; no direct parent pip package pulls it
+RUN conda run -n base python -m pip install --upgrade 'wheel>=0.46.2' 'cryptography>=46.0.7' 'setuptools>=82.0.1' 'PyJWT>=2.12.0' 'aiohttp>=3.13.4' 'urllib3>=2.6.3' 'requests>=2.33.0' 'python-dotenv>=1.2.2'
 
 
 # Upgrade ptca environment conda packages

assets/training/general/environments/acpt-pytorch-2.8-cuda12.6/context/Dockerfile

@@ -28,19 +28,32 @@ ENV WORKER_TIMEOUT=400
 EXPOSE 5001 8883 8888
 
 # support Deepspeed launcher requirement of passwordless ssh login
-RUN apt-get update
-RUN apt-get install -y openssh-server openssh-client
+RUN apt-get update && \
+    apt-get install -y openssh-server openssh-client && \
+    apt-get upgrade -y && \
+    apt-get clean && rm -rf /var/lib/apt/lists/*
+
 
+# Fix security vulnerabilities in ptca conda env (active environment)
+# pip>=26.0: GHSA-4xh5-x5gv-qwph, GHSA-6vgw-5pg2-w6jp
+# wheel>=0.46.2: CVE-2026-24049
+# cryptography>=46.0.7: CVE-2026-41727 (NOTE: azureml-mlflow~=1.62.0 pins cryptography<46.0.0; upgrading anyway)
+# pillow>=12.1.1: GHSA-cfh3-3jmp-rvhc; transitive dep of matplotlib (requires >=8), no parent upgrade resolves this
+# protobuf>=6.33.5: CVE-2026-40186; transitive dep of onnxruntime/tensorboard, parents use loose floors
+# setuptools>=82.0.1: GHSA-58pv-8j8x-9vj2 (vendored jaraco.context), GHSA-8rrh-rw8j-w5fx (vendored wheel)
+# pytest>=9.0.3: GHSA-6w46-j5rx-g56g; pre-installed in ptca env from ACPT base image (7.4.3), no parent to upgrade
+RUN pip install --upgrade 'pip>=26.0' 'wheel>=0.46.2' 'cryptography>=46.0.7' 'pillow>=12.1.1' 'protobuf>=6.33.5' 'setuptools>=82.0.1' 'pytest>=9.0.3'
 
-# Fix security vulnerabilities
-# NOTE: azureml-mlflow~=1.62.0 pins cryptography<46.0.0; upgrading anyway for CVE fix
-# setuptools vendors jaraco.context internally; >=82.0.1 bundles the patched version (GHSA-58pv-8j8x-9vj2)
-# pytest 7.4.3 (GHSA-6w46-j5rx-g56g) is installed in ptca env from ACPT base image; overriding since base image hasn't been patched yet
-RUN pip install --upgrade 'pip>=26.0' 'wheel>=0.46.2' 'cryptography>=46.0.5' 'pillow>=12.1.1' 'protobuf>=6.33.5' 'setuptools>=82.0.1' 'pytest>=9.0.3'
-# vulnerability in base conda env
-# PyJWT 2.10.1 (CVE-2026-32597) is installed in the base conda env (python3.13) from ACPT base image; manually upgrading since base image hasn't been patched yet
-# aiohttp 3.13.3 (GHSA-63hf-3vf5-4wqf et al.) is in base conda env; upgrading to >=3.13.4 for multiple CVE fixes
-RUN conda run -n base python -m pip install --upgrade 'pip>=26.0' 'wheel>=0.46.2' 'cryptography>=46.0.5' 'pillow>=12.1.1' 'protobuf>=6.33.5' 'setuptools>=82.0.1' 'PyJWT>=2.12.0' 'aiohttp>=3.13.4'
+# Fix security vulnerabilities in base conda env (python 3.13)
+# Same CVEs as above for pip/wheel/cryptography/pillow/protobuf/setuptools, plus:
+# PyJWT>=2.12.0: CVE-2026-32597; transitive dep of msal (>=1.0.0,<3) and azureml-core (<3.0.0), no parent pins safe floor
+# aiohttp>=3.13.4: GHSA-63hf-3vf5-4wqf et al. (8 CVEs); transitive dep of azure SDK, parents use loose floors
+# python-dotenv>=1.2.2: CVE-2026-28684 (GHSA-mf9w-mj56-hr94); transitive dep chain: azureml-defaults
+#   -> azureml-inference-server-http -> pydantic-settings -> python-dotenv>=0.21.0.
+#   pydantic-settings (all versions through 2.14.0) only requires >=0.21.0, no parent upgrade resolves this
+# requests>=2.33.0: GHSA-gc5v-m9x4-r6x2; transitive dep of msal (>=2.0.0,<3) and azureml-core (>=2.19.1,<3), no parent pins safe floor
+# urllib3>=2.6.3: CVE-2026-37152; transitive dep of requests/azure SDK, parents use loose floors
+RUN conda run -n base python -m pip install --upgrade 'pip>=26.0' 'wheel>=0.46.2' 'cryptography>=46.0.7' 'pillow>=12.1.1' 'protobuf>=6.33.5' 'setuptools>=82.0.1' 'PyJWT>=2.12.0' 'aiohttp>=3.13.4' 'python-dotenv>=1.2.2' 'requests>=2.33.0' 'urllib3>=2.6.3'
 # pip install updates the binary but conda-meta still references old versions; conda install syncs both
 RUN conda install -y -n ptca pip>=26.0.1
 RUN conda clean -a -y && rm -rf /opt/miniconda/pkgs/

assets/training/general/environments/acpt-pytorch-2.8-cuda12.6/context/requirements.txt

@@ -10,7 +10,7 @@ MarkupSafe
 regex
 pybind11
 urllib3>=2.6.3
-requests
+requests>=2.33.0
 pillow>=12.1.1
 transformers
 aiohttp>=3.13.4
@@ -24,3 +24,18 @@ tqdm
 py-cpuinfo
 torch-tb-profiler
 filelock>=3.20.1
+starlette>=0.49.1
+wheel>=0.46.2
+protobuf>=6.33.5
+# onnx: transitive dep of onnxruntime in base image; parent uses loose floor (>=1.16.0);
+# override needed for GHSA-p433-9wv8-28xj, GHSA-538c-55jv-c5g9, GHSA-cmw6-hcpp-c6jp
+onnx>=1.21.0
+# python-dotenv: transitive dep chain: azureml-defaults -> azureml-inference-server-http
+#   -> pydantic-settings -> python-dotenv>=0.21.0. pydantic-settings (all versions through
+#   2.14.0) only requires >=0.21.0 which does not constrain to a CVE-safe version.
+#   No parent upgrade resolves this. Floor pin required to fix CVE-2026-28684 (GHSA-mf9w-mj56-hr94).
+python-dotenv>=1.2.2
+# PyJWT: transitive dep of msal (requires >=1.0.0,<3) and azureml-core (requires <3.0.0).
+#   No parent version constrains PyJWT to a CVE-safe floor.
+#   Floor pin required to fix CVE-2026-32597.
+PyJWT>=2.12.0

assets/training/vision/environments/automl-dnn-vision-gpu/context/Dockerfile

@@ -90,18 +90,22 @@ RUN pip install --upgrade 'onnx>=1.21.0'
 
 # Security: fix base conda env (python 3.13) — aiohttp (CVE-2026-37899), wheel (CVE-2026-24049),
 # cryptography (CVE-2026-41727), PyJWT (CVE-2026-32597), urllib3, filelock, pillow, bokeh
+# python-dotenv>=1.2.2: CVE-2026-28684 (GHSA-mf9w-mj56-hr94); transitive dep chain: azureml-defaults
+#   -> azureml-inference-server-http -> pydantic-settings -> python-dotenv>=0.21.0.
+#   pydantic-settings (all versions through 2.14.0) only requires >=0.21.0, no parent upgrade resolves this
 RUN /opt/conda/bin/pip install --no-cache-dir --upgrade 'requests>=2.33.0' 'urllib3>=2.6.3' 'aiohttp>=3.13.4' 'wheel>=0.46.2' \
     'setuptools>=82.0.1' 'cryptography>=46.0.7' 'PyJWT>=2.12.0' 'pip>=26.0' \
     'filelock>=3.20.3' 'pillow>=12.2.0' 'onnx>=1.21.0' \
-    'bokeh>=3.8.2'  # bokeh: conda env installs 2.4.3, override needed for GHSA-793v-589g-574v
+    'python-dotenv>=1.2.2' \
+    'bokeh>=3.8.2'
 # Security: fix ptca conda env — torch (CVE-2025-32434), protobuf (CVE-2026-40186),
 # wheel/setuptools, urllib3, filelock, pillow, PyJWT, bokeh overrides for conda env
 RUN /opt/conda/envs/ptca/bin/pip install --no-cache-dir --upgrade torch==2.8.0 torchvision==0.23.0 
 RUN /opt/conda/envs/ptca/bin/pip install --no-cache-dir --upgrade 'urllib3>=2.6.3' 'filelock>=3.20.3' \
     'wheel>=0.46.2' 'setuptools>=82.0.1' 'protobuf>=6.33.5' \
     'PyJWT>=2.12.0' 'pillow>=12.2.0' 'onnx>=1.21.0' 'requests>=2.33.0' \
     'aiohttp>=3.13.4' 'cryptography>=46.0.7' 'pytest>=9.0.3' \
-    'bokeh>=3.8.2'  # bokeh: conda env installs 2.4.3, override needed for GHSA-793v-589g-574v
+    'python-dotenv>=1.2.2' 'bokeh>=3.8.2'
 
 # Patch pillow vulnerability (GHSA-whj4-6x5x-4v2j) across all three conda environments
 RUN pip install --upgrade 'pillow>=12.2.0'
@@ -115,10 +119,13 @@ RUN /opt/conda/envs/ptca/bin/pip install --upgrade 'pillow>=12.2.0'
 # requests: transitive dep of azure SDK packages; parent uses template version, cannot upgrade parent
 # aiohttp: transitive dep of azure SDK; parent uses template version, cannot upgrade parent
 # pytest: dev dep in ptca base image; no parent to upgrade
+# python-dotenv>=1.2.2: CVE-2026-28684 (GHSA-mf9w-mj56-hr94); transitive dep chain: azureml-defaults
+#   -> azureml-inference-server-http -> pydantic-settings -> python-dotenv>=0.21.0.
+#   pydantic-settings (all versions through 2.14.0) only requires >=0.21.0, no parent upgrade resolves this
 RUN pip install --upgrade 'aiohttp>=3.13.4' 'distributed>=2026.1.0' 'protobuf>=6.33.5' 'pip>=26.0' 'cryptography>=46.0.7' \
     'filelock>=3.20.3' 'wheel>=0.46.2' 'setuptools>=82.0.1' 'PyJWT>=2.12.0' 'urllib3>=2.6.3' 'pillow>=12.2.0' \
-    'onnx>=1.21.0' 'requests>=2.33.0' \
-    'bokeh>=3.8.2'  # bokeh: conda env installs 2.4.3, override needed for GHSA-793v-589g-574v
+    'onnx>=1.21.0' 'requests>=2.33.0' 'python-dotenv>=1.2.2' \
+    'bokeh>=3.8.2'
 # Remove stale vendored metadata that scanners pick up
 RUN rm -rf /opt/conda/lib/python3.13/site-packages/setuptools/__vendor/jaraco.context-5.3.0.dist-info \
     /opt/conda/lib/python3.13/site-packages/setuptools/__vendor/wheel-0.45.1.dist-info \