Skip to content

Commit 4d80156

Browse files
authored
Merge branch 'main' into bump_v8
2 parents 3faf425 + c6e328b commit 4d80156

33 files changed

Lines changed: 65 additions & 34 deletions

File tree

eng/docker-tools/DEV-GUIDE.md

Lines changed: 25 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -143,6 +143,7 @@ Build Stage
143143
144144
Post_Build Stage
145145
├── Merge image info files
146+
├── Create multi-arch manifests
146147
└── Consolidate SBOMs
147148
148149
@@ -191,13 +192,35 @@ Common patterns:
191192
- `"publish"` - Publish only (when re-running a failed publish from a previous build)
192193
- `"build,test,sign,publish"` - Full pipeline
193194
194-
**Note:** The `Post_Build` stage is implicitly included whenever `build` is in the stages list. You don't need to specify it separately—it automatically runs after Build to merge image info files and consolidate SBOMs.
195+
**Note:** The `Post_Build` stage is implicitly included whenever `build` is in the stages list. You don't need to specify it separately—it automatically runs after Build to merge image info files, create and validate multi-arch manifests, and consolidate SBOMs.
195196

196197
The stages variable is useful for:
197198
- Re-running just the publish stage after fixing a transient failure
198199
- Skipping tests during initial development
199200
- Running isolated stages for debugging
200201

202+
### Decoupling build OS from the base image OS
203+
204+
By default, a platform's `osVersion` represents the base image OS version, but also determines what
205+
build leg an image is built in. This can cause problems when build image and base image don't match
206+
up. For example, building a .NET app on Windows Server 2025 and copying the artifacts into a
207+
Windows Server 2019 base image won't work, because the build matrix generation will attempt to
208+
build the image on the Server 2019 build leg (which can't run Server 2025 images).
209+
210+
To fix this, set the optional `buildOsVersion` field in order to override only the OS used in the
211+
build matrix generation. Here is an example of building a Windows Server 2019 image using Windows
212+
Server 2025:
213+
214+
```jsonc
215+
{
216+
// ...
217+
"os": "windows",
218+
"osVersion": "windowsservercore-ltsc2019",
219+
"buildOsVersion": "windowsservercore-ltsc2025"
220+
// ...
221+
}
222+
```
223+
201224
### Image Info Files: The Build's Memory
202225

203226
Image info files (defined by [`ImageArtifactDetails`](https://github.com/dotnet/docker-tools/blob/main/src/ImageBuilder/Models/Image/ImageArtifactDetails.cs)) are the mechanism that tracks what was built:
@@ -458,7 +481,7 @@ If the Dockerfile is in the manifest but you don't see a build job for it, the b
458481
```yaml
459482
windowsLtsc2025Amd64:
460483
src-windowsservercore-ltsc2025-helix-graph:
461-
imageBuilderPaths: --path src/windowsservercore/ltsc2025/helix/amd64 --path src/windowsservercore/ltsc2025/helix/webassembly/amd64
484+
imageBuilderPaths: --path src/windowsservercore/ltsc2025/helix/amd64 --path src/windowsservercore/ltsc2025/helix/webassembly-net8/amd64 --path src/windowsservercore/ltsc2025/helix/webassembly/amd64
462485
legName: windows-ltsc2025amd64src-windowsservercore-ltsc2025-helix-graph
463486
osType: windows
464487
architecture: amd64

eng/pipelines/steps/pip-authenticate.yml

Lines changed: 6 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -10,7 +10,12 @@ steps:
1010
artifactFeeds: public/dotnet-public-pypi,public/helix-client-prod
1111
onlyAddExtraIndex: false
1212
- powershell: |
13+
# common.yml seeds imageBuilderBuildArgs with the anonymous PIP_INDEX_URL so
14+
# public pipelines can build. On internal builds, replace that default with
15+
# the authenticated URL produced by PipAuthenticate above (it embeds a PAT),
16+
# rather than appending a duplicate --build-arg.
1317
$options = "$(imageBuilderBuildArgs)"
14-
$options += " --build-arg PIP_INDEX_URL=$env:PIP_INDEX_URL --build-arg NPM_TOKEN=$(System.AccessToken)"
18+
$options = $options -replace '--build-arg PIP_INDEX_URL=\S+', "--build-arg PIP_INDEX_URL=$env:PIP_INDEX_URL"
19+
$options += " --build-arg NPM_TOKEN=$(System.AccessToken)"
1520
echo "##vso[task.setvariable variable=imageBuilderBuildArgs]$options"
1621
displayName: Set PIP and NPM Build Args

eng/pipelines/variables/common.yml

Lines changed: 4 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -24,7 +24,10 @@ variables:
2424
- name: publishReadme
2525
value: false
2626
- name: imageBuilderBuildArgs
27-
value: ''
27+
# Default build args. PIP_INDEX_URL defaults to the public PyPI (anonymous and
28+
# unthrottled, unlike the Azure Artifacts mirror) and is overwritten with an
29+
# authenticated mirror URL by pip-authenticate.yml in internal pipelines.
30+
value: '--build-arg PIP_INDEX_URL=https://pypi.org/simple/ --build-arg HELIX_FEED=https://dnceng.pkgs.visualstudio.com/public/_packaging/helix-client-prod/pypi/simple'
2831
- name: publicGitRepoUri
2932
value: https://github.com/dotnet/dotnet-buildtools-prereqs-docker
3033
- name: publicSourceBranch

src/almalinux/10/helix/amd64/Dockerfile

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -2,6 +2,7 @@ FROM library/almalinux:10 AS venv
22

33
ARG PIP_INDEX_URL
44
ENV PIP_INDEX_URL=${PIP_INDEX_URL}
5+
ARG HELIX_FEED
56

67
RUN dnf upgrade --refresh -y \
78
&& dnf install --setopt tsflags=nodocs -y \
@@ -13,7 +14,6 @@ RUN dnf upgrade --refresh -y \
1314

1415
RUN python3 -m venv /venv \
1516
&& . /venv/bin/activate \
16-
&& HELIX_FEED=https://dnceng.pkgs.visualstudio.com/public/_packaging/helix-client-prod/pypi/simple \
1717
&& pip download --no-deps --index-url $HELIX_FEED --extra-index-url $PIP_INDEX_URL helix-scripts \
1818
&& pip install --index-url $PIP_INDEX_URL ./helix_scripts-*-py3-none-any.whl \
1919
&& rm ./helix_scripts-*-py3-none-any.whl

src/almalinux/8/helix/amd64/Dockerfile

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -2,6 +2,7 @@ FROM library/almalinux:8
22

33
ARG PIP_INDEX_URL
44
ENV PIP_INDEX_URL=${PIP_INDEX_URL}
5+
ARG HELIX_FEED
56

67
RUN dnf upgrade --refresh -y \
78
&& dnf install --setopt tsflags=nodocs -y \
@@ -38,7 +39,6 @@ ENV VIRTUAL_ENV=/home/helixbot/.vsts-env
3839

3940
RUN python -m venv $VIRTUAL_ENV && \
4041
${VIRTUAL_ENV}/bin/pip install --upgrade pip setuptools && \
41-
HELIX_FEED=https://dnceng.pkgs.visualstudio.com/public/_packaging/helix-client-prod/pypi/simple && \
4242
${VIRTUAL_ENV}/bin/pip download --no-deps --index-url $HELIX_FEED --extra-index-url $PIP_INDEX_URL helix-scripts && \
4343
${VIRTUAL_ENV}/bin/pip install --index-url $PIP_INDEX_URL ./helix_scripts-*-py3-none-any.whl && \
4444
rm ./helix_scripts-*-py3-none-any.whl

src/almalinux/9/helix/amd64/Dockerfile

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -2,6 +2,7 @@ FROM library/almalinux:9 AS venv
22

33
ARG PIP_INDEX_URL
44
ENV PIP_INDEX_URL=${PIP_INDEX_URL}
5+
ARG HELIX_FEED
56

67
RUN dnf upgrade --refresh -y \
78
&& dnf install --setopt tsflags=nodocs -y \
@@ -14,7 +15,6 @@ RUN dnf upgrade --refresh -y \
1415
RUN python3 -m venv /venv \
1516
&& . /venv/bin/activate \
1617
&& pip install --upgrade pip setuptools \
17-
&& HELIX_FEED=https://dnceng.pkgs.visualstudio.com/public/_packaging/helix-client-prod/pypi/simple \
1818
&& pip download --no-deps --index-url $HELIX_FEED --extra-index-url $PIP_INDEX_URL helix-scripts \
1919
&& pip install --index-url $PIP_INDEX_URL ./helix_scripts-*-py3-none-any.whl \
2020
&& rm ./helix_scripts-*-py3-none-any.whl

src/alpine/3.23/helix/Dockerfile

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -2,6 +2,7 @@ FROM library/alpine:3.23 AS venv
22

33
ARG PIP_INDEX_URL
44
ENV PIP_INDEX_URL=${PIP_INDEX_URL}
5+
ARG HELIX_FEED
56

67
RUN apk add --upgrade --no-cache \
78
cargo \
@@ -14,7 +15,6 @@ RUN apk add --upgrade --no-cache \
1415

1516
RUN python3 -m venv /venv && \
1617
source /venv/bin/activate && \
17-
HELIX_FEED=https://dnceng.pkgs.visualstudio.com/public/_packaging/helix-client-prod/pypi/simple && \
1818
pip download --no-deps --index-url $HELIX_FEED --extra-index-url $PIP_INDEX_URL helix-scripts && \
1919
pip install --index-url $PIP_INDEX_URL ./helix_scripts-*-py3-none-any.whl && \
2020
rm ./helix_scripts-*-py3-none-any.whl

src/alpine/3.24/helix/Dockerfile

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -2,6 +2,7 @@ FROM library/alpine:3.24 AS venv
22

33
ARG PIP_INDEX_URL
44
ENV PIP_INDEX_URL=${PIP_INDEX_URL}
5+
ARG HELIX_FEED
56

67
RUN apk add --upgrade --no-cache \
78
cargo \
@@ -14,7 +15,6 @@ RUN apk add --upgrade --no-cache \
1415

1516
RUN python3 -m venv /venv && \
1617
source /venv/bin/activate && \
17-
HELIX_FEED=https://dnceng.pkgs.visualstudio.com/public/_packaging/helix-client-prod/pypi/simple && \
1818
pip download --no-deps --index-url $HELIX_FEED --extra-index-url $PIP_INDEX_URL helix-scripts && \
1919
pip install --index-url $PIP_INDEX_URL ./helix_scripts-*-py3-none-any.whl && \
2020
rm ./helix_scripts-*-py3-none-any.whl

src/alpine/edge/helix/Dockerfile

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -2,6 +2,7 @@ FROM library/alpine:edge AS venv
22

33
ARG PIP_INDEX_URL
44
ENV PIP_INDEX_URL=${PIP_INDEX_URL}
5+
ARG HELIX_FEED
56

67
RUN apk add --upgrade --no-cache \
78
cargo \
@@ -14,7 +15,6 @@ RUN apk add --upgrade --no-cache \
1415

1516
RUN python3 -m venv /venv && \
1617
source /venv/bin/activate && \
17-
HELIX_FEED=https://dnceng.pkgs.visualstudio.com/public/_packaging/helix-client-prod/pypi/simple && \
1818
pip download --no-deps --index-url $HELIX_FEED --extra-index-url $PIP_INDEX_URL helix-scripts && \
1919
pip install --index-url $PIP_INDEX_URL ./helix_scripts-*-py3-none-any.whl && \
2020
rm ./helix_scripts-*-py3-none-any.whl

src/azurelinux/3.0/helix/Dockerfile

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -2,6 +2,7 @@ FROM mcr.microsoft.com/azurelinux/base/core:3.0 as venv
22

33
ARG PIP_INDEX_URL
44
ENV PIP_INDEX_URL=${PIP_INDEX_URL}
5+
ARG HELIX_FEED
56

67
RUN tdnf install --refresh -y \
78
build-essential \
@@ -14,7 +15,6 @@ RUN tdnf install --refresh -y \
1415

1516
RUN python3 -m venv /venv && \
1617
source /venv/bin/activate && \
17-
HELIX_FEED=https://dnceng.pkgs.visualstudio.com/public/_packaging/helix-client-prod/pypi/simple && \
1818
pip download --no-deps --index-url $HELIX_FEED --extra-index-url $PIP_INDEX_URL helix-scripts && \
1919
pip install --index-url $PIP_INDEX_URL ./helix_scripts-*-py3-none-any.whl && \
2020
rm ./helix_scripts-*-py3-none-any.whl

0 commit comments

Comments
 (0)