Skip to content

Commit 40b21eb

Browse files
roehrich-hperoehrich-hpe
authored
Update to match latest DWS/NNF (#7)
 Signed-off-by: Dean Roehrich <dean.roehrich@hpe.com>
1 parent 24b5c79 commit 40b21eb

8 files changed

Lines changed: 356 additions & 76 deletions

File tree

‎.github/workflows/main.yml‎

Lines changed: 61 additions & 45 deletions
Original file line numberDiff line numberDiff line change
@@ -1,16 +1,6 @@
11
name: Docker build and push
22

3-
on:
4-
push:
5-
branches:
6-
- '*'
7-
tags:
8-
- 'v*'
9-
pull_request:
10-
branches:
11-
- 'master'
12-
- 'releases/v*'
13-
- 'feature/*'
3+
on: [push]
144

155
env:
166
# TEST_TARGET: Name of the testing target in the Dockerfile
@@ -27,30 +17,49 @@ jobs:
2717
runs-on: ubuntu-latest
2818

2919
steps:
30-
- uses: actions/checkout@v2
20+
- uses: actions/checkout@v4
21+
with:
22+
fetch-depth: 0
23+
ref: ${{ github.event.pull_request.head.sha }}
3124

3225
- name: Lowercase repository name for docker build
3326
id: lowercase-repository-name
3427
run: |
35-
echo "HPE_BUILD_REPO=$(echo ${{ github.repository }} | tr '[:upper:]' '[:lower:]')" >> $GITHUB_ENV
36-
echo "HPE_DEPLOY_REPO=$(echo ${{ github.repository }} | tr '[:upper:]' '[:lower:]')" >> $GITHUB_ENV
37-
38-
- name: set TAG & TESTTAG for main/master
39-
if: ${{ github.event.repository.default_branch == 'main' || github.event.repository.default_branch == 'master' }}
40-
run: |
41-
echo "TAG=${{ github.sha }}" >> ${GITHUB_ENV}
42-
echo "LATESTTAG=latest" >> ${GITHUB_ENV}
43-
echo "TESTTAG=test-${{ github.sha }}" >> ${GITHUB_ENV}
28+
echo "REPO_NAME=$(echo ${{ github.repository }} | tr '[:upper:]' '[:lower:]')" >> $GITHUB_ENV
4429
45-
- name: set TAG & TESTTAG for all other branches
46-
if: ${{ github.event.repository.default_branch != 'main' && github.event.repository.default_branch != 'master' }}
30+
- name: "Set tags for main/master"
31+
id: set_tags
4732
run: |
48-
echo "TAG=dev-${{ github.sha }}" >> ${GITHUB_ENV}
49-
echo "LATESTTAG=dev-latest" >> ${GITHUB_ENV}
50-
echo "TESTTAG=test-${{ github.sha }}" >> ${GITHUB_ENV}
33+
echo "VERSION_TAG=$(./git-version-gen | grep -v UNKNOWN)" >> ${GITHUB_ENV}
34+
echo "TEST_TAG=$(git rev-parse HEAD)-test" >> ${GITHUB_ENV}
35+
echo "SHA_TAG=$(git rev-parse HEAD)" >> ${GITHUB_ENV}
36+
echo "${GITHUB_ENV}:"
37+
cat ${GITHUB_ENV}
38+
shell: bash
39+
40+
- name: "Docker metadata"
41+
id: meta
42+
uses: docker/metadata-action@v5
43+
with:
44+
images: |
45+
ghcr.io/${{ env.REPO_NAME }}
46+
tags: |
47+
# For merge to master branch, tag example: 'master'
48+
type=ref,event=branch
49+
# For PR event, tag example: 'pr-3'
50+
type=ref,event=pr
51+
# For PR event or merge event, tag example: 1.0.1.12-5667
52+
type=raw,value=${{ env.VERSION_TAG }}
53+
# For PR event or merge, tag example: 566769e04d2436cf5f42ae4f46092c7dff6e668e
54+
type=raw,value=${{ env.SHA_TAG }}
55+
# For push to semver tag, tag example: 1.0.2
56+
# This also sets 'latest'.
57+
type=semver,pattern={{version}}
58+
# For push to semver tag, tag example: 1.0
59+
type=semver,pattern={{major}}.{{minor}}
5160
5261
- name: Docker login
53-
uses: docker/login-action@v1
62+
uses: docker/login-action@v3
5463
with:
5564
registry: ghcr.io
5665
username: ${{ github.actor }}
@@ -59,36 +68,43 @@ jobs:
5968
- name: Build the test Docker image
6069
if: ${{ env.DO_TEST == 'true' }}
6170
id: docker_build_test_target
62-
uses: docker/build-push-action@v2
71+
uses: docker/build-push-action@v6
6372
with:
6473
push: false
6574
target: ${{ env.TEST_TARGET }}
66-
tags: ${{ env.HPE_BUILD_REPO }}:${{ env.TESTTAG }}
75+
tags: ${{ env.REPO_NAME }}:${{ env.TEST_TAG }}
6776

6877
- name: Run the Docker image unit tests
6978
if: ${{ env.DO_TEST == 'true' }}
70-
run: docker run ${HPE_BUILD_REPO}:${TESTTAG}
79+
run: docker run ${REPO_NAME}:${TEST_TAG}
7180

7281
- name: Build the final Docker image
7382
id: docker_build
74-
uses: docker/build-push-action@v2
83+
uses: docker/build-push-action@v6
7584
with:
76-
push: false
77-
tags: ${{ env.HPE_BUILD_REPO }}:${{ env.TAG }}
85+
push: true
86+
tags: ${{ steps.meta.outputs.tags }}
7887

7988
- name: Peek at the docker images
8089
run: docker images
8190

82-
- name: Tag the build
83-
run: docker tag docker.io/${HPE_BUILD_REPO}:${TAG} ghcr.io/${HPE_DEPLOY_REPO}:${TAG}
84-
85-
- name: Tag the build as latest
86-
run: docker tag ${HPE_BUILD_REPO}:${TAG} ghcr.io/${HPE_DEPLOY_REPO}:${LATESTTAG}
87-
88-
- name: Push the tagged build
89-
if: ${{ env.DO_PUSH == 'true' }}
90-
run: docker push ghcr.io/${HPE_DEPLOY_REPO}:${TAG}
91+
create_release:
92+
needs: build
93+
if: startsWith(github.ref, 'refs/tags/v')
94+
runs-on: ubuntu-latest
95+
steps:
96+
- name: Checkout
97+
uses: actions/checkout@v4
98+
with:
99+
fetch-tags: true
100+
fetch-depth: 0
101+
- name: Repair tag
102+
run: git fetch -f origin ${{ github.ref }}:${{ github.ref }}
103+
- name: Verify that the tag is annotated
104+
run: if test x$(git for-each-ref ${{ github.ref }} | awk '{print $2}') = xtag; then /bin/true; else echo "\"${{ github.ref }}\" does not look like an annotated tag!"; /bin/false; fi
105+
- name: Release
106+
uses: softprops/action-gh-release@v1
107+
with:
108+
#prerelease: true
109+
generate_release_notes: true
91110

92-
- name: Push the build tagged as latest
93-
if: ${{ env.DO_PUSH == 'true' }}
94-
run: docker push ghcr.io/${HPE_DEPLOY_REPO}:${LATESTTAG}

‎.gitignore‎

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,2 @@
1+
.version
2+

‎Makefile‎

Lines changed: 27 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -1,3 +1,28 @@
1+
# IMAGE_TAG_BASE defines the docker.io namespace and part of the image name for remote images.
2+
# This variable is used to construct full image tags for bundle and catalog images.
3+
#
4+
# For example, running 'make bundle-build bundle-push catalog-build catalog-push' will build and push both
5+
# cray.com/nnf-sos-bundle:$VERSION and cray.com/nnf-sos-catalog:$VERSION.
6+
IMAGE_TAG_BASE ?= ghcr.io/nearnodeflash/nnf-container-example
7+
8+
# CONTAINER_TOOL defines the container tool to be used for building images.
9+
# Be aware that the target commands are only tested with Docker which is
10+
# scaffolded by default. However, you might want to replace it to use other
11+
# tools. (i.e. podman)
12+
CONTAINER_TOOL ?= docker
13+
14+
all: docker-build
15+
16+
docker-build: VERSION ?= $(shell cat .version)
17+
docker-build: .version
18+
$(CONTAINER_TOOL) build . -t $(IMAGE_TAG_BASE):$(VERSION)
19+
20+
# Let .version be phony so that a git update to the workarea can be reflected
21+
# in it each time it's needed.
22+
.PHONY: .version
23+
.version: ## Uses the git-version-gen script to generate a tag version
24+
./git-version-gen --fallback `git rev-parse HEAD` > .version
25+
26+
clean:
27+
rm -f .version
128

2-
all:
3-
docker build . -t nnf-container-example:0.0.1

‎README.md‎

Lines changed: 18 additions & 13 deletions
Original file line numberDiff line numberDiff line change
@@ -69,7 +69,7 @@ storage is persistent storage, then it must start with `DW_PERSISTENT` rather th
6969

7070
```yaml
7171
---
72-
apiVersion: nnf.cray.hpe.com/v1alpha1
72+
apiVersion: nnf.cray.hpe.com/v1alpha2
7373
kind: NnfContainerProfile
7474
metadata:
7575
name: demo
@@ -88,7 +88,7 @@ Next, we define the container specification. For MPI applications, this is done
8888
`mpiSpec` allows us to define the Launcher and Worker containers.
8989

9090
```yaml
91-
apiVersion: nnf.cray.hpe.com/v1alpha1
91+
apiVersion: nnf.cray.hpe.com/v1alpha2
9292
kind: NnfContainerProfile
9393
metadata:
9494
name: demo
@@ -104,22 +104,25 @@ data:
104104
spec:
105105
containers:
106106
- name: nnf-container-example
107-
image: nnf-container-example:master
107+
image: ghcr.io/nearnodeflash/nnf-container-example:master
108108
command:
109109
- mpirun
110+
- --tag-output
110111
- mpi_hello_world
111112
- "$(DW_JOB_my_storage)"
112113
Worker:
113114
template:
114115
spec:
115116
containers:
116117
- name: nnf-container-example
117-
image: nnf-container-example:master
118+
image: ghcr.io/nearnodeflash/nnf-container-example:master
118119
```
119120

120121
Both the `Launcher` and `Worker` must be defined. The main pieces here are to set the images for both
121122
and the command for the `Launcher`. Boiled down, these are just Kubernetes [PodTemplateSpecs](https://pkg.go.dev/k8s.io/api/core/v1#PodTemplateSpec).
122123

124+
The container image tag may need to be changed from "master" to match the tag for your build.
125+
123126
Our `mpi_hello_world` application takes in a command line argument for the storage file path. We are
124127
using the name of the storage we defined above in the `storages` object to pass into our command:
125128

@@ -251,15 +254,17 @@ You can use kubectl to inspect the log to get your application's output:
251254

252255
```shell
253256
$ kubectl logs demo-container-launcher-wcvcs
254-
Defaulted container "nnf-container-example" out of: nnf-container-example, mpi-wait-for-worker-0 (init), mpi-wait-for-worker-1 (init), mpi-init-passwd (init)
255-
Warning: Permanently added 'demo-container-worker-1.demo-container-worker.default.svc,10.244.1.6' (ECDSA) to the list of known hosts.
256-
Warning: Permanently added 'demo-container-worker-0.demo-container-worker.default.svc,10.244.3.5' (ECDSA) to the list of known hosts.
257-
Hello world from processor demo-container-worker-1, rank 1 out of 2 processors. NNF Storage path: /mnt/nnf/100db033-c9f2-4cf8-b085-505aebf571c1-0, hostname: demo-container-worker-1
258-
rank 1: test file: /mnt/nnf/100db033-c9f2-4cf8-b085-505aebf571c1-0/0/testfile
259-
rank 1: wrote file to '/mnt/nnf/100db033-c9f2-4cf8-b085-505aebf571c1-0/0/testfile'
260-
Hello world from processor demo-container-worker-0, rank 0 out of 2 processors. NNF Storage path: /mnt/nnf/100db033-c9f2-4cf8-b085-505aebf571c1-0, hostname: demo-container-worker-0
261-
rank 0: test file: /mnt/nnf/100db033-c9f2-4cf8-b085-505aebf571c1-0/0/testfile
262-
rank 0: wrote file to '/mnt/nnf/100db033-c9f2-4cf8-b085-505aebf571c1-0/0/testfile'
257+
Defaulted container "nnf-container-example" out of: nnf-container-example, mpi-init-passwd (init), mpi-wait-for-worker-2 (init)
258+
Warning: Permanently added '[demo-container-worker-1.demo-container.default.svc]:2222,[10.42.3.146]:2222' (ECDSA) to the list of known hosts.
259+
Warning: Permanently added '[demo-container-worker-0.demo-container.default.svc]:2222,[10.42.2.118]:2222' (ECDSA) to the list of known hosts.
260+
[1,0]<stdout>:Hello world from processor demo-container-worker-0, rank 0 out of 2 processors. NNF Storage path: /mnt/nnf/7f33f3bf-4559-4dda-892c-ff6148116a08-0, hostname: demo-container-worker-0
261+
[1,0]<stdout>:Found indexed dir: /mnt/nnf/7f33f3bf-4559-4dda-892c-ff6148116a08-0/rabbit-node-1-0
262+
[1,0]<stdout>:rank 0: test file: /mnt/nnf/7f33f3bf-4559-4dda-892c-ff6148116a08-0/rabbit-node-1-0/testfile
263+
[1,1]<stdout>:Hello world from processor demo-container-worker-1, rank 1 out of 2 processors. NNF Storage path: /mnt/nnf/7f33f3bf-4559-4dda-892c-ff6148116a08-0, hostname: demo-container-worker-1
264+
[1,1]<stdout>:Found indexed dir: /mnt/nnf/7f33f3bf-4559-4dda-892c-ff6148116a08-0/rabbit-node-2-0
265+
[1,1]<stdout>:rank 1: test file: /mnt/nnf/7f33f3bf-4559-4dda-892c-ff6148116a08-0/rabbit-node-2-0/testfile
266+
[1,0]<stdout>:rank 0: wrote file to '/mnt/nnf/7f33f3bf-4559-4dda-892c-ff6148116a08-0/rabbit-node-1-0/testfile'
267+
[1,1]<stdout>:rank 1: wrote file to '/mnt/nnf/7f33f3bf-4559-4dda-892c-ff6148116a08-0/rabbit-node-2-0/testfile'
263268
```
264269

265270
You can see that the Storage path is passed into the container application and printed to the log:

‎allocation-servers.yaml‎

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -4,6 +4,6 @@ spec:
44
label: gfs2
55
storage:
66
- name: rabbit-node-1
7-
allocationCount: 2
7+
allocationCount: 1
88
- name: rabbit-node-2
9-
allocationCount: 2
9+
allocationCount: 1

0 commit comments

Comments
 (0)