Skip to content

Commit e4a80c4

Browse files
rokroskarolevski
rokroskar
authored and
olevski
committed
chore: add buildpack base images (#4086)
1 parent c0ea439 commit e4a80c4

8 files changed

Lines changed: 1722 additions & 4 deletions

File tree

‎.github/dependabot.yml‎

Lines changed: 5 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -1,18 +1,19 @@
11
version: 2
22
updates:
3-
- package-ecosystem: "pip"
4-
directories:
3+
- package-ecosystem: "pip"
4+
directories:
55
- "docs/"
66
- "scripts/init-realm"
77
- "acceptance-tests/"
8+
- "global-images/**"
89
schedule:
910
interval: "weekly"
1011
open-pull-requests-limit: 5
1112
ignore:
1213
- dependency-name: "python-keycloak"
13-
# I would like to replace this with "version-update:semver-patch", "version-update:semver-minor"
14+
# I would like to replace this with "version-update:semver-patch", "version-update:semver-minor"
1415
# but just want to test this first...
15-
update-types: ["version-update:semver-patch"]
16+
update-types: ["version-update:semver-patch"]
1617
groups:
1718
python:
1819
patterns:

‎.github/workflows/build-global-renkulab-images.yml‎

Lines changed: 54 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,54 @@
1+
name: Build global base images
2+
3+
on:
4+
push:
5+
paths:
6+
- "global-images/**"
7+
- ".github/workflows/build-global-renkulab-images.yml"
8+
env:
9+
DOCKER_PREFIX: ghcr.io/swissdatasciencecenter/renku
10+
BUILDER_IMAGE: ghcr.io/swissdatasciencecenter/renku-frontend-buildpacks/selector:0.0.6
11+
RUN_IMAGE: ghcr.io/swissdatasciencecenter/renku-frontend-buildpacks/base-image:0.0.6
12+
jobs:
13+
build-images:
14+
runs-on: ubuntu-24.04
15+
strategy:
16+
fail-fast: true
17+
matrix:
18+
FLAVOR:
19+
- basic
20+
- datascience
21+
FRONTEND:
22+
- jupyterlab
23+
- vscodium
24+
steps:
25+
- uses: actions/checkout@v4
26+
- name: Login to GitHub Container Registry
27+
uses: docker/login-action@v3
28+
with:
29+
registry: ghcr.io
30+
username: ${{ github.actor }}
31+
password: ${{ secrets.GITHUB_TOKEN }}
32+
- name: Set up Docker Buildx
33+
uses: docker/setup-buildx-action@v3
34+
- name: Docker meta
35+
id: meta
36+
uses: docker/metadata-action@v5
37+
with:
38+
images: ${{ env.DOCKER_PREFIX }}/py-${{ matrix.FLAVOR }}-${{ matrix.FRONTEND }}
39+
tags: |
40+
type=sha,prefix=
41+
type=raw,value=latest,enable=${{ github.ref == 'refs/heads/master' }}
42+
type=semver,pattern={{version}},event=tag
43+
- uses: buildpacks/github-actions/setup-pack@v5.0.0
44+
- name: Build image
45+
run: |
46+
tags=$(echo "${{ steps.meta.outputs.tags }}" | tr '\n' ',')
47+
tags=${tags::-1}
48+
TAGS=$tags \
49+
FRONTEND=${{ matrix.FRONTEND }} \
50+
RUN_IMAGE=${{ env.RUN_IMAGE }} \
51+
BUILDER=${{ env.BUILDER_IMAGE }} \
52+
PUBLISH=1 \
53+
EXTRA_FLAGS="--cache-image ${{ env.DOCKER_PREFIX }}/py-${{ matrix.FLAVOR }}-${{ matrix.FRONTEND }}-cache:latest" \
54+
make -C global-images ${{ matrix.FLAVOR }}

‎global-images/Makefile‎

Lines changed: 32 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,32 @@
1+
FRONTEND ?= vscodium
2+
BUILDER ?= ghcr.io/swissdatasciencecenter/renku-frontend-buildpacks/selector:0.0.6
3+
RUN_IMAGE ?= ghcr.io/swissdatasciencecenter/renku-frontend-buildpacks/base-image:0.0.6
4+
GIT_SHA := $(shell git rev-parse --short=7 HEAD)
5+
6+
.PHONY: all basic datascience
7+
8+
all: basic datascience
9+
10+
basic datascience:
11+
@BASE=$@; \
12+
PUBLISH_FLAG=""; \
13+
if [ -n "$$PUBLISH" ]; then PUBLISH_FLAG="--publish"; fi; \
14+
if [ -n "$$DOCKER_PREFIX" ]; then \
15+
case "$$DOCKER_PREFIX" in \
16+
*/) FINAL_PREFIX="$$DOCKER_PREFIX";; \
17+
*) FINAL_PREFIX="$$DOCKER_PREFIX/";; \
18+
esac; \
19+
else \
20+
FINAL_PREFIX=""; \
21+
fi; \
22+
TAGS=$${TAGS:-$${FINAL_PREFIX}py-$$BASE-$(FRONTEND):$(GIT_SHA)}; \
23+
echo "Building from \"$$BASE\" sub-directory"; \
24+
FIRST_TAG=$${TAGS%%,*}; \
25+
pack build $$FIRST_TAG \
26+
-p $$BASE \
27+
--builder $(BUILDER) \
28+
--run-image $(RUN_IMAGE) \
29+
--env BP_RENKU_FRONTENDS=$(FRONTEND) \
30+
--tag $$TAGS \
31+
$$PUBLISH_FLAG \
32+
$$EXTRA_FLAGS

‎global-images/README.md‎

Lines changed: 49 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,49 @@
1+
# Collection of global images for renkulab
2+
3+
The subdirectories here contain basic spec for images that can be used in global session launchers in RenkuLab.
4+
They are built using [buildpacks](https://github.com/SwissDataScienceCenter/renku-frontend-buildpacks/) and contain
5+
some basic libraries. Consult the environment definition files in the subdirectories to see which packages they contain.
6+
7+
## Building
8+
9+
To build the images, install [pack](https://buildpacks.io/docs/for-platform-operators/how-to/integrate-ci/pack/) then run:
10+
11+
```shell
12+
$ make all
13+
```
14+
15+
## Configuration
16+
17+
There are a few environment variables you can set to modify the build.
18+
19+
| Environment Variable | Description | Default |
20+
------------------------------------------------
21+
| BUILDER | Builder image to use | ghcr.io/swissdatasciencecenter/renku-frontend-buildpacks/selector:0.0.6 |
22+
| DOCKER_PREFIX | Prefix to use for the image | None |
23+
| FRONTEND | The frontend to add (vscodium or jupyterlab) | vscodium |
24+
| PUBLISH | Push the image to the registry | False |
25+
| RUN_IMAGE | Run image to use | ghcr.io/swissdatasciencecenter/renku-frontend-buildpacks/base-image:0.0.6 |
26+
| TAGS | Comma-separated list of image names | None |
27+
28+
## Adding packages to existing images
29+
30+
The python images use poetry (< 2.0) for dependency management. Use poetry to add packages, for example:
31+
32+
```bash
33+
$ cd datascience
34+
$ poetry add tensorflow
35+
```
36+
37+
Make sure `pyproject.toml` and `poetry.lock` are updated and push the changes.
38+
39+
## Adding additional images
40+
41+
To configure an additional image spec to be built, follow these steps:
42+
43+
- create a new sub-directory with the corresponding environment definition file
44+
- add the new directory name to the Makefile targets
45+
- add the directory name to the `FLAVOR` list in the build matrix in `build-global-renkulab-images.yml`
46+
47+
## Publishing
48+
49+
These images are automatically built and published by a GitHub Action whenever the repository is tagged.

‎global-images/basic/poetry.lock‎

Lines changed: 7 additions & 0 deletions
Some generated files are not rendered by default. Learn more about customizing how changed files appear on GitHub.

‎global-images/basic/pyproject.toml‎

Lines changed: 15 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,15 @@
1+
[tool.poetry]
2+
name = "basic-python-renku-dependencies"
3+
version = "0.0.1"
4+
description = "Basic Python dependencies for renkulab.io"
5+
authors = ["Renku developers <hello@renku.io>"]
6+
license = "Apache 2.0"
7+
readme = "README.md"
8+
package-mode = false
9+
10+
[tool.poetry.dependencies]
11+
python = ">=3.12,<3.14"
12+
13+
[build-system]
14+
requires = ["poetry-core"]
15+
build-backend = "poetry.core.masonry.api"

0 commit comments

Comments
 (0)