The Data Manager Job Operator

This repo contains a Kubernetes Operator based on the kopf and kubernetes Python packages that is used by the Informatics Matters Squonk2 Data Manager API to create transient Jobs (Kubernetes Pods) for the Data Manager service.

Prerequisites: -

• Python
• Docker
• A kubernetes config file
• A compatible Kubernetes (e.g. 1.32 thru 1.34 if the operator is built for 1.33)

Contributing

The project uses: -

• pre-commit to enforce linting of files prior to committing them to the upstream repository
• Commitizen to enforce a Conventional Commit commit message format
• Black as a code formatter

You MUST comply with these choices in order to contribute to the project.

To get started review the pre-commit utility and the conventional commit style and then set-up your local clone by following the Installation and Quick Start sections: -

pip install -r build-requirements.txt
pre-commit install -t commit-msg -t pre-commit

Now the project's rules will run on every commit, and you can check the current health of your clone with: -

pre-commit run --all-files

Building the operator (local development)

Pre-requisites: -

• Docker Compose (v2)

The operator container, residing in the operator directory, is automatically built and pushed to Docker Hub using GitHub Actions.

You can build the image yourself using docker compose. The following will build and push an operator image with a specific tag: -

export IMAGE_TAG=34.0.0-alpha.1
docker compose build
docker compose push

Versioning

We adopt a different approach for operator naming. At the time of writing we were on version 33 and major changes do not result in changes to this number. Why?

The major revision is actually used to identify the Kubernetes 1.x release the operator is built against. So the 33.x.x operator is built using the Python 33.x Kubernetes package.

See the kubernetes package version in operator/requrements.txt.

When we make feature changes we update the minor value and for bug-fixes we adjust the patch value. So, for a build against Kubernetes 1.33 our major version will always be 33.

Deploying into the Data Manager API

We use our peer repository squonk2-data-manager-job-operator-ansible to deploy this operator.

---