Skip to content

Commit 3ef76aa

Browse files
Add manual deployment script as single source of truth
CI is unavailable, so releases must be deployed by hand. Add scripts/deploy.sh, which reproduces the Release workflow's deployment steps, and have the workflow call it so the two never drift. scripts/deploy.sh: - nix-cache: build komet-node once, push the closure to the k-framework cache (cachix), publish the binary to k-framework-binary (kup), and verify the push and pin. - docker: build the image, smoke-test it, push to Docker Hub; logs out on exit so credentials do not linger. - Preflight guards: required tools on PATH, a clean worktree (so a manual build matches the published REV), and required secrets. scripts/README.md: manual-deployment runbook. release.yml: the nix-cache and dockerhub jobs now invoke the script instead of duplicating the commands inline. .dockerignore: exclude local secret/credential files so a manual `docker build .` cannot bake them into a published image layer. .devcontainer: install cachix and add the docker-in-docker feature so both subcommands run inside the container.
1 parent 35b2946 commit 3ef76aa

8 files changed

Lines changed: 308 additions & 51 deletions

File tree

.devcontainer/Dockerfile

Lines changed: 6 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -59,13 +59,18 @@ ENV PATH=/home/vscode/.nix-profile/bin:/home/vscode/.npm-global/bin:$PATH
5959
# - gh → GitHub CLI for PRs/issues/auth; global so it's usable
6060
# from anywhere. Auth is persisted across rebuilds via a
6161
# named volume on ~/.config/gh (see devcontainer.json).
62+
# - cachix → required by scripts/deploy.sh `nix-cache` to push the
63+
# build closure to the k-framework cache. kup (the other
64+
# half of that deploy step) is fetched via nix at run
65+
# time, so only cachix needs provisioning here.
6266
RUN . /home/vscode/.nix-profile/etc/profile.d/nix.sh \
6367
&& nix registry add nixpkgs github:NixOS/nixpkgs/nixos-25.05 \
6468
&& nix profile install \
6569
nixpkgs#direnv \
6670
nixpkgs#nix-direnv \
6771
nixpkgs#nodejs_22 \
68-
nixpkgs#gh
72+
nixpkgs#gh \
73+
nixpkgs#cachix
6974

7075
# Claude Code CLI — installed to a per-user prefix so root isn't required.
7176
RUN . /home/vscode/.nix-profile/etc/profile.d/nix.sh \
Lines changed: 9 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,9 @@
1+
{
2+
"features": {
3+
"ghcr.io/devcontainers/features/docker-in-docker:2": {
4+
"version": "2.17.0",
5+
"resolved": "ghcr.io/devcontainers/features/docker-in-docker@sha256:25b9f05705ffba7dbe503230ac76081419306f8c8bc88e0ce78c4ecd99a0c78c",
6+
"integrity": "sha256:25b9f05705ffba7dbe503230ac76081419306f8c8bc88e0ce78c4ecd99a0c78c"
7+
}
8+
}
9+
}

.devcontainer/devcontainer.json

Lines changed: 7 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -3,6 +3,13 @@
33
"build": {
44
"dockerfile": "Dockerfile"
55
},
6+
// An isolated Docker daemon inside the container, so `scripts/deploy.sh docker`
7+
// can build and push the release image without touching the host's Docker. The
8+
// feature adds the docker CLI, runs dockerd, and puts `vscode` in the docker
9+
// group; it also marks the container privileged, which dockerd requires.
10+
"features": {
11+
"ghcr.io/devcontainers/features/docker-in-docker:2": {}
12+
},
613
"remoteUser": "vscode",
714
"workspaceFolder": "/workspace",
815
"workspaceMount": "source=${localWorkspaceFolder},target=/workspace,type=bind",

.dockerignore

Lines changed: 13 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -15,3 +15,16 @@ cov-*
1515
result
1616
result-*
1717
state.kore
18+
19+
# Never ship local secrets. The build context is the whole repo, and the
20+
# Dockerfile does `COPY . komet-node`, so any credential file sitting here
21+
# would be baked into a published image layer (a later `rm` does not remove
22+
# it from history). deploy.sh is run by hand with tokens exported, often via a
23+
# direnv `.envrc` or a `.env`, so keep those and any key/token files out.
24+
.envrc
25+
.env
26+
.env.*
27+
*.pem
28+
*.key
29+
*.token
30+
secrets*

.github/scripts/check-cachix-pin.sh

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -2,7 +2,7 @@
22
set -euo pipefail
33

44
# Verifies that `kup publish` actually pushed AND pinned komet-node in the
5-
# private k-framework-binary cache. Cachix has intermittently dropped the
5+
# k-framework-binary cache. Cachix has intermittently dropped the
66
# `cachix pin` requests kup makes under the hood, so we confirm both the pin
77
# (visible via the pin API) and the narinfo (the pushed store path) before
88
# treating the publish as successful.

.github/workflows/release.yml

Lines changed: 7 additions & 49 deletions
Original file line numberDiff line numberDiff line change
@@ -40,38 +40,16 @@ jobs:
4040
ref: ${{ github.sha }}
4141
fetch-depth: 0
4242

43-
# Build the komet-node derivation once, then push it to both the public
44-
# k-framework cache (full build closure, via `cachix push`) and the
45-
# private k-framework-binary cache (the kup-installable binary, via
46-
# `kup publish`). Both pushes reuse the build output already present in
47-
# this runner's Nix store, so the derivation is never built twice.
43+
# The build-and-publish logic lives in scripts/deploy.sh so CI and manual
44+
# deployments stay in sync (single source of truth). See scripts/README.md.
4845
- name: 'Build and publish komet-node to both Nix caches'
4946
shell: bash
5047
env:
51-
GC_DONT_GC: '1'
52-
# Enable flakes for every nix invocation in this step (the initial
53-
# build, the kup build, and the builds kup runs internally) so the
54-
# job does not depend on flakes being globally enabled on the runner.
55-
NIX_CONFIG: 'extra-experimental-features = nix-command flakes'
5648
CACHIX_PUBLIC_TOKEN: '${{ secrets.CACHIX_PUBLIC_TOKEN }}'
5749
CACHIX_PRIVATE_KFB_TOKEN: '${{ secrets.CACHIX_PRIVATE_KFB_TOKEN }}'
5850
OWNER_REPO: '${{ github.repository }}'
5951
REV: '${{ github.sha }}'
60-
run: |
61-
KOMET_NODE=$(nix build .#komet-node --no-link --print-out-paths)
62-
DRV=$(nix-store --query --deriver "${KOMET_NODE}")
63-
64-
# Push the full build closure to the public k-framework cache.
65-
export CACHIX_AUTH_TOKEN="${CACHIX_PUBLIC_TOKEN}"
66-
nix-store --query --requisites --include-outputs "${DRV}" | cachix push k-framework
67-
68-
# Publish the binary to the private k-framework-binary cache. kup
69-
# reuses the store paths built above.
70-
export CACHIX_AUTH_TOKEN="${CACHIX_PRIVATE_KFB_TOKEN}"
71-
export PATH="$(nix build github:runtimeverification/kup --no-link --print-out-paths)/bin:$PATH"
72-
kup publish k-framework-binary .#komet-node --keep-days 180
73-
# Cachix has not been responding to 'cachix pin' requests made under the hood by kup. Verify the push and pin manually.
74-
bash .github/scripts/check-cachix-pin.sh
52+
run: ./scripts/deploy.sh nix-cache
7553

7654
- name: 'On failure, delete drafted release'
7755
if: failure()
@@ -96,32 +74,12 @@ jobs:
9674
ref: ${{ github.sha }}
9775
fetch-depth: 0
9876

99-
- name: 'Set environment'
100-
run: |
101-
KOMET_NODE_VERSION=$(cat package/version)
102-
TAG=runtimeverificationinc/komet-node:ubuntu-jammy-${KOMET_NODE_VERSION}
103-
echo "TAG=${TAG}" >> ${GITHUB_ENV}
104-
105-
- name: 'Build Docker image'
106-
run: |
107-
K_VERSION=$(cat deps/k_release)
108-
docker build . --no-cache --tag ${TAG} --build-arg K_VERSION=${K_VERSION}
109-
110-
- name: 'Run Docker image'
111-
run: docker run --rm ${TAG} komet-node --help
112-
113-
- name: 'Push Docker image to Docker Hub'
77+
# Build, smoke-test, and push logic lives in scripts/deploy.sh so CI and
78+
# manual deployments stay in sync (single source of truth). See scripts/README.md.
79+
- name: 'Build, test, and push Docker image to Docker Hub'
11480
env:
115-
DOCKERHUB_USERNAME: rvdockerhub
11681
DOCKERHUB_PASSWORD: ${{ secrets.DOCKERHUB_PASSWORD }}
117-
DOCKERHUB_NAMESPACE: runtimeverificationinc
118-
DOCKERHUB_REPO: komet-node
119-
run: |
120-
# Log in to Docker Hub
121-
echo "${DOCKERHUB_PASSWORD}" | docker login --username "${DOCKERHUB_USERNAME}" --password-stdin
122-
123-
# Push the image
124-
docker image push ${TAG}
82+
run: ./scripts/deploy.sh docker
12583

12684
- name: 'On failure, delete drafted release'
12785
if: failure()

scripts/README.md

Lines changed: 84 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,84 @@
1+
# Deployment scripts
2+
3+
`deploy.sh` builds and publishes the komet-node release artifacts. It is the
4+
single source of truth for the deployment steps: the `Release` workflow
5+
(`.github/workflows/release.yml`) invokes its subcommands, and you run the same
6+
subcommands by hand when CI is unavailable.
7+
8+
## What it does
9+
10+
The script has three subcommands:
11+
12+
| Subcommand | Action |
13+
| ----------- | ------ |
14+
| `nix-cache` | Builds `.#komet-node`, pushes the full build closure to the `k-framework` Cachix cache, publishes the kup-installable binary to the `k-framework-binary` cache, then verifies the push and pin. Both caches are public; `binary` is a historical name, not an access level. |
15+
| `docker` | Builds the runtime image from `Dockerfile`, runs `komet-node --help` in it as a smoke test, then pushes it to Docker Hub. |
16+
| `all` | Runs `nix-cache` then `docker`. |
17+
18+
The image tag is `runtimeverificationinc/komet-node:ubuntu-jammy-<version>`,
19+
where `<version>` comes from `package/version`.
20+
21+
## Prerequisites
22+
23+
Both subcommands run from a checkout of the revision you are releasing, with the
24+
following tools on `PATH`:
25+
26+
- `nix-cache` needs `nix` and `cachix`. It fetches `kup` from
27+
`github:runtimeverification/kup` itself, so you do not install kup separately.
28+
- `docker` needs `docker`.
29+
30+
Each subcommand checks its tools up front and exits with a clear message if one
31+
is missing, before starting the build.
32+
33+
The project devcontainer (`.devcontainer/`) provides all of these: `nix` and
34+
`cachix` are installed into the Nix profile, and the docker-in-docker feature
35+
supplies `docker`. If you added these to an existing container, rebuild it
36+
("Dev Containers: Rebuild Container") so the new tools are present.
37+
38+
Provide the secrets as environment variables. Everything else has a default (see
39+
the `Shared configuration` block in `deploy.sh`), so a normal checkout needs no
40+
further setup.
41+
42+
| Subcommand | Required environment variables |
43+
| ----------- | ------------------------------ |
44+
| `nix-cache` | `CACHIX_PUBLIC_TOKEN`, `CACHIX_PRIVATE_KFB_TOKEN` |
45+
| `docker` | `DOCKERHUB_PASSWORD` |
46+
47+
`OWNER_REPO` and `REV` default to the current checkout's `origin` remote and
48+
`HEAD`. Override them if you are publishing a revision other than the one checked
49+
out. `DOCKERHUB_USERNAME`, `DOCKERHUB_NAMESPACE`, and `DOCKERHUB_REPO` default to
50+
the release account and repository; override them only to publish elsewhere.
51+
52+
Both subcommands refuse to run if the working tree is not clean. CI builds from a
53+
fresh checkout, so a manual build must too: uncommitted changes and untracked
54+
files would otherwise be baked into the published image or flake build without
55+
matching the `REV` they are published under. Commit, stash, or clean the tree
56+
first. `git status --porcelain` ignores `.gitignore`d paths, so runtime artifacts
57+
and dev caches do not count. Set `ALLOW_DIRTY=1` to override the check when you
58+
deliberately want to publish an uncommitted state.
59+
60+
## Manual release while CI is down
61+
62+
The workflow also drafts, cleans up, and finalizes the GitHub release around the
63+
deployment jobs. When you deploy by hand, run those `gh` steps yourself in this
64+
order. `nix-cache` runs once per architecture (the workflow uses an `x86_64` and
65+
an `ARM64` runner), so run it on a machine of each architecture that should be
66+
cached.
67+
68+
```sh
69+
VERSION=v$(cat package/version)
70+
71+
# 1. Draft the release.
72+
gh release create "$VERSION" --draft --title "$VERSION" --target "$(git rev-parse HEAD)"
73+
74+
# 2. Deploy. Export the secrets first (see the table above).
75+
# Run nix-cache once per architecture; docker once.
76+
./scripts/deploy.sh nix-cache
77+
./scripts/deploy.sh docker
78+
79+
# 3. Finalize the release once every deployment has succeeded.
80+
gh release edit "$VERSION" --draft=false
81+
82+
# If a deployment fails, remove the draft instead:
83+
# gh release delete "$VERSION" --yes --cleanup-tag
84+
```

0 commit comments

Comments
 (0)