Fix/cachix pin no response (#950)

Something I have been noticing when testing pin and push with kup /
cachix is we get a failure on 'cachix pin' and it times out after 120
seconds, and then a few minutes later the pin shows up.

The workaround here may likely be just wait and keep trying for a few
more minutes to determine if it actually did fail or if the error we're
getting from cachix is a false failure. -- This very much seems to be
the case.

Author: F-WRunTime

.github/scripts/check-cachix-pin.sh

@@ -0,0 +1,49 @@
+#!/usr/bin/env bash
+set -xeuo pipefail
+
+# Kup relies on cachix registry k-framework-binary.
+CACHE="k-framework-binary"
+OWNER_REPO="$(git remote get-url origin | sed -E 's#(git@github.com:|https://github.com/)##; s#\.git$##')"
+REV="$(git rev-parse HEAD)"
+# Get the output of the nix build for kmir.
+OUT="$(nix build --no-link --json ".#kmir" | jq -r '.[0].outputs.out')"
+KEY="github:${OWNER_REPO}/${REV}#kmir"
+
+SUMMARY="${GITHUB_STEP_SUMMARY:-/dev/stdout}"
+
+{
+  echo "## Cachix Publish Summary"
+  echo "CACHE: $CACHE"
+  echo "OUT: $OUT"
+  echo "KEY: $KEY"
+} >> "$SUMMARY"
+
+# Verify push + pin together. Both can become visible with some delay.
+HASH="$(basename "$OUT" | cut -d- -f1)"
+PUSH_NARINFO_URL="https://${CACHE}.cachix.org/${HASH}.narinfo"
+PIN_API_URL="https://app.cachix.org/api/v1/cache/${CACHE}/pin"
+PIN_VISIBILITY_TIMEOUT_SECONDS=120 # 2 minutes
+PIN_VISIBILITY_INTERVAL_SECONDS=5 # 5 seconds
+PIN_VISIBILITY_ATTEMPTS=$((PIN_VISIBILITY_TIMEOUT_SECONDS / PIN_VISIBILITY_INTERVAL_SECONDS))
+for i in $(seq 1 "$PIN_VISIBILITY_ATTEMPTS"); do
+  PUSH_STATUS="$(curl -sS -o /dev/null -w '%{http_code}' "$PUSH_NARINFO_URL")" || PUSH_STATUS="000"
+  if curl -fsSL "$PIN_API_URL" | jq -e --arg k "$KEY" 'any(.[]; .name == $k)' > /dev/null; then
+    PIN_STATUS="pin-ok"
+  else
+    PIN_STATUS="pin-missing"
+  fi
+
+  echo "push-http: ${PUSH_STATUS}" >> "$SUMMARY"
+  echo "pin-status: ${PIN_STATUS}" >> "$SUMMARY"
+
+  if [ "$PUSH_STATUS" = "200" ] && [ "$PIN_STATUS" = "pin-ok" ]; then
+    echo "cachix-status: push-and-pin-ok" >> "$SUMMARY"
+    exit 0
+  fi
+
+  echo "cachix-check-attempt-${i}: not-ready, retrying in ${PIN_VISIBILITY_INTERVAL_SECONDS}s" >> "$SUMMARY"
+  sleep "$PIN_VISIBILITY_INTERVAL_SECONDS"
+done
+
+echo "cachix-status: push-or-pin-missing-after-${PIN_VISIBILITY_TIMEOUT_SECONDS}s" >> "$SUMMARY"
+exit 1

.github/workflows/release.yml

@@ -171,8 +171,11 @@ jobs:
           packages: jq
           script: |
             export PATH="$(nix build github:runtimeverification/kup --no-link --json | jq -r '.[].outputs | to_entries[].value')/bin:$PATH"
-            kup publish k-framework-binary .#kmir --keep-days 180
-            kup publish k-framework-binary .#kmir.rust --keep-days 180
+            kup publish k-framework-binary .#kmir --keep-days 180 || true
+            kup publish k-framework-binary .#kmir.rust --keep-days 180 || true
+            
+            # Cachix has not been responding to 'cachix pin' requests made under the hood by kup. So we need to manually verify the push and pin.
+            .github/scripts/check-cachix-pin.sh
 
       - name: 'On failure, delete drafted release'
         if: failure()

.github/workflows/test-cachix-pin.yml

@@ -0,0 +1,28 @@
+name: 'Test Cachix Pin'
+
+on:
+  workflow_dispatch:
+    inputs:
+      ref:
+        description: 'Git ref (tag or SHA) of an existing release to verify'
+        required: true
+        type: string
+
+jobs:
+  verify-cachix-pin:
+    name: 'Verify Cachix Pin'
+    runs-on: [self-hosted, linux, normal]
+    steps:
+      - name: 'Check out code'
+        uses: actions/checkout@v4
+        with:
+          ref: ${{ inputs.ref }}
+
+      - name: 'Verify cachix publish and pin'
+        uses: workflow/nix-shell-action@v3
+        env:
+          CACHIX_AUTH_TOKEN: '${{ secrets.CACHIX_PRIVATE_KFB_TOKEN }}'
+          GC_DONT_GC: '1'
+        with:
+          packages: jq
+          script: bash .github/scripts/check-cachix-pin.sh