Skip to content

Commit 91b803c

Browse files
samroseclaude
samrose
and
claude
authored
feat(nix/docker)Cli docker image size reduce (#2019)
* feat(nix): add latestOnly parameter support to postgres.nix Add latestOnly parameter that flows through makeOurPostgresPkgs, makePostgresBin, and makePostgres functions. When true, this parameter will enable building only the latest version of each extension. Also adds psql_17_slim package that uses latestOnly=true. feat(ext): add latestOnly support to pgvector feat(ext): add latestOnly support to wrappers Add latestOnly parameter to wrappers extension that: - Filters versionsToUse when latestOnly=true to build only latest version - Skips migration SQL file generation when latestOnly=true - Skips previously packaged version symlinks when latestOnly=true - Updates library count verification for slim builds (expects 2 files) - Updates passthru metadata with versionsBuilt and latestOnly flag This is part of the psql_17_slim implementation to reduce image size. feat(ext): add latestOnly support to pg_graphql feat(ext): add latestOnly support to pg_net * feat: variant of pkgs for image size reduce * fix: deal with systemd in slim vs regular * feat: slimmed image with tests and image analysis tool * tests: workflow to check image size and test images with pg_regress * fix: running docker test ci * fix: running apps fix * fix: pg_isready in proper place * docs: add pg-startup-profiler design document Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * docs: add pg-startup-profiler implementation plan Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * feat(pg-startup-profiler): scaffold project structure Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * feat(pg-startup-profiler): add pluggable rules system Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * feat(pg-startup-profiler): add Docker client wrapper Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * feat(pg-startup-profiler): add log parser Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * feat: docker startup profiler * cli smoke test, nix package to trigger test runs * feat: shared action use nix derivation to check changes and run test if change in inputs * feat: local and ci cli-smoke and image tests * chore: test suffix * fix: use http_mock_host * fix: match develop branch on these changes fix: docker image test fix: resolve merge conflicts --------- Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>
1 parent 08fc008 commit 91b803c

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

73 files changed

+6924
-1318
lines changed

‎.github/actions/check-docker-image-changes/action.yml‎

Lines changed: 80 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,80 @@
1+
name: Check Docker Image Changes
2+
description: Determines if Docker image inputs have changed between current and base branch
3+
4+
inputs:
5+
base_ref:
6+
description: 'Base branch ref for comparison (typically github.base_ref)'
7+
required: false
8+
default: ''
9+
event_name:
10+
description: 'GitHub event name (typically github.event_name)'
11+
required: true
12+
13+
outputs:
14+
should_run:
15+
description: 'Whether tests should run based on input changes'
16+
value: ${{ steps.check.outputs.should_run }}
17+
input_hash:
18+
description: 'Current Docker image inputs hash'
19+
value: ${{ steps.check.outputs.input_hash }}
20+
base_hash:
21+
description: 'Base branch Docker image inputs hash (empty if not a PR)'
22+
value: ${{ steps.check.outputs.base_hash }}
23+
24+
runs:
25+
using: composite
26+
steps:
27+
- name: Get current Docker image inputs hash
28+
id: current
29+
shell: bash
30+
run: |
31+
HASH=$(nix run --accept-flake-config .#docker-image-inputs -- hash)
32+
echo "hash=$HASH" >> "$GITHUB_OUTPUT"
33+
echo "Current Docker image inputs hash: $HASH"
34+
35+
- name: Get base branch Docker image inputs hash
36+
id: base
37+
if: inputs.event_name == 'pull_request'
38+
shell: bash
39+
run: |
40+
# Fetch base branch
41+
git fetch origin ${{ inputs.base_ref }} --depth=1
42+
43+
# Checkout base branch files temporarily
44+
git checkout FETCH_HEAD -- . 2>/dev/null || true
45+
46+
# Get hash from base branch
47+
BASE_HASH=$(nix run --accept-flake-config .#docker-image-inputs -- hash 2>/dev/null || echo "")
48+
49+
# Restore current branch
50+
git checkout HEAD -- .
51+
52+
echo "hash=$BASE_HASH" >> "$GITHUB_OUTPUT"
53+
echo "Base branch Docker image inputs hash: $BASE_HASH"
54+
55+
- name: Determine if tests should run
56+
id: check
57+
shell: bash
58+
run: |
59+
CURRENT_HASH="${{ steps.current.outputs.hash }}"
60+
BASE_HASH="${{ steps.base.outputs.hash }}"
61+
62+
echo "input_hash=$CURRENT_HASH" >> "$GITHUB_OUTPUT"
63+
echo "base_hash=$BASE_HASH" >> "$GITHUB_OUTPUT"
64+
65+
if [[ "${{ inputs.event_name }}" == "workflow_dispatch" ]]; then
66+
echo "Workflow dispatch - running tests"
67+
echo "should_run=true" >> "$GITHUB_OUTPUT"
68+
elif [[ "${{ inputs.event_name }}" == "push" ]]; then
69+
echo "Push to protected branch - running tests"
70+
echo "should_run=true" >> "$GITHUB_OUTPUT"
71+
elif [[ -z "$BASE_HASH" ]]; then
72+
echo "Could not get base hash - running tests to be safe"
73+
echo "should_run=true" >> "$GITHUB_OUTPUT"
74+
elif [[ "$CURRENT_HASH" != "$BASE_HASH" ]]; then
75+
echo "Docker image inputs changed ($BASE_HASH -> $CURRENT_HASH) - running tests"
76+
echo "should_run=true" >> "$GITHUB_OUTPUT"
77+
else
78+
echo "Docker image inputs unchanged - skipping tests"
79+
echo "should_run=false" >> "$GITHUB_OUTPUT"
80+
fi

‎.github/workflows/cli-smoke-test.yml‎

Lines changed: 112 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,112 @@
1+
name: CLI Smoke Test
2+
3+
on:
4+
pull_request:
5+
types: [opened, reopened, synchronize]
6+
push:
7+
branches:
8+
- develop
9+
- release/*
10+
workflow_dispatch:
11+
12+
permissions:
13+
id-token: write
14+
contents: read
15+
16+
jobs:
17+
check-changes:
18+
name: Check Docker Image Changes
19+
runs-on: blacksmith-2vcpu-ubuntu-2404
20+
outputs:
21+
should_run: ${{ steps.check.outputs.should_run }}
22+
input_hash: ${{ steps.check.outputs.input_hash }}
23+
base_hash: ${{ steps.check.outputs.base_hash }}
24+
steps:
25+
- name: Checkout Repo
26+
uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # v4.2.2
27+
28+
- name: Install nix
29+
uses: ./.github/actions/nix-install-ephemeral
30+
with:
31+
push-to-cache: 'false'
32+
env:
33+
DEV_AWS_ROLE: ${{ secrets.DEV_AWS_ROLE }}
34+
NIX_SIGN_SECRET_KEY: ${{ secrets.NIX_SIGN_SECRET_KEY }}
35+
36+
- name: Check Docker image changes
37+
id: check
38+
uses: ./.github/actions/check-docker-image-changes
39+
with:
40+
event_name: ${{ github.event_name }}
41+
base_ref: ${{ github.base_ref }}
42+
43+
cli-smoke-test:
44+
name: CLI Smoke Test (PG ${{ matrix.pg_version }})
45+
needs: check-changes
46+
if: needs.check-changes.outputs.should_run == 'true'
47+
runs-on: large-linux-arm
48+
timeout-minutes: 60
49+
strategy:
50+
fail-fast: false
51+
matrix:
52+
pg_version: ['15', '17']
53+
steps:
54+
- name: Checkout Repo
55+
uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # v4.2.2
56+
57+
- name: Install nix
58+
uses: ./.github/actions/nix-install-ephemeral
59+
with:
60+
push-to-cache: 'false'
61+
env:
62+
DEV_AWS_ROLE: ${{ secrets.DEV_AWS_ROLE }}
63+
NIX_SIGN_SECRET_KEY: ${{ secrets.NIX_SIGN_SECRET_KEY }}
64+
65+
- name: Create Docker context
66+
run: docker context create builders
67+
68+
- name: Set up Docker Buildx
69+
uses: docker/setup-buildx-action@8d2750c68a42422c14e847fe6c8ac0403b4cbd6f # v3.12.0
70+
with:
71+
endpoint: builders
72+
73+
- name: Build Docker image
74+
run: |
75+
DOCKERFILE="Dockerfile-${{ matrix.pg_version }}"
76+
echo "Building $DOCKERFILE..."
77+
# Tag with ECR prefix since CLI uses public.ecr.aws/supabase/postgres as base
78+
docker build -f "$DOCKERFILE" -t "public.ecr.aws/supabase/postgres:${{ matrix.pg_version }}" .
79+
80+
- name: Run CLI smoke test
81+
run: |
82+
echo "Running CLI smoke test for PostgreSQL ${{ matrix.pg_version }}..."
83+
nix run --accept-flake-config .#cli-smoke-test -- --no-build ${{ matrix.pg_version }}
84+
timeout-minutes: 10
85+
86+
- name: Show logs on failure
87+
if: failure()
88+
run: |
89+
echo "=== Supabase Status ==="
90+
nix run --accept-flake-config .#supabase-cli -- status || true
91+
92+
echo "=== Docker containers ==="
93+
docker ps -a
94+
95+
echo "=== Database container logs ==="
96+
docker logs supabase_db_postgres 2>&1 | tail -100 || true
97+
98+
- name: Cleanup
99+
if: always()
100+
run: |
101+
nix run --accept-flake-config .#supabase-cli -- stop --no-backup || true
102+
103+
skip-notification:
104+
name: CLI Smoke Test (Skipped)
105+
needs: check-changes
106+
if: needs.check-changes.outputs.should_run == 'false'
107+
runs-on: ubuntu-latest
108+
steps:
109+
- name: Report skipped
110+
run: |
111+
echo "CLI smoke test skipped - Docker image inputs unchanged"
112+
echo "Input hash: ${{ needs.check-changes.outputs.input_hash }}"

‎.github/workflows/docker-image-test.yml‎

Lines changed: 139 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,139 @@
1+
name: Docker Image Test
2+
3+
on:
4+
pull_request:
5+
types: [opened, reopened, synchronize]
6+
push:
7+
branches:
8+
- develop
9+
- release/*
10+
workflow_call:
11+
secrets:
12+
DEV_AWS_ROLE:
13+
required: true
14+
NIX_SIGN_SECRET_KEY:
15+
required: true
16+
workflow_dispatch:
17+
inputs:
18+
dockerfile:
19+
description: 'Specific Dockerfile to test (leave empty for all)'
20+
required: false
21+
default: ''
22+
type: string
23+
24+
permissions:
25+
id-token: write
26+
contents: read
27+
28+
jobs:
29+
check-changes:
30+
name: Check Docker Image Changes
31+
runs-on: blacksmith-2vcpu-ubuntu-2404
32+
outputs:
33+
should_run: ${{ steps.check.outputs.should_run }}
34+
input_hash: ${{ steps.check.outputs.input_hash }}
35+
steps:
36+
- name: Checkout Repo
37+
uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # v4.2.2
38+
39+
- name: Install nix
40+
uses: ./.github/actions/nix-install-ephemeral
41+
with:
42+
push-to-cache: 'false'
43+
env:
44+
DEV_AWS_ROLE: ${{ secrets.DEV_AWS_ROLE }}
45+
NIX_SIGN_SECRET_KEY: ${{ secrets.NIX_SIGN_SECRET_KEY }}
46+
47+
- name: Check Docker image changes
48+
id: check
49+
uses: ./.github/actions/check-docker-image-changes
50+
with:
51+
event_name: ${{ github.event_name }}
52+
base_ref: ${{ github.base_ref }}
53+
54+
docker-image-test:
55+
name: Test ${{ matrix.dockerfile }}
56+
needs: check-changes
57+
if: needs.check-changes.outputs.should_run == 'true'
58+
runs-on: large-linux-arm
59+
timeout-minutes: 120
60+
strategy:
61+
fail-fast: false
62+
matrix:
63+
dockerfile:
64+
- Dockerfile-15
65+
- Dockerfile-17
66+
- Dockerfile-orioledb-17
67+
steps:
68+
- name: Checkout Repo
69+
uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # v4.2.2
70+
71+
- name: Install nix
72+
uses: ./.github/actions/nix-install-ephemeral
73+
with:
74+
push-to-cache: 'false'
75+
env:
76+
DEV_AWS_ROLE: ${{ secrets.DEV_AWS_ROLE }}
77+
NIX_SIGN_SECRET_KEY: ${{ secrets.NIX_SIGN_SECRET_KEY }}
78+
79+
- name: Create Docker context
80+
run: docker context create builders
81+
82+
- name: Set up Docker Buildx
83+
uses: docker/setup-buildx-action@8d2750c68a42422c14e847fe6c8ac0403b4cbd6f # v3.12.0
84+
with:
85+
endpoint: builders
86+
87+
- name: Build Docker image
88+
run: |
89+
echo "Building ${{ matrix.dockerfile }}..."
90+
VERSION="${{ matrix.dockerfile }}"
91+
VERSION="${VERSION#Dockerfile-}"
92+
# Build with tags expected by both tools
93+
docker build -f ${{ matrix.dockerfile }} \
94+
-t "pg-docker-test:${VERSION}" \
95+
-t "supabase-postgres:${VERSION}-analyze" \
96+
.
97+
98+
- name: Run image size analysis
99+
run: |
100+
echo "=== Image Size Analysis for ${{ matrix.dockerfile }} ==="
101+
nix run --accept-flake-config .#image-size-analyzer -- --image ${{ matrix.dockerfile }} --no-build
102+
103+
- name: Run Docker image tests
104+
run: |
105+
echo "=== Running tests for ${{ matrix.dockerfile }} ==="
106+
nix run --accept-flake-config .#docker-image-test -- --no-build ${{ matrix.dockerfile }}
107+
108+
- name: Show container logs on failure
109+
if: failure()
110+
run: |
111+
VERSION="${{ matrix.dockerfile }}"
112+
VERSION="${VERSION#Dockerfile-}"
113+
CONTAINER_NAME=$(docker ps -a --filter "name=pg-test-${VERSION}" --format "{{.Names}}" | head -1)
114+
if [[ -n "$CONTAINER_NAME" ]]; then
115+
echo "=== Container logs for $CONTAINER_NAME ==="
116+
docker logs "$CONTAINER_NAME" 2>&1 || true
117+
fi
118+
119+
- name: Cleanup
120+
if: always()
121+
run: |
122+
VERSION="${{ matrix.dockerfile }}"
123+
VERSION="${VERSION#Dockerfile-}"
124+
# Remove test containers
125+
docker ps -a --filter "name=pg-test-${VERSION}" -q | xargs -r docker rm -f || true
126+
# Remove test images
127+
docker rmi "pg-docker-test:${VERSION}" || true
128+
docker rmi "supabase-postgres:${VERSION}-analyze" || true
129+
130+
skip-notification:
131+
name: Docker Image Test (Skipped)
132+
needs: check-changes
133+
if: needs.check-changes.outputs.should_run == 'false'
134+
runs-on: ubuntu-latest
135+
steps:
136+
- name: Report skipped
137+
run: |
138+
echo "Docker image tests skipped - inputs unchanged"
139+
echo "Input hash: ${{ needs.check-changes.outputs.input_hash }}"

‎.github/workflows/nix-build.yml‎

Lines changed: 12 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -217,3 +217,15 @@ jobs:
217217
(needs.nix-build-packages-x86_64-linux.result == 'skipped' || needs.nix-build-packages-x86_64-linux.result == 'success') &&
218218
(needs.nix-build-checks-x86_64-linux.result == 'skipped' || needs.nix-build-checks-x86_64-linux.result == 'success')
219219
uses: ./.github/workflows/test.yml
220+
221+
docker-image-test:
222+
needs: [nix-eval, nix-build-packages-aarch64-linux, nix-build-checks-aarch64-linux]
223+
if: |
224+
!cancelled() &&
225+
needs.nix-eval.result == 'success' &&
226+
(needs.nix-build-packages-aarch64-linux.result == 'skipped' || needs.nix-build-packages-aarch64-linux.result == 'success') &&
227+
(needs.nix-build-checks-aarch64-linux.result == 'skipped' || needs.nix-build-checks-aarch64-linux.result == 'success')
228+
uses: ./.github/workflows/docker-image-test.yml
229+
secrets:
230+
DEV_AWS_ROLE: ${{ secrets.DEV_AWS_ROLE }}
231+
NIX_SIGN_SECRET_KEY: ${{ secrets.NIX_SIGN_SECRET_KEY }}

0 commit comments

Comments
 (0)