Skip to content

Commit 71ff17f

Browse files
authored
feat: add portable CLI binaries for multi-platform Supabase CLI distribution (#2033)
This commit implements portable, self-contained PostgreSQL binaries for the Supabase CLI across macOS (ARM), Linux (x64), and Linux (ARM64), along with automated CI/CD workflows for building and releasing these artifacts. The Supabase CLI needs to ship PostgreSQL binaries that work on user machines without requiring Nix or other system dependencies. This means extracting the actual binaries from Nix's wrapper scripts, bundling all necessary shared libraries, and patching them to use relative paths instead of hardcoded Nix store paths. A `variant` parameter was added to the postgres build pipeline to distinguish between "full" (all extensions) and "cli" (minimal extensions for Supabase CLI). The `cliExtensions` list contains 6 extensions required for running Supabase migrations: supautils, pg_graphql, pgsodium, supabase_vault, pg_net, and pg_cron. Built-in extensions (uuid-ossp, pgcrypto, pg_stat_statements) are included automatically with PostgreSQL. `makeOurPostgresPkgs`/`makePostgresBin` were modified to accept this parameter. A new `psql_17_cli` package is created using `variant = "cli"`, while the full extension set is preserved for base packages (`psql_15`, `psql_17`, `psql_orioledb-17`). The portable CLI variant (`psql_17_cli_portable`) includes 6 extensions for migration support while maintaining a significantly smaller size than the full build. The implementation in `nix/packages/postgres-portable.nix` extracts binaries from `psql_17_cli` using a `resolve_binary()` function that follows wrapper layers to find the actual ELF/Mach-O binaries behind Nix's environment setup scripts. All Nix-provided libraries (ICU, readline, zlib, etc.) are bundled while excluding system libraries (`libc`, `libpthread`, `libm`, `glibc`, `libdl`) that must come from the host. This distinction is critical: Linux bundles must exclude glibc due to kernel ABI dependencies, while macOS can include more libs due to its different linking model. Dependency resolution runs multiple passes to catch transitive deps (e.g., ICU → charset → etc.). Platform-specific patching is applied: Linux binaries use the system interpreter (`/lib64/ld-linux-*.so.2`) and `$ORIGIN`-based RPATHs, while macOS binaries use `@rpath` with `@executable_path`. Wrapper scripts set `LD_LIBRARY_PATH` (Linux) or `DYLD_LIBRARY_PATH` (macOS) to find bundled libraries. The bundle includes PostgreSQL config templates (`postgresql.conf`, `pg_hba.conf`, `pg_ident.conf`) tailored for CLI usage with minimal local dev settings, plus the complete Supabase migration script (`migrate.sh`) with all init-scripts and migration SQL files (55 files, 236KB). A Docker-style initialization script (`supabase-postgres-init.sh`) provides one-command database setup via environment variables (`POSTGRES_USER`, `POSTGRES_PASSWORD`, `POSTGRES_DB`, `PGDATA`). To achieve true portability, a `portable` parameter was added to the PostgreSQL build in `nix/postgresql/generic.nix`. When `portable = true`, three critical hardcoded paths are excluded from the build: 1. `--with-system-tzdata` is removed from configure flags, allowing PostgreSQL to use bundled timezone data from the `share/` directory instead of a hardcoded `/nix/store/.../tzdata` path; 2. the `locale-binary-path.patch` is excluded, so PostgreSQL calls `locale -a` from system PATH rather than using an absolute path to glibc's locale command; 3. the `postFixup` initdb wrapper is disabled to avoid hardcoding glibc paths. The `portable` parameter defaults to `false` in `nix/postgresql/default.nix` but is overridden to `true` for the CLI variant in `nix/packages/postgres.nix` using `.override { portable = true; }`. This ensures standard PostgreSQL builds remain unchanged while the CLI variant produces truly portable binaries without any `/nix/store` references. A GitHub Actions workflow builds portable binaries across all three platforms using a matrix strategy. Each build runs automated portability checks that verify no `/nix/store` references remain, validate RPATH configuration, confirm transitive dependencies are bundled, ensure system libraries are NOT bundled, and check wrapper scripts contain proper library path setup. Post-build testing validates binaries work without Nix (`postgres --version`, `psql --version`). On tagged releases (`v*-cli`), the workflow creates GitHub releases with tarball artifacts and checksums. The test infrastructure needed significant changes to support variants with different extension sets. An `isCliVariant` parameter was added to `makeCheckHarness`, and the hardcoded `shared_preload_libraries` list in `postgresql.conf.in` was replaced with a `@PRELOAD_LIBRARIES@` placeholder. A `generatePreloadLibs` script now parses `receipt.json` at test time and dynamically builds the preload list based on available extensions, removing the previous timescaledb removal hack for OrioleDB.
1 parent c2a3550 commit 71ff17f

14 files changed

Lines changed: 1572 additions & 43 deletions

.github/workflows/cli-release.yml

Lines changed: 446 additions & 0 deletions
Large diffs are not rendered by default.

nix/checks.nix

Lines changed: 430 additions & 25 deletions
Large diffs are not rendered by default.
Lines changed: 9 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,9 @@
1+
# Supabase CLI Host-Based Authentication
2+
# TYPE DATABASE USER ADDRESS METHOD
3+
4+
# Local connections
5+
local all all scram-sha-256
6+
# IPv4 local connections
7+
host all all 127.0.0.1/32 scram-sha-256
8+
# IPv6 local connections
9+
host all all ::1/128 scram-sha-256
Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,2 @@
1+
# Supabase CLI Ident Map Configuration
2+
# MAPNAME SYSTEM-USERNAME PG-USERNAME
Lines changed: 14 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,14 @@
1+
#!/bin/bash
2+
3+
set -euo pipefail
4+
5+
KEY_FILE="${PGSODIUM_KEY_FILE:-$HOME/.supabase/pgsodium_root.key}"
6+
KEY_DIR="$(dirname "$KEY_FILE")"
7+
8+
# Create directory if it doesn't exist
9+
mkdir -p "$KEY_DIR"
10+
11+
if [[ ! -f "${KEY_FILE}" ]]; then
12+
head -c 32 /dev/urandom | od -A n -t x1 | tr -d ' \n' > "${KEY_FILE}"
13+
fi
14+
cat "$KEY_FILE"
Lines changed: 43 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,43 @@
1+
# Supabase CLI PostgreSQL Configuration
2+
# Minimal configuration for local development
3+
4+
# Connection Settings
5+
listen_addresses = '127.0.0.1'
6+
port = 54322
7+
max_connections = 100
8+
unix_socket_directories = '/tmp'
9+
10+
# Memory Settings (conservative for local dev)
11+
shared_buffers = 128MB
12+
effective_cache_size = 256MB
13+
work_mem = 4MB
14+
maintenance_work_mem = 64MB
15+
16+
# Write Ahead Log
17+
wal_level = replica
18+
max_wal_senders = 0
19+
20+
# Logging
21+
log_destination = 'stderr'
22+
logging_collector = off
23+
log_min_messages = warning
24+
log_min_error_statement = error
25+
26+
# Locale
27+
lc_messages = 'C'
28+
lc_monetary = 'C'
29+
lc_numeric = 'C'
30+
lc_time = 'C'
31+
32+
# Extensions
33+
shared_preload_libraries = 'pg_stat_statements, pg_cron, pg_net, pgsodium, supabase_vault, supautils'
34+
35+
# pgsodium and vault configuration
36+
# Set the path to your pgsodium_getkey.sh script
37+
# Default location when using portable bundle: share/supabase-cli/config/pgsodium_getkey.sh
38+
#pgsodium.getkey_script = '/path/to/pgsodium_getkey.sh'
39+
#vault.getkey_script = '/path/to/pgsodium_getkey.sh'
40+
41+
# Supautils configuration
42+
supautils.reserved_roles = 'supabase_admin,supabase_auth_admin,supabase_storage_admin,supabase_read_only_user,supabase_replication_admin,supabase_realtime_admin,supabase_functions_admin'
43+
supautils.reserved_memberships = 'pg_read_server_files,pg_write_server_files,pg_execute_server_program'
Lines changed: 191 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,191 @@
1+
#!/bin/bash
2+
set -Eeo pipefail
3+
4+
# Supabase PostgreSQL Initialization Script
5+
# Similar to docker-entrypoint.sh from official postgres image
6+
# Handles database initialization, password setup, and configuration
7+
8+
# Default values
9+
PGDATA="${PGDATA:-./data}"
10+
POSTGRES_USER="${POSTGRES_USER:-supabase_admin}"
11+
POSTGRES_PASSWORD="${POSTGRES_PASSWORD:-postgres}"
12+
POSTGRES_DB="${POSTGRES_DB:-postgres}"
13+
14+
# Get the directory where this script lives
15+
SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
16+
# Script is at share/supabase-cli/bin/, so go up 3 levels to reach bundle root
17+
BUNDLE_DIR="$(cd "$SCRIPT_DIR/../../.." && pwd)"
18+
PGBIN="$BUNDLE_DIR/bin"
19+
20+
# Logging functions
21+
postgres_log() {
22+
local type="$1"; shift
23+
printf '%s [%s] %s\n' "$(date '+%Y-%m-%d %H:%M:%S')" "$type" "$*"
24+
}
25+
26+
postgres_note() {
27+
postgres_log Note "$@"
28+
}
29+
30+
postgres_error() {
31+
postgres_log ERROR "$@" >&2
32+
}
33+
34+
# Check if PGDATA is initialized
35+
postgres_is_initialized() {
36+
[ -s "$PGDATA/PG_VERSION" ]
37+
}
38+
39+
# Setup initial database
40+
postgres_setup_db() {
41+
postgres_note "Initializing database in $PGDATA"
42+
43+
# Create PGDATA directory if it doesn't exist
44+
mkdir -p "$PGDATA"
45+
46+
# Run initdb
47+
"$PGBIN/initdb" \
48+
-D "$PGDATA" \
49+
-U "$POSTGRES_USER" \
50+
--encoding=UTF8 \
51+
--locale=C \
52+
--no-instructions
53+
54+
postgres_note "Database initialized"
55+
}
56+
57+
# Setup configuration files
58+
postgres_setup_config() {
59+
postgres_note "Setting up configuration files"
60+
61+
# Copy config templates
62+
cp "$BUNDLE_DIR/share/supabase-cli/config/postgresql.conf.template" "$PGDATA/postgresql.conf"
63+
cp "$BUNDLE_DIR/share/supabase-cli/config/pg_hba.conf.template" "$PGDATA/pg_hba.conf"
64+
cp "$BUNDLE_DIR/share/supabase-cli/config/pg_ident.conf.template" "$PGDATA/pg_ident.conf"
65+
66+
# Set absolute path to getkey script in postgresql.conf
67+
GETKEY_SCRIPT="$BUNDLE_DIR/share/supabase-cli/config/pgsodium_getkey.sh"
68+
69+
# Ensure getkey script is executable
70+
if [ -f "$GETKEY_SCRIPT" ]; then
71+
chmod +x "$GETKEY_SCRIPT"
72+
fi
73+
74+
cat >> "$PGDATA/postgresql.conf" << EOF
75+
76+
# pgsodium and vault configuration (set by supabase-postgres-init)
77+
pgsodium.getkey_script = '$GETKEY_SCRIPT'
78+
vault.getkey_script = '$GETKEY_SCRIPT'
79+
EOF
80+
81+
postgres_note "Configuration files set up"
82+
}
83+
84+
# Set password for superuser using single-user mode
85+
postgres_setup_password() {
86+
if [ -n "$POSTGRES_PASSWORD" ]; then
87+
postgres_note "Setting password for user: $POSTGRES_USER"
88+
89+
# Use single-user mode to set password before server starts
90+
# This allows us to use scram-sha-256 authentication
91+
# Must specify 'postgres' database as the target database
92+
# -j flag allows natural multi-line SQL (terminated by semicolon + empty line)
93+
"$PGBIN/postgres" --single -j -D "$PGDATA" postgres <<-EOSQL
94+
ALTER USER "$POSTGRES_USER" WITH PASSWORD '$POSTGRES_PASSWORD';
95+
EOSQL
96+
97+
postgres_note "Password set successfully"
98+
fi
99+
}
100+
101+
# Create additional database if specified and different from default
102+
postgres_create_db() {
103+
if [ "$POSTGRES_DB" != "postgres" ]; then
104+
postgres_note "Creating database: $POSTGRES_DB"
105+
106+
# Must specify 'postgres' database as the target database for single-user mode
107+
# -j flag allows natural multi-line SQL (terminated by semicolon + empty line)
108+
"$PGBIN/postgres" --single -j -D "$PGDATA" postgres <<-EOSQL
109+
CREATE DATABASE "$POSTGRES_DB";
110+
EOSQL
111+
112+
postgres_note "Database created"
113+
fi
114+
}
115+
116+
# Run initialization scripts from a directory
117+
postgres_process_init_files() {
118+
local initdir="$1"
119+
120+
if [ -d "$initdir" ]; then
121+
postgres_note "Running initialization scripts from $initdir"
122+
123+
# Process files in sorted order
124+
for f in "$initdir"/*; do
125+
if [ ! -f "$f" ]; then
126+
continue
127+
fi
128+
129+
case "$f" in
130+
*.sh)
131+
if [ -x "$f" ]; then
132+
postgres_note "Running $f"
133+
"$f"
134+
else
135+
postgres_note "Sourcing $f"
136+
. "$f"
137+
fi
138+
;;
139+
*.sql)
140+
postgres_note "Running $f"
141+
"$PGBIN/postgres" --single -j -D "$PGDATA" postgres < "$f"
142+
;;
143+
*.sql.gz)
144+
postgres_note "Running $f"
145+
gunzip -c "$f" | "$PGBIN/postgres" --single -j -D "$PGDATA" postgres
146+
;;
147+
*)
148+
postgres_note "Ignoring $f"
149+
;;
150+
esac
151+
done
152+
fi
153+
}
154+
155+
# Main initialization flow
156+
postgres_init() {
157+
if postgres_is_initialized; then
158+
postgres_note "Database already initialized, skipping setup"
159+
return
160+
fi
161+
162+
postgres_setup_db
163+
postgres_setup_config
164+
postgres_setup_password
165+
postgres_create_db
166+
167+
# Process any initialization scripts in standard location
168+
# This allows users to add custom init scripts similar to Docker
169+
postgres_process_init_files "$BUNDLE_DIR/share/supabase-cli/init-scripts"
170+
171+
postgres_note "Initialization complete"
172+
}
173+
174+
# Main entrypoint
175+
main() {
176+
# Validate environment
177+
if [ -z "$PGDATA" ]; then
178+
postgres_error "PGDATA environment variable must be set"
179+
exit 1
180+
fi
181+
182+
# Initialize database if needed
183+
postgres_init
184+
185+
# Start PostgreSQL server
186+
postgres_note "Starting PostgreSQL server"
187+
exec "$PGBIN/postgres" -D "$PGDATA" "$@"
188+
}
189+
190+
# Run main function
191+
main "$@"

nix/packages/default.nix

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -62,6 +62,9 @@
6262
psql_orioledb-17 = self'.packages."psql_orioledb-17/bin";
6363
inherit (self.supabase) defaults;
6464
};
65+
psql_17_cli_portable = pkgs.callPackage ./postgres-portable.nix {
66+
psql_17_cli = self'.legacyPackages.psql_17_cli;
67+
};
6568
start-replica = pkgs.callPackage ./start-replica.nix {
6669
psql_15 = self'.packages."psql_15/bin";
6770
inherit pgsqlSuperuser;

0 commit comments

Comments
 (0)