Skip to content

Commit 50fef18

Browse files
encimaencima
committed
update major PG instructions and ansible file for handling pg18 config
1 parent 7233077 commit 50fef18

File tree

3 files changed

+152
-30
lines changed

3 files changed

+152
-30
lines changed

‎ansible/tasks/stage2-setup-postgres.yml‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -10,7 +10,7 @@
1010
- name: Execute tasks when (is_psql_oriole or is_psql_17) and stage2_nix
1111
become: true
1212
when:
13-
- (is_psql_oriole or is_psql_17)
13+
- (is_psql_oriole or is_psql_17 or is_psql_18)
1414
- stage2_nix
1515
block:
1616
- name: Remove specified extensions from postgresql.conf if orioledb-17 or 17 build

‎nix/docs/new-major-postgres.md‎

Lines changed: 148 additions & 21 deletions
Original file line numberDiff line numberDiff line change
@@ -1,31 +1,158 @@
1+
# Adding a new major PostgreSQL version
2+
3+
This guide walks through every file that needs to change when adding a new major
4+
PostgreSQL version (e.g. 18) to the repository. The PostgreSQL 18 branch
5+
(`feat/pg-18`) is a good reference for the full set of changes.
6+
7+
## 1. Upstream nixpkgs
8+
19
PostgreSQL versions are managed in upstream nixpkgs.
10+
[See this example PR](https://github.com/NixOS/nixpkgs/pull/249030) for adding
11+
a version to nixpkgs. Once the version lands upstream, run `nix flake update` in
12+
this repository to pull it in.
13+
14+
## 2. Nix configuration (`nix/config.nix`)
15+
16+
Add a new entry under `postgresql` with the major version, full version string,
17+
and source hash:
18+
19+
```nix
20+
"18" = {
21+
version = "18.3";
22+
hash = "sha256-...";
23+
};
24+
```
25+
26+
## 3. Packages (`nix/packages/`)
27+
28+
Several files need to reference the new version:
29+
30+
### `nix/packages/postgres.nix`
31+
32+
- Define a `dbExtensionsXX` list (can reuse the previous version's if identical,
33+
e.g. `dbExtensions18 = dbExtensions17;`).
34+
- Add an `else if` branch to select those extensions for the new version.
35+
- Add `psql_XX = makePostgres "XX" { };` to `basePackages`.
36+
- Add `psql_XX_cli = makePostgres "XX" { variant = "cli"; };` to `cliPackages`.
37+
38+
### `nix/packages/default.nix`
39+
40+
- Pass `psql_XX` to the `lib.nix` and `start-client.nix` callPackage sites.
41+
42+
### `nix/packages/lib.nix`
43+
44+
- Add `psql_XX` to the function arguments and expose `PSQLXX_BINDIR` in the
45+
environment.
46+
47+
### `nix/packages/start-client.nix`
48+
49+
- Add `psql_XX` to the inputs and wire it into the version dispatch logic.
50+
51+
### `nix/packages/dbmate-tool.nix`
52+
53+
- Add the new version to the help text and `--restrict-key` conditional.
54+
55+
## 4. Overlays (`nix/overlays/default.nix`)
56+
57+
Add `postgresql_XX` to the `inherit` list so the package is available as an
58+
overlay.
59+
60+
## 5. Checks (`nix/checks.nix`)
61+
62+
This file has several places that need updating:
63+
64+
- Pass `psql_XX` to `pkgs-lib`.
65+
- Add version-matching regex (`isXXMatch`) and dispatch branch in the version
66+
detection logic.
67+
- Assign a unique port number for the new version's test server.
68+
- Add version regex matching for test file filtering (e.g. `builtins.match
69+
"18.*" name`).
70+
- Add version-specific test output filtering (`z_18_.*`).
71+
- Add `psql_XX` and `psql_XX_cli` check harness entries.
72+
- Add `postgresql_XX_debug` and `postgresql_XX_src` to the exported packages.
73+
74+
## 6. Extension versions (`nix/ext/versions.json`)
75+
76+
For each extension, either:
77+
78+
- **Add the new major version to an existing version entry's `postgresql` list**
79+
if the extension already works with the new PostgreSQL without changes.
80+
- **Add a new version entry** if the extension needed to be upgraded for
81+
compatibility. Include the new hash and any extra fields like `pgrx`/`rust`
82+
versions for Rust-based extensions.
83+
84+
Some extensions may need upstream updates to compile against new PostgreSQL
85+
C API changes. Common examples from the PG 18 effort:
86+
87+
- `pg_cron`, `pgsql-http`, `supautils`, `pg_tle`, `pgaudit` — bumped due to
88+
internal PostgreSQL C API changes.
89+
- `pg_graphql`, `pg_jsonschema` — bumped for pgrx compatibility with the new
90+
major version (pgrx version and Rust toolchain may need updating).
91+
- `pgroonga` / `groonga` — patches may need amending due to line number changes.
92+
- `pgvector`, `pgtap`, `plpgsql_check`, `rum`, `pg_stat_monitor`, `postgis` —
93+
bumped to versions that added PG 18 support.
94+
95+
## 7. Extension nix files (`nix/ext/*.nix`)
96+
97+
Each extension's `.nix` file uses `versions.json` to select a compatible
98+
version. No per-extension `.nix` changes are needed in most cases — the
99+
`versions.json` entry is sufficient. However, if an extension fails to build for
100+
any major version (i.e. no entry in `versions.json` matches), the build will
101+
fail with an assertion error.
102+
103+
## 8. Test infrastructure
104+
105+
### `nix/ext/tests/lib.nix`
106+
107+
Update version comparisons so the new version gets the correct config
108+
adjustments (e.g. removing `timescaledb` from `shared_preload_libraries` for
109+
versions ≥ 17).
110+
111+
### `nix/tools/run-server.sh.in`
112+
113+
- Add an `elif` branch to handle the new version, setting `BINDIR` to the
114+
corresponding `@PSQLXX_BINDIR@` substitution.
115+
- Update help text to list the new version.
116+
- Add any version-specific config overrides (e.g. disabling `db_user_namespace`,
117+
removing unsupported extensions from preload libraries on certain platforms).
118+
119+
### Test expected outputs (`nix/tests/expected/`)
120+
121+
Some tests produce version-specific output. You may need to add new
122+
`z_XX_*.out` files or alternative expected output files for tests where the
123+
output differs from previous versions.
124+
125+
## 9. Migration schema (`migrations/`)
126+
127+
Add a `schema-XX.sql` dump generated by running `pg_dump --schema-only` against
128+
a freshly initialized database of the new version. This is used by the dbmate
129+
migration tooling.
130+
131+
## 10. Dockerfile
132+
133+
Add a `Dockerfile-XX` for the new version. This can generally be copied from the
134+
previous version's Dockerfile with the `postgresql_major` build arg updated.
135+
136+
## 11. Ansible (AMI builds)
2137

3-
[See this example PR](https://github.com/NixOS/nixpkgs/pull/249030) to add a
4-
new version of PostgreSQL; this version is for 16 beta3, but any version is
5-
roughly the same. In short, you need to:
138+
If the new version will ship in AMI builds:
6139

7-
- Add a new version and hash to `nix/config.nix`
8-
- Possibly patch the source code for minor refactorings
9-
- In this example, an old patch had to be rewritten because a function was
10-
split into two different functions; the patch is functionally equivalent but
11-
textually different
12-
- Add the changes to `all-packages.nix`
13-
- Integrate inside the CI and get code review
14-
- Run `nix flake update` to get a new version, once it's ready
140+
- Add the version to the `postgres_major` list in `ansible/vars.yml`.
141+
- Add the full release string to `postgres_release`.
142+
- Update any version-specific Ansible tasks in `ansible/tasks/setup-postgres.yml`
143+
(e.g. config file differences, `supautils.conf` adjustments).
15144

16-
## Adding the major version to this repository
145+
## 12. Verification
17146

18-
It isn't well abstracted, unfortunately. In short: look for the strings `14` and
19-
`15` under the nix configuration files. More specifically:
147+
After making all changes, run:
20148

21-
- Add `psql_XX` to `basePackages` in `nix/packages/postgres.nix`
22-
- Ditto with `checks` in `nix/checks.nix`
23-
- Modify the tools under `nix/packages/` to understand the new major version
24-
- Make sure the CI is integrated under the GitHub Actions.
149+
```bash
150+
nix flake check
151+
```
25152

26-
The third step and fourth steps are the most annoying, really. The first two are
27-
easy and by that point you can run `nix flake check` in order to test the build,
28-
at least.
153+
This builds all packages and runs the test harnesses. Fix any failures
154+
iteratively — extension build failures are the most common issue and usually
155+
require bumping the extension version or adjusting patches.
29156

30157
## Other notes
31158

‎nix/tools/run-server.sh.in‎

Lines changed: 3 additions & 8 deletions
Original file line numberDiff line numberDiff line change
@@ -275,17 +275,12 @@ orioledb_config_items() {
275275
perl -pi -e 's/(shared_preload_libraries\s*=\s*'\''.*?)'\''/\1, orioledb'\''/' "$DATDIR/postgresql.conf"
276276

277277
echo "default_table_access_method = 'orioledb'" >> "$DATDIR/postgresql.conf"
278-
elif [[ "$VERSION" == "17" && "$CURRENT_SYSTEM" != "aarch64-darwin" ]]; then
279-
echo "non-macos pg 17 conf"
278+
elif [[ "$VERSION" =~ ^(17|18)$ && "$CURRENT_SYSTEM" != "aarch64-darwin" ]]; then
279+
echo "non-macos pg 17+ conf"
280280
sed -i 's/ timescaledb,//g;' "$DATDIR/postgresql.conf"
281281
sed -i 's/db_user_namespace = off/#db_user_namespace = off/g;' "$DATDIR/postgresql.conf"
282282
sed -i 's/ timescaledb,//g; s/ plv8,//g;' "$DATDIR/supautils.conf"
283-
elif [[ "$VERSION" == "17" && "$CURRENT_SYSTEM" = "aarch64-darwin" ]]; then
284-
perl -pi -e 's/db_user_namespace = off/#db_user_namespace = off/g;' "$DATDIR/postgresql.conf"
285-
perl -pi -e 's/ timescaledb,//g' "$DATDIR/postgresql.conf"
286-
perl -pi -e 's/ timescaledb,//g' "$DATDIR/supautils.conf"
287-
perl -pi -e 's/ plv8,//g;' "$DATDIR/supautils.conf"
288-
elif [[ "$VERSION" == "18" && "$CURRENT_SYSTEM" = "aarch64-darwin" ]]; then
283+
elif [[ "$VERSION" =~ ^(17|18)$ && "$CURRENT_SYSTEM" = "aarch64-darwin" ]]; then
289284
perl -pi -e 's/db_user_namespace = off/#db_user_namespace = off/g;' "$DATDIR/postgresql.conf"
290285
perl -pi -e 's/ timescaledb,//g' "$DATDIR/postgresql.conf"
291286
perl -pi -e 's/ timescaledb,//g' "$DATDIR/supautils.conf"

0 commit comments

Comments
 (0)