Fix upgrade test: build default install SQL from HEAD, not initial commit (#2397)

The upgrade test previously built age--<CURR>.sql from the initial
version-bump commit, meaning CREATE EXTENSION installed 'day-one'
SQL. This caused all 31 non-upgrade regression tests to run WITHOUT
SQL functions added after the version bump (e.g., age_invalidate_graph_cache,
age_prepare_pg_upgrade, age_vertex_stats, etc.). These functions were
never registered in pg_proc, so features depending on them (like VLE
cache invalidation triggers) were silently disabled during testing.

Fix by inverting the upgrade test direction:

Before (installs incomplete SQL, upgrades to complete):
  age--1.7.0.sql                     built from version-bump commit (incomplete)
  age--1.7.0_upgrade_test.sql        built from HEAD (complete)
  age--1.7.0--1.7.0_upgrade_test.sql stamped from template
  Test: CREATE EXTENSION age -> data -> ALTER EXTENSION UPDATE TO '1.7.0_upgrade_test'

After (installs complete SQL, upgrades from synthetic initial):
  age--1.7.0.sql                     built from HEAD (complete)
  age--1.7.0_initial.sql             built from version-bump commit (synthetic)
  age--1.7.0_initial--1.7.0.sql      stamped from template
  Test: CREATE EXTENSION age VERSION '1.7.0_initial' -> data -> ALTER EXTENSION UPDATE TO '1.7.0'

This ensures:
  - All 31 non-upgrade tests run with every SQL function registered
  - The upgrade template is still validated (initial -> current)
  - The test ends at the default version (clean state for drop test)
  - Tarball builds (no git) work identically (cat sql/sql_files)
  - All synthetic files are cleaned up after the test

Makefile changes:
  - age_sql rule: cat current HEAD sql/sql_files (was: git show from commit)
  - New age_init_sql rule: git show from version-bump commit
  - age_upgrade_test_sql: stamps template as INIT->CURR (was CURR->NEXT)
  - EXTRA_CLEAN, _install_upgrade_test_files: updated filenames
  - Removed AGE_NEXT_VER, age_next_sql; added AGE_INIT_VER, age_init_sql

Test SQL changes:
  - Step 3: CREATE EXTENSION age VERSION '<init>' (was: CREATE EXTENSION age)
  - Step 6: ALTER EXTENSION age UPDATE TO default_version (was: LIKE '%_upgrade_test')
  - Step 7: Check installed = default (was: installed <> default)
  - Comments updated throughout to reflect inverted direction

All 32 regression tests pass.

modified:   Makefile
modified:   regress/expected/age_upgrade.out
modified:   regress/sql/age_upgrade.sql

Author: jrgemignani

Makefile

@@ -24,24 +24,25 @@ age_sql = age--1.7.0.sql
 # Validates the upgrade template (age--<VER>--y.y.y.sql) by simulating an
 # extension version upgrade entirely within "make installcheck". The test:
 #
-#   1. Builds the install SQL from the INITIAL version-bump commit (the "from"
-#      version). This is the age--<CURR>.sql used by CREATE EXTENSION age.
-#   2. Builds the install SQL from current HEAD as a synthetic "next" version
-#      (the "to" version). This is age--<NEXT>.sql where NEXT = CURR_upgrade_test.
-#   3. Stamps the upgrade template with the synthetic version, producing the
-#      upgrade script age--<CURR>--<NEXT>.sql.
-#   4. Temporarily installs both synthetic files into the PG extension directory
-#      so that ALTER EXTENSION age UPDATE TO '<NEXT>' can find them.
+#   1. Builds the default install SQL (age--<CURR>.sql) from current HEAD's
+#      sql/sql_files. This is what CREATE EXTENSION age installs.
+#   2. Builds a synthetic "initial" version install SQL from the version-bump
+#      commit in git history. This represents the pre-upgrade state.
+#   3. Stamps the upgrade template to upgrade from the initial version to the
+#      current version.
+#   4. Temporarily installs the synthetic files into the PG extension directory
+#      so that CREATE EXTENSION age VERSION '<INIT>' and ALTER EXTENSION
+#      age UPDATE TO '<CURR>' can find them.
 #   5. The age_upgrade regression test exercises the full upgrade path: install
-#      at CURR, create data, ALTER EXTENSION UPDATE to NEXT, verify data.
+#      at INIT, create data, ALTER EXTENSION UPDATE to CURR, verify data.
 #   6. The test SQL cleans up the synthetic files via a generated shell script.
 #
 # This forces developers to keep the upgrade template in sync: any SQL object
 # added after the version-bump commit must also appear in the template, or the
 # upgrade test will fail (the object will be missing after ALTER EXTENSION UPDATE).
 #
-# The .so (shared library) is always built from current HEAD, so C-level changes
-# in the PR are tested by every regression test, not just the upgrade test.
+# Because the default install SQL comes from HEAD, all 31 non-upgrade tests
+# run with every SQL function registered — no functions are missing.
 #
 # Graceful degradation — the upgrade test is silently skipped when:
 #   - No git history (tarball build): AGE_VER_COMMIT is empty.
@@ -53,15 +54,15 @@ age_sql = age--1.7.0.sql
 AGE_CURR_VER := $(shell awk -F"'" '/default_version/ {print $$2}' age.control 2>/dev/null)
 # Git commit that last changed age.control — the "initial release" commit
 AGE_VER_COMMIT := $(shell git log -1 --format=%H -- age.control 2>/dev/null)
-# Synthetic next version: current version with _upgrade_test suffix (e.g., 1.7.0 -> 1.7.0_upgrade_test)
-AGE_NEXT_VER := $(AGE_CURR_VER)_upgrade_test
+# Synthetic initial version: current version with _initial suffix
+AGE_INIT_VER := $(AGE_CURR_VER)_initial
 # The upgrade template file (e.g., age--1.7.0--y.y.y.sql); empty if not present
 AGE_UPGRADE_TEMPLATE := $(wildcard age--$(AGE_CURR_VER)--y.y.y.sql)
 
 # Synthetic filenames — these are NOT installed permanently; they are temporarily
 # placed in $(SHAREDIR)/extension/ during installcheck and removed after.
-age_next_sql = $(if $(AGE_NEXT_VER),age--$(AGE_NEXT_VER).sql)
-age_upgrade_test_sql = $(if $(AGE_NEXT_VER),age--$(AGE_CURR_VER)--$(AGE_NEXT_VER).sql)
+age_init_sql = $(if $(AGE_INIT_VER),age--$(AGE_INIT_VER).sql)
+age_upgrade_test_sql = $(if $(AGE_INIT_VER),age--$(AGE_INIT_VER)--$(AGE_CURR_VER).sql)
 
 # Real (git-tracked, non-template) upgrade scripts FROM the current version.
 # If any exist (e.g., age--1.7.0--1.8.0.sql is committed), the synthetic
@@ -196,7 +197,7 @@ ag_regress_dir = $(srcdir)/regress
 REGRESS_OPTS = --load-extension=age --inputdir=$(ag_regress_dir) --outputdir=$(ag_regress_dir) --temp-instance=$(ag_regress_dir)/instance --port=61958 --encoding=UTF-8 --temp-config $(ag_regress_dir)/age_regression.conf
 
 ag_regress_out = instance/ log/ results/ regression.*
-EXTRA_CLEAN = $(addprefix $(ag_regress_dir)/, $(ag_regress_out)) src/backend/parser/cypher_gram.c src/include/parser/cypher_gram_def.h src/include/parser/cypher_kwlist_d.h $(all_age_sql) $(age_next_sql) $(age_upgrade_test_sql) $(ag_regress_dir)/age_upgrade_cleanup.sh
+EXTRA_CLEAN = $(addprefix $(ag_regress_dir)/, $(ag_regress_out)) src/backend/parser/cypher_gram.c src/include/parser/cypher_gram_def.h src/include/parser/cypher_kwlist_d.h $(all_age_sql) $(age_init_sql) $(age_upgrade_test_sql) $(ag_regress_dir)/age_upgrade_cleanup.sh
 
 GEN_KEYWORDLIST = $(PERL) -I ./tools/ ./tools/gen_keywordlist.pl
 GEN_KEYWORDLIST_DEPS = ./tools/gen_keywordlist.pl tools/PerfectHash.pm
@@ -226,59 +227,44 @@ src/backend/parser/cypher_parser.bc: src/backend/parser/cypher_gram.c src/includ
 src/backend/parser/cypher_keywords.o: src/backend/parser/cypher_gram.c src/include/parser/cypher_gram_def.h
 src/backend/parser/cypher_keywords.bc: src/backend/parser/cypher_gram.c src/include/parser/cypher_gram_def.h
 
-# Build the default install SQL (age--<VER>.sql) from the INITIAL version-bump
-# commit in git history. This means CREATE EXTENSION age installs the "day-one
-# release" SQL — the state of sql/ at the moment the version was bumped in
-# age.control. All other regression tests run against this SQL + the current
-# HEAD .so, implicitly validating that the .so is backward-compatible with the
-# initial release SQL.
-#
-# The current HEAD SQL goes into the synthetic next version (age--<NEXT>.sql)
-# which is only reachable via ALTER EXTENSION UPDATE in the upgrade test.
-ifneq ($(AGE_VER_COMMIT),)
-$(age_sql): age.control
-	@echo "Building initial-release install SQL: $@ from commit $(AGE_VER_COMMIT)"
-	@for f in $$(git show $(AGE_VER_COMMIT):sql/sql_files 2>/dev/null); do \
-		git show $(AGE_VER_COMMIT):sql/$${f}.sql 2>/dev/null; \
-	done > $@
-ifeq ($(SIZEOF_DATUM),4)
-	@echo "32-bit build: removing PASSEDBYVALUE from graphid type"
-	@sed 's/^  PASSEDBYVALUE,$$/  -- PASSEDBYVALUE removed for 32-bit (see Makefile)/' $@ > $@.tmp && mv $@.tmp $@
-endif
-else
-# Fallback: no git history (tarball build) — use current HEAD SQL fragments
+# Build the default install SQL (age--<VER>.sql) from current HEAD's sql/sql_files.
+# This is what CREATE EXTENSION age installs — it contains ALL current functions.
+# All 31 non-upgrade regression tests run against this complete SQL.
 $(age_sql): $(SQLS)
+	@echo "Building install SQL: $@ from HEAD"
 	@cat $(SQLS) > $@
 ifeq ($(SIZEOF_DATUM),4)
 	@echo "32-bit build: removing PASSEDBYVALUE from graphid type"
 	@sed 's/^  PASSEDBYVALUE,$$/  -- PASSEDBYVALUE removed for 32-bit (see Makefile)/' $@ > $@.tmp && mv $@.tmp $@
-	@grep -q 'PASSEDBYVALUE removed for 32-bit' $@ || { echo "Error: PASSEDBYVALUE replacement failed in $@"; exit 1; }
-endif
 endif
 
-# Build synthetic "next" version install SQL from current HEAD's sql/sql_files.
-# This represents what the extension SQL looks like in the PR being tested.
+# Build synthetic "initial" version install SQL from the version-bump commit.
+# This represents the pre-upgrade state — the SQL at the time the version was
+# bumped in age.control. Used only by the upgrade test.
 ifneq ($(AGE_HAS_UPGRADE_TEST),)
-$(age_next_sql): $(SQLS)
-	@echo "Building synthetic next version install SQL: $@ ($(AGE_NEXT_VER))"
-	@cat $(SQLS) > $@
+$(age_init_sql): age.control
+	@echo "Building synthetic initial version install SQL: $@ from commit $(AGE_VER_COMMIT)"
+	@for f in $$(git show $(AGE_VER_COMMIT):sql/sql_files 2>/dev/null); do \
+		git show $(AGE_VER_COMMIT):sql/$${f}.sql 2>/dev/null; \
+	done > $@
 ifeq ($(SIZEOF_DATUM),4)
 	@sed 's/^  PASSEDBYVALUE,$$/  -- PASSEDBYVALUE removed for 32-bit (see Makefile)/' $@ > $@.tmp && mv $@.tmp $@
 endif
 
-# Stamp upgrade template as upgrade from current to synthetic next version
+# Stamp upgrade template as upgrade from initial to current version
 $(age_upgrade_test_sql): $(AGE_UPGRADE_TEMPLATE)
-	@echo "Stamping upgrade template: $< -> $@"
-	@sed -e "s/1\.X\.0/$(AGE_NEXT_VER)/g" -e "s/y\.y\.y/$(AGE_NEXT_VER)/g" $< > $@
+	@echo "Stamping upgrade template: $< -> $@ ($(AGE_INIT_VER) -> $(AGE_CURR_VER))"
+	@sed -e "s/1\.X\.0/$(AGE_CURR_VER)/g" -e "s/y\.y\.y/$(AGE_CURR_VER)/g" $< > $@
 endif
 
 src/backend/parser/ag_scanner.c: FLEX_NO_BACKUP=yes
 
 # --- Upgrade test file lifecycle during installcheck ---
 #
-# Problem: The upgrade test needs age--<NEXT>.sql and age--<CURR>--<NEXT>.sql
-# in the PG extension directory for ALTER EXTENSION UPDATE to find them, but
-# we must not leave them installed permanently (they would confuse users).
+# Problem: The upgrade test needs age--<INIT>.sql and age--<INIT>--<CURR>.sql
+# in the PG extension directory for CREATE EXTENSION VERSION and ALTER
+# EXTENSION UPDATE to find them, but we must not leave them installed
+# permanently (they would confuse users).
 #
 # Solution: A Make prerequisite installs them before pg_regress runs, and the
 # test SQL removes them at the end via \! (shell escape in psql). A generated
@@ -293,10 +279,10 @@ SHAREDIR = $(shell $(PG_CONFIG) --sharedir)
 installcheck: export LC_COLLATE=C
 ifneq ($(AGE_HAS_UPGRADE_TEST),)
 .PHONY: _install_upgrade_test_files
-_install_upgrade_test_files: $(age_next_sql) $(age_upgrade_test_sql)  ## Build, install synthetic files, generate cleanup script
+_install_upgrade_test_files: $(age_init_sql) $(age_upgrade_test_sql)  ## Build, install synthetic files, generate cleanup script
 	@echo "Installing upgrade test files to $(SHAREDIR)/extension/"
-	@$(INSTALL_DATA) $(age_next_sql) $(age_upgrade_test_sql) '$(SHAREDIR)/extension/'
-	@printf '#!/bin/sh\nrm -f "$(SHAREDIR)/extension/$(age_next_sql)" "$(SHAREDIR)/extension/$(age_upgrade_test_sql)"\nrm -f "$(age_next_sql)" "$(age_upgrade_test_sql)" "$(ag_regress_dir)/age_upgrade_cleanup.sh"\n' > $(ag_regress_dir)/age_upgrade_cleanup.sh
+	@$(INSTALL_DATA) $(age_init_sql) $(age_upgrade_test_sql) '$(SHAREDIR)/extension/'
+	@printf '#!/bin/sh\nrm -f "$(SHAREDIR)/extension/$(age_init_sql)" "$(SHAREDIR)/extension/$(age_upgrade_test_sql)"\nrm -f "$(age_init_sql)" "$(age_upgrade_test_sql)" "$(ag_regress_dir)/age_upgrade_cleanup.sh"\n' > $(ag_regress_dir)/age_upgrade_cleanup.sh
 	@chmod +x $(ag_regress_dir)/age_upgrade_cleanup.sh
 
 installcheck: _install_upgrade_test_files

regress/expected/age_upgrade.out

@@ -20,17 +20,17 @@
 -- Extension upgrade regression test
 --
 -- This test validates the upgrade template (age--<VER>--y.y.y.sql) by:
---   1. Installing AGE at the current version (built from the initial
---      version-bump commit's SQL — the "day-one release" state)
+--   1. Dropping AGE and reinstalling at the synthetic "initial" version
+--      (built from the version-bump commit's SQL — the "day-one" state)
 --   2. Creating three graphs with multiple labels, edges, GIN indexes,
 --      and numeric properties that serve as integrity checksums
---   3. Upgrading to a synthetic "next" version via the stamped template
+--   3. Upgrading to the current (default) version via the stamped template
 --   4. Verifying all data, structure, and checksums survived the upgrade
 --
 -- The Makefile builds:
---   age--<CURR>.sql        from the initial version-bump commit (git history)
---   age--<NEXT>.sql        from current HEAD's sql/sql_files
---   age--<CURR>--<NEXT>.sql stamped from the upgrade template
+--   age--<CURR>.sql            from current HEAD's sql/sql_files (default)
+--   age--<CURR>_initial.sql    from the version-bump commit (synthetic)
+--   age--<CURR>_initial--<CURR>.sql  stamped from the upgrade template
 --
 -- All version discovery is dynamic — no hardcoded versions anywhere.
 -- This test is version-agnostic and works on any branch for any version.
@@ -39,7 +39,7 @@ LOAD 'age';
 SET search_path TO ag_catalog;
 -- Step 1: Clean up any state left by prior tests, then drop AGE entirely.
 -- The --load-extension=age flag installed AGE at the current (default) version.
--- We need to remove it so we can cleanly re-create for this test.
+-- We need to remove it so we can reinstall at the synthetic initial version.
 SELECT drop_graph(name, true) FROM ag_graph ORDER BY name;
  drop_graph 
 ------------
@@ -54,8 +54,22 @@ FROM pg_available_extension_versions WHERE name = 'age';
  t
 (1 row)
 
--- Step 3: Install AGE at the default version (the initial release SQL).
-CREATE EXTENSION age;
+-- Step 3: Install AGE at the synthetic initial version (pre-upgrade state).
+DO $$
+DECLARE init_ver text;
+BEGIN
+    SELECT version INTO init_ver
+    FROM pg_available_extension_versions
+    WHERE name = 'age' AND version LIKE '%_initial'
+    ORDER BY version DESC
+    LIMIT 1;
+    IF init_ver IS NULL THEN
+        RAISE EXCEPTION 'No initial version available for upgrade test';
+    END IF;
+
+    EXECUTE format('CREATE EXTENSION age VERSION %L', init_ver);
+END;
+$$;
 SELECT extversion IS NOT NULL AS version_installed FROM pg_extension WHERE extname = 'age';
  version_installed 
 -------------------
@@ -248,27 +262,26 @@ WHERE ag.name <> '_ag_catalog';
                   21
 (1 row)
 
--- Step 6: Upgrade AGE to the synthetic next version via the stamped template.
+-- Step 6: Upgrade AGE from the initial version to the current (default) version
+-- via the stamped upgrade template.
 DO $$
-DECLARE next_ver text;
+DECLARE curr_ver text;
 BEGIN
-    SELECT version INTO next_ver
-    FROM pg_available_extension_versions
-    WHERE name = 'age' AND version LIKE '%_upgrade_test'
-    ORDER BY version DESC
-    LIMIT 1;
-    IF next_ver IS NULL THEN
-        RAISE EXCEPTION 'No next version available for upgrade test';
+    SELECT default_version INTO curr_ver
+    FROM pg_available_extensions
+    WHERE name = 'age';
+    IF curr_ver IS NULL THEN
+        RAISE EXCEPTION 'No default version found for upgrade test';
     END IF;
 
-    EXECUTE format('ALTER EXTENSION age UPDATE TO %L', next_ver);
+    EXECUTE format('ALTER EXTENSION age UPDATE TO %L', curr_ver);
 END;
 $$;
--- Step 7: Confirm version changed.
-SELECT installed_version <> default_version AS upgraded_past_default
+-- Step 7: Confirm version is now the default (current HEAD) version.
+SELECT installed_version = default_version AS upgraded_to_current
 FROM pg_available_extensions WHERE name = 'age';
- upgraded_past_default 
------------------------
+ upgraded_to_current 
+---------------------
  t
 (1 row)
 

regress/sql/age_upgrade.sql

@@ -21,17 +21,17 @@
 -- Extension upgrade regression test
 --
 -- This test validates the upgrade template (age--<VER>--y.y.y.sql) by:
---   1. Installing AGE at the current version (built from the initial
---      version-bump commit's SQL — the "day-one release" state)
+--   1. Dropping AGE and reinstalling at the synthetic "initial" version
+--      (built from the version-bump commit's SQL — the "day-one" state)
 --   2. Creating three graphs with multiple labels, edges, GIN indexes,
 --      and numeric properties that serve as integrity checksums
---   3. Upgrading to a synthetic "next" version via the stamped template
+--   3. Upgrading to the current (default) version via the stamped template
 --   4. Verifying all data, structure, and checksums survived the upgrade
 --
 -- The Makefile builds:
---   age--<CURR>.sql        from the initial version-bump commit (git history)
---   age--<NEXT>.sql        from current HEAD's sql/sql_files
---   age--<CURR>--<NEXT>.sql stamped from the upgrade template
+--   age--<CURR>.sql            from current HEAD's sql/sql_files (default)
+--   age--<CURR>_initial.sql    from the version-bump commit (synthetic)
+--   age--<CURR>_initial--<CURR>.sql  stamped from the upgrade template
 --
 -- All version discovery is dynamic — no hardcoded versions anywhere.
 -- This test is version-agnostic and works on any branch for any version.
@@ -42,16 +42,30 @@ SET search_path TO ag_catalog;
 
 -- Step 1: Clean up any state left by prior tests, then drop AGE entirely.
 -- The --load-extension=age flag installed AGE at the current (default) version.
--- We need to remove it so we can cleanly re-create for this test.
+-- We need to remove it so we can reinstall at the synthetic initial version.
 SELECT drop_graph(name, true) FROM ag_graph ORDER BY name;
 DROP EXTENSION age;
 
 -- Step 2: Verify we have multiple installable versions.
 SELECT count(*) > 1 AS has_upgrade_path
 FROM pg_available_extension_versions WHERE name = 'age';
 
--- Step 3: Install AGE at the default version (the initial release SQL).
-CREATE EXTENSION age;
+-- Step 3: Install AGE at the synthetic initial version (pre-upgrade state).
+DO $$
+DECLARE init_ver text;
+BEGIN
+    SELECT version INTO init_ver
+    FROM pg_available_extension_versions
+    WHERE name = 'age' AND version LIKE '%_initial'
+    ORDER BY version DESC
+    LIMIT 1;
+    IF init_ver IS NULL THEN
+        RAISE EXCEPTION 'No initial version available for upgrade test';
+    END IF;
+
+    EXECUTE format('CREATE EXTENSION age VERSION %L', init_ver);
+END;
+$$;
 SELECT extversion IS NOT NULL AS version_installed FROM pg_extension WHERE extname = 'age';
 
 -- Step 4: Create three test graphs with diverse labels, edges, and data.
@@ -194,25 +208,24 @@ SELECT count(*)::int AS total_labels_before
 FROM ag_label al JOIN ag_graph ag ON al.graph = ag.graphid
 WHERE ag.name <> '_ag_catalog';
 
--- Step 6: Upgrade AGE to the synthetic next version via the stamped template.
+-- Step 6: Upgrade AGE from the initial version to the current (default) version
+-- via the stamped upgrade template.
 DO $$
-DECLARE next_ver text;
+DECLARE curr_ver text;
 BEGIN
-    SELECT version INTO next_ver
-    FROM pg_available_extension_versions
-    WHERE name = 'age' AND version LIKE '%_upgrade_test'
-    ORDER BY version DESC
-    LIMIT 1;
-    IF next_ver IS NULL THEN
-        RAISE EXCEPTION 'No next version available for upgrade test';
+    SELECT default_version INTO curr_ver
+    FROM pg_available_extensions
+    WHERE name = 'age';
+    IF curr_ver IS NULL THEN
+        RAISE EXCEPTION 'No default version found for upgrade test';
     END IF;
 
-    EXECUTE format('ALTER EXTENSION age UPDATE TO %L', next_ver);
+    EXECUTE format('ALTER EXTENSION age UPDATE TO %L', curr_ver);
 END;
 $$;
 
--- Step 7: Confirm version changed.
-SELECT installed_version <> default_version AS upgraded_past_default
+-- Step 7: Confirm version is now the default (current HEAD) version.
+SELECT installed_version = default_version AS upgraded_to_current
 FROM pg_available_extensions WHERE name = 'age';
 
 -- Step 8: Verify all data survived — reload and recheck.