VLE cache + performance improvements (#2376)

VLE cache + perf: version counter, thin entries, variadic elimination,
edge cleanup list removal

Replace snapshot-based VLE cache invalidation with per-graph version
counters and add performance optimizations for VLE traversal.

Cache invalidation:
- Add DSM (PG 17+) and shmem (PG <17) per-graph monotonic version
  counters for cross-backend cache invalidation
- Replace snapshot xmin/xmax/curcid comparison with version counter
  check in is_ggctx_invalid(), with snapshot fallback for safety
- Add executor hooks in CREATE/DELETE/SET/MERGE to increment the
  graph version counter on mutations
- Add SQL trigger function (age_invalidate_graph_cache) for catching
  SQL-level mutations (INSERT/UPDATE/DELETE/TRUNCATE)
- Auto-install trigger on new label tables via label_commands.c
  with LookupFuncName check for backward compatibility
- Add TRUNCATE interception in ProcessUtility hook (ag_catalog.c)
- Add shmem_request/startup hooks for PG <17 (age.c)
- Use search_label_relation_cache() in get_graph_oid_for_table()
  for fast label-to-graph lookup instead of catalog table scan

Thin entries (lazy property fetch):
- Replace Datum edge_properties/vertex_properties with 6-byte
  ItemPointerData TID in vertex_entry and edge_entry
- Add get_vertex_entry_properties() and get_edge_entry_properties()
  that do heap_fetch via stored TID on demand
- Add is_an_edge_match() fast path: skip property access entirely
  for label-only VLE patterns (the common case)

Edge cleanup list removal:
- Remove ggctx->edges linked list from GRAPH_global_context. With
  thin entries, edge_entry has no palloc'd sub-structures to free
  (TIDs are inline). The cleanup list existed to pfree datumCopy'd
  edge_properties Datums, which no longer exist. hash_destroy()
  handles all edge hash table memory. Saves ~5.6 GB at SF10.

Performance optimizations:
- Reduce VERTEX/EDGE_HTAB_INITIAL_SIZE from 1,000,000 to 10,000.
  PG dynahash grows automatically; large initial size wastes memory.
- Eliminate extract_variadic_args in age_match_vle_terminal_edge:
  direct PG_GETARG_DATUM + cached arg types via fn_extra
- Eliminate extract_variadic_args in age_match_vle_edge_to_id_qual:
  same pattern
- Add agtype_access_operator 2-arg fast path: bypasses
  extract_variadic_args_min for the common property access case
- Add age_tointeger 1-arg fast path: bypasses extract_variadic_args
  with cached arg type
- Add GraphIdStack (flat array-based) DFS stacks replacing
  ListGraphId linked-list stacks for push/pop without palloc/pfree

Upgrade script:
- Add trigger installation on pre-existing label tables during
  extension upgrade via DO block in age--1.7.0--y.y.y.sql. Tables
  created after upgrade get triggers automatically via label_commands.c.

Regression tests:
- Add VLE cache invalidation tests (CREATE, DELETE, SET mutations)
- Add thin entry edge property fetch tests (RETURN p, UNWIND)
- Add direct SQL trigger tests (INSERT, UPDATE, DELETE, TRUNCATE on
  label tables with VLE cache invalidation verification)

All 32 regression tests pass.

SF3 Benchmarks (9.3M vertices, 52.7M edges, warm cache, median):

Total IC1-IC12: 1,530s -> 1,159s (-24.3%, 371s saved)

VLE-heavy queries:
  IC3  (KNOWS*1..2 + 2 countries):    34.9s -> 20.7s  (-40.6%)
  IC5  (forum members, KNOWS*1..2):   46.2s -> 29.9s  (-35.4%)
  IC6  (tag co-occurrence, KNOWS*1..2): 28.2s -> 19.2s (-31.8%)
  IC9  (recent messages, KNOWS*1..2): 86.0s -> 60.4s  (-29.7%)
  IC11 (KNOWS*1..2 + WORK_AT):       18.7s -> 11.0s  (-41.5%)
  IC1  (KNOWS*1..3 + profile):     1269.4s -> 974.9s  (-23.2%)

Short Reads (IS1-IS7): No meaningful change -- non-VLE queries,
sub-millisecond to sub-second. Within run-to-run variance.

Updates (IU1-IU8, SF3, median, 50 ops each):

  IU2 (Like Post):     99.5ms ->  6.3ms  (-93.6%)
  IU3 (Like Comment): 318.3ms ->  5.7ms  (-98.2%)
  IU7 (Add Comment):  344.4ms -> 11.3ms  (-96.7%)
  IU5 (Forum Member):  12.3ms ->  5.9ms  (-52.0%)

Version counter eliminates redundant VLE cache rebuilds on mutations.
Previously, every INSERT/UPDATE/DELETE invalidated the cache via
snapshot comparison, forcing a full rebuild on the next VLE query.
Now, mutations just bump a counter; rebuild only occurs when VLE
actually runs and finds the counter changed.

Graph cache memory (SF3, calculated):

Total: ~15.7 GB -> ~8.7 GB (-45%, 7.0 GB saved)

  Thin entries (TID replaces datumCopy'd properties):  -5.3 GB
  Edge cleanup list removal (no longer needed):        -1.7 GB

Co-authored-by: Claude Opus <noreply@anthropic.com>

modified:   age--1.7.0--y.y.y.sql
modified:   regress/expected/age_global_graph.out
modified:   regress/sql/age_global_graph.sql
modified:   sql/age_main.sql
modified:   src/backend/age.c
modified:   src/backend/catalog/ag_catalog.c
modified:   src/backend/commands/label_commands.c
modified:   src/backend/executor/cypher_create.c
modified:   src/backend/executor/cypher_delete.c
modified:   src/backend/executor/cypher_merge.c
modified:   src/backend/executor/cypher_set.c
modified:   src/backend/utils/adt/age_global_graph.c
modified:   src/backend/utils/adt/age_graphid_ds.c
modified:   src/backend/utils/adt/age_vle.c
modified:   src/backend/utils/adt/agtype.c
modified:   src/include/utils/age_global_graph.h
modified:   src/include/utils/age_graphid_ds.h

Author: jrgemignani

age--1.7.0--y.y.y.sql

@@ -408,3 +408,54 @@ $function$;
 
 COMMENT ON FUNCTION ag_catalog.age_pg_upgrade_status() IS
 'Returns the current pg_upgrade readiness status of the AGE installation.';
+
+--
+-- VLE cache invalidation trigger function
+-- Installed on graph label tables to catch SQL-level mutations
+-- and increment the per-graph version counter for VLE cache invalidation.
+--
+CREATE FUNCTION ag_catalog.age_invalidate_graph_cache()
+    RETURNS trigger
+    LANGUAGE c
+AS 'MODULE_PATHNAME';
+
+--
+-- Install the cache invalidation trigger on all pre-existing label tables.
+-- New label tables created after this upgrade will get the trigger
+-- automatically via label_commands.c. This DO block handles tables
+-- that were created before the upgrade.
+--
+DO $$
+DECLARE
+    r RECORD;
+BEGIN
+    FOR r IN
+        SELECT n.nspname AS schema_name, c.relname AS table_name
+        FROM ag_catalog.ag_label l
+        JOIN pg_catalog.pg_class c ON c.oid = l.relation
+        JOIN pg_catalog.pg_namespace n ON n.oid = c.relnamespace
+        WHERE l.name != '_ag_label_vertex'
+          AND l.name != '_ag_label_edge'
+    LOOP
+        -- Skip if trigger already exists on this table
+        IF NOT EXISTS (
+            SELECT 1 FROM pg_catalog.pg_trigger t
+            JOIN pg_catalog.pg_class c ON c.oid = t.tgrelid
+            JOIN pg_catalog.pg_namespace n ON n.oid = c.relnamespace
+            WHERE n.nspname = r.schema_name
+              AND c.relname = r.table_name
+              AND t.tgname = '_age_cache_invalidate'
+        )
+        THEN
+            EXECUTE format(
+                'CREATE TRIGGER _age_cache_invalidate '
+                'AFTER INSERT OR UPDATE OR DELETE OR TRUNCATE '
+                'ON %I.%I '
+                'FOR EACH STATEMENT '
+                'EXECUTE FUNCTION ag_catalog.age_invalidate_graph_cache()',
+                r.schema_name, r.table_name
+            );
+        END IF;
+    END LOOP;
+END;
+$$;

regress/expected/age_global_graph.out

@@ -413,6 +413,294 @@ NOTICE:  graph "ag_graph_3" has been dropped
 
 (1 row)
 
+-----------------------------------------------------------------------------------------------------------------------------
+--
+-- VLE cache invalidation tests
+--
+-- These tests verify that the graph version counter properly invalidates
+-- the VLE hash table cache when the graph is mutated, and that thin
+-- entry lazy property fetch returns correct data.
+--
+-- Setup: create a graph with a chain a->b->c->d
+SELECT * FROM create_graph('vle_cache_test');
+NOTICE:  graph "vle_cache_test" has been created
+ create_graph 
+--------------
+ 
+(1 row)
+
+SELECT * FROM cypher('vle_cache_test', $$
+  CREATE (a:Node {name: 'a'})-[:Edge]->(b:Node {name: 'b'})-[:Edge]->(c:Node {name: 'c'})-[:Edge]->(d:Node {name: 'd'})
+$$) AS (v agtype);
+ v 
+---
+(0 rows)
+
+-- VLE query: find all paths from a's neighbors (should find b, b->c, b->c->d)
+SELECT * FROM cypher('vle_cache_test', $$
+  MATCH (a:Node {name: 'a'})-[:Edge*1..3]->(n:Node)
+  RETURN n.name
+  ORDER BY n.name
+$$) AS (name agtype);
+ name 
+------
+ "b"
+ "c"
+ "d"
+(3 rows)
+
+-- Now add a new node e connected to d. This should invalidate the cache.
+SELECT * FROM cypher('vle_cache_test', $$
+  MATCH (d:Node {name: 'd'})
+  CREATE (d)-[:Edge]->(:Node {name: 'e'})
+$$) AS (v agtype);
+ v 
+---
+(0 rows)
+
+-- VLE query again: should now also find e via a->b->c->d->e (4 hops won't reach,
+-- but d->e is 1 hop from d, and a->b->c->d->e would be 4 hops from a).
+-- Increase range to *1..4 to include e
+SELECT * FROM cypher('vle_cache_test', $$
+  MATCH (a:Node {name: 'a'})-[:Edge*1..4]->(n:Node)
+  RETURN n.name
+  ORDER BY n.name
+$$) AS (name agtype);
+ name 
+------
+ "b"
+ "c"
+ "d"
+ "e"
+(4 rows)
+
+-- Test cache invalidation on DELETE: remove node c and its edges
+SELECT * FROM cypher('vle_cache_test', $$
+  MATCH (c:Node {name: 'c'})
+  DETACH DELETE c
+$$) AS (v agtype);
+ v 
+---
+(0 rows)
+
+-- VLE query: should only find b now (c is gone, so b->c path is broken)
+SELECT * FROM cypher('vle_cache_test', $$
+  MATCH (a:Node {name: 'a'})-[:Edge*1..4]->(n:Node)
+  RETURN n.name
+  ORDER BY n.name
+$$) AS (name agtype);
+ name 
+------
+ "b"
+(1 row)
+
+-- Test cache invalidation on SET: change b's name property
+SELECT * FROM cypher('vle_cache_test', $$
+  MATCH (b:Node {name: 'b'})
+  SET b.name = 'b_modified'
+  RETURN b.name
+$$) AS (name agtype);
+     name     
+--------------
+ "b_modified"
+(1 row)
+
+-- VLE query: verify the updated property is returned via lazy fetch
+SELECT * FROM cypher('vle_cache_test', $$
+  MATCH (a:Node {name: 'a'})-[:Edge*1..4]->(n:Node)
+  RETURN n.name
+  ORDER BY n.name
+$$) AS (name agtype);
+     name     
+--------------
+ "b_modified"
+(1 row)
+
+-- Test VLE with edge properties (exercises thin entry edge property fetch)
+SELECT * FROM drop_graph('vle_cache_test', true);
+NOTICE:  drop cascades to 4 other objects
+DETAIL:  drop cascades to table vle_cache_test._ag_label_vertex
+drop cascades to table vle_cache_test._ag_label_edge
+drop cascades to table vle_cache_test."Node"
+drop cascades to table vle_cache_test."Edge"
+NOTICE:  graph "vle_cache_test" has been dropped
+ drop_graph 
+------------
+ 
+(1 row)
+
+SELECT * FROM create_graph('vle_cache_test2');
+NOTICE:  graph "vle_cache_test2" has been created
+ create_graph 
+--------------
+ 
+(1 row)
+
+SELECT * FROM cypher('vle_cache_test2', $$
+  CREATE (a:N {name: 'a'})-[:E {weight: 1}]->(b:N {name: 'b'})-[:E {weight: 2}]->(c:N {name: 'c'})
+$$) AS (v agtype);
+ v 
+---
+(0 rows)
+
+-- VLE path output to verify edge properties are fetched correctly via
+-- thin entry lazy fetch. Returning the full path forces build_path()
+-- to call get_edge_entry_properties() for each edge in the result.
+-- The output must contain the correct weight values (1 and 2).
+SELECT * FROM cypher('vle_cache_test2', $$
+  MATCH p=(a:N {name: 'a'})-[:E *1..2]->(n:N)
+  RETURN p
+  ORDER BY n.name
+$$) AS (p agtype);
+                                                                                                                                                                                                                                                        p                                                                                                                                                                                                                                                         
+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
+ [{"id": 844424930131969, "label": "N", "properties": {"name": "a"}}::vertex, {"id": 1125899906842626, "label": "E", "end_id": 844424930131970, "start_id": 844424930131969, "properties": {"weight": 1}}::edge, {"id": 844424930131970, "label": "N", "properties": {"name": "b"}}::vertex]::path
+ [{"id": 844424930131969, "label": "N", "properties": {"name": "a"}}::vertex, {"id": 1125899906842626, "label": "E", "end_id": 844424930131970, "start_id": 844424930131969, "properties": {"weight": 1}}::edge, {"id": 844424930131970, "label": "N", "properties": {"name": "b"}}::vertex, {"id": 1125899906842625, "label": "E", "end_id": 844424930131971, "start_id": 844424930131970, "properties": {"weight": 2}}::edge, {"id": 844424930131971, "label": "N", "properties": {"name": "c"}}::vertex]::path
+(2 rows)
+
+-- VLE edge properties via UNWIND + relationships() to individually verify
+-- each edge's properties are correctly fetched from the heap via TID.
+SELECT * FROM cypher('vle_cache_test2', $$
+  MATCH p=(a:N {name: 'a'})-[:E *1..2]->(n:N)
+  WITH p, n
+  UNWIND relationships(p) AS e
+  RETURN n.name, e.weight
+  ORDER BY n.name, e.weight
+$$) AS (name agtype, weight agtype);
+ name | weight 
+------+--------
+ "b"  | 1
+ "c"  | 1
+ "c"  | 2
+(3 rows)
+
+-- Cleanup
+SELECT * FROM drop_graph('vle_cache_test2', true);
+NOTICE:  drop cascades to 4 other objects
+DETAIL:  drop cascades to table vle_cache_test2._ag_label_vertex
+drop cascades to table vle_cache_test2._ag_label_edge
+drop cascades to table vle_cache_test2."N"
+drop cascades to table vle_cache_test2."E"
+NOTICE:  graph "vle_cache_test2" has been dropped
+ drop_graph 
+------------
+ 
+(1 row)
+
+-----------------------------------------------------------------------------------------------------------------------------
+--
+-- VLE cache invalidation via direct SQL (trigger tests)
+--
+-- These tests verify that the SQL trigger (age_invalidate_graph_cache)
+-- properly invalidates the VLE cache when label tables are mutated
+-- via direct SQL INSERT, UPDATE, DELETE, and TRUNCATE — not via Cypher.
+--
+-- Setup: create graph with a chain a->b->c using Cypher
+SELECT * FROM create_graph('vle_trigger_test');
+NOTICE:  graph "vle_trigger_test" has been created
+ create_graph 
+--------------
+ 
+(1 row)
+
+SELECT * FROM cypher('vle_trigger_test', $$
+  CREATE (a:Node {name: 'a'})-[:Edge]->(b:Node {name: 'b'})-[:Edge]->(c:Node {name: 'c'})
+$$) AS (v agtype);
+ v 
+---
+(0 rows)
+
+-- Prime the VLE cache: find all nodes reachable from a
+SELECT * FROM cypher('vle_trigger_test', $$
+  MATCH (a:Node {name: 'a'})-[:Edge*1..3]->(n:Node)
+  RETURN n.name
+  ORDER BY n.name
+$$) AS (name agtype);
+ name 
+------
+ "b"
+ "c"
+(2 rows)
+
+-- Direct SQL INSERT on vertex: add a new vertex via SQL.
+-- This should fire the trigger and invalidate the VLE cache.
+-- Use _graphid() with the label's id and nextval for the entry id.
+INSERT INTO vle_trigger_test."Node" (id, properties)
+SELECT ag_catalog._graphid(l.id,
+           nextval('vle_trigger_test."Node_id_seq"'::regclass)),
+       '{"name": "d"}'::agtype
+FROM ag_catalog.ag_label l
+JOIN ag_catalog.ag_graph g ON g.graphid = l.graph
+WHERE g.name = 'vle_trigger_test' AND l.name = 'Node';
+-- VLE query: results should be unchanged (d has no edges) but cache was rebuilt
+SELECT * FROM cypher('vle_trigger_test', $$
+  MATCH (a:Node {name: 'a'})-[:Edge*1..3]->(n:Node)
+  RETURN n.name
+  ORDER BY n.name
+$$) AS (name agtype);
+ name 
+------
+ "b"
+ "c"
+(2 rows)
+
+-- Direct SQL UPDATE on vertex: change b's name to 'b_updated'
+-- This should fire the trigger and invalidate the VLE cache.
+UPDATE vle_trigger_test."Node"
+SET properties = '{"name": "b_updated"}'::agtype
+WHERE properties @> '{"name": "b"}'::agtype;
+-- VLE query: verify updated property is visible (cache invalidated by trigger)
+SELECT * FROM cypher('vle_trigger_test', $$
+  MATCH (a:Node {name: 'a'})-[:Edge*1..3]->(n:Node)
+  RETURN n.name
+  ORDER BY n.name
+$$) AS (name agtype);
+    name     
+-------------
+ "b_updated"
+ "c"
+(2 rows)
+
+-- Direct SQL DELETE on edge: remove the edge from b_updated to c
+DELETE FROM vle_trigger_test."Edge"
+WHERE end_id = (SELECT id FROM vle_trigger_test."Node"
+                WHERE properties @> '{"name": "c"}'::agtype);
+-- VLE query: only b_updated reachable now (edge to c is gone)
+SELECT * FROM cypher('vle_trigger_test', $$
+  MATCH (a:Node {name: 'a'})-[:Edge*1..3]->(n:Node)
+  RETURN n.name
+  ORDER BY n.name
+$$) AS (name agtype);
+    name     
+-------------
+ "b_updated"
+(1 row)
+
+-- Direct SQL TRUNCATE on edge table: remove all edges
+TRUNCATE vle_trigger_test."Edge";
+-- VLE query: no edges exist, should return no rows
+SELECT * FROM cypher('vle_trigger_test', $$
+  MATCH (a:Node {name: 'a'})-[:Edge*1..3]->(n:Node)
+  RETURN n.name
+  ORDER BY n.name
+$$) AS (name agtype);
+ name 
+------
+(0 rows)
+
+-- Cleanup
+SELECT * FROM drop_graph('vle_trigger_test', true);
+NOTICE:  drop cascades to 4 other objects
+DETAIL:  drop cascades to table vle_trigger_test._ag_label_vertex
+drop cascades to table vle_trigger_test._ag_label_edge
+drop cascades to table vle_trigger_test."Node"
+drop cascades to table vle_trigger_test."Edge"
+NOTICE:  graph "vle_trigger_test" has been dropped
+ drop_graph 
+------------
+ 
+(1 row)
+
 -----------------------------------------------------------------------------------------------------------------------------
 --
 -- End of tests