Declared Local Temporary Tables in PSQL (#9058)

Author: asfernandes

doc/sql.extensions/README.declared_local_temporary_tables.md

@@ -0,0 +1,255 @@
+# Declared Local Temporary Tables in PSQL (FB 6.0)
+
+Firebird 6.0 supports declaring local temporary tables inside PSQL blocks, procedures, functions and triggers.
+
+Declared Local Temporary Tables are local to the compiled PSQL object or `EXECUTE BLOCK`. They are not stored in
+database metadata and they do not require an `ON COMMIT` clause. Their data is scoped to the current execution frame.
+
+This feature is distinct from:
+
+- SQL-created Local Temporary Tables (`CREATE LOCAL TEMPORARY TABLE ...`), whose definitions are attachment-private
+  DDL objects.
+- Packaged temporary tables (`TEMPORARY TABLE ...` inside packages), whose definitions are persistent package-owned
+  metadata.
+
+## Syntax
+
+Declared Local Temporary Tables are declared in the same declaration section as variables, cursors and subroutines.
+
+```sql
+DECLARE [LOCAL] TEMPORARY TABLE <table_name>
+(
+    <column_definition> [, ...]
+);
+```
+
+There is no `ON COMMIT` clause.
+
+Example in `EXECUTE BLOCK`:
+
+```sql
+set term !;
+
+execute block returns (n integer)
+as
+    declare local temporary table t (
+        id integer not null,
+        val varchar(20)
+    );
+begin
+    insert into t(id, val) values (1, 'a');
+    insert into t(id, val) values (2, 'b');
+
+    update t set id = id + 10 where id = 1;
+    delete from t where id = 2;
+
+    select count(*) from t into n;
+    suspend;
+end!
+
+set term ;!
+```
+
+Example in a procedure:
+
+```sql
+set term !;
+
+create procedure p_count_values returns (n integer)
+as
+    declare local temporary table t (
+        id integer not null
+    );
+begin
+    insert into t(id) values (1);
+    insert into t(id) values (2);
+
+    select count(*) from t into n;
+    suspend;
+end!
+
+set term ;!
+```
+
+## Semantics
+
+Declared Local Temporary Tables have execution-frame data scope.
+
+- The table structure is part of the compiled PSQL statement, procedure, function or trigger.
+- Rows are private to the current execution frame.
+- Rows are discarded when that execution frame finishes.
+- Transaction commit or rollback does not define the table lifetime; there is no `ON COMMIT` behavior.
+- An autonomous transaction block inside the same execution frame sees the same rows.
+- Local subroutines may access declared local temporary tables from the containing PSQL scope. They see and modify the
+  rows of the current containing execution frame.
+- Local subroutines may also declare their own local temporary tables. Those rows are scoped to the subroutine execution
+  frame.
+- Recursive calls reuse the same compiled table structure, but each recursive execution frame has separate rows.
+
+Recursive example:
+
+```sql
+set term !;
+
+create procedure p_ltt_rec(d integer)
+    returns (depth integer, cnt_before integer, cnt_after integer)
+as
+    declare local temporary table t (
+        id integer
+    );
+begin
+    insert into t values (:d);
+    select count(*) from t into cnt_before;
+
+    if (d > 0) then
+    begin
+        for
+            select depth, cnt_before, cnt_after
+                from p_ltt_rec(:d - 1)
+                into depth, cnt_before, cnt_after
+        do
+            suspend;
+    end
+
+    select count(*) from t into cnt_after;
+    depth = d;
+    suspend;
+end!
+
+set term ;!
+```
+
+Each row returned by `p_ltt_rec(2)` reports `cnt_before = 1` and `cnt_after = 1`, because every recursive frame has
+its own table data.
+
+Autonomous transaction example:
+
+```sql
+set term !;
+
+execute block returns (n integer)
+as
+    declare local temporary table t (
+        id integer
+    );
+begin
+    insert into t values (10);
+
+    in autonomous transaction do
+    begin
+        select count(*) from t into n;
+    end
+
+    suspend;
+end!
+
+set term ;!
+```
+
+The autonomous block sees the row inserted by its parent execution frame.
+
+Local subroutine example:
+
+```sql
+set term !;
+
+execute block returns (n integer, s integer)
+as
+    declare local temporary table t (
+        id integer
+    );
+
+    declare procedure p_add(v integer)
+    as
+    begin
+        insert into t values (:v);
+    end
+
+    declare function f_count returns integer
+    as
+        declare variable ret integer;
+    begin
+        select count(*) from t into ret;
+        return ret;
+    end
+begin
+    insert into t values (1);
+    execute procedure p_add(2);
+
+    n = f_count();
+    select sum(id) from t into s;
+    suspend;
+end!
+
+set term ;!
+```
+
+The local procedure and function access `t` declared by the outer `EXECUTE BLOCK`. They use the same row set as the
+current outer execution frame.
+
+## Visibility and Name Resolution
+
+Declared Local Temporary Tables are visible only to SQL statements compiled inside the declaring PSQL scope and its
+local subroutines.
+
+- Unqualified references to the declared name resolve to the declared local table.
+- SQL statements in local subroutines may also reference declared local temporary tables from the containing PSQL scope.
+- The declaration name cannot be schema-qualified or package-qualified.
+- The table is not present in `RDB$RELATIONS`, `RDB$RELATION_FIELDS`, monitoring metadata or other persistent metadata.
+- Dynamic SQL, including `EXECUTE STATEMENT`, is parsed separately and does not see declared local temporary tables.
+
+The name shares the local declaration namespace. It cannot duplicate an input parameter, output parameter, local
+variable, cursor or another declared local table in the same declaration scope.
+
+## Supported Operations
+
+Declared Local Temporary Tables can be used by regular DML statements compiled in the same PSQL scope:
+
+```sql
+insert into t (...) values (...);
+select ... from t ...;
+update t set ... where ...;
+delete from t where ...;
+```
+
+They can be used in subqueries, joins and cursor loops like other record sources, subject to the restrictions below.
+
+## Restrictions
+
+Declared Local Temporary Tables intentionally support a small table definition surface.
+
+Column restrictions:
+
+- `DEFAULT` values are not supported.
+- `COMPUTED BY` columns are not supported.
+- `IDENTITY` columns are not supported.
+- Array columns are not supported.
+- Field-level `NOT NULL` is supported, but only as an unnamed constraint.
+
+Constraint restrictions:
+
+- Table constraints are not supported.
+- Primary keys are not supported.
+- Foreign keys are not supported.
+- Check constraints are not supported.
+- Named constraints are not supported.
+
+Other restrictions:
+
+- A single PSQL statement, procedure, function or trigger may declare at most 1024 local temporary tables.
+- Indexes are not supported.
+- Triggers on declared local temporary tables are not supported.
+- Explicit privileges are not supported.
+- `ALTER TABLE`, `DROP TABLE`, `CREATE INDEX`, `ALTER INDEX` and `DROP INDEX` are not valid for declared local
+  temporary tables.
+- Declared local temporary tables cannot be referenced by persistent metadata objects outside the PSQL object where
+  they are declared.
+
+## Notes
+
+Declared Local Temporary Tables are useful for temporary intermediate data that should not be exposed through database
+metadata and should be automatically isolated per PSQL execution frame.
+
+Use SQL-created Local Temporary Tables when the temporary table definition must be created and addressed dynamically by
+the attachment. Use packaged temporary tables when the definition must be part of package metadata and visible through
+package name resolution.

src/dsql/DdlNodes.epp

@@ -10229,7 +10229,7 @@ void CreateRelationNode::defineLocalTempTable(thread_db* tdbb, DsqlCompilerScrat
 	if (externalFile)
 		status_exception::raise(Arg::Gds(isc_random) << "External file is not allowed for local temporary tables");
 
-	if (attachment->att_local_temporary_tables.count() >= MAX_LTT_COUNT)
+	if (attachment->att_local_temporary_tables.count() >= MAX_CREATED_LTT_COUNT)
 	{
 		ERR_post(Arg::Gds(isc_imp_exc) <<
 				 Arg::Gds(isc_random) <<
@@ -10299,14 +10299,14 @@ void CreateRelationNode::defineLocalTempTable(thread_db* tdbb, DsqlCompilerScrat
 	// Assign LTT relation ID
 
 	if (!attachment->att_next_ltt_id.has_value())
-		attachment->att_next_ltt_id = MAX_LTT_ID;
+		attachment->att_next_ltt_id = MAX_CREATED_LTT_ID;
 
 	const auto startLttId = attachment->att_next_ltt_id.value();
 
 	while (true)
 	{
-		if (attachment->att_next_ltt_id.value() >= MAX_LTT_ID)
-			attachment->att_next_ltt_id = MIN_LTT_ID;
+		if (attachment->att_next_ltt_id.value() >= MAX_CREATED_LTT_ID)
+			attachment->att_next_ltt_id = MIN_CREATED_LTT_ID;
 		else
 			++attachment->att_next_ltt_id.value();
 

src/dsql/DsqlCompilerScratch.cpp

@@ -486,7 +486,7 @@ void DsqlCompilerScratch::putLocalVariableInit(dsql_var* variable, const Declare
 // Put maps in subroutines for outer variables/parameters usage.
 void DsqlCompilerScratch::putOuterMaps()
 {
-	if (!outerMessagesMap.count() && !outerVarsMap.count())
+	if (outerMessagesMap.isEmpty() && outerVarsMap.isEmpty() && outerLocalTablesMap.isEmpty())
 		return;
 
 	appendUChar(blr_outer_map);
@@ -505,6 +505,13 @@ void DsqlCompilerScratch::putOuterMaps()
 		appendUShort(outer);
 	}
 
+	for (auto& [outer, inner] : outerLocalTablesMap)
+	{
+		appendUChar(blr_outer_map_local_table);
+		appendUShort(outer);
+		appendUShort(inner);
+	}
+
 	appendUChar(blr_end);
 }
 
@@ -553,6 +560,41 @@ dsql_var* DsqlCompilerScratch::resolveVariable(const MetaName& varName)
 	return NULL;
 }
 
+DeclareLocalTableNode* DsqlCompilerScratch::getLocalTable(const MetaName& name, bool* outerDecl)
+{
+	DeclareLocalTableNode* table = nullptr;
+	localTableNames.get(name, table);
+
+	if (outerDecl)
+		*outerDecl = false;
+
+	if (!table && mainScratch)
+	{
+		table = mainScratch->getLocalTable(name);
+
+		if (table && outerDecl)
+			*outerDecl = true;
+	}
+
+	return table;
+}
+
+USHORT DsqlCompilerScratch::getOuterLocalTableNumber(USHORT tableNumber)
+{
+	if (const auto innerNumber = outerLocalTablesMap.get(tableNumber))
+		return *innerNumber;
+
+	const auto innerNumber = localTableNumber++;
+	outerLocalTablesMap.put(tableNumber, innerNumber);
+
+	return innerNumber;
+}
+
+void DsqlCompilerScratch::putLocalTable(DeclareLocalTableNode* table)
+{
+	localTableNames.put(table->dsqlName, table);
+}
+
 // Generate BLR for a return.
 void DsqlCompilerScratch::genReturn(bool eosFlag)
 {

src/dsql/DsqlCompilerScratch.h

@@ -97,6 +97,7 @@ class DsqlCompilerScratch : public BlrDebugWriter
 		  labels(p),
 		  cursors(p),
 		  localTables(p),
+		  localTableNames(p),
 		  aliasRelationPrefix(p),
 		  package(p),
 		  currCtes(p),
@@ -106,6 +107,7 @@ class DsqlCompilerScratch : public BlrDebugWriter
 		  mainScratch(aMainScratch),
 		  outerMessagesMap(p),
 		  outerVarsMap(p),
+		  outerLocalTablesMap(p),
 		  ddlSchema(p),
 		  ctes(p),
 		  cteAliases(p),
@@ -198,6 +200,11 @@ class DsqlCompilerScratch : public BlrDebugWriter
 	dsql_var* makeVariable(dsql_fld*, const char*, const dsql_var::Type type, USHORT,
 		USHORT, std::optional<USHORT> = std::nullopt);
 	dsql_var* resolveVariable(const MetaName& varName);
+
+	DeclareLocalTableNode* getLocalTable(const MetaName& name, bool* outerDecl = nullptr);
+	USHORT getOuterLocalTableNumber(USHORT tableNumber);
+	void putLocalTable(DeclareLocalTableNode* table);
+
 	void genReturn(bool eosFlag = false);
 
 	void genParameters(Firebird::Array<NestConst<ParameterClause> >& parameters,
@@ -323,6 +330,7 @@ class DsqlCompilerScratch : public BlrDebugWriter
 	Firebird::Array<DeclareCursorNode*> cursors; // Cursors
 	USHORT localTableNumber = 0;			// Local table number
 	Firebird::Array<DeclareLocalTableNode*> localTables; // Local tables
+	Firebird::LeftPooledMap<MetaName, DeclareLocalTableNode*> localTableNames;
 	USHORT inSelectList = 0;				// now processing "select list"
 	USHORT inWhereClause = 0;				// processing "where clause"
 	USHORT inGroupByClause = 0;				// processing "group by clause"
@@ -350,6 +358,7 @@ class DsqlCompilerScratch : public BlrDebugWriter
 	DsqlCompilerScratch* mainScratch = nullptr;
 	Firebird::NonPooledMap<USHORT, USHORT> outerMessagesMap;	// <outer, inner>
 	Firebird::NonPooledMap<USHORT, USHORT> outerVarsMap;		// <outer, inner>
+	Firebird::NonPooledMap<USHORT, USHORT> outerLocalTablesMap;	// <outer, inner>
 	MetaName ddlSchema;
 	Firebird::AutoPtr<Firebird::ObjectsArray<Firebird::MetaString>> cachedDdlSchemaSearchPath;
 	dsql_msg* recordKeyMessage = nullptr;	// Side message for positioned DML