Skip to content

Commit 67b9ac2

Browse files
committed
Recursive CTEs: WITH name AS (anchor UNION ALL recursive [UNION ALL …]) SELECT … with multi-anchor / multi-recursive support and OPTION (MAXRECURSION N) (default 100, 0 = unlimited). Branch-by-branch body parser via internalized Selection.ParseIntersectChain detects self-reference per branch through binding.IsRecursivePartParse + a per-branch counter; CteBinding gains Schema / ColumnNames captured from anchor plus a CurrentIterationRows runtime slot the recursive Selection rebinds between iterations. FromRecursiveCte runs anchors once into the seed, then iterates each recursive branch against the previous-iteration rowset until empty (or MaxRecursion trips Msg 530 with literal limit). Strict per-column type-equality between anchor and recursive (Msg 240) — diverges from regular UNION ALL's Promote. Errors: 240 / 247 anchor-after-recursive / 252 missing UNION ALL / 253 multiple self-refs / 530. Recursive-part feature restrictions (460 / 461 / 462 / 465 / 467) deferred — accepted with possibly-incorrect semantics.
1 parent 32680cc commit 67b9ac2

9 files changed

Lines changed: 591 additions & 42 deletions

File tree

CLAUDE.md

Lines changed: 22 additions & 9 deletions
Original file line numberDiff line numberDiff line change
@@ -339,20 +339,33 @@ Three entry points share one per-connection undo log: implicit (statement-level
339339
### `rowversion` (legacy synonym `timestamp`)
340340
8-byte big-endian database-scoped monotonic counter; `Simulation.AllocateRowVersion` advances on every INSERT into a rowversion-bearing table and every UPDATE that affects a row in one. Storage type name surfaces as `timestamp` in `information_schema` regardless of declaration keyword. Explicit insert → Msg 273; explicit update → Msg 272; second column on a table → Msg 2738. Outbound CAST: `varbinary(N)` / `binary(N)` copy the 8 bytes; `bigint` reads big-endian. `Promote(RowVersion, Varbinary)``Varbinary` so EF Core's `WHERE [rv] = @originalRv` optimistic-concurrency parameter works directly. `EF Core [Timestamp]` SaveChanges round-trips end-to-end via `UPDATE ... OUTPUT INSERTED.[RowVersion] WHERE [Id] = @p AND [RowVersion] = @originalRv`.
341341

342-
### Common table expressions (non-recursive)
343-
`WITH name [(col, …)] AS (SELECT …) [, …] {SELECT|INSERT|UPDATE|DELETE|MERGE} …`. The WITH prefix scopes to exactly one immediately-following statement; the binding is gone after that statement runs. Probe-confirmed against SQL Server 2025 on 2026-05-10.
342+
### Common table expressions
343+
`WITH name [(col, …)] AS (SELECT …) [, …] {SELECT|INSERT|UPDATE|DELETE|MERGE} …`. The WITH prefix scopes to exactly one immediately-following statement; the binding is gone after that statement runs. Both non-recursive and recursive forms are modeled. Probe-confirmed against SQL Server 2025 on 2026-05-10 / 2026-05-11.
344344

345-
`Simulation.ParseCteBindings` runs at the statement-loop top before dispatch, populating `ParserContext.CteBindings` (a `Dictionary<string, CteBinding>`). Each binding's body parses at `depth: 1` because the `(SELECT …)` brackets it; cleared at the top of the next iteration. `ParseSingleFromSource` consults the bindings before falling through to `Simulation.HeapTables` — a CTE name shadows a real table for the prefixed statement (probe-confirmed). Resolution builds a `FromSource` with `lateralPlan: cteBinding.Plan`, so each FROM-side reference re-runs the inner Selection (parallel to derived tables in FROM). Multiple comma-separated CTEs cascade: later ones see earlier ones because the prior bindings are already in the dictionary when the later body is parsed.
345+
`Simulation.ParseCteBindings` runs at the statement-loop top before dispatch, populating `ParserContext.CteBindings` (a `Dictionary<string, CteBinding>`). Each body parses branch-by-branch via `Selection.ParseIntersectChain` (the higher-precedence set-op chain parser); the surrounding loop walks UNION / UNION ALL / EXCEPT operators and tracks per-branch self-reference. Bindings are cleared at the top of the next statement-loop iteration. `ParseSingleFromSource` consults the bindings before falling through to `Simulation.HeapTables` — a CTE name shadows a real table for the prefixed statement (probe-confirmed). Multiple comma-separated CTEs cascade: later ones see earlier ones because the prior bindings are already in the dictionary when the later body is parsed.
346346

347-
**Recursive CTEs aren't modeled.** Self-reference is detected via a sentinel: `ParseCteBindings` registers a `CteBinding` with `Plan = null` *before* parsing the body, so a body's `FROM cte_name` resolves to the sentinel and `ParseSingleFromSource` raises `NotSupportedException("Recursive common table expressions are not modeled.")`. The non-recursive sibling replaces the sentinel with the resolved plan once the body finishes parsing.
347+
**Non-recursive form**: when the body has no self-references, branches are folded via the standard `Selection.CombineSetOps` (so type-promotion across UNION / UNION ALL / EXCEPT matches a regular set-op chain). Resolution builds a `FromSource` with `lateralPlan: binding.Plan`, so each FROM-side reference re-runs the inner Selection (parallel to derived tables in FROM).
348348

349-
**Errors modeled:** **Msg 239** duplicate CTE name (`"Duplicate common table expression name 'a' was specified."`); **Msg 8158** rename list has fewer columns than body projection (`"'name' has more columns than were specified in the column list."`); **Msg 8159** rename list has more columns than body projection (`"'name' has fewer columns than were specified in the column list."`); **Msg 1033** ORDER BY in CTE body without TOP / OFFSET / FETCH (`"The ORDER BY clause is invalid in views, inline functions, derived tables, subqueries, and common table expressions, unless TOP, OFFSET or FOR XML is also specified."`).
349+
**Recursive form**: when any branch self-references, the body parser splits branches into anchor (no self-ref) and recursive (one self-ref) lists, then builds a Selection via `Selection.FromRecursiveCte(anchors, recursives, binding)`. The recursive Selection runs anchors once into a seed rowset, then iterates: each iteration rebinds `binding.CurrentIterationRows` to the previous iteration's output and runs every recursive branch, accumulating all rows. Termination: an iteration produces zero rows, OR `binding.MaxRecursion` (default 100, override via `OPTION (MAXRECURSION N)` at the end of the outer SELECT) is exceeded → **Msg 530** with the literal limit value in the message. `OPTION (MAXRECURSION 0)` disables the cap.
350350

351-
The Msg 1033 enforcement requires knowing whether the body's plan has a TOP / OFFSET / FETCH at any layer — added a new `Selection.HasTopOrOffsetOrFetch` field threaded through all four `new Selection(...)` call sites (`BuildSynthesizedSqlRow` / `BuildSqlProjection` / `CombineSetOps` / `ApplyTopLevelOrderBy`). The set-op combiner inherits from its branches; the top-level ORDER-BY wrapper inherits from the inner plan plus its own OFFSET / FETCH counts. Derived tables face the same SQL Server rule but the simulator currently allows ORDER BY without TOP / OFFSET there — pre-existing divergence orthogonal to this bundle.
351+
Self-reference resolution: during the recursive-part parse, `binding.IsRecursivePartParse` is set after the anchor branch completes (which captures the binding's `Schema` / `ColumnNames` from the anchor's projection). Subsequent branch parses route self-references through a FromSource backed by `SelfReferenceRows(binding)` — a closure that reads `binding.CurrentIterationRows` at iterator-start time. Each branch's `binding.SelfReferenceCountInCurrentBranch` is reset before parse; the body parser inspects the count after parse to classify the branch as anchor (count = 0) or recursive (count = 1) and to enforce one-self-ref-per-branch (Msg 253).
352352

353-
**Msg 319** (WITH after a non-terminated previous statement) isn't enforced. The simulator's statement loop consumes one statement per iteration and naturally treats WITH at iteration top as a fresh prefix regardless of prior `;` — more permissive than real SQL Server. Apps that idiomatically terminate statements (or use a single statement) won't notice.
353+
**Recursive CTE errors modeled:**
354+
- **Msg 240**: anchor and recursive parts produce different per-column types. Recursive CTEs require strict type-equality (no Promote-style widening, unlike regular UNION ALL); the user must explicitly cast.
355+
- **Msg 247**: an anchor branch (no self-ref) appears after a recursive branch — anchors must precede recursives.
356+
- **Msg 252**: the body has a self-reference but no top-level UNION ALL splitting it from an anchor (e.g. `WITH c AS (SELECT n+1 FROM c WHERE n < 5) …` — no anchor); also fires when UNION-without-ALL is used between branches.
357+
- **Msg 253**: one recursive branch references the CTE more than once (e.g. `c CROSS JOIN c`).
358+
- **Msg 530**: MAXRECURSION exceeded; the literal limit value appears in the message.
354359

355-
EF Core 10 emits non-recursive CTEs in some shapes (TPC inheritance, certain `Distinct/OrderBy/Skip` patterns); also reachable from raw SQL (`FromSqlInterpolated` / direct command text) including the `WITH … INSERT … SELECT` shape.
360+
**Recursive part restrictions not enforced** (pre-existing CLAUDE.md gap, deferred to a follow-up bundle): DISTINCT / TOP / OFFSET / aggregate / GROUP BY / OUTER JOIN inside a recursive branch should raise Msg 460/461/467/462 respectively in real SQL Server. The simulator silently accepts those and produces possibly-incorrect semantics; users hitting these constructs in recursive parts are writing SQL that real SQL Server rejects anyway. Recursive references inside subqueries (Msg 465) likewise aren't enforced.
361+
362+
**Non-recursive errors modeled:** **Msg 239** duplicate CTE name; **Msg 8158** / **Msg 8159** rename-list count mismatch; **Msg 1033** ORDER BY in CTE body without TOP / OFFSET / FETCH (gated via `Selection.HasTopOrOffsetOrFetch` threaded through all `new Selection(...)` call sites — set-op combiner inherits from branches; top-level ORDER-BY wrapper inherits from the inner plan plus its own OFFSET / FETCH).
363+
364+
**Msg 319** (WITH after a non-terminated previous statement) isn't enforced — the simulator's statement loop consumes one statement per iteration and treats WITH at iteration top as a fresh prefix regardless of prior `;`; apps that idiomatically terminate statements won't notice.
365+
366+
`OPTION (MAXRECURSION N)` parses inside `Selection.ParseQueryExpression` after the optional ORDER BY / OFFSET / FETCH; the parser walks `context.CteBindings.Values` and writes `MaxRecursion` to each, so recursive Selections see the override at execute time. `MAXRECURSION` is the only OPTION hint modeled; other hints (`OPTIMIZE FOR`, `RECOMPILE`, `MERGE JOIN`, etc.) raise `NotSupportedException`.
367+
368+
EF Core 10 emits non-recursive CTEs in some shapes (TPC inheritance, certain `Distinct/OrderBy/Skip` patterns); recursive CTEs are reachable only via raw SQL (`FromSqlInterpolated` / direct command text) — EF Core's LINQ surface doesn't compile to recursive CTEs from idiomatic queries.
356369

357370
### INSERT … SELECT
358371
`INSERT [INTO] target [(cols)] SELECT …` accepts the same Selection grammar as a top-level SELECT — WHERE / JOIN / GROUP BY / aggregates / ORDER BY / TOP / OFFSET-FETCH / UNION / INTERSECT / EXCEPT all work on the source side. Probe-confirmed against SQL Server 2025 on 2026-05-10.
@@ -412,7 +425,7 @@ Full `DbDataReader` contract. Typed accessors read `SqlValue` directly via the c
412425
- `UNION` / `UNION ALL` inside a subquery body.
413426
- Row-constructor `IN ((1,2), (3,4))`.
414427
- Window functions other than `ROW_NUMBER` and the aggregate-OVER family (see "Window functions" above).
415-
- Recursive CTEs (`WITH name AS (… UNION ALL … FROM name …)`); self-reference detected and raises `NotSupportedException`. Non-recursive CTEs are modeled.
428+
- Recursive-part feature restrictions (Msg 460 DISTINCT / Msg 461 TOP / Msg 462 OUTER JOIN / Msg 467 aggregate-or-GROUP-BY / Msg 465 ref-in-subquery) — the recursive iteration accepts these constructs and produces possibly-incorrect semantics rather than raising. Apps that exercise these in real SQL Server hit the rejection there too, so the simulator's behavior diverges from real-server errors but matches the broader "don't write that" guidance. Recursive CTEs themselves and the structural error paths (Msg 240 / 247 / 252 / 253 / 530) are modeled.
416429
- `LIKE` with `COLLATE` override (default collation only — case-insensitive Latin1_General-shaped).
417430
- `CONVERT` / `TRY_CONVERT` style codes other than `0` / `120` / `121` for date-like → string.
418431
- `LEN(ntext)` raising Msg 8116 (function-level text/ntext/image restrictions); legacy `READTEXT` / `WRITETEXT` / `UPDATETEXT`.

SqlServerSimulator.Tests/CommonTableExpressionTests.cs

Lines changed: 138 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -161,14 +161,146 @@ public void Cte_OrderByWithOffsetFetch_Allowed()
161161
=> AreEqual(2, WithSourceTable().ExecuteScalar("with c as (select id from src order by id desc offset 1 rows fetch next 1 rows only) select id from c"));
162162

163163
[TestMethod]
164-
public void Cte_RecursiveSelfReference_RaisesNotSupported()
165-
=> Throws<NotSupportedException>(() => WithSourceTable().ExecuteNonQuery("""
166-
with a as (
164+
public void Cte_Recursive_Counter()
165+
{
166+
using var reader = new Simulation().ExecuteReader("""
167+
with c as (
167168
select 1 as n
168169
union all
169-
select n + 1 from a where n < 5
170-
) select * from a
171-
"""));
170+
select n + 1 from c where n < 5
171+
) select n from c order by n
172+
""");
173+
var values = new List<int>();
174+
while (reader.Read())
175+
values.Add(reader.GetInt32(0));
176+
CollectionAssert.AreEqual(new[] { 1, 2, 3, 4, 5 }, values);
177+
}
178+
179+
[TestMethod]
180+
public void Cte_Recursive_HierarchyTraversal()
181+
{
182+
var simulation = new Simulation();
183+
_ = simulation.ExecuteNonQuery("""
184+
create table emp (id int, mgr_id int, name nvarchar(20));
185+
insert emp values (1, null, 'CEO'), (2, 1, 'VP1'), (3, 1, 'VP2'), (4, 2, 'Dir1'), (5, 4, 'Mgr1')
186+
""");
187+
using var reader = simulation.ExecuteReader("""
188+
with org as (
189+
select id, mgr_id, name, 0 as depth from emp where mgr_id is null
190+
union all
191+
select e.id, e.mgr_id, e.name, o.depth + 1
192+
from emp e inner join org o on e.mgr_id = o.id
193+
)
194+
select id, name, depth from org order by depth, id
195+
""");
196+
var rows = new List<(int id, string name, int depth)>();
197+
while (reader.Read())
198+
rows.Add((reader.GetInt32(0), reader.GetString(1), reader.GetInt32(2)));
199+
CollectionAssert.AreEqual(new[] { (1, "CEO", 0), (2, "VP1", 1), (3, "VP2", 1), (4, "Dir1", 2), (5, "Mgr1", 3) }, rows);
200+
}
201+
202+
[TestMethod]
203+
public void Cte_Recursive_DefaultMaxRecursion100_RaisesMsg530()
204+
=> new Simulation().AssertSqlError("with c as (select 1 as n union all select n+1 from c) select count(*) from c", 530,
205+
"The statement terminated. The maximum recursion 100 has been exhausted before statement completion.");
206+
207+
[TestMethod]
208+
public void Cte_Recursive_OptionMaxRecursionLow_LimitInMessage()
209+
=> new Simulation().AssertSqlError("with c as (select 1 as n union all select n+1 from c) select count(*) from c option (maxrecursion 5)", 530,
210+
"The statement terminated. The maximum recursion 5 has been exhausted before statement completion.");
211+
212+
[TestMethod]
213+
public void Cte_Recursive_OptionMaxRecursionZero_Unlimited()
214+
=> AreEqual(200, new Simulation().ExecuteScalar("with c as (select 1 as n union all select n+1 from c where n < 200) select count(*) from c option (maxrecursion 0)"));
215+
216+
[TestMethod]
217+
public void Cte_Recursive_MultipleAnchors()
218+
{
219+
using var reader = new Simulation().ExecuteReader("""
220+
with c as (
221+
select 1 as n
222+
union all select 100
223+
union all select n+1 from c where n < 3
224+
) select n from c order by n
225+
""");
226+
var values = new List<int>();
227+
while (reader.Read())
228+
values.Add(reader.GetInt32(0));
229+
CollectionAssert.AreEqual(new[] { 1, 2, 3, 100 }, values);
230+
}
231+
232+
[TestMethod]
233+
public void Cte_Recursive_AnchorAfterRecursive_RaisesMsg247()
234+
=> new Simulation().AssertSqlError("with c as (select 1 as n union all select 2 union all select n+1 from c where n < 4 union all select 99) select n from c", 247,
235+
"An anchor member was found in the recursive part of recursive query \"c\".");
236+
237+
[TestMethod]
238+
public void Cte_Recursive_MultipleSelfRefInOneBranch_RaisesMsg253()
239+
=> new Simulation().AssertSqlError("with c as (select 1 as n union all select c1.n + c2.n from c c1 cross join c c2 where c1.n < 5) select n from c", 253,
240+
"Recursive member of a common table expression 'c' has multiple recursive references.");
241+
242+
[TestMethod]
243+
public void Cte_Recursive_TypeMismatch_RaisesMsg240()
244+
=> new Simulation().AssertSqlError("with c as (select cast(1 as smallint) as n union all select cast(n+1 as int) from c where n < 5) select n from c", 240,
245+
"Types don't match between the anchor and the recursive part in column \"n\" of recursive query \"c\".");
246+
247+
[TestMethod]
248+
public void Cte_Recursive_UnionWithoutAll_RaisesMsg252()
249+
=> new Simulation().AssertSqlError("with c as (select 1 as n union select n+1 from c where n < 3) select n from c", 252,
250+
"Recursive common table expression 'c' does not contain a top-level UNION ALL operator.");
251+
252+
[TestMethod]
253+
public void Cte_Recursive_NoUnionAtAll_RaisesMsg252()
254+
=> new Simulation().AssertSqlError("with c as (select n+1 from c where n < 5) select n from c", 252);
255+
256+
[TestMethod]
257+
public void Cte_Recursive_ZeroIterations_OnlyAnchor()
258+
{
259+
// Anchor produces a row whose WHERE in the recursive part rejects
260+
// immediately; result is just the anchor's rows.
261+
using var reader = new Simulation().ExecuteReader("with c as (select 100 as n union all select n+1 from c where n < 50) select n from c");
262+
var values = new List<int>();
263+
while (reader.Read())
264+
values.Add(reader.GetInt32(0));
265+
CollectionAssert.AreEqual(new[] { 100 }, values);
266+
}
267+
268+
[TestMethod]
269+
public void Cte_Recursive_ConsumesParentRowsPerIteration()
270+
{
271+
// The recursive branch reads the previous-iteration rowset, NOT
272+
// the cumulative result-so-far. With anchor n=1 and recursive
273+
// `select n+1 from c where n < 3`, iteration 1 produces n=2,
274+
// iteration 2 produces n=3, iteration 3 produces no rows (n=3
275+
// doesn't satisfy n<3), so the result is {1, 2, 3}.
276+
using var reader = new Simulation().ExecuteReader("with c as (select 1 as n union all select n+1 from c where n < 3) select n from c order by n");
277+
var values = new List<int>();
278+
while (reader.Read())
279+
values.Add(reader.GetInt32(0));
280+
CollectionAssert.AreEqual(new[] { 1, 2, 3 }, values);
281+
}
282+
283+
[TestMethod]
284+
public void Cte_Recursive_AggregateOverFinalResult()
285+
=> AreEqual(15, new Simulation().ExecuteScalar("with c as (select 1 as n union all select n+1 from c where n < 5) select sum(n) from c"));
286+
287+
[TestMethod]
288+
public void Cte_Recursive_StringConcatenation()
289+
{
290+
using var reader = new Simulation().ExecuteReader("""
291+
with path as (
292+
select cast('root' as varchar(100)) as p, 0 as depth
293+
union all
294+
select cast(p + '/' + cast(depth + 1 as varchar(10)) as varchar(100)), depth + 1
295+
from path where depth < 3
296+
)
297+
select p from path order by depth
298+
""");
299+
var values = new List<string>();
300+
while (reader.Read())
301+
values.Add(reader.GetString(0));
302+
CollectionAssert.AreEqual(new[] { "root", "root/1", "root/1/2", "root/1/2/3" }, values);
303+
}
172304

173305
[TestMethod]
174306
public void Cte_BindingClearsBetweenStatements()

0 commit comments

Comments
 (0)