Skip to content

Commit 67d13f8

Browse files
committed
Non-recursive CTE prefix: WITH name [(cols)] AS (SELECT …) [, …] {SELECT|INSERT|UPDATE|DELETE|MERGE} …. ParseCteBindings runs at the statement-loop top, populating ParserContext.CteBindings (cleared on the next iteration); body parses at depth 1 because of the surrounding parens. ParseSingleFromSource consults the bindings before HeapTables — CTE name shadows a real table for the prefixed statement; each FROM-side reference re-runs the inner Selection via the existing LateralPlan slot. Multiple comma-separated CTEs cascade. Recursive CTE detection via a null-Plan sentinel registered before body parse — self-reference raises NotSupportedException. Errors: Msg 239 duplicate name, Msg 8158/8159 rename-list count mismatch, Msg 1033 ORDER BY without TOP/OFFSET (gated via a new Selection.HasTopOrOffsetOrFetch field threaded through all four ctor sites).
1 parent bd3db51 commit 67d13f8

10 files changed

Lines changed: 479 additions & 5 deletions

File tree

CLAUDE.md

Lines changed: 16 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -339,6 +339,21 @@ Three entry points share one per-connection undo log: implicit (statement-level
339339
### `rowversion` (legacy synonym `timestamp`)
340340
8-byte big-endian database-scoped monotonic counter; `Simulation.AllocateRowVersion` advances on every INSERT into a rowversion-bearing table and every UPDATE that affects a row in one. Storage type name surfaces as `timestamp` in `information_schema` regardless of declaration keyword. Explicit insert → Msg 273; explicit update → Msg 272; second column on a table → Msg 2738. Outbound CAST: `varbinary(N)` / `binary(N)` copy the 8 bytes; `bigint` reads big-endian. `Promote(RowVersion, Varbinary)``Varbinary` so EF Core's `WHERE [rv] = @originalRv` optimistic-concurrency parameter works directly. `EF Core [Timestamp]` SaveChanges round-trips end-to-end via `UPDATE ... OUTPUT INSERTED.[RowVersion] WHERE [Id] = @p AND [RowVersion] = @originalRv`.
341341

342+
### Common table expressions (non-recursive)
343+
`WITH name [(col, …)] AS (SELECT …) [, …] {SELECT|INSERT|UPDATE|DELETE|MERGE} …`. The WITH prefix scopes to exactly one immediately-following statement; the binding is gone after that statement runs. Probe-confirmed against SQL Server 2025 on 2026-05-10.
344+
345+
`Simulation.ParseCteBindings` runs at the statement-loop top before dispatch, populating `ParserContext.CteBindings` (a `Dictionary<string, CteBinding>`). Each binding's body parses at `depth: 1` because the `(SELECT …)` brackets it; cleared at the top of the next iteration. `ParseSingleFromSource` consults the bindings before falling through to `Simulation.HeapTables` — a CTE name shadows a real table for the prefixed statement (probe-confirmed). Resolution builds a `FromSource` with `lateralPlan: cteBinding.Plan`, so each FROM-side reference re-runs the inner Selection (parallel to derived tables in FROM). Multiple comma-separated CTEs cascade: later ones see earlier ones because the prior bindings are already in the dictionary when the later body is parsed.
346+
347+
**Recursive CTEs aren't modeled.** Self-reference is detected via a sentinel: `ParseCteBindings` registers a `CteBinding` with `Plan = null` *before* parsing the body, so a body's `FROM cte_name` resolves to the sentinel and `ParseSingleFromSource` raises `NotSupportedException("Recursive common table expressions are not modeled.")`. The non-recursive sibling replaces the sentinel with the resolved plan once the body finishes parsing.
348+
349+
**Errors modeled:** **Msg 239** duplicate CTE name (`"Duplicate common table expression name 'a' was specified."`); **Msg 8158** rename list has fewer columns than body projection (`"'name' has more columns than were specified in the column list."`); **Msg 8159** rename list has more columns than body projection (`"'name' has fewer columns than were specified in the column list."`); **Msg 1033** ORDER BY in CTE body without TOP / OFFSET / FETCH (`"The ORDER BY clause is invalid in views, inline functions, derived tables, subqueries, and common table expressions, unless TOP, OFFSET or FOR XML is also specified."`).
350+
351+
The Msg 1033 enforcement requires knowing whether the body's plan has a TOP / OFFSET / FETCH at any layer — added a new `Selection.HasTopOrOffsetOrFetch` field threaded through all four `new Selection(...)` call sites (`BuildSynthesizedSqlRow` / `BuildSqlProjection` / `CombineSetOps` / `ApplyTopLevelOrderBy`). The set-op combiner inherits from its branches; the top-level ORDER-BY wrapper inherits from the inner plan plus its own OFFSET / FETCH counts. Derived tables face the same SQL Server rule but the simulator currently allows ORDER BY without TOP / OFFSET there — pre-existing divergence orthogonal to this bundle.
352+
353+
**Msg 319** (WITH after a non-terminated previous statement) isn't enforced. The simulator's statement loop consumes one statement per iteration and naturally treats WITH at iteration top as a fresh prefix regardless of prior `;` — more permissive than real SQL Server. Apps that idiomatically terminate statements (or use a single statement) won't notice.
354+
355+
EF Core 10 emits non-recursive CTEs in some shapes (TPC inheritance, certain `Distinct/OrderBy/Skip` patterns); also reachable from raw SQL (`FromSqlInterpolated` / direct command text) including the `WITH … INSERT … SELECT` shape.
356+
342357
### INSERT … SELECT
343358
`INSERT [INTO] target [(cols)] SELECT …` accepts the same Selection grammar as a top-level SELECT — WHERE / JOIN / GROUP BY / aggregates / ORDER BY / TOP / OFFSET-FETCH / UNION / INTERSECT / EXCEPT all work on the source side. Probe-confirmed against SQL Server 2025 on 2026-05-10.
344359

@@ -375,6 +390,7 @@ Full `DbDataReader` contract. Typed accessors read `SqlValue` directly via the c
375390
- `UNION` / `UNION ALL` inside a subquery body.
376391
- Row-constructor `IN ((1,2), (3,4))`.
377392
- Window functions other than `ROW_NUMBER` and the aggregate-OVER family (see "Window functions" above).
393+
- Recursive CTEs (`WITH name AS (… UNION ALL … FROM name …)`); self-reference detected and raises `NotSupportedException`. Non-recursive CTEs are modeled.
378394
- `LIKE` with `COLLATE` override (default collation only — case-insensitive Latin1_General-shaped).
379395
- `CONVERT` / `TRY_CONVERT` style codes other than `0` / `120` / `121` for date-like → string.
380396
- `LEN(ntext)` raising Msg 8116 (function-level text/ntext/image restrictions); legacy `READTEXT` / `WRITETEXT` / `UPDATETEXT`.
Lines changed: 209 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,209 @@
1+
using System.Data.Common;
2+
using static Microsoft.VisualStudio.TestTools.UnitTesting.Assert;
3+
4+
namespace SqlServerSimulator;
5+
6+
/// <summary>
7+
/// Behavioral tests for the <c>WITH name [(col, …)] AS (SELECT …)</c>
8+
/// non-recursive CTE prefix that scopes to one following SELECT / INSERT
9+
/// / UPDATE / DELETE / MERGE statement. Recursive CTEs are not modeled.
10+
/// </summary>
11+
[TestClass]
12+
public sealed class CommonTableExpressionTests
13+
{
14+
private static Simulation WithSourceTable()
15+
{
16+
var simulation = new Simulation();
17+
_ = simulation.ExecuteNonQuery("""
18+
create table src (id int, score int);
19+
insert src (id, score) values (1, 10), (2, 20), (3, 30)
20+
""");
21+
return simulation;
22+
}
23+
24+
[TestMethod]
25+
public void Cte_Vanilla_ProjectsBodyRows()
26+
{
27+
var simulation = WithSourceTable();
28+
using var reader = simulation.ExecuteReader("with c as (select id, score from src) select id, score from c order by id");
29+
var rows = new List<(int id, int score)>();
30+
while (reader.Read())
31+
rows.Add((reader.GetInt32(0), reader.GetInt32(1)));
32+
CollectionAssert.AreEqual(new[] { (1, 10), (2, 20), (3, 30) }, rows);
33+
}
34+
35+
[TestMethod]
36+
public void Cte_RenameList_RenamesProjection()
37+
{
38+
var simulation = WithSourceTable();
39+
using var reader = simulation.ExecuteReader("with c (a, b) as (select id, score from src) select a, b from c order by a");
40+
IsTrue(reader.Read());
41+
AreEqual("a", reader.GetName(0));
42+
AreEqual("b", reader.GetName(1));
43+
AreEqual(1, reader.GetInt32(0));
44+
AreEqual(10, reader.GetInt32(1));
45+
}
46+
47+
[TestMethod]
48+
public void Cte_MultipleBindings_LaterReferencesEarlier()
49+
=> AreEqual(2, WithSourceTable().ExecuteScalar("""
50+
with a as (select id, score from src),
51+
b as (select id from a where score > 10)
52+
select count(*) from b
53+
"""));
54+
55+
[TestMethod]
56+
public void Cte_MultipleReferences_EachReExecutesPlan()
57+
{
58+
var simulation = WithSourceTable();
59+
using var reader = simulation.ExecuteReader("""
60+
with c as (select id from src)
61+
select x.id, y.id
62+
from c x cross join c y
63+
where x.id < y.id
64+
order by x.id, y.id
65+
""");
66+
var rows = new List<(int x, int y)>();
67+
while (reader.Read())
68+
rows.Add((reader.GetInt32(0), reader.GetInt32(1)));
69+
CollectionAssert.AreEqual(new[] { (1, 2), (1, 3), (2, 3) }, rows);
70+
}
71+
72+
[TestMethod]
73+
public void Cte_AliasOnCteReference_QualifiesColumns()
74+
=> AreEqual(20, WithSourceTable().ExecuteScalar("with c as (select id, score from src) select alias.score from c as alias where alias.id = 2"));
75+
76+
[TestMethod]
77+
public void Cte_WhereReferencesCteColumns()
78+
=> AreEqual(2, WithSourceTable().ExecuteScalar("with c as (select id, score from src) select count(*) from c where score >= 20"));
79+
80+
[TestMethod]
81+
public void Cte_StarProjectionInBody_ExpandsBeforeRename()
82+
{
83+
var simulation = WithSourceTable();
84+
using var reader = simulation.ExecuteReader("with c as (select * from src) select id, score from c order by id");
85+
IsTrue(reader.Read());
86+
AreEqual(1, reader.GetInt32(0));
87+
AreEqual(10, reader.GetInt32(1));
88+
}
89+
90+
[TestMethod]
91+
public void Cte_UnionInBody_Works()
92+
=> AreEqual(6, new Simulation().ExecuteScalar("""
93+
with c as (select 1 as v union all select 2 union all select 3)
94+
select sum(v) from c
95+
"""));
96+
97+
// CTE shadows a real table of the same name for the prefixed statement.
98+
[TestMethod]
99+
public void Cte_ShadowsRealTable()
100+
{
101+
var simulation = WithSourceTable();
102+
AreEqual(42, simulation.ExecuteScalar("with src as (select 42 as score) select score from src"));
103+
}
104+
105+
[TestMethod]
106+
public void Cte_PrefixesInsertSelect_RowsLandInDestination()
107+
{
108+
var simulation = WithSourceTable();
109+
_ = simulation.ExecuteNonQuery("create table dst (a int)");
110+
AreEqual(2, simulation.ExecuteNonQuery("with c as (select score from src where score >= 20) insert dst (a) select score from c"));
111+
AreEqual(2, simulation.ExecuteScalar("select count(*) from dst"));
112+
AreEqual(50, simulation.ExecuteScalar("select sum(a) from dst"));
113+
}
114+
115+
[TestMethod]
116+
public void Cte_PrefixesUpdate_FromJoin()
117+
{
118+
var simulation = WithSourceTable();
119+
_ = simulation.ExecuteNonQuery("create table u (id int, score int); insert u select id, score from src");
120+
AreEqual(2, simulation.ExecuteNonQuery("with c as (select id from src where score >= 20) update u set score = score * 10 from u inner join c on c.id = u.id"));
121+
AreEqual(10, simulation.ExecuteScalar("select score from u where id = 1"));
122+
AreEqual(200, simulation.ExecuteScalar("select score from u where id = 2"));
123+
AreEqual(300, simulation.ExecuteScalar("select score from u where id = 3"));
124+
}
125+
126+
[TestMethod]
127+
public void Cte_PrefixesDelete_FromJoin()
128+
{
129+
var simulation = WithSourceTable();
130+
_ = simulation.ExecuteNonQuery("create table d (id int, score int); insert d select id, score from src");
131+
AreEqual(2, simulation.ExecuteNonQuery("with c as (select id from src where score >= 20) delete d from d inner join c on c.id = d.id"));
132+
AreEqual(1, simulation.ExecuteScalar("select count(*) from d"));
133+
}
134+
135+
[TestMethod]
136+
public void Cte_DuplicateName_RaisesMsg239()
137+
=> new Simulation().AssertSqlError("with a as (select 1 as v), a as (select 2 as v) select * from a", 239,
138+
"Duplicate common table expression name 'a' was specified.");
139+
140+
[TestMethod]
141+
public void Cte_RenameTooFew_RaisesMsg8158()
142+
=> WithSourceTable().AssertSqlError("with c (x) as (select id, score from src) select * from c", 8158,
143+
"'c' has more columns than were specified in the column list.");
144+
145+
[TestMethod]
146+
public void Cte_RenameTooMany_RaisesMsg8159()
147+
=> WithSourceTable().AssertSqlError("with c (x, y, z) as (select id, score from src) select * from c", 8159,
148+
"'c' has fewer columns than were specified in the column list.");
149+
150+
[TestMethod]
151+
public void Cte_OrderByWithoutTopOrOffset_RaisesMsg1033()
152+
=> WithSourceTable().AssertSqlError("with c as (select id from src order by id) select * from c", 1033,
153+
"The ORDER BY clause is invalid in views, inline functions, derived tables, subqueries, and common table expressions, unless TOP, OFFSET or FOR XML is also specified.");
154+
155+
[TestMethod]
156+
public void Cte_OrderByWithTop_Allowed()
157+
=> AreEqual(3, WithSourceTable().ExecuteScalar("with c as (select top 1 id from src order by id desc) select id from c"));
158+
159+
[TestMethod]
160+
public void Cte_OrderByWithOffsetFetch_Allowed()
161+
=> AreEqual(2, WithSourceTable().ExecuteScalar("with c as (select id from src order by id desc offset 1 rows fetch next 1 rows only) select id from c"));
162+
163+
[TestMethod]
164+
public void Cte_RecursiveSelfReference_RaisesNotSupported()
165+
=> Throws<NotSupportedException>(() => WithSourceTable().ExecuteNonQuery("""
166+
with a as (
167+
select 1 as n
168+
union all
169+
select n + 1 from a where n < 5
170+
) select * from a
171+
"""));
172+
173+
[TestMethod]
174+
public void Cte_BindingClearsBetweenStatements()
175+
{
176+
var simulation = WithSourceTable();
177+
// First statement uses a CTE binding named 'c'; the second statement
178+
// should NOT see 'c' anymore — it must fail with a missing-table error.
179+
_ = simulation.ExecuteNonQuery("with c as (select id from src) select count(*) from c");
180+
_ = Throws<DbException>(() => simulation.ExecuteNonQuery("select count(*) from c"));
181+
}
182+
183+
[TestMethod]
184+
public void Cte_WithParameter()
185+
{
186+
var simulation = WithSourceTable();
187+
using var connection = simulation.CreateOpenConnection();
188+
var command = connection.CreateCommand("with c as (select id, score from src where id >= @minId) select count(*) from c", ("minId", 2));
189+
AreEqual(2, command.ExecuteScalar());
190+
}
191+
192+
// CTE is reachable from a scalar subquery inside the prefixed statement
193+
// (the binding lives in ParserContext.CteBindings for the whole statement).
194+
[TestMethod]
195+
public void Cte_ReachableFromScalarSubqueryInProjection()
196+
{
197+
var simulation = WithSourceTable();
198+
using var reader = simulation.ExecuteReader("""
199+
with c as (select id, score from src)
200+
select id, (select max(score) from c) as max_score
201+
from c
202+
order by id
203+
""");
204+
var rows = new List<(int id, int max)>();
205+
while (reader.Read())
206+
rows.Add((reader.GetInt32(0), reader.GetInt32(1)));
207+
CollectionAssert.AreEqual(new[] { (1, 30), (2, 30), (3, 30) }, rows);
208+
}
209+
}

SqlServerSimulator/Errors/SimulatedSqlException.QueryErrors.cs

Lines changed: 31 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -191,6 +191,37 @@ internal static SimulatedSqlException InsertSelectListFewerThanInsertList() =>
191191
internal static SimulatedSqlException InsertSelectListMoreThanInsertList() =>
192192
new("The select list for the INSERT statement contains more items than the insert list. The number of SELECT values must match the number of INSERT columns.", 121, 15, 1);
193193

194+
/// <summary>
195+
/// Mimics SQL Server error 239: a <c>WITH</c> prefix declares two or more
196+
/// CTEs with the same name in one statement.
197+
/// </summary>
198+
internal static SimulatedSqlException DuplicateCteName(string name) =>
199+
new($"Duplicate common table expression name '{name}' was specified.", 239, 16, 1);
200+
201+
/// <summary>
202+
/// Mimics SQL Server error 8158: a CTE's column-rename list has fewer
203+
/// names than the body's projection produces. Counterpart to Msg 8159.
204+
/// </summary>
205+
internal static SimulatedSqlException CteHasMoreColumnsThanList(string name) =>
206+
new($"'{name}' has more columns than were specified in the column list.", 8158, 16, 1);
207+
208+
/// <summary>
209+
/// Mimics SQL Server error 8159: a CTE's column-rename list has more
210+
/// names than the body's projection produces. Counterpart to Msg 8158.
211+
/// </summary>
212+
internal static SimulatedSqlException CteHasFewerColumnsThanList(string name) =>
213+
new($"'{name}' has fewer columns than were specified in the column list.", 8159, 16, 1);
214+
215+
/// <summary>
216+
/// Mimics SQL Server error 1033: <c>ORDER BY</c> appears inside a CTE
217+
/// body without an accompanying <c>TOP</c> / <c>OFFSET</c> / <c>FOR XML</c>.
218+
/// Real SQL Server's wording lists the broader set of contexts (views,
219+
/// inline functions, derived tables, subqueries, common table expressions);
220+
/// the simulator enforces the rule for CTE bodies only.
221+
/// </summary>
222+
internal static SimulatedSqlException OrderByInvalidInCte() =>
223+
new("The ORDER BY clause is invalid in views, inline functions, derived tables, subqueries, and common table expressions, unless TOP, OFFSET or FOR XML is also specified.", 1033, 15, 1);
224+
194225
/// <summary>
195226
/// Mimics SQL Server error 108: a positional <c>ORDER BY</c> ordinal
196227
/// (e.g. <c>order by 0</c>, <c>order by 5</c> with only 3 columns) is
Lines changed: 28 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,28 @@
1+
namespace SqlServerSimulator.Parser;
2+
3+
/// <summary>
4+
/// One named binding from a <c>WITH</c> prefix. Lifetime is exactly one
5+
/// following statement: <see cref="Simulation.CreateResultSetsForCommand"/>
6+
/// populates <see cref="ParserContext.CteBindings"/> via <c>ParseCteBindings</c>
7+
/// before dispatching to SELECT / INSERT / UPDATE / DELETE / MERGE, and
8+
/// clears the slot at the start of the next iteration.
9+
/// </summary>
10+
/// <remarks>
11+
/// <see cref="Plan"/> is null while the body is mid-parse — a sentinel so
12+
/// <c>ParseSingleFromSource</c> can detect self-references (recursive CTEs)
13+
/// and raise <see cref="NotSupportedException"/>. Once the body's
14+
/// <see cref="Selection"/> resolves, the sentinel is replaced by the real
15+
/// plan and the binding becomes resolvable from the FROM side.
16+
/// </remarks>
17+
internal sealed class CteBinding(string name, string[] columnNames)
18+
{
19+
public readonly string Name = name;
20+
public readonly string[] ColumnNames = columnNames;
21+
22+
/// <summary>
23+
/// The body's parsed plan, or <see langword="null"/> while parsing is
24+
/// in flight. <see cref="Plan"/> is the only mutable field — set once
25+
/// when the body's <c>Selection</c> resolves.
26+
/// </summary>
27+
public Selection? Plan;
28+
}

SqlServerSimulator/Parser/ParserContext.cs

Lines changed: 13 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -117,6 +117,19 @@ internal sealed class ParserContext(SimulatedDbCommand command)
117117
/// </summary>
118118
public Func<MultiPartName, SqlType>? OuterTypeResolver;
119119

120+
/// <summary>
121+
/// Common-table-expression bindings registered by a <c>WITH</c> prefix
122+
/// that scope to the immediately-following statement. Populated by
123+
/// <c>Simulation.ParseCteBindings</c> before the SELECT / INSERT /
124+
/// UPDATE / DELETE / MERGE dispatch and cleared at the top of the next
125+
/// statement iteration. Consulted by <c>Selection.ParseSingleFromSource</c>
126+
/// before falling through to <see cref="Simulation.HeapTables"/>; matching
127+
/// names build a deferred-plan <see cref="FromSource"/> (re-runs per
128+
/// reference, parallel to derived tables in FROM). Null when no WITH
129+
/// prefix is in scope.
130+
/// </summary>
131+
public Dictionary<string, CteBinding>? CteBindings;
132+
120133
private readonly FrozenDictionary<string, SqlValue> variables = command
121134
.Parameters
122135
.Cast<DbParameter>()

0 commit comments

Comments
 (0)