Skip to content

Commit 6bb4b81

Browse files
committed
Added ROW_NUMBER() OVER(PARTITION BY … ORDER BY …) window-function support, unlocking EF Core 10's top-N / Skip+Take per-group emission shape.
1 parent ccb2a60 commit 6bb4b81

9 files changed

Lines changed: 610 additions & 11 deletions

File tree

CLAUDE.md

Lines changed: 8 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -178,6 +178,13 @@ Top-level OFFSET/FETCH (post-set-op chain) attaches alongside the top-level ORDE
178178
### Aggregates
179179
`COUNT(*)` / `COUNT(expr)` / `COUNT(DISTINCT expr)` / `COUNT_BIG`, `SUM` / `AVG` (integer-truncating; `decimal(38, max(s, 6))` widening for AVG over decimals), `MAX` / `MIN`, statistical (`STDEV` / `STDEVP` / `VAR` / `VARP`), `STRING_AGG`, `CHECKSUM_AGG`, `APPROX_COUNT_DISTINCT`. Standalone and inside `GROUP BY` / `HAVING`.
180180

181+
### Window functions
182+
`ROW_NUMBER() OVER([PARTITION BY <expr-list>] ORDER BY <expr-list-with-direction>)` only — the single shape EF Core 10 emits for top-N / Skip+Take per group. `RANK` / `DENSE_RANK` / analytic family (`LAG` / `LEAD` / `FIRST_VALUE`) and frame specs (`ROWS BETWEEN`, `RANGE BETWEEN`) aren't modeled — EF Core 10 doesn't emit them from idiomatic LINQ. Result type is `bigint`. ORDER BY is required inside OVER (without it, parse fails — SQL Server raises Msg 4112).
183+
184+
`WindowExpression` (`Parser/Expressions/WindowExpression.cs`) registers itself in `ParserContext.WindowCollector` like aggregates do with `AggregateCollector`. The executor's `ProjectWindowedRows` buffers post-WHERE tuples, partitions by each window's PARTITION BY keys, sorts each partition by ORDER BY keys, assigns row numbers per partition, then walks the buffer in original order binding per-tuple results before projecting. Combining window functions with GROUP BY / HAVING / aggregates raises `NotSupportedException` (EF Core 10 doesn't emit the combination, so the simulator hasn't built it).
185+
186+
EF Core 10 always wraps ROW_NUMBER in a derived-table subquery: `SELECT ... FROM (SELECT cols, ROW_NUMBER() OVER(...) AS row FROM T) AS sub WHERE sub.row <= N` (Take) or `WHERE 1 < sub.row AND sub.row <= K` (Skip+Take). The simulator's plain-derived-table-doesn't-see-outer-scope limitation doesn't bite here because the ROW_NUMBER subquery has no outer correlation — its OVER refers only to the inner FROM.
187+
181188
### Date scalar functions: `DATEPART` / `DATEADD`
182189
Both take a bare datepart keyword as the first argument (parse-time `Name` token, not an expression). Canonical keywords + common aliases: `year`/`yy`/`yyyy`, `quarter`/`qq`/`q`, `month`/`mm`/`m`, `dayofyear`/`dy`/`y`, `day`/`dd`/`d`, `week`/`wk`/`ww`, `iso_week`/`isowk`/`isoww`, `weekday`/`dw`, `hour`/`hh`, `minute`/`mi`/`n`, `second`/`ss`/`s`, `millisecond`/`ms`, `microsecond`/`mcs`, `nanosecond`/`ns`, `tzoffset`/`tz`. `DATEPART` always returns `int`; `DATEADD` preserves the input's SQL type (`date` stays `date`, `time(N)` stays `time(N)`, etc.).
183190

@@ -246,7 +253,7 @@ Type-metadata accessors (`GetDataTypeName` / `GetFieldType`) read from `Simulate
246253
- `ANY` / `SOME` / `ALL` quantifiers.
247254
- `UNION` / `UNION ALL` inside a subquery body.
248255
- Row-constructor `IN ((1,2), (3,4))`.
249-
- Window-function aggregate form (`OVER (...)`).
256+
- Window functions other than `ROW_NUMBER`: `RANK` / `DENSE_RANK`, analytic (`LAG` / `LEAD` / `FIRST_VALUE` / `LAST_VALUE`), aggregate-OVER form (`SUM(x) OVER(...)` / `COUNT(*) OVER(...)`), frame specs (`ROWS BETWEEN` / `RANGE BETWEEN`). EF Core 10 doesn't emit any of these from idiomatic LINQ.
250257
- `STRING_AGG`'s `WITHIN GROUP (ORDER BY ...)`.
251258
- `LIKE` with `COLLATE` override (default collation only — case-insensitive Latin1_General-shaped).
252259
- `CONVERT` / `TRY_CONVERT` style codes other than `0` / `120` / `121` for date-like → string. Other styles raise Msg 281; money / float / binary style codes and `CONVERT(date, str, 103)`-style date parsing not modeled.
Lines changed: 84 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,84 @@
1+
namespace SqlServerSimulator;
2+
3+
/// <summary>
4+
/// End-to-end tests for the LINQ shapes EF Core 10 translates to
5+
/// <c>ROW_NUMBER() OVER(...)</c>: <c>SelectMany</c> + <c>OrderBy</c> +
6+
/// <c>Take</c> over a collection navigation, the same with <c>Skip</c>,
7+
/// and the per-group "latest record" pattern that emits ROW_NUMBER + 1.
8+
/// EF Core 10 wraps every emission in a derived-table subquery filtered
9+
/// by <c>WHERE row &lt;= N</c> (Take) or <c>WHERE 1 &lt; row AND row &lt;= K</c>
10+
/// (Skip+Take); see CLAUDE.md for the full shape.
11+
/// </summary>
12+
[TestClass]
13+
public class EFCoreWindow
14+
{
15+
public TestContext TestContext { get; set; } = null!;
16+
17+
private static TestDbContext SeededContext()
18+
{
19+
var context = new TestDbContext(TestDbContext.CreateAuthorsSimulation());
20+
context.Authors.AddRange(
21+
new Author { Name = "alice" },
22+
new Author { Name = "bob" });
23+
_ = context.SaveChanges();
24+
// alice=1: 4 books with scores 10/20/30/15. bob=2: 2 books with scores 5/40.
25+
context.Books.AddRange(
26+
new Book { AuthorId = 1, Title = "B1", Score = 10 },
27+
new Book { AuthorId = 1, Title = "B2", Score = 20 },
28+
new Book { AuthorId = 1, Title = "B3", Score = 30 },
29+
new Book { AuthorId = 1, Title = "B4", Score = 15 },
30+
new Book { AuthorId = 2, Title = "B5", Score = 5 },
31+
new Book { AuthorId = 2, Title = "B6", Score = 40 });
32+
_ = context.SaveChanges();
33+
return context;
34+
}
35+
36+
[TestMethod]
37+
public void Take_PerGroup_EmitsRowNumberFilter()
38+
{
39+
// EF Core 10 emits ROW_NUMBER() OVER(PARTITION BY AuthorId ORDER BY Score DESC)
40+
// wrapped in a derived table, then WHERE row <= 2.
41+
using var context = SeededContext();
42+
var titles = context.Authors
43+
.SelectMany(a => a.Books.OrderByDescending(b => b.Score).Take(2),
44+
(a, b) => new { Author = a.Name, b.Title, b.Score })
45+
.OrderBy(x => x.Author).ThenByDescending(x => x.Score)
46+
.Select(x => x.Title)
47+
.ToArray();
48+
// alice: top-2 by score desc → B3 (30), B2 (20). bob: B6 (40), B5 (5).
49+
CollectionAssert.AreEqual(new[] { "B3", "B2", "B6", "B5" }, titles);
50+
}
51+
52+
[TestMethod]
53+
public void SkipTake_PerGroup_EmitsRowNumberRange()
54+
{
55+
// Skip(1).Take(2) per group → WHERE 1 < row AND row <= 3 in the wrapped query.
56+
using var context = SeededContext();
57+
var titles = context.Authors
58+
.SelectMany(a => a.Books.OrderBy(b => b.Score).Skip(1).Take(2),
59+
(a, b) => new { Author = a.Name, b.Title, b.Score })
60+
.OrderBy(x => x.Author).ThenBy(x => x.Score)
61+
.Select(x => x.Title)
62+
.ToArray();
63+
// alice: ranks 2-3 by score asc (10, 15, 20, 30) → B4 (15), B2 (20).
64+
// bob: ranks 2-3 by score asc (5, 40) → only rank 2: B6 (40).
65+
CollectionAssert.AreEqual(new[] { "B4", "B2", "B6" }, titles);
66+
}
67+
68+
[TestMethod]
69+
public void LatestPerGroup_EmitsRowNumberLimitOne()
70+
{
71+
// Probe-confirmed shape for "latest record per group" via FirstOrDefault projection
72+
// on a navigation collection — translates to ROW_NUMBER + LEFT JOIN with row <= 1.
73+
using var context = SeededContext();
74+
var pairs = context.Authors
75+
.Select(a => new { a.Name, Latest = a.Books.OrderByDescending(b => b.Score).FirstOrDefault() })
76+
.OrderBy(x => x.Name)
77+
.ToArray();
78+
Assert.HasCount(2, pairs);
79+
Assert.AreEqual("alice", pairs[0].Name);
80+
Assert.AreEqual("B3", pairs[0].Latest!.Title);
81+
Assert.AreEqual("bob", pairs[1].Name);
82+
Assert.AreEqual("B6", pairs[1].Latest!.Title);
83+
}
84+
}
Lines changed: 161 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,161 @@
1+
using System.Data.Common;
2+
using static Microsoft.VisualStudio.TestTools.UnitTesting.Assert;
3+
4+
namespace SqlServerSimulator;
5+
6+
/// <summary>
7+
/// Behavioral tests for window functions — currently only
8+
/// <c>ROW_NUMBER() OVER(PARTITION BY ... ORDER BY ...)</c>, the single
9+
/// shape EF Core 10 emits for <c>Take</c>-per-group / <c>Skip+Take</c>-per-group
10+
/// patterns. The function's value depends on the entire row stream
11+
/// (sorted within partition), so the executor buffers post-WHERE tuples
12+
/// before binding per-row results.
13+
/// </summary>
14+
[TestClass]
15+
public sealed class WindowFunctionTests
16+
{
17+
private static DbConnection SeededPosts()
18+
{
19+
var connection = new Simulation().CreateOpenConnection();
20+
_ = connection.CreateCommand("create table posts (id int, blog_id int, score int)").ExecuteNonQuery();
21+
_ = connection.CreateCommand(
22+
"insert into posts values (1, 1, 10), (2, 1, 30), (3, 1, 20), (4, 2, 5), (5, 2, 50), (6, 2, 5)").ExecuteNonQuery();
23+
return connection;
24+
}
25+
26+
private static List<(int Id, long Row)> ReadIdRow(DbCommand command)
27+
{
28+
using var reader = command.ExecuteReader();
29+
var rows = new List<(int, long)>();
30+
while (reader.Read())
31+
rows.Add((reader.GetInt32(0), reader.GetInt64(1)));
32+
return rows;
33+
}
34+
35+
[TestMethod]
36+
public void RowNumber_PartitionBy_AssignsPerPartitionSequence()
37+
{
38+
// Partition by blog_id, order by score asc. Each partition gets 1, 2, 3...
39+
using var connection = SeededPosts();
40+
var rows = ReadIdRow(connection.CreateCommand(
41+
"select id, row_number() over(partition by blog_id order by score) as rn from posts"));
42+
// blog_id=1: id=1 (score=10) → 1, id=3 (score=20) → 2, id=2 (score=30) → 3
43+
// blog_id=2: id=4 or id=6 (score=5, tie) → 1 / 2 in some order, id=5 (score=50) → 3
44+
var byId = rows.ToDictionary(r => r.Id, r => r.Row);
45+
AreEqual(1L, byId[1]);
46+
AreEqual(3L, byId[2]);
47+
AreEqual(2L, byId[3]);
48+
AreEqual(3L, byId[5]);
49+
// The tied rows get 1 and 2 (in some stable order); both must appear.
50+
IsTrue((byId[4] == 1 && byId[6] == 2) || (byId[4] == 2 && byId[6] == 1));
51+
}
52+
53+
[TestMethod]
54+
public void RowNumber_OrderByDesc_AssignsLargestFirst()
55+
{
56+
using var connection = SeededPosts();
57+
var rows = ReadIdRow(connection.CreateCommand(
58+
"select id, row_number() over(partition by blog_id order by score desc) as rn from posts"));
59+
var byId = rows.ToDictionary(r => r.Id, r => r.Row);
60+
// blog_id=1: id=2 (30) → 1, id=3 (20) → 2, id=1 (10) → 3
61+
AreEqual(1L, byId[2]);
62+
AreEqual(2L, byId[3]);
63+
AreEqual(3L, byId[1]);
64+
// blog_id=2: id=5 (50) → 1, ties for 2/3 between id=4 and id=6
65+
AreEqual(1L, byId[5]);
66+
}
67+
68+
[TestMethod]
69+
public void RowNumber_NoPartitionBy_AssignsGlobalSequence()
70+
{
71+
using var connection = SeededPosts();
72+
var rows = ReadIdRow(connection.CreateCommand(
73+
"select id, row_number() over(order by score) as rn from posts"));
74+
// 6 rows total; row numbers 1..6 (ties get adjacent numbers).
75+
var rowNumbers = rows.Select(r => r.Row).OrderBy(n => n).ToArray();
76+
CollectionAssert.AreEqual(new long[] { 1, 2, 3, 4, 5, 6 }, rowNumbers);
77+
}
78+
79+
[TestMethod]
80+
public void RowNumber_WrappedInDerivedTable_FilterByRow()
81+
{
82+
// The exact shape EF Core 10 emits for SelectMany.OrderBy.Take(2):
83+
// wrap ROW_NUMBER in a derived table, outer WHERE filters by the row column.
84+
using var connection = SeededPosts();
85+
var ids = new List<int>();
86+
using var reader = connection.CreateCommand(
87+
"select id from (select id, row_number() over(partition by blog_id order by score desc) as rn from posts) as p where rn <= 2 order by id").ExecuteReader();
88+
while (reader.Read()) ids.Add(reader.GetInt32(0));
89+
// blog_id=1 top-2 by score desc: id=2 (30), id=3 (20). blog_id=2: id=5 (50), then one of {id=4, id=6}.
90+
HasCount(4, ids);
91+
Contains(2, ids);
92+
Contains(3, ids);
93+
Contains(5, ids);
94+
IsTrue(ids.Contains(4) || ids.Contains(6));
95+
}
96+
97+
[TestMethod]
98+
public void RowNumber_WrappedInDerivedTable_SkipTakeRange()
99+
{
100+
// Skip+Take per group: WHERE 1 < rn AND rn <= 3 → ranks 2 and 3.
101+
using var connection = SeededPosts();
102+
var ids = new List<int>();
103+
using var reader = connection.CreateCommand(
104+
"select id from (select id, row_number() over(partition by blog_id order by score) as rn from posts) as p where 1 < rn and rn <= 3 order by id").ExecuteReader();
105+
while (reader.Read()) ids.Add(reader.GetInt32(0));
106+
// blog_id=1 ranks 2-3 (score asc): id=3 (20), id=2 (30).
107+
// blog_id=2 ranks 2-3 (score asc, ties on score=5): one of {4,6} at rank 2, id=5 (50) at rank 3.
108+
HasCount(4, ids);
109+
Contains(2, ids);
110+
Contains(3, ids);
111+
Contains(5, ids);
112+
}
113+
114+
[TestMethod]
115+
public void RowNumber_PartitionByMultipleColumns()
116+
{
117+
using var connection = SeededPosts();
118+
var rows = ReadIdRow(connection.CreateCommand(
119+
"select id, row_number() over(partition by blog_id, score order by id) as rn from posts"));
120+
// Tied (blog_id=2, score=5) rows partition by (2, 5): id=4 and id=6 each get their own sequence.
121+
// Order by id: id=4 → 1, id=6 → 2.
122+
var byId = rows.ToDictionary(r => r.Id, r => r.Row);
123+
AreEqual(1L, byId[4]);
124+
AreEqual(2L, byId[6]);
125+
// Other partitions are singleton: each gets row 1.
126+
AreEqual(1L, byId[1]);
127+
AreEqual(1L, byId[2]);
128+
AreEqual(1L, byId[3]);
129+
AreEqual(1L, byId[5]);
130+
}
131+
132+
// === Parser rejections ===
133+
134+
[TestMethod]
135+
public void RowNumber_RequiresOver()
136+
{
137+
using var connection = SeededPosts();
138+
_ = Throws<DbException>(() =>
139+
_ = connection.CreateCommand("select row_number() from posts").ExecuteScalar());
140+
}
141+
142+
[TestMethod]
143+
public void RowNumber_RequiresOrderByInsideOver()
144+
{
145+
// ROW_NUMBER without ORDER BY in OVER is invalid SQL.
146+
using var connection = SeededPosts();
147+
_ = Throws<DbException>(() =>
148+
_ = connection.CreateCommand("select row_number() over() from posts").ExecuteScalar());
149+
}
150+
151+
[TestMethod]
152+
public void RowNumber_CombinedWithGroupBy_NotSupported()
153+
{
154+
// SQL Server allows this in real life, but EF Core 10 doesn't emit
155+
// the combination, so the simulator hasn't built it.
156+
using var connection = SeededPosts();
157+
_ = Throws<NotSupportedException>(() =>
158+
_ = connection.CreateCommand(
159+
"select blog_id, row_number() over(order by blog_id), count(*) from posts group by blog_id").ExecuteScalar());
160+
}
161+
}

SqlServerSimulator/Parser/ContextualKeyword.cs

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -30,6 +30,7 @@ enum ContextualKeyword
3030
Offset,
3131
Only,
3232
Output,
33+
Partition,
3334
Persisted,
3435
Row,
3536
Rows,

SqlServerSimulator/Parser/Expression.cs

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -258,6 +258,7 @@ private static Expression ResolveBuiltIn(string name, ParserContext context)
258258
10 => uppercaseName switch
259259
{
260260
"DATALENGTH" => new DataLength(context),
261+
"ROW_NUMBER" => WindowExpression.ParseRowNumber(context),
261262
"STRING_AGG" => AggregateExpression.Parse(context, AggregateKind.StringAgg),
262263
_ => null
263264
},

0 commit comments

Comments
 (0)