Skip to content

Commit 8454339

Browse files
committed
JSON_QUERY scalar function. SQL Server's complement to JSON_VALUE: object/array match → raw JSON text, scalar match → SQL NULL. Two WWI computed columns (Application.People.OtherLanguages and Warehouse.StockItems.Tags, both shaped as json_query([CustomFields], N'$.<key>')) now load — WWI computed columns complete at 8/8; two extended properties attached to those columns also unblock (SqlExtendedProperty 83 → 81).
**`JsonQuery` expression class** (`SqlServerSimulator/Parser/Expressions/JsonQuery.cs`). Structural parallel to `JsonValue` — reuses `JsonPath.Parse` / `JsonDocument.Parse` / `JsonPath.Walk` verbatim and differs only at the leaf materialization: - Object / Array → `SqlValue.FromNVarchar(element.GetRawText())` — preserves the input's whitespace shape, matching SQL Server's observable round-trip behavior. - String / Number / True / False / Null → `SqlValue.Null(SqlType.NVarchar)` (lax-mode behavior; strict mode would raise Msg 13624, which EF / DACFx never depend on). Return type is `nvarchar(MAX)` semantically; the runtime `SqlValue` carries `SqlType.NVarchar` (same convention as JSON_VALUE — the simulator's runtime value type doesn't carry the max-length distinction; only the static `GetSqlType` does, and JSON_QUERY's static type here is `SqlType.NVarchar`). Two-arg form only (the SQL Server 1-arg shorthand `JSON_QUERY(json)` ≡ `JSON_QUERY(json, '$')` raises Msg 102 at parse via `SyntaxErrorNear`) — DACFx-emitted computed columns always supply explicit paths so this is harmless. **Dispatcher entry** (`SqlServerSimulator/Parser/Expression.cs`). `"JSON_QUERY" => new JsonQuery(context)` lands in the length-10 switch arm alongside JSON_VALUE / DATALENGTH / ROW_NUMBER / etc. **Test coverage** (`SqlServerSimulator.Tests/JsonScalarTests.cs`, +7 entries): - `JsonQuery_ObjectMatch_ReturnsRawJson` — `{"obj":{"a":1,"b":2}}` / `$.obj` → `{"a":1,"b":2}`. - `JsonQuery_ArrayMatch_ReturnsRawJson` — `{"arr":[1,2,3]}` / `$.arr` → `[1,2,3]`. - `JsonQuery_ScalarMatch_ReturnsNullLax` — complement to JsonValue's object-match-returns-null. - `JsonQuery_MissingPath_ReturnsNullLax`, `JsonQuery_NullJson_ReturnsNull`, `JsonQuery_NullPath_ReturnsNull` — null/missing-input coverage matching JSON_VALUE's set. - `JsonQuery_RoundTripThroughOpenJson` — `JSON_QUERY` extracts an array, `OPENJSON` consumes it, value comes back — proves the raw-text re-emission survives a parser round-trip. `Load_WWI_Known_Gaps_Recorded_In_Skipped` updated: `IsFalse(grouped.ContainsKey("SqlComputedColumn"))` replaces the prior `AreEqual(2, …)` assertion. `Load_WWI_Computed_Columns_Land_With_is_computed_Set` bumps from 6 → 8 (full WWI coverage). The end-to-end-recompute test (`Load_WWI_Persisted_Computed_Column_Evaluates_On_Read`) is unchanged; it was already exercising one of the non-JSON_QUERY computed columns (`SearchName`). **Doc updates**: `docs/claude/json.md` title gains JSON_QUERY, a new paragraph documents the semantics (raw-text re-emission for object/array, NULL for scalar, two-arg form only, OPENJSON-pipeline-friendly), and the "Not modeled" trailer drops JSON_QUERY. `CLAUDE.md`'s BACPAC pointer refreshes the WWI census (8/8 computed columns now, SqlExtendedProperty 81, drops SqlComputedColumn from the gap catalog). `docs/claude/bacpac-prerequisites.md` gets a new step 6 covering this bundle; step 9's gap inventory updates with the new counts and strikes through SqlComputedColumn ("shipped in step 6").
1 parent 9da889e commit 8454339

7 files changed

Lines changed: 128 additions & 16 deletions

File tree

CLAUDE.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -164,7 +164,7 @@ Per-feature deep-dives live under `docs/claude/`. Each entry below is a trigger:
164164
- **Touching `XmlSqlType` (singleton in `Storage/XmlType.cs`, SystemTypeId=241, IsLob=true), `XmlSchemaCollection` storage class + `Schema.XmlSchemaCollections` dict + `Database.AllocateXmlCollectionId` (seeds at 65536 per probe), `HeapColumn.XmlSchemaCollection` per-column binding, the `xml`-recognizing arms of `SqlType.ResolveSimpleKeyword` / `Precedence` / `SystemTypeId`, the `xml(name)` / `xml(CONTENT name)` / `xml(DOCUMENT name)` column-type parsing in `ParseOneColumnIntoLists` (via `PeekIsXmlSchemaArgument` / `ParseXmlSchemaCollectionArgument`), `Simulation.Xml.cs` partial (CREATE XML SCHEMA COLLECTION / DROP XML SCHEMA COLLECTION / CREATE [PRIMARY] XML INDEX / FOR PATH|VALUE|PROPERTY secondary form), the `Xml` contextual keyword + `Keyword.Primary` dispatch in `TryParseCreate`, `XmlIndex` / `XmlSecondaryIndexType` storage on `HeapTable.XmlIndexes`, the `XmlMethodCall` expression in `Parser/Expressions/XmlMethodCall.cs` (closed accept-list of `value` / `nodes` / `query` / `exist` / `modify`, parses cleanly, throws `NotSupportedException` at `Run`), or the `sys.xml_schema_collections` / `sys.xml_indexes` catalog views** → [`docs/claude/bacpac-prerequisites.md`](docs/claude/bacpac-prerequisites.md) (the "`xml` data type + XML schema collections + XML methods + XML indexes" section carries the implementation detail).
165165
- **Touching `GeographySqlType` / `GeometrySqlType` (singletons in `Storage/SpatialType.cs`, both inherit `SpatialSqlType : SqlType(SqlTypeCategory.String)`, SystemTypeId=240, UserTypeId=130/129, IsLob=true), `SqlValue.FromGeography` / `FromGeometry` factories + the spatial branches in `FromString`, the `geography` / `geometry`-recognizing arms of `SqlType.ResolveSimpleKeyword` / `Precedence` / `SystemTypeId` / `UserTypeId`, the `SpatialMethodCall` expression in `Parser/Expressions/SpatialMethodCall.cs` (broad ~70-name accept-list of OGC + Microsoft-extension methods; parses cleanly + throws `NotSupportedException` at `Run` except `.ToString()` returns the stored WKT), the `SpatialStaticCall` expression in `Parser/Expressions/SpatialStaticCall.cs` (`geography::Parse(wkt)` / `geography::STGeomFromText(wkt, srid)` / `geometry::Point(x, y, srid)` work; other static methods throw), the `::` dispatch for `geography`/`geometry` in `Expression.cs` alongside `hierarchyid::`, the `.ToString()` polymorphic delegation in `HierarchyIdMethodCall.Run` (spatial receivers return WKT through the spatial path), the `SpatialIndex` storage class + `SpatialIndexKind` enum on `HeapTable.SpatialIndexes`, the `Simulation.Spatial.cs` partial (CREATE SPATIAL INDEX with USING / BOUNDING_BOX / GRIDS / CELLS_PER_OBJECT parsing + default tessellation_scheme), the `Spatial` contextual keyword routing in `TryParseCreate`, or the `sys.spatial_indexes` (23-col probe-confirmed) / `sys.spatial_index_tessellations` (16-col probe-confirmed) / `sys.spatial_reference_systems` (6-col, empty seed) catalog views** → [`docs/claude/bacpac-prerequisites.md`](docs/claude/bacpac-prerequisites.md) (the "`geography` / `geometry` data types" section carries the implementation detail).
166166
- **Adding a new top-level statement parser or changing the dispatch loop's statement-separator rules**[`docs/claude/grammar.md`](docs/claude/grammar.md) + [`docs/claude/control-flow.md`](docs/claude/control-flow.md).
167-
- **Working on BACPAC import (`Simulation.FromBacpac(string, out BacpacLoadResult)` + Stream overload, internal until externally consumed) — the model.xml dispatcher in `SqlServerSimulator/Storage/Bacpac/ModelXmlReader.cs` (9 phases: DB options + schemas + UDDTs + **sequences + roles + table types** → tables → constraints → FKs → indexes → views → programmable objects → **deferred computed columns via ALTER TABLE ADD AS** → extended properties → BCP data), the `BacpacReader` + `BcpRowReader` + `HierarchyIdWireDecoder` data path, the `BacpacLoadResult.Skipped` + `ElementCounts` + `Warnings` + `TableColumnIsAlias` diagnostics carriers, the resilient-loader contract (per-element exceptions land on Skipped with `"Load failed: …"` prefix and don't abort the load; computed-column failures use `"Deferred: …"` prefix so the AW guard stays meaningful), the **BCP-filters-computed-columns contract** (`BacpacReader.LoadRowsFromBcp` strips columns with `HeapColumn.Computed != null` before passing to `BcpRowReader`/`RowEncoder` — DACFx-emitted BCP files exclude computed columns from the wire layout regardless of PERSISTED; the loader emits computed columns *without* the PERSISTED qualifier so the simulator recomputes on every read), the wire-format matrix (fixed-raw / 1-byte-prefix / 2-byte-prefix / 8-byte-prefix tiers — see [`docs/claude/bacpac-prerequisites.md`](docs/claude/bacpac-prerequisites.md) "BCP wire format" section)** → [`docs/claude/bacpac-prerequisites.md`](docs/claude/bacpac-prerequisites.md). Loader baseline ships (2026-05-15) with AdventureWorks2025 at **100% row coverage** (760,167 / 760,167; zero BCP-file failures); WideWorldImporters-Standard loads end-to-end with **48/48 tables + 26/26 sequences + 9/9 roles + 4/4 table types + 6/8 computed columns + 42/42 procedures + 4.7M rows**, remaining Skipped categories cataloged in `Load_WWI_Known_Gaps_Recorded_In_Skipped` (83 SqlExtendedProperty, 3 SqlIndex, 2 SqlComputedColumn (json_query unmodeled), 2 SqlCheckConstraint, 2 SqlPermissionStatement, 1 each SqlView / SqlScalarFunction / SqlFilegroup / SqlDatabaseOptions-Collation). MAX types (varchar(MAX) / nvarchar(MAX) / varbinary(MAX)) + xml + CLR-UDT family all use the simple inline 8-byte-prefix wire shape (NOT TDS-PLP chunked form); plain `bit` uses 1-byte length prefix regardless of nullability (probe-confirmed via Production.Document.FolderFlag, the sole non-UDDT bit column in AW). Hierarchyid wire decoder covers AW's [0..79] positive-ordinal envelope via a 4-prefix order-preserving prefix code (`01` / `100` / `101` / `110`); negative ordinals + ordinals ≥ 80 + dotted sub-ordinals raise `NotSupportedException` for a follow-up bundle to extend. Geography rows load with `SpatialLocation = NULL` (WKB→WKT decoding deferred).
167+
- **Working on BACPAC import (`Simulation.FromBacpac(string, out BacpacLoadResult)` + Stream overload, internal until externally consumed) — the model.xml dispatcher in `SqlServerSimulator/Storage/Bacpac/ModelXmlReader.cs` (9 phases: DB options + schemas + UDDTs + **sequences + roles + table types** → tables → constraints → FKs → indexes → views → programmable objects → **deferred computed columns via ALTER TABLE ADD AS** → extended properties → BCP data), the `BacpacReader` + `BcpRowReader` + `HierarchyIdWireDecoder` data path, the `BacpacLoadResult.Skipped` + `ElementCounts` + `Warnings` + `TableColumnIsAlias` diagnostics carriers, the resilient-loader contract (per-element exceptions land on Skipped with `"Load failed: …"` prefix and don't abort the load; computed-column failures use `"Deferred: …"` prefix so the AW guard stays meaningful), the **BCP-filters-computed-columns contract** (`BacpacReader.LoadRowsFromBcp` strips columns with `HeapColumn.Computed != null` before passing to `BcpRowReader`/`RowEncoder` — DACFx-emitted BCP files exclude computed columns from the wire layout regardless of PERSISTED; the loader emits computed columns *without* the PERSISTED qualifier so the simulator recomputes on every read), the wire-format matrix (fixed-raw / 1-byte-prefix / 2-byte-prefix / 8-byte-prefix tiers — see [`docs/claude/bacpac-prerequisites.md`](docs/claude/bacpac-prerequisites.md) "BCP wire format" section)** → [`docs/claude/bacpac-prerequisites.md`](docs/claude/bacpac-prerequisites.md). Loader baseline ships (2026-05-15) with AdventureWorks2025 at **100% row coverage** (760,167 / 760,167; zero BCP-file failures); WideWorldImporters-Standard loads end-to-end with **48/48 tables + 26/26 sequences + 9/9 roles + 4/4 table types + 8/8 computed columns + 42/42 procedures + 4.7M rows**, remaining Skipped categories cataloged in `Load_WWI_Known_Gaps_Recorded_In_Skipped` (81 SqlExtendedProperty, 3 SqlIndex, 2 SqlCheckConstraint, 2 SqlPermissionStatement, 1 each SqlView / SqlScalarFunction / SqlFilegroup / SqlDatabaseOptions-Collation). MAX types (varchar(MAX) / nvarchar(MAX) / varbinary(MAX)) + xml + CLR-UDT family all use the simple inline 8-byte-prefix wire shape (NOT TDS-PLP chunked form); plain `bit` uses 1-byte length prefix regardless of nullability (probe-confirmed via Production.Document.FolderFlag, the sole non-UDDT bit column in AW). Hierarchyid wire decoder covers AW's [0..79] positive-ordinal envelope via a 4-prefix order-preserving prefix code (`01` / `100` / `101` / `110`); negative ordinals + ordinals ≥ 80 + dotted sub-ordinals raise `NotSupportedException` for a follow-up bundle to extend. Geography rows load with `SpatialLocation = NULL` (WKB→WKT decoding deferred).
168168

169169
## Not modeled
170170

SqlServerSimulator.Tests.Internal/Storage/BacpacLoaderTests.cs

Lines changed: 6 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -600,8 +600,8 @@ public void Load_WWI_Known_Gaps_Recorded_In_Skipped()
600600
.ToDictionary(g => g.Key, g => g.Count());
601601
IsFalse(grouped.ContainsKey("SqlTableType"), "SqlTableType is dispatched; should not appear in Skipped.");
602602
IsFalse(grouped.ContainsKey("SqlProcedure"), "All 3 previously-failing sysname-using procs now load (sysname keyword wired into SqlType.ResolveSimpleKeyword).");
603-
AreEqual(83, grouped["SqlExtendedProperty"], "Extended properties on the 2 still-deferred computed columns + on table-type columns + on filegroup.");
604-
AreEqual(2, grouped["SqlComputedColumn"], "2 WWI computed columns invoke json_query (not modeled); both routed via 'Deferred:' prefix so the AW guard stays meaningful.");
603+
IsFalse(grouped.ContainsKey("SqlComputedColumn"), "JSON_QUERY now supported; all 8 WWI computed columns load.");
604+
AreEqual(81, grouped["SqlExtendedProperty"], "Extended properties on table-type columns + filegroup + a handful of computed-col-adjacent columns the loader doesn't yet host-route.");
605605
AreEqual(3, grouped["SqlIndex"], "3 filtered indexes reference computed columns that don't yet exist at phase 5 (computed columns land in phase 8). Reordering indexes to a later phase tripped an unrelated UDF-resolution gap in ALTER TABLE ADD AS for AW; deferred.");
606606
AreEqual(2, grouped["SqlCheckConstraint"], "2 JSON check constraints deferred.");
607607
AreEqual(2, grouped["SqlPermissionStatement"], "GRANT/REVOKE wire-up deferred (same as AW).");
@@ -618,13 +618,13 @@ public void Load_WWI_Computed_Columns_Land_With_is_computed_Set()
618618
using var connection = (SimulatedDbConnection)simulation.CreateDbConnection();
619619
connection.Open();
620620
using var command = connection.CreateCommand();
621-
// 6 of WWI's 8 computed columns succeed (the 2 referencing json_query
622-
// remain deferred). Verify the 6 surface in sys.columns with
623-
// is_computed = 1.
621+
// All 8 of WWI's computed columns succeed after the 2026-05-15
622+
// JSON_QUERY bundle (was 6/8 before). Verify all 8 surface in
623+
// sys.columns with is_computed = 1.
624624
command.CommandText = "SELECT COUNT(*) FROM sys.columns WHERE is_computed = 1;";
625625
using var reader = command.ExecuteReader();
626626
IsTrue(reader.Read());
627-
AreEqual(6, reader.GetInt32(0));
627+
AreEqual(8, reader.GetInt32(0));
628628
}
629629

630630
[TestMethod]

SqlServerSimulator.Tests/JsonScalarTests.cs

Lines changed: 31 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -138,4 +138,35 @@ public void JsonModify_EfPartialUpdateShape()
138138
"{\"City\":\"Shelbyville\",\"Street\":\"1 Main\"}",
139139
simulation.ExecuteScalar("select json_modify('{\"City\":\"Springfield\",\"Street\":\"1 Main\"}', 'strict $.City', json_value('{\"\":\"Shelbyville\"}', '$.\"\"'))"));
140140
}
141+
142+
[TestMethod]
143+
public void JsonQuery_ObjectMatch_ReturnsRawJson()
144+
=> AreEqual("{\"a\":1,\"b\":2}", new Simulation().ExecuteScalar("select json_query('{\"obj\":{\"a\":1,\"b\":2}}', '$.obj')"));
145+
146+
[TestMethod]
147+
public void JsonQuery_ArrayMatch_ReturnsRawJson()
148+
=> AreEqual("[1,2,3]", new Simulation().ExecuteScalar("select json_query('{\"arr\":[1,2,3]}', '$.arr')"));
149+
150+
/// <summary>Complement of <see cref="JsonValue_ObjectMatch_ReturnsNullLax"/>: JSON_QUERY returns NULL on scalar matches in lax mode.</summary>
151+
[TestMethod]
152+
public void JsonQuery_ScalarMatch_ReturnsNullLax()
153+
=> IsInstanceOfType<DBNull>(new Simulation().ExecuteScalar("select json_query('{\"n\":42}', '$.n')"));
154+
155+
[TestMethod]
156+
public void JsonQuery_MissingPath_ReturnsNullLax()
157+
=> IsInstanceOfType<DBNull>(new Simulation().ExecuteScalar("select json_query('{\"x\":[1]}', '$.missing')"));
158+
159+
[TestMethod]
160+
public void JsonQuery_NullJson_ReturnsNull()
161+
=> IsInstanceOfType<DBNull>(new Simulation().ExecuteScalar("select json_query(null, '$.x')"));
162+
163+
[TestMethod]
164+
public void JsonQuery_NullPath_ReturnsNull()
165+
=> IsInstanceOfType<DBNull>(new Simulation().ExecuteScalar("select json_query('{\"x\":[1]}', null)"));
166+
167+
[TestMethod]
168+
public void JsonQuery_RoundTripThroughOpenJson()
169+
=> AreEqual(
170+
"Spanish",
171+
new Simulation().ExecuteScalar("select [value] from openjson(json_query('{\"OtherLanguages\":[\"Spanish\"]}', '$.OtherLanguages'))"));
141172
}

SqlServerSimulator/Parser/Expression.cs

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -614,6 +614,7 @@ private static Expression ResolveBuiltIn(string name, ParserContext context)
614614
"DENSE_RANK" => WindowExpression.ParseDenseRank(context),
615615
"ERROR_LINE" => new ErrorLineFunction(context),
616616
"GETUTCDATE" => new CurrentTimeFunction(context, CurrentTimeKind.GetUtcDate),
617+
"JSON_QUERY" => new JsonQuery(context),
617618
"JSON_VALUE" => new JsonValue(context),
618619
"LAST_VALUE" => WindowExpression.ParseLastValue(context),
619620
"ROW_NUMBER" => WindowExpression.ParseRowNumber(context),
Lines changed: 77 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,77 @@
1+
using System.Text.Json;
2+
using SqlServerSimulator.Storage;
3+
4+
namespace SqlServerSimulator.Parser.Expressions;
5+
6+
/// <summary>
7+
/// SQL <c>JSON_QUERY(json, path)</c>: extracts an object or array subtree
8+
/// from a JSON text by path. Returns <c>nvarchar(MAX)</c>; scalar matches
9+
/// and missing-path cases yield SQL NULL under the default lax mode —
10+
/// complement of <see cref="JsonValue"/> (which returns NULL for non-scalar
11+
/// matches and the scalar text otherwise).
12+
/// </summary>
13+
/// <remarks>
14+
/// The path argument is optional in real SQL Server (default <c>'$'</c> —
15+
/// returns the whole document). DACFx-emitted computed columns (WWI's
16+
/// <c>Application.People.OtherLanguages</c>, <c>Warehouse.StockItems.Tags</c>)
17+
/// supply an explicit path so this implementation requires it; the bare
18+
/// 1-arg form raises Msg 102 at parse via SyntaxErrorNear.
19+
/// </remarks>
20+
internal sealed class JsonQuery : Expression
21+
{
22+
private readonly Expression jsonInput;
23+
private readonly Expression pathInput;
24+
25+
public JsonQuery(ParserContext context)
26+
{
27+
this.jsonInput = Parse(context);
28+
if (context.Token is not Tokens.Operator { Character: ',' })
29+
throw SimulatedSqlException.SyntaxErrorNear(context);
30+
this.pathInput = Parse(context.MoveNextRequiredReturnSelf());
31+
}
32+
33+
public override SqlValue Run(RuntimeContext runtime)
34+
{
35+
var jsonValue = this.jsonInput.Run(runtime);
36+
var pathValue = this.pathInput.Run(runtime);
37+
if (jsonValue.IsNull || pathValue.IsNull)
38+
return SqlValue.Null(SqlType.NVarchar);
39+
40+
var path = JsonPath.Parse(pathValue.AsString);
41+
42+
JsonDocument doc;
43+
try
44+
{
45+
doc = JsonDocument.Parse(jsonValue.AsString);
46+
}
47+
catch (JsonException)
48+
{
49+
return path.Mode == JsonPathMode.Strict ? throw SimulatedSqlException.JsonInvalidText() : SqlValue.Null(SqlType.NVarchar);
50+
}
51+
52+
using (doc)
53+
{
54+
var match = path.Walk(doc.RootElement);
55+
if (match is null)
56+
return SqlValue.Null(SqlType.NVarchar);
57+
58+
var element = match.Value;
59+
return element.ValueKind switch
60+
{
61+
// Object / Array — JSON_QUERY returns the raw JSON text of
62+
// the subtree (re-serialized via GetRawText, which preserves
63+
// the input's whitespace shape — matching SQL Server's
64+
// observable behavior on round-trip).
65+
JsonValueKind.Object or JsonValueKind.Array => SqlValue.FromNVarchar(element.GetRawText()),
66+
// Scalar match (String / Number / True / False / Null) —
67+
// JSON_QUERY returns NULL in lax mode (Msg 13624 in strict,
68+
// which EF / DACFx never depend on).
69+
_ => SqlValue.Null(SqlType.NVarchar),
70+
};
71+
}
72+
}
73+
74+
public override SqlType GetSqlType(Func<MultiPartName, SqlType> resolveColumnType) => SqlType.NVarchar;
75+
76+
internal override string DebugDisplay() => $"JSON_QUERY({this.jsonInput.DebugDisplay()}, {this.pathInput.DebugDisplay()})";
77+
}

0 commit comments

Comments
 (0)