Skip to content

Commit 61be39c

Browse files
committed
The TDS endpoint now accepts TDS 8.0 Encrypt=Strict connections. First-wire-byte routing onto an unpinned TLS-first SslStream negotiating ALPN tds/8.0 with prelogin/LOGIN7/MARS/SqlBulkCopy all inside the channel and LOGINACK echoing version 0x08000000, exposing SimulatedNetworkListener.ServerCertificate because SqlClient ignores TrustServerCertificate in strict mode and must pin via its ServerCertificate keyword. One-side-variant comparisons now convert the base-typed operand up to sql_variant, replacing the unwrap-and-promote residual.
1 parent 0fe030a commit 61be39c

12 files changed

Lines changed: 332 additions & 56 deletions

File tree

CLAUDE.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -270,7 +270,7 @@ Each entry below is a trigger: read the linked file on demand when working in th
270270
- **New top-level statement parser or dispatch-loop separator rules, double-quoted identifiers / QUOTED_IDENTIFIER**[`grammar.md`](docs/claude/grammar.md) + [`control-flow.md`](docs/claude/control-flow.md).
271271
- **BACPAC import** (`Simulation.ImportBacpac` — multi-database via repeated calls, `BacpacImportOptions`, `ModelXmlReader` dispatcher, BCP wire format, `BacpacBuilder` test harness) → [`bacpac-loader.md`](docs/claude/bacpac-loader.md).
272272
- **Linked servers** (`Simulation.AddRemoteSimulation`, `sp_addlinkedserver` / `sp_dropserver`, four-part FROM routing through the remote's ADO.NET pipeline, `OPENQUERY(server,'query')` ad-hoc pass-through + compile-time schema discovery, `sys.servers`) → [`linked-servers.md`](docs/claude/linked-servers.md).
273-
- **TDS network endpoint** (`Simulation.ListenLocalAsync` (loopback) / `Simulation.ListenNetworkAsync` (all interfaces, requires a registered login) → `SimulatedNetworkListener`; real SqlClient over loopback TCP+TLS; SQLBatch/RPC/TM/BulkLoad (`SqlBulkCopy`); legacy `text`/`ntext`/`image` result columns + `image` RPC input params in the in-band textptr wire form; TVP (`0xF3` `SqlDbType.Structured`) RPC params via the shared `TdsColumnDecoder`; `sp_cursor*` API-server-cursor RPC family; mid-stream attention (cancel / `CommandTimeout`) via a carry-forward concurrent read → `DONE_ATTN` ack + probed XACT_ABORT tx semantics; MARS (`MultipleActiveResultSets=True`) via an SMP demux/mux over one shared connection with serialized cooperative multiplexing + per-session attention; credential enforcement via the `CREATE LOGIN` registry, Msg 18456 on mismatch; EF via plain `UseSqlServer`; oracles = `*.Tests.SqlClient` + `*.Tests.Smo`, the real-SMO consumer oracle) → [`tds-endpoint.md`](docs/claude/tds-endpoint.md).
273+
- **TDS network endpoint** (`Simulation.ListenLocalAsync` (loopback) / `Simulation.ListenNetworkAsync` (all interfaces, requires a registered login) → `SimulatedNetworkListener`; real SqlClient over loopback TCP+TLS; SQLBatch/RPC/TM/BulkLoad (`SqlBulkCopy`); legacy `text`/`ntext`/`image` result columns + `image` RPC input params in the in-band textptr wire form; TVP (`0xF3` `SqlDbType.Structured`) RPC params via the shared `TdsColumnDecoder`; `sp_cursor*` API-server-cursor RPC family; mid-stream attention (cancel / `CommandTimeout`) via a carry-forward concurrent read → `DONE_ATTN` ack + probed XACT_ABORT tx semantics; MARS (`MultipleActiveResultSets=True`) via an SMP demux/mux over one shared connection with serialized cooperative multiplexing + per-session attention; TDS 8.0 `Encrypt=Strict` (TLS-first, ALPN `tds/8.0`, client pins the exposed `SimulatedNetworkListener.ServerCertificate` since SqlClient ignores `TrustServerCertificate` in strict mode); credential enforcement via the `CREATE LOGIN` registry, Msg 18456 on mismatch; EF via plain `UseSqlServer`; oracles = `*.Tests.SqlClient` + `*.Tests.Smo`, the real-SMO consumer oracle) → [`tds-endpoint.md`](docs/claude/tds-endpoint.md).
274274

275275
## Not modeled
276276

docs/claude/backlog.md

Lines changed: 4 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -37,7 +37,7 @@ Remaining phases, roughly in value order:
3737
Candidate follow-on legs within tool scope: Visual Studio's SQL Server Object Explorer (DacFx-driven, a different query dialect from SMO) and LINQPad.
3838
- **SMO API sweep campaign**`.vs/smo-sweep` (gitignored local harness) walks SMO's full reachable read surface against the self-hosted simulator and, identically, against the live reference, draining every `Property.Value` and `Script()`-ing every `IScriptable`; modes `sweep` / `sweep --live` / `diff` → sorted JSON reports + `reports/triage.md`; workflow = sweep both sides → triage → fix bundles → graduated `Tests.Smo` tests → re-sweep.
3939
Open items from the latest triage: (a) `DBCC SHOW_STATISTICS … WITH STATS_STREAM` (SMO `Statistic.Stream`) stays `NotSupportedException` — it wants the raw serialized statistics-histogram blob, which the simulator has no faithful source for; (b) the unmodeled runtime/OS surfaces SMO reaches as absent objects (backup history `msdb.dbo.backupset`, `sys.dm_tran_persistent_version_store_stats`, file-space/IO DMVs, `sys.dm_os_process_memory`, `master.dbo.sysprocesses`, `FILEPROPERTY`, registry/OS xps) — surfaced as `PropertyCannotBeRetrievedException` / defaults, the legitimate-gap category.
40-
- **Smaller fidelity items**: TDS 8.0 / `Encrypt=Strict` (ALPN via `SslApplicationProtocol`); user-supplied `X509Certificate2` and per-interface bind-address selection — add-on-demand public-surface expansions.
40+
- **Smaller fidelity items**: user-supplied `X509Certificate2` (a CA-trusted certificate instead of the ephemeral self-signed one; strict-mode clients can already pin the exposed `SimulatedNetworkListener.ServerCertificate`) and per-interface bind-address selection — add-on-demand public-surface expansions.
4141
Open residuals of shipped wire features (details in [`tds-endpoint.md`](tds-endpoint.md)): cancel/attention reaction is bounded by the in-flight statement's materialization, not interruptible inside a single statement's row loop; MARS never raises Msg 8628/8651 and fully materializes each session's response under the execution gate.
4242
- **Chunked `OFFSET/FETCH` paging: per-page constant factor, complexity class matches real** (probed 2026-07-18, plans + timings): real SQL Server also redoes the work on every page — `Top(OFFSET…)` over an ordered index scan reading offset+fetch rows when an index supplies the order, or a full scan + `Sort(TOP offset+fetch)` re-sorted per page when not; no cross-query sorted-result caching exists on either side, so "sort once, serve many pages" is rejected (it would invent behavior real doesn't have).
4343
The residual is constant-factor only: the simulator materializes (and for non-index order, sorts) all n rows per page regardless of offset where real's indexed plan touches only offset+fetch (measured at 150k rows / fetch 100 / offset 140k: sim ~41 ms vs real ~13 ms indexed, ~116 ms vs ~22 ms unindexed — real's sort is also 16-way parallel).
@@ -68,7 +68,9 @@ Low priority / niche — simulatable (as placeholder constants or a small model)
6868
A named-filegroup registry now ships (`Database.Filegroups`, seeded PRIMARY=1, extended by the bacpac loader's `SqlFilegroup` dispatch; surfaced by `sys.filegroups`/`sys.data_spaces` and FILEGROUP-scoped extended properties) — the scalar helpers above could read it, though a `file_id` 1 placeholder is still needed for the file-level ones.
6969
- **Certificates** — CERTENCODED / CERTPRIVATEKEY (needs a small certificate-name → bytes model, or NULL placeholders).
7070
- **Full-text properties** — FULLTEXTCATALOGPROPERTY (the `CREATE FULLTEXT CATALOG` / `INDEX` DDL already ships; this reads its metadata).
71-
- **`sql_variant` minor quirks** (cross-type family ordering shipped 2026-07-19 — see [`scalars.md`](scalars.md#sql_variant-expression-semantics)): a decimal-declared inner reports BaseType `numeric` rather than real's `decimal` (the single-decimal-family naming divergence); a mixed variant-vs-base-typed comparison unwraps and promotes where real converts the base side up to variant and applies the family rules (a string variant never matches a numeric literal on real — unprobed, retained).
71+
- **`sql_variant` minor quirk** (cross-type family ordering and one-side-variant comparison both shipped 2026-07-19 — see [`scalars.md`](scalars.md#sql_variant-expression-semantics)): a decimal-declared inner reports BaseType `numeric` rather than real's `decimal`.
72+
Probed 2026-07-19: real preserves the declared keyword *distinctly*`decimal` and `numeric` never collapse — through literals, table columns, variant columns, and variant variables assigned from typed variables.
73+
The faithful fix splits the per-`(p, s)` `DecimalSqlType` singleton by declared keyword, forking the reference-identity space the row encoder, promote paths, and catalog surfaces key on — a medium refactor whose blast radius far exceeds the one metadata string it corrects, so it's deliberately deferred.
7274
Deliberate exclusion, don't re-pitch: `msdb.dbo.syspolicy_configuration.current_value` stays `nvarchar` — it's a *view-body* projection (not a resource column) mixing `int` rows with a `binary` GUID row, every consumer reads a single named row and CASTs it, so a variant migration there would only touch the view SQL text for no observable gain.
7375
- **FILESTREAM** — GET_FILESTREAM_TRANSACTION_CONTEXT (needs a FILESTREAM storage binding; NULL is a faithful "no FILESTREAM context" placeholder).
7476

docs/claude/scalars.md

Lines changed: 6 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -421,7 +421,7 @@ These carry real per-session state on `SimulatedDbConnection` (not placeholder c
421421
A missing key reads as NULL; a NULL key argument to `SESSION_CONTEXT` raises **Msg 8116** (`session_context` lowercase in the wording).
422422
`sp_set_session_context` with a NULL `@key` raises **Msg 225**; re-setting a key previously stored with `@read_only = 1` raises **Msg 15664**.
423423
Like real SQL Server, `SESSION_CONTEXT` returns **`sql_variant`** preserving the stored value's base type — an `int` stored round-trips as `int`, an `nvarchar` as `nvarchar`.
424-
The common `WHERE int_col = SESSION_CONTEXT(N'key')` shape works by the comparison path unwrapping the variant to its inner value.
424+
The common `WHERE int_col = SESSION_CONTEXT(N'key')` shape works by the comparison path converting the column side up to `sql_variant` and matching within the exact-numeric family (the family rules below).
425425
- **`SESSIONPROPERTY(name)`** (`Parser/Expressions/SessionProperty.cs`) — the current session setting for one of the ANSI / arithmetic SET options: `ANSI_NULLS`, `ANSI_PADDING`, `ANSI_WARNINGS`, `ARITHABORT`, `CONCAT_NULL_YIELDS_NULL`, `NUMERIC_ROUNDABORT`, `QUOTED_IDENTIFIER`.
426426
DacFx's bacpac-export preamble reads `ISNULL(SESSIONPROPERTY('ANSI_NULLS'), 0)` / `ISNULL(SESSIONPROPERTY('QUOTED_IDENTIFIER'), 1)`.
427427
Like real SQL Server the result is **`sql_variant`** with an inner base type of `int` (each option reads back 1 / 0).
@@ -507,9 +507,9 @@ Probe-confirmed against SQL Server 2025 (2026-07-16):
507507
- **ExecuteScalar / GetValue** surface the inner CLR object (a bare `int` for `SERVERPROPERTY('EngineEdition')`, `string` for `Edition`), so most existing value assertions are unchanged; `GetDataTypeName` reports `sql_variant` and `GetFieldType` is `object`.
508508
- **`ISNULL` / `COALESCE` / `CASE`** keep the `sql_variant` result type (variant has highest data-type precedence in `SqlType.Promote`) and preserve each value's inner type — `ISNULL(SESSIONPROPERTY('ANSI_NULLS'), 0)` stays `sql_variant`.
509509
- **`UNION [ALL]`** keeps each row's own inner base type — no schema unification/promotion (a variant column can be `int` on one row, `nvarchar` on the next).
510-
- **Comparison** against a non-variant (`WHERE int_col = SESSION_CONTEXT(N'k')`) unwraps the variant to its inner value.
511-
**Both-operands-variant comparison** follows the family rules below (probe-confirmed: variant `bigint 1000 < float 0.5`, variant `int 5 = bigint 5`, variant `nvarchar N'x' = varchar 'x'`).
512-
Residual: real converts the base-typed side of a mixed pair *up* to `sql_variant` (highest precedence) and applies the family rules — so a string variant never matches a numeric literal on real, while the unwrap path converts and can match; unprobed, retained.
510+
- **Comparison** with a `sql_variant` on *either* side follows the family rules below — the base-typed side of a mixed pair converts *up* to `sql_variant` (probe-confirmed as `CONVERT_IMPLICIT(sql_variant, …)` in real's plan; the variant never unwraps into ordinary type-precedence promotion).
511+
So a string variant is less than any exact-numeric value and never equal to it (`variant nvarchar N'5' < 5`, never `=`), cross-family comparison stays value-blind even when one side is a plain literal or column, and **no comparison error is possible**`variant nvarchar N'abc'` vs `int 5` is cleanly `<`, never Msg 245 (all probe-confirmed, including WHERE and JOIN forms over mixed-inner-type variant columns).
512+
A bare string literal against a `datetime` variant promotes to a *character*-family variant (not to `datetime`), so `variant datetime = '2020-01-01'` is false while `= CAST('2020-01-01' AS datetime)` is true.
513513
- **`SELECT … INTO`** from a variant-producing built-in creates a `sql_variant` column (probe-confirmed).
514514
- **Arithmetic** rejects: `variant + non-variant`**Msg 257** (`Implicit conversion from data type sql_variant to <target> is not allowed. Use the CONVERT function to run this query.`); `variant + variant` and `string + variant`**Msg 402** (`… incompatible in the add operator`).
515515
`PromoteForArithmetic` is the single source; a runtime guard in `IntegerArithmetic` routes through it so `Run`-time and projection-schema errors agree.
@@ -519,7 +519,8 @@ Probe-confirmed against SQL Server 2025 (2026-07-16):
519519
NULL sorts lowest.
520520
`MIN`/`MAX` pick the family-hierarchy extremes.
521521
Same-collation character pairs compare under that collation; cross-collation pairs compare by code point **without** a Msg 468 conflict (probed) — the character family hashes by rank alone since no single hash agrees with both regimes.
522-
The `SqlValue` variant arms (`CompareTo` / `Equals` / `GetHashCode`) implement the rules, so ORDER BY, GROUP BY, DISTINCT, MIN/MAX, and hash joins all inherit them via `SqlValueKey`.
522+
The `SqlValue` variant arms (`CompareTo` / `Equals` / `GetHashCode`) implement the rules, so ORDER BY, GROUP BY, DISTINCT, MIN/MAX, and hash joins all inherit them via `SqlValueKey`; a variant-vs-base equi-join key promotes to `sql_variant` (`Promote``CoerceTo` wraps the base side), so the hash fast path keys by the same family semantics.
523+
Oracle: `SqlVariantOrderingTests` (in-process; the one-side-variant comparison tests live there too).
523524

524525
## Built-in TVF: `STRING_SPLIT`
525526
`STRING_SPLIT(input, separator [, enable_ordinal])` dispatches in `ParseSingleFromSource` alongside `OPENJSON` — case-insensitive name match before generic name resolution.

docs/claude/tds-endpoint.md

Lines changed: 11 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -8,9 +8,15 @@ The listener binds IPv4 loopback plus best-effort IPv6 loopback on the same port
88
Oracle: `NetworkListenerTests`.
99

1010
Connection-string requirements: `TrustServerCertificate=true` (the endpoint presents an ephemeral in-memory self-signed cert generated per listener) and credentials per the enforcement rule below (any credentials when no logins exist).
11-
`Encrypt=Strict` (TDS 8.0) is not supported; default/`Mandatory`/`Optional` all negotiate to full encryption via `ENCRYPT_REQ`.
11+
Default/`Mandatory`/`Optional` all negotiate to full encryption via `ENCRYPT_REQ`.
1212
A client that cannot do TLS at all (`ENCRYPT_NOT_SUP`) is disconnected after the prelogin response — there is no plaintext mode.
1313

14+
**`Encrypt=Strict` (TDS 8.0) ships**: the client opens with a bare TLS ClientHello negotiating ALPN `tds/8.0`, and every TDS packet — prelogin included — flows inside the TLS channel; the session routes on the first wire byte (TLS handshake record `0x16` vs cleartext PRELOGIN `0x12`).
15+
SqlClient **ignores `TrustServerCertificate` in strict mode** and always validates the certificate (chain + hostname), so a strict client must pin instead: export `SimulatedNetworkListener.ServerCertificate` (the presented certificate, public part only) to a file and reference it with the connection string's `ServerCertificate` keyword (empirically confirmed against SqlClient 7.0.2 — `TrustServerCertificate=True` alone fails strict with `UntrustedRoot` + hostname mismatch).
16+
The strict TLS handshake is not version-pinned (TLS 1.3 negotiates; raw records make NewSessionTicket harmless), and LOGINACK echoes TDS version `0x08000000` when LOGIN7 requested it.
17+
MARS and `SqlBulkCopy` ride the strict channel unchanged.
18+
Oracle: `StrictEncryptionTests`.
19+
1420
## Credential enforcement
1521

1622
The knob is T-SQL, not API: `CREATE LOGIN name WITH PASSWORD = '…'` (run through any in-process connection — those never authenticate) populates the server-scope registry `Simulation.Logins`, and the endpoint enforces it.
@@ -30,14 +36,15 @@ Login DDL details in [`permissions.md`](permissions.md).
3036
## Architecture
3137

3238
Everything lives in `Network/` (internal) except the public `SimulatedNetworkListener` (root) and `Simulation.Listen.cs` (the `ListenLocalAsync` / `ListenNetworkAsync` partial).
33-
One task per accepted socket runs `TdsSession.RunAsync`: prelogin → TLS handshake → LOGIN7 → batch loop.
39+
One task per accepted socket runs `TdsSession.RunAsync`: prelogin → TLS handshake → LOGIN7 → batch loop (TDS 7.x), or TLS handshake → prelogin → LOGIN7 → batch loop (TDS 8.0 strict, routed by the peeked first byte).
3440
The session maps 1:1 onto a `SimulatedDbConnection`; execution flows through `Simulation.CreateResultSetsForCommand`, which yields both result sets and per-statement `RecordsAffected` — the TDS layer is a pure translator and touches no engine code.
3541

3642
- `TdsPacketTransport` — packet framing both directions: reassembles inbound packet sequences into `TdsMessage` (EOM-terminated), stamps outbound headers (type 0x04, SPID truncated to 16 bits, incrementing packet id).
3743
The stream it rides is swapped from the raw `NetworkStream` to the `SslStream` after the handshake.
3844
- `TlsHandshakeFramingStream` — the TDS 7.x TLS seam: handshake records travel wrapped in PRELOGIN-type packets, so this shim strips/adds packet headers under `SslStream` during `AuthenticateAsServerAsync`, then flips to transparent passthrough.
39-
**TLS is pinned to 1.2**: a TLS 1.3 server emits NewSessionTicket records at handshake completion, which would still be prelogin-wrapped after the client switched to reading raw records ("cannot determine frame size" on the client).
45+
**TLS is pinned to 1.2 on this path**: a TLS 1.3 server emits NewSessionTicket records at handshake completion, which would still be prelogin-wrapped after the client switched to reading raw records ("cannot determine frame size" on the client).
4046
Matches SqlClient/real-server behavior for pre-TDS-8 encryption.
47+
The TDS 8.0 strict path needs no shim (the `SslStream` sits directly on the socket, records flow raw) and no version pin.
4148
- `Login7Request` — parses TDS version, packet size (accepted when 512–32767 and acked via ENVCHANGE type 4), hostname/username/password (de-obfuscated)/appname/database.
4249
An **empty** requested database maps to the default (user) database; **any non-empty name — including `master` — resolves genuinely through `ChangeDatabase`** (master is a real seeded database now, so `Database=master` lands in master rather than being aliased to the default).
4350
A `ChangeDatabase` failure becomes the probe-confirmed login pair — Msg 4060 severity 11 (`Cannot open database "x" requested by the login. The login failed.`, double-quoted name) then Msg 18456 severity 14 — before the connection closes.
@@ -385,7 +392,7 @@ A real build number is load-bearing for SSMS's per-build client feature gates (A
385392
## Divergences / deferred
386393

387394
- Login INFO states are approximations.
388-
- MARS ships (see [MARS](#mars-multiple-active-result-sets) below); no TDS 8.0 / `Encrypt=Strict`, no plaintext sessions, no integrated auth (an SSPI/FedAuth login presents an empty SQL username, which under a non-empty registry fails as Msg 18456 rather than negotiating).
395+
- MARS and TDS 8.0 / `Encrypt=Strict` ship (see [MARS](#mars-multiple-active-result-sets) below and the strict paragraph up top); no plaintext sessions, no integrated auth (an SSPI/FedAuth login presents an empty SQL username, which under a non-empty registry fails as Msg 18456 rather than negotiating).
389396
- Credential-enforcement edges not modeled: `ALTER LOGIN … DISABLE` parses but doesn't block login; password policy (`CHECK_POLICY` / expiration / lockout) never enforced; no login auditing.
390397
- RPC parameters are gap-free in both directions: every input TYPE_INFO is accepted — TVP / UDT / `sql_variant` / `text` / `ntext` / `image` — every client value-stream column type (bulk / TVP) decodes through the shared `TdsWireValue` / `TdsColumnDecoder`, and output-direction UDT / `sql_variant` parameters write back as RETURNVALUE tokens (see [CLR-UDT / sql_variant parameters](#clr-udt--sql_variant-parameters)).
391398
Non-cursor well-known ProcIDs beyond the sp_execute/sp_prepare family are rejected with ERROR 50000 naming the id.

0 commit comments

Comments
 (0)