You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
The TDS endpoint now accepts TDS 8.0 Encrypt=Strict connections. First-wire-byte routing onto an unpinned TLS-first SslStream negotiating ALPN tds/8.0 with prelogin/LOGIN7/MARS/SqlBulkCopy all inside the channel and LOGINACK echoing version 0x08000000, exposing SimulatedNetworkListener.ServerCertificate because SqlClient ignores TrustServerCertificate in strict mode and must pin via its ServerCertificate keyword. One-side-variant comparisons now convert the base-typed operand up to sql_variant, replacing the unwrap-and-promote residual.
-**BACPAC import** (`Simulation.ImportBacpac` — multi-database via repeated calls, `BacpacImportOptions`, `ModelXmlReader` dispatcher, BCP wire format, `BacpacBuilder` test harness) → [`bacpac-loader.md`](docs/claude/bacpac-loader.md).
272
272
-**Linked servers** (`Simulation.AddRemoteSimulation`, `sp_addlinkedserver` / `sp_dropserver`, four-part FROM routing through the remote's ADO.NET pipeline, `OPENQUERY(server,'query')` ad-hoc pass-through + compile-time schema discovery, `sys.servers`) → [`linked-servers.md`](docs/claude/linked-servers.md).
273
-
-**TDS network endpoint** (`Simulation.ListenLocalAsync` (loopback) / `Simulation.ListenNetworkAsync` (all interfaces, requires a registered login) → `SimulatedNetworkListener`; real SqlClient over loopback TCP+TLS; SQLBatch/RPC/TM/BulkLoad (`SqlBulkCopy`); legacy `text`/`ntext`/`image` result columns + `image` RPC input params in the in-band textptr wire form; TVP (`0xF3``SqlDbType.Structured`) RPC params via the shared `TdsColumnDecoder`; `sp_cursor*` API-server-cursor RPC family; mid-stream attention (cancel / `CommandTimeout`) via a carry-forward concurrent read → `DONE_ATTN` ack + probed XACT_ABORT tx semantics; MARS (`MultipleActiveResultSets=True`) via an SMP demux/mux over one shared connection with serialized cooperative multiplexing + per-session attention; credential enforcement via the `CREATE LOGIN` registry, Msg 18456 on mismatch; EF via plain `UseSqlServer`; oracles = `*.Tests.SqlClient` + `*.Tests.Smo`, the real-SMO consumer oracle) → [`tds-endpoint.md`](docs/claude/tds-endpoint.md).
273
+
- **TDS network endpoint** (`Simulation.ListenLocalAsync` (loopback) / `Simulation.ListenNetworkAsync` (all interfaces, requires a registered login) → `SimulatedNetworkListener`; real SqlClient over loopback TCP+TLS; SQLBatch/RPC/TM/BulkLoad (`SqlBulkCopy`); legacy `text`/`ntext`/`image` result columns + `image` RPC input params in the in-band textptr wire form; TVP (`0xF3` `SqlDbType.Structured`) RPC params via the shared `TdsColumnDecoder`; `sp_cursor*` API-server-cursor RPC family; mid-stream attention (cancel / `CommandTimeout`) via a carry-forward concurrent read → `DONE_ATTN` ack + probed XACT_ABORT tx semantics; MARS (`MultipleActiveResultSets=True`) via an SMP demux/mux over one shared connection with serialized cooperative multiplexing + per-session attention; TDS 8.0 `Encrypt=Strict` (TLS-first, ALPN `tds/8.0`, client pins the exposed `SimulatedNetworkListener.ServerCertificate` since SqlClient ignores `TrustServerCertificate` in strict mode); credential enforcement via the `CREATE LOGIN` registry, Msg 18456 on mismatch; EF via plain `UseSqlServer`; oracles = `*.Tests.SqlClient` + `*.Tests.Smo`, the real-SMO consumer oracle) → [`tds-endpoint.md`](docs/claude/tds-endpoint.md).
Copy file name to clipboardExpand all lines: docs/claude/backlog.md
+4-2Lines changed: 4 additions & 2 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -37,7 +37,7 @@ Remaining phases, roughly in value order:
37
37
Candidate follow-on legs within tool scope: Visual Studio's SQL Server Object Explorer (DacFx-driven, a different query dialect from SMO) and LINQPad.
38
38
-**SMO API sweep campaign** — `.vs/smo-sweep` (gitignored local harness) walks SMO's full reachable read surface against the self-hosted simulator and, identically, against the live reference, draining every `Property.Value` and `Script()`-ing every `IScriptable`; modes `sweep` / `sweep --live` / `diff` → sorted JSON reports + `reports/triage.md`; workflow = sweep both sides → triage → fix bundles → graduated `Tests.Smo` tests → re-sweep.
39
39
Open items from the latest triage: (a) `DBCC SHOW_STATISTICS … WITH STATS_STREAM` (SMO `Statistic.Stream`) stays `NotSupportedException` — it wants the raw serialized statistics-histogram blob, which the simulator has no faithful source for; (b) the unmodeled runtime/OS surfaces SMO reaches as absent objects (backup history `msdb.dbo.backupset`, `sys.dm_tran_persistent_version_store_stats`, file-space/IO DMVs, `sys.dm_os_process_memory`, `master.dbo.sysprocesses`, `FILEPROPERTY`, registry/OS xps) — surfaced as `PropertyCannotBeRetrievedException` / defaults, the legitimate-gap category.
40
-
-**Smaller fidelity items**: TDS 8.0 / `Encrypt=Strict` (ALPN via `SslApplicationProtocol`); user-supplied `X509Certificate2` and per-interface bind-address selection — add-on-demand public-surface expansions.
40
+
-**Smaller fidelity items**: user-supplied `X509Certificate2` (a CA-trusted certificate instead of the ephemeral self-signed one; strict-mode clients can already pin the exposed `SimulatedNetworkListener.ServerCertificate`) and per-interface bind-address selection — add-on-demand public-surface expansions.
41
41
Open residuals of shipped wire features (details in [`tds-endpoint.md`](tds-endpoint.md)): cancel/attention reaction is bounded by the in-flight statement's materialization, not interruptible inside a single statement's row loop; MARS never raises Msg 8628/8651 and fully materializes each session's response under the execution gate.
42
42
-**Chunked `OFFSET/FETCH` paging: per-page constant factor, complexity class matches real** (probed 2026-07-18, plans + timings): real SQL Server also redoes the work on every page — `Top(OFFSET…)` over an ordered index scan reading offset+fetch rows when an index supplies the order, or a full scan + `Sort(TOP offset+fetch)` re-sorted per page when not; no cross-query sorted-result caching exists on either side, so "sort once, serve many pages" is rejected (it would invent behavior real doesn't have).
43
43
The residual is constant-factor only: the simulator materializes (and for non-index order, sorts) all n rows per page regardless of offset where real's indexed plan touches only offset+fetch (measured at 150k rows / fetch 100 / offset 140k: sim ~41 ms vs real ~13 ms indexed, ~116 ms vs ~22 ms unindexed — real's sort is also 16-way parallel).
@@ -68,7 +68,9 @@ Low priority / niche — simulatable (as placeholder constants or a small model)
68
68
A named-filegroup registry now ships (`Database.Filegroups`, seeded PRIMARY=1, extended by the bacpac loader's `SqlFilegroup` dispatch; surfaced by `sys.filegroups`/`sys.data_spaces` and FILEGROUP-scoped extended properties) — the scalar helpers above could read it, though a `file_id` 1 placeholder is still needed for the file-level ones.
69
69
-**Certificates** — CERTENCODED / CERTPRIVATEKEY (needs a small certificate-name → bytes model, or NULL placeholders).
70
70
-**Full-text properties** — FULLTEXTCATALOGPROPERTY (the `CREATE FULLTEXT CATALOG` / `INDEX` DDL already ships; this reads its metadata).
71
-
-**`sql_variant` minor quirks** (cross-type family ordering shipped 2026-07-19 — see [`scalars.md`](scalars.md#sql_variant-expression-semantics)): a decimal-declared inner reports BaseType `numeric` rather than real's `decimal` (the single-decimal-family naming divergence); a mixed variant-vs-base-typed comparison unwraps and promotes where real converts the base side up to variant and applies the family rules (a string variant never matches a numeric literal on real — unprobed, retained).
71
+
-**`sql_variant` minor quirk** (cross-type family ordering and one-side-variant comparison both shipped 2026-07-19 — see [`scalars.md`](scalars.md#sql_variant-expression-semantics)): a decimal-declared inner reports BaseType `numeric` rather than real's `decimal`.
72
+
Probed 2026-07-19: real preserves the declared keyword *distinctly* — `decimal` and `numeric` never collapse — through literals, table columns, variant columns, and variant variables assigned from typed variables.
73
+
The faithful fix splits the per-`(p, s)``DecimalSqlType` singleton by declared keyword, forking the reference-identity space the row encoder, promote paths, and catalog surfaces key on — a medium refactor whose blast radius far exceeds the one metadata string it corrects, so it's deliberately deferred.
72
74
Deliberate exclusion, don't re-pitch: `msdb.dbo.syspolicy_configuration.current_value` stays `nvarchar` — it's a *view-body* projection (not a resource column) mixing `int` rows with a `binary` GUID row, every consumer reads a single named row and CASTs it, so a variant migration there would only touch the view SQL text for no observable gain.
73
75
-**FILESTREAM** — GET_FILESTREAM_TRANSACTION_CONTEXT (needs a FILESTREAM storage binding; NULL is a faithful "no FILESTREAM context" placeholder).
Copy file name to clipboardExpand all lines: docs/claude/scalars.md
+6-5Lines changed: 6 additions & 5 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -421,7 +421,7 @@ These carry real per-session state on `SimulatedDbConnection` (not placeholder c
421
421
A missing key reads as NULL; a NULL key argument to `SESSION_CONTEXT` raises **Msg 8116** (`session_context` lowercase in the wording).
422
422
`sp_set_session_context` with a NULL `@key` raises **Msg 225**; re-setting a key previously stored with `@read_only = 1` raises **Msg 15664**.
423
423
Like real SQL Server, `SESSION_CONTEXT` returns **`sql_variant`** preserving the stored value's base type — an `int` stored round-trips as `int`, an `nvarchar` as `nvarchar`.
424
-
The common `WHERE int_col = SESSION_CONTEXT(N'key')` shape works by the comparison path unwrapping the variant to its inner value.
424
+
The common `WHERE int_col = SESSION_CONTEXT(N'key')` shape works by the comparison path converting the column side up to `sql_variant` and matching within the exact-numeric family (the family rules below).
425
425
-**`SESSIONPROPERTY(name)`** (`Parser/Expressions/SessionProperty.cs`) — the current session setting for one of the ANSI / arithmetic SET options: `ANSI_NULLS`, `ANSI_PADDING`, `ANSI_WARNINGS`, `ARITHABORT`, `CONCAT_NULL_YIELDS_NULL`, `NUMERIC_ROUNDABORT`, `QUOTED_IDENTIFIER`.
Like real SQL Server the result is **`sql_variant`** with an inner base type of `int` (each option reads back 1 / 0).
@@ -507,9 +507,9 @@ Probe-confirmed against SQL Server 2025 (2026-07-16):
507
507
-**ExecuteScalar / GetValue** surface the inner CLR object (a bare `int` for `SERVERPROPERTY('EngineEdition')`, `string` for `Edition`), so most existing value assertions are unchanged; `GetDataTypeName` reports `sql_variant` and `GetFieldType` is `object`.
508
508
-**`ISNULL` / `COALESCE` / `CASE`** keep the `sql_variant` result type (variant has highest data-type precedence in `SqlType.Promote`) and preserve each value's inner type — `ISNULL(SESSIONPROPERTY('ANSI_NULLS'), 0)` stays `sql_variant`.
509
509
-**`UNION [ALL]`** keeps each row's own inner base type — no schema unification/promotion (a variant column can be `int` on one row, `nvarchar` on the next).
510
-
-**Comparison**against a non-variant (`WHERE int_col = SESSION_CONTEXT(N'k')`) unwraps the variant to its inner value.
Residual: real converts the base-typed side of a mixed pair *up*to `sql_variant` (highest precedence) and applies the family rules — so a string variant never matches a numeric literal on real, while the unwrap path converts and can match; unprobed, retained.
510
+
-**Comparison**with a `sql_variant` on *either* side follows the family rules below — the base-typed side of a mixed pair converts *up* to `sql_variant` (probe-confirmed as `CONVERT_IMPLICIT(sql_variant, …)` in real's plan; the variant never unwraps into ordinary type-precedence promotion).
511
+
So a string variant is less than any exact-numeric value and never equal to it (`variant nvarchar N'5' < 5`, never `=`), cross-family comparison stays value-blind even when one side is a plain literal or column, and **no comparison error is possible** — `variant nvarchar N'abc'` vs `int 5` is cleanly `<`, never Msg 245 (all probe-confirmed, including WHERE and JOIN forms over mixed-inner-type variant columns).
512
+
A bare string literal against a `datetime` variant promotes to a *character*-family variant (not to `datetime`), so `variant datetime = '2020-01-01'` is false while `= CAST('2020-01-01' AS datetime)` is true.
513
513
-**`SELECT … INTO`** from a variant-producing built-in creates a `sql_variant` column (probe-confirmed).
514
514
-**Arithmetic** rejects: `variant + non-variant` → **Msg 257** (`Implicit conversion from data type sql_variant to <target> is not allowed. Use the CONVERT function to run this query.`); `variant + variant` and `string + variant` → **Msg 402** (`… incompatible in the add operator`).
515
515
`PromoteForArithmetic` is the single source; a runtime guard in `IntegerArithmetic` routes through it so `Run`-time and projection-schema errors agree.
@@ -519,7 +519,8 @@ Probe-confirmed against SQL Server 2025 (2026-07-16):
519
519
NULL sorts lowest.
520
520
`MIN`/`MAX` pick the family-hierarchy extremes.
521
521
Same-collation character pairs compare under that collation; cross-collation pairs compare by code point **without** a Msg 468 conflict (probed) — the character family hashes by rank alone since no single hash agrees with both regimes.
522
-
The `SqlValue` variant arms (`CompareTo` / `Equals` / `GetHashCode`) implement the rules, so ORDER BY, GROUP BY, DISTINCT, MIN/MAX, and hash joins all inherit them via `SqlValueKey`.
522
+
The `SqlValue` variant arms (`CompareTo` / `Equals` / `GetHashCode`) implement the rules, so ORDER BY, GROUP BY, DISTINCT, MIN/MAX, and hash joins all inherit them via `SqlValueKey`; a variant-vs-base equi-join key promotes to `sql_variant` (`Promote` → `CoerceTo` wraps the base side), so the hash fast path keys by the same family semantics.
523
+
Oracle: `SqlVariantOrderingTests` (in-process; the one-side-variant comparison tests live there too).
523
524
524
525
## Built-in TVF: `STRING_SPLIT`
525
526
`STRING_SPLIT(input, separator [, enable_ordinal])` dispatches in `ParseSingleFromSource` alongside `OPENJSON` — case-insensitive name match before generic name resolution.
Copy file name to clipboardExpand all lines: docs/claude/tds-endpoint.md
+11-4Lines changed: 11 additions & 4 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -8,9 +8,15 @@ The listener binds IPv4 loopback plus best-effort IPv6 loopback on the same port
8
8
Oracle: `NetworkListenerTests`.
9
9
10
10
Connection-string requirements: `TrustServerCertificate=true` (the endpoint presents an ephemeral in-memory self-signed cert generated per listener) and credentials per the enforcement rule below (any credentials when no logins exist).
11
-
`Encrypt=Strict` (TDS 8.0) is not supported; default/`Mandatory`/`Optional` all negotiate to full encryption via `ENCRYPT_REQ`.
11
+
Default/`Mandatory`/`Optional` all negotiate to full encryption via `ENCRYPT_REQ`.
12
12
A client that cannot do TLS at all (`ENCRYPT_NOT_SUP`) is disconnected after the prelogin response — there is no plaintext mode.
13
13
14
+
**`Encrypt=Strict` (TDS 8.0) ships**: the client opens with a bare TLS ClientHello negotiating ALPN `tds/8.0`, and every TDS packet — prelogin included — flows inside the TLS channel; the session routes on the first wire byte (TLS handshake record `0x16` vs cleartext PRELOGIN `0x12`).
15
+
SqlClient **ignores `TrustServerCertificate` in strict mode** and always validates the certificate (chain + hostname), so a strict client must pin instead: export `SimulatedNetworkListener.ServerCertificate` (the presented certificate, public part only) to a file and reference it with the connection string's `ServerCertificate` keyword (empirically confirmed against SqlClient 7.0.2 — `TrustServerCertificate=True` alone fails strict with `UntrustedRoot` + hostname mismatch).
16
+
The strict TLS handshake is not version-pinned (TLS 1.3 negotiates; raw records make NewSessionTicket harmless), and LOGINACK echoes TDS version `0x08000000` when LOGIN7 requested it.
17
+
MARS and `SqlBulkCopy` ride the strict channel unchanged.
18
+
Oracle: `StrictEncryptionTests`.
19
+
14
20
## Credential enforcement
15
21
16
22
The knob is T-SQL, not API: `CREATE LOGIN name WITH PASSWORD = '…'` (run through any in-process connection — those never authenticate) populates the server-scope registry `Simulation.Logins`, and the endpoint enforces it.
@@ -30,14 +36,15 @@ Login DDL details in [`permissions.md`](permissions.md).
30
36
## Architecture
31
37
32
38
Everything lives in `Network/` (internal) except the public `SimulatedNetworkListener` (root) and `Simulation.Listen.cs` (the `ListenLocalAsync` / `ListenNetworkAsync` partial).
33
-
One task per accepted socket runs `TdsSession.RunAsync`: prelogin → TLS handshake → LOGIN7 → batch loop.
39
+
One task per accepted socket runs `TdsSession.RunAsync`: prelogin → TLS handshake → LOGIN7 → batch loop (TDS 7.x), or TLS handshake → prelogin → LOGIN7 → batch loop (TDS 8.0 strict, routed by the peeked first byte).
34
40
The session maps 1:1 onto a `SimulatedDbConnection`; execution flows through `Simulation.CreateResultSetsForCommand`, which yields both result sets and per-statement `RecordsAffected` — the TDS layer is a pure translator and touches no engine code.
35
41
36
42
-`TdsPacketTransport` — packet framing both directions: reassembles inbound packet sequences into `TdsMessage` (EOM-terminated), stamps outbound headers (type 0x04, SPID truncated to 16 bits, incrementing packet id).
37
43
The stream it rides is swapped from the raw `NetworkStream` to the `SslStream` after the handshake.
38
44
-`TlsHandshakeFramingStream` — the TDS 7.x TLS seam: handshake records travel wrapped in PRELOGIN-type packets, so this shim strips/adds packet headers under `SslStream` during `AuthenticateAsServerAsync`, then flips to transparent passthrough.
39
-
**TLS is pinned to 1.2**: a TLS 1.3 server emits NewSessionTicket records at handshake completion, which would still be prelogin-wrapped after the client switched to reading raw records ("cannot determine frame size" on the client).
45
+
**TLS is pinned to 1.2 on this path**: a TLS 1.3 server emits NewSessionTicket records at handshake completion, which would still be prelogin-wrapped after the client switched to reading raw records ("cannot determine frame size" on the client).
40
46
Matches SqlClient/real-server behavior for pre-TDS-8 encryption.
47
+
The TDS 8.0 strict path needs no shim (the `SslStream` sits directly on the socket, records flow raw) and no version pin.
41
48
-`Login7Request` — parses TDS version, packet size (accepted when 512–32767 and acked via ENVCHANGE type 4), hostname/username/password (de-obfuscated)/appname/database.
42
49
An **empty** requested database maps to the default (user) database; **any non-empty name — including `master` — resolves genuinely through `ChangeDatabase`** (master is a real seeded database now, so `Database=master` lands in master rather than being aliased to the default).
43
50
A `ChangeDatabase` failure becomes the probe-confirmed login pair — Msg 4060 severity 11 (`Cannot open database "x" requested by the login. The login failed.`, double-quoted name) then Msg 18456 severity 14 — before the connection closes.
@@ -385,7 +392,7 @@ A real build number is load-bearing for SSMS's per-build client feature gates (A
385
392
## Divergences / deferred
386
393
387
394
- Login INFO states are approximations.
388
-
- MARS ships (see [MARS](#mars-multiple-active-result-sets) below); no TDS 8.0 / `Encrypt=Strict`, no plaintext sessions, no integrated auth (an SSPI/FedAuth login presents an empty SQL username, which under a non-empty registry fails as Msg 18456 rather than negotiating).
395
+
- MARS and TDS 8.0 / `Encrypt=Strict` ship (see [MARS](#mars-multiple-active-result-sets) below and the strict paragraph up top); no plaintext sessions, no integrated auth (an SSPI/FedAuth login presents an empty SQL username, which under a non-empty registry fails as Msg 18456 rather than negotiating).
389
396
- Credential-enforcement edges not modeled: `ALTER LOGIN … DISABLE` parses but doesn't block login; password policy (`CHECK_POLICY` / expiration / lockout) never enforced; no login auditing.
390
397
- RPC parameters are gap-free in both directions: every input TYPE_INFO is accepted — TVP / UDT / `sql_variant` / `text` / `ntext` / `image` — every client value-stream column type (bulk / TVP) decodes through the shared `TdsWireValue` / `TdsColumnDecoder`, and output-direction UDT / `sql_variant` parameters write back as RETURNVALUE tokens (see [CLR-UDT / sql_variant parameters](#clr-udt--sql_variant-parameters)).
391
398
Non-cursor well-known ProcIDs beyond the sp_execute/sp_prepare family are rejected with ERROR 50000 naming the id.
0 commit comments