Skip to content

Commit ac187f1

Browse files
committed
A table variable or TVP bound to a stored-proc READONLY parameter now copies off-row data correctly.
1 parent b7fd9ab commit ac187f1

7 files changed

Lines changed: 120 additions & 11 deletions

File tree

docs/claude/backlog.md

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -42,7 +42,8 @@ Remaining phases, roughly in value order:
4242
**`text`/`ntext`/`image` ship fully** (2026-07-19) — result columns, `image` (0x22) input parameters, and **`SqlBulkCopy` into a legacy-LOB column** (2026-07-19: the BCP LONGLEN COLMETADATA + in-band text-pointer ROW value now decode); see [`tds-endpoint.md`](tds-endpoint.md#legacy-text--ntext--image-wire-forms).
4343
**`sql_variant` and UDT (`geography`/`geometry`/`hierarchyid`) ship as RPC *input* parameters AND as *columns inside a TVP*** (2026-07-19 — the shared decoder made the TVP columns near-free; see [`tds-endpoint.md`](tds-endpoint.md#tvp-parameters-sqldbtypestructured)).
4444
**TVP parameters (0xF3, `SqlDbType.Structured`) ship**`DataTable` / `IEnumerable<SqlDataRecord>` / `DbDataReader` sources over proc-RPC + sp_executesql, error parity for arity / type / NOT NULL / CHECK / PK / UNIQUE / unknown-type.
45-
Residuals: **output-direction** UDT / `sql_variant` params; **`SET TEXTSIZE`** doesn't truncate returned LOB data (parse-and-discard); a **LOB-backed TVP column** (`nvarchar(max)` / `geography` / `geometry`) bound to a **stored-proc READONLY parameter** hits a pre-existing table-variable LOB-copy gap in proc-parameter binding (the copied row's LOB pointers dangle — reproducible in-process, unrelated to the wire; the `sp_executesql` text path is clean, and the crash boundary surfaces it as a severe error not a silent reset).
45+
Residuals: **output-direction** UDT / `sql_variant` params; **`SET TEXTSIZE`** doesn't truncate returned LOB data (parse-and-discard).
46+
(The LOB-backed TVP column → stored-proc READONLY parameter dangling-pointer bug was fixed 2026-07-19: the proc-parameter copy re-homes off-row values into the parameter's heap; see [`table-valued-parameters.md`](table-valued-parameters.md).)
4647
- **Smaller fidelity items**: TDS 8.0 / `Encrypt=Strict` (ALPN via `SslApplicationProtocol`); user-supplied `X509Certificate2` — an add-on-demand public-surface expansion.
4748
(Off-loopback binding shipped 2026-07-19 as `ListenNetworkAsync`, gated on a registered login; per-interface bind-address selection remains add-on-demand.)
4849
(**Terminal crash boundary** shipped 2026-07-19 — an exception outside the typed catch list now emits a best-effort Msg 0 / severity 20 ERROR at a token boundary instead of a silent transport reset, `AtTokenBoundary`-gated for mid-token safety, on both the non-MARS and MARS session loops; see [`tds-endpoint.md`](tds-endpoint.md#terminal-crash-boundary).

docs/claude/table-valued-parameters.md

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -101,6 +101,8 @@ Two-shape recognition in `ParseExecArgument`:
101101
- Allocates a parallel `boundTableValues` array alongside `boundValues`.
102102
- For each TVP parameter: clones the type into a fresh TVP-flagged `HeapTable`, then bulk-copies the supplied source's rows via `Heap.EnumerateRows()``Heap.Insert()`.
103103
Pass-by-value semantics matching real SQL Server's TVP copy.
104+
When the type has any off-row-capable column (`Heap.ReclaimColumns` non-null — a LOB-typed column or any variable-length column, since bounded var columns overflow-push past 8060 bytes), each row decodes from the source heap and re-encodes against the clone's heap so LOB chains are re-homed; raw row bytes would carry pointers into the *source* heap's `LobPages` and dangle.
105+
Pointer-free schemas keep the raw byte copy.
104106
- Unsupplied TVP parameters get an empty clone (probe-confirmed: `EXEC p` with the TVP arg omitted is legal).
105107
- Scalar argument passed to a TVP parameter raises **Msg 206** ("Operand type clash: int is incompatible with `<typeleaf>`") — probe-confirmed wording, with the table-type leaf name (no schema qualifier).
106108

docs/claude/tds-endpoint.md

Lines changed: 1 addition & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -254,8 +254,7 @@ Probed against SQL Server 2025 + SqlClient 7.0.2 (2026-07-18).
254254
Enforcing NOT NULL / CHECK / PK / UNIQUE on the structured path is a fidelity upgrade shared with the in-process path (which previously skipped them via a direct `Heap.Insert`).
255255
- **`sql_variant` / UDT columns** decode (2026-07-19): a `sql_variant` TVP column arrives as `0x62` + a 4-byte max length with a 4-byte-total-length (0 = NULL) self-describing body value; a `hierarchyid` / `geography` / `geometry` column arrives as `0xF0` + three B_VARCHARs (db / schema / type) with a PLP value — **the same wire forms the matching RPC parameters carry** (cleartext-probed inside TVP_COLMETADATA / TVP_ROW against SQL Server 2025 + SqlClient 7.0.2), so unification made them decode for near-free.
256256
Oracle: `TvpVariantUdtColumnTests`.
257-
**Caveat**: a LOB-backed TVP column (`geography` / `geometry`, like `nvarchar(max)`) round-trips fine through the `sp_executesql` text path (the parameter is a batch table variable), but routing it through a **stored-procedure READONLY parameter** hits a pre-existing table-variable LOB-copy gap in proc-parameter binding (the copied row's LOB pointers dangle) — reproducible in-process too, and unrelated to the wire decode.
258-
The terminal crash boundary now surfaces that as a clean severe error rather than a silent transport reset; the tests use the text path.
257+
A LOB-backed TVP column (`geography` / `geometry`, like `nvarchar(max)`) round-trips through both the `sp_executesql` text path (the parameter is a batch table variable) **and** a **stored-procedure READONLY parameter** — the proc-parameter copy re-homes off-row values into the parameter's own heap (see [`table-valued-parameters.md`](table-valued-parameters.md)); `TvpVariantUdtColumnTests.LobBackedColumns_ThroughProcReadonlyParameter_RoundTrip` covers the proc route.
259258
- **Residuals**: a `DBNull`-valued Structured parameter is rejected by SqlClient client-side (`NotSupportedException`) before any bytes hit the wire, so the server never sees a null TVP (a wire TVP_NULL, column count `0xFFFF`, is handled defensively as an empty table).
260259
TVP_ORDER_UNIQUE (0x10) / TVP_COLUMN_ORDERING (0x11) optional metadata — never emitted by `DataTable` / `SqlDataReader` sources — is rejected with a clear ERROR rather than a guessed parse.
261260
An identity column present in a TVP raises **Msg 1077** on real SQL Server even for a `DBNull` value; the simulator's structured path auto-generates on `DBNull` (pre-existing in-process divergence, unchanged).

src/SqlServerSimulator/Network/CLAUDE.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -70,7 +70,7 @@ These notes are the local implementation contracts.
7070
UDT **input** *parameters* decode (`TdsRpc.DecodeClrUdt``TdsWireValue.BuildUdtValue`): the client UDT_INFO is three B_VARCHARs (db/schema/type, no max-size, no AQN — shorter than COLMETADATA) + PLP value; hierarchyid OrdPath bytes bind verbatim, spatial WKB decodes via `SpatialWkbDecoder.TryDecode`, resolved case-insensitively into a pre-built `SqlValue` (unknown type → Msg 8064, invalid spatial bytes → Msg 8023).
7171
**UDT columns inside a TVP** decode through the same builder (`TdsColumnDecoder`: 0xF0 + three B_VARCHARs + PLP value).
7272
**Output** direction stays rejected.
73-
A LOB-backed UDT TVP column (`geography`/`geometry`) round-trips via the `sp_executesql` text path but hits a pre-existing table-variable LOB-copy gap when bound to a stored-proc READONLY parameter (docs/claude/tds-endpoint.md).
73+
A LOB-backed UDT TVP column (`geography`/`geometry`) round-trips via both the `sp_executesql` text path and a stored-proc READONLY parameter (the proc-parameter copy re-homes off-row values — docs/claude/table-valued-parameters.md).
7474
Oracles: `UdtRpcParameterTests`, `TvpVariantUdtColumnTests`.
7575
- **Length-prefix trap**: most strings are char-counted (B_VARCHAR / US_VARCHAR), but TM-request transaction names are byte-counted — misreading them as char-counted overruns the payload on every `SqlTransaction.Save`.
7676
LOGIN7's password length pair is char-counted like the other LOGIN7 fields (oracle-confirmed with a surrogate-pair password); only its *bytes* are special, obfuscated per MS-TDS (client swaps nibbles then XORs 0xA5 — decode inverts the order).

src/SqlServerSimulator/Simulation/Simulation.InvokeProcedure.cs

Lines changed: 11 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -141,8 +141,18 @@ internal IEnumerable<SimulatedStatementOutcome> InvokeProcedure(
141141
var clone = tvpType.Clone("@" + param.Name, outerBatch, isTableValuedParameter: true);
142142
if (boundTableValues[i] is { } supplied)
143143
{
144+
// Row bytes can point into the source heap's off-row
145+
// pages (LOB chains, overflow-pushed var columns), so
146+
// off-row-capable schemas decode and re-encode each row
147+
// against the clone's heap; pointer-free schemas copy
148+
// the bytes as-is.
149+
var reencode = supplied.Heap.ReclaimColumns is not null;
144150
foreach (var row in supplied.Heap.EnumerateRows())
145-
_ = clone.Heap.Insert(row);
151+
{
152+
_ = clone.Heap.Insert(reencode
153+
? RowEncoder.EncodeRow(supplied.StoredColumns, RowDecoder.DecodeRow(supplied.StoredColumns, row, supplied.Heap), clone.Heap)
154+
: row);
155+
}
146156
}
147157
tableVariables[param.Name] = clone;
148158
continue;

tests/SqlServerSimulator.Tests.SqlClient/TvpVariantUdtColumnTests.cs

Lines changed: 41 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -21,12 +21,10 @@ namespace SqlServerSimulator;
2121
/// value — both the same forms as the matching RPC parameters.
2222
/// </summary>
2323
/// <remarks>
24-
/// These drive the TVP through the <c>sp_executesql</c> text path (the parameter
25-
/// binds as its own <c>@rows</c> table variable). Routing a LOB-backed TVP column
26-
/// (<c>geography</c> / <c>geometry</c>, like <c>nvarchar(max)</c>) through a
27-
/// stored-procedure READONLY parameter hits a pre-existing table-variable LOB-copy
28-
/// gap in proc-parameter binding, unrelated to the wire decode; see
29-
/// docs/claude/tds-endpoint.md.
24+
/// Most of these drive the TVP through the <c>sp_executesql</c> text path (the
25+
/// parameter binds as its own <c>@rows</c> table variable); the LOB-backed
26+
/// proc test covers the stored-procedure READONLY route, whose parameter copy
27+
/// re-homes off-row values into the parameter's own heap.
3028
/// </remarks>
3129
[TestClass]
3230
public sealed class TvpVariantUdtColumnTests
@@ -127,6 +125,43 @@ public async Task GeographyColumn_RoundTripsToWkt()
127125
.ExecuteScalarAsync(TestContext.CancellationToken));
128126
}
129127

128+
// LOB-backed columns (nvarchar(max), geography) bound to a stored-proc
129+
// READONLY parameter: the proc-parameter copy must carry the off-row
130+
// values into the parameter's table variable, not just the row bytes.
131+
[TestMethod]
132+
public async Task LobBackedColumns_ThroughProcReadonlyParameter_RoundTrip()
133+
{
134+
var simulation = new Simulation();
135+
Wire.ExecInProc(simulation, "create type dbo.LobRows as table (id int, doc nvarchar(max), g geography)");
136+
Wire.ExecInProc(simulation, "create table dbo.sink (id int, doc nvarchar(max), g geography)");
137+
Wire.ExecInProc(simulation, "create proc dbo.ins_lob @rows dbo.LobRows readonly as insert into dbo.sink select id, doc, g from @rows");
138+
await using var listener = await simulation.ListenLocalAsync(0, TestContext.CancellationToken);
139+
await using var connection = await Wire.OpenAsync(listener, TestContext.CancellationToken);
140+
141+
var metadata = new[]
142+
{
143+
new SqlMetaData("id", SqlDbType.Int),
144+
new SqlMetaData("doc", SqlDbType.NVarChar, SqlMetaData.Max),
145+
new SqlMetaData("g", SqlDbType.Udt, typeof(SqlGeography), "geography"),
146+
};
147+
var record = new SqlDataRecord(metadata);
148+
record.SetInt32(0, 1);
149+
record.SetString(1, new string('x', 100000));
150+
record.SetValue(2, SqlGeography.STGeomFromText(new SqlChars("POINT(-122.3 47.6)"), 4326));
151+
152+
await using var command = new SqlCommand("dbo.ins_lob", connection) { CommandType = CommandType.StoredProcedure };
153+
var parameter = command.Parameters.AddWithValue("@rows", new[] { record });
154+
parameter.SqlDbType = SqlDbType.Structured;
155+
parameter.TypeName = "dbo.LobRows";
156+
_ = await command.ExecuteNonQueryAsync(TestContext.CancellationToken);
157+
158+
await using var reader = await new SqlCommand("select cast(len(doc) as int), g.ToString() from dbo.sink", connection)
159+
.ExecuteReaderAsync(TestContext.CancellationToken);
160+
IsTrue(await reader.ReadAsync(TestContext.CancellationToken));
161+
AreEqual(100000, reader.GetInt32(0));
162+
AreEqual("POINT (-122.3 47.6)", reader.GetString(1));
163+
}
164+
130165
[TestMethod]
131166
public async Task GeometryColumn_RoundTripsToWkt()
132167
{

tests/SqlServerSimulator.Tests/TableValuedParameterTests.cs

Lines changed: 62 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -262,6 +262,68 @@ public void Exec_TvpArg_NestedProcPassThrough_PreservesRows()
262262
AreEqual(2, sim.ExecuteScalar("declare @t dbo.t1; insert @t values (1), (2); exec dbo.outer_p @t"));
263263
}
264264

265+
// ---- Off-row values crossing the proc-parameter copy ----
266+
267+
// A table variable's off-row bytes (LOB chains for MAX-typed columns,
268+
// and bounded var columns overflow-pushed past 8060) live in the source
269+
// heap; the proc-parameter copy must re-home them into the parameter's
270+
// own heap or the copied rows' pointers dangle.
271+
272+
[TestMethod]
273+
public void Exec_TvpArg_NvarcharMaxColumn_LobValueSurvivesParameterCopy()
274+
{
275+
var sim = new Simulation();
276+
sim.ExecuteBatches(
277+
"create type dbo.t1 as table (id int, doc nvarchar(max))",
278+
"create proc dbo.p1 @rows dbo.t1 readonly as select cast(len(doc) as int) l, substring(doc, 99999, 2) tail from @rows");
279+
using var rdr = sim.ExecuteReader("""
280+
declare @t dbo.t1;
281+
insert @t values (1, replicate(cast(N'x' as nvarchar(max)), 99999) + N'y');
282+
exec dbo.p1 @t
283+
""");
284+
IsTrue(rdr.Read());
285+
AreEqual(100000, rdr.GetInt32(0));
286+
AreEqual("xy", rdr.GetString(1));
287+
}
288+
289+
[TestMethod]
290+
public void Exec_TvpArg_OverflowPushedBoundedColumn_SurvivesParameterCopy()
291+
{
292+
var sim = new Simulation();
293+
sim.ExecuteBatches(
294+
"create type dbo.t1 as table (id int, a varchar(8000), b varchar(8000))",
295+
"create proc dbo.p1 @rows dbo.t1 readonly as select len(a) + len(b) from @rows");
296+
AreEqual(16000, sim.ExecuteScalar("""
297+
declare @t dbo.t1;
298+
insert @t values (1, replicate('a', 8000), replicate('b', 8000));
299+
exec dbo.p1 @t
300+
"""));
301+
}
302+
303+
[TestMethod]
304+
public void Structured_DataTable_NvarcharMaxColumn_LobValueSurvivesParameterCopy()
305+
{
306+
var simulation = new Simulation();
307+
_ = simulation.ExecuteNonQuery("create type dbo.t1 as table (id int, doc nvarchar(max))");
308+
_ = simulation.ExecuteNonQuery("create proc dbo.p1 @rows dbo.t1 readonly as select cast(len(doc) as int) from @rows");
309+
310+
using var con = simulation.CreateDbConnection();
311+
con.Open();
312+
using var cmd = con.CreateCommand();
313+
cmd.CommandText = "exec dbo.p1 @rows";
314+
var dt = new DataTable();
315+
_ = dt.Columns.Add("id", typeof(int));
316+
_ = dt.Columns.Add("doc", typeof(string));
317+
_ = dt.Rows.Add(1, new string('x', 100000));
318+
var p = cmd.CreateParameter();
319+
p.ParameterName = "@rows";
320+
p.Value = dt;
321+
p.TypeName = "dbo.t1";
322+
_ = cmd.Parameters.Add(p);
323+
324+
AreEqual(100000, cmd.ExecuteScalar());
325+
}
326+
265327
// ---- ADO.NET Structured parameter (DataTable / IDataReader) ----
266328

267329
[TestMethod]

0 commit comments

Comments
 (0)