Commit 5433e91
committed
BACPAC loader Phase I —
**OrdPath binary encoding** (derived empirically by probing SQL Server 2025 across a synthetic `/N/` sweep over [0..300] + [-1..-50] + boundary values + the canonical AW hierarchyid byte ↔ path table for HR.Employee.OrganizationNode and Production.Document.DocumentNode). Each ordinal is a self-contained variable-bit sequence: prefix bits identify the range, value bits encode the position within range, a terminator bit `1` ends the ordinal. Multiple ordinals concatenate left-to-right, the byte stream pads with `0`s to a byte boundary, and the path ends when remaining bits are all zero. The four prefix codes the decoder ships:
| Prefix | Bits | Range | Encoding length | Shape |
|---|---|---|---|---|
| `01` | 2 | [0..3] | 5 bits total | `01_XX_1` (2 value bits) |
| `100` | 3 | [4..7] | 6 bits | `100_XX_1` (2 value bits) |
| `101` | 3 | [8..15] | 7 bits | `101_XXX_1` (3 value bits) |
| `110` | 3 | [16..79] | 12 bits | `110_AB_0_C_1_DEF_1` (6 value bits with structural-bit insertion at positions 6 = static `0` and 8 = static `1`) |
The `110`-prefix encoding's structural bits at positions 6 and 8 are load-bearing for round-trip but carry no value information — they're likely artifacts of the original byte-aligned design but reproduce verbatim from probe data. Value composition: `value = (high2 << 4) | (midHigh << 3) | low3` for the 6-bit value, then `ordinal = 16 + value`. Probe-confirmed across [16..79] via /16/=`C110`, /17/=`C130`, /23/=`C1F0`, /24/=`C310`, /48/=`D110`, /79/=`DBF0` plus 32 sample points in between. AW's actual ordinal usage (probed via `SELECT OrganizationNode.ToString(), CAST(OrganizationNode AS varbinary(892))` against the reference instance's AdventureWorks2025 database) spans 0..22 with a single excursion to /6/ at the top level — all comfortably within the [0..79] envelope.
**Deferred ordinal shapes** (raise `NotSupportedException` so the BCP file lands on Skipped if encountered): negative ordinals (`00` prefix, 1-byte single-byte encoding for [-1..-8], 2-byte for [-9..-24], etc. — encoding empirically reverse-engineerable but not exercised by AW), ordinals ≥ 80 (`1110` prefix and beyond — would extend the value-bit / structural-bit layout, also not exercised by AW), and the dotted-sub-ordinal form `/N.M/` (the encoding for `/1.5/` = `6460` differs from `/1/` followed by `/5/` — context-sensitive sub-ordinal encoding). All three combine into a follow-up bundle if a non-AW bacpac requires them.
**`BitReader` ref struct** (private to `HierarchyIdWireDecoder`): `ReadBit()` / `ReadBits(N)` consume bits MSB-first within each byte; `HasMoreNonZeroBits()` peeks the remaining unread tail for any `1` to distinguish "more ordinals follow" from "padding tail". Ref-struct + readonly-where-applicable keeps the per-row decoding allocation-free for AW's 290-row HR.Employee hot path.
**BcpRowReader integration**: new `EightBytePayload.HierarchyId` enum case dispatched from `DecodeColumn`'s 8-byte-prefix family alongside the MAX / xml / spatial cases (the 8-byte LE length-prefix wire shape is shared across the whole CLR-UDT + MAX family per the probe finding from Phase H). Decoded `int[][]` flows through `SqlValue.FromHierarchyId`; the simulator's existing hierarchyid storage encoder writes the segment-array LE form to row bytes, which round-trips back through `HierarchyIdSqlType.Decode` cleanly. Cross-engine CAST byte equality (the simulator's own `varbinary(hierarchyid)` matching SQL Server's documented binary form) is the symmetric companion work — the wire **decoder** ships, the wire **encoder** is the remaining piece for a future bundle.
**Test coverage**: 45 new `HierarchyIdWireDecoderTests` in `SqlServerSimulator.Tests.Internal/Storage/`. Each `[DataRow]` pairs a probe-confirmed hex byte sequence with its canonical string form: range [0..3] (4 cases) + [4..7] (4) + [8..15] (8) + [16..79] (10 sample points) + multi-level paths (8 single-byte and multi-byte sample paths) + AW-derived ground-truth paths (8 cases from HR.Employee and Production.Document). Plus dedicated tests for: empty input → `/` root; large-ordinal input → `NotSupportedException`; negative-range input → `NotSupportedException`. Total decoder-test count: 45.
**BacpacLoaderTests updates**: `Load_AW_Bcp_Data_Loads` row floor raised from `>= 759_500` to `== 760_167` (locked at full AW row count); `_DataFile` Skipped list locked at empty (zero failures). New `Load_AW_Hierarchyid_Column_Carries_Path` test exercises the end-to-end path: HumanResources.Employee row count = 290 with exactly 1 NULL OrganizationNode (the CEO at BusinessEntityID=1), employee 2's OrganizationNode.ToString() = `/1/`, Production.Document row count = 12 with the "Documents" root entry's DocumentNode.ToString() = `/`. The internal tests test count grows by 46 (45 decoder + 1 new loader) from 244 → 290.
CLAUDE.md "BACPAC import" pointer rewrites to reflect 100% AW coverage + the new `HierarchyIdWireDecoder` deep-dive. `docs/claude/bacpac-prerequisites.md` Status section updates to "100% row coverage" + lifts the hierarchyid-blocked-rows caveat; the order-of-operations checklist marks the wire decoder shipped with the symmetric encoder (byte-identical CAST round-trip) called out as separate future work. The "BCP wire format" matrix's CLR-UDT row updates from "hierarchyid blocks via NotSupportedException" to "hierarchyid decodes via HierarchyIdWireDecoder".hierarchyid binary wire decoder. AdventureWorks2025 row coverage rises from 759,833 → **760,167 / 760,167 (100%)**; every BCP shard loads cleanly. HR.Employee's 290 employees ship with their organization paths intact, Production.Document's 12 documents likewise. Cross-engine round-trip: SQL Server's hierarchyid varbinary payload → simulator's int[][] segment-array form via a brand-new HierarchyIdWireDecoder (SqlServerSimulator/Storage/Bacpac/HierarchyIdWireDecoder.cs).1 parent f1be9d2 commit 5433e91
6 files changed
Lines changed: 330 additions & 23 deletions
File tree
- SqlServerSimulator.Tests.Internal/Storage
- SqlServerSimulator/Storage/Bacpac
- docs/claude
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
164 | 164 | | |
165 | 165 | | |
166 | 166 | | |
167 | | - | |
| 167 | + | |
168 | 168 | | |
169 | 169 | | |
170 | 170 | | |
| |||
Lines changed: 40 additions & 18 deletions
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
259 | 259 | | |
260 | 260 | | |
261 | 261 | | |
262 | | - | |
263 | | - | |
264 | | - | |
265 | | - | |
266 | | - | |
267 | | - | |
| 262 | + | |
| 263 | + | |
| 264 | + | |
| 265 | + | |
268 | 266 | | |
269 | | - | |
| 267 | + | |
270 | 268 | | |
271 | | - | |
272 | 269 | | |
273 | 270 | | |
274 | | - | |
275 | 271 | | |
| 272 | + | |
| 273 | + | |
276 | 274 | | |
277 | 275 | | |
278 | 276 | | |
279 | 277 | | |
280 | | - | |
281 | | - | |
282 | | - | |
| 278 | + | |
| 279 | + | |
283 | 280 | | |
284 | | - | |
285 | | - | |
286 | | - | |
287 | | - | |
288 | | - | |
289 | | - | |
| 281 | + | |
| 282 | + | |
| 283 | + | |
| 284 | + | |
| 285 | + | |
| 286 | + | |
| 287 | + | |
| 288 | + | |
| 289 | + | |
| 290 | + | |
| 291 | + | |
| 292 | + | |
| 293 | + | |
| 294 | + | |
| 295 | + | |
| 296 | + | |
| 297 | + | |
| 298 | + | |
| 299 | + | |
| 300 | + | |
| 301 | + | |
| 302 | + | |
| 303 | + | |
| 304 | + | |
| 305 | + | |
| 306 | + | |
| 307 | + | |
| 308 | + | |
| 309 | + | |
| 310 | + | |
| 311 | + | |
290 | 312 | | |
291 | 313 | | |
292 | 314 | | |
| |||
Lines changed: 104 additions & 0 deletions
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
| 1 | + | |
| 2 | + | |
| 3 | + | |
| 4 | + | |
| 5 | + | |
| 6 | + | |
| 7 | + | |
| 8 | + | |
| 9 | + | |
| 10 | + | |
| 11 | + | |
| 12 | + | |
| 13 | + | |
| 14 | + | |
| 15 | + | |
| 16 | + | |
| 17 | + | |
| 18 | + | |
| 19 | + | |
| 20 | + | |
| 21 | + | |
| 22 | + | |
| 23 | + | |
| 24 | + | |
| 25 | + | |
| 26 | + | |
| 27 | + | |
| 28 | + | |
| 29 | + | |
| 30 | + | |
| 31 | + | |
| 32 | + | |
| 33 | + | |
| 34 | + | |
| 35 | + | |
| 36 | + | |
| 37 | + | |
| 38 | + | |
| 39 | + | |
| 40 | + | |
| 41 | + | |
| 42 | + | |
| 43 | + | |
| 44 | + | |
| 45 | + | |
| 46 | + | |
| 47 | + | |
| 48 | + | |
| 49 | + | |
| 50 | + | |
| 51 | + | |
| 52 | + | |
| 53 | + | |
| 54 | + | |
| 55 | + | |
| 56 | + | |
| 57 | + | |
| 58 | + | |
| 59 | + | |
| 60 | + | |
| 61 | + | |
| 62 | + | |
| 63 | + | |
| 64 | + | |
| 65 | + | |
| 66 | + | |
| 67 | + | |
| 68 | + | |
| 69 | + | |
| 70 | + | |
| 71 | + | |
| 72 | + | |
| 73 | + | |
| 74 | + | |
| 75 | + | |
| 76 | + | |
| 77 | + | |
| 78 | + | |
| 79 | + | |
| 80 | + | |
| 81 | + | |
| 82 | + | |
| 83 | + | |
| 84 | + | |
| 85 | + | |
| 86 | + | |
| 87 | + | |
| 88 | + | |
| 89 | + | |
| 90 | + | |
| 91 | + | |
| 92 | + | |
| 93 | + | |
| 94 | + | |
| 95 | + | |
| 96 | + | |
| 97 | + | |
| 98 | + | |
| 99 | + | |
| 100 | + | |
| 101 | + | |
| 102 | + | |
| 103 | + | |
| 104 | + | |
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
112 | 112 | | |
113 | 113 | | |
114 | 114 | | |
| 115 | + | |
115 | 116 | | |
116 | 117 | | |
117 | 118 | | |
| |||
147 | 148 | | |
148 | 149 | | |
149 | 150 | | |
| 151 | + | |
| 152 | + | |
| 153 | + | |
| 154 | + | |
| 155 | + | |
| 156 | + | |
| 157 | + | |
| 158 | + | |
| 159 | + | |
| 160 | + | |
150 | 161 | | |
151 | 162 | | |
152 | 163 | | |
| |||
176 | 187 | | |
177 | 188 | | |
178 | 189 | | |
| 190 | + | |
179 | 191 | | |
180 | 192 | | |
181 | 193 | | |
| |||
0 commit comments