sql_variant values with mixed inner base types order, group, and compare by SQL Server's two-level family rules.

Author: RyanLamansky

docs/claude/backlog.md

@@ -68,9 +68,7 @@ Low priority / niche — simulatable (as placeholder constants or a small model)
   A named-filegroup registry now ships (`Database.Filegroups`, seeded PRIMARY=1, extended by the bacpac loader's `SqlFilegroup` dispatch; surfaced by `sys.filegroups`/`sys.data_spaces` and FILEGROUP-scoped extended properties) — the scalar helpers above could read it, though a `file_id` 1 placeholder is still needed for the file-level ones.
 - **Certificates** — CERTENCODED / CERTPRIVATEKEY (needs a small certificate-name → bytes model, or NULL placeholders).
 - **Full-text properties** — FULLTEXTCATALOGPROPERTY (the `CREATE FULLTEXT CATALOG` / `INDEX` DDL already ships; this reads its metadata).
-- **`sql_variant` mixed-inner-family ordering** — `ORDER BY` / `GROUP BY` over a variant column with *mixed inner base-type families* (int + nvarchar in one column) isn't modeled: real orders by a datatype-family rank, while the inner `CompareTo` rejects the cross-type pair (surfaces as `InvalidOperationException` from the sort).
-  Same-inner-family ordering, mixed-inner `DISTINCT`, and the single-scalar / comparison / `CAST` flows all work — see [`scalars.md`](scalars.md#sql_variant-expression-semantics), [`catalog-views.md`](catalog-views.md#the-sql_variant-type).
-  Related open quirk: a decimal-declared sequence/column's inner reports BaseType `numeric` rather than real's `decimal` — the single-decimal-family divergence.
+- **`sql_variant` minor quirks** (cross-type family ordering shipped 2026-07-19 — see [`scalars.md`](scalars.md#sql_variant-expression-semantics)): a decimal-declared inner reports BaseType `numeric` rather than real's `decimal` (the single-decimal-family naming divergence); a mixed variant-vs-base-typed comparison unwraps and promotes where real converts the base side up to variant and applies the family rules (a string variant never matches a numeric literal on real — unprobed, retained).
   Deliberate exclusion, don't re-pitch: `msdb.dbo.syspolicy_configuration.current_value` stays `nvarchar` — it's a *view-body* projection (not a resource column) mixing `int` rows with a `binary` GUID row, every consumer reads a single named row and CASTs it, so a variant migration there would only touch the view SQL text for no observable gain.
 - **FILESTREAM** — GET_FILESTREAM_TRANSACTION_CONTEXT (needs a FILESTREAM storage binding; NULL is a faithful "no FILESTREAM context" placeholder).
 

docs/claude/catalog-views.md

@@ -143,18 +143,15 @@ Contracts:
   This is what lets the DSC catalog view — which re-encodes each generator row through `RowEncoder` — carry variant cells.
 - **Coercion** — `CAST(x AS sql_variant)` wraps (`CoerceTo` a variant target); `CAST(variant AS <concrete>)` unwraps the inner and re-coerces.
   `ISNULL(variant, x)` / `COALESCE(variant, …)` stay `sql_variant` (ISNULL fixes to the first arg's type; `Promote(variant, X) → variant`).
-- **Comparison** — a variant operand unwraps to its inner value before the promote-and-compare (`CompareValuesPromoted`), so `value = 0` unwraps a variant int/bit and compares numerically.
-  Identity (Equals / GetHashCode for GROUP BY / DISTINCT) is strict on the inner value, so mixed-inner values are all distinct.
-  **Residual:** `ORDER BY` / `GROUP BY` over a variant column with *mixed inner base-type families* (int and nvarchar in one column) isn't modeled — SQL Server orders by a documented datatype-family rank, but the inner `CompareTo` rejects the cross-type pair (surfaces as an `InvalidOperationException` from the sort).
-  Same-inner-family ordering works; the built-ins realistically feed single-scalar / comparison / `CAST` flows that are unaffected.
+- **Comparison** — a variant operand against a base-typed one unwraps to its inner value before the promote-and-compare (`CompareValuesPromoted`), so `value = 0` unwraps a variant int/bit and compares numerically; a both-variant pair follows the datatype-family rules.
+  Identity and ordering (Equals / GetHashCode / CompareTo for GROUP BY / DISTINCT / ORDER BY) follow SQL Server's two-level family comparison — family rank, then value within the family, with cross-type equal values collapsing into one bucket (`Storage/SqlVariantOrdering.cs`; probed rules in [`scalars.md`](scalars.md#sql_variant-expression-semantics)).
 - **Arithmetic / concatenation** — probe-confirmed against SQL Server 2025: `variant + non-variant` raises **Msg 257** (`Implicit conversion from data type sql_variant to <target> is not allowed. Use the CONVERT function to run this query.`); `variant + variant` and `string + variant` raise **Msg 402** (`… incompatible in the <op> operator`).
   `SqlType.PromoteForArithmetic` is the single source; a runtime guard in `IntegerArithmetic` routes through it so `Run`-time and projection-schema errors agree.
-- **TDS wire** — a `sql_variant` result column emits COLMETADATA type `0x62` (SSVARIANTTYPE) and the MS-TDS 2.2.5.5.3 per-value body (see [`tds-endpoint.md`](tds-endpoint.md)).
-  RPC `sql_variant` *parameters* remain rejected.
+- **TDS wire** — a `sql_variant` result column emits COLMETADATA type `0x62` (SSVARIANTTYPE) and the MS-TDS 2.2.5.5.3 per-value body; RPC `sql_variant` parameters ship in both directions, TVP variant columns included (see [`tds-endpoint.md`](tds-endpoint.md)).
 
 **Consumers surfacing true `sql_variant`:** the `SERVERPROPERTY` / `SESSIONPROPERTY` / `CONNECTIONPROPERTY` / `COLLATIONPROPERTY` / `LOGINPROPERTY` / `SESSION_CONTEXT` / `SQL_VARIANT_PROPERTY` / `DATABASEPROPERTYEX` / `OBJECTPROPERTYEX` scalar family (see [`scalars.md`](scalars.md)), `sys.configurations`.`value*`, `sys.database_scoped_configurations`.`value*`, `sys.sequences` (`start_value` / `increment` / `minimum_value` / `maximum_value` / `current_value` / `last_used_value`, inner = the sequence's declared type), `sys.identity_columns` (`seed_value` / `increment_value` / `last_value`, inner = the identity column's declared type), `sys.parameters.default_value` (always a NULL variant), `sys.partition_range_values.value` (always empty), and the `sys.symmetric_keys` / `sys.asymmetric_keys` sql_variant columns (`key_thumbprint` / `cryptographic_provider_algid`, always empty).
 No inner-type substitution remains.
-The one **semantic residual** is `ORDER BY` / `GROUP BY` over a variant column with *mixed inner base-type families* (see [`scalars.md`](scalars.md#sql_variant-expression-semantics)); a decimal-declared sequence's inner reports BaseType `numeric` rather than real's `decimal` (the single-decimal-family quirk).
+Remaining quirk: a decimal-declared sequence's inner reports BaseType `numeric` rather than real's `decimal` (the single-decimal-family naming divergence).
 - **`sys.filegroups`** is the row-filegroup subset of `sys.data_spaces` (same `Database.Filegroups` enumeration via `EnumerateFilegroupRows` — PRIMARY plus any bacpac-registered filegroup like WWI's `[USERDATA]`) plus `filegroup_guid uniqueidentifier` (NULL), `log_filegroup_id int` (NULL), `is_read_only` / `is_autogrow_all_files bit` (0).
   SMO's CREATE-scripting index/full-text queries read it; DacFx's bacpac export re-emits a standalone `SqlFilegroup` element per non-PRIMARY row.
 - **`sys.periods`** projects one row per table carrying a `PERIOD FOR SYSTEM_TIME` declaration (`HeapTable.PeriodColumns`), **history siblings excluded** (they hold no period of their own even though the simulator copies `PeriodColumns` onto them for the `FOR SYSTEM_TIME` query machinery).

docs/claude/scalars.md

@@ -508,11 +508,18 @@ Probe-confirmed against SQL Server 2025 (2026-07-16):
 - **`ISNULL` / `COALESCE` / `CASE`** keep the `sql_variant` result type (variant has highest data-type precedence in `SqlType.Promote`) and preserve each value's inner type — `ISNULL(SESSIONPROPERTY('ANSI_NULLS'), 0)` stays `sql_variant`.
 - **`UNION [ALL]`** keeps each row's own inner base type — no schema unification/promotion (a variant column can be `int` on one row, `nvarchar` on the next).
 - **Comparison** against a non-variant (`WHERE int_col = SESSION_CONTEXT(N'k')`) unwraps the variant to its inner value.
+  **Both-operands-variant comparison** follows the family rules below (probe-confirmed: variant `bigint 1000 < float 0.5`, variant `int 5 = bigint 5`, variant `nvarchar N'x' = varchar 'x'`).
+  Residual: real converts the base-typed side of a mixed pair *up* to `sql_variant` (highest precedence) and applies the family rules — so a string variant never matches a numeric literal on real, while the unwrap path converts and can match; unprobed, retained.
 - **`SELECT … INTO`** from a variant-producing built-in creates a `sql_variant` column (probe-confirmed).
 - **Arithmetic** rejects: `variant + non-variant` → **Msg 257** (`Implicit conversion from data type sql_variant to <target> is not allowed. Use the CONVERT function to run this query.`); `variant + variant` and `string + variant` → **Msg 402** (`… incompatible in the add operator`).
   `PromoteForArithmetic` is the single source; a runtime guard in `IntegerArithmetic` routes through it so `Run`-time and projection-schema errors agree.
-- **Residual**: `ORDER BY` / `GROUP BY` over a variant column with *mixed inner base-type families* (e.g. `int` and `nvarchar` in the same column) isn't modeled — real orders by a documented type-family rank, the simulator's inner `CompareTo` rejects the cross-type pair (surfaces as an `InvalidOperationException` from the sort).
-  Same-inner-family ordering and mixed-inner `DISTINCT` (equality unwraps per value) both work; single-scalar / `ISNULL` / comparison / `CAST` flows the built-ins realistically feed are unaffected.
+- **Cross-type ordering / grouping** (`Storage/SqlVariantOrdering.cs`, probe-confirmed 2026-07-19): comparison is two-level — datatype-family rank first, then value within the family.
+  Six families, lowest to highest: **1 `uniqueidentifier`; 2 binary** (`binary`/`varbinary`, byte-lexicographic); **3 character** (`char`/`varchar`/`nchar`/`nvarchar` — Unicode and non-Unicode are ONE family); **4 exact numeric** (`bit`, integer types, `decimal`, `money`/`smallmoney`, compared as decimal); **5 approximate** (`real`/`float` — above *every* exact value regardless of magnitude); **6 date/time** (compared as an instant: `time` anchored to 1900-01-01, `datetimeoffset` by UTC instant).
+  Cross-family comparison is value-blind (`float 0.5 > bigint 1000000`); within a family, cross-type values compare by value and equal values are truly equal — `int 5` / `bigint 5` / `decimal 5.00` are one GROUP BY / DISTINCT bucket whose representative is the first value encountered (matching real's plan-order representative), and their ORDER BY tie order is undefined on real (plan-dependent), so tests must not pin it.
+  NULL sorts lowest.
+  `MIN`/`MAX` pick the family-hierarchy extremes.
+  Same-collation character pairs compare under that collation; cross-collation pairs compare by code point **without** a Msg 468 conflict (probed) — the character family hashes by rank alone since no single hash agrees with both regimes.
+  The `SqlValue` variant arms (`CompareTo` / `Equals` / `GetHashCode`) implement the rules, so ORDER BY, GROUP BY, DISTINCT, MIN/MAX, and hash joins all inherit them via `SqlValueKey`.
 
 ## Built-in TVF: `STRING_SPLIT`
 `STRING_SPLIT(input, separator [, enable_ordinal])` dispatches in `ParseSingleFromSource` alongside `OPENJSON` — case-insensitive name match before generic name resolution.

src/SqlServerSimulator/Parser/BooleanExpression.cs

@@ -1233,12 +1233,21 @@ private protected override bool TryAppendFilterDefinition(StringBuilder sb, Batc
     /// </summary>
     internal static bool? CompareValuesPromoted(SqlValue l, SqlValue r, string operatorName, Func<SqlValue, SqlValue, bool> compare)
     {
-        // sql_variant operands compare by their inner value: unwrap here so the
+        // Both operands sql_variant: real compares by datatype-family rank
+        // then value within the family (probe-confirmed — bigint 1000 < float
+        // 0.5, variant int 5 = variant bigint 5), which the SqlValue variant
+        // arms implement; apply the operator to the wrapped pair directly.
+        if (l.Type is SqlVariantSqlType && r.Type is SqlVariantSqlType)
+            return l.IsNull || r.IsNull ? null : compare(l, r);
+
+        // One sql_variant operand against a base-typed one: unwrap so the
         // downstream promote-and-compare runs on the base types (a variant int
         // vs an int literal, a variant bit vs 0, etc.). A NULL variant stays
-        // wrapped — the NULL short-circuit below folds it to UNKNOWN. Full
-        // sql_variant datatype-family comparison rules (where a string variant
-        // never matches a numeric literal) are out of scope; see docs.
+        // wrapped — the NULL short-circuit below folds it to UNKNOWN.
+        // Residual: real converts the base-typed side UP to sql_variant
+        // (highest precedence) and applies the family rules, so a string
+        // variant never matches a numeric literal on real, while the unwrap
+        // path here converts and can match — unprobed, retained.
         if (l.Type is SqlVariantSqlType && !l.IsNull)
             l = l.AsVariantInner;
         if (r.Type is SqlVariantSqlType && !r.IsNull)

src/SqlServerSimulator/Storage/SqlValue.cs

@@ -748,7 +748,7 @@ public bool Equals(SqlValue other) =>
         && this.IsNull == other.IsNull
         && (this.IsNull
             || (this.Type is SqlVariantSqlType
-                ? this.AsVariantInner.Equals(other.AsVariantInner)
+                ? SqlVariantOrdering.Compare(this.AsVariantInner, other.AsVariantInner) == 0
                 : IsStringTypeRef(this.Type)
                 ? this.Type.Collation!.Equals(TrimTrailing((string)this.reference!), TrimTrailing((string)other.reference!))
                 : this.Type is DateTimeOffsetSqlType
@@ -848,11 +848,9 @@ SqlTypeCategory.Integer or SqlTypeCategory.DateTime or SqlTypeCategory.Money
                     // OrdPath's defining property: unsigned bytewise order equals
                     // depth-first tree order, so hierarchyid comparison is a memcmp.
                     HierarchyIdSqlType => this.AsHierarchyIdBytes.AsSpan().SequenceCompareTo(other.AsHierarchyIdBytes),
-                    // sql_variant ordering compares the inner values; differing
-                    // inner base types fall to the inner CompareTo's own
-                    // cross-type rejection (SQL Server's full sql_variant sort
-                    // by type-family rank is out of scope — see docs).
-                    SqlVariantSqlType => this.AsVariantInner.CompareTo(other.AsVariantInner),
+                    // sql_variant orders by datatype-family rank, then value
+                    // within the family (probe-confirmed; see SqlVariantOrdering).
+                    SqlVariantSqlType => SqlVariantOrdering.Compare(this.AsVariantInner, other.AsVariantInner),
                     _ => throw new NotSupportedException($"Comparison for {this.Type} isn't implemented yet."),
                 },
                 _ => throw new NotSupportedException($"Comparison for {this.Type} isn't implemented yet."),
@@ -870,8 +868,9 @@ public override int GetHashCode()
 
         if (this.Type is SqlVariantSqlType)
         {
-            // Identity is the inner value alone; equality unwraps to it.
-            hash.Add(this.AsVariantInner.GetHashCode());
+            // Identity is the family-canonical inner value, agreeing with the
+            // family-rank equality (int 5 / bigint 5 / decimal 5.00 hash alike).
+            hash.Add(SqlVariantOrdering.InnerHashCode(this.AsVariantInner));
         }
         else if (IsStringTypeRef(this.Type))
         {