fix(list,detail): sorting a lookup column stops ordering by an invisible key (#3096) (#3102)

A relational column (lookup / master_detail / user / tree) never holds the
string its cell shows: it holds the $expand-ed record, or a raw foreign-key id
whose label was resolved separately. Every sort path took that raw value as its
key, so a column of names came back in an order with no relation to the names —
sorting looked broken, and nothing said the key was something else.

The two halves can order by different things, so they are fixed differently.

Client-side sorts (grid column headers, any data-table, a non-windowed related
list) now key off the label the cell renders, via getSortValue /
compareSortValues in @object-ui/core — which resolves an expanded record
through getRecordDisplayName (ADR-0079), so the sort key and the lookup cell
agree on which field names a record. That replaces two broken comparators:
`a[col] < b[col]` is always false between two objects (the comparator collapsed
to a constant and permuted the rows), and `String(a[col])` is "[object Object]"
(every row compared equal, so the sort silently did nothing).

Server $orderby sorts cannot be fixed here — the key is the stored id by
construction, and objectstack#4256 settled that no relation join is coming. So
those entry points stop offering the illusion: the ListView toolbar sort picker
withholds relational fields and explains why (pointing at a formula field as the
supported way to sort by a related name), and a windowed related list renders no
sort button for them. A relational field the view's CURRENT sort already uses
stays listed, labelled "(by ID)", so view metadata authored or saved with such a
sort round-trips instead of rendering a blank row and losing the sort on the
next edit.

Each regression test was run against the unfixed code first: the data-table one
showed the constant comparator leaving rows untouched, the RelatedList one
showed the old raw-id order, the ListView one showed all four relational fields
in the picker. Verified live in the console against a CRM backend: crm_lead's
two lookups are gone from the sort picker with the hint rendered beneath it, and
the contacts grid's account column now sorts Acme/Globex/Initech ↔ reverse by
the shown company name.

Co-authored-by: Jack Zhuang <277994282+os-zhuang@users.noreply.github.com>
Co-authored-by: Claude Opus 5 <noreply@anthropic.com>

Author: os-zhuang

.changeset/relational-column-sort-honesty.md

@@ -0,0 +1,37 @@
+---
+"@object-ui/core": patch
+"@object-ui/components": patch
+"@object-ui/plugin-list": patch
+"@object-ui/plugin-detail": patch
+"@object-ui/i18n": patch
+---
+
+fix(list,detail): sorting a lookup column no longer orders by an invisible key — #3096
+
+A relational column (`lookup` / `master_detail` / `user` / `tree`) never holds
+the string its cell shows: it holds the `$expand`-ed record, or a raw foreign-key
+id whose label was resolved separately. Every sort path took that raw value as
+its key, so the column of names came back in an order with no relation to the
+names — sorting looked broken, with nothing saying the key was something else.
+
+The two halves are fixed differently, because they can order by different things:
+
+- **Client-side sorts** (grid column headers, any `data-table`, a non-windowed
+  related list) now key off the label the cell renders, via the new
+  `getSortValue` / `compareSortValues` in `@object-ui/core` — which resolves an
+  expanded record through `getRecordDisplayName` (ADR-0079), so the sort key and
+  the lookup cell agree on which field names a record. This replaces two broken
+  comparators: `a[col] < b[col]` is always false between two objects (the
+  comparator collapsed to a constant and permuted the rows), and
+  `String(a[col])` is `"[object Object]"` (every row compared equal, so the sort
+  silently did nothing).
+- **Server `$orderby` sorts** cannot be fixed here — the key is the stored id by
+  construction, and `objectstack#4256` settled that no relation join is coming.
+  So those entry points stop offering the illusion: the ListView toolbar sort
+  picker withholds relational fields and explains why (pointing at a formula
+  field as the supported way to sort by a related name), and a windowed related
+  list renders no sort button for them.
+
+A relational field the view's CURRENT sort already uses stays listed, labelled
+`(by ID)`, so view metadata authored or saved with such a sort round-trips
+instead of rendering a blank row and losing the sort on the next edit.

packages/components/src/renderers/complex/__tests__/data-table-relational-sort.test.tsx

@@ -0,0 +1,101 @@
+/**
+ * ObjectUI
+ * Copyright (c) 2024-present ObjectStack Inc.
+ *
+ * This source code is licensed under the MIT license found in the
+ * LICENSE file in the root directory of this source tree.
+ */
+
+/**
+ * Regression (objectui#3096): clicking a lookup column header sorted by the
+ * `$expand`-ed RECORD OBJECT, not by the name in the cell.
+ *
+ * `aValue < bValue` is always false for two objects, so the comparator returned
+ * the constant `1` for every pair — the rows moved (the arrow turned, the order
+ * changed) but the resulting order had no relation to the names on screen. The
+ * header sort now keys off the same display name the cell renders.
+ *
+ * The fixture is chosen so the three candidate orders are all distinct: the
+ * incoming order and the foreign-key-id order are Mallory/Zoe/Alice, while the
+ * displayed-name order is Alice/Mallory/Zoe. Only a sort that reads the label
+ * can produce the assertions below.
+ */
+import { describe, it, expect, beforeAll } from 'vitest';
+import { render, screen, fireEvent, within } from '@testing-library/react';
+import '@testing-library/jest-dom';
+import React from 'react';
+import { ComponentRegistry } from '@object-ui/core';
+import '../data-table';
+
+const rows = [
+  { id: '1', code: 'C-1', owner: { id: 'rec_1a', name: 'Mallory' } },
+  { id: '2', code: 'C-2', owner: { id: 'rec_4m', name: 'Zoe' } },
+  { id: '3', code: 'C-3', owner: { id: 'rec_9z', name: 'Alice' } },
+];
+
+const schema = {
+  data: rows,
+  pagination: false,
+  searchable: false,
+  columns: [
+    { header: 'Code', accessorKey: 'code' },
+    {
+      header: 'Owner',
+      accessorKey: 'owner',
+      // Mirrors the lookup cell renderer: the cell shows the related record's
+      // name while the row value stays the expanded object.
+      cell: (value: any) => <span>{value?.name ?? '—'}</span>,
+    },
+  ],
+};
+
+function renderTable() {
+  const DataTable = ComponentRegistry.get('data-table') as any;
+  if (!DataTable) throw new Error('data-table not registered');
+  return render(<DataTable schema={schema} />);
+}
+
+/** Owner names in DOM row order. */
+function ownerColumn(): string[] {
+  return screen
+    .getAllByRole('row')
+    .slice(1) // drop the header row
+    .map((row) => within(row).getAllByRole('cell')[1]?.textContent?.trim() ?? '');
+}
+
+describe('data-table sorting a relational column', () => {
+  beforeAll(() => {
+    expect(ComponentRegistry.has('data-table')).toBe(true);
+  });
+
+  it('orders by the displayed name, ascending then descending', () => {
+    renderTable();
+    // Unsorted — and this is also the order the raw foreign-key ids produce,
+    // so neither "no sort" nor "sorted by id" can satisfy the next assertion.
+    expect(ownerColumn()).toEqual(['Mallory', 'Zoe', 'Alice']);
+    expect([...rows].sort((a, b) => a.owner.id.localeCompare(b.owner.id)).map((r) => r.owner.name))
+      .toEqual(['Mallory', 'Zoe', 'Alice']);
+
+    fireEvent.click(screen.getByText('Owner'));
+    expect(ownerColumn()).toEqual(['Alice', 'Mallory', 'Zoe']);
+
+    fireEvent.click(screen.getByText('Owner'));
+    expect(ownerColumn()).toEqual(['Zoe', 'Mallory', 'Alice']);
+  });
+
+  it('leaves a third click unsorted, back to the incoming row order', () => {
+    renderTable();
+    fireEvent.click(screen.getByText('Owner'));
+    fireEvent.click(screen.getByText('Owner'));
+    fireEvent.click(screen.getByText('Owner'));
+    expect(ownerColumn()).toEqual(['Mallory', 'Zoe', 'Alice']);
+  });
+
+  it('still sorts a plain text column', () => {
+    renderTable();
+    fireEvent.click(screen.getByText('Code'));
+    expect(ownerColumn()).toEqual(['Mallory', 'Zoe', 'Alice']);
+    fireEvent.click(screen.getByText('Code'));
+    expect(ownerColumn()).toEqual(['Alice', 'Zoe', 'Mallory']);
+  });
+});

packages/components/src/renderers/complex/data-table.tsx

@@ -11,7 +11,7 @@ import React, { useState, useMemo, useRef, useEffect, useLayoutEffect } from 're
 import { cn } from '../../lib/utils';
 import { resolveIcon } from '../action/resolve-icon';
 import { useGridFieldAuthoring } from '../../context/gridFieldAuthoring';
-import { ComponentRegistry } from '@object-ui/core';
+import { ComponentRegistry, compareSortValues, getSortValue } from '@object-ui/core';
 import type { DataTableSchema } from '@object-ui/types';
 import { useRowPredicate } from '@object-ui/react';
 import { createSafeTranslation } from '@object-ui/i18n';
@@ -599,19 +599,26 @@ const DataTableRenderer = ({ schema }: { schema: DataTableSchema }) => {
     );
   }, [data, searchQuery, columns]);
 
-  // Sorting
+  // Sorting — client-side, over the rows this table was handed.
+  //
+  // Sort keys go through `getSortValue` so a relational column orders by the
+  // label its cell shows, not by the `$expand`-ed record object (objectui#3096).
+  // The old `aValue < bValue` was always false for two objects, so a lookup
+  // column's comparator collapsed to the constant `1` and the sort produced an
+  // order unrelated to anything on screen.
+  //
+  // Decorate → sort → undecorate: the key is resolved ONCE per row rather than
+  // on every one of the O(n log n) comparisons.
   const sortedData = useMemo(() => {
     if (!sortColumn || !sortDirection) return filteredData;
-    
-    return [...filteredData].sort((a, b) => {
-      const aValue = a[sortColumn];
-      const bValue = b[sortColumn];
-      
-      if (aValue === bValue) return 0;
-      
-      const comparison = aValue < bValue ? -1 : 1;
+
+    const keyed = filteredData.map((row) => ({ row, key: getSortValue(row[sortColumn]) }));
+    // Array#sort is stable, so rows with equal keys keep their incoming order.
+    keyed.sort((a, b) => {
+      const comparison = compareSortValues(a.key, b.key);
       return sortDirection === 'asc' ? comparison : -comparison;
     });
+    return keyed.map((entry) => entry.row);
   }, [filteredData, sortColumn, sortDirection]);
 
   // Pagination. Under manual (server-side) pagination the parent controls the

packages/core/src/index.ts

@@ -18,6 +18,7 @@ export * from './utils/filter-converter.js';
 export * from './utils/managedBy.js';
 export * from './utils/extract-records.js';
 export * from './utils/expand-fields.js';
+export * from './utils/sort-values.js';
 export * from './evaluator/index.js';
 export * from './actions/index.js';
 export * from './query/index.js';

packages/core/src/utils/__tests__/sort-values.test.ts

@@ -0,0 +1,124 @@
+/**
+ * ObjectUI
+ * Copyright (c) 2024-present ObjectStack Inc.
+ *
+ * This source code is licensed under the MIT license found in the
+ * LICENSE file in the root directory of this source tree.
+ */
+
+/**
+ * objectui#3096 — a client-side sort must order by the string the cell shows.
+ *
+ * The comparators these helpers replace failed in two distinct ways on a
+ * relational column: `a < b` between two `$expand`-ed record objects is always
+ * false (so the comparator returned a constant and the rows were permuted into
+ * a meaningless order), and `String(record)` is `"[object Object]"` (so every
+ * row compared equal and the sort silently did nothing). Both are pinned below.
+ */
+import { describe, it, expect } from 'vitest';
+import { compareCellValues, compareSortValues, getSortValue } from '../sort-values.js';
+
+const sortBy = <T,>(rows: T[], read: (row: T) => unknown, labels?: Record<string, string>): T[] =>
+  [...rows].sort((a, b) => compareCellValues(read(a), read(b), { labels }));
+
+describe('getSortValue', () => {
+  it('reduces an expanded reference record to its display name', () => {
+    expect(getSortValue({ id: 'rec_7f3', name: 'Alice' })).toBe('Alice');
+    expect(getSortValue({ id: 'rec_001', full_name: 'Bob Lin' })).toBe('Bob Lin');
+    expect(getSortValue({ id: 'rec_002', title: 'Q3 Renewal' })).toBe('Q3 Renewal');
+  });
+
+  it('maps a raw foreign-key id through the client-resolved label map', () => {
+    const labels = { u1: 'Carol', u2: 'Alice' };
+    expect(getSortValue('u1', { labels })).toBe('Carol');
+    expect(getSortValue(7, { labels: { '7': 'Dave' } })).toBe('Dave');
+    // An expanded record still honours a label resolved for the same id.
+    expect(getSortValue({ id: 'u2', name: 'stale' }, { labels })).toBe('Alice');
+    // No entry → the raw value, never a crash.
+    expect(getSortValue('u9', { labels })).toBe('u9');
+  });
+
+  it('keeps primitives in their own domain so numbers stay numeric', () => {
+    expect(getSortValue(42)).toBe(42);
+    expect(getSortValue(true)).toBe(true);
+    expect(getSortValue('Alice')).toBe('Alice');
+    expect(getSortValue(new Date('2024-03-01T00:00:00Z'))).toBe(Date.parse('2024-03-01T00:00:00Z'));
+  });
+
+  it('treats every flavour of empty as one blank value', () => {
+    expect(getSortValue(null)).toBeNull();
+    expect(getSortValue(undefined)).toBeNull();
+    expect(getSortValue('')).toBeNull();
+    expect(getSortValue(Number.NaN)).toBeNull();
+    // A record with nothing name-like must NOT become the literal "Untitled" —
+    // that would bunch unrelated empty records together under "U".
+    expect(getSortValue({})).toBeNull();
+  });
+
+  it('joins multi-value cells the way they read', () => {
+    expect(getSortValue([{ id: 'a', name: 'Alice' }, { id: 'b', name: 'Bob' }])).toBe('Alice, Bob');
+    expect(getSortValue([])).toBeNull();
+  });
+});
+
+describe('compareSortValues', () => {
+  it('sorts blanks last ascending (so callers that negate get them first)', () => {
+    expect(compareSortValues(null, 'Alice')).toBe(1);
+    expect(compareSortValues('Alice', null)).toBe(-1);
+    expect(compareSortValues(null, null)).toBe(0);
+  });
+
+  it('compares numbers numerically, not lexicographically', () => {
+    expect(compareSortValues(9, 10)).toBeLessThan(0);
+    // `numeric: true` extends that to numbers embedded in strings.
+    expect(compareSortValues('Item 9', 'Item 10')).toBeLessThan(0);
+  });
+
+  it('orders booleans false before true', () => {
+    expect(compareSortValues(false, true)).toBeLessThan(0);
+    expect(compareSortValues(true, true)).toBe(0);
+  });
+});
+
+describe('sorting a relational column (the objectui#3096 regression)', () => {
+  const rows = [
+    { id: '1', owner: { id: 'rec_ccc', name: 'Carol' } },
+    { id: '2', owner: { id: 'rec_aaa', name: 'Alice' } },
+    { id: '3', owner: { id: 'rec_bbb', name: 'Bob' } },
+  ];
+
+  it('orders expanded lookup cells by the label', () => {
+    expect(sortBy(rows, (r) => r.owner).map((r) => r.owner.name)).toEqual(['Alice', 'Bob', 'Carol']);
+  });
+
+  it('disagrees with id order whenever the ids are opaque', () => {
+    const opaque = [
+      { owner: { id: 'rec_9z', name: 'Alice' } },
+      { owner: { id: 'rec_1a', name: 'Carol' } },
+      { owner: { id: 'rec_4m', name: 'Bob' } },
+    ];
+    expect(sortBy(opaque, (r) => r.owner).map((r) => r.owner.name)).toEqual([
+      'Alice',
+      'Bob',
+      'Carol',
+    ]);
+    // The old server-shaped key (bare id) produces a different, meaningless order.
+    expect([...opaque].sort((a, b) => a.owner.id.localeCompare(b.owner.id)).map((r) => r.owner.name))
+      .toEqual(['Carol', 'Bob', 'Alice']);
+  });
+
+  it('does not collapse to a no-op the way String(record) did', () => {
+    // Every record stringifies to "[object Object]" — the old RelatedList
+    // comparator therefore returned 0 for every pair and left rows untouched.
+    const stringified = rows.map((r) => String(r.owner));
+    expect(new Set(stringified).size).toBe(1);
+    expect(sortBy(rows, (r) => r.owner).map((r) => r.id)).not.toEqual(rows.map((r) => r.id));
+  });
+
+  it('orders raw-id cells by the client-resolved labels', () => {
+    const idRows = [{ owner: 'u3' }, { owner: 'u1' }, { owner: 'u2' }];
+    const labels = { u1: 'Zoe', u2: 'Alice', u3: 'Mallory' };
+    expect(sortBy(idRows, (r) => r.owner, labels).map((r) => labels[r.owner as keyof typeof labels]))
+      .toEqual(['Alice', 'Mallory', 'Zoe']);
+  });
+});