Fix multi-column column index rebuild ordering

[saulshanabrook]

Summary

Fixes a debug-only panic in ColumnIndex::rebuild_full for multi-column indexes after a major table-version rebuild.

The regression comes from #862, specifically commit 70677c590534ba06ecebdaa9f16ac3bb8b1c8a0c (cleanup). That commit switched multi-column ColumnIndex::rebuild_full to the value-only radix sort path. Multi-column collection is column-major, so rows for the same value can be ordered like 95..126, 0..30, 127; BufferedSubset::Sparse then creates a SortedOffsetSlice from unsorted row IDs.

Why this stayed hidden

The invariant violation is caught by a debug_assert! in core-relations/src/offsets/mod.rs:136. The egglog-experimental CI job runs make test, which uses cargo nextest run --release, so that debug assertion is disabled in CI. In release mode, the math_backoff test passes even though the sorted-offset invariant is violated.

This was investigated while updating egraphs-good/egglog-experimental#46, but it is not caused by that PR. A clean egglog-experimental origin/main checkout at 1530719693f56c1186ab867629140c0db185783a, pinned to egg-smol codex/split-scheduler-can-stop-report#ebba7bb902bdc1b0f377b6bb22c06ac305912674, already reproduces the same debug-mode math_backoff panic in core-relations/src/offsets/mod.rs:136.

So the timeline is:

1. Claude experiment day 4 #862 introduced the multi-column ordering bug.
2. egglog-experimental picked up an egg-smol commit containing that bug.
3. egglog-experimental CI stayed green because it runs release tests.
4. Local debug-mode validation for Add primitive command support egglog-experimental#46 exposed the latent invariant violation.

Changes

• For multi-column ColumnIndex::rebuild_full, sort each equal-Value group by RowId before deduplicating.
• Add a regression test that forces a major-version rebuild and checks a multi-column column index returns sorted row IDs.

Validation

• Confirmed the new test fails on unmodified current main with slice is not sorted.
• Confirmed egglog-experimental origin/main fails cargo test -q math_backoff --test files -- --exact in debug mode with the same upstream panic.
• Confirmed egglog-experimental origin/main passes cargo test --release -q math_backoff --test files -- --exact, matching CI's release-mode behavior.
• cargo fmt --check
• cargo check -q -p egglog-core-relations
• cargo test -q -p egglog-core-relations --lib multi_column_column_index_rebuild_orders_each_value_by_row
• cargo test -q -p egglog-core-relations --lib

[codspeed-hq]

Merging this PR will improve performance by 27.43%

⚠️ Different runtime environments detected

Some benchmarks with significant performance changes were compared across different runtime environments,
which may affect the accuracy of the results.

Open the report in CodSpeed to investigate

⚡ 2 improved benchmarks
❌ 1 regressed benchmark
✅ 22 untouched benchmarks
🆕 1 new benchmark
⏩ 215 skipped benchmarks¹

Warning

Please fix the performance issues or acknowledge them on CodSpeed.

Performance Changes

	Mode	Benchmark	BASE	HEAD	Efficiency
⚡	Simulation	rust_rule_fib[rule_run_1000]	280.7 ms	202.8 ms	+38.39%
🆕	Simulation	tests[repro-herbie-vanilla]	N/A	1.4 s	N/A
⚡	Simulation	tests[cykjson]	440.2 ms	270.9 ms	+62.51%
❌	Simulation	tests[python_array_optimize]	1.5 s	1.7 s	-7.99%

Tip

Investigate this regression by commenting @codspeedbot fix this regression on this PR, or directly use the CodSpeed MCP with your agent.

---

_{Comparing saulshanabrook:codex/core-relations-index-order-fix (5740bec) with main (8c1c70b)²}

Footnotes

1. 215 benchmarks were skipped, so the baseline results were used instead. If they were deleted from the codebase, click here and archive them to remove them from the performance reports. ↩

2. No successful run was found on main (2e1796e) during the generation of this report, so 8c1c70b was used instead as the comparison base. There might be some changes unrelated to this pull request in this report. ↩

[codecov-commenter]

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 87.16%. Comparing base (2e1796e) to head (5740bec).

Additional details and impacted files

@@            Coverage Diff             @@
##             main     #899      +/-   ##
==========================================
+ Coverage   87.12%   87.16%   +0.03%     
==========================================
  Files          88       88              
  Lines       25904    25905       +1     
==========================================
+ Hits        22568    22579      +11     
+ Misses       3336     3326      -10     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
• 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

[saulshanabrook]

Closing in favor of #914