Skip to content

Pin per-system dolt_schema_diff shapes for schema objects#1668

Merged
timsehn merged 2 commits into
masterfrom
test/schema-diff-oracle-schema-objects
Jul 17, 2026
Merged

Pin per-system dolt_schema_diff shapes for schema objects#1668
timsehn merged 2 commits into
masterfrom
test/schema-diff-oracle-schema-objects

Conversation

@timsehn

@timsehn timsehn commented Jul 17, 2026

Copy link
Copy Markdown
Collaborator

Final piece of the schema-objects × VC oracle campaign (#1657 add, #1663 merge, #1665 checkout, #1666 revert/cherry-pick, #1667 remotes).

The surface divergence, pinned rather than papered over

dolt_schema_diff is the one VC surface where doltlite and Dolt intentionally disagree on shape:

change doltlite Dolt
index add/drop/modify own row named idx folded into t → t
view add / last drop own row named v dolt_schemas table created/dropped
view modify v → v no row at all (dolt_schemas schema unchanged)
trigger add (schemas table exists) own row named trg no row at all

doltlite's per-object rows follow the SQLite surface (indexes/views/triggers are first-class schema objects) and are strictly more informative for view/trigger edits. Cross-comparing outputs is meaningless, so the new scenarios pin each system to its own expected shape — if either doltlite regresses or upstream Dolt changes behavior, the suite flags it for a deliberate decision.

Coverage added (10 scenarios)

Index add/drop/modify, view add/drop/modify, trigger add alongside an existing view, a mixed commit (table ALTER + index + view in one commit), and doltlite's 3-arg filter semantics: filtering by object name returns that object's row; filtering by the owning table returns only the table's own DDL changes (the index is its own object — flagging this judgment call for review, since a user filtering by table might expect its indexes included).

No engine changes

Suite total: 59/59.

🤖 Generated with Claude Code

The two systems intentionally present schema objects differently:
doltlite emits one row per SQLite schema object while Dolt folds index
changes into the owning table's row and stores views and triggers as
dolt_schemas rows, whose own schema only changes on creation or drop —
a view modification produces no Dolt schema_diff row at all. Cross-
comparing is meaningless, so ten scenarios pin each system to its own
expected shape (index add/drop/modify, view add/drop/modify, trigger
add, a mixed commit, and doltlite's object-name filter semantics); a
change on either side surfaces as a failure.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@github-actions

github-actions Bot commented Jul 17, 2026

Copy link
Copy Markdown

Version-Control Performance Ceilings

Runs: median of 3 executions per benchmark, excluding fixture setup. The
working-set fixtures use 800 committed tables with 125 rows each
(100000 rows total); data-dirty cases update 1 row in 40
tables, and schema-dirty cases update 1 row plus add a column in 20 tables.
Branch tests use 300 branches, checkout uses a clean 400-table
branch switch over 100000 rows, and merge
tests use 100000-row tables with 2000 changed or conflicting
rows per side.

Benchmark Median ms Ceiling ms Result
status_clean_many_tables 91 200 PASS
status_dirty_many_tables 95 200 PASS
diff_regular_working_one_table 87 150 PASS
diff_regular_working_many_tables 99 200 PASS
diff_stat_working_many_tables 99 200 PASS
diff_schema_working_many_tables 100 200 PASS
branch_list_many_branches 25 100 PASS
branch_create_delete 27 100 PASS
checkout_branch_clean 60 200 PASS
merge_data_no_conflicts 32 150 PASS
merge_schema_no_conflicts 24 100 PASS
merge_data_conflicts 130 250 PASS
merge_data_conflicts_with_resolve 129 250 PASS

@github-actions

github-actions Bot commented Jul 17, 2026

Copy link
Copy Markdown

Sysbench-Style Benchmark (TEXT PK): Doltlite vs SQLite

Companion to the classic Sysbench-Style Benchmark. Every workload here
runs against tables with a 32-char hex TEXT PRIMARY KEY (UUID-shaped).

Individual ratios gated at 2.5×; section averages gated at 2×. Autocommit writes use 6× / 5×.

In-Memory

Reads

Test SQLite (us) Doltlite (us) Multiplier
oltp_point_select 30,087 35,936 1.19
oltp_range_select 14,432 13,975 0.97
oltp_sum_range 12,657 13,864 1.10
oltp_order_range 3,261 3,317 1.02
oltp_distinct_range 4,279 4,377 1.02
oltp_index_scan 4,749 6,214 1.31
select_random_points 18,302 20,941 1.14
select_random_ranges 4,298 5,402 1.26
covering_index_scan 4,820 4,573 0.95
groupby_scan 34,317 37,695 1.10
index_join 6,906 9,164 1.33
index_join_scan 4,727 5,771 1.22
types_table_scan 1,179,043 1,263,497 1.07
table_scan 1,404,219 1,378,409 0.98
oltp_read_only 123,145 134,512 1.09
Average 1.12

Writes

Test SQLite (us) Doltlite (us) Multiplier
oltp_bulk_insert 229,402 352,384 1.54
oltp_insert 22,325 41,005 1.84
oltp_update_index 76,752 143,882 1.87
oltp_update_non_index 52,295 93,897 1.80
oltp_delete_insert 56,792 113,726 2.00
oltp_write_only 30,961 66,347 2.14
types_delete_insert 34,898 58,625 1.68
oltp_read_write 96,034 146,027 1.52
Average 1.80

File-Backed

Reads

Test SQLite (us) Doltlite (us) Multiplier
oltp_point_select 116,388 67,167 0.58
oltp_range_select 25,213 17,362 0.69
oltp_sum_range 22,430 17,311 0.77
oltp_order_range 4,225 3,655 0.87
oltp_distinct_range 5,290 4,809 0.91
oltp_index_scan 13,544 9,516 0.70
select_random_points 28,139 25,174 0.89
select_random_ranges 12,802 8,476 0.66
covering_index_scan 14,430 7,994 0.55
groupby_scan 36,367 37,866 1.04
index_join 12,487 11,819 0.95
index_join_scan 5,948 6,431 1.08
types_table_scan 1,284,419 1,392,610 1.08
table_scan 1,789,857 1,471,273 0.82
oltp_read_only 257,520 182,193 0.71
Average 0.82

Writes

Test SQLite (us) Doltlite (us) Multiplier
oltp_bulk_insert 251,892 388,575 1.54
oltp_insert 50,237 54,408 1.08
oltp_update_index 125,274 189,103 1.51
oltp_update_non_index 117,457 121,991 1.04
oltp_delete_insert 95,679 146,218 1.53
oltp_write_only 81,976 92,646 1.13
types_delete_insert 59,239 78,607 1.33
oltp_read_write 145,365 167,805 1.15
Average 1.29

File-Backed (autocommit)

Each statement runs as its own transaction — exposes per-commit
fixed costs that the wrapped-in-BEGIN/COMMIT tests amortize away.
SQLite uses WAL mode with synchronous=FULL in this section so
the comparison uses SQLite's durable WAL autocommit path.

Reads

Reads have no commit cost; these are the same SQL files as the
File-Backed Reads section, included here for symmetry and to
catch any per-statement overhead doltlite pays on the read path.

Test SQLite (us) Doltlite (us) Multiplier
oltp_point_select 59,906 66,352 1.11
oltp_range_select 19,095 17,358 0.91
oltp_sum_range 17,147 17,446 1.02
oltp_order_range 3,837 3,684 0.96
oltp_distinct_range 4,774 4,875 1.02
oltp_index_scan 8,117 9,638 1.19
select_random_points 22,429 24,687 1.10
select_random_ranges 7,192 8,385 1.17
covering_index_scan 9,199 7,827 0.85
groupby_scan 35,528 37,851 1.07
index_join 9,742 11,607 1.19
index_join_scan 5,487 6,711 1.22
types_table_scan 1,371,568 1,383,376 1.01
table_scan 1,791,106 1,413,918 0.79
oltp_read_only 169,979 180,232 1.06
Average 1.04

Writes

Test SQLite (us) Doltlite (us) Multiplier
oltp_bulk_insert_ac 15,762 60,461 3.84
oltp_insert_ac 18,507 75,135 4.06
oltp_update_index_ac 20,984 93,336 4.45
oltp_update_non_index_ac 16,222 73,261 4.52
oltp_delete_insert_ac 18,689 85,480 4.57
oltp_write_only_ac 18,564 84,043 4.53
types_delete_insert_ac 16,311 72,768 4.46
oltp_read_write_ac 26,107 92,528 3.54
Average 4.25

100000 rows, median of 5 invocations per test; autocommit writes use 9, workload-only timing via host monotonic clock when available.

Performance Ceiling Check (2.5x individual, 2x average; autocommit writes: 6x / 5x)

All tests within ceilings.

@github-actions

github-actions Bot commented Jul 17, 2026

Copy link
Copy Markdown

Sysbench-Style Benchmark (BLOB PK): Doltlite vs SQLite

Companion to the classic Sysbench-Style Benchmark. Every workload here
runs against tables with a 16-byte big-endian BLOB PRIMARY KEY.

Individual ratios gated at 2.5×; section averages gated at 2×. Autocommit writes use 6× / 5×.

In-Memory

Reads

Test SQLite (us) Doltlite (us) Multiplier
oltp_point_select 30,514 35,049 1.15
oltp_range_select 13,098 13,723 1.05
oltp_sum_range 11,739 13,283 1.13
oltp_order_range 3,039 3,194 1.05
oltp_distinct_range 4,074 4,343 1.07
oltp_index_scan 4,460 5,840 1.31
select_random_points 17,345 19,735 1.14
select_random_ranges 4,201 5,210 1.24
covering_index_scan 4,492 4,402 0.98
groupby_scan 33,625 36,794 1.09
index_join 6,733 8,735 1.30
index_join_scan 4,171 5,556 1.33
types_table_scan 1,130,556 1,256,256 1.11
table_scan 1,309,101 1,358,371 1.04
oltp_read_only 117,359 132,790 1.13
Average 1.14

Writes

Test SQLite (us) Doltlite (us) Multiplier
oltp_bulk_insert 240,115 343,752 1.43
oltp_insert 20,197 39,341 1.95
oltp_update_index 69,467 129,779 1.87
oltp_update_non_index 49,810 85,003 1.71
oltp_delete_insert 49,606 103,516 2.09
oltp_write_only 28,223 63,166 2.24
types_delete_insert 32,367 53,284 1.65
oltp_read_write 79,443 134,186 1.69
Average 1.83

File-Backed

Reads

Test SQLite (us) Doltlite (us) Multiplier
oltp_point_select 112,501 62,843 0.56
oltp_range_select 21,372 16,513 0.77
oltp_sum_range 19,620 16,163 0.82
oltp_order_range 3,816 3,563 0.93
oltp_distinct_range 4,793 4,750 0.99
oltp_index_scan 12,759 8,892 0.70
select_random_points 26,086 22,812 0.87
select_random_ranges 12,473 8,206 0.66
covering_index_scan 12,396 7,728 0.62
groupby_scan 34,405 37,285 1.08
index_join 11,085 11,227 1.01
index_join_scan 5,170 6,020 1.16
types_table_scan 1,127,948 1,252,154 1.11
table_scan 1,309,773 1,363,103 1.04
oltp_read_only 237,494 171,620 0.72
Average 0.87

Writes

Test SQLite (us) Doltlite (us) Multiplier
oltp_bulk_insert 255,903 366,093 1.43
oltp_insert 30,154 52,240 1.73
oltp_update_index 101,516 161,980 1.60
oltp_update_non_index 78,373 108,583 1.39
oltp_delete_insert 80,210 132,340 1.65
oltp_write_only 51,431 86,061 1.67
types_delete_insert 50,222 71,526 1.42
oltp_read_write 107,913 155,609 1.44
Average 1.54

File-Backed (autocommit)

Each statement runs as its own transaction — exposes per-commit
fixed costs that the wrapped-in-BEGIN/COMMIT tests amortize away.
SQLite uses WAL mode with synchronous=FULL in this section so
the comparison uses SQLite's durable WAL autocommit path.

Reads

Reads have no commit cost; these are the same SQL files as the
File-Backed Reads section, included here for symmetry and to
catch any per-statement overhead doltlite pays on the read path.

Test SQLite (us) Doltlite (us) Multiplier
oltp_point_select 57,114 62,717 1.10
oltp_range_select 16,349 16,588 1.01
oltp_sum_range 14,976 16,149 1.08
oltp_order_range 3,524 3,572 1.01
oltp_distinct_range 4,407 4,761 1.08
oltp_index_scan 7,611 9,001 1.18
select_random_points 21,021 22,777 1.08
select_random_ranges 6,943 8,182 1.18
covering_index_scan 8,052 7,574 0.94
groupby_scan 34,032 37,190 1.09
index_join 8,692 11,130 1.28
index_join_scan 4,760 5,962 1.25
types_table_scan 1,130,584 1,255,791 1.11
table_scan 1,308,489 1,359,676 1.04
oltp_read_only 156,086 172,763 1.11
Average 1.10

Writes

Test SQLite (us) Doltlite (us) Multiplier
oltp_bulk_insert_ac 15,668 63,056 4.02
oltp_insert_ac 18,040 84,174 4.67
oltp_update_index_ac 21,583 97,925 4.54
oltp_update_non_index_ac 16,551 76,695 4.63
oltp_delete_insert_ac 17,699 85,034 4.80
oltp_write_only_ac 17,949 85,423 4.76
types_delete_insert_ac 15,327 76,119 4.97
oltp_read_write_ac 24,161 94,388 3.91
Average 4.54

100000 rows, median of 5 invocations per test; autocommit writes use 9, workload-only timing via host monotonic clock when available.

Performance Ceiling Check (2.5x individual, 2x average; autocommit writes: 6x / 5x)

All tests within ceilings.

@github-actions

github-actions Bot commented Jul 17, 2026

Copy link
Copy Markdown

Sysbench-Style Benchmark (composite PK): Doltlite vs SQLite

Companion to the classic Sysbench-Style Benchmark. Every workload here
runs against tables with a 2-column INTEGER PRIMARY KEY(a, b) WITHOUT ROWID.

Individual ratios gated at 2.5×; section averages gated at 2×. Autocommit writes use 6× / 5×.

In-Memory

Reads

Test SQLite (us) Doltlite (us) Multiplier
oltp_point_select 35,119 40,568 1.16
oltp_range_select 19,759 22,223 1.12
oltp_sum_range 18,580 21,339 1.15
oltp_order_range 3,551 3,937 1.11
oltp_distinct_range 4,653 5,231 1.12
oltp_index_scan 4,712 6,433 1.37
select_random_points 28,881 32,510 1.13
select_random_ranges 7,946 9,054 1.14
covering_index_scan 4,223 4,253 1.01
groupby_scan 35,898 41,117 1.15
index_join 8,252 10,896 1.32
index_join_scan 4,285 5,453 1.27
types_table_scan 1,108,138 1,248,543 1.13
table_scan 1,281,523 1,347,266 1.05
oltp_read_only 156,449 175,374 1.12
Average 1.16

Writes

Test SQLite (us) Doltlite (us) Multiplier
oltp_bulk_insert 247,896 364,474 1.47
oltp_insert 19,120 36,864 1.93
oltp_update_index 68,462 118,757 1.73
oltp_update_non_index 51,423 85,558 1.66
oltp_delete_insert 51,911 99,363 1.91
oltp_write_only 27,644 58,565 2.12
types_delete_insert 33,652 55,163 1.64
oltp_read_write 105,288 158,254 1.50
Average 1.75

File-Backed

Reads

Test SQLite (us) Doltlite (us) Multiplier
oltp_point_select 102,918 66,126 0.64
oltp_range_select 27,194 25,024 0.92
oltp_sum_range 26,165 24,388 0.93
oltp_order_range 4,455 4,290 0.96
oltp_distinct_range 5,538 5,509 0.99
oltp_index_scan 11,978 9,082 0.76
select_random_points 38,529 35,792 0.93
select_random_ranges 15,178 12,007 0.79
covering_index_scan 11,550 7,135 0.62
groupby_scan 36,725 41,525 1.13
index_join 12,489 12,928 1.04
index_join_scan 5,188 5,958 1.15
types_table_scan 1,119,379 1,260,882 1.13
table_scan 1,386,637 1,349,994 0.97
oltp_read_only 261,961 217,051 0.83
Average 0.92

Writes

Test SQLite (us) Doltlite (us) Multiplier
oltp_bulk_insert 260,335 387,009 1.49
oltp_insert 26,913 47,000 1.75
oltp_update_index 102,883 148,942 1.45
oltp_update_non_index 78,663 106,062 1.35
oltp_delete_insert 77,225 121,994 1.58
oltp_write_only 51,455 79,870 1.55
types_delete_insert 52,577 71,031 1.35
oltp_read_write 133,944 182,541 1.36
Average 1.48

File-Backed (autocommit)

Each statement runs as its own transaction — exposes per-commit
fixed costs that the wrapped-in-BEGIN/COMMIT tests amortize away.
SQLite uses WAL mode with synchronous=FULL in this section so
the comparison uses SQLite's durable WAL autocommit path.

Reads

Reads have no commit cost; these are the same SQL files as the
File-Backed Reads section, included here for symmetry and to
catch any per-statement overhead doltlite pays on the read path.

Test SQLite (us) Doltlite (us) Multiplier
oltp_point_select 58,997 66,385 1.13
oltp_range_select 22,453 24,982 1.11
oltp_sum_range 21,510 24,492 1.14
oltp_order_range 3,988 4,256 1.07
oltp_distinct_range 5,019 5,487 1.09
oltp_index_scan 7,621 9,213 1.21
select_random_points 33,705 37,182 1.10
select_random_ranges 10,639 11,992 1.13
covering_index_scan 7,025 7,178 1.02
groupby_scan 36,456 41,608 1.14
index_join 10,186 13,057 1.28
index_join_scan 4,825 6,027 1.25
types_table_scan 1,128,491 1,260,465 1.12
table_scan 1,396,459 1,358,561 0.97
oltp_read_only 194,453 216,578 1.11
Average 1.12

Writes

Test SQLite (us) Doltlite (us) Multiplier
oltp_bulk_insert_ac 24,899 95,598 3.84
oltp_insert_ac 26,760 106,797 3.99
oltp_update_index_ac 28,517 120,095 4.21
oltp_update_non_index_ac 24,490 97,604 3.99
oltp_delete_insert_ac 27,208 111,615 4.10
oltp_write_only_ac 26,762 112,294 4.20
types_delete_insert_ac 22,708 106,380 4.68
oltp_read_write_ac 33,040 116,463 3.52
Average 4.07

100000 rows, median of 5 invocations per test; autocommit writes use 9, workload-only timing via host monotonic clock when available.

Performance Ceiling Check (2.5x individual, 2x average; autocommit writes: 6x / 5x)

All tests within ceilings.

The pinned literals were captured under macOS collation, where sort
places '~' before letters; CI's C locale orders it after, failing the
multi-row scenario. Both pinned-shape sorts now force LC_ALL=C and the
literals use byte order.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@github-actions

Copy link
Copy Markdown

Sysbench-Style Benchmark: Doltlite vs SQLite

In-Memory

Reads

Test SQLite (us) Doltlite (us) Multiplier
oltp_point_select 26,122 30,138 1.15
oltp_range_select 10,671 13,102 1.23
oltp_sum_range 10,086 12,668 1.26
oltp_order_range 2,680 2,927 1.09
oltp_distinct_range 3,661 4,094 1.12
oltp_index_scan 4,007 5,656 1.41
select_random_points 10,794 11,503 1.07
select_random_ranges 3,144 4,103 1.31
covering_index_scan 4,276 4,530 1.06
groupby_scan 30,110 34,094 1.13
index_join 6,106 8,633 1.41
index_join_scan 3,430 4,651 1.36
types_table_scan 1,123,752 1,288,255 1.15
table_scan 1,291,171 1,401,757 1.09
oltp_read_only 109,563 125,965 1.15
Average 1.20

Writes

Test SQLite (us) Doltlite (us) Multiplier
oltp_bulk_insert 180,087 260,048 1.44
oltp_insert 15,983 29,589 1.85
oltp_update_index 54,292 96,966 1.79
oltp_update_non_index 36,667 61,587 1.68
oltp_delete_insert 47,147 75,484 1.60
oltp_write_only 23,309 47,945 2.06
types_delete_insert 25,699 41,874 1.63
oltp_read_write 73,782 114,405 1.55
Average 1.70

File-Backed

Reads

Test SQLite (us) Doltlite (us) Multiplier
oltp_point_select 95,281 55,119 0.58
oltp_range_select 18,176 15,617 0.86
oltp_sum_range 17,693 15,332 0.87
oltp_order_range 3,435 3,290 0.96
oltp_distinct_range 4,562 4,388 0.96
oltp_index_scan 11,352 8,263 0.73
select_random_points 19,496 14,488 0.74
select_random_ranges 10,134 6,580 0.65
covering_index_scan 11,573 7,128 0.62
groupby_scan 31,232 34,886 1.12
index_join 10,647 10,397 0.98
index_join_scan 4,472 5,073 1.13
types_table_scan 1,112,564 1,303,911 1.17
table_scan 1,274,695 1,392,174 1.09
oltp_read_only 214,463 166,134 0.77
Average 0.88

Writes

Test SQLite (us) Doltlite (us) Multiplier
oltp_bulk_insert 193,546 279,635 1.44
oltp_insert 23,239 37,156 1.60
oltp_update_index 82,457 123,504 1.50
oltp_update_non_index 62,153 84,928 1.37
oltp_delete_insert 70,925 99,848 1.41
oltp_write_only 46,491 67,585 1.45
types_delete_insert 42,237 55,413 1.31
oltp_read_write 100,086 134,339 1.34
Average 1.43

File-Backed (autocommit)

Each statement runs as its own transaction — exposes per-commit
fixed costs that the wrapped-in-BEGIN/COMMIT tests amortize away.
SQLite uses WAL mode with synchronous=FULL in this section so
the comparison uses SQLite's durable WAL autocommit path.

Reads

Reads have no commit cost; these are the same SQL files as the
File-Backed Reads section, included here for symmetry and to
catch any per-statement overhead doltlite pays on the read path.

Test SQLite (us) Doltlite (us) Multiplier
oltp_point_select 49,080 55,432 1.13
oltp_range_select 13,820 15,815 1.14
oltp_sum_range 13,233 15,183 1.15
oltp_order_range 2,997 3,277 1.09
oltp_distinct_range 4,001 4,470 1.12
oltp_index_scan 6,818 8,519 1.25
select_random_points 14,653 14,673 1.00
select_random_ranges 5,637 6,532 1.16
covering_index_scan 6,982 7,241 1.04
groupby_scan 30,343 35,105 1.16
index_join 7,884 10,353 1.31
index_join_scan 3,969 5,266 1.33
types_table_scan 1,120,359 1,304,850 1.16
table_scan 1,284,180 1,392,878 1.08
oltp_read_only 145,351 163,905 1.13
Average 1.15

Writes

Test SQLite (us) Doltlite (us) Multiplier
oltp_bulk_insert_ac 22,977 82,844 3.61
oltp_insert_ac 25,311 102,363 4.04
oltp_update_index_ac 28,689 116,206 4.05
oltp_update_non_index_ac 24,471 95,402 3.90
oltp_delete_insert_ac 25,683 104,669 4.08
oltp_write_only_ac 25,385 105,772 4.17
types_delete_insert_ac 22,619 98,227 4.34
oltp_read_write_ac 30,915 112,462 3.64
Average 3.98

100000 rows, median of 5 invocations per test; autocommit writes use 9, workload-only timing via host monotonic clock when available.

Performance Ceiling Check (2.5x individual, 2x average; autocommit writes: 6x / 5x)

All tests within ceilings.

@timsehn
timsehn merged commit 205ce5a into master Jul 17, 2026
27 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant