Optimize write transaction begin refresh

[timsehn]

Summary

• avoid the pre-lock disk refresh when beginning a write transaction
• keep the locked chunkStoreLockAndRefreshChanged() refresh/check as the authoritative write-path freshness check
• preserve the existing read transaction refresh path

Notes

This removes duplicated external-change probing on autocommit writes. Local autocommit benchmark runs were noisy, so this is intentionally a narrow transaction-path cleanup rather than a claimed large benchmark win.

Testing

• make -j8 doltlite doltlite-lib
• make c-tests
• ../test/run_c_tests.sh .
  • C gating suite passed: 0 / 12 failures

[github-actions]

Sysbench-Style Benchmark: Doltlite vs SQLite

In-Memory

Reads

Test	SQLite (us)	Doltlite (us)	Multiplier
oltp_point_select	23,477	37,111	1.58
oltp_range_select	9,682	14,200	1.47
oltp_sum_range	9,079	14,105	1.55
oltp_order_range	2,441	3,199	1.31
oltp_distinct_range	3,531	4,260	1.21
oltp_index_scan	3,807	6,787	1.78
select_random_points	9,157	17,306	1.89
select_random_ranges	2,733	5,360	1.96
covering_index_scan	4,084	4,462	1.09
groupby_scan	30,440	34,910	1.15
index_join	5,878	9,214	1.57
index_join_scan	3,249	5,359	1.65
types_table_scan	1,042,920	1,370,808	1.31
table_scan	1,163,356	1,557,448	1.34
oltp_read_only	100,289	135,751	1.35
Average			1.48

Writes

Test	SQLite (us)	Doltlite (us)	Multiplier
oltp_bulk_insert	180,377	246,516	1.37
oltp_insert	15,136	27,055	1.79
oltp_update_index	48,354	101,610	2.10
oltp_update_non_index	32,871	67,009	2.04
oltp_delete_insert	43,502	77,342	1.78
oltp_write_only	21,170	47,906	2.26
types_delete_insert	24,020	44,958	1.87
oltp_read_write	70,304	126,752	1.80
Average			1.88

File-Backed

Reads

Test	SQLite (us)	Doltlite (us)	Multiplier
oltp_point_select	98,572	103,841	1.05
oltp_range_select	19,017	35,287	1.86
oltp_sum_range	18,643	34,959	1.88
oltp_order_range	3,511	6,324	1.80
oltp_distinct_range	4,590	7,341	1.60
oltp_index_scan	12,000	17,079	1.42
select_random_points	23,725	62,176	2.62
select_random_ranges	10,684	14,780	1.38
covering_index_scan	11,162	7,865	0.70
groupby_scan	33,183	50,165	1.51
index_join	10,237	17,662	1.73
index_join_scan	4,350	12,024	2.76
types_table_scan	1,196,967	2,644,300	2.21
table_scan	1,385,149	3,338,893	2.41
oltp_read_only	218,103	283,510	1.30
Average			1.75

Writes

Test	SQLite (us)	Doltlite (us)	Multiplier
oltp_bulk_insert	188,791	265,103	1.40
oltp_insert	21,808	37,521	1.72
oltp_update_index	123,265	222,317	1.80
oltp_update_non_index	80,011	145,807	1.82
oltp_delete_insert	89,409	154,981	1.73
oltp_write_only	57,718	101,647	1.76
types_delete_insert	48,175	84,696	1.76
oltp_read_write	119,147	262,517	2.20
Average			1.78

File-Backed (autocommit)

Each statement runs as its own transaction — exposes per-commit
fixed costs that the wrapped-in-BEGIN/COMMIT tests amortize away.
SQLite uses WAL mode with synchronous=FULL in this section so
the comparison uses SQLite's durable WAL autocommit path.

Reads

Reads have no commit cost; these are the same SQL files as the
File-Backed Reads section, included here for symmetry and to
catch any per-statement overhead doltlite pays on the read path.

Test	SQLite (us)	Doltlite (us)	Multiplier
oltp_point_select	54,138	104,072	1.92
oltp_range_select	14,267	34,999	2.45
oltp_sum_range	14,111	35,126	2.49
oltp_order_range	3,048	6,263	2.05
oltp_distinct_range	4,150	7,283	1.75
oltp_index_scan	7,509	17,945	2.39
select_random_points	19,903	64,138	3.22
select_random_ranges	6,324	15,592	2.47
covering_index_scan	6,764	8,512	1.26
groupby_scan	33,047	50,606	1.53
index_join	7,896	17,465	2.21
index_join_scan	3,883	12,033	3.10
types_table_scan	1,191,223	2,661,758	2.23
table_scan	1,403,917	3,370,481	2.40
oltp_read_only	155,138	281,663	1.82
Average			2.22

Writes

Test	SQLite (us)	Doltlite (us)	Multiplier
oltp_bulk_insert_ac	20,418	85,717	4.20
oltp_insert_ac	24,004	96,414	4.02
oltp_update_index_ac	25,519	106,337	4.17
oltp_update_non_index_ac	22,664	87,846	3.88
oltp_delete_insert_ac	23,925	96,315	4.03
oltp_write_only_ac	28,847	106,936	3.71
types_delete_insert_ac	21,884	92,570	4.23
oltp_read_write_ac	32,470	113,521	3.50
Average			3.96

100000 rows, median of 5 invocations per test, workload-only timing via host monotonic clock when available.

Performance Ceiling Check (6x individual, 5x average)

All tests within ceilings.

[github-actions]

Sysbench-Style Benchmark (composite PK): Doltlite vs SQLite

Companion to the classic Sysbench-Style Benchmark. Every workload here
runs against tables with a 2-column INTEGER PRIMARY KEY(a, b) WITHOUT ROWID.
Individual ratios gated at 6×; section averages gated at 5×.

In-Memory

Reads

Test	SQLite (us)	Doltlite (us)	Multiplier
oltp_point_select	32,783	48,476	1.48
oltp_range_select	18,699	27,082	1.45
oltp_sum_range	17,467	25,924	1.48
oltp_order_range	3,554	4,619	1.30
oltp_distinct_range	4,615	5,687	1.23
oltp_index_scan	4,505	7,441	1.65
select_random_points	27,624	40,328	1.46
select_random_ranges	7,830	10,195	1.30
covering_index_scan	4,149	4,988	1.20
groupby_scan	39,327	44,413	1.13
index_join	8,046	12,750	1.58
index_join_scan	3,985	6,857	1.72
types_table_scan	1,049,926	1,371,272	1.31
table_scan	1,260,032	1,732,039	1.37
oltp_read_only	146,235	203,527	1.39
Average			1.40

Writes

Test	SQLite (us)	Doltlite (us)	Multiplier
oltp_bulk_insert	243,183	330,516	1.36
oltp_insert	19,106	33,381	1.75
oltp_update_index	66,639	127,823	1.92
oltp_update_non_index	49,876	86,211	1.73
oltp_delete_insert	48,932	97,070	1.98
oltp_write_only	26,532	57,203	2.16
types_delete_insert	24,161	45,962	1.90
oltp_read_write	101,478	173,959	1.71
Average			1.81

File-Backed

Reads

Test	SQLite (us)	Doltlite (us)	Multiplier
oltp_point_select	108,884	118,920	1.09
oltp_range_select	28,248	51,525	1.82
oltp_sum_range	27,616	49,613	1.80
oltp_order_range	4,489	7,920	1.76
oltp_distinct_range	5,738	9,152	1.59
oltp_index_scan	12,810	19,996	1.56
select_random_points	44,847	92,888	2.07
select_random_ranges	15,751	21,126	1.34
covering_index_scan	11,095	10,323	0.93
groupby_scan	39,795	61,633	1.55
index_join	13,186	26,660	2.02
index_join_scan	5,158	16,308	3.16
types_table_scan	1,202,307	2,694,751	2.24
table_scan	1,454,264	3,822,928	2.63
oltp_read_only	272,956	373,700	1.37
Average			1.80

Writes

Test	SQLite (us)	Doltlite (us)	Multiplier
oltp_bulk_insert	258,430	356,396	1.38
oltp_insert	26,005	49,144	1.89
oltp_update_index	155,305	266,102	1.71
oltp_update_non_index	104,288	176,472	1.69
oltp_delete_insert	112,170	189,927	1.69
oltp_write_only	65,200	121,544	1.86
types_delete_insert	48,913	88,192	1.80
oltp_read_write	167,383	335,613	2.01
Average			1.75

File-Backed (autocommit)

Each statement runs as its own transaction — exposes per-commit
fixed costs that the wrapped-in-BEGIN/COMMIT tests amortize away.
SQLite uses WAL mode with synchronous=FULL in this section so
the comparison uses SQLite's durable WAL autocommit path.

Reads

Reads have no commit cost; these are the same SQL files as the
File-Backed Reads section, included here for symmetry and to
catch any per-statement overhead doltlite pays on the read path.

Test	SQLite (us)	Doltlite (us)	Multiplier
oltp_point_select	64,728	122,427	1.89
oltp_range_select	23,490	51,651	2.20
oltp_sum_range	22,760	50,571	2.22
oltp_order_range	4,186	8,081	1.93
oltp_distinct_range	5,165	9,065	1.76
oltp_index_scan	8,295	19,846	2.39
select_random_points	38,675	91,472	2.37
select_random_ranges	11,125	21,021	1.89
covering_index_scan	6,843	10,363	1.51
groupby_scan	39,565	61,719	1.56
index_join	10,653	26,217	2.46
index_join_scan	4,749	16,370	3.45
types_table_scan	1,203,989	2,709,448	2.25
table_scan	1,462,440	3,852,451	2.63
oltp_read_only	206,820	373,150	1.80
Average			2.15

Writes

Test	SQLite (us)	Doltlite (us)	Multiplier
oltp_bulk_insert_ac	23,939	79,294	3.31
oltp_insert_ac	27,842	98,415	3.53
oltp_update_index_ac	31,734	114,085	3.60
oltp_update_non_index_ac	24,797	98,213	3.96
oltp_delete_insert_ac	24,851	104,939	4.22
oltp_write_only_ac	26,837	103,558	3.86
types_delete_insert_ac	26,205	132,245	5.05
oltp_read_write_ac	36,797	117,698	3.20
Average			3.84

100000 rows, median of 5 invocations per test, workload-only timing via host monotonic clock when available.

Performance Ceiling Check (6x individual, 5x average)

All tests within ceilings.

[github-actions]

Sysbench-Style Benchmark (TEXT PK): Doltlite vs SQLite

Companion to the classic Sysbench-Style Benchmark. Every workload here
runs against tables with a 32-char hex TEXT PRIMARY KEY (UUID-shaped).
Individual ratios gated at 6×; section averages gated at 5×.

In-Memory

Reads

Test	SQLite (us)	Doltlite (us)	Multiplier
oltp_point_select	25,188	40,561	1.61
oltp_range_select	11,333	19,134	1.69
oltp_sum_range	10,751	18,566	1.73
oltp_order_range	2,616	3,638	1.39
oltp_distinct_range	3,355	4,422	1.32
oltp_index_scan	3,800	6,824	1.80
select_random_points	15,491	27,089	1.75
select_random_ranges	3,385	5,873	1.74
covering_index_scan	3,978	4,769	1.20
groupby_scan	28,868	36,093	1.25
index_join	5,938	11,430	1.92
index_join_scan	3,750	7,565	2.02
types_table_scan	934,599	1,252,874	1.34
table_scan	1,178,150	1,653,561	1.40
oltp_read_only	102,552	152,935	1.49
Average			1.58

Writes

Test	SQLite (us)	Doltlite (us)	Multiplier
oltp_bulk_insert	196,291	292,544	1.49
oltp_insert	18,439	33,694	1.83
oltp_update_index	61,853	131,937	2.13
oltp_update_non_index	42,053	81,320	1.93
oltp_delete_insert	42,876	95,737	2.23
oltp_write_only	24,045	55,795	2.32
types_delete_insert	20,979	39,784	1.90
oltp_read_write	72,268	143,859	1.99
Average			1.98

File-Backed

Reads

Test	SQLite (us)	Doltlite (us)	Multiplier
oltp_point_select	62,018	87,162	1.41
oltp_range_select	16,283	39,706	2.44
oltp_sum_range	15,852	39,286	2.48
oltp_order_range	3,188	6,419	2.01
oltp_distinct_range	3,882	7,334	1.89
oltp_index_scan	8,223	15,632	1.90
select_random_points	25,282	63,567	2.51
select_random_ranges	7,121	12,963	1.82
covering_index_scan	7,731	10,102	1.31
groupby_scan	30,402	53,586	1.76
index_join	10,447	26,437	2.53
index_join_scan	6,081	18,940	3.11
types_table_scan	1,067,466	2,115,782	1.98
table_scan	1,321,530	3,520,852	2.66
oltp_read_only	160,942	280,210	1.74
Average			2.10

Writes

Test	SQLite (us)	Doltlite (us)	Multiplier
oltp_bulk_insert	206,480	312,056	1.51
oltp_insert	43,284	47,405	1.10
oltp_update_index	135,773	242,852	1.79
oltp_update_non_index	120,562	147,020	1.22
oltp_delete_insert	96,448	167,763	1.74
oltp_write_only	72,296	104,742	1.45
types_delete_insert	37,139	67,374	1.81
oltp_read_write	128,678	266,735	2.07
Average			1.59

File-Backed (autocommit)

Each statement runs as its own transaction — exposes per-commit
fixed costs that the wrapped-in-BEGIN/COMMIT tests amortize away.
SQLite uses WAL mode with synchronous=FULL in this section so
the comparison uses SQLite's durable WAL autocommit path.

Reads

Reads have no commit cost; these are the same SQL files as the
File-Backed Reads section, included here for symmetry and to
catch any per-statement overhead doltlite pays on the read path.

Test	SQLite (us)	Doltlite (us)	Multiplier
oltp_point_select	40,556	88,227	2.18
oltp_range_select	14,015	40,227	2.87
oltp_sum_range	13,644	39,713	2.91
oltp_order_range	3,063	6,530	2.13
oltp_distinct_range	3,840	7,254	1.89
oltp_index_scan	6,390	15,772	2.47
select_random_points	22,928	63,575	2.77
select_random_ranges	5,156	13,244	2.57
covering_index_scan	5,978	10,113	1.69
groupby_scan	30,209	53,256	1.76
index_join	9,590	25,966	2.71
index_join_scan	6,077	18,946	3.12
types_table_scan	1,041,045	2,114,670	2.03
table_scan	1,296,787	3,536,640	2.73
oltp_read_only	132,543	281,925	2.13
Average			2.40

Writes

Test	SQLite (us)	Doltlite (us)	Multiplier
oltp_bulk_insert_ac	31,048	94,333	3.04
oltp_insert_ac	33,619	116,544	3.47
oltp_update_index_ac	37,071	133,554	3.60
oltp_update_non_index_ac	35,421	114,250	3.23
oltp_delete_insert_ac	33,820	120,617	3.57
oltp_write_only_ac	29,317	120,300	4.10
types_delete_insert_ac	25,018	115,071	4.60
oltp_read_write_ac	38,606	135,062	3.50
Average			3.64

100000 rows, median of 5 invocations per test, workload-only timing via host monotonic clock when available.

Performance Ceiling Check (6x individual, 5x average)

All tests within ceilings.

[github-actions]

Sysbench-Style Benchmark (BLOB PK): Doltlite vs SQLite

Companion to the classic Sysbench-Style Benchmark. Every workload here
runs against tables with a 16-byte big-endian BLOB PRIMARY KEY.
Individual ratios gated at 6×; section averages gated at 5×.

In-Memory

Reads

Test	SQLite (us)	Doltlite (us)	Multiplier
oltp_point_select	29,476	44,746	1.52
oltp_range_select	12,595	20,885	1.66
oltp_sum_range	11,588	19,272	1.66
oltp_order_range	2,815	3,969	1.41
oltp_distinct_range	3,927	5,075	1.29
oltp_index_scan	4,329	7,434	1.72
select_random_points	17,248	27,837	1.61
select_random_ranges	3,769	6,364	1.69
covering_index_scan	4,234	5,180	1.22
groupby_scan	31,953	39,390	1.23
index_join	6,544	12,337	1.89
index_join_scan	3,781	7,611	2.01
types_table_scan	1,043,413	1,365,317	1.31
table_scan	1,183,810	1,744,200	1.47
oltp_read_only	113,084	172,035	1.52
Average			1.55

Writes

Test	SQLite (us)	Doltlite (us)	Multiplier
oltp_bulk_insert	238,955	333,227	1.39
oltp_insert	19,409	37,526	1.93
oltp_update_index	65,890	139,042	2.11
oltp_update_non_index	46,457	88,569	1.91
oltp_delete_insert	46,882	104,672	2.23
oltp_write_only	26,658	62,491	2.34
types_delete_insert	24,585	45,615	1.86
oltp_read_write	81,171	160,635	1.98
Average			1.97

File-Backed

Reads

Test	SQLite (us)	Doltlite (us)	Multiplier
oltp_point_select	104,452	118,932	1.14
oltp_range_select	21,788	49,329	2.26
oltp_sum_range	20,861	48,057	2.30
oltp_order_range	3,799	7,858	2.07
oltp_distinct_range	4,878	8,934	1.83
oltp_index_scan	12,679	20,964	1.65
select_random_points	32,945	80,925	2.46
select_random_ranges	11,306	17,312	1.53
covering_index_scan	11,496	12,860	1.12
groupby_scan	34,983	61,093	1.75
index_join	12,937	32,406	2.50
index_join_scan	6,348	22,793	3.59
types_table_scan	1,200,700	2,652,732	2.21
table_scan	1,454,788	4,272,796	2.94
oltp_read_only	235,297	356,638	1.52
Average			2.06

Writes

Test	SQLite (us)	Doltlite (us)	Multiplier
oltp_bulk_insert	249,174	359,220	1.44
oltp_insert	35,322	57,222	1.62
oltp_update_index	169,058	300,197	1.78
oltp_update_non_index	101,962	184,616	1.81
oltp_delete_insert	110,919	212,629	1.92
oltp_write_only	70,205	134,026	1.91
types_delete_insert	48,682	85,916	1.76
oltp_read_write	153,599	341,751	2.22
Average			1.81

File-Backed (autocommit)

Each statement runs as its own transaction — exposes per-commit
fixed costs that the wrapped-in-BEGIN/COMMIT tests amortize away.
SQLite uses WAL mode with synchronous=FULL in this section so
the comparison uses SQLite's durable WAL autocommit path.

Reads

Reads have no commit cost; these are the same SQL files as the
File-Backed Reads section, included here for symmetry and to
catch any per-statement overhead doltlite pays on the read path.

Test	SQLite (us)	Doltlite (us)	Multiplier
oltp_point_select	60,952	121,069	1.99
oltp_range_select	17,547	49,973	2.85
oltp_sum_range	17,078	48,166	2.82
oltp_order_range	3,501	7,978	2.28
oltp_distinct_range	4,450	8,903	2.00
oltp_index_scan	8,373	21,185	2.53
select_random_points	28,342	82,715	2.92
select_random_ranges	7,007	17,483	2.50
covering_index_scan	7,316	13,017	1.78
groupby_scan	34,547	61,274	1.77
index_join	11,001	32,620	2.97
index_join_scan	6,168	22,810	3.70
types_table_scan	1,195,928	2,654,885	2.22
table_scan	1,463,876	4,234,701	2.89
oltp_read_only	171,425	356,908	2.08
Average			2.49

Writes

Test	SQLite (us)	Doltlite (us)	Multiplier
oltp_bulk_insert_ac	21,268	72,321	3.40
oltp_insert_ac	25,958	93,006	3.58
oltp_update_index_ac	26,185	105,623	4.03
oltp_update_non_index_ac	22,061	89,922	4.08
oltp_delete_insert_ac	23,719	102,268	4.31
oltp_write_only_ac	26,366	98,256	3.73
types_delete_insert_ac	21,207	81,957	3.86
oltp_read_write_ac	32,476	111,122	3.42
Average			3.80

100000 rows, median of 5 invocations per test, workload-only timing via host monotonic clock when available.

Performance Ceiling Check (6x individual, 5x average)

All tests within ceilings.