You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Each statement runs as its own transaction — exposes per-commit fixed costs that the wrapped-in-BEGIN/COMMIT tests amortize away. SQLite uses WAL mode with synchronous=FULL in this section so the comparison uses SQLite's durable WAL autocommit path.
Reads
Reads have no commit cost; these are the same SQL files as the File-Backed Reads section, included here for symmetry and to catch any per-statement overhead doltlite pays on the read path.
Test
SQLite (us)
Doltlite (us)
Multiplier
oltp_point_select
53,392
103,450
1.94
oltp_range_select
14,316
35,284
2.46
oltp_sum_range
14,123
35,165
2.49
oltp_order_range
3,103
6,221
2.00
oltp_distinct_range
4,141
7,305
1.76
oltp_index_scan
7,428
17,490
2.35
select_random_points
19,127
62,928
3.29
select_random_ranges
6,120
14,728
2.41
covering_index_scan
6,704
7,875
1.17
groupby_scan
32,814
50,301
1.53
index_join
7,974
17,639
2.21
index_join_scan
4,040
12,156
3.01
types_table_scan
1,199,735
2,704,809
2.25
table_scan
1,411,218
3,377,019
2.39
oltp_read_only
153,966
289,538
1.88
Average
2.21
Writes
Test
SQLite (us)
Doltlite (us)
Multiplier
oltp_bulk_insert_ac
22,083
79,842
3.62
oltp_insert_ac
24,456
97,977
4.01
oltp_update_index_ac
26,832
111,079
4.14
oltp_update_non_index_ac
22,188
88,880
4.01
oltp_delete_insert_ac
26,946
108,686
4.03
oltp_write_only_ac
27,471
97,054
3.53
types_delete_insert_ac
21,899
97,608
4.46
oltp_read_write_ac
31,299
115,604
3.69
Average
3.94
100000 rows, median of 5 invocations per test, workload-only timing via host monotonic clock when available.
Sysbench-Style Benchmark (composite PK): Doltlite vs SQLite
Companion to the classic Sysbench-Style Benchmark. Every workload here
runs against tables with a 2-column INTEGER PRIMARY KEY(a, b) WITHOUT ROWID. Individual ratios gated at 6×; section averages gated at 5×.
In-Memory
Reads
Test
SQLite (us)
Doltlite (us)
Multiplier
oltp_point_select
34,001
44,556
1.31
oltp_range_select
20,535
25,420
1.24
oltp_sum_range
18,477
25,702
1.39
oltp_order_range
3,796
4,431
1.17
oltp_distinct_range
4,854
5,507
1.13
oltp_index_scan
4,574
7,095
1.55
select_random_points
26,878
36,773
1.37
select_random_ranges
7,338
9,176
1.25
covering_index_scan
4,325
4,754
1.10
groupby_scan
40,329
45,218
1.12
index_join
8,242
12,123
1.47
index_join_scan
4,077
6,977
1.71
types_table_scan
1,126,095
1,517,707
1.35
table_scan
1,296,677
1,732,511
1.34
oltp_read_only
151,972
190,393
1.25
Average
1.32
Writes
Test
SQLite (us)
Doltlite (us)
Multiplier
oltp_bulk_insert
241,969
319,812
1.32
oltp_insert
18,923
33,638
1.78
oltp_update_index
69,023
130,340
1.89
oltp_update_non_index
52,534
87,483
1.67
oltp_delete_insert
50,152
97,555
1.95
oltp_write_only
27,343
57,917
2.12
types_delete_insert
32,657
57,882
1.77
oltp_read_write
99,229
165,542
1.67
Average
1.77
File-Backed
Reads
Test
SQLite (us)
Doltlite (us)
Multiplier
oltp_point_select
125,635
126,282
1.01
oltp_range_select
31,282
54,419
1.74
oltp_sum_range
29,191
53,715
1.84
oltp_order_range
4,870
8,227
1.69
oltp_distinct_range
5,899
9,321
1.58
oltp_index_scan
14,339
21,301
1.49
select_random_points
45,990
99,378
2.16
select_random_ranges
16,280
21,462
1.32
covering_index_scan
12,699
11,201
0.88
groupby_scan
43,259
68,802
1.59
index_join
13,566
27,991
2.06
index_join_scan
5,402
18,652
3.45
types_table_scan
1,335,666
3,412,771
2.56
table_scan
1,591,043
4,245,656
2.67
oltp_read_only
294,889
389,001
1.32
Average
1.82
Writes
Test
SQLite (us)
Doltlite (us)
Multiplier
oltp_bulk_insert
251,113
346,245
1.38
oltp_insert
25,377
49,961
1.97
oltp_update_index
158,696
292,217
1.84
oltp_update_non_index
104,768
186,522
1.78
oltp_delete_insert
108,089
205,460
1.90
oltp_write_only
63,467
132,922
2.09
types_delete_insert
60,968
108,693
1.78
oltp_read_write
167,890
350,376
2.09
Average
1.85
File-Backed (autocommit)
Each statement runs as its own transaction — exposes per-commit fixed costs that the wrapped-in-BEGIN/COMMIT tests amortize away. SQLite uses WAL mode with synchronous=FULL in this section so the comparison uses SQLite's durable WAL autocommit path.
Reads
Reads have no commit cost; these are the same SQL files as the File-Backed Reads section, included here for symmetry and to catch any per-statement overhead doltlite pays on the read path.
Test
SQLite (us)
Doltlite (us)
Multiplier
oltp_point_select
70,124
130,394
1.86
oltp_range_select
26,237
54,727
2.09
oltp_sum_range
24,656
54,928
2.23
oltp_order_range
4,424
8,378
1.89
oltp_distinct_range
5,506
9,327
1.69
oltp_index_scan
8,825
21,758
2.47
select_random_points
40,499
100,246
2.48
select_random_ranges
11,005
21,648
1.97
covering_index_scan
7,142
11,093
1.55
groupby_scan
42,767
67,094
1.57
index_join
11,095
27,978
2.52
index_join_scan
4,909
18,249
3.72
types_table_scan
1,341,460
3,436,590
2.56
table_scan
1,598,182
4,271,081
2.67
oltp_read_only
218,837
383,586
1.75
Average
2.20
Writes
Test
SQLite (us)
Doltlite (us)
Multiplier
oltp_bulk_insert_ac
15,568
58,884
3.78
oltp_insert_ac
19,689
78,904
4.01
oltp_update_index_ac
19,351
91,505
4.73
oltp_update_non_index_ac
16,606
73,663
4.44
oltp_delete_insert_ac
17,453
82,513
4.73
oltp_write_only_ac
20,078
83,366
4.15
types_delete_insert_ac
15,644
69,953
4.47
oltp_read_write_ac
25,727
101,690
3.95
Average
4.28
100000 rows, median of 5 invocations per test, workload-only timing via host monotonic clock when available.
Sysbench-Style Benchmark (BLOB PK): Doltlite vs SQLite
Companion to the classic Sysbench-Style Benchmark. Every workload here
runs against tables with a 16-byte big-endian BLOB PRIMARY KEY. Individual ratios gated at 6×; section averages gated at 5×.
In-Memory
Reads
Test
SQLite (us)
Doltlite (us)
Multiplier
oltp_point_select
30,232
45,609
1.51
oltp_range_select
12,270
19,957
1.63
oltp_sum_range
11,622
19,246
1.66
oltp_order_range
2,870
3,831
1.33
oltp_distinct_range
3,932
4,891
1.24
oltp_index_scan
4,315
7,651
1.77
select_random_points
17,742
28,281
1.59
select_random_ranges
4,151
6,373
1.54
covering_index_scan
4,257
5,238
1.23
groupby_scan
32,275
39,141
1.21
index_join
6,696
12,755
1.90
index_join_scan
4,132
7,908
1.91
types_table_scan
1,140,774
1,597,725
1.40
table_scan
1,343,509
1,753,043
1.30
oltp_read_only
116,551
168,920
1.45
Average
1.51
Writes
Test
SQLite (us)
Doltlite (us)
Multiplier
oltp_bulk_insert
239,516
338,322
1.41
oltp_insert
19,606
38,121
1.94
oltp_update_index
66,147
140,982
2.13
oltp_update_non_index
48,004
90,463
1.88
oltp_delete_insert
47,531
107,811
2.27
oltp_write_only
28,115
62,795
2.23
types_delete_insert
32,342
59,269
1.83
oltp_read_write
83,602
162,435
1.94
Average
1.96
File-Backed
Reads
Test
SQLite (us)
Doltlite (us)
Multiplier
oltp_point_select
104,652
119,406
1.14
oltp_range_select
21,819
48,896
2.24
oltp_sum_range
21,128
48,400
2.29
oltp_order_range
3,844
7,840
2.04
oltp_distinct_range
4,924
8,952
1.82
oltp_index_scan
12,840
21,283
1.66
select_random_points
33,304
82,513
2.48
select_random_ranges
11,418
17,464
1.53
covering_index_scan
11,690
12,910
1.10
groupby_scan
35,305
61,869
1.75
index_join
13,016
33,102
2.54
index_join_scan
6,550
23,180
3.54
types_table_scan
1,255,188
3,571,135
2.85
table_scan
1,476,246
4,254,377
2.88
oltp_read_only
238,517
361,128
1.51
Average
2.09
Writes
Test
SQLite (us)
Doltlite (us)
Multiplier
oltp_bulk_insert
249,646
363,026
1.45
oltp_insert
36,139
58,663
1.62
oltp_update_index
172,757
308,326
1.78
oltp_update_non_index
105,259
187,845
1.78
oltp_delete_insert
113,683
214,882
1.89
oltp_write_only
71,208
134,766
1.89
types_delete_insert
63,178
115,971
1.84
oltp_read_write
155,863
346,999
2.23
Average
1.81
File-Backed (autocommit)
Each statement runs as its own transaction — exposes per-commit fixed costs that the wrapped-in-BEGIN/COMMIT tests amortize away. SQLite uses WAL mode with synchronous=FULL in this section so the comparison uses SQLite's durable WAL autocommit path.
Reads
Reads have no commit cost; these are the same SQL files as the File-Backed Reads section, included here for symmetry and to catch any per-statement overhead doltlite pays on the read path.
Test
SQLite (us)
Doltlite (us)
Multiplier
oltp_point_select
61,165
120,386
1.97
oltp_range_select
17,249
49,136
2.85
oltp_sum_range
16,607
48,912
2.95
oltp_order_range
3,489
7,901
2.26
oltp_distinct_range
4,596
9,009
1.96
oltp_index_scan
8,484
21,266
2.51
select_random_points
28,787
84,530
2.94
select_random_ranges
7,260
17,452
2.40
covering_index_scan
7,680
13,019
1.70
groupby_scan
34,978
62,005
1.77
index_join
11,000
33,532
3.05
index_join_scan
6,173
23,451
3.80
types_table_scan
1,265,086
3,598,708
2.84
table_scan
1,499,077
4,290,122
2.86
oltp_read_only
173,300
365,031
2.11
Average
2.53
Writes
Test
SQLite (us)
Doltlite (us)
Multiplier
oltp_bulk_insert_ac
22,550
85,545
3.79
oltp_insert_ac
23,980
100,792
4.20
oltp_update_index_ac
28,453
114,617
4.03
oltp_update_non_index_ac
22,613
99,466
4.40
oltp_delete_insert_ac
25,517
106,718
4.18
oltp_write_only_ac
25,102
103,070
4.11
types_delete_insert_ac
23,158
93,426
4.03
oltp_read_write_ac
31,331
119,385
3.81
Average
4.07
100000 rows, median of 5 invocations per test, workload-only timing via host monotonic clock when available.
Sysbench-Style Benchmark (TEXT PK): Doltlite vs SQLite
Companion to the classic Sysbench-Style Benchmark. Every workload here
runs against tables with a 32-char hex TEXT PRIMARY KEY (UUID-shaped). Individual ratios gated at 6×; section averages gated at 5×.
In-Memory
Reads
Test
SQLite (us)
Doltlite (us)
Multiplier
oltp_point_select
31,015
46,239
1.49
oltp_range_select
13,751
20,469
1.49
oltp_sum_range
12,282
20,003
1.63
oltp_order_range
3,059
3,931
1.29
oltp_distinct_range
4,071
5,068
1.24
oltp_index_scan
4,759
8,043
1.69
select_random_points
18,402
29,959
1.63
select_random_ranges
4,177
6,721
1.61
covering_index_scan
4,869
5,447
1.12
groupby_scan
32,541
39,568
1.22
index_join
6,974
13,052
1.87
index_join_scan
4,545
8,037
1.77
types_table_scan
1,230,983
1,655,183
1.34
table_scan
1,512,557
1,807,207
1.19
oltp_read_only
128,252
178,896
1.39
Average
1.46
Writes
Test
SQLite (us)
Doltlite (us)
Multiplier
oltp_bulk_insert
233,296
345,506
1.48
oltp_insert
22,368
39,894
1.78
oltp_update_index
76,789
156,852
2.04
oltp_update_non_index
48,868
94,332
1.93
oltp_delete_insert
55,105
111,169
2.02
oltp_write_only
30,558
64,827
2.12
types_delete_insert
33,641
63,298
1.88
oltp_read_write
85,211
168,077
1.97
Average
1.90
File-Backed
Reads
Test
SQLite (us)
Doltlite (us)
Multiplier
oltp_point_select
108,110
128,238
1.19
oltp_range_select
22,980
53,700
2.34
oltp_sum_range
22,293
53,168
2.38
oltp_order_range
3,946
8,334
2.11
oltp_distinct_range
5,066
9,316
1.84
oltp_index_scan
13,364
22,211
1.66
select_random_points
34,963
88,953
2.54
select_random_ranges
11,724
18,514
1.58
covering_index_scan
12,965
13,643
1.05
groupby_scan
36,205
65,617
1.81
index_join
15,243
35,162
2.31
index_join_scan
8,061
25,252
3.13
types_table_scan
1,337,334
3,996,239
2.99
table_scan
1,572,982
4,794,824
3.05
oltp_read_only
249,230
391,596
1.57
Average
2.10
Writes
Test
SQLite (us)
Doltlite (us)
Multiplier
oltp_bulk_insert
248,660
383,986
1.54
oltp_insert
49,950
62,350
1.25
oltp_update_index
200,735
338,008
1.68
oltp_update_non_index
126,684
202,841
1.60
oltp_delete_insert
142,374
229,782
1.61
oltp_write_only
121,153
142,343
1.17
types_delete_insert
72,260
126,747
1.75
oltp_read_write
211,786
357,582
1.69
Average
1.54
File-Backed (autocommit)
Each statement runs as its own transaction — exposes per-commit fixed costs that the wrapped-in-BEGIN/COMMIT tests amortize away. SQLite uses WAL mode with synchronous=FULL in this section so the comparison uses SQLite's durable WAL autocommit path.
Reads
Reads have no commit cost; these are the same SQL files as the File-Backed Reads section, included here for symmetry and to catch any per-statement overhead doltlite pays on the read path.
Test
SQLite (us)
Doltlite (us)
Multiplier
oltp_point_select
62,687
130,677
2.08
oltp_range_select
18,786
54,561
2.90
oltp_sum_range
17,682
54,485
3.08
oltp_order_range
3,775
8,518
2.26
oltp_distinct_range
4,740
9,482
2.00
oltp_index_scan
9,254
22,397
2.42
select_random_points
30,966
89,089
2.88
select_random_ranges
7,444
18,800
2.53
covering_index_scan
8,510
13,668
1.61
groupby_scan
35,402
66,077
1.87
index_join
13,074
36,737
2.81
index_join_scan
7,949
26,644
3.35
types_table_scan
1,319,857
4,036,013
3.06
table_scan
1,587,038
4,754,230
3.00
oltp_read_only
185,062
389,062
2.10
Average
2.53
Writes
Test
SQLite (us)
Doltlite (us)
Multiplier
oltp_bulk_insert_ac
22,189
93,314
4.21
oltp_insert_ac
26,877
105,911
3.94
oltp_update_index_ac
30,928
128,512
4.16
oltp_update_non_index_ac
25,996
107,368
4.13
oltp_delete_insert_ac
26,642
114,150
4.28
oltp_write_only_ac
26,887
107,146
3.99
types_delete_insert_ac
22,601
106,214
4.70
oltp_read_write_ac
34,551
127,313
3.68
Average
4.14
100000 rows, median of 5 invocations per test, workload-only timing via host monotonic clock when available.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Benchmark target
Latest merged PR #1035 sysbench comments highest multiplier:
PRIMARY KEY(a,b) WITHOUT ROWIDtypes_delete_insert_acOn current master the benchmark schema has been corrected, so the local targeted SQL used composite keys for
sbtest_types.Local timing
Targeted 100k-row corrected composite
types_delete_insert_ac, Doltlite only, 5 samples each:49,195 27,048 41,305 36,694 42,577 us, median41,305 us38,643 20,950 26,351 21,582 37,605 us, median26,351 usTests
make -j8 sqlite3.o libdoltlite.amake -j8 invariant_test corruption_test && ./invariant_test && ./corruption_testBENCH_ROWS=200 BENCH_RUNS=1 BENCH_MAX_MULTIPLIER=1000 BENCH_AVG_MAX_MULTIPLIER=1000 ./test/sysbench_compare_compositepk.shmake -j8 c-tests && test/run_c_tests.sh .