Skip to content

Commit 362157d

Browse files
KKouldKKould
authored
docs: update TPCC benchmark results (#373)
1 parent dec3643 commit 362157d

2 files changed

Lines changed: 104 additions & 90 deletions

File tree

‎README.md‎

Lines changed: 6 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -203,12 +203,13 @@ Recent stable-run 720-second local comparison on the machine above:
203203
204204
| Backend | TpmC | New-Order p90 | Payment p90 | Order-Status p90 | Delivery p90 | Stock-Level p90 |
205205
| --- | ---: | ---: | ---: | ---: | ---: | ---: |
206-
| KiteSQL LMDB | 71345 | 0.001s | 0.001s | 0.001s | 0.002s | 0.001s |
207-
| KiteSQL RocksDB | 40563 | 0.001s | 0.001s | 0.002s | 0.009s | 0.001s |
208-
| SQLite balanced | 67527 | 0.001s | 0.001s | 0.001s | 0.001s | 0.001s |
209-
| SQLite practical | 64774 | 0.001s | 0.001s | 0.001s | 0.001s | 0.001s |
206+
| KiteSQL LMDB | 82871 | 0.001s | 0.001s | 0.001s | 0.002s | 0.001s |
207+
| KiteSQL RocksDB | 40960 | 0.001s | 0.001s | 0.001s | 0.011s | 0.001s |
208+
| SQLite balanced | 51637 | 0.001s | 0.001s | 0.001s | 0.001s | 0.001s |
209+
| SQLite practical | 61424 | 0.001s | 0.001s | 0.001s | 0.001s | 0.001s |
210+
211+
These rows are from the stable runs on `2026-07-11`; the detailed raw outputs are recorded in [tpcc/README.md](tpcc/README.md).
210212
211-
These rows are from `./scripts/run_tpcc_stable.py --build` at `2026-06-23_00-50-48`; the detailed raw outputs are recorded in [tpcc/README.md](tpcc/README.md).
212213
#### 👉[check more](tpcc/README.md)
213214
214215
## Roadmap

‎tpcc/README.md‎

Lines changed: 98 additions & 85 deletions
Original file line numberDiff line numberDiff line change
@@ -46,12 +46,12 @@ Local stable-run 720-second comparison on the machine above:
4646

4747
| Backend | TpmC | New-Order p90 | Payment p90 | Order-Status p90 | Delivery p90 | Stock-Level p90 |
4848
| --- | ---: | ---: | ---: | ---: | ---: | ---: |
49-
| KiteSQL LMDB | 71345 | 0.001s | 0.001s | 0.001s | 0.002s | 0.001s |
50-
| KiteSQL RocksDB | 40563 | 0.001s | 0.001s | 0.002s | 0.009s | 0.001s |
51-
| SQLite balanced | 67527 | 0.001s | 0.001s | 0.001s | 0.001s | 0.001s |
52-
| SQLite practical | 64774 | 0.001s | 0.001s | 0.001s | 0.001s | 0.001s |
49+
| KiteSQL LMDB | 82871 | 0.001s | 0.001s | 0.001s | 0.002s | 0.001s |
50+
| KiteSQL RocksDB | 40960 | 0.001s | 0.001s | 0.001s | 0.011s | 0.001s |
51+
| SQLite balanced | 51637 | 0.001s | 0.001s | 0.001s | 0.001s | 0.001s |
52+
| SQLite practical | 61424 | 0.001s | 0.001s | 0.001s | 0.001s | 0.001s |
5353

54-
- All rows are from `./scripts/run_tpcc_stable.py --build` at `2026-06-23_00-50-48`.
54+
- The KiteSQL rows are from `2026-07-11_17-20-24`; the SQLite rows are from `2026-07-11_20-25-01`.
5555
- The stable-run gates were `temp<=65.0C`, `cpu<=20.0%`, `min_cooldown=300s`, `stable_samples=3`, and `sample_interval=10.0s`.
5656
- All rows use `--num-ware 1`, `--max-retry 5`, and TPCC's default 720-second measure time.
5757
- SQLite rows use the `balanced` and `practical` profiles respectively.
@@ -62,11 +62,11 @@ Transaction Summary (elapsed 720.0s)
6262
+--------------+---------+------+---------+-------+
6363
| Transaction | Success | Late | Failure | Total |
6464
+--------------+---------+------+---------+-------+
65-
| New-Order | 856143 | 0 | 8829 | 864972 |
66-
| Payment | 856120 | 0 | 0 | 856120 |
67-
| Order-Status | 85612 | 0 | 0 | 85612 |
68-
| Delivery | 85612 | 0 | 0 | 85612 |
69-
| Stock-Level | 85612 | 0 | 0 | 85612 |
65+
| New-Order | 994454 | 0 | 10137 | 1004591 |
66+
| Payment | 994430 | 0 | 0 | 994430 |
67+
| Order-Status | 99443 | 0 | 0 | 99443 |
68+
| Delivery | 99443 | 0 | 0 | 99443 |
69+
| Stock-Level | 99443 | 0 | 0 | 99443 |
7070
+--------------+---------+------+---------+-------+
7171
<Constraint Check> (all must be [OK])
7272
[transaction percentage]
@@ -86,35 +86,42 @@ Transaction Summary (elapsed 720.0s)
8686

8787
1.New-Order
8888

89-
0.001, 856143
89+
0.001, 994388
90+
0.002, 64
91+
0.003, 1
92+
0.004, 1
9093

9194
2.Payment
9295

93-
0.001, 856120
96+
0.001, 994428
97+
0.002, 2
9498

9599
3.Order-Status
96100

97-
0.001, 83223
98-
0.002, 2103
99-
0.003, 286
101+
0.001, 96207
102+
0.002, 2704
103+
0.003, 532
100104

101105
4.Delivery
102106

103-
0.002, 85606
104-
0.003, 6
107+
0.001, 89134
108+
0.002, 10294
109+
0.003, 12
110+
0.004, 2
111+
0.005, 1
105112

106113
5.Stock-Level
107114

108-
0.001, 85612
115+
0.001, 99443
109116

110117
<90th Percentile RT (MaxRT)>
111-
New-Order : 0.001 (0.001)
112-
Payment : 0.001 (0.000)
113-
Order-Status : 0.001 (0.002)
114-
Delivery : 0.002 (0.002)
115-
Stock-Level : 0.001 (0.000)
118+
New-Order : 0.001 (0.003)
119+
Payment : 0.001 (0.001)
120+
Order-Status : 0.001 (0.003)
121+
Delivery : 0.002 (0.004)
122+
Stock-Level : 0.001 (0.001)
116123
<TpmC>
117-
71345 Tpmc
124+
82871 Tpmc
118125
```
119126

120127
### KiteSQL RocksDB
@@ -123,11 +130,11 @@ Transaction Summary (elapsed 720.0s)
123130
+--------------+---------+------+---------+-------+
124131
| Transaction | Success | Late | Failure | Total |
125132
+--------------+---------+------+---------+-------+
126-
| New-Order | 486763 | 0 | 5029 | 491792 |
127-
| Payment | 486739 | 0 | 0 | 486739 |
128-
| Order-Status | 48674 | 0 | 0 | 48674 |
129-
| Delivery | 48674 | 0 | 0 | 48674 |
130-
| Stock-Level | 48674 | 0 | 0 | 48674 |
133+
| New-Order | 491520 | 0 | 4973 | 496493 |
134+
| Payment | 491493 | 0 | 0 | 491493 |
135+
| Order-Status | 49150 | 0 | 0 | 49150 |
136+
| Delivery | 49150 | 0 | 0 | 49150 |
137+
| Stock-Level | 49149 | 0 | 0 | 49149 |
131138
+--------------+---------+------+---------+-------+
132139
<Constraint Check> (all must be [OK])
133140
[transaction percentage]
@@ -147,53 +154,58 @@ Transaction Summary (elapsed 720.0s)
147154

148155
1.New-Order
149156

150-
0.001, 486709
151-
0.002, 54
157+
0.001, 491385
158+
0.002, 131
159+
0.003, 4
152160

153161
2.Payment
154162

155-
0.001, 486736
156-
0.002, 2
163+
0.001, 491475
164+
0.002, 15
165+
0.003, 1
157166

158167
3.Order-Status
159168

160-
0.001, 43202
161-
0.002, 4215
162-
0.003, 717
163-
0.004, 395
164-
0.005, 137
165-
0.006, 7
166-
0.007, 1
169+
0.001, 44281
170+
0.002, 3740
171+
0.003, 721
172+
0.004, 320
173+
0.005, 80
167174

168175
4.Delivery
169176

170-
0.002, 3572
171-
0.003, 8048
172-
0.004, 7083
173-
0.005, 4455
174-
0.006, 5238
175-
0.007, 6637
176-
0.008, 6138
177-
0.009, 4702
178-
0.010, 2626
179-
0.011, 164
180-
0.012, 4
181-
0.013, 3
182-
0.014, 3
183-
0.022, 1
177+
0.002, 5008
178+
0.003, 6120
179+
0.004, 5573
180+
0.005, 2405
181+
0.006, 4233
182+
0.007, 4778
183+
0.008, 5329
184+
0.009, 5870
185+
0.010, 4851
186+
0.011, 3188
187+
0.012, 1585
188+
0.013, 191
189+
0.014, 4
190+
0.015, 2
191+
0.016, 5
192+
0.017, 2
193+
0.018, 4
194+
0.019, 1
195+
0.020, 1
184196

185197
5.Stock-Level
186198

187-
0.001, 48674
199+
0.001, 49149
188200

189201
<90th Percentile RT (MaxRT)>
190-
New-Order : 0.001 (0.001)
202+
New-Order : 0.001 (0.003)
191203
Payment : 0.001 (0.007)
192-
Order-Status : 0.002 (0.007)
193-
Delivery : 0.009 (0.022)
204+
Order-Status : 0.001 (0.008)
205+
Delivery : 0.011 (0.019)
194206
Stock-Level : 0.001 (0.001)
195207
<TpmC>
196-
40563 Tpmc
208+
40960 Tpmc
197209
```
198210

199211
### SQLite balanced
@@ -202,11 +214,11 @@ Transaction Summary (elapsed 720.0s)
202214
+--------------+---------+------+---------+-------+
203215
| Transaction | Success | Late | Failure | Total |
204216
+--------------+---------+------+---------+-------+
205-
| New-Order | 810343 | 0 | 8322 | 818665 |
206-
| Payment | 810320 | 0 | 0 | 810320 |
207-
| Order-Status | 81032 | 0 | 0 | 81032 |
208-
| Delivery | 81032 | 0 | 0 | 81032 |
209-
| Stock-Level | 81032 | 0 | 0 | 81032 |
217+
| New-Order | 619643 | 0 | 6299 | 625942 |
218+
| Payment | 619618 | 0 | 0 | 619618 |
219+
| Order-Status | 61961 | 0 | 0 | 61961 |
220+
| Delivery | 61962 | 0 | 0 | 61962 |
221+
| Stock-Level | 61962 | 0 | 0 | 61962 |
210222
+--------------+---------+------+---------+-------+
211223
<Constraint Check> (all must be [OK])
212224
[transaction percentage]
@@ -226,33 +238,34 @@ Transaction Summary (elapsed 720.0s)
226238

227239
1.New-Order
228240

229-
0.001, 810329
230-
0.002, 14
241+
0.001, 619561
242+
0.002, 82
231243

232244
2.Payment
233245

234-
0.001, 810320
246+
0.001, 619618
235247

236248
3.Order-Status
237249

238-
0.001, 81032
250+
0.001, 61961
239251

240252
4.Delivery
241253

242-
0.001, 81032
254+
0.001, 61956
255+
0.002, 6
243256

244257
5.Stock-Level
245258

246-
0.001, 81032
259+
0.001, 61962
247260

248261
<90th Percentile RT (MaxRT)>
249262
New-Order : 0.001 (0.001)
250-
Payment : 0.001 (0.000)
263+
Payment : 0.001 (0.001)
251264
Order-Status : 0.001 (0.000)
252265
Delivery : 0.001 (0.001)
253266
Stock-Level : 0.001 (0.000)
254267
<TpmC>
255-
67527 Tpmc
268+
51637 Tpmc
256269
```
257270

258271
### SQLite practical
@@ -261,11 +274,11 @@ Transaction Summary (elapsed 720.0s)
261274
+--------------+---------+------+---------+-------+
262275
| Transaction | Success | Late | Failure | Total |
263276
+--------------+---------+------+---------+-------+
264-
| New-Order | 777304 | 0 | 8128 | 785432 |
265-
| Payment | 777280 | 0 | 0 | 777280 |
266-
| Order-Status | 77728 | 0 | 0 | 77728 |
267-
| Delivery | 77728 | 0 | 0 | 77728 |
268-
| Stock-Level | 77728 | 0 | 0 | 77728 |
277+
| New-Order | 737086 | 0 | 7349 | 744435 |
278+
| Payment | 737064 | 0 | 0 | 737064 |
279+
| Order-Status | 73706 | 0 | 0 | 73706 |
280+
| Delivery | 73706 | 0 | 0 | 73706 |
281+
| Stock-Level | 73706 | 0 | 0 | 73706 |
269282
+--------------+---------+------+---------+-------+
270283
<Constraint Check> (all must be [OK])
271284
[transaction percentage]
@@ -285,33 +298,33 @@ Transaction Summary (elapsed 720.0s)
285298

286299
1.New-Order
287300

288-
0.001, 777302
289-
0.002, 2
301+
0.001, 737069
302+
0.002, 17
290303

291304
2.Payment
292305

293-
0.001, 777280
306+
0.001, 737064
294307

295308
3.Order-Status
296309

297-
0.001, 77728
310+
0.001, 73706
298311

299312
4.Delivery
300313

301-
0.001, 77728
314+
0.001, 73706
302315

303316
5.Stock-Level
304317

305-
0.001, 77728
318+
0.001, 73706
306319

307320
<90th Percentile RT (MaxRT)>
308321
New-Order : 0.001 (0.001)
309-
Payment : 0.001 (0.000)
322+
Payment : 0.001 (0.001)
310323
Order-Status : 0.001 (0.000)
311324
Delivery : 0.001 (0.001)
312325
Stock-Level : 0.001 (0.000)
313326
<TpmC>
314-
64774 Tpmc
327+
61424 Tpmc
315328
```
316329

317330
## Refer to

0 commit comments

Comments
 (0)