22
33[ View source code] ({{ config.repo_url }}/blob/{{ config.extra.version_tag }}/bindings/python/examples/07_stackoverflow_tables_oltp.py){ .md-button }
44
5- This example loads Stack Overflow XML tables and runs a mixed OLTP workload with
6- point-oriented CRUD operations .
5+ This example loads the Stack Overflow XML tables into a relational-style layout and
6+ runs a point-oriented CRUD workload .
77
88## Overview
99
10- Example 07 is the table-oriented OLTP benchmark in the Python examples set .
10+ Example 07 is the table OLTP benchmark for the Stack Overflow dataset .
1111
12- - Loads the Stack Overflow XML tables into the selected backend
13- - Runs a mixed CRUD operation stream against one table at a time
14- - Measures throughput, latency, load time, disk usage, and peak RSS
15- - Supports deterministic single-thread verification for repeatability checks
12+ - It loads eight XML-derived tables.
13+ - It uses a fixed mixed workload: 60% read, 20% update, 10% insert, 10% delete.
14+ - Every operation picks one table and one target row at random.
15+ - It measures preload time, index time, OLTP latency, throughput, disk usage, and RSS.
1616
17- ## Current Repository Guidance
17+ ## Source Tables
1818
19- - ArcadeDB preload uses async SQL insert for the initial document load
20- - Keep that preload path on a single async worker; do not rely on multi-threaded async
21- insert for this workload in the current Python examples
22- - For cross-database comparability, ` --threads 1 ` is the recommended baseline
23- - ` --verify-single-thread-series ` checks deterministic repeatability for a single
24- backend configuration, not strict cross-database equality
19+ The example defines these tables directly in ` TABLE_DEFS ` .
20+
21+ | Table | Source file | Columns |
22+ | --- | --- | --- |
23+ | ` User ` | ` Users.xml ` | ` Id ` , ` Reputation ` , ` CreationDate ` , ` DisplayName ` , ` LastAccessDate ` , ` WebsiteUrl ` , ` Location ` , ` AboutMe ` , ` Views ` , ` UpVotes ` , ` DownVotes ` , ` AccountId ` |
24+ | ` Post ` | ` Posts.xml ` | ` Id ` , ` PostTypeId ` , ` ParentId ` , ` AcceptedAnswerId ` , ` CreationDate ` , ` Score ` , ` ViewCount ` , ` Body ` , ` OwnerUserId ` , ` LastEditorUserId ` , ` LastEditorDisplayName ` , ` LastEditDate ` , ` LastActivityDate ` , ` Title ` , ` Tags ` , ` AnswerCount ` , ` CommentCount ` , ` FavoriteCount ` , ` ClosedDate ` , ` CommunityOwnedDate ` |
25+ | ` Comment ` | ` Comments.xml ` | ` Id ` , ` PostId ` , ` Score ` , ` Text ` , ` CreationDate ` , ` UserId ` |
26+ | ` Badge ` | ` Badges.xml ` | ` Id ` , ` UserId ` , ` Name ` , ` Date ` , ` Class ` , ` TagBased ` |
27+ | ` Vote ` | ` Votes.xml ` | ` Id ` , ` PostId ` , ` VoteTypeId ` , ` CreationDate ` , ` UserId ` , ` BountyAmount ` |
28+ | ` PostLink ` | ` PostLinks.xml ` | ` Id ` , ` CreationDate ` , ` PostId ` , ` RelatedPostId ` , ` LinkTypeId ` |
29+ | ` Tag ` | ` Tags.xml ` | ` Id ` , ` TagName ` , ` Count ` , ` ExcerptPostId ` , ` WikiPostId ` |
30+ | ` PostHistory ` | ` PostHistory.xml ` | ` Id ` , ` PostHistoryTypeId ` , ` PostId ` , ` RevisionGUID ` , ` CreationDate ` , ` UserId ` , ` UserDisplayName ` , ` Comment ` , ` Text ` , ` CloseReasonId ` |
31+
32+ Each backend also creates a unique ` Id ` index for every table.
2533
2634## Supported Backends
2735
@@ -45,18 +53,203 @@ python 07_stackoverflow_tables_oltp.py \
4553 --verify-single-thread-series
4654```
4755
48- ## Key Options
56+ ## Workload Model
57+
58+ The operation planner is fixed in source as:
59+
60+ ``` python
61+ DEFAULT_OLTP_MIX = {" read" : 0.60 , " update" : 0.20 , " insert" : 0.10 , " delete" : 0.10 }
62+ ```
63+
64+ The selected table for each operation is random across all eight tables.
65+
66+ ### Read Projections
67+
68+ Reads always filter on ` Id ` , but the projected columns vary by table. The source code
69+ uses ` get_read_projection() ` , which chooses one of two projections per table.
70+
71+ | Table | Projection A | Projection B |
72+ | --- | --- | --- |
73+ | ` User ` | ` Id, Reputation, CreationDate ` | ` Id, Reputation, DisplayName ` |
74+ | ` Post ` | ` Id, PostTypeId, ParentId ` | ` Id, PostTypeId, AcceptedAnswerId ` |
75+ | ` Comment ` | ` Id, PostId, Score ` | ` Id, PostId, Text ` |
76+ | ` Badge ` | ` Id, UserId, Name ` | ` Id, UserId, Date ` |
77+ | ` Vote ` | ` Id, PostId, VoteTypeId ` | ` Id, PostId, CreationDate ` |
78+ | ` PostLink ` | ` Id, CreationDate, PostId ` | ` Id, CreationDate, RelatedPostId ` |
79+ | ` Tag ` | ` Id, TagName, Count ` | ` Id, TagName, ExcerptPostId ` |
80+ | ` PostHistory ` | ` Id, PostHistoryTypeId, PostId ` | ` Id, PostHistoryTypeId, RevisionGUID ` |
81+
82+ ### Update Targets
83+
84+ The update column is the first non-` Id ` field whose declared type is ` INTEGER ` or
85+ ` BOOLEAN ` .
86+
87+ | Table | Updated column |
88+ | --- | --- |
89+ | ` User ` | ` Reputation ` |
90+ | ` Post ` | ` PostTypeId ` |
91+ | ` Comment ` | ` PostId ` |
92+ | ` Badge ` | ` UserId ` |
93+ | ` Vote ` | ` PostId ` |
94+ | ` PostLink ` | ` PostId ` |
95+ | ` Tag ` | ` Count ` |
96+ | ` PostHistory ` | ` PostHistoryTypeId ` |
97+
98+ ### Insert Payloads
99+
100+ Insert rows are synthetic. The source uses ` build_synthetic_row() ` :
101+
102+ - ` Id ` is the next generated integer for that table.
103+ - ` INTEGER ` fields get a random value in ` [1, 1000] ` .
104+ - ` BOOLEAN ` fields get a random boolean.
105+ - ` DATETIME ` fields get the current UTC timestamp.
106+ - string fields get ` synthetic_<Table>_<Field>_<Id> ` .
107+
108+ ## Exact Query Templates
109+
110+ ### ArcadeDB SQL
111+
112+ The ArcadeDB path issues SQL directly.
113+
114+ #### ArcadeDB Read
115+
116+ ``` sql
117+ SELECT {projection} FROM {table_name} WHERE Id = {target_id}
118+ ```
119+
120+ #### ArcadeDB Update
121+
122+ ``` sql
123+ UPDATE {table_name} SET {update_col} = coalesce({update_col}, 0 ) + 1 WHERE Id = {target_id}
124+ ```
125+
126+ #### ArcadeDB Insert
127+
128+ ``` sql
129+ INSERT INTO {table_name} SET {col1} = ?, {col2} = ?, ...
130+ ```
131+
132+ The same statement shape is used both for preload batches and OLTP inserts.
133+
134+ #### ArcadeDB Delete
135+
136+ ``` sql
137+ DELETE FROM {table_name} WHERE Id = {target_id}
138+ ```
139+
140+ ### SQLite
141+
142+ The SQLite path uses parameterized SQL with quoted identifiers.
143+
144+ #### SQLite Read
145+
146+ ``` sql
147+ SELECT {projection} FROM " {table_name}" WHERE " Id" = ?
148+ ```
149+
150+ #### SQLite Update
151+
152+ ``` sql
153+ UPDATE " {table_name}" SET " {update_col}" = coalesce(" {update_col}" , 0 ) + 1 WHERE " Id" = ?
154+ ```
155+
156+ #### SQLite Insert
157+
158+ ``` sql
159+ INSERT INTO " {table_name}" (" col1" , " col2" , ...) VALUES (?, ?, ...)
160+ ```
161+
162+ #### SQLite Delete
163+
164+ ``` sql
165+ DELETE FROM " {table_name}" WHERE " Id" = ?
166+ ```
167+
168+ ### DuckDB
169+
170+ The DuckDB OLTP path uses the same SQL shapes as SQLite, with explicit transactions.
171+
172+ #### DuckDB Read
173+
174+ ``` sql
175+ SELECT {projection} FROM " {table_name}" WHERE " Id" = ?
176+ ```
177+
178+ #### DuckDB Update
179+
180+ ``` sql
181+ UPDATE " {table_name}" SET " {update_col}" = coalesce(" {update_col}" , 0 ) + 1 WHERE " Id" = ?
182+ ```
183+
184+ #### DuckDB Insert
185+
186+ ``` sql
187+ INSERT INTO " {table_name}" (" col1" , " col2" , ...) VALUES (?, ?, ...)
188+ ```
189+
190+ #### DuckDB Delete
191+
192+ ``` sql
193+ DELETE FROM " {table_name}" WHERE " Id" = ?
194+ ```
195+
196+ ### PostgreSQL
197+
198+ The PostgreSQL path uses quoted identifiers and ` %s ` parameters.
199+
200+ #### Read
201+
202+ ``` sql
203+ SELECT {projection_sql} FROM " {table_name}" WHERE " Id" = %s
204+ ```
205+
206+ The ` projection_sql ` string is built by quoting each projected column returned by
207+ ` get_read_projection() ` .
208+
209+ #### Update
210+
211+ ``` sql
212+ UPDATE " {table_name}" SET " {update_col}" = coalesce(" {update_col}" , 0 ) + 1 WHERE " Id" = %s
213+ ```
214+
215+ #### Insert
216+
217+ ``` sql
218+ INSERT INTO " {table_name}" (" col1" , " col2" , ...) VALUES (%s, %s, ...)
219+ ```
220+
221+ #### Delete
222+
223+ ``` sql
224+ DELETE FROM " {table_name}" WHERE " Id" = %s
225+ ```
226+
227+ ## Preload Paths
228+
229+ The benchmark does not use the same preload mechanism for every backend.
230+
231+ - ArcadeDB preload uses async ` INSERT INTO ... SET ... ` SQL.
232+ - SQLite preload uses batched ` INSERT INTO ... VALUES ... ` statements.
233+ - DuckDB preload uses per-table CSV materialization followed by:
234+
235+ ``` sql
236+ COPY " {table_name}" FROM ' {csv_path}' (AUTO_DETECT TRUE, HEADER TRUE)
237+ ```
238+
239+ - PostgreSQL preload writes CSV and then streams it through:
240+
241+ ``` sql
242+ COPY " {table_name}" (" col1" , " col2" , ...) FROM STDIN WITH (FORMAT CSV, HEADER TRUE)
243+ ```
49244
50- - ` --dataset ` : dataset size from ` stackoverflow-tiny ` through ` stackoverflow-full `
51- - ` --db ` : backend to test
52- - ` --threads ` : worker threads for the OLTP run
53- - ` --transactions ` : number of OLTP operations
54- - ` --batch-size ` : preload insert batch size
55- - ` --mem-limit ` : Docker and JVM memory budget
56- - ` --sqlite-profile ` : SQLite tuning profile when using SQLite
245+ Those load-path differences matter for ingest timing, but they do not change the
246+ OLTP CRUD statements listed above.
57247
58248## Result Notes
59249
60- - ` du_mib ` in the summarized outputs reflects real post-run filesystem usage
61- - ` disk_after_* ` fields in the JSON payload are benchmark-reported logical size values
62- - Per-operation latency is summarized from ` latency_summary.ops `
250+ - ` du_mib ` is real post-run filesystem usage.
251+ - ` disk_after_* ` fields are benchmark-reported logical size counters.
252+ - Per-operation latency is summarized from the recorded ` read ` , ` update ` , ` insert ` ,
253+ and ` delete ` latency buckets.
254+ - ` --verify-single-thread-series ` checks deterministic repeatability for one backend
255+ configuration, not strict equality across different engines.
0 commit comments