Skip to content

Commit 7c9222a

Browse files
committed
Refactor documentation for Stack Overflow benchmarks and vector index examples
- Updated Example 08 documentation to clarify the analytical SQL suite and query definitions. - Enhanced Example 09 documentation with detailed graph projection rules and validation checklist. - Expanded Example 10 documentation to include graph projection assumptions and schema used by the query suite. - Revised Example 11 documentation to clarify vector ingest and index build processes across multiple backends. - Improved Example 12 documentation to detail search performance evaluation logic and backend-specific operations.
1 parent 092851b commit 7c9222a

6 files changed

Lines changed: 1623 additions & 86 deletions

bindings/python/docs/examples/07_stackoverflow_tables_oltp.md

Lines changed: 218 additions & 25 deletions
Original file line numberDiff line numberDiff line change
@@ -2,26 +2,34 @@
22

33
[View source code]({{ config.repo_url }}/blob/{{ config.extra.version_tag }}/bindings/python/examples/07_stackoverflow_tables_oltp.py){ .md-button }
44

5-
This example loads Stack Overflow XML tables and runs a mixed OLTP workload with
6-
point-oriented CRUD operations.
5+
This example loads the Stack Overflow XML tables into a relational-style layout and
6+
runs a point-oriented CRUD workload.
77

88
## Overview
99

10-
Example 07 is the table-oriented OLTP benchmark in the Python examples set.
10+
Example 07 is the table OLTP benchmark for the Stack Overflow dataset.
1111

12-
- Loads the Stack Overflow XML tables into the selected backend
13-
- Runs a mixed CRUD operation stream against one table at a time
14-
- Measures throughput, latency, load time, disk usage, and peak RSS
15-
- Supports deterministic single-thread verification for repeatability checks
12+
- It loads eight XML-derived tables.
13+
- It uses a fixed mixed workload: 60% read, 20% update, 10% insert, 10% delete.
14+
- Every operation picks one table and one target row at random.
15+
- It measures preload time, index time, OLTP latency, throughput, disk usage, and RSS.
1616

17-
## Current Repository Guidance
17+
## Source Tables
1818

19-
- ArcadeDB preload uses async SQL insert for the initial document load
20-
- Keep that preload path on a single async worker; do not rely on multi-threaded async
21-
insert for this workload in the current Python examples
22-
- For cross-database comparability, `--threads 1` is the recommended baseline
23-
- `--verify-single-thread-series` checks deterministic repeatability for a single
24-
backend configuration, not strict cross-database equality
19+
The example defines these tables directly in `TABLE_DEFS`.
20+
21+
| Table | Source file | Columns |
22+
| --- | --- | --- |
23+
| `User` | `Users.xml` | `Id`, `Reputation`, `CreationDate`, `DisplayName`, `LastAccessDate`, `WebsiteUrl`, `Location`, `AboutMe`, `Views`, `UpVotes`, `DownVotes`, `AccountId` |
24+
| `Post` | `Posts.xml` | `Id`, `PostTypeId`, `ParentId`, `AcceptedAnswerId`, `CreationDate`, `Score`, `ViewCount`, `Body`, `OwnerUserId`, `LastEditorUserId`, `LastEditorDisplayName`, `LastEditDate`, `LastActivityDate`, `Title`, `Tags`, `AnswerCount`, `CommentCount`, `FavoriteCount`, `ClosedDate`, `CommunityOwnedDate` |
25+
| `Comment` | `Comments.xml` | `Id`, `PostId`, `Score`, `Text`, `CreationDate`, `UserId` |
26+
| `Badge` | `Badges.xml` | `Id`, `UserId`, `Name`, `Date`, `Class`, `TagBased` |
27+
| `Vote` | `Votes.xml` | `Id`, `PostId`, `VoteTypeId`, `CreationDate`, `UserId`, `BountyAmount` |
28+
| `PostLink` | `PostLinks.xml` | `Id`, `CreationDate`, `PostId`, `RelatedPostId`, `LinkTypeId` |
29+
| `Tag` | `Tags.xml` | `Id`, `TagName`, `Count`, `ExcerptPostId`, `WikiPostId` |
30+
| `PostHistory` | `PostHistory.xml` | `Id`, `PostHistoryTypeId`, `PostId`, `RevisionGUID`, `CreationDate`, `UserId`, `UserDisplayName`, `Comment`, `Text`, `CloseReasonId` |
31+
32+
Each backend also creates a unique `Id` index for every table.
2533

2634
## Supported Backends
2735

@@ -45,18 +53,203 @@ python 07_stackoverflow_tables_oltp.py \
4553
--verify-single-thread-series
4654
```
4755

48-
## Key Options
56+
## Workload Model
57+
58+
The operation planner is fixed in source as:
59+
60+
```python
61+
DEFAULT_OLTP_MIX = {"read": 0.60, "update": 0.20, "insert": 0.10, "delete": 0.10}
62+
```
63+
64+
The selected table for each operation is random across all eight tables.
65+
66+
### Read Projections
67+
68+
Reads always filter on `Id`, but the projected columns vary by table. The source code
69+
uses `get_read_projection()`, which chooses one of two projections per table.
70+
71+
| Table | Projection A | Projection B |
72+
| --- | --- | --- |
73+
| `User` | `Id, Reputation, CreationDate` | `Id, Reputation, DisplayName` |
74+
| `Post` | `Id, PostTypeId, ParentId` | `Id, PostTypeId, AcceptedAnswerId` |
75+
| `Comment` | `Id, PostId, Score` | `Id, PostId, Text` |
76+
| `Badge` | `Id, UserId, Name` | `Id, UserId, Date` |
77+
| `Vote` | `Id, PostId, VoteTypeId` | `Id, PostId, CreationDate` |
78+
| `PostLink` | `Id, CreationDate, PostId` | `Id, CreationDate, RelatedPostId` |
79+
| `Tag` | `Id, TagName, Count` | `Id, TagName, ExcerptPostId` |
80+
| `PostHistory` | `Id, PostHistoryTypeId, PostId` | `Id, PostHistoryTypeId, RevisionGUID` |
81+
82+
### Update Targets
83+
84+
The update column is the first non-`Id` field whose declared type is `INTEGER` or
85+
`BOOLEAN`.
86+
87+
| Table | Updated column |
88+
| --- | --- |
89+
| `User` | `Reputation` |
90+
| `Post` | `PostTypeId` |
91+
| `Comment` | `PostId` |
92+
| `Badge` | `UserId` |
93+
| `Vote` | `PostId` |
94+
| `PostLink` | `PostId` |
95+
| `Tag` | `Count` |
96+
| `PostHistory` | `PostHistoryTypeId` |
97+
98+
### Insert Payloads
99+
100+
Insert rows are synthetic. The source uses `build_synthetic_row()`:
101+
102+
- `Id` is the next generated integer for that table.
103+
- `INTEGER` fields get a random value in `[1, 1000]`.
104+
- `BOOLEAN` fields get a random boolean.
105+
- `DATETIME` fields get the current UTC timestamp.
106+
- string fields get `synthetic_<Table>_<Field>_<Id>`.
107+
108+
## Exact Query Templates
109+
110+
### ArcadeDB SQL
111+
112+
The ArcadeDB path issues SQL directly.
113+
114+
#### ArcadeDB Read
115+
116+
```sql
117+
SELECT {projection} FROM {table_name} WHERE Id = {target_id}
118+
```
119+
120+
#### ArcadeDB Update
121+
122+
```sql
123+
UPDATE {table_name} SET {update_col} = coalesce({update_col}, 0) + 1 WHERE Id = {target_id}
124+
```
125+
126+
#### ArcadeDB Insert
127+
128+
```sql
129+
INSERT INTO {table_name} SET {col1} = ?, {col2} = ?, ...
130+
```
131+
132+
The same statement shape is used both for preload batches and OLTP inserts.
133+
134+
#### ArcadeDB Delete
135+
136+
```sql
137+
DELETE FROM {table_name} WHERE Id = {target_id}
138+
```
139+
140+
### SQLite
141+
142+
The SQLite path uses parameterized SQL with quoted identifiers.
143+
144+
#### SQLite Read
145+
146+
```sql
147+
SELECT {projection} FROM "{table_name}" WHERE "Id" = ?
148+
```
149+
150+
#### SQLite Update
151+
152+
```sql
153+
UPDATE "{table_name}" SET "{update_col}" = coalesce("{update_col}", 0) + 1 WHERE "Id" = ?
154+
```
155+
156+
#### SQLite Insert
157+
158+
```sql
159+
INSERT INTO "{table_name}" ("col1", "col2", ...) VALUES (?, ?, ...)
160+
```
161+
162+
#### SQLite Delete
163+
164+
```sql
165+
DELETE FROM "{table_name}" WHERE "Id" = ?
166+
```
167+
168+
### DuckDB
169+
170+
The DuckDB OLTP path uses the same SQL shapes as SQLite, with explicit transactions.
171+
172+
#### DuckDB Read
173+
174+
```sql
175+
SELECT {projection} FROM "{table_name}" WHERE "Id" = ?
176+
```
177+
178+
#### DuckDB Update
179+
180+
```sql
181+
UPDATE "{table_name}" SET "{update_col}" = coalesce("{update_col}", 0) + 1 WHERE "Id" = ?
182+
```
183+
184+
#### DuckDB Insert
185+
186+
```sql
187+
INSERT INTO "{table_name}" ("col1", "col2", ...) VALUES (?, ?, ...)
188+
```
189+
190+
#### DuckDB Delete
191+
192+
```sql
193+
DELETE FROM "{table_name}" WHERE "Id" = ?
194+
```
195+
196+
### PostgreSQL
197+
198+
The PostgreSQL path uses quoted identifiers and `%s` parameters.
199+
200+
#### Read
201+
202+
```sql
203+
SELECT {projection_sql} FROM "{table_name}" WHERE "Id" = %s
204+
```
205+
206+
The `projection_sql` string is built by quoting each projected column returned by
207+
`get_read_projection()`.
208+
209+
#### Update
210+
211+
```sql
212+
UPDATE "{table_name}" SET "{update_col}" = coalesce("{update_col}", 0) + 1 WHERE "Id" = %s
213+
```
214+
215+
#### Insert
216+
217+
```sql
218+
INSERT INTO "{table_name}" ("col1", "col2", ...) VALUES (%s, %s, ...)
219+
```
220+
221+
#### Delete
222+
223+
```sql
224+
DELETE FROM "{table_name}" WHERE "Id" = %s
225+
```
226+
227+
## Preload Paths
228+
229+
The benchmark does not use the same preload mechanism for every backend.
230+
231+
- ArcadeDB preload uses async `INSERT INTO ... SET ...` SQL.
232+
- SQLite preload uses batched `INSERT INTO ... VALUES ...` statements.
233+
- DuckDB preload uses per-table CSV materialization followed by:
234+
235+
```sql
236+
COPY "{table_name}" FROM '{csv_path}' (AUTO_DETECT TRUE, HEADER TRUE)
237+
```
238+
239+
- PostgreSQL preload writes CSV and then streams it through:
240+
241+
```sql
242+
COPY "{table_name}" ("col1", "col2", ...) FROM STDIN WITH (FORMAT CSV, HEADER TRUE)
243+
```
49244

50-
- `--dataset`: dataset size from `stackoverflow-tiny` through `stackoverflow-full`
51-
- `--db`: backend to test
52-
- `--threads`: worker threads for the OLTP run
53-
- `--transactions`: number of OLTP operations
54-
- `--batch-size`: preload insert batch size
55-
- `--mem-limit`: Docker and JVM memory budget
56-
- `--sqlite-profile`: SQLite tuning profile when using SQLite
245+
Those load-path differences matter for ingest timing, but they do not change the
246+
OLTP CRUD statements listed above.
57247

58248
## Result Notes
59249

60-
- `du_mib` in the summarized outputs reflects real post-run filesystem usage
61-
- `disk_after_*` fields in the JSON payload are benchmark-reported logical size values
62-
- Per-operation latency is summarized from `latency_summary.ops`
250+
- `du_mib` is real post-run filesystem usage.
251+
- `disk_after_*` fields are benchmark-reported logical size counters.
252+
- Per-operation latency is summarized from the recorded `read`, `update`, `insert`,
253+
and `delete` latency buckets.
254+
- `--verify-single-thread-series` checks deterministic repeatability for one backend
255+
configuration, not strict equality across different engines.

0 commit comments

Comments
 (0)