Refactor documentation and examples to promote chunked transactions over batch_context for embedded workloads

Author: tae898

bindings/python/docs/api/batch.md

@@ -12,6 +12,11 @@ The `BatchContext` class simplifies bulk data operations by:
 - **Context Manager**: Automatic cleanup and completion waiting
 - **High Performance**: 50,000-200,000 records/sec throughput
 
+> **Embedded note:** For embedded workloads, explicit chunked transactions (`with
+> db.transaction():` in fixed-size slices) currently outperform `batch_context`. Use
+> `batch_context` primarily for API demonstrations or tests until performance parity is
+> improved.
+
 ## Class: BatchContext
 
 High-level batch processing context manager.

bindings/python/docs/api/database.md

@@ -518,32 +518,6 @@ db.async_executor() -> AsyncExecutor
 
 ---
 
-### batch_context
-
-```python
-db.batch_context(
-    batch_size: int = 5000,
-    parallel: int = 4,
-    use_wal: bool = True,
-    back_pressure: int = 50,
-    progress: bool = False,
-    progress_desc: str = "Processing",
-)
-```
-
-Convenience context for bulk ingestion built on the async executor (auto-commit, back-pressure, optional progress).
-
-**Example:**
-
-```python
-with db.batch_context(batch_size=5000, parallel=8, progress=True) as batch:
-    batch.set_total(len(users))
-    for user in users:
-        batch.create_vertex("User", **user)
-```
-
----
-
 ### export_database
 
 ```python

bindings/python/docs/development/testing/test-exporter.md

@@ -2,7 +2,7 @@
 
 [View source code](https://github.com/humemai/arcadedb-embedded-python/blob/main/bindings/python/tests/test_exporter.py){ .md-button }
 
-These notes mirror the Python tests in [test_exporter.py](https://github.com/humemai/arcadedb-embedded-python/blob/main/bindings/python/tests/test_exporter.py). There are 5 test classes with 12+ tests covering JSONL (with type/edge filters), GraphML/GraphSON (skipped if Gremlin unavailable), CSV, round-trip (export→import), batch-context integration, and all data types.
+These notes mirror the Python tests in [test_exporter.py](https://github.com/humemai/arcadedb-embedded-python/blob/main/bindings/python/tests/test_exporter.py). There are 5 test classes with 12+ tests covering JSONL (with type/edge filters), GraphML/GraphSON (skipped if Gremlin unavailable), CSV, round-trip (export→import), bulk insert (chunked transactions), and all data types.
 
 ## Test Classes & Cases
 
@@ -42,9 +42,9 @@ Fixture `sample_db` creates 20 users, 15 movies, 10 actors, 50 Rated edges, 30 A
 
 - **jsonl_export_import_roundtrip**: Exports sample_db to JSONL, closes original, creates new DB, imports via `IMPORT DATABASE file://...`, then verifies counts (User 20, Movie 15, Actor 10, LogEntry 10, Config 5) and data integrity (first user name is "User0"). See [test_exporter.py#L508-L567](https://github.com/humemai/arcadedb-embedded-python/blob/main/bindings/python/tests/test_exporter.py#L508-L567).
 
-### TestExportWithBatchContext
+### TestExportWithBulkInsert
 
-- **export_after_batch_insert**: Uses `batch_context(batch_size=100, parallel=2)` to create 500 Product vertices, exports to JSONL, asserts `vertices == 500`. See [test_exporter.py#L570-L593](https://github.com/humemai/arcadedb-embedded-python/blob/main/bindings/python/tests/test_exporter.py#L570-L593).
+- **export_after_chunked_insert**: Uses chunked transactions (no `batch_context`) to create 500 Product vertices, exports to JSONL, asserts `vertices == 500`. See [test_exporter.py#L567-L605](https://github.com/humemai/arcadedb-embedded-python/blob/main/bindings/python/tests/test_exporter.py#L567-L605).
 
 ### TestAllDataTypes
 

bindings/python/docs/development/testing/test-vector.md

@@ -18,7 +18,7 @@ What the tests cover:
 - ✅ **Overquery factor** tuning (`overquery_factor`)
 - ✅ **Distance functions** (cosine default, euclidean variants)
 - ✅ **Persistence & size checks** (index files survive reopen)
-- ✅ **Batch inserts** through `BatchContext`
+- ✅ **Chunked inserts** via explicit transactions (preferred for embedded)
 
 ## Test Coverage (high level)
 
@@ -85,27 +85,32 @@ with arcadedb.create_database("./test_db") as db:
     )
 ```
 
-### Batch insert vectors
+### Chunked insert vectors (preferred)
 ```python
 with arcadedb.create_database("./test_db") as db:
     # Schema operations are auto-transactional
     db.schema.create_vertex_type("Doc")
     db.schema.create_property("Doc", "docId", "INTEGER")
     db.schema.create_property("Doc", "embedding", "ARRAY_OF_FLOATS")
 
-    # Batch insert (auto-transactional)
+    # Prefer chunked transactions for embedded (avoids batch_context overhead)
     vectors = [[1.0, 0.0, 0.0], [0.0, 1.0, 0.0], [0.0, 0.0, 1.0]]
-    with db.batch_context(batch_size=1000, parallel=4) as batch:
-        for i, vec in enumerate(vectors):
-            batch.create_vertex("Doc", docId=i, embedding=vec)
+    chunk_size = 100
+    for start in range(0, len(vectors), chunk_size):
+        with db.transaction():
+            for idx, vec in enumerate(vectors[start : start + chunk_size]):
+                doc = db.new_vertex("Doc")
+                doc.set("docId", start + idx)
+                doc.set("embedding", vec)
+                doc.save()
 ```
 
 ## Key Takeaways
 
 1. JVector is fully Java-native and LSM-backed; no legacy hnswlib path remains.
 2. Use `allowed_rids` for pre-filtered searches and `overquery_factor` for recall/speed trade-offs.
 3. `max_connections` and `beam_width` map to JVector graph degree and search beam; tune per workload.
-4. All tests run through the Python bindings to ensure parity with the Java engine.
+4. Prefer chunked `db.transaction()` inserts for embedded workloads; reserve `batch_context` for legacy/tests that explicitly need it.
 
 ## See Also
 

bindings/python/docs/guide/core/queries.md

@@ -46,22 +46,23 @@ with db.transaction():
     bob.save()
 ```
 
-### Bulk Inserts with BatchContext
+### Bulk Inserts (preferred: chunked transactions)
 
 ```python
-# Efficient bulk insertion
-from jpype import JClass
-LocalDate = JClass("java.time.LocalDate")
-
-with db.batch_context(batch_size=100, parallel=2) as batch:
-    for name, age, city in people_data:
-        batch.create_vertex("Person",
-            name=name,
-            age=age,
-            city=city,
-            joined_date=LocalDate.parse("2024-01-15")
-        )
-# Automatically waits for completion
+# Efficient bulk insertion (embedded-friendly): use chunked transactions
+chunk_size = 500
+for start in range(0, len(people_data), chunk_size):
+    with db.transaction():
+        for name, age, city in people_data[start : start + chunk_size]:
+            person = db.new_vertex("Person")
+            person.set("name", name)
+            person.set("age", age)
+            person.set("city", city)
+            person.save()
+
+# If you specifically need the Java batching API (e.g., remote/server workflows),
+# you can still use db.batch_context(...), but embedded users should prefer
+# explicit chunked transactions to avoid batch_context overhead.
 ```
 
 ## SQL for Queries

bindings/python/docs/guide/core/transactions.md

@@ -1,8 +1,12 @@
 # Transactions
 
-Prefer the Pythonic wrappers (`new_vertex`, `new_document`, `new_edge`, `modify`,
-`batch_context`) over raw SQL. When you see `temp_db_path`, substitute your own path if
-you are not in the test harness.
+Prefer the Pythonic wrappers (`new_vertex`, `new_document`, `new_edge`, `modify`) over
+raw SQL. When you see `temp_db_path`, substitute your own path if you are not in the
+test harness.
+
+> **Embedded note:** For bulk ingest in embedded mode, explicit chunked transactions are
+> currently faster and more stable. Use `with db.transaction():` in fixed-size slices for
+> production ingest.
 
 ## Basic commit and rollback
 
@@ -113,30 +117,6 @@ with arcadedb.create_database(temp_db_path) as db:
     assert updated.get("country") == "USA"
 ```
 
-## Batch context: edge creation must be inside a transaction
-
-```python
-with db.transaction():
-    with db.batch_context(batch_size=10) as batch:
-        batch.create_edge(alice, bob, "KNOWS", since=2020)
-        batch.create_edge(bob, charlie, "KNOWS", since=2021)
-        batch.create_edge(charlie, alice, "KNOWS", since=2022)
-```
-
-## Batch updates with `modify()` + `batch_context`
-
-```python
-counters = list(db.query("sql", "SELECT FROM Counter"))
-
-with db.transaction():
-    with db.batch_context(batch_size=50) as batch:
-        for counter in counters:
-            vertex = counter.get_vertex()
-            mutable_vertex = vertex.modify()
-            mutable_vertex.set("value", counter.get("value") * 2)
-            batch.update_record(mutable_vertex._java_document)
-```
-
 ## Chunked bulk inserts with manual commit/renew
 
 ```python

bindings/python/examples/02_social_network_graph.py

@@ -257,57 +257,64 @@ def create_sample_data(db):
         ]
 
         print("  📝 Creating people...")
-        print("  💡 Using BatchContext for efficient bulk insertion")
+        print("  💡 Using chunked transactions for efficient bulk insertion")
 
         # Parse date strings to Java dates once (for reuse)
         from jpype import JClass
 
         LocalDate = JClass("java.time.LocalDate")
 
-        # Create people using BatchContext for cleaner, more efficient code
-        with db.batch_context(batch_size=100, parallel=2) as batch:
-            for person in people_data:
-                name, age, city, joined_date, email, phone, verified, reputation = (
-                    person
-                )
-
-                # Parse date string to Java date
-                date_obj = LocalDate.parse(joined_date)
-
-                # Create vertex properties dict
-                # (BatchContext handles None/NULL automatically)
-                properties = {
-                    "name": name,
-                    "age": age,
-                    "city": city,
-                    "joined_date": date_obj,
-                    "verified": verified,
-                }
-
-                # Add optional fields only if present
-                if email:
-                    properties["email"] = email
-                if phone:
-                    properties["phone"] = phone
-                if reputation is not None:
-                    properties["reputation"] = reputation
-
-                # Create vertex using batch context
-                # (automatically queued for async insertion)
-                batch.create_vertex("Person", **properties)
-
-                # Show which fields are NULL
-                null_fields = []
-                if not email:
-                    null_fields.append("email")
-                if not phone:
-                    null_fields.append("phone")
-                if reputation is None:
-                    null_fields.append("reputation")
-
-                null_str = f" (NULL: {', '.join(null_fields)})" if null_fields else ""
-                print(f"    ✓ Queued person: {name} ({age}, {city}){null_str}")
-        # BatchContext automatically waits for completion on exit
+        # Create people using chunked transactions (more stable for embedded)
+        chunk_size = 100
+        for start in range(0, len(people_data), chunk_size):
+            end = start + chunk_size
+            with db.transaction():
+                for person in people_data[start:end]:
+                    (
+                        name,
+                        age,
+                        city,
+                        joined_date,
+                        email,
+                        phone,
+                        verified,
+                        reputation,
+                    ) = person
+
+                    date_obj = LocalDate.parse(joined_date)
+
+                    properties = {
+                        "name": name,
+                        "age": age,
+                        "city": city,
+                        "joined_date": date_obj,
+                        "verified": verified,
+                    }
+
+                    if email:
+                        properties["email"] = email
+                    if phone:
+                        properties["phone"] = phone
+                    if reputation is not None:
+                        properties["reputation"] = reputation
+
+                    v = db.new_vertex("Person")
+                    for k, vval in properties.items():
+                        v.set(k, vval)
+                    v.save()
+
+                    null_fields = []
+                    if not email:
+                        null_fields.append("email")
+                    if not phone:
+                        null_fields.append("phone")
+                    if reputation is None:
+                        null_fields.append("reputation")
+
+                    null_str = (
+                        f" (NULL: {', '.join(null_fields)})" if null_fields else ""
+                    )
+                    print(f"    ✓ Inserted person: {name} ({age}, {city}){null_str}")
 
         print("  ✅ All people created successfully")
 

bindings/python/examples/benchmark-vector/benchmark_vector_params.py

@@ -222,9 +222,17 @@ def setup_database(db, data):
     db.schema.create_property("VectorData", "id", "INTEGER")
     db.schema.create_property("VectorData", "vector", "ARRAY_OF_FLOATS")
 
-    with db.batch_context(batch_size=10000) as batch:
-        for i, vec in enumerate(data):
-            batch.create_vertex("VectorData", id=i, vector=vec)
+    # Use explicit chunked transactions for ingest (faster than batch_context in
+    # embedded mode)
+    chunk_size = 10000
+    for start in range(0, len(data), chunk_size):
+        end = start + chunk_size
+        with db.transaction():
+            for i, vec in enumerate(data[start:end], start=start):
+                v = db.new_vertex("VectorData")
+                v.set("id", i)
+                v.set("vector", vec)
+                v.save()
 
     setup_time = time.perf_counter() - start_setup
     print(f"  Database setup completed in {setup_time:.4f}s")