version for release

Author: carsontung666

README.md

@@ -8,23 +8,17 @@
   Store, navigate, and query hierarchical document structures with LLM-powered reasoning retrieval.
 </p>
 
-<h4 align="center">
-  <a href="https://github.com/VectifyAI/PageIndex">PageIndex</a>&nbsp; | &nbsp;
-  <a href="https://docs.pageindex.ai">Docs</a>&nbsp; | &nbsp;
-  <a href="https://discord.com/invite/VuXuf29EUj">Discord</a>
-</h4>
-
 </div>
 
 ---
 
 ## What is ConDB?
 
-**ConDB** is the storage and retrieval engine behind [PageIndex](https://github.com/VectifyAI/PageIndex). It stores hierarchical document trees (generated by PageIndex or other sources) in a SQLite database, and provides LLM-powered **reasoning-based retrieval** to query them — no vector DB, no chunking.
+**ConDB** stores hierarchical document trees in a SQLite database and provides LLM-powered **reasoning-based retrieval** to query them — no vector DB, no chunking. It accepts pageindex-compatible trees, chat trees, and custom hierarchical JSON without taking a runtime code dependency on PageIndex itself.
 
 **Key capabilities:**
 
-- **Hierarchical storage** — store tree-structured documents (PDFs, Markdown, custom JSON) in SQLite
+- **Hierarchical storage** — store document trees, chat trees, and custom hierarchical JSON in SQLite
 - **Reasoning-based retrieval** — LLM navigates the tree to find relevant content, like a human expert
 - **Multiple retrieval strategies** — beam search for small trees, block retrieval for large documents
 - **Multi-provider LLM support** — works with Anthropic (Claude) and OpenAI (GPT) out of the box
@@ -38,7 +32,6 @@
 
 ```bash
 pip install -r requirements.txt
-pip install pageindex  # optional, for PDF/Markdown indexing
 ```
 
 ### Basic Usage
@@ -52,29 +45,28 @@ db = contextdb.open("my_docs.sqlite")
 # Configure LLM
 db.set_llm(provider="anthropic", model="claude-sonnet-4-20250514")
 
-# Store a PageIndex tree
-tree_id = db.store(pageindex_json, format="pageindex")
+# Store a document tree
+tree_id = db.store(document_tree_json, format="document")
 
 # Query with LLM reasoning
 result = db.query(tree_id, "What are the key findings?")
 print(result.contents)
 ```
 
-### Index from files (requires `pageindex`)
+### Index from files with an external tree builder
 
 ```python
-from contextdb import ContextTree, LLMClient
+from contextdb import ContextTree
 
-llm = LLMClient("anthropic", api_key="sk-...", model="claude-sonnet-4-20250514")
-ct = ContextTree("context.sqlite", llm=llm)
+def build_markdown_tree(path: str) -> dict:
+    ...
 
-# Index documents
-tree_id = ct.index_pdf_file("report.pdf")
-tree_id = ct.index_markdown_file("doc.md")
+ct = ContextTree("context.sqlite")
 
-# Query
-result = ct.query(tree_id, "What are the main topics?", use_llm=True, max_turns=5)
-print(result.contents)
+tree_id = ct.index_markdown_file("doc.md", tree_builder=build_markdown_tree)
+
+# You can also generate a tree out of process and call:
+# tree_id = ct.index_document_tree(document_tree_json)
 
 ct.close()
 ```
@@ -118,17 +110,37 @@ result = db.query(tree_id, "question", strategy="block", beam_size=3)
 
 ---
 
+## Benchmark Snapshot
+
+Current filesystem benchmark summary lives in [bench/fs_block_beam_vertical.md](bench/fs_block_beam_vertical.md).
+
+Run setup for the snapshot below: `beam_size=3`, `max_turns=10`, `5` filesystem queries on `context7` only.
+
+### Block vs Beam vs Vertical
+
+| Retriever | Avg Time (s) | Avg LLM Calls | Hit@1 | Hit@10 | Total Cost (USD) |
+|---|---:|---:|---:|---:|---:|
+| **Block** | 5.47 | 1.00 | 1.00 | 1.00 | 0.0762 |
+| **Vertical** | 7.31 | 1.60 | 1.00 | 1.00 | 0.1486 |
+| **Beam** | 20.18 | 4.60 | 0.60 | 0.80 | 0.1328 |
+
+`Block` is the best default on this `context7` snapshot: same retrieval quality as `Vertical`, with lower latency and fewer model calls. `Beam` is still workable, but it trails clearly on retrieval accuracy.
+
+These numbers are benchmark snapshots, not hard guarantees; exact cost and latency will vary with model choice, provider pricing, prompt-cache behavior, and corpus shape.
+
+---
+
 ## Architecture
 
 ```
 contextdb/
 ├── api/
 │   ├── condb.py          # ConDB — main entry point
-│   └── context_tree.py   # ContextTree — file indexing API
+│   └── context_tree.py   # ContextTree — tree indexing + query API
 ├── core/
 │   └── storage.py        # TreeDB (SQLite), StorageProtocol
 ├── adapter/
-│   └── base.py           # PageIndex, ChatIndex, Generic adapters
+│   └── base.py           # DocumentTree, ChatIndex, Generic adapters
 ├── retriever/
 │   ├── base.py           # Retriever protocols
 │   └── algorithm/        # Beam, Block retrieval strategies
@@ -182,7 +194,7 @@ ct = ContextTree("db.sqlite", llm=MyLLM())
 
 ## Related Projects
 
-- [**PageIndex**](https://github.com/VectifyAI/PageIndex) — reasoning-based RAG framework that generates hierarchical tree indices from documents
+- [**PageIndex**](https://github.com/VectifyAI/PageIndex) — one possible external producer of pageindex-compatible document trees
 - [**AgentFS**](https://github.com/anthropics/agentfs) — filesystem for AI agents
 
 ---
@@ -191,7 +203,7 @@ ct = ContextTree("db.sqlite", llm=MyLLM())
 
 ## License
 
-MIT
+Apache-2.0
 
 <br/>
 

bench/benchmark_retrievers.py

@@ -10,13 +10,11 @@
         python bench/benchmark_retrievers.py --mode doc --doc <document.json> --config <queries.json>
 
     Filesystem mode:
-        python bench/benchmark_retrievers.py --mode fs --repo-dir <path> --queries-config <queries.json>
+        python bench/benchmark_retrievers.py --mode fs --fs-root <path> --queries-config <queries.json>
 
 Examples:
     python bench/benchmark_retrievers.py --mode doc --doc examples/large_doc.json --config bench/queries.json
-    python bench/benchmark_retrievers.py --mode fs --repo-dir bench/filesystem/repo --queries-config bench/filesystem/repo/queries.json
-    python bench/benchmark_retrievers.py --mode fs --repo-dir bench/filesystem/arxiv --queries-config bench/filesystem/arxiv/queries.json
-    python bench/benchmark_retrievers.py --mode fs --repo-dir bench/filesystem/context7 --queries-config bench/filesystem/context7/queries.json
+    python bench/benchmark_retrievers.py --mode fs --fs-root bench/filesystem/context7 --queries-config bench/filesystem/context7/queries.json
 """
 
 import argparse
@@ -27,7 +25,7 @@
 import time
 from dataclasses import asdict, dataclass, field
 from pathlib import Path
-from typing import Any, Optional
+from typing import Optional
 
 sys.path.insert(0, str(Path(__file__).parent.parent))
 
@@ -38,6 +36,8 @@
 
 ALGORITHM_DIR = Path(__file__).parent.parent / "contextdb/retriever/algorithm"
 EXCLUDED_FILES = {"base_retriever.py", "block_cutter.py", "block_types.py", "__init__.py"}
+FS_HIT_LEVELS = (1, 3, 5, 10)
+FS_DISPLAY_HIT_LEVELS = (1, 10)
 
 
 def discover_retrievers() -> dict[str, type]:
@@ -91,7 +91,6 @@ class RetrieverMetrics:
     # FS-specific
     retrieved_files: Optional[list[str]] = None
     hit_at: Optional[dict[int, bool]] = None
-    mrr: Optional[float] = None
 
 
 @dataclass
@@ -145,11 +144,9 @@ def total(items, attr):
 
             # FS hit metrics
             if self.mode == "fs":
-                for k in [1, 3, 5, 10]:
+                for k in FS_HIT_LEVELS:
                     hits = [r for r in valid if r.hit_at and k in r.hit_at]
                     s[f"hit@{k}"] = sum(1 for r in hits if r.hit_at[k]) / len(hits) if hits else 0
-                mrr_vals = [r.mrr for r in valid if r.mrr is not None]
-                s["mrr"] = sum(mrr_vals) / len(mrr_vals) if mrr_vals else 0
 
             summary[name] = s
 
@@ -205,25 +202,21 @@ def build_entities(flat_list: list) -> dict:
 # ── FS Mode Helpers ─────────────────────────────────────────────────
 
 
-def compute_fs_metrics(retrieved_files: list[str], ground_truth: list[str]) -> tuple[dict[int, bool], float]:
-    """Returns (hit_at_k_dict, mrr)."""
+def compute_fs_metrics(retrieved_files: list[str], ground_truth: list[str]) -> dict[int, bool]:
+    """Returns hit-at-k booleans for the retrieved file list."""
     gt_set = set(ground_truth)
     hit_at = {}
-    first_hit_rank = None
 
     for i, f in enumerate(retrieved_files):
-        if f in gt_set and first_hit_rank is None:
-            first_hit_rank = i + 1
-        for k in [1, 3, 5, 10]:
+        for k in FS_HIT_LEVELS:
             if i < k and f in gt_set:
                 hit_at.setdefault(k, False)
                 hit_at[k] = True
 
-    for k in [1, 3, 5, 10]:
+    for k in FS_HIT_LEVELS:
         hit_at.setdefault(k, False)
 
-    mrr = 1.0 / first_hit_rank if first_hit_rank else 0.0
-    return hit_at, mrr
+    return hit_at
 
 
 def extract_file_paths(db: TreeDB, tree_id: str, node_ids: list[str]) -> list[str]:
@@ -299,10 +292,9 @@ def run_retriever(
 
         if mode == "fs" and db and ground_truth:
             retrieved_files = extract_file_paths(db, tree_id, result.nodes)
-            hit_at, mrr = compute_fs_metrics(retrieved_files, ground_truth)
+            hit_at = compute_fs_metrics(retrieved_files, ground_truth)
             metrics.retrieved_files = retrieved_files
             metrics.hit_at = hit_at
-            metrics.mrr = mrr
 
         return metrics
     except Exception as e:
@@ -328,7 +320,7 @@ def run_benchmark(
     *,
     mode: str = "doc",
     doc_path: Path = None,
-    repo_dir: Path = None,
+    fs_root: Path = None,
     query_list: list = None,
     queries_with_gt: list[dict] = None,
     queries_config_path: Path = None,
@@ -382,25 +374,25 @@ def make_tree(db):
         ignore_patterns = list(DEFAULT_IGNORE_PATTERNS)
         effective_queries_config = queries_config_path
         if effective_queries_config is None:
-            default_queries = repo_dir / "queries.json"
+            default_queries = fs_root / "queries.json"
             if default_queries.exists():
                 effective_queries_config = default_queries
 
         if effective_queries_config is not None:
             try:
-                rel_cfg = str(effective_queries_config.resolve().relative_to(repo_dir.resolve())).replace("\\", "/")
+                rel_cfg = str(effective_queries_config.resolve().relative_to(fs_root.resolve())).replace("\\", "/")
                 ignore_patterns.append(rel_cfg)
             except ValueError:
                 pass
 
         if fs_query_order == "prefix":
             queries_with_gt = reorder_fs_queries_by_prefix(queries_with_gt)
 
-        adapter = FileSystemAdapter(str(repo_dir), ignore_patterns=ignore_patterns)
+        adapter = FileSystemAdapter(str(fs_root), ignore_patterns=ignore_patterns)
         tree_structure, entities = adapter.convert()
         queries = [q["query"] for q in queries_with_gt]
         ground_truths = [q["ground_truth"] for q in queries_with_gt]
-        doc_info = str(repo_dir)
+        doc_info = str(fs_root)
         sections_count = len(entities)
 
         def make_tree(db):
@@ -462,7 +454,7 @@ def make_tree(db):
                     parts.append(f"Cache read: {metrics.cache_read_tokens:,}")
                 if mode == "fs" and metrics.hit_at:
                     parts.append(f"Hit@1: {metrics.hit_at.get(1, False)}")
-                    parts.append(f"MRR: {metrics.mrr:.3f}")
+                    parts.append(f"Hit@10: {metrics.hit_at.get(10, False)}")
                     if metrics.retrieved_files:
                         parts.append(f"Files: {metrics.retrieved_files[:5]}")
                 print(f"    {', '.join(parts)}")
@@ -486,7 +478,7 @@ def print_summary(result: BenchmarkResult):
     print(title)
     print("=" * 70)
     print(f"\nModel: {Config.LLM_PROVIDER}/{Config.LLM_MODEL}")
-    print(f"{'Repository' if result.mode == 'fs' else 'Document'}: {result.document_path}")
+    print(f"{'Filesystem Root' if result.mode == 'fs' else 'Document'}: {result.document_path}")
     print(f"Entities: {result.document_sections}")
     print(f"Queries: {summary['queries_run']}")
     print(f"Retrievers: {', '.join(result.retriever_names)}")
@@ -508,16 +500,12 @@ def print_summary(result: BenchmarkResult):
         rows.append(row)
 
     if result.mode == "fs":
-        for k in [1, 3, 5, 10]:
+        for k in FS_DISPLAY_HIT_LEVELS:
             row = [f"Hit@{k}"]
             for name in result.retriever_names:
                 val = summary[name].get(f"hit@{k}", 0)
                 row.append(f"{val:.1%}")
             rows.append(row)
-        mrr_row = ["MRR"]
-        for name in result.retriever_names:
-            mrr_row.append(f"{summary[name].get('mrr', 0):.3f}")
-        rows.append(mrr_row)
 
     col_widths = [max(len(str(row[i])) for row in [headers] + rows) for i in range(len(headers))]
     header_line = " | ".join(h.ljust(w) for h, w in zip(headers, col_widths))
@@ -549,8 +537,8 @@ def main():
                         help="Path to document JSON file (doc mode)")
     parser.add_argument("--config", "-c", type=Path,
                         help="Path to queries config JSON (doc mode)")
-    parser.add_argument("--repo-dir", type=Path,
-                        help="Path to repository directory (fs mode)")
+    parser.add_argument("--fs-root", type=Path,
+                        help="Path to filesystem root directory (fs mode)")
     parser.add_argument("--queries-config", type=Path,
                         help="Path to queries+ground_truth JSON (fs mode)")
     parser.add_argument("--output", "-o", choices=["text", "json"], default="text")
@@ -589,7 +577,8 @@ def main():
         config = load_config(args.config)
         queries = config.get("queries", [])
         if not queries:
-            print("ERROR: No queries in config"); sys.exit(1)
+            print("ERROR: No queries in config")
+            sys.exit(1)
 
         print("=" * 70)
         print("Retriever Benchmark (DOC mode)")
@@ -603,22 +592,23 @@ def main():
         )
 
     elif args.mode == "fs":
-        if not args.repo_dir or not args.repo_dir.exists():
-            print("ERROR: --repo-dir required and must exist for fs mode")
+        if not args.fs_root or not args.fs_root.exists():
+            print("ERROR: --fs-root required and must exist for fs mode")
             sys.exit(1)
         if not args.queries_config or not args.queries_config.exists():
             print("ERROR: --queries-config required and must exist for fs mode")
             sys.exit(1)
         config = load_config(args.queries_config)
         queries_with_gt = config.get("queries", [])
         if not queries_with_gt:
-            print("ERROR: No queries in config"); sys.exit(1)
+            print("ERROR: No queries in config")
+            sys.exit(1)
 
         print("=" * 70)
         print("Retriever Benchmark (FS mode)")
         print("=" * 70)
         result = run_benchmark(
-            mode="fs", repo_dir=args.repo_dir, queries_with_gt=queries_with_gt,
+            mode="fs", fs_root=args.fs_root, queries_with_gt=queries_with_gt,
             queries_config_path=args.queries_config,
             beam_size=args.beam_size, max_turns=args.max_turns,
             clear_cache=args.clear_cache,