Merge pull request #112 from vectorlessflow/dev

Dev

Author: zTgx

.gitignore

@@ -84,6 +84,3 @@ wheels/
 .venv/
 venv/
 ENV/
-
-# Test workspace
-workspace*

Cargo.toml

@@ -10,11 +10,13 @@ members = [
     "vectorless-core/vectorless-metrics",
     "vectorless-core/vectorless-llm",
     "vectorless-core/vectorless-storage",
-    "vectorless-core/vectorless-query",
+    # Strategy layer moved to Python — crates kept but not compiled:
+    # "vectorless-core/vectorless-query",
+    # "vectorless-core/vectorless-agent",
+    # "vectorless-core/vectorless-retrieval",
     "vectorless-core/vectorless-index",
-    "vectorless-core/vectorless-agent",
-    "vectorless-core/vectorless-retrieval",
     "vectorless-core/vectorless-rerank",
+    "vectorless-core/vectorless-primitives",
     "vectorless-core/vectorless-engine",
     "vectorless-core/vectorless-py",
 ]

README.md

@@ -1,126 +1,43 @@
-<div align="center">
-
-<img src="https://vectorless.dev/img/with-title.png" alt="Vectorless" width="400">
-
-<h1>Document Understanding Engine for AI</h1>
-<h3>Reason, don't vector · Structure, not chunks · Think, then answer</h3>
+<h1>Vectorless</h1>
 
 [![PyPI](https://img.shields.io/pypi/v/vectorless.svg)](https://pypi.org/project/vectorless/)
 [![PyPI Downloads](https://static.pepy.tech/badge/vectorless/month)](https://pepy.tech/projects/vectorless)
-[![Crates.io](https://img.shields.io/crates/v/vectorless.svg)](https://crates.io/crates/vectorless)
-[![Crates.io Downloads](https://img.shields.io/crates/d/vectorless.svg)](https://crates.io/crates/vectorless)
-[![Docs](https://docs.rs/vectorless/badge.svg)](https://docs.rs/vectorless)
-[![License](https://img.shields.io/badge/license-Apache--2.0-blue.svg)](LICENSE)
-
-</div>
-
-**Vectorless** is a document understanding engine for AI. It compiles documents into structured trees of meaning, then dispatches multiple agents to reason through headings, sections, and paragraphs — evaluating how each part relates to the whole. The problem it solves is not "where to look", but "what does this mean in context". Every answer is a reasoning act, not a retrieval result.
-
-Light up a star and shine with us! ⭐
-
-## Three Rules
-- **Reason, don't vector.** Understanding is reasoning, not similarity.
-- **Model fails, we fail.** No heuristic fallbacks, no silent degradation.
-- **No thought, no answer.** Only reasoned output counts as an answer.
 
-## How It Works
+<p>Knowing by reasoning, not vectors.</p>
+<p>Deep and reliable. Vectorless plays nicely with your documents. Ask questions in plain language; get answers by reasoning with Vectorless.</p>
 
-### Four-Artifact Index Architecture
+## Installation
 
-When a document is indexed, the compile pipeline builds four artifacts:
+Install using `pip install -U vectorless`. For more details, see the [Installation](https://vectorless.dev/docs/installation) section in the documentation.
 
-```
-Content Layer          Navigation Layer              Reasoning Index            Document Card
-DocumentTree          NavigationIndex               ReasoningIndex            DocCard
-(TreeNode)            (NavEntry, ChildRoute)        (topic_paths, hot_nodes)  (title, overview,
-      │                      │                              │                 question hints)
-      │                      │                              │                    │
- Agent reads           Agent reads every            Agent's targeted        Orchestrator reads
- only on cat           decision round               search tool (grep)      for multi-doc routing
-```
-
-- **Content Layer** — The raw document tree. The agent only accesses this when reading specific paragraphs (`cat`).
-- **Navigation Layer** — Each non-leaf node stores an overview, question hints, and child routes (title + description). The agent reads this every round to decide where to go next.
-- **Reasoning Index** — Keyword-topic mappings with weights. Provides the agent's `grep` tool with structured keyword data for targeted search within a document.
-- **DocCard** — A compact document-level summary. The Orchestrator reads DocCards to decide which documents to navigate in multi-document queries, without loading full documents.
-
-This separation means the agent makes routing decisions from lightweight metadata, not by scanning full content.
-
-### Agent-Based Understanding
-
-```
-Engine.query("What drove the revenue decline?")
-  │
-  ├─ Query Understanding ── intent, concepts, strategy (LLM)
-  │
-  ├─ Orchestrator ── analyzes query, dispatches Workers
-  │   │
-  │   ├─ Worker 1 ── ls → cd "Financials" → ls → cd "Revenue" → cat
-  │   └─ Worker 2 ── ls → cd "Risk Factors" → grep "decline" → cat
-  │   │
-  │   └─ evaluate ── insufficient? → replan → dispatch new paths → loop
-  │
-  └─ Synthesis ── dedup, evidence scoring, reasoned answer with source chain
-```
-
-Worker navigation commands:
-
-| Command | Action | Reads |
-|---------|--------|-------|
-| `ls` | List child sections | Navigation Layer (ChildRoute) |
-| `cd` | Enter a child section | Navigation Layer |
-| `cat` | Read content at current node | Content Layer (DocumentTree) |
-| `grep` | Search by keyword | Reasoning Index (topic_paths) |
-
-The Orchestrator evaluates Worker results after each round. If evidence is insufficient, it **replans** — adjusting strategy, dispatching new paths, or deepening exploration. This continues until enough evidence is collected.
-
-## Quick Start
-
-```bash
-pip install vectorless
-```
+## A Simple Example
 
 ```python
 import asyncio
-from vectorless import Engine, IndexContext, QueryContext
+from vectorless import Engine
 
 async def main():
     engine = Engine(api_key="sk-...", model="gpt-4o", endpoint="https://api.openai.com/v1")
 
-    # Index a document
-    result = await engine.index(IndexContext.from_path("./report.pdf"))
+    # Compile a document
+    result = await engine.compile(path="./report.pdf")
     doc_id = result.doc_id
 
-    # Query
-    result = await engine.query(
-        QueryContext("What is the total revenue?").with_doc_ids([doc_id])
-    )
-    print(result.single().content)
+    # Ask a question
+    response = await engine.ask("What is the total revenue?", doc_ids=[doc_id])
+    print(response.single().content)
 
 asyncio.run(main())
 ```
 
-## Resources
+## Help
 
-- [Documentation](https://vectorless.dev) — Guides, architecture, API reference
-- [Rust API Docs](https://docs.rs/vectorless) — Auto-generated crate documentation
-- [PyPI](https://pypi.org/project/vectorless/) — Python package
-- [Crates.io](https://crates.io/crates/vectorless) — Rust crate
-- [Examples](examples/) — Complete usage patterns for Python and Rust
+See [documentation](https://vectorless.dev/docs/getting-started) for more details.
 
-## Contributing
 
-Contributions welcome! If you find this useful, please ⭐ the repo — it helps others discover it.
-
-## Star History
+## Contributing
 
-<a href="https://www.star-history.com/?repos=vectorlessflow%2Fvectorless&type=date&legend=top-left">
- <picture>
-   <source media="(prefers-color-scheme: dark)" srcset="https://api.star-history.com/chart?repos=vectorlessflow/vectorless&type=date&theme=dark&legend=top-left" />
-   <source media="(prefers-color-scheme: light)" srcset="https://api.star-history.com/chart?repos=vectorlessflow/vectorless&type=date&legend=top-left" />
-   <img alt="Star History Chart" src="https://api.star-history.com/chart?repos=vectorlessflow/vectorless&type=date&legend=top-left" />
- </picture>
-</a>
+Contributions welcome! See [Contributing](CONTRIBUTING.md) for setup and guidelines.
 
 ## License
 

docs/docs/api-reference.mdx

@@ -1,7 +1,7 @@
 ---
 sidebar_position: 9
 title: API Reference
-description: Complete API reference for Vectorless Rust crate and Python SDK.
+description: Complete API reference for the Vectorless Python SDK.
 ---
 
 # API Reference
@@ -10,8 +10,7 @@ description: Complete API reference for Vectorless Rust crate and Python SDK.
 
 In the meantime, you can refer to the following resources:
 
-- **Rust crate docs**: [docs.rs/vectorless](https://docs.rs/vectorless) — auto-generated documentation from source code
 - **Python SDK docs**: Available via `help(vectorless)` in an interactive Python session
 - **Source code**: [github.com/vectorlessflow/vectorless](https://github.com/vectorlessflow/vectorless)
 
-For usage examples, see [Quick Query](/docs/examples/quick-query), [Multi-Document](/docs/examples/multi-document), and [Batch Indexing](/docs/examples/batch-indexing).
+For usage examples, see [Quick Query](/docs/examples/quick-query), [Multi-Document](/docs/examples/multi-document), and [Batch Compiling](/docs/examples/batch-indexing).

docs/docs/architecture.mdx

@@ -67,7 +67,7 @@ The retrieval pipeline is a supervisor loop driven entirely by LLM reasoning. Ev
 ### Flow
 
 ```text
-Engine.query()
+Engine.ask()
   → Dispatcher
     → Query Understanding (LLM) → QueryPlan (intent, concepts, strategy)
     → Orchestrator (always — single or multi-doc)

docs/docs/examples/batch-indexing.mdx

@@ -2,28 +2,24 @@
 sidebar_position: 3
 ---
 
-# Batch Indexing
+# Batch Compiling
 
-Index multiple documents efficiently with progress tracking and error handling.
-
-## Python
+Compile multiple documents efficiently with progress tracking and error handling.
 
 ```python
 import asyncio
-from vectorless import Engine, IndexContext, IndexOptions
+from vectorless import Engine
 
 async def main():
     engine = Engine(
         api_key="sk-...",
         model="gpt-4o",
     )
 
-    # Index a directory of documents
-    result = await engine.index(
-        IndexContext.from_dir("./documents/")
-    )
+    # Compile a directory of documents
+    result = await engine.compile(directory="./documents/")
 
-    print(f"Indexed {len(result.items)} documents")
+    print(f"Compiled {len(result.items)} documents")
     print(f"Failures: {len(result.failed)}")
 
     for item in result.items:
@@ -37,50 +33,19 @@ async def main():
     for fail in result.failed:
         print(f"  ✗ {fail.source}: {fail.error}")
 
-    # List all indexed documents
-    docs = await engine.list()
-    print(f"\nTotal indexed: {len(docs)} documents")
+    # List all compiled documents
+    docs = await engine.list_documents()
+    print(f"\nTotal compiled: {len(docs)} documents")
 
 asyncio.run(main())
 ```
 
-## Rust
-
-```rust
-use vectorless::{Engine, EngineBuilder, IndexContext};
-
-#[tokio::main]
-async fn main() -> vectorless::Result<()> {
-    let engine = EngineBuilder::new()
-        .with_key("sk-...")
-        .with_model("gpt-4o")
-        .build()
-        .await?;
-
-    // Index a directory
-    let result = engine.index(IndexContext::from_dir("./documents/")).await?;
-
-    println!("Indexed {} documents", result.items.len());
-    println!("Failures: {}", result.failed.len());
-
-    for item in &result.items {
-        println!("  ✓ {} ({:?}) → {}", item.name, item.format, item.doc_id);
-    }
-
-    // List all documents
-    let docs = engine.list().await?;
-    println!("Total indexed: {} documents", docs.len());
-
-    Ok(())
-}
-```
-
 ## Error Handling
 
-Each item in the result is either successful or failed. Failures don't prevent other documents from being indexed:
+Each item in the result is either successful or failed. Failures don't prevent other documents from being compiled:
 
 ```python
-result = await engine.index(IndexContext.from_paths(mixed_paths))
+result = await engine.compile(paths=mixed_paths)
 
 # Successful items
 for item in result.items:

docs/docs/examples/multi-document.mdx

@@ -4,31 +4,26 @@ sidebar_position: 2
 
 # Multi-Document Retrieval
 
-Query across multiple indexed documents using the cross-document strategy with graph-based score boosting.
-
-## Python
+Query across multiple compiled documents using the cross-document strategy with graph-based score boosting.
 
 ```python
 import asyncio
-from vectorless import (
-    Engine, IndexContext, QueryContext,
-    IndexOptions,
-)
+from vectorless import Engine
 
 async def main():
     engine = Engine(
         api_key="sk-...",
         model="gpt-4o",
     )
 
-    # Index multiple documents
+    # Compile multiple documents
     docs = ["./report-q1.pdf", "./report-q2.pdf", "./report-q3.pdf"]
     doc_ids = []
 
     for path in docs:
-        result = await engine.index(IndexContext.from_path(path))
+        result = await engine.compile(path=path)
         doc_ids.append(result.doc_id)
-        print(f"Indexed: {path} → {result.doc_id}")
+        print(f"Compiled: {path} → {result.doc_id}")
 
     # Check the cross-document graph
     graph = await engine.get_graph()
@@ -40,25 +35,26 @@ async def main():
                 print(f"  {doc_id[:8]}... → {edge.target_doc_id[:8]}... ({edge.weight:.2f})")
 
     # Query across all documents
-    result = await engine.query(
-        QueryContext("Compare quarterly revenue trends")
-        .with_doc_ids(doc_ids)
+    response = await engine.ask(
+        "Compare quarterly revenue trends",
+        doc_ids=doc_ids,
     )
 
-    for item in result.items:
-        print(f"\n[{item.doc_id[:8]}...] Score: {item.score:.2f}")
+    for item in response.items:
+        print(f"\n[{item.doc_id[:8]}...] Confidence: {item.confidence:.2f}")
         print(item.content[:300])
 
     # Or query entire workspace
-    result = await engine.query(
-        QueryContext("What documents discuss risk factors?")
+    response = await engine.ask(
+        "What documents discuss risk factors?",
+        workspace_scope=True,
     )
 
-    print(f"\nFound in {len(result.items)} document(s)")
+    print(f"\nFound in {len(response.items)} document(s)")
 
     # Cleanup
     for doc_id in doc_ids:
-        await engine.remove(doc_id)
+        await engine.remove_document(doc_id)
 
 asyncio.run(main())
 ```
@@ -67,19 +63,8 @@ asyncio.run(main())
 
 ### Document Graph
 
-After indexing, documents are connected in a graph based on shared keywords. The graph enables:
+After compiling, documents are connected in a graph based on shared keywords. The graph enables:
 
 - **Score boosting** — High-confidence results in one document boost neighbor documents
 - **Relationship discovery** — Automatically find related documents
 - **Cross-referencing** — Results from connected documents are surfaced together
-
-### Merge Strategies
-
-The cross-document strategy supports multiple merge modes:
-
-| Strategy | Description |
-|----------|-------------|
-| **TopK** | Return top-K results across all documents |
-| **BestPerDocument** | Best result from each document |
-| **WeightedByRelevance** | Weight by each document's best score |
-| **GraphBoosted** | Use graph connections to boost scores |