Skip to content

Commit c3ea944

Browse files
authored
Merge pull request #46 from vectorlessflow/dev
Dev
2 parents fc7b25b + f1c0fb2 commit c3ea944

64 files changed

Lines changed: 33543 additions & 2244 deletions

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

.gitignore

Lines changed: 4 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -35,8 +35,10 @@ workspace/
3535
test_output/
3636
temp/
3737

38-
# Documentation
39-
docs/_build/
38+
# Documentation (Docusaurus)
39+
docs/node_modules/
40+
docs/build/
41+
docs/.docusaurus/
4042

4143
# Benchmarks
4244
.criterion/

Cargo.toml

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -84,7 +84,8 @@ rand = "0.8"
8484
bm25 = { version = "2.3.2", features = ["parallelism"] }
8585

8686
# Python bindings
87-
pyo3 = { version = "0.22", features = ["extension-module"] }
87+
pyo3 = { version = "0.28", features = ["extension-module"] }
88+
pyo3-async-runtimes = { version = "0.28", features = ["tokio-runtime"] }
8889

8990
# Dev dependencies
9091
tempfile = "3.10"

README.md

Lines changed: 19 additions & 16 deletions
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,6 @@
11
<div align="center">
22

3-
<img src="https://raw.githubusercontent.com/vectorlessflow/vectorless/main/docs/design/with-title.png" alt="Vectorless" width="400" style="vertical-align:middle;">
3+
<img src="https://vectorless.dev/img/with-title.png" alt="Vectorless" width="400" style="vertical-align:middle;">
44

55
<h1>Reasoning-native Document Intelligence Engine</h1>
66

@@ -27,23 +27,26 @@ pip install vectorless
2727
### Index and Query
2828

2929
```python
30+
import asyncio
3031
from vectorless import Engine, IndexContext
3132

32-
# Create engine — api_key and model are required
33-
engine = Engine(
34-
workspace="./data",
35-
api_key="sk-...",
36-
model="gpt-4o",
37-
)
38-
39-
# Index a document (PDF or Markdown)
40-
result = engine.index(IndexContext.from_file("./report.pdf"))
41-
doc_id = result.doc_id
42-
43-
# Query
44-
result = engine.query(doc_id, "What is the total revenue?")
45-
print(result.content)
46-
print(f"Score: {result.score}")
33+
async def main():
34+
# Create engine — api_key and model are required
35+
engine = Engine(
36+
workspace="./data",
37+
api_key="sk-...",
38+
model="gpt-4o",
39+
)
40+
41+
# Index a document (PDF or Markdown)
42+
result = await engine.index(IndexContext.from_file("./report.pdf"))
43+
doc_id = result.doc_id
44+
45+
# Query
46+
result = await engine.query(doc_id, "What is the total revenue?")
47+
print(result.single().content)
48+
49+
asyncio.run(main())
4750
```
4851

4952
<details>

docs/.gitignore

Lines changed: 20 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,20 @@
1+
# Dependencies
2+
/node_modules
3+
4+
# Production
5+
/build
6+
7+
# Generated files
8+
.docusaurus
9+
.cache-loader
10+
11+
# Misc
12+
.DS_Store
13+
.env.local
14+
.env.development.local
15+
.env.test.local
16+
.env.production.local
17+
18+
npm-debug.log*
19+
yarn-debug.log*
20+
yarn-error.log*

docs/README.md

Lines changed: 41 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,41 @@
1+
# Website
2+
3+
This website is built using [Docusaurus](https://docusaurus.io/), a modern static website generator.
4+
5+
## Installation
6+
7+
```bash
8+
yarn
9+
```
10+
11+
## Local Development
12+
13+
```bash
14+
yarn start
15+
```
16+
17+
This command starts a local development server and opens up a browser window. Most changes are reflected live without having to restart the server.
18+
19+
## Build
20+
21+
```bash
22+
yarn build
23+
```
24+
25+
This command generates static content into the `build` directory and can be served using any static contents hosting service.
26+
27+
## Deployment
28+
29+
Using SSH:
30+
31+
```bash
32+
USE_SSH=true yarn deploy
33+
```
34+
35+
Not using SSH:
36+
37+
```bash
38+
GIT_USER=<Your GitHub username> yarn deploy
39+
```
40+
41+
If you are using GitHub pages for hosting, this command is a convenient way to build the website and push to the `gh-pages` branch.
Lines changed: 84 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,84 @@
1+
---
2+
slug: welcome
3+
title: Welcome to Vectorless
4+
authors: [zTgx]
5+
tags: [vectorless, rag, llm, announcement]
6+
---
7+
8+
Vectorless is a reasoning-native document intelligence engine written in Rust — **no vector database, no embeddings, no similarity search**.
9+
10+
{/* truncate */}
11+
12+
## Why Vectorless?
13+
14+
Traditional RAG systems rely on vector embeddings and similarity search. This approach loses document structure, requires a vector database, and often returns chunks that lack context.
15+
16+
Vectorless takes a different path:
17+
18+
- **Hierarchical Semantic Trees** — Documents are parsed into a tree of sections, preserving structure and relationships.
19+
- **LLM Navigation** — Queries are resolved by intelligently traversing the tree, not by comparing vectors.
20+
- **Zero Infrastructure** — No vector DB, no embedding models, no similarity search. Just an LLM API key.
21+
22+
## Quick Start
23+
24+
### Python
25+
26+
```python
27+
import asyncio
28+
from vectorless import Engine, IndexContext
29+
30+
async def main():
31+
engine = Engine(
32+
workspace="./data",
33+
api_key="sk-...",
34+
model="gpt-4o",
35+
)
36+
37+
# Index a document
38+
result = await engine.index(IndexContext.from_file("./report.pdf"))
39+
doc_id = result.doc_id
40+
41+
# Query
42+
answer = await engine.query(doc_id, "What is the total revenue?")
43+
print(answer.single().content)
44+
45+
asyncio.run(main())
46+
```
47+
48+
### Rust
49+
50+
```rust
51+
use vectorless::{EngineBuilder, IndexContext, QueryContext};
52+
53+
#[tokio::main]
54+
async fn main() -> vectorless::Result<()> {
55+
let engine = EngineBuilder::new()
56+
.with_workspace("./data")
57+
.with_key("sk-...")
58+
.with_model("gpt-4o")
59+
.build()
60+
.await?;
61+
62+
let result = engine.index(IndexContext::from_path("./report.pdf")).await?;
63+
let doc_id = result.doc_id().unwrap();
64+
65+
let result = engine.query(
66+
QueryContext::new("What is the total revenue?").with_doc_id(doc_id)
67+
).await?;
68+
println!("{}", result.content);
69+
70+
Ok(())
71+
}
72+
```
73+
74+
## What's Next?
75+
76+
- Cross-document relationship graph
77+
- Incremental indexing with content fingerprinting
78+
- Multi-format support (Markdown, PDF, DOCX)
79+
80+
The project is open source under Apache-2.0. Contributions welcome!
81+
82+
- [GitHub](https://github.com/vectorlessflow/vectorless)
83+
- [PyPI](https://pypi.org/project/vectorless/)
84+
- [crates.io](https://crates.io/crates/vectorless)

docs/blog/authors.yml

Lines changed: 8 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,8 @@
1+
zTgx:
2+
name: zTgx
3+
title: Rust Developer
4+
url: https://beautifularea.com
5+
page: true
6+
socials:
7+
x: pendingcode
8+
github: zTgx

docs/blog/tags.yml

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,4 @@
1+
hello:
2+
label: Hello
3+
permalink: /hello
4+
description: Hello tag description

docs/docs/intro.mdx

Lines changed: 84 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,84 @@
1+
---
2+
sidebar_position: 1
3+
---
4+
5+
# Introduction
6+
7+
**Vectorless** is a reasoning-native document intelligence engine written in Rust — **no vector database, no embeddings, no similarity search**.
8+
9+
It transforms documents into hierarchical semantic trees and uses LLMs to navigate the structure, retrieving the most relevant content through deep contextual understanding instead of vector math.
10+
11+
## How It Works
12+
13+
1. **Parse** — Documents (Markdown, PDF) are parsed into hierarchical semantic trees, preserving structure and relationships between sections.
14+
2. **Index** — Trees are stored with metadata, keywords, and optional summaries. Incremental indexing skips unchanged files via content fingerprinting.
15+
3. **Query** — An LLM navigates the tree to find the most relevant sections. No embeddings, no similarity search — just structural reasoning.
16+
17+
## Quick Start
18+
19+
### Python
20+
21+
```bash
22+
pip install vectorless
23+
```
24+
25+
```python
26+
import asyncio
27+
from vectorless import Engine, IndexContext
28+
29+
async def main():
30+
engine = Engine(
31+
workspace="./data",
32+
api_key="sk-...",
33+
model="gpt-4o",
34+
)
35+
36+
result = await engine.index(IndexContext.from_file("./report.pdf"))
37+
doc_id = result.doc_id
38+
39+
answer = await engine.query(doc_id, "What is the total revenue?")
40+
print(answer.single().content)
41+
42+
asyncio.run(main())
43+
```
44+
45+
### Rust
46+
47+
```toml
48+
[dependencies]
49+
vectorless = "0.1"
50+
tokio = { version = "1", features = ["full"] }
51+
```
52+
53+
```rust
54+
use vectorless::{EngineBuilder, IndexContext, QueryContext};
55+
56+
#[tokio::main]
57+
async fn main() -> vectorless::Result<()> {
58+
let engine = EngineBuilder::new()
59+
.with_workspace("./data")
60+
.with_key("sk-...")
61+
.with_model("gpt-4o")
62+
.build()
63+
.await?;
64+
65+
let result = engine.index(IndexContext::from_path("./report.pdf")).await?;
66+
let doc_id = result.doc_id().unwrap();
67+
68+
let result = engine.query(
69+
QueryContext::new("What is the total revenue?").with_doc_id(doc_id)
70+
).await?;
71+
println!("{}", result.content);
72+
73+
Ok(())
74+
}
75+
```
76+
77+
## Features
78+
79+
- **Hierarchical Semantic Trees** — Preserves document structure, not flat chunks
80+
- **LLM-Powered Retrieval** — Structural reasoning over the tree, not vector similarity
81+
- **Incremental Indexing** — Content fingerprinting skips unchanged files
82+
- **Cross-Document Graph** — Automatic relationship discovery between documents
83+
- **Multi-Format** — Markdown and PDF support
84+
- **Zero Infrastructure** — No vector DB, no embedding models, just an LLM API key

0 commit comments

Comments
 (0)