Skip to content

Commit 10a4898

Browse files
docs: sync README with docker-compose, helm, and notebooks (#18)
1 parent 832da2c commit 10a4898

1 file changed

Lines changed: 95 additions & 158 deletions

File tree

README.md

Lines changed: 95 additions & 158 deletions
Original file line numberDiff line numberDiff line change
@@ -1,227 +1,164 @@
11
# enVector - Encrypted Vector Search
22

33
[![License](https://img.shields.io/badge/license-Proprietary-red.svg)](LICENSE)
4-
[![PyPI](https://img.shields.io/pypi/v/es2)](https://pypi.org/project/es2/)
4+
[![PyPI](https://img.shields.io/pypi/v/pyenvector)](https://pypi.org/project/pyenvector/)
55
[![Docker](https://img.shields.io/badge/docker-private-blue.svg)](https://www.docker.com/)
66
[![Kubernetes](https://img.shields.io/badge/kubernetes-ready-blue.svg)](https://kubernetes.io/)
77
[![Python](https://img.shields.io/badge/python-3.9%20%7C%203.10%20%7C%203.11%20%7C%203.12%20%7C%203.13-blue.svg)](https://www.python.org/)
88
[![OS](https://img.shields.io/badge/os-linux%20%7C%20macOS%2011.0+-green.svg)](https://www.apple.com/macos/)
99

10-
> **enVector** is a product that provides secure vector search functionality using **ES2 (Encrypted Similarity Search)**, which is based on Fully Homomorphic Encryption (FHE). This repository contains self-hosted deployment scripts and client SDK examples.
10+
> **enVector** provides secure vector search using **ES2 (Encrypted Similarity Search)** based on Fully Homomorphic Encryption (FHE). This repository contains self-hosted deployment assets and Python SDK notebooks.
1111
12-
## 🚀 Features
12+
## Features
1313

14-
- **🔐 End-to-End Encryption**: Secure vector search with FHE (Fully Homomorphic Encryption)
15-
- **⚡ High Performance**: Optimized vector similarity search algorithms
16-
- **🏗️ Microservices Architecture**: Scalable and maintainable service design
17-
- **🐳 Multi-Platform Deployment**: Docker Compose and Kubernetes (Helm) support
18-
- **📱 Python SDK**: Easy-to-use client library for integration
14+
- End-to-end encrypted vector search
15+
- Docker Compose and Kubernetes (Helm) deployment support
16+
- Multi-service architecture for scaling and HA
17+
- Python SDK notebooks for quick start and API flow
1918

19+
## Architecture
2020

21-
## 🔒 Security Features
21+
enVector consists of five main services:
2222

23-
- **Fully Homomorphic Encryption**: Secure computation on encrypted data
24-
- **Client-side Key Management**: Secret key never leave the client
25-
- **Encrypted Vector Storage**: All vector data is encrypted at rest (at-rest)
26-
- **Secure Search**: Search operations performed on encrypted data (in-use)
23+
- **envector-endpoint**: API gateway and client entrypoint
24+
- **envector-backend**: metadata/service management
25+
- **envector-orchestrator**: request coordination and scheduling
26+
- **envector-compute**: encrypted vector compute workers
27+
- **envector-shaper**: shard split/merge and storage shaping tasks
2728

28-
## 📊 Performance
29+
Infrastructure dependencies:
2930

30-
- **Vector Dimensions**: Support for 32-4096 dimensional vectors
31-
- **Search Speed**: Optimized encrtyped similarity search algorithms
32-
- **Scalability**: Horizontal scaling with Kubernetes
33-
- **GPU Support**: Optional GPU acceleration for encrypted vector search
31+
- PostgreSQL (metadata)
32+
- Object storage (S3-compatible or GCS via Helm values)
3433

35-
## 🛠️ Development
34+
## Project Structure
3635

37-
### Prerequisites
38-
39-
- Python 3.9-3.13
40-
- Docker and Docker Compose
41-
- Kubernetes cluster (for K8s deployment)
42-
- Helm 3.0+
43-
- Linux or macOS 11.0+
44-
45-
## 🏗️ Architecture
46-
47-
enVector consists of four main microservices:
48-
49-
- **es2e (Endpoint)**: Main API gateway and client interface
50-
- **es2b (Backend)**: Service orchestration and metadata management
51-
- **es2o (Orchestrator)**: Manages and schedules compute requests
52-
- **es2c (Compute)**: Executes vector search and similarity computations
53-
54-
### Infrastructure Dependencies
55-
56-
- **PostgreSQL**: Metadata storage and management
57-
- **MinIO**: Vector data storage (S3-compatible)
58-
59-
## 📁 Project Structure
60-
61-
```
36+
```text
6237
envector-deployment/
63-
├── docker-compose/ # Docker Compose deployment
64-
│ ├── docker-compose.envector.yml # Core application services
65-
│ ├── docker-compose.infra.yml # Postgres + MinIO (adds readiness deps to core)
66-
│ ├── docker-compose.gpu.yml # GPU override for es2c
67-
│ ├── .env.example # environment variables for es2
68-
│ ├── start_envector.sh # easy-to-use helper script
69-
│ └── README.md # Docker setup guide
70-
├── kubernetes-manifests/ # Kubernetes deployment
71-
│ ├── helm/ # Helm chart for K8s
72-
│ │ ├── Chart.yaml # Chart metadata
73-
│ │ ├── values.yaml # Configurable values
74-
│ │ └── templates/ # K8s manifest templates
75-
│ └── README.md # K8s deployment guide
76-
└── notebooks/ # Python SDK examples
38+
├── docker-compose/
39+
│ ├── docker-compose.envector.yml
40+
│ ├── docker-compose.infra.yml
41+
│ ├── docker-compose.gpu.yml
42+
│ ├── start_envector.sh
43+
│ └── README.md
44+
├── kubernetes-manifests/
45+
│ ├── helm/
46+
│ │ ├── Chart.yaml
47+
│ │ ├── values.yaml
48+
│ │ └── templates/
49+
│ └── README.md
50+
└── notebooks/
51+
├── 00-quick-start.ipynb
52+
├── 01-api-flow.ipynb
53+
├── 02-simple-rag.ipynb
54+
├── 03-rag-with-langchain.ipynb
55+
├── 04-ann-api-flow.ipynb
56+
└── 05-insert-load-capacity.ipynb
7757
```
7858

79-
## 🚀 Quick Start
59+
## Quick Start
8060

81-
**⚠️ Important**: Docker images are stored in private repositories. Please contact [heaan](hello@heaan.com) for access credentials before proceeding with deployment.
61+
**Important**: Docker images are in private repositories. Contact [heaan](mailto:hello@heaan.com) for access.
8262

8363
### Option 1: Docker Compose
8464

85-
Recommended for Development. See more details in [docker-compose README](docker-compose/README.md).
65+
See full guide: [docker-compose/README.md](docker-compose/README.md)
8666

87-
#### Method A: Clone Repository
8867
```bash
89-
# Clone the repository
9068
git clone https://github.com/CryptoLabInc/envector-deployment.git
9169
cd envector-deployment/docker-compose
9270

93-
# Copy environment file (optional)
94-
# If .env is missing, ./start_envector.sh will be created from .env.example automatically
71+
# optional (script auto-creates from .env.example if missing)
9572
cp .env.example .env
9673

97-
# Start services (performs preflight: Docker, PAT login if needed, license token)
74+
# preflight + up (docker, image access, license token)
9875
./start_envector.sh
99-
# OR docker compose -f docker-compose.envector.yml -f docker-compose.infra.yml -p envector up -d
10076
```
10177

102-
#### Method B: Direct HTTP Usage
78+
Useful variants:
79+
10380
```bash
104-
# Download and run directly from GitHub
105-
curl -O https://raw.githubusercontent.com/cryptolabinc/envector-deployment/main/docker-compose/docker-compose.envector.yml
106-
curl -O https://raw.githubusercontent.com/cryptolabinc/envector-deployment/main/docker-compose/docker-compose.infra.yml
107-
curl -O https://raw.githubusercontent.com/cryptolabinc/envector-deployment/main/docker-compose/.env.example
81+
# print merged compose config only
82+
./start_envector.sh --config
10883

109-
# Copy environment file
110-
cp .env.example .env
84+
# GPU override
85+
./start_envector.sh --gpu
11186

112-
# Start services
113-
docker compose -f docker-compose.envector.yml -f docker-compose.infra.yml -p envector up -d
114-
```
87+
# scale compute/orchestrator
88+
./start_envector.sh --num-compute 4 --num-orchestrator 2
11589

90+
# stop stack
91+
./start_envector.sh --down
92+
```
11693

117-
### Option 2: Kubernetes
94+
### Option 2: Kubernetes (Helm)
11895

119-
Recommended for production.
96+
See full guide: [kubernetes-manifests/README.md](kubernetes-manifests/README.md)
12097

121-
#### Clone Repository
12298
```bash
123-
# Clone the repository
12499
git clone https://github.com/CryptoLabInc/envector-deployment.git
125-
cd envector-deployment
100+
cd envector-deployment/kubernetes-manifests
126101

127-
# Install Helm chart
128-
helm install envector ./kubernetes-manifests/helm
102+
# install chart
103+
helm install envector ./helm
104+
```
129105

130-
# Check deployment status
131-
kubectl get pods
106+
For production, configure before install/upgrade:
132107

133-
# Access services
134-
kubectl get svc
135-
```
108+
- `externalServices.metadatadb` / `externalServices.storage`
109+
- `compute.license` (token secret creation or `existingSecret`)
110+
- `externalSecrets.*` if using External Secrets Operator
136111

137-
## 🔧 Configuration
112+
## Configuration
138113

139-
### Environment Variables
114+
### Key Docker Compose Environment Variables
140115

141116
| Variable | Description | Default |
142-
|----------|-------------|---------|
143-
| `ES2E_TAG` | es2e service image tag | `latest` |
144-
| `ES2B_TAG` | es2b service image tag | `latest` |
145-
| `ES2O_TAG` | es2o service image tag | `latest` |
146-
| `ES2C_TAG` | es2c service image tag | `latest` |
147-
| `ES2_LOG_LEVEL` | Logging level | `INFO` |
148-
| `ES2E_HOST_PORT` | es2e external port | `50050` |
117+
|---|---|---|
118+
| `ENVECTOR_ENDPOINT_HOST_PORT` | Host port for gRPC endpoint | `50050` |
119+
| `ENVECTOR_HTTP_HEALTH_HOST_PORT` | Host port for HTTP health/admin | `18080` |
120+
| `ENVECTOR_ADMIN_API_ENABLED` | Enable admin API on endpoint | `true` |
121+
| `ENVECTOR_LOG_LEVEL` | Service log level | `INFO` |
122+
| `ENVECTOR_COMPUTE_TAG` | Compute image tag | `latest` |
123+
| `ENVECTOR_LICENSE_TOKEN` | In-container license token path | `/envector/license/token.jwt` |
124+
| `ENVECTOR_DB_*` | PostgreSQL connection parts | varies |
125+
| `ENVECTOR_STORAGE_*` | Storage connection settings | varies |
149126

150-
### Helm Values
127+
### Key Helm Values
151128

152-
Edit `kubernetes-manifests/helm/values.yaml` to customize:
153-
- Service ports and types
154-
- Resource limits and replicas
155-
- External database connections
156-
- Image repositories and tags
129+
`kubernetes-manifests/helm/values.yaml` 주요 항목:
157130

131+
- `endpoint.*`, `backend.*`, `orchestrator.*`, `compute.*`, `shaper.*`
132+
- `externalServices.metadatadb.*`, `externalServices.storage.*`
133+
- `compute.license.*` (createSecret/existingSecret/mountAsFile/injectAsEnv)
134+
- `externalSecrets.*` (ESO 기반 시크릿 주입)
135+
- `ingress.*` (TLS/HTTPS)
158136

159-
## 📚 Python SDK Usage
160-
* Python Version: 3.9-3.13
161-
* OS: Linux/macOS 11.0+
137+
## Python SDK / Notebooks
162138

163139
```bash
164-
pip install es2
140+
pip install pyenvector
165141
```
166142

167-
### Basic Setup
143+
Basic init example:
168144

169145
```python
170-
import es2
146+
import pyenvector as ev
171147

172-
# Initialize connection
173-
es2.init(
174-
host="localhost",
175-
port=50050,
148+
ev.init(
149+
address="localhost:50050",
176150
key_path="./keys",
177-
key_id="my_key"
151+
key_id="my_key",
178152
)
179-
180-
# Create index
181-
index = es2.create_index("my_index", dim=512)
182-
183-
# Insert vectors
184-
vectors = [
185-
[0.001 * i for i in range(512)],
186-
[0.001 * i + 0.001 for i in range(512)],
187-
]
188-
index.insert(vectors, metadata=["doc1", "doc2"])
189-
190-
# Search
191-
results = index.search(vectors[0], top_k=5)
192-
```
193-
194-
### Key Management
195-
196-
```python
197-
from es2.crypto import KeyGenerator, Cipher
198-
199-
# Generate FHE keys
200-
keygen = KeyGenerator("./keys/my_key")
201-
keygen.generate_keys()
202-
203-
# Create cipher for encryption/decryption
204-
cipher = Cipher(dim=512, enc_key_path="./keys/my_key/EncKey.json")
205153
```
206154

155+
Notebook examples are in `notebooks/`.
207156

208-
## 📄 License
209-
210-
This project is proprietary software. For licensing information, please contact [heaan](mailto:hello@heaan.com).
211-
212-
## 🤝 Contributing
213-
214-
This is a proprietary software project. For contribution inquiries, please contact [heaan](mailto:hello@heaan.com).
215-
216-
## 📞 Support
217-
218-
- **Product Information**: [enVector at heaan](https://heaan.com)
219-
- **Technical Support**: Please contact [heaan](mailto:es2.support@heaan.com)
157+
## License
220158

221-
## 🔗 Related Links
159+
This project is proprietary software. For licensing information, contact [heaan](mailto:hello@heaan.com).
222160

223-
- [enVector Product Page](https://heaan.com)
224-
- [heaan](https://heaan.com)
225-
- [FHE Resources](https://fhe.org)
161+
## Support
226162

227-
---
163+
- Product Information: [heaan](https://heaan.com)
164+
- Technical Support: [es2.support@heaan.com](mailto:es2.support@heaan.com)

0 commit comments

Comments
 (0)