Skip to content

07-production-deployment.md

Latest commit

 

History

History
334 lines (249 loc) · 10.4 KB

07-production-deployment.md

File metadata and controls

334 lines (249 loc) · 10.4 KB
layout default
title Chapter 7: Production Deployment
parent Chroma Tutorial
nav_order 7

Chapter 7: Production Deployment

Welcome to Chapter 7: Production Deployment. In this part of ChromaDB Tutorial: Building AI-Native Vector Databases, you will build an intuitive mental model first, then move into concrete implementation details and practical production tradeoffs.

Scale Chroma for production workloads! This chapter covers deployment strategies, scaling, monitoring, and operational best practices for production Chroma deployments.

Production Architecture

Scalable Deployment

from chromadb.config import Settings
import chromadb

# Production configuration
production_settings = Settings(
    chroma_server_host="0.0.0.0",
    chroma_server_http_port=8000,
    chroma_server_cors_allow_origins=["*"],
    anonymized_telemetry=False,
    allow_reset=False,  # Disable reset in production
    is_persistent=True,
    persist_directory="/data/chroma"
)

# Initialize production client
client = chromadb.PersistentClient(settings=production_settings)

Docker Deployment

# docker-compose.yml for production
version: '3.8'
services:
  chroma:
    image: chromadb/chroma:latest
    ports:
      - "8000:8000"
    volumes:
      - ./data:/chroma/chroma
    environment:
      - CHROMA_SERVER_HOST=0.0.0.0
      - CHROMA_SERVER_HTTP_PORT=8000
      - ANONYMIZED_TELEMETRY=False
      - ALLOW_RESET=False
      - IS_PERSISTENT=True
    restart: unless-stopped

Scaling Strategies

Horizontal Scaling

# Load balancing across multiple Chroma instances
class ChromaLoadBalancer:
    def __init__(self, instances):
        self.instances = instances
        self.current_index = 0

    def get_next_instance(self):
        instance = self.instances[self.current_index]
        self.current_index = (self.current_index + 1) % len(self.instances)
        return instance

    def distribute_operation(self, operation, *args, **kwargs):
        instance = self.get_next_instance()
        return getattr(instance, operation)(*args, **kwargs)

# Usage
load_balancer = ChromaLoadBalancer([
    chromadb.Client(host="chroma-1:8000"),
    chromadb.Client(host="chroma-2:8000"),
    chromadb.Client(host="chroma-3:8000")
])

# Distribute operations
collection = load_balancer.distribute_operation("get_collection", "my_collection")

Monitoring and Observability

Performance Monitoring

import time
import psutil

class ChromaMonitor:
    def __init__(self, client):
        self.client = client
        self.metrics = {}

    def collect_metrics(self):
        # System metrics
        self.metrics['cpu_percent'] = psutil.cpu_percent()
        self.metrics['memory_percent'] = psutil.virtual_memory().percent

        # Chroma-specific metrics
        collections = self.client.list_collections()
        self.metrics['collection_count'] = len(collections)
        self.metrics['total_documents'] = sum(c.count() for c in collections)

        return self.metrics

    def monitor_query_performance(self, query_func, *args, **kwargs):
        start_time = time.time()
        result = query_func(*args, **kwargs)
        query_time = time.time() - start_time

        self.metrics['last_query_time'] = query_time
        return result, query_time

# Usage
monitor = ChromaMonitor(client)
metrics = monitor.collect_metrics()

result, query_time = monitor.monitor_query_performance(
    collection.query,
    query_texts=["test query"],
    n_results=5
)

Backup and Recovery

Automated Backup

import json
import os
from datetime import datetime
import shutil

class ChromaBackupManager:
    def __init__(self, chroma_data_dir, backup_dir):
        self.chroma_data_dir = chroma_data_dir
        self.backup_dir = backup_dir
        os.makedirs(backup_dir, exist_ok=True)

    def create_backup(self):
        timestamp = datetime.now().strftime("%Y%m%d_%H%M%S")
        backup_name = f"chroma_backup_{timestamp}"

        backup_path = os.path.join(self.backup_dir, backup_name)

        # Copy entire data directory
        shutil.copytree(self.chroma_data_dir, backup_path)

        # Create metadata
        metadata = {
            "timestamp": timestamp,
            "chroma_data_dir": self.chroma_data_dir,
            "collections": self._get_collection_info()
        }

        with open(os.path.join(backup_path, "backup_metadata.json"), 'w') as f:
            json.dump(metadata, f, indent=2)

        return backup_path

    def restore_backup(self, backup_path):
        # Stop Chroma service
        self._stop_chroma()

        # Restore data
        shutil.copytree(backup_path, self.chroma_data_dir, dirs_exist_ok=True)

        # Restart Chroma service
        self._start_chroma()

    def _get_collection_info(self):
        # Get collection information for backup metadata
        try:
            client = chromadb.PersistentClient(path=self.chroma_data_dir)
            collections = client.list_collections()
            return [{"name": c.name, "count": c.count()} for c in collections]
        except:
            return []

# Usage
backup_manager = ChromaBackupManager("/data/chroma", "/backups")
backup_path = backup_manager.create_backup()

Security Best Practices

Access Control

class ChromaSecurityManager:
    def __init__(self, client):
        self.client = client
        self.permissions = {}

    def set_collection_permissions(self, collection_name, user_permissions):
        self.permissions[collection_name] = user_permissions

    def check_permission(self, user, collection_name, operation):
        if collection_name not in self.permissions:
            return False

        user_perms = self.permissions[collection_name].get(user, [])
        return operation in user_perms

    def secure_query(self, user, collection_name, query_func, *args, **kwargs):
        if not self.check_permission(user, collection_name, "query"):
            raise PermissionError(f"User {user} cannot query {collection_name}")

        return query_func(*args, **kwargs)

# Usage
security = ChromaSecurityManager(client)
security.set_collection_permissions("sensitive_data", {
    "admin": ["query", "add", "delete"],
    "user": ["query"]
})

# Secure query execution
try:
    results = security.secure_query("user", "sensitive_data", collection.query,
                                   query_texts=["test"])
except PermissionError as e:
    print(f"Access denied: {e}")

Performance Optimization

Query Optimization

# Production query optimization
def optimize_production_query(collection, query_texts, **kwargs):
    # Use optimized search parameters
    optimized_kwargs = {
        "ef": min(128, kwargs.get("n_results", 10) * 10),  # Adaptive ef
        "n_results": kwargs.get("n_results", 10),
        "include": ["documents", "metadatas"]  # Only include needed fields
    }

    # Add metadata filters if provided
    if "where" in kwargs:
        optimized_kwargs["where"] = kwargs["where"]

    return collection.query(query_texts=query_texts, **optimized_kwargs)

# Batch processing for better performance
def batch_process_queries(collection, queries, batch_size=10):
    results = []

    for i in range(0, len(queries), batch_size):
        batch = queries[i:i + batch_size]
        batch_results = collection.query(
            query_texts=batch,
            n_results=5
        )
        results.extend(batch_results['documents'])

    return results

What We've Accomplished

This chapter covered production deployment strategies for Chroma, including scaling, monitoring, backup/recovery, security, and performance optimization.

Next Steps

Time for optimization! In Chapter 8: Performance Optimization, we'll dive deep into tuning and optimizing Chroma performance.

Practice what you've learned:

  1. Set up a production Chroma deployment
  2. Implement monitoring and alerting
  3. Create backup and recovery procedures
  4. Configure security and access control
  5. Optimize queries for production workloads

How will you deploy Chroma in production? 🚀

What Problem Does This Solve?

Most teams struggle here because the hard part is not writing more code, but deciding clear boundaries for self, client, kwargs so behavior stays predictable as complexity grows.

In practical terms, this chapter helps you avoid three common failures:

  • coupling core logic too tightly to one implementation path
  • missing the handoff boundaries between setup, execution, and validation
  • shipping changes without clear rollback or observability strategy

After working through this chapter, you should be able to reason about Chapter 7: Production Deployment as an operating subsystem inside ChromaDB Tutorial: Building AI-Native Vector Databases, with explicit contracts for inputs, state transitions, and outputs.

Use the implementation notes around query, metrics, chroma as your checklist when adapting these patterns to your own repository.

How it Works Under the Hood

Under the hood, Chapter 7: Production Deployment usually follows a repeatable control path:

  1. Context bootstrap: initialize runtime config and prerequisites for self.
  2. Input normalization: shape incoming data so client receives stable contracts.
  3. Core execution: run the main logic branch and propagate intermediate state through kwargs.
  4. Policy and safety checks: enforce limits, auth scopes, and failure boundaries.
  5. Output composition: return canonical result payloads for downstream consumers.
  6. Operational telemetry: emit logs/metrics needed for debugging and performance tuning.

When debugging, walk this sequence in order and confirm each stage has explicit success/failure conditions.

Source Walkthrough

Use the following upstream sources to verify implementation details while reading this chapter:

  • View Repo Why it matters: authoritative reference on View Repo (github.com).

Suggested trace strategy:

  • search upstream code for self and client to map concrete implementation paths
  • compare docs claims against actual runtime/config code before reusing patterns in production

Chapter Connections