Skip to content
Merged
Show file tree
Hide file tree
Changes from 13 commits
Commits
Show all changes
17 commits
Select commit Hold shift + click to select a range
70db1c4
chore(release): bump version to 1.0.0 and configure Tauri signing
AlexMikhalev Nov 4, 2025
b03f48f
fix(ci): resolve GitHub Actions workflow failures
AlexMikhalev Nov 4, 2025
ab6a854
fix(build): add SCSS preprocessing and suppress dependency warnings
AlexMikhalev Nov 4, 2025
4bce62e
chore: prepare for v1.0.0 release - update docs and dependencies
AlexMikhalev Nov 5, 2025
774ebee
feat: sync full QueryRs implementation from private repo
AlexMikhalev Nov 5, 2025
10da54c
feat: comprehensive functional validation and html2md fix
AlexMikhalev Nov 5, 2025
f1ff00a
fix: sync TUI implementation from private repo and fix panic strategy…
AlexMikhalev Nov 5, 2025
7b93eac
release: v1.0.0 release build successful
AlexMikhalev Nov 5, 2025
bf2791f
release: v1.0.0 - Complete release with Tauri desktop app
AlexMikhalev Nov 5, 2025
9f44b21
chore: add compressed app bundles and GitHub release notes
AlexMikhalev Nov 5, 2025
5ab3d07
Fix Desktop app: Add missing role selector UI to ThemeSwitcher compon…
AlexMikhalev Nov 5, 2025
b54ba04
Fix system tray synchronization and add comprehensive test report for…
AlexMikhalev Nov 5, 2025
d947810
Fix GitHub Actions release workflows for all platforms
AlexMikhalev Nov 6, 2025
549b422
fix(tui): Replace add_command with register_command in tests
AlexMikhalev Nov 6, 2025
234cf34
chore(server): Update version to 1.0.0
AlexMikhalev Nov 6, 2025
b7ee3af
fix(desktop): Attempt to fix svelte-jsoneditor build issue
AlexMikhalev Nov 6, 2025
7e0efd8
fix(desktop): Remove svelte-jsoneditor dependency completely
AlexMikhalev Nov 6, 2025
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 5 additions & 0 deletions .cargo/config.toml
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
[profile.release]
panic = "unwind"
opt-level = 3
lto = true
codegen-units = 1
48 changes: 30 additions & 18 deletions .docs/summary-CLAUDE.md
Original file line number Diff line number Diff line change
@@ -1,24 +1,36 @@
# Summary: CLAUDE.md

## File Purpose
Primary guidance file for Claude Code when working with the Terraphim AI repository, providing comprehensive development guidelines and project context.
## Purpose
Provides comprehensive guidance to Claude Code (claude.ai/code) when working with the Terraphim AI codebase. Serves as the primary reference for AI assistants to understand project structure, conventions, and development workflows.

## Key Functionality
- **Development Guidelines**: Rust, async programming, and concurrent systems best practices
- **Memory Management**: Documentation organization using `.docs/` folder structure
- **Agent Instructions**: Consolidated machine-readable instructions for AI agents
- **Mandatory /init Command**: Two-step process for documentation maintenance
- **Rust & Async Programming Guidelines**: Tokio-based async patterns, channels, concurrency best practices
- **Project Architecture**: 29-crate workspace structure with specialized components for search, knowledge graphs, and AI integration
- **Development Commands**: Complete build, test, and deployment workflows including TUI, desktop app, and server
- **Testing Infrastructure**: Comprehensive testing strategy with unit, integration, and live tests; includes specialized testing scripts
- **Agent Systems**: Documents two agent systems (Superpowers Skills for workflows, Terraphim .agents for automation)
- **Configuration Management**: Role-based configs, feature flags, environment variables
- **Firecracker Integration**: Secure VM execution with sub-2 second boot times

## Important Details
- **Rust Expertise**: Focus on tokio async runtime, error handling, and performance
- **Documentation Organization**: Uses `.docs/` folder for file summaries and comprehensive overview
- **Agent Systems**: Two agent systems (Superpowers Skills and Terraphim .agents)
- **Quality Standards**: No mocks in tests, IDE diagnostics, comprehensive testing
- **Development Tools**: tmux for background tasks, gh tool for GitHub issues
## Critical Sections
- **Workspace Structure**: 29 library crates + 2 binaries (terraphim_server, terraphim_firecracker)
- **Critical Crates**: Service layer (terraphim_service, terraphim_middleware), agent system (6 crates), haystack integrations (4 crates)
- **Development Workflow**: Pre-commit hooks, testing scripts, watch commands, feature flags
- **Knowledge Graph System**: Thesaurus format, automata construction, rolegraph management
- **AI Integration**: OpenRouter, Ollama support with LLM client abstraction

## Recent Updates
- Added workspace structure section
- Expanded crate documentation (agent systems, haystacks)
- Added TUI build variations and feature flags
- Documented Firecracker integration
- Added testing scripts section
- Expanded desktop app development commands
- Added dependency constraints and troubleshooting

## Architecture Impact
- Establishes core development principles for the entire project
- Defines documentation structure and maintenance workflows
- Provides comprehensive guidance for AI agents working on the codebase
- Ensures consistent code quality and development practices
- Mandates systematic documentation updates through `/init` command
## Important Details
- Never use mocks in tests
- Pre-commit hooks mandatory for all commits
- Feature flags: `openrouter`, `mcp-rust-sdk`, `repl-full` for TUI
- Preferred storage backends: memory, dashmap, sqlite, redb (avoid RocksDB)
- Testing scripts location: `./quick-start-autocomplete.sh`, `./start-autocomplete-test.sh`
46 changes: 46 additions & 0 deletions .docs/summary-CONTRIBUTING.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,46 @@
# Summary: CONTRIBUTING.md

## Purpose
Guide for contributors to understand how to participate in the Terraphim AI project, covering setup, standards, and submission process.

## Key Sections
- **Getting Started**: Clone, install hooks, environment setup
- **Code Quality Standards**: Conventional Commits, cargo fmt, Biome for JS/TS
- **Development Workflow**: Feature branches, proper testing, pull requests
- **Dependency Management**: Version pinning constraints, Dependabot configuration

## Code Quality Requirements
- **Commit Format**: Conventional Commits (feat:, fix:, docs:, etc.)
- **Rust**: Automatic formatting with `cargo fmt`
- **JavaScript/TypeScript**: Biome for linting and formatting
- **Security**: No secrets or large files allowed in commits
- **Testing**: Proper test coverage required for all changes

## Development Process
1. Create feature branch: `git checkout -b feat/your-feature`
2. Make changes with proper tests
3. Commit with conventional format: `git commit -m "feat: add amazing feature"`
4. Push and create Pull Request
5. Automated checks must pass (format, lint, tests, build)

## Dependency Constraints
Critical pinned versions:
- `wiremock = "0.6.4"` - Version 0.6.5 uses unstable Rust features
- `schemars = "0.8.22"` - Version 1.0+ introduces breaking API changes
- `thiserror = "1.0.x"` - Version 2.0+ requires code migration

These are enforced in `.github/dependabot.yml` to prevent automatic breaking updates.

## Setup Commands
```bash
git clone https://github.com/terraphim/terraphim-ai.git
cd terraphim-ai
./scripts/install-hooks.sh # Sets up code quality tools
```

## Important Notes
- Pre-commit hooks are **required** (enforced in CI)
- All PRs must pass automated checks
- Follow existing code style and conventions
- Update documentation for user-facing changes
- Add tests for new functionality
44 changes: 44 additions & 0 deletions .docs/summary-Cargo.toml
Original file line number Diff line number Diff line change
@@ -0,0 +1,44 @@
# Summary: Cargo.toml

## Purpose
Workspace-level Cargo configuration for the Terraphim AI project. Defines workspace members, shared dependencies, and build settings for the entire multi-crate project.

## Key Functionality
- **Workspace Definition**: Organizes 29+ crates under unified workspace
- **Dependency Management**: Centralizes common dependencies for consistency across crates
- **Build Configuration**: Sets Rust edition 2024 and resolver version 2

## Workspace Structure
- **Members**:
- `crates/*` - All library crates (29 crates)
- `terraphim_server` - Main HTTP API server binary
- `desktop/src-tauri` - Tauri desktop application
- `terraphim_firecracker` - Firecracker microVM binary
- **Excluded**: `crates/terraphim_agent_application` (experimental with incomplete APIs)
- **Default Member**: `terraphim_server`

## Shared Dependencies
- **Async Runtime**: `tokio` 1.0 with full features
- **HTTP Client**: `reqwest` 0.12 with JSON and rustls-tls (no default features)
- **Serialization**: `serde` 1.0, `serde_json` 1.0
- **UUID**: `uuid` 1.0 with v4 and serde features
- **Time**: `chrono` 0.4 with serde
- **Async Utilities**: `async-trait` 0.1
- **Error Handling**: `thiserror` 1.0, `anyhow` 1.0
- **Logging**: `log` 0.4

## Patches
- **genai**: Patched to use terraphim fork from `https://github.com/terraphim/rust-genai.git` (main branch)

## Critical Details
- Edition 2024: Uses latest Rust edition features
- Resolver 2: Modern dependency resolution algorithm
- Centralized dependencies reduce version conflicts across workspace
- OpenRouter AI integration dependencies grouped together
- Default features disabled for reqwest (explicit feature selection)

## Build Implications
- All crates share same Rust edition and resolver
- Common dependencies version-locked across workspace
- Workspace-level features can be enabled/disabled per crate
- Patch enables custom genai fork without version conflicts
51 changes: 38 additions & 13 deletions .docs/summary-README.md
Original file line number Diff line number Diff line change
@@ -1,20 +1,45 @@
# Summary: README.md

## Purpose
This is the main project README for Terraphim AI, serving as the primary documentation and entry point for users, developers, and contributors.
Main project documentation for Terraphim AI, a privacy-first AI assistant that operates locally. Provides installation instructions, quick start guide, and project overview for end users and developers.

## Key Functionality
- Introduces Terraphim as a privacy-first AI assistant with local operation and deterministic behavior
- Explains core concepts: Haystacks (data sources), Knowledge Graphs, Profiles, Roles, and Rolegraphs
- Provides comprehensive getting started guide with quick start, installation methods (Homebrew, Docker, Debian), and development setup
- Documents configuration for storage backends (local vs cloud), environment variables, and deployment scenarios
- Includes terminology definitions, contribution guidelines, and code style standards
- Features build/lint/test commands for both Rust backend and Svelte frontend
- **Project Vision**: Privacy-first local AI assistant for searching across personal, team, and public knowledge repositories
- **Installation Methods**: Docker, Homebrew, Debian/Ubuntu packages, direct download, development setup
- **Quick Start**: Automated installation scripts, manual setup instructions
- **TUI Interface**: Comprehensive terminal UI with REPL, sub-2 second VM boot, markdown command system
- **Multiple Interfaces**: Backend server, web frontend (Svelte), desktop app (Tauri), terminal UI

## Critical Components
- **Haystack**: Data sources (folders, Notion, email, Jira, Confluence)
- **Knowledge Graph**: Structured information with node/edge relationships
- **Role**: User profiles with specific knowledge domains and search preferences
- **Rolegraph**: Knowledge graph using Aho-Corasick automata for ranking

## Installation Options
- **Docker**: `docker run ghcr.io/terraphim/terraphim-server:latest`
- **Homebrew**: `brew install terraphim/terraphim-ai/terraphim-ai`
- **Quick Install**: `curl -fsSL https://raw.githubusercontent.com/terraphim/terraphim-ai/main/release/v0.2.3/install.sh | bash`

## Development Setup
1. Clone repository
2. Install pre-commit hooks: `./scripts/install-hooks.sh`
3. Start backend: `cargo run`
4. Start frontend: `cd desktop && yarn run dev` (web) or `yarn run tauri dev` (desktop)
5. TUI: `cargo build -p terraphim_tui --features repl-full --release`

## Important Details
- Emphasizes privacy-first design with local infrastructure operation
- Supports multiple installation methods for end users and developers
- Includes detailed code style guidelines for Rust (Tokio, snake_case, Result<T,E>) and frontend (Svelte, Bulma CSS)
- Mandates conventional commits and pre-commit checks for contributions
- Documents deployment patterns using Caddy reverse proxy and 1Password CLI for secrets
- Contains troubleshooting guides and links to additional documentation
- Storage backends: Local by default (memory, dashmap, sqlite, redb); optional AWS S3 for cloud
- No cloud dependencies required for local development
- Dependency constraints enforced: wiremock 0.6.4, schemars 0.8.22, thiserror 1.0.x
- Code quality: Conventional Commits, cargo fmt, Biome for JS/TS, no secrets in commits
- License: Apache 2.0
- Trademark: Terraphim registered in UK, US, and internationally (WIPO)

## Key Features (TUI)
- AI Chat Integration (OpenRouter, Ollama)
- Sub-500ms VM allocation
- Firecracker microVM pools
- Markdown command system (YAML frontmatter)
- Security-first execution modes (Local, Firecracker, Hybrid)
- Knowledge graph validation before execution
54 changes: 54 additions & 0 deletions .docs/summary-TESTING_SCRIPTS_README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,54 @@
# Summary: TESTING_SCRIPTS_README.md

## Purpose
Comprehensive documentation for testing scripts used in Novel editor autocomplete integration with Terraphim's knowledge graph system. Provides automated testing workflows and service management.

## Key Scripts
- **quick-start-autocomplete.sh**: Interactive menu with preset configurations (full, mcp, dev, test, status, stop)
- **start-autocomplete-test.sh**: Main testing script with full control over services and configuration
- **stop-autocomplete-test.sh**: Clean shutdown of all testing services with graceful or force stop options
- **test-novel-autocomplete-integration.js**: Validation script for MCP server integration

## Services & Ports
- **MCP Server**: Port 8001 (default) - Autocomplete API
- **Axum Server**: Dynamic port - Alternative backend
- **Desktop App**: N/A - Tauri window for Novel editor UI

## Testing Scenarios
1. **Full Testing**: Everything needed (MCP + Desktop + Tests)
2. **MCP Only**: Backend development
3. **Desktop Development**: UI work
4. **Tests Only**: Validation
5. **Development Mode**: MCP + Desktop without tests

## Key Features
- Process management with PID files (`pids/`)
- Log aggregation (`logs/mcp_server.log`, `logs/desktop_app.log`)
- Real-time monitoring capabilities
- Force stop option for stuck processes
- Prerequisites checking (Rust, Node.js, commands)

## Command Options
**start-autocomplete-test.sh**:
- `-m, --mcp-only`: Start only MCP server
- `-d, --desktop-only`: Start only desktop app
- `-t, --test-only`: Run only integration tests
- `-p, --port PORT`: Set MCP server port (default: 8001)
- `--no-desktop`: Skip desktop app startup
- `--no-tests`: Skip integration tests
- `--verbose`: Enable verbose logging

**stop-autocomplete-test.sh**:
- `-s, --status`: Check status of running services
- `-f, --force`: Force kill all processes

## Troubleshooting
- Port conflicts: `lsof -i :8001` to find conflicting process
- MCP not responding: Check `logs/mcp_server.log`
- Desktop won't start: Verify `cd desktop && yarn install`
- No autocomplete: Verify MCP server running, backend connection, known terms

## Success Indicators
- MCP server: "βœ… MCP server is ready!"
- Desktop app: "βœ… Desktop app started (PID: 12345)"
- Integration tests: "βœ… autocomplete_terms working"
81 changes: 81 additions & 0 deletions .docs/summary-lessons-learned.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,81 @@
# Summary: lessons-learned.md

## Purpose
Captures critical technical insights, development patterns, and lessons from Terraphim AI implementation. Serves as knowledge repository for repeatable patterns and anti-patterns to avoid.

## Key Functionality
- **Pattern Discovery**: Documents successful implementation patterns for reuse
- **Anti-patterns**: Records mistakes and their solutions for future reference
- **Technology Insights**: Lessons about tools, frameworks, deployment strategies
- **Decision Context**: Explains why certain technical approaches were chosen
- **Best Practices**: Establishes project-specific development guidelines

## Critical Lessons

**Pattern 1: Pattern Discovery Through Reading Existing Code**
- **Context**: Deployment infrastructure implementation
- **Learning**: Always read existing scripts before creating new infrastructure
- **Example**: Reading `deploy-to-bigbox.sh` revealed correct Caddy+rsync pattern (not Docker/nginx)
- **When to Apply**: Any new feature deployment, integration with existing infrastructure

**Pattern 2: Vanilla JavaScript over Framework for Simple UIs**
- **Context**: TruthForge UI implementation
- **Learning**: No build step = instant deployment; check project patterns before choosing technology
- **Implementation**: Class-based separation (TruthForgeClient, TruthForgeUI), progressive enhancement
- **Benefits**: Static files work immediately, no compilation required

**Pattern 3: Rsync + Caddy Deployment Pattern**
- **Context**: Bigbox infrastructure deployment
- **Learning**: Project uses rsync for file copying, Caddy for reverse proxy (not Docker/nginx)
- **Steps**: Copy files β†’ Configure Caddy β†’ Update endpoints β†’ Start backend β†’ Verify
- **Example**: `alpha.truthforge.terraphim.cloud` deployment

**Pattern 4: 1Password CLI for Secret Management**
- **Context**: Production secret injection
- **Learning**: Use `op run --env-file=.env` in systemd services, never commit secrets
- **Benefits**: Centralized management, audit trail, automatic rotation
- **Implementation**: `.env` file contains vault references (`op://Shared/Key/field`)

**Pattern 5: Test-Driven Security Implementation**
- **Context**: Security vulnerability fixes
- **Learning**: Write tests first for security issues, then implement fixes
- **Categories**: Prompt injection, command injection, memory safety, network validation
- **Coverage**: 99 comprehensive tests across multiple attack vectors

## Technical Insights

**UI Development**:
- WebSocket client reusability from shared libraries (agent-workflows/shared/)
- Protocol-aware URL configuration for file:// vs http:// protocols
- Fallback mechanisms for graceful degradation

**Security**:
- Defense-in-depth patterns with multiple validation layers
- Unicode obfuscation detection critical for prompt sanitizers
- Concurrent security testing required for production readiness
- Regex catastrophic backtracking prevention in validation

**Deployment**:
- Phase-based deployment makes debugging easier (copy, configure, update, verify)
- Caddy reverse proxy with automatic HTTPS and zero-downtime reloads
- Systemd services with proper EnvironmentFile for secret loading

**Testing**:
- Browser automation (Playwright) for E2E validation
- Protocol validation prevents future regressions
- Comprehensive test suites build confidence in changes

## Anti-Patterns to Avoid
- Assuming Docker deployment without checking existing patterns
- Creating new infrastructure without reading existing scripts
- Using frameworks when vanilla JS suffices for simple UIs
- Storing secrets in .env files or environment variables
- Skipping security tests for "simple" changes
- Using blocking operations in async functions

## When to Apply These Lessons
- **Pattern Discovery**: Beginning any new feature or integration
- **Vanilla JS**: Building simple UIs, demos, or examples
- **Deployment Pattern**: Any production deployment or service configuration
- **Secret Management**: All production deployments with sensitive data
- **Security Testing**: Any code handling user input, system commands, or network operations
Loading
Loading