ARCHITECTURE.md

Monocle Architecture

This document describes the architecture of the monocle project: a BGP information toolkit that can be used as both a Rust library, a command-line application, and an HTTP/SSE server.

Goals and Design Principles

1. Library-first: the core capability lives in the library; the CLI is a thin wrapper.
2. Clear separation of concerns:
   • persistence and caching in database/
   • domain operations in lens/
   • presentation and UX concerns in the CLI (bin/)
3. Extensible: new functionality should be added as new lenses (and optionally wired into the CLI).
4. Composability: lenses should be usable programmatically and in batch/automation contexts.

Layering Rules

The codebase follows strict layering rules to maintain separation of concerns:

Repository Layer (database/)

Repositories are data access only:

• CRUD operations (Create, Read, Update, Delete)
• Query methods that return raw data
• No business logic or policy decisions
• No output formatting

Lens Layer (lens/)

Lenses contain business logic and policy:

• Interpretation of data (e.g., RPKI validation logic)
• Coordination between multiple repositories
• Output formatting
• Cache refresh decisions

CLI Layer (bin/)

CLI commands are thin wrappers:

• Argument parsing
• Output format selection
• Progress display
• Error presentation

High-level Architecture

Monocle is organized into three primary layers:

• CLI layer (src/bin/):
  • parses flags/arguments
  • loads configuration
  • selects output format
  • calls into lenses
• Lens layer (src/lens/):
  • provides "use-case" APIs (e.g., search, parse, RPKI lookups)
  • controls output shaping via a unified OutputFormat
  • uses database/ and external libraries as needed
• Database layer (src/database/):
  • manages storage (SQLite), schema initialization, and caching primitives
  • contains repositories for specific datasets

Directory Structure

src/
├── lib.rs                    # Library entry point / exports
├── config.rs                 # Configuration and shared status helpers
│
├── database/                 # Persistence + caching
│   ├── mod.rs
│   ├── README.md
│   │
│   ├── core/                 # Connection + schema
│   │   ├── mod.rs
│   │   ├── connection.rs
│   │   └── schema.rs
│   │
│   ├── session/              # Ephemeral / per-run databases
│   │   ├── mod.rs
│   │   └── msg_store.rs
│   │
│   └── monocle/              # Main persistent monocle database
│       ├── mod.rs
│       ├── asinfo.rs         # Unified AS information (from bgpkit-commons)
│       ├── as2rel.rs         # AS relationships
│       ├── rpki.rs           # ROAs/ASPAs cache (SQLite with blob prefixes)
│       └── pfx2as.rs         # Prefix-to-ASN mappings (SQLite with blob prefixes)
│
├── lens/                     # Business logic ("use-cases")
│   ├── mod.rs
│   ├── README.md
│   ├── utils.rs              # OutputFormat, formatting helpers
│   ├── country.rs            # Country code/name lookup
│   │
│   ├── as2rel/               # AS relationship lens
│   │   ├── mod.rs
│   │   ├── args.rs
│   │   └── types.rs
│   │
│   ├── inspect/              # Unified AS/prefix inspection
│   │   ├── mod.rs            # InspectLens implementation
│   │   └── types.rs          # Result types, section selection
│   │
│   ├── ip/                   # IP information lookup
│   │   └── mod.rs
│   │
│   ├── parse/                # MRT file parsing
│   │   └── mod.rs
│   │
│   ├── pfx2as/               # Prefix-to-ASN mapping types
│   │   └── mod.rs            # Types only; repository handles lookups
│   │
│   ├── rpki/                 # RPKI validation and data
│   │   ├── mod.rs            # RpkiLens with validation logic
│   │   └── commons.rs        # bgpkit-commons integration
│   │
│   ├── search/               # BGP message search
│   │   ├── mod.rs
│   │   └── query_builder.rs
│   │
│   └── time/                 # Time parsing and formatting
│       └── mod.rs
│
├── server/                   # HTTP/SSE server (requires `server` feature)
│   ├── mod.rs                # Server state, startup, /health, auth wiring
│   ├── http.rs               # REST router, API error types, system/info
│   ├── search.rs             # SSE search streaming handler
│   ├── auth.rs               # Token auth middleware
│   └── rest/                 # REST endpoint handlers
│       ├── mod.rs
│       ├── time.rs           # time/parse
│       ├── country.rs        # country/lookup
│       ├── ip.rs             # ip/lookup, ip/public
│       ├── rpki.rs           # rpki/roa/lookup, rpki/aspa/lookup, rpki/roa/validate
│       ├── pfx2as.rs         # pfx2as/lookup
│       ├── as2rel.rs         # as2rel/search, as2rel/relationship, as2rel/refresh
│       ├── inspect.rs        # inspect/query
│       └── database.rs       # database/status, database/refresh, inspect/refresh
│
└── bin/
    ├── monocle.rs            # CLI entry point
    └── commands/             # Command handlers (thin wrappers around lenses)
        ├── as2rel.rs
        ├── config.rs         # Config display + update, backup, sources
        ├── country.rs
        ├── inspect.rs        # Unified inspect command (replaces whois, pfx2as)
        ├── ip.rs
        ├── parse.rs
        ├── rpki.rs
        ├── search.rs
        └── time.rs

Key Modules

inspect - Unified AS/Prefix Information

The inspect command and lens consolidate multiple data sources into a single query interface:

• ASInfo: Core AS data from bgpkit-commons (replaces as2org)
• Connectivity: AS2Rel-based upstream/peer/downstream relationships
• RPKI: ROAs and ASPA records
• Pfx2as: Prefix-to-ASN mappings

Features:

• Auto-detects query type (ASN, prefix, IP address, or name)
• Section selection (--show basic/prefixes/connectivity/rpki/all)
• Display limits with --full, --full-roas, --full-prefixes, --full-connectivity
• Auto-refresh of stale data
• Multiple output formats

rpki - RPKI Validation

The RPKI lens (RpkiLens) provides:

• Validation logic (RFC 6811): Valid/Invalid/NotFound states
• Cache management: Uses RpkiRepository for current data (SQLite with blob prefixes)
• Historical queries: Uses bgpkit-commons for date-specific lookups

Layering:

• RpkiRepository (database): Raw data access only (CRUD, prefix range queries)
• RpkiLens (lens): Validation logic, cache refresh, formatting

pfx2as - Prefix-to-ASN Mapping

The Pfx2as repository (Pfx2asRepository) provides:

• Lookup modes: Exact, longest prefix match, covering (supernets), covered (subnets)
• ASN queries: Get all prefixes for an ASN
• SQLite storage: IP prefixes stored as 16-byte start/end address pairs
• Cache management: 24-hour TTL with automatic refresh

Note: The file-based cache has been removed; all pfx2as data now uses SQLite.

server - HTTP/SSE API

The HTTP/SSE server (monocle server) provides programmatic access to monocle functionality:

• Protocol: HTTP REST with JSON request/response bodies
• Streaming: SSE (text/event-stream) for search with progress, element batches, and terminal events
• Cancellation: Client closes HTTP connection; server detects drop via Arc<AtomicBool>
• Backpressure: Bounded mpsc channel (capacity 32); element batches never dropped
• Terminal invariant: Exactly one terminal event (completed, cancelled, or error)
• Auth: Optional token-based middleware (Authorization: Bearer <token>)
• DB access: Per-request MonocleDatabase in spawn_blocking (connection is !Send)

Available endpoint groups:

• /health: Health check (always open)
• /api/v1/system/*: Server introspection
• /api/v1/search/*: SSE streaming BGP search
• /api/v1/time/*, /api/v1/ip/*, /api/v1/country/*: Utility lookups
• /api/v1/rpki/*: RPKI ROA/ASPA lookup and validation
• /api/v1/as2rel/*: AS relationship queries
• /api/v1/pfx2as/*: Prefix-to-ASN mapping
• /api/v1/inspect/*: Unified AS/prefix inspection
• /api/v1/database/*: Database status and refresh

Module Architecture

config/ (Configuration + Status Reporting)

Responsibilities:

• compute default paths and load config file overrides
• provide shared helpers used by config CLI command to display:
  • SQLite database info (size, table counts, last update time)
  • cache settings and cache directory info
  • database management (refresh, backup, sources)

This module is intentionally "infra-ish": it should not implement domain logic.

database/ (Persistence and Caching)

The database module handles local persistence and caches shared across commands.

database/core/

Responsibilities:

• create/configure SQLite connections
• define and initialize schema
• expose schema/version checks (used on open)

Notes:

• schema management is owned here; higher-level modules should not issue CREATE TABLE etc.

database/monocle/

Responsibilities:

• main persistent monocle dataset store (SQLite DB under the monocle data directory)
• repositories for datasets:
  • ASInfo (unified AS information from bgpkit-commons)
  • AS2Rel (AS-level relationships)
  • RPKI (ROAs/ASPAs with blob-based prefix storage)
  • Pfx2as (prefix-to-ASN mappings with blob-based prefix storage)
• file cache helpers for auxiliary file-based caching

Key idea:

• MonocleDatabase is the entry point for accessing the persistent DB/repositories.

database/session/

Responsibilities:

• short-lived, per-operation SQLite storage (e.g., search results)
• optimized for write-heavy temporary usage and easy export

lens/ (Business Logic / Use-cases)

Lenses are the primary public-facing API surface for functionality. A lens:

• takes a &MonocleDatabase reference for data access
• defines argument types (often serde-serializable; optionally clap-derivable under cli)
• defines result types (serde-serializable)
• performs the operation (may call into Broker, Parser, SQLite repositories, file caches, etc.)
• emits output using the unified OutputFormat

Output formatting

lens/utils.rs contains the global OutputFormat used across the CLI to keep formatting consistent and predictable.

Progress reporting

Certain lenses support progress callbacks (e.g., parse/search). Progress types are designed to be:

• thread-safe (Send + Sync)
• serializable (for GUI or other frontends)

bin/ (CLI)

The CLI layer wires together:

• clap argument parsing
• config loading
• output selection (--format, --json)
• invocation of lens operations
• printing human-readable messages to stderr and data output to stdout (to support piping)

The CLI should not duplicate core logic. It should:

• validate/normalize CLI inputs
• call library APIs
• format/print results

Typical Data Flows

CLI flow (conceptual)

1. User runs monocle <command> ...
2. CLI parses args and loads MonocleConfig
3. CLI determines OutputFormat
4. CLI opens MonocleDatabase and constructs the lens
5. Lens executes operation:
   • uses repository for data access
   • applies business logic
   • returns typed results (and optionally progress events via callback)
6. CLI prints results via OutputFormat

Library flow (conceptual)

1. Application creates (or opens) the MonocleDatabase
2. Application constructs lens with database reference
3. Application calls lens methods
4. Application consumes typed results directly, or uses OutputFormat to format for display

HTTP/SSE flow (conceptual)

1. Client sends HTTP request (POST with JSON body, or GET)
2. Axum router dispatches to handler
3. Handler opens MonocleDatabase in spawn_blocking (for DB-backed endpoints)
4. Handler calls lens methods, formats result as JSON response
5. For search: handler returns Sse stream; worker sends events via bounded channel
6. Client disconnect cancels the search worker via Arc<AtomicBool>

Feature Flags

Monocle supports conditional compilation via Cargo features with a simplified three-tier structure:

Feature Hierarchy

cli (default)
 ├── server
 │    └── lib
 └── lib

Quick Guide:

• Need the CLI binary? Use cli (includes everything)
• Need HTTP/SSE server without CLI? Use server (includes lib)
• Need only library/data access? Use lib (database + all lenses + display)

Feature Descriptions

Feature	Description	Key Dependencies
lib	Complete library: database + all lenses + display	rusqlite, bgpkit-parser, bgpkit-broker, tabled, etc.
server	HTTP/SSE server (implies lib)	axum, tokio, serde_json
cli	Full CLI binary with progress bars (implies lib and server)	clap, indicatif

Use Case Scenarios

Scenario 1: Library Only

Features: lib

Use when building applications that need:

• Database operations (SQLite, data loading)
• All lenses (TimeLens, ParseLens, SearchLens, RPKI, Country, InspectLens, etc.)
• Table formatting with tabled

monocle = { version = "1.1", default-features = false, features = ["lib"] }

use monocle::database::MonocleDatabase;
use monocle::lens::inspect::{InspectLens, InspectQueryOptions};

let db = MonocleDatabase::open_in_dir("~/.local/share/monocle")?;
let lens = InspectLens::new(&db);
let result = lens.query("AS13335", &InspectQueryOptions::default())?;

Scenario 2: Library with HTTP/SSE Server

Features: server

Use when building applications that need:

• Everything in lib
• HTTP/SSE server for remote API access

monocle = { version = "1.1", default-features = false, features = ["server"] }

use monocle::config::MonocleConfig;
use monocle::server::start_server;

let config = MonocleConfig::new(&None)?;
// Start HTTP/SSE server on default port
start_server(config).await?;

Scenario 3: CLI Binary (Default)

Features: cli (default)

The full CLI binary with all features, HTTP/SSE server, and terminal UI:

monocle = "1.1"

Or explicitly:

monocle = { version = "1.1", features = ["cli"] }

Valid Feature Combinations

All of these combinations compile successfully:

Combination	Use Case
(none)	Config types only, no functionality
lib	Full library functionality
server	Library + HTTP/SSE server
cli	Full CLI (includes everything)

Feature Dependencies

When you enable a higher-tier feature, lower-tier features are automatically included:

• server → automatically enables lib
• cli → automatically enables lib and server

Related Documents

• README.md — user-facing CLI and library overview
• CHANGELOG.md — version history and breaking changes
• DEVELOPMENT.md — contributor guide for adding lenses and fixing bugs
• src/server/README.md — HTTP/SSE API specification
• src/database/README.md — database module notes
• src/lens/README.md — lens module patterns and conventions
• examples/README.md — example code organized by feature tier