tool-discovery.mdx

title	Tool Discovery Implementation
description	Technical implementation of MCP server tool discovery in DeployStack Satellite - unified architecture supporting both HTTP/SSE remote servers and stdio subprocess servers.

DeployStack Satellite implements automatic tool discovery from MCP servers across both HTTP/SSE remote endpoints and stdio subprocess servers. This unified system provides dynamic tool availability without manual configuration.

**Current Implementation**: Tool discovery fully supports both HTTP/SSE remote MCP servers and stdio subprocess servers through a unified architecture. The `UnifiedToolDiscoveryManager` coordinates discovery across both transport types, merging tools into a single cache.

This document focuses on the internal tool discovery mechanism. To learn how tools are exposed to MCP clients through the hierarchical router pattern, see Hierarchical Router Implementation.

The overall satellite architecture is documented in Satellite Architecture Design. MCP transport protocol details can be found in MCP Transport Protocols.

Technical Overview

Unified Discovery Architecture

Tool discovery operates through three coordinated managers that handle different transport types and merge results:

┌─────────────────────────────────────────────────────────────────────────────────┐
│                     Unified Tool Discovery Architecture                         │
│                                                                                 │
│  ┌─────────────────────────────────────────────────────────────────┐           │
│  │              UnifiedToolDiscoveryManager                        │           │
│  │                                                                 │           │
│  │  • Coordinates both HTTP and stdio discovery                   │           │
│  │  • Merges tools from both managers                            │           │
│  │  • Single interface for MCP clients                           │           │
│  └─────────────────────────────────────────────────────────────────┘           │
│                            │                  │                                 │
│                            ▼                  ▼                                 │
│  ┌───────────────────────────┐    ┌───────────────────────────┐               │
│  │ RemoteToolDiscoveryManager │    │ StdioToolDiscoveryManager │               │
│  │                           │    │                           │               │
│  │ • HTTP/SSE servers        │    │ • stdio subprocesses      │               │
│  │ • Startup discovery       │    │ • Post-spawn discovery    │               │
│  │ • SSE parsing             │    │ • JSON-RPC over stdin/out │               │
│  │ • Static configuration    │    │ • Process lifecycle aware │               │
│  └───────────────────────────┘    └───────────────────────────┘               │
└─────────────────────────────────────────────────────────────────────────────────┘

Core Components

UnifiedToolDiscoveryManager:

• Coordinates tool discovery across both HTTP/SSE and stdio transport types
• Merges discovered tools from both managers into a unified cache
• Routes discovery requests to the appropriate transport type
• Provides single interface for MCP protocol handlers

RemoteToolDiscoveryManager:

• Queries remote HTTP/SSE MCP servers during startup
• Parses Server-Sent Events responses
• Maintains in-memory cache with namespacing
• Handles differential configuration updates

StdioToolDiscoveryManager:

• Discovers tools from stdio subprocess MCP servers
• Executes discovery after process spawn and handshake
• Tools persist in cache even when processes go dormant
• Tracks tools by server with namespacing

Discovery Process by Transport Type

HTTP/SSE Discovery (Startup)

Remote HTTP/SSE servers are discovered during satellite initialization:

Startup → Config Load → HTTP Servers → Query tools/list → Cache Tools → Ready
    │           │             │               │               │          │
 Init      Enabled Only   POST Request    Parse Response   Namespace   Serve

HTTP Discovery Flow:

1. Load enabled HTTP/SSE servers from dynamic configuration
2. Query each server with tools/list JSON-RPC request
3. Parse SSE or JSON responses
4. Cache tools with namespacing (server_slug-tool_name)
5. Expose through MCP transport endpoints

stdio Discovery (Post-Spawn)

stdio subprocess servers are discovered after process spawning:

Process Spawn → Handshake → Running → Discover Tools → Cache → Auto-Cleanup
      │             │          │            │           │           │
  Backend Cmd   Initialize   Status     tools/list   Namespace   On Exit

stdio Discovery Flow:

1. Process spawned via Backend command
2. MCP handshake completes (initialize + initialized)
3. Discovery triggered automatically after handshake
4. Tools cached with namespacing (server_slug-tool_name)
5. Tools persist in cache even when process terminates (for fast respawn)

Discovery Timing Differences

HTTP/SSE (Eager):

• Discovered at startup before serving requests
• All HTTP tools available immediately
• Configuration changes trigger rediscovery

stdio (Lazy):

• Discovered after process spawn completes
• Tools become available post-handshake
• Tools persist even when process goes dormant (enables fast respawn)

Tool Caching Strategy

Unified Cache Design

Both transport types use identical caching and namespacing:

interface UnifiedCachedTool {
  serverName: string;           // Installation name
  originalName: string;         // Tool name from server
  namespacedName: string;       // server_slug-tool_name
  description: string;          // Tool description
  inputSchema: object;          // JSON Schema
  transport: 'stdio' | 'http';  // Transport type for routing
  discoveredAt?: Date;          // Discovery timestamp (HTTP only)
}

Cache Characteristics:

• Unified Namespace: Same format across both transport types
• Memory Storage: No persistent storage or database
• Persistent Caching: stdio tools remain cached even when processes go dormant
• Conflict Prevention: server_slug ensures unique names

Namespacing Strategy

Both HTTP and stdio tools use identical namespacing:

HTTP Tool Example:
  Server Slug: "context7"
  Original: "resolve-library-id"
  Namespaced: "context7-resolve-library-id"

stdio Tool Example:
  Server Slug: "filesystem" (extracted from "filesystem-john-abc123")
  Original: "read_file"
  Namespaced: "filesystem-read_file"

Namespacing Rules:

• Format: {server_slug}-{originalToolName}
• HTTP: Uses server_slug from configuration
• stdio: Extracts slug from installation name
• Routing: Internal server names used for team isolation
• User Display: Friendly namespaced names shown to clients

For team-based server resolution, see Team Isolation Implementation.

Configuration Management

Dynamic Configuration Updates

The unified manager handles configuration changes intelligently:

Differential Updates:

• Only discovers tools for added/modified servers
• Preserves tools for unchanged servers
• Removes tools for deleted servers
• Minimizes network overhead and latency

Configuration Sources:

• HTTP/SSE: Static configuration from Backend polling
• stdio: Dynamic spawning via Backend commands
• Both: Support three-tier configuration system

Tool Execution Flow

Transport-Aware Routing

Tool execution routes to the correct transport based on discovery:

MCP Client → tools/call → Parse Name → Lookup Tool → Route by Transport
    │            │            │            │              │
  Request    Namespaced   Extract Slug   Get Cache    stdio/HTTP/SSE

HTTP Transport:

• Routes to remote HTTP/SSE endpoint
• Uses HTTP Proxy Manager
• Handles SSE streaming responses

stdio Transport:

• Routes to local subprocess
• Uses ProcessManager JSON-RPC
• Communicates over stdin/stdout

Error Handling & Recovery

Discovery Failures

Both managers implement graceful failure handling:

HTTP Discovery:

• Server unreachable → Skip and continue
• Parse errors → Log and skip malformed tools
• Timeout → Mark server as failed

stdio Discovery:

• Process not running → Error with status check
• No tools returned → Empty array (valid response)
• Communication failure → Process restart logic

Automatic Cleanup

stdio tools persist in cache for optimal performance:

Process Lifecycle:

• Spawn: Tools discovered after handshake
• Running: Tools available for execution
• Idle/Dormant: Process terminated, tools remain cached for fast respawn
• Respawn: Process restarts automatically, tools already available (no rediscovery)
• Uninstall: Tools cleared only when server is explicitly removed

**Idle Process Management**: stdio processes that remain inactive for the configured idle timeout (default: 3 minutes) are automatically terminated to save memory. However, **tools remain cached** so when a client requests them, the process respawns instantly without needing to rediscover tools. This reduces respawn time from 1-3 seconds to 1-2 seconds. See [Idle Process Management](/development/satellite/idle-process-management) for details.

Tool Metadata Collection

After tool discovery completes, the satellite emits tool metadata to the backend for storage and analysis.

Event Emission (Post-Discovery)

Following successful tool discovery (both HTTP/SSE and stdio), the satellite:

1. Calculates token consumption using the token-counter.ts utility
2. Builds event payload with tool metadata including per-tool token counts
3. Emits mcp.tools.discovered event to backend via EventBus
4. Backend stores metadata in mcpToolMetadata table for team visibility

Event Payload Structure:

{
  installation_id: string;
  installation_name: string;
  team_id: string;
  server_slug: string;
  tool_count: number;           // Total tools discovered
  total_tokens: number;         // Sum of all tool token counts
  tools: Array<{
    tool_name: string;
    description: string;
    input_schema: Record<string, unknown>;
    token_count: number;        // Tokens for this specific tool
  }>;
  discovered_at: string;        // ISO 8601 timestamp
}

Integration Points:

• StdioToolDiscoveryManager: Emits after stdio tool discovery completes
• RemoteToolDiscoveryManager: Emits after HTTP/SSE tool discovery completes
• EventBus: Batches events every 3 seconds for efficient transmission
• Backend handler: Stores tools with delete-then-insert strategy

Token Calculation: The satellite uses estimateMcpServerTokens() from token-counter.ts to calculate:

• Per-tool tokens: name + description + JSON.stringify(inputSchema)
• Total server tokens: Sum of all tool tokens
• Uses gpt-tokenizer library (provider-agnostic)

Purpose:

• Store tool metadata in backend database for team visibility
• Calculate hierarchical router token savings (traditional vs 2-meta-tool approach)
• Enable frontend tool catalog display with token consumption metrics
• Provide analytics on MCP server complexity and context window usage

See Event System for event batching and delivery details.

Development Considerations

Debugging Support

The debug endpoint shows tools from both transport types:

curl http://localhost:3001/api/status/debug

Debug Information:

• Tools grouped by transport type (HTTP/stdio)
• Tools grouped by server name
• Discovery statistics for both managers
• Process status for stdio servers

**Security Notice**: The debug endpoint exposes detailed system information. Disable in production with `DEPLOYSTACK_STATUS_SHOW_MCP_DEBUG_ROUTE=false`.

Performance Characteristics

HTTP/SSE Performance

• Discovery Time: 2-5 seconds at startup
• Memory: ~1KB per tool
• Overhead: Single HTTP request per server
• Caching: Persistent until configuration change

stdio Performance

• Discovery Time: 1-2 seconds post-spawn (first time only)
• Memory: ~1KB per tool (persists even when process dormant)
• Overhead: Single JSON-RPC request per process (cached for respawns)
• Caching: Persistent - tools remain even when process goes dormant

Scalability

Combined Limits:

• No hard server limit for either transport
• Memory-bound by total tool count
• HTTP: Limited by network connection pool
• stdio: Limited by system process limits

**Implementation Status**: Tool discovery is fully operational for both HTTP/SSE remote servers and stdio subprocess servers. The unified manager successfully coordinates discovery, merges tools, and routes execution requests to the appropriate transport.

Future Enhancements

Dynamic Capabilities

Planned Features:

• Runtime refresh for HTTP servers without restart
• Configuration hot-reload for both transport types
• Health monitoring with automatic server detection
• Tool versioning support

Advanced Features

Under Consideration:

• Load balancing across multiple server instances
• Circuit breakers for automatic failure recovery
• Detailed usage and performance analytics
• Cache persistence for faster startup (HTTP only)

The unified tool discovery implementation provides a solid foundation for multi-transport MCP server integration while maintaining simplicity and reliability for development and production use.