Skip to content

operations.md

Latest commit

 

History

History
171 lines (125 loc) · 5.74 KB

operations.md

File metadata and controls

171 lines (125 loc) · 5.74 KB

Operations Guide

This guide covers running LDK Server in production: process management, backups, security, monitoring, and remote access.

Process Management

systemd

LDK Server integrates with systemd via sd_notify. It sends READY=1 after the gRPC listener is bound and STOPPING=1 when shutting down. A sample unit file can be found in contrib/ldk-server.service.

Graceful Shutdown

The server handles SIGTERM and CTRL-C (SIGINT). On receipt, it:

  1. Signals all active streaming clients (SubscribeEvents) to disconnect
  2. Stops the LDK Node (persists channel state)
  3. Exits cleanly

Log Rotation

By default, LDK Server logs to stdout/stderr. When running under systemd or Docker, this allows the environment (e.g., journald) to handle persistence, rotation, and compression automatically.

If you enable log_to_file in the configuration, LDK Server will automatically rotate logs when they exceed 50MB or 24 hours (configurable) and keep the last 5 uncompressed log files.

If you prefer to use system logrotate for file logs, the server still reopens its log file on SIGHUP. Save the following config to /etc/logrotate.d/ldk-server (adjust the log path to match your setup):

/var/lib/ldk-server/regtest/ldk-server.log {
    daily
    rotate 14
    compress
    missingok
    notifempty
    postrotate
        systemctl kill --signal=HUP ldk-server.service
    endscript
}

Backups

What to Back Up

File Priority Description
<storage_dir>/keys_seed Critical Node identity and master secret. Required to recover on-chain funds.
<network_dir>/ldk_node_data.sqlite Critical Channel state and on-chain wallet data. Required to recover channel funds.
<network_dir>/ldk_server_data.sqlite Nice-to-have Payment and forwarding history

What is Reconstructable

  • Network graph data (re-synced from gossip or RGS)
  • Fee rate cache (re-fetched from the chain backend)
  • The API key (can be regenerated, but clients will need the new one)
  • The TLS certificate (can be regenerated, but clients will need the new one)

Warning: Do not restore a backup onto two running nodes simultaneously. Running the same node identity on two instances will cause channel state conflicts and potential fund loss.

Security

API Key

  • Auto-generated as 32 random bytes on first startup
  • Stored at <network_dir>/api_key with 0400 permissions (read-only for owner)
  • The hex-encoded form of this key is used for HMAC authentication
  • Treat it as a secret: anyone with the API key and network access to the gRPC port can control the node

TLS

  • Self-signed ECDSA P-256 certificate generated automatically
  • Private key stored at <storage_dir>/tls.key with 0400 permissions
  • Certificate includes localhost and 127.0.0.1 in SANs by default
  • Add your server's hostname/IP to [tls] hosts for remote access

Network Exposure

The gRPC service binds to 127.0.0.1:3536 by default. For remote access, either:

  1. Change grpc_service_address to bind to 0.0.0.0:3536 and add the server's hostname to [tls] hosts, or
  2. Use a reverse proxy (e.g., nginx, Caddy) that terminates TLS and forwards to the loopback address

Monitoring

Prometheus Metrics

LDK Server can expose metrics in Prometheus text format. Prometheus is an open-source monitoring toolkit that scrapes HTTP endpoints and stores time-series data for alerting and dashboards.

Enable metrics in the config:

[metrics]
enabled = true
poll_metrics_interval = 60

Metrics are served at GET /metrics on the same port as the gRPC service (default 3536). This is a plain HTTP endpoint (not gRPC), returning Prometheus text format.

Basic Auth is recommended to prevent unauthorized access to node metrics:

[metrics]
enabled = true
username = "prometheus"
password = "secret"

The Prometheus scrape config would then use:

scrape_configs:
  - job_name: ldk-server
    scheme: https
    tls_config:
      ca_file: /path/to/tls.crt
    basic_auth:
      username: prometheus
      password: secret
    static_configs:
      - targets: [ 'localhost:3536' ]

Available Metrics

Metrics cover:

  • On-chain and Lightning balances
  • Public and Private Channel counts
  • Payment counts (successful, failed, pending)
  • Peer count

Remote Access

To allow clients to connect from other machines:

  1. Update TLS hosts: Add the server's hostname or IP to [tls] hosts in the config so the certificate's SANs cover the address clients will use.
  2. Update bind address: Set grpc_service_address to bind on the appropriate interface (e.g., 0.0.0.0:3536).
  3. Distribute the TLS certificate: Copy <storage_dir>/tls.crt to each client machine. Clients must pin this certificate since it is self-signed.
  4. Share the API key: Provide the hex-encoded API key to authorized clients.

If you regenerate the TLS certificate (by deleting tls.crt and tls.key and restarting), all clients will need the new certificate.

Network-Specific Notes

Data is stored in per-network subdirectories (bitcoin/, testnet/, signet/, regtest/, etc.) under the storage root. This means you can run multiple networks from one storage directory without conflicts.

The keys_seed file is shared across networks (stored at the storage root, not per-network). Keys are split by network at the derivation path level, so the same seed will produce different keys.