BoostBridge is a Clojure application that serves as a bridge between the Lightning Network's bolt11 invoices and keysend payments for Podcasting 2.0. The system handles the entire flow from user boost requests to final value distribution to podcast participants.
- Web Interface: A REST API (using Reitit) and simple HTML views for user interaction, including QR code generation.
- Event-Driven Core: An event sourcing pattern manages state. A single-threaded event processor ensures sequential, durable processing of commands.
- Lightning Network Integration: Communicates with a Core Lightning (CLN) node via its REST API for bolt11 invoice creation and keysend payment execution.
- Data Management: Uses Datalevin for both the event log and materialized views, providing a durable, queryable data store.
- Boost Request: A user submits a boost via the web form.
- Event Creation: A
boost-requestedevent is created and logged. - Invoice Generation: An associated bolt11 invoice is created via the CLN node.
- Payment Monitoring: The system watches for invoice payment using the
pay_index. - Value Distribution: Upon payment, the system distributes value to the podcast's splits via keysend, using intent logging to ensure at-most-once execution for payments.
- State Updates: All state changes (for boosts, splits, etc.) are driven by events.
The system is built on an event sourcing model. Commands produce events, which are stored in an immutable log. A single-threaded processor consumes these events to update materialized views and trigger side-effects (like payments).
Rationale:
- Durability & Auditability: Critical for handling payments. The event log provides a complete, replayable history of the system's state.
- Simplicity: A single-threaded processor simplifies reasoning about state changes and avoids concurrency issues.
- Observability: The entire system's history is captured in the event log, making debugging and analysis straightforward.
The processor itself is implemented as a loop that blocks on a core.async channel. When a new event is submitted, a message is put on this channel to wake the processor up. This channel uses a dropping buffer of size 1, ensuring that event producers never block and that multiple rapid events only trigger a single processing cycle. This provides a simple, robust back-pressure mechanism.
An actor-based model was considered but rejected due to the complexity of ensuring durability and recovering from crashes, which would have required re-implementing many features that event sourcing provides naturally.
A key challenge is managing the mix of idempotent and non-idempotent operations.
- Idempotent Operations: Database updates, view materialization, and invoice creation (with deterministic IDs) can be safely retried.
- Non-Idempotent Operations: Sending keysend payments moves real funds and must not be executed more than once.
The system handles this by using an intent log pattern. The intent to send a payment is recorded first. The actual payment is a separate step. This ensures that even if the process restarts, we can determine if a payment has already been sent, guaranteeing at-most-once execution.
The initial design uses a single in-process event loop, which is sufficient for V1. Future scaling can be achieved through several patterns without a full rewrite:
- Partitioning: The event stream can be partitioned by
boost-id, allowing multiple workers (in-process or distributed) to process events concurrently. - External Event Store: For larger scale, the in-process event log can be replaced with a dedicated system like Kafka, RedPanda, or Redis Streams.
The current architecture provides a clear path to scaling as needed.
The application is composed of components managed by the init library, ensuring clear dependency management and a well-defined startup/shutdown lifecycle.