Persistent monitor events for off-chain outbound fails

We are in the process of moving the resolution of off-chain HTLCs from the
Channel to the ChannelMonitor, via MonitorEvents. This will simplify how we
reconstruct/reconcile the ChannelManager's pending HTLC set on startup and
avoid needing to persist pending HTLCs explicitly in the manager, as we can
just replay pending monitor events to reconstruct the set instead.

In a recent commit, we started generating persistent monitor events whenever an
htlc is failed off-chain, but those events were no-ops at the time.

In this commit, we make those monitor events the sole driver of off-chain
outbound payment failure. When `persistent_monitor_events` is enabled, we will
skip the existing failure path for these HTLCs and instead fail them when the
monitor event is processed. Similar to what we do for claims, the monitor event
will not be acked and will continue to be re-provided on startup until the
resulting PaymentFailed event is processed by the user.

Forward and on-chain failure paths are unchanged and will be migrated
separately as part of the broader refactor.

Author: valentinewallace

lightning/src/ln/channelmanager.rs

@@ -10547,6 +10547,13 @@ This indicates a bug inside LDK. Please report this error at https://github.com/
 		self.forward_htlcs(htlc_forwards);
 		self.finalize_claims(finalized_claimed_htlcs);
 		for failure in failed_htlcs {
+			if self.persistent_monitor_events
+				&& matches!(failure.0, HTLCSource::OutboundRoute { .. })
+			{
+				// The MonitorEvent::HTLCEvent generated when the previous counterparty commitment
+				// is pruned will drive the failure instead.
+				continue;
+			}
 			let failure_type = failure.0.failure_type(counterparty_node_id, channel_id);
 			self.fail_htlc_backwards_internal(
 				&failure.0,
@@ -13971,26 +13978,40 @@ This indicates a bug inside LDK. Please report this error at https://github.com/
 							OutboundHTLCResolution::Failed { reason } => {
 								let from_onchain =
 									self.channel_is_closed(&channel_id, &counterparty_node_id);
-								if from_onchain {
+								let we_are_sender =
+									matches!(htlc_update.source, HTLCSource::OutboundRoute { .. });
+								if from_onchain | we_are_sender {
 									log_trace!(logger, "Failing HTLC from our monitor");
 									let failure_type = htlc_update
 										.source
 										.failure_type(counterparty_node_id, channel_id);
-									let completion_update = Some(EventCompletionAction::ReleasePaymentCompleteChannelMonitorUpdate(PaymentCompleteUpdate {
-										counterparty_node_id,
-										channel_funding_outpoint: funding_outpoint,
-										channel_id,
-										htlc_id: SentHTLCId::from_source(&htlc_update.source),
-									}));
+
+									let completion_update = if self.persistent_monitor_events
+										&& we_are_sender && !from_onchain
+									{
+										EventCompletionAction::AckMonitorEvent {
+											event_id: monitor_event_source,
+										}
+									} else {
+										EventCompletionAction::ReleasePaymentCompleteChannelMonitorUpdate(PaymentCompleteUpdate {
+											counterparty_node_id,
+											channel_funding_outpoint: funding_outpoint,
+											channel_id,
+											htlc_id: SentHTLCId::from_source(&htlc_update.source),
+										})
+									};
+
 									self.fail_htlc_backwards_internal(
 										&htlc_update.source,
 										&htlc_update.payment_hash,
 										&reason,
 										failure_type,
-										completion_update,
+										Some(completion_update),
 									);
 								}
-								self.chain_monitor.ack_monitor_event(monitor_event_source);
+								if from_onchain | !we_are_sender {
+									self.chain_monitor.ack_monitor_event(monitor_event_source);
+								}
 							},
 						}
 					},