Skip to content

Commit 2c97001

Browse files
committed
fix(cluster): release descriptor drain on every DDL exit path
The implicit drain clear only ran after a successful catalog apply, so outcomes that write nothing — an entry superseded during replay, or an if-absent create for a descriptor that already exists — left the drain installed. Every read of that descriptor was then rejected as a retryable schema change until the drain TTL lapsed. Extract the clear into a helper and call it from every exit path that concludes the DDL.
1 parent 55da2be commit 2c97001

1 file changed

Lines changed: 32 additions & 5 deletions

File tree

nodedb/src/control/cluster/metadata_applier/catalog_ddl.rs

Lines changed: 32 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -15,6 +15,30 @@ use super::audit::emit_ddl_audit;
1515
use super::types::MetadataCommitApplier;
1616

1717
impl MetadataCommitApplier {
18+
/// Release the descriptor drain a `Put*` DDL installed.
19+
///
20+
/// A drain is proposed *before* the DDL and is meant to end when that DDL
21+
/// concludes. Concluding includes the outcomes that write nothing — an
22+
/// entry superseded during replay, or an if-absent create for a descriptor
23+
/// that already exists. The drain is keyed to the DDL, not to whether the
24+
/// catalog changed, so every path that finishes handling the entry must
25+
/// clear it.
26+
///
27+
/// Missing one of those paths does not fail loudly: the drain simply
28+
/// survives, and `is_draining` then rejects every plan for that descriptor
29+
/// as a retryable schema change until the TTL (`max_wait +
30+
/// DRAIN_TTL_GRACE`) lapses — the collection reads as broken for a minute
31+
/// with no error explaining why.
32+
fn clear_implicit_drain(&self, stamped: &catalog_entry::CatalogEntry) {
33+
if let Some(weak) = self.shared.get()
34+
&& let Some(shared) = weak.upgrade()
35+
&& let Some(drained_id) =
36+
crate::control::lease::drain_propose::descriptor_id_for_implicit_clear(stamped)
37+
{
38+
shared.lease_drain.install_end(&drained_id);
39+
}
40+
}
41+
1842
pub(super) fn apply_catalog_ddl(
1943
&self,
2044
entry: &MetadataEntry,
@@ -72,11 +96,18 @@ impl MetadataCommitApplier {
7296
kind = stamped.kind(),
7397
"catalog_entry: descriptor entry already superseded or applied"
7498
);
99+
// The DDL that installed the drain is over even though this entry
100+
// changed nothing — release it, or every read of the descriptor
101+
// stays rejected until the drain TTL lapses.
102+
self.clear_implicit_drain(&stamped);
75103
return Ok(());
76104
}
77105

78106
debug!(kind = stamped.kind(), "catalog_entry: applying to redb");
79107
if !catalog_entry::apply::apply_to(&stamped, catalog) {
108+
// A `Put*` that wrote nothing (e.g. an if-absent create for a
109+
// descriptor that already exists) still concludes its DDL.
110+
self.clear_implicit_drain(&stamped);
80111
return Ok(());
81112
}
82113
// Implicit drain clear: if the entry is a `Put*` for one
@@ -88,11 +119,7 @@ impl MetadataCommitApplier {
88119
if let Some(weak) = self.shared.get()
89120
&& let Some(shared) = weak.upgrade()
90121
{
91-
if let Some(drained_id) =
92-
crate::control::lease::drain_propose::descriptor_id_for_implicit_clear(&stamped)
93-
{
94-
shared.lease_drain.install_end(&drained_id);
95-
}
122+
self.clear_implicit_drain(&stamped);
96123
// Run synchronous post-apply side effects INLINE so every
97124
// in-memory cache update (install_replicated_user,
98125
// install_replicated_owner, etc.) is visible before the

0 commit comments

Comments
 (0)