Skip to content

Commit 879bfeb

Browse files
committed
storage: admit tiered_cloud partitions to space management
tiered_cloud local log segments were never reclaimed by space management, so a saturating produce workload fills local disk and cascades to KAFKA_STORAGE_ERROR while legacy tiered storage stays bounded. The reclaim executor (ctp_stm) and the offset channel (compute_gc_offset returning a pinned _cloud_gc_offset) already exist; cloud topics were only gated out of the eviction-policy decision side. Relax the four is_cloud_retention_active / remote_partition gates to admit tiered_cloud partitions, bounded by the reconciled L1 offset (max_removable). Spike for bench validation; the _cloud_gc_offset clear/refresh semantics for ctp_stm still need work.
1 parent 3103408 commit 879bfeb

2 files changed

Lines changed: 10 additions & 4 deletions

File tree

src/v/resource_mgmt/storage.cc

Lines changed: 7 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -289,7 +289,13 @@ eviction_policy::collect_reclaimable_offsets() {
289289
*/
290290
chunked_vector<ss::lw_shared_ptr<cluster::partition>> partitions;
291291
for (const auto& p : _pm->local().partitions()) {
292-
if (!p.second->remote_partition()) {
292+
// SPIKE(space-mgmt): also admit tiered_cloud partitions. Their local log
293+
// is a cache bounded by the reconciled (L1) offset; the pinned
294+
// _cloud_gc_offset flows to ctp_stm via compute_gc_offset. (cloud mode
295+
// already trims to max_removable and ignores the pin, so exclude it.)
296+
if (
297+
!p.second->remote_partition()
298+
&& !p.second->log()->config().is_tiered_cloud()) {
293299
continue;
294300
}
295301
partitions.push_back(p.second);

src/v/storage/disk_log_impl.cc

Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -4279,7 +4279,7 @@ ss::future<usage_report> disk_log_impl::disk_usage(gc_config cfg) {
42794279
chunked_vector<ss::lw_shared_ptr<segment>>
42804280
disk_log_impl::cloud_gc_eligible_segments() {
42814281
vassert(
4282-
is_cloud_retention_active(),
4282+
is_cloud_retention_active() || config().is_tiered_cloud(),
42834283
"Expected {} to have cloud retention enabled",
42844284
config().ntp());
42854285

@@ -4313,7 +4313,7 @@ disk_log_impl::cloud_gc_eligible_segments() {
43134313
}
43144314

43154315
void disk_log_impl::set_cloud_gc_offset(model::offset offset) {
4316-
if (!is_cloud_retention_active()) {
4316+
if (!is_cloud_retention_active() && !config().is_tiered_cloud()) {
43174317
vlog(
43184318
stlog.debug,
43194319
"Ignoring request to set GC offset on non-cloud enabled partition "
@@ -4338,7 +4338,7 @@ disk_log_impl::get_reclaimable_offsets(gc_config cfg) {
43384338

43394339
reclaimable_offsets res;
43404340

4341-
if (!is_cloud_retention_active()) {
4341+
if (!is_cloud_retention_active() && !config().is_tiered_cloud()) {
43424342
vlog(
43434343
stlog.debug,
43444344
"Reporting no reclaimable offsets for non-cloud partition {}",

0 commit comments

Comments
 (0)