Skip to content

Commit 96c4efa

Browse files
[Fix] Sync Tablet::_cumulative_point with TabletMeta::_cumulative_layer_point
## Problem Description There is a critical synchronization issue between the in-memory cumulative point and its persistent storage: - Tablet::_cumulative_point (in-memory, runtime value) - TabletMeta::_cumulative_layer_point (persistent, stored in RocksDB) The root cause is that these two values were never properly synchronized: 1. After compaction updates _cumulative_point, it was never written to _cumulative_layer_point 2. On BE restart, Tablet constructor hardcoded _cumulative_point to -1, ignoring the value loaded from RocksDB 3. This caused cumulative compaction to restart from scratch after every BE restart ## Data Flow Analysis ### BE Startup/Restart Flow: ``` DataDir::load() → TabletMetaManager::traverse_headers() [iterate RocksDB] → TabletManager::load_tablet_from_meta(meta_binary from RocksDB) → TabletMeta::deserialize(meta_binary) → TabletMeta::init_from_pb() → _cumulative_layer_point = tablet_meta_pb.cumulative_layer_point() → std::make_shared<Tablet>(_engine, tablet_meta, data_dir) → Tablet::Tablet() constructor ``` **Before this fix**: Constructor hardcoded _cumulative_point = -1, losing RocksDB value **After this fix**: Constructor loads _cumulative_point from TabletMeta ### Compaction Update Flow: ``` CumulativeCompaction::execute_compact() → update_cumulative_point() → Tablet::set_cumulative_layer_point(new_point) [updates _cumulative_point] → Tablet::save_meta() ``` **Before this fix**: save_meta() only saved TabletMeta without syncing _cumulative_point **After this fix**: save_meta() syncs _cumulative_point to TabletMeta before persisting ## Solution This commit adds bidirectional synchronization: 1. **Load path** (tablet.cpp:260): ```cpp _cumulative_point(_tablet_meta->cumulative_layer_point()) ``` Initialize from TabletMeta on construction (BE restart/clone/restore) 2. **Save path** (tablet.cpp:341): ```cpp _tablet_meta->set_cumulative_layer_point(_cumulative_point); ``` Sync to TabletMeta before persisting to RocksDB ## Impact ### Fixed Scenarios: - ✅ BE restart: Cumulative point persists across restarts - ✅ Clone: Target replica inherits correct cumulative point - ✅ Restore: Restored tablet keeps original cumulative point ### New Tablet Creation: - Still correctly initializes to -1 (TabletMeta constructor sets it to -1) ## Related Issue This also resolves the existing TODO comment: ```cpp // TODO(ygl): lost some information here, such as cumulative layer point // engine_storage_migration_task.cpp:348 ```
1 parent 9ec8abb commit 96c4efa

1 file changed

Lines changed: 3 additions & 1 deletion

File tree

be/src/olap/tablet.cpp

Lines changed: 3 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -257,7 +257,7 @@ Tablet::Tablet(StorageEngine& engine, TabletMetaSharedPtr tablet_meta, DataDir*
257257
_last_cumu_compaction_success_millis(0),
258258
_last_base_compaction_success_millis(0),
259259
_last_full_compaction_success_millis(0),
260-
_cumulative_point(K_INVALID_CUMULATIVE_POINT),
260+
_cumulative_point(_tablet_meta->cumulative_layer_point()),
261261
_newly_created_rowset_num(0),
262262
_last_checkpoint_time(0),
263263
_cumulative_compaction_type(cumulative_compaction_type),
@@ -337,6 +337,8 @@ Status Tablet::init() {
337337
// if it's a primary replica
338338
void Tablet::save_meta() {
339339
check_table_size_correctness();
340+
// Sync the in-memory cumulative point to tablet meta before saving
341+
_tablet_meta->set_cumulative_layer_point(_cumulative_point);
340342
auto res = _tablet_meta->save_meta(_data_dir);
341343
CHECK_EQ(res, Status::OK()) << "fail to save tablet_meta. res=" << res
342344
<< ", root=" << _data_dir->path();

0 commit comments

Comments
 (0)