Skip to content

Feature/iotconsensus snap multi dir#16811

Merged
jt2594838 merged 14 commits into
apache:masterfrom
hongzhi-gao:feature/iotconsensus-snap-multi-dir
Jun 3, 2026
Merged

Feature/iotconsensus snap multi dir#16811
jt2594838 merged 14 commits into
apache:masterfrom
hongzhi-gao:feature/iotconsensus-snap-multi-dir

Conversation

@hongzhi-gao

@hongzhi-gao hongzhi-gao commented Nov 26, 2025

Copy link
Copy Markdown
Contributor

Description

  1. FolderManager relocated to commons module for consensus module accessibility.

img_v3_02sl_2c9a9abc-2110-482d-a770-5f9717ee5f5g


This PR has:

  • been self-reviewed.
    • concurrent read
    • concurrent write
    • concurrent read and write
  • added documentation for new or modified features or behaviors.
  • added Javadocs for most classes and all non-trivial methods.
  • added or updated version, license, or notice information
  • added comments explaining the "why" and the intent of the code wherever would not be obvious
    for an unfamiliar reader.
  • added unit tests or modified existing tests to cover new code paths, ensuring the threshold
    for code coverage.
  • added integration tests.
  • been tested in a test IoTDB cluster.

Key changed/added classes (or packages if there are too many classes) in this PR

@jt2594838

Copy link
Copy Markdown
Contributor

Should include some compatibility design, like adding storageDir to storageDirs.

# Conflicts:
#	iotdb-core/datanode/src/main/java/org/apache/iotdb/db/pipe/receiver/protocol/thrift/IoTDBDataNodeReceiver.java
#	iotdb-core/datanode/src/main/java/org/apache/iotdb/db/storageengine/load/util/LoadUtil.java
@hongzhi-gao
hongzhi-gao force-pushed the feature/iotconsensus-snap-multi-dir branch from 85037c6 to 7f86ba6 Compare December 4, 2025 09:58
…-snap-multi-dir

# Conflicts:
#	iotdb-core/consensus/src/main/java/org/apache/iotdb/consensus/iot/IoTConsensusServerImpl.java
#	iotdb-core/datanode/src/main/java/org/apache/iotdb/db/pipe/receiver/protocol/thrift/IoTDBDataNodeReceiver.java
#	iotdb-core/datanode/src/main/java/org/apache/iotdb/db/pipe/sink/util/builder/PipeTsFileBuilder.java
#	iotdb-core/datanode/src/main/java/org/apache/iotdb/db/storageengine/dataregion/DataRegion.java
#	iotdb-core/datanode/src/main/java/org/apache/iotdb/db/storageengine/dataregion/compaction/execute/task/InnerSpaceCompactionTask.java
#	iotdb-core/datanode/src/main/java/org/apache/iotdb/db/storageengine/dataregion/compaction/selector/impl/SizeTieredCompactionSelector.java
#	iotdb-core/datanode/src/main/java/org/apache/iotdb/db/storageengine/dataregion/snapshot/SnapshotLoader.java
#	iotdb-core/datanode/src/main/java/org/apache/iotdb/db/storageengine/dataregion/tsfile/generator/TsFileNameGenerator.java
#	iotdb-core/datanode/src/main/java/org/apache/iotdb/db/storageengine/dataregion/wal/allocation/AbstractNodeAllocationStrategy.java
#	iotdb-core/datanode/src/main/java/org/apache/iotdb/db/storageengine/load/util/LoadUtil.java
#	iotdb-core/datanode/src/main/java/org/apache/iotdb/db/storageengine/rescon/disk/TierManager.java
#	iotdb-core/datanode/src/test/java/org/apache/iotdb/db/storageengine/dataregion/snapshot/IoTDBSnapshotTest.java
#	iotdb-core/node-commons/src/main/java/org/apache/iotdb/commons/disk/FolderManager.java
#	iotdb-core/node-commons/src/main/java/org/apache/iotdb/commons/disk/strategy/DirectoryStrategy.java
#	iotdb-core/node-commons/src/main/java/org/apache/iotdb/commons/disk/strategy/MinFolderOccupiedSpaceFirstStrategy.java
@luoluoyuyu
luoluoyuyu force-pushed the feature/iotconsensus-snap-multi-dir branch from a1720e0 to 8ff057c Compare June 3, 2026 04:15
@jt2594838
jt2594838 merged commit c818b28 into apache:master Jun 3, 2026
41 checks passed
MileaRobertStefan pushed a commit to MileaRobertStefan/iotdb that referenced this pull request Jun 26, 2026
* move FolderManager to node-common module

* move FolderManager to node-common module

* Multi-directory snapshot support for IoTConsensus receiver

* Revert "Multi-directory snapshot support for IoTConsensus receiver"

This reverts commit 1fef8e4.

* IoTConsensus receiver snapshot multi dirs

* fix imports

* Add keep same disk when loading snapshot

* Use commons i18n for disk messages

* Fix multi directory strategy package prefix

* Fix IoTConsensus snapshot fragment placement

* Use local data dirs for IoTConsensus snapshot receive

* Honor folder availability for same disk selection

---------

Co-authored-by: luoluoyuyu <zhenyu@apache.org>
JackieTien97 pushed a commit that referenced this pull request Jun 29, 2026
* move FolderManager to node-common module

* move FolderManager to node-common module

* Multi-directory snapshot support for IoTConsensus receiver

* Revert "Multi-directory snapshot support for IoTConsensus receiver"

This reverts commit 1fef8e4.

* IoTConsensus receiver snapshot multi dirs

* fix imports

* Add keep same disk when loading snapshot

* Use commons i18n for disk messages

* Fix multi directory strategy package prefix

* Fix IoTConsensus snapshot fragment placement

* Use local data dirs for IoTConsensus snapshot receive

* Honor folder availability for same disk selection

---------

Co-authored-by: luoluoyuyu <zhenyu@apache.org>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants