Skip to content

HDDS-15080. DirectoryDeletingService is using single thread#10125

Merged
ashishkumar50 merged 9 commits intoapache:masterfrom
aryangupta1998:HDDS-15080
Apr 29, 2026
Merged

HDDS-15080. DirectoryDeletingService is using single thread#10125
ashishkumar50 merged 9 commits intoapache:masterfrom
aryangupta1998:HDDS-15080

Conversation

@aryangupta1998
Copy link
Copy Markdown
Contributor

What changes were proposed in this pull request?

Switched DirectoryDeletingService to consistently use a fixed-size parallel deletion pool instead of the previous core-0 setup.
Unified pool creation so constructor/start/restart all follow the same path and behave the same after reconfiguration.
Also fixed thread-count reconfig handling so updated values are actually used by worker creation and pool recreation.

What is the link to the Apache JIRA

https://issues.apache.org/jira/browse/HDDS-15080

How was this patch tested?

Tested Manually on a cluster.

2026-04-22 21:05:50,910 [pool-37-thread-2] INFO org.apache.hadoop.ozone.om.service.DirectoryDeletingService: Number of dirs deleted: 224, Number of sub-dir deleted: 0, Number of sub-files moved: 447 to DeletedTable, Number of sub-dirs moved 110 to DeletedDirectoryTable, iteration elapsed: 445ms,  totalRunCount: 1
2026-04-22 21:05:50,921 [pool-37-thread-8] INFO org.apache.hadoop.ozone.om.service.DirectoryDeletingService: Number of dirs deleted: 159, Number of sub-dir deleted: 0, Number of sub-files moved: 568 to DeletedTable, Number of sub-dirs moved 89 to DeletedDirectoryTable, iteration elapsed: 456ms,  totalRunCount: 1
2026-04-22 21:05:50,929 [pool-37-thread-7] INFO org.apache.hadoop.ozone.om.service.DirectoryDeletingService: Number of dirs deleted: 262, Number of sub-dir deleted: 0, Number of sub-files moved: 339 to DeletedTable, Number of sub-dirs moved 135 to DeletedDirectoryTable, iteration elapsed: 465ms,  totalRunCount: 1
2026-04-22 21:05:50,940 [pool-37-thread-6] INFO org.apache.hadoop.ozone.om.service.DirectoryDeletingService: Number of dirs deleted: 211, Number of sub-dir deleted: 0, Number of sub-files moved: 437 to DeletedTable, Number of sub-dirs moved 133 to DeletedDirectoryTable, iteration elapsed: 475ms,  totalRunCount: 1
2026-04-22 21:05:50,951 [pool-37-thread-9] INFO org.apache.hadoop.ozone.om.service.DirectoryDeletingService: Number of dirs deleted: 221, Number of sub-dir deleted: 0, Number of sub-files moved: 474 to DeletedTable, Number of sub-dirs moved 116 to DeletedDirectoryTable, iteration elapsed: 487ms,  totalRunCount: 1

Copy link
Copy Markdown
Contributor

@ptlrs ptlrs left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the patch @aryangupta1998. LGTM

Copy link
Copy Markdown
Contributor

@adoroszlai adoroszlai left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @aryangupta1998 for the patch.

Copy link
Copy Markdown
Contributor

@ashishkumar50 ashishkumar50 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@aryangupta1998 Thanks for working on this, can you also have a test to see if multiple threads are executing in parallel.

@ashishkumar50
Copy link
Copy Markdown
Contributor

@aryangupta1998 Can you add a test if multiple threads are executing in parallel.

Copy link
Copy Markdown
Contributor

@ptlrs ptlrs left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the updates @aryangupta1998. LGTM

Copy link
Copy Markdown
Contributor

@ashishkumar50 ashishkumar50 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@aryangupta1998 Thanks for updating patch, LGTM.

@ashishkumar50 ashishkumar50 merged commit be850c1 into apache:master Apr 29, 2026
89 of 91 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants