Skip to content

HDDS-14518. Support open keys and MPU in ContainerToKeyMapping tool#9703

Merged
sumitagrawl merged 7 commits intoapache:masterfrom
sarvekshayr:HDDS-14518
Feb 25, 2026
Merged

HDDS-14518. Support open keys and MPU in ContainerToKeyMapping tool#9703
sumitagrawl merged 7 commits intoapache:masterfrom
sarvekshayr:HDDS-14518

Conversation

@sarvekshayr
Copy link
Copy Markdown
Contributor

@sarvekshayr sarvekshayr commented Feb 3, 2026

What changes were proposed in this pull request?

Added a flag --in-progress to include open files/keys and multipart uploads in ContainerToKeyMapping tool.

What is the link to the Apache JIRA

HDDS-14518

How was this patch tested?

Only shows files and keys from fileTable and keyTable respectively.

ozone debug om container-key-mapping --db /data/metadata/om.db --containers 1,2,3,4
{
  "containers" : {
    "1" : {
      "keys" : [ "/vol1/fso-bucket/dir1/file1" ],
      "totalKeys" : 1
    },
    "2" : {
      "keys" : [ "/vol1/obs-bucket/key1" ],
      "totalKeys" : 1
    },
    "3" : {
      "keys" : [ ],
      "unreferencedKeys" : 1,
      "totalKeys" : 1
    },
    "4" : {
      "keys" : [ ],
      "totalKeys" : 0
    }
  }
}

Shows open files and keys from openFileTable and openKeyTable respectively. Also includes multipart uploads info from multipartInfoTable

ozone debug om container-key-mapping --db /data/metadata/om.db --containers 1,2,3,4 --in-progress
{
  "containers" : {
    "1" : {
      "keys" : [ "/vol1/fso-bucket/dir1/file1" ],
      "openKeys" : [ "/100/200/300/openFile/1" ],
      "totalKeys" : 2
    },
    "2" : {
      "keys" : [ "/vol1/obs-bucket/key1" ],
      "openKeys" : [ "/vol1/obs-bucket/openKey" ],
      "totalKeys" : 2
    },
    "3" : {
      "keys" : [ ],
      "openKeys" : [ ],
      "unreferencedKeys" : 1,
      "totalKeys" : 1
    },
    "4" : {
      "keys" : [ ],
      "openKeys" : [ "/vol1/obs-bucket/mpuKey/test-upload-id" ],
      "totalKeys" : 1
    }
  }
}

Show only file names (does not apply to in-progress files/keys)

ozone debug om container-key-mapping --db /data/metadata/om.db --containers 1,2,3,4 --in-progress --onlyFileNames
{
  "containers" : {
    "1" : {
      "keys" : [ "file1" ],
      "openKeys" : [ "/100/200/300/openFile/1" ],
      "totalKeys" : 2
    },
    "2" : {
      "keys" : [ "key1" ],
      "openKeys" : [ "/vol1/obs-bucket/openKey" ],
      "totalKeys" : 2
    },
    "3" : {
      "keys" : [ "unreferencedFile" ],
      "openKeys" : [ ],
      "totalKeys" : 1
    },
    "4" : {
      "keys" : [ ],
      "openKeys" : [ "/vol1/obs-bucket/mpuKey/test-upload-id" ],
      "totalKeys" : 1
    }
  }
}

Copy link
Copy Markdown
Contributor

@sreejasahithi sreejasahithi left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @sarvekshayr for working on this.
Could you please reduce the duplication between processOpenFiles() and processOpenKeys().

@sarvekshayr sarvekshayr added the tools Tools that helps with debugging label Feb 10, 2026
Copy link
Copy Markdown
Contributor

@sumitagrawl sumitagrawl left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@sarvekshayr Thanks for working over this, given few comments

Copy link
Copy Markdown
Contributor

@sumitagrawl sumitagrawl left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Plz check comments

@sumitagrawl sumitagrawl merged commit 2642f37 into apache:master Feb 25, 2026
46 checks passed
Copy link
Copy Markdown
Contributor

@sumitagrawl sumitagrawl left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

tools Tools that helps with debugging

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants