Pass instanceId to mongodb metadata backend by welansari · Pull Request #5826 · scality/cloudserver

welansari · 2025-06-12T14:16:47Z

instanceId will be used when generating versionIds.
This will ensure uniqueness of the versionID across
all Cloudserver pods in the cluster.

The instanceId is a combination of the short hash
contained in the name of the pods, which is unique
across all pods within the same deployment, and a
prefix for differenciating the different types of
deployments (internal vs external).

see. https://scality.atlassian.net/wiki/spaces/OS/pages/3225583692/VersionID+Collisions+in+Zenko

Related PR: scality/Arsenal#2445

Issue: CLDSRV-665

bert-e · 2025-06-12T14:16:50Z

Hello kerkesni,

My role is to assist you with the merge of this
pull request. Please type @bert-e help to get information
on this process, or consult the user documentation.

Available options

name	description	privileged	authored
`/after_pull_request`	Wait for the given pull request id to be merged before continuing with the current one.
`/bypass_author_approval`	Bypass the pull request author's approval	⭐
`/bypass_build_status`	Bypass the build and test status	⭐
`/bypass_commit_size`	Bypass the check on the size of the changeset `TBA`	⭐
`/bypass_incompatible_branch`	Bypass the check on the source branch prefix	⭐
`/bypass_jira_check`	Bypass the Jira issue check	⭐
`/bypass_peer_approval`	Bypass the pull request peers' approval	⭐
`/bypass_leader_approval`	Bypass the pull request leaders' approval	⭐
`/approve`	Instruct Bert-E that the author has approved the pull request.		✍️
`/create_pull_requests`	Allow the creation of integration pull requests.
`/create_integration_branches`	Allow the creation of integration branches.
`/no_octopus`	Prevent Wall-E from doing any octopus merge and use multiple consecutive merge instead
`/unanimity`	Change review acceptance criteria from `one reviewer at least` to `all reviewers`
`/wait`	Instruct Bert-E not to run until further notice.

Available commands

name	description	privileged
`/help`	Print Bert-E's manual in the pull request.
`/status`	Print Bert-E's current status in the pull request `TBA`
`/clear`	Remove all comments from Bert-E from the history `TBA`
`/retry`	Re-start a fresh build `TBA`
`/build`	Re-start a fresh build `TBA`
`/force_reset`	Delete integration branches & pull requests, and restart merge process from the beginning.
`/reset`	Try to remove integration branches unless there are commits on them which do not appear on the source branch.

Status report is not available.

codecov · 2025-06-12T14:19:57Z

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 65.11%. Comparing base (b6a78b9) to head (17cab88).
Report is 8 commits behind head on development/8.8.

✅ All tests successful. No failed tests found.

Additional details and impacted files

Files with missing lines	Coverage Δ
lib/Config.js	`75.68% <100.00%> (+0.36%)`	⬆️
lib/metadata/wrapper.js	`55.55% <ø> (ø)`

@@                 Coverage Diff                 @@
##           development/8.8    #5826      +/-   ##
===================================================
+ Coverage            65.08%   65.11%   +0.03%     
===================================================
  Files                  188      188              
  Lines                11948    11961      +13     
===================================================
+ Hits                  7776     7789      +13     
  Misses                4172     4172

Flag	Coverage Δ
unit	`65.11% <100.00%> (+0.03%)`	⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

🚀 New features to boost your workflow:

📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

bert-e · 2025-06-12T14:20:06Z

Request integration branches

Waiting for integration branch creation to be requested by the user.

To request integration branches, please comment on this pull request with the following command:

/create_integration_branches

Alternatively, the /approve and /create_pull_requests commands will automatically
create the integration branches.

ghost

LGTM, as discussed, let's add the base62 as the default during tests as well, so we are actually testing what is used in production

lib/Config.js

tests/unit/Config.js

lib/Config.js

welansari · 2025-07-01T17:04:33Z

Note for reviewers, i updated this PR following the changes in the TAD.
We no longer use the replicationGroupId as a local unique id but a new field called instanceId.

ghost · 2025-07-02T08:21:44Z

@Kerkesni do we want to target 8.8? Some clients might not be ready to upgrade to the latest product version right away

bert-e · 2025-07-02T09:08:53Z

Request integration branches

Waiting for integration branch creation to be requested by the user.

To request integration branches, please comment on this pull request with the following command:

/create_integration_branches

Alternatively, the /approve and /create_pull_requests commands will automatically
create the integration branches.

welansari · 2025-07-02T09:09:07Z

@williamlardier yes forgot about that part, thanks

ghost

lgtm otherwise, but let's wait for the finalized Arsenal part

lib/Config.js

bert-e · 2025-07-21T10:28:40Z

Integration data created

I have created the integration data for the additional destination branches.

this pull request will merge bugfix/CLDSRV-665 into
development/8.8
w/9.0/bugfix/CLDSRV-665 will be merged into development/9.0
w/9.1/bugfix/CLDSRV-665 will be merged into development/9.1

The following branches will NOT be impacted:

development/7.10
development/7.4
development/7.70

You can set option create_pull_requests if you need me to create
integration pull requests in addition to integration branches, with:

@bert-e create_pull_requests

The following options are set: create_integration_branches

bert-e · 2025-07-21T10:28:41Z

Waiting for approval

The following approvals are needed before I can proceed with the merge:

the author
2 peers

The following options are set: create_integration_branches

lib/Config.js

.github/workflows/tests.yaml

francoisferrand · 2025-07-21T15:26:34Z

.github/docker/docker-compose.yaml

      - SCUBA_HEALTHCHECK_FREQUENCY
      - S3QUOTA
      - QUOTA_ENABLE_INFLIGHTS
+      - S3_VERSION_ID_ENCODING_TYPE


instead of setting this here, should we (can we?) set it automatically in config.js when backend is mongo ?

This will need quite a rework as S3_VERSION_ID_ENCODING_TYPE is used by the versionID generation code inside of Arsenal, it's not a simple change in Cloudserver

bert-e · 2025-07-22T11:27:22Z

Waiting for approval

The following approvals are needed before I can proceed with the merge:

the author
2 peers

The following options are set: create_integration_branches

instanceId will be used when generating versionIds. This will ensure uniqueness of the versionID across all Cloudserver pods in the cluster. The instanceId is a combination of the short hash contained in the name of the pods, which is unique across all pods within the same deployment, and a prefix for differenciating the different types of deployments (internal vs external). see. https://scality.atlassian.net/wiki/spaces/OS/pages/3225583692/VersionID+Collisions+in+Zenko Issue: CLDSRV-665

Cloudserver in Zenko always uses base62 encoding, so we enforce it in the mongo tests. Issue: CLDSRV-665

Issue: CLDSRV-665

bert-e · 2025-07-22T11:45:06Z

History mismatch

Merge commit #e1fb0290322cc75e24b8e90c8fd2a9d4e6cd4177 on the integration branch
w/9.0/bugfix/CLDSRV-665 is merging a branch which is neither the current
branch bugfix/CLDSRV-665 nor the development branch
development/9.0.

It is likely due to a rebase of the branch bugfix/CLDSRV-665 and the
merge is not possible until all related w/* branches are deleted or updated.

Please use the reset command to have me reinitialize these branches.

The following options are set: create_integration_branches

welansari · 2025-07-22T11:59:22Z

/force_reset

bert-e · 2025-07-22T11:59:31Z

Reset complete

I have successfully deleted this pull request's integration branches.

The following options are set: create_integration_branches

bert-e · 2025-07-22T12:01:51Z

Integration data created

I have created the integration data for the additional destination branches.

this pull request will merge bugfix/CLDSRV-665 into
development/8.8
w/9.0/bugfix/CLDSRV-665 will be merged into development/9.0
w/9.1/bugfix/CLDSRV-665 will be merged into development/9.1

The following branches will NOT be impacted:

development/7.10
development/7.4
development/7.70

You can set option create_pull_requests if you need me to create
integration pull requests in addition to integration branches, with:

@bert-e create_pull_requests

The following options are set: create_integration_branches

bert-e · 2025-07-22T12:01:52Z

Waiting for approval

The following approvals are needed before I can proceed with the merge:

the author
2 peers

The following options are set: create_integration_branches

welansari · 2025-07-22T12:20:40Z

/approve

bert-e · 2025-07-22T12:20:57Z

In the queue

The changeset has received all authorizations and has been added to the
relevant queue(s). The queue(s) will be merged in the target development
branch(es) as soon as builds have passed.

The changeset will be merged in:

✔️ development/8.8
✔️ development/9.0
✔️ development/9.1

The following branches will NOT be impacted:

development/7.10
development/7.4
development/7.70

There is no action required on your side. You will be notified here once
the changeset has been merged. In the unlikely event that the changeset
fails permanently on the queue, a member of the admin team will
contact you to help resolve the matter.

IMPORTANT

Please do not attempt to modify this pull request.

Any commit you add on the source branch will trigger a new cycle after the
current queue is merged.
Any commit you add on one of the integration branches will be lost.

If you need this pull request to be removed from the queue, please contact a
member of the admin team now.

The following options are set: approve, create_integration_branches

bert-e · 2025-07-22T12:39:56Z

I have successfully merged the changeset of this pull request
into targetted development branches:

✔️ development/8.8
✔️ development/9.0
✔️ development/9.1

The following branches have NOT changed:

development/7.10
development/7.4
development/7.70

Please check the status of the associated issue CLDSRV-665.

Goodbye kerkesni.

scality deleted a comment from bert-e Jun 12, 2025

welansari requested review from a user and francoisferrand June 12, 2025 14:35

ghost reviewed Jun 12, 2025

View reviewed changes

lib/Config.js Outdated Show resolved Hide resolved

welansari force-pushed the bugfix/CLDSRV-665 branch 2 times, most recently from 187bb49 to 526ed7b Compare June 12, 2025 15:05

ghost approved these changes Jun 12, 2025

View reviewed changes

welansari force-pushed the bugfix/CLDSRV-665 branch from 526ed7b to 892f4cd Compare June 12, 2025 15:15

jonathan-gramain approved these changes Jun 13, 2025

View reviewed changes

tests/unit/Config.js Outdated Show resolved Hide resolved

lib/Config.js Outdated Show resolved Hide resolved

welansari force-pushed the bugfix/CLDSRV-665 branch 2 times, most recently from 06d051c to 85ab804 Compare July 1, 2025 17:00

welansari changed the title ~~remove hardcoded replicationGroupId and generate from pod name~~ Pass instanceId to mongodb metadata backend Jul 1, 2025

welansari mentioned this pull request Jul 1, 2025

support new versionId format scality/Arsenal#2445

Merged

welansari requested review from a user and jonathan-gramain July 1, 2025 17:05

welansari force-pushed the bugfix/CLDSRV-665 branch from 85ab804 to 6db3320 Compare July 2, 2025 09:08

welansari changed the base branch from development/9.0 to development/8.8 July 2, 2025 09:08

scality deleted a comment from bert-e Jul 2, 2025

welansari force-pushed the bugfix/CLDSRV-665 branch 3 times, most recently from 735f683 to 3e340d4 Compare July 2, 2025 13:46

ghost reviewed Jul 2, 2025

View reviewed changes

lib/Config.js Outdated Show resolved Hide resolved

francoisferrand reviewed Jul 21, 2025

View reviewed changes

lib/Config.js Show resolved Hide resolved

francoisferrand reviewed Jul 21, 2025

View reviewed changes

.github/workflows/tests.yaml Outdated Show resolved Hide resolved

francoisferrand reviewed Jul 21, 2025

View reviewed changes

francoisferrand approved these changes Jul 21, 2025

View reviewed changes

welansari force-pushed the bugfix/CLDSRV-665 branch 3 times, most recently from 09cba5d to 01a80c2 Compare July 22, 2025 10:56

scality deleted a comment from bert-e Jul 22, 2025

Kerkesni added 4 commits July 22, 2025 13:44

run mongo functional tests with base62 versionId encoding

98a23de

Cloudserver in Zenko always uses base62 encoding, so we enforce it in the mongo tests. Issue: CLDSRV-665

update arsenal to include versionID changes

eb2a9e1

Issue: CLDSRV-665

bump version to 8.8.55

17cab88

Issue: CLDSRV-665

welansari force-pushed the bugfix/CLDSRV-665 branch from 01a80c2 to 17cab88 Compare July 22, 2025 11:44

scality deleted a comment from bert-e Jul 22, 2025

bert-e merged commit 766ecba into development/8.8 Jul 22, 2025
26 checks passed

bert-e deleted the bugfix/CLDSRV-665 branch July 22, 2025 12:40

Conversation

welansari commented Jun 12, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

bert-e commented Jun 12, 2025

Hello kerkesni,

Uh oh!

codecov bot commented Jun 12, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

bert-e commented Jun 12, 2025

Request integration branches

Uh oh!

ghost left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

welansari commented Jul 1, 2025

Uh oh!

ghost commented Jul 2, 2025

Uh oh!

bert-e commented Jul 2, 2025

Request integration branches

Uh oh!

welansari commented Jul 2, 2025

Uh oh!

ghost left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

bert-e commented Jul 21, 2025

Integration data created

Uh oh!

bert-e commented Jul 21, 2025

Waiting for approval

Uh oh!

Uh oh!

Uh oh!

francoisferrand Jul 21, 2025

Choose a reason for hiding this comment

Uh oh!

welansari Jul 22, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

bert-e commented Jul 22, 2025

Waiting for approval

Uh oh!

bert-e commented Jul 22, 2025

History mismatch

Uh oh!

welansari commented Jul 22, 2025

Uh oh!

bert-e commented Jul 22, 2025

Reset complete

Uh oh!

bert-e commented Jul 22, 2025

Integration data created

Uh oh!

bert-e commented Jul 22, 2025

Waiting for approval

Uh oh!

welansari commented Jul 22, 2025

Uh oh!

bert-e commented Jul 22, 2025

In the queue

Uh oh!

bert-e commented Jul 22, 2025

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

welansari commented Jun 12, 2025 •

edited

Loading

codecov bot commented Jun 12, 2025 •

edited

Loading

welansari Jul 22, 2025 •

edited

Loading