Skip to content

K8SPSMDB-1031: Add the possibility to configure an external node as arbiter #2307

Open
tariktunahanakan wants to merge 3 commits intopercona:mainfrom
tariktunahanakan:fix/external-arbiter-tags
Open

K8SPSMDB-1031: Add the possibility to configure an external node as arbiter #2307
tariktunahanakan wants to merge 3 commits intopercona:mainfrom
tariktunahanakan:fix/external-arbiter-tags

Conversation

@tariktunahanakan
Copy link
Copy Markdown

MongoDB does not allow setting tags on arbiter members (replSetReconfig returns InvalidReplicaSetConfig). When an external node is also an arbiter, FixMemberConfigs was unconditionally attempting to apply the hardcoded
{"external":"true"} tag, causing the operator to enter a permanent error reconciliation loop.

Add !member.ArbiterOnly guard so tag reconciliation is skipped for arbiters, matching MongoDB's own constraint.

Fixes: operator sets cluster status to error with message "Cannot set tags on arbiters" when externalNodes contains an arbiter member.

CHANGE DESCRIPTION

Problem:
Short explanation of the problem.

Cause:
Short explanation of the root cause of the issue if applicable.

Solution:
Short explanation of the solution we are providing with this PR.

CHECKLIST

Jira

  • Is the Jira ticket created and referenced properly?
  • Does the Jira ticket have the proper statuses for documentation (Needs Doc) and QA (Needs QA)?
  • Does the Jira ticket link to the proper milestone (Fix Version field)?

Tests

  • Is an E2E test/test case added for the new feature/change?
  • Are unit tests added where appropriate?
  • Are OpenShift compare files changed for E2E tests (compare/*-oc.yml)?

Config/Logging/Testability

  • Are all needed new/changed options added to default YAML files?
  • Are all needed new/changed options added to the Helm Chart?
  • Did we add proper logging messages for operator actions?
  • Did we ensure compatibility with the previous version or cluster upgrade process?
  • Does the change support oldest and newest supported MongoDB version?
  • Does the change support oldest and newest supported Kubernetes version?

MongoDB does not allow setting tags on arbiter members
(replSetReconfig returns InvalidReplicaSetConfig). When an
external node is also an arbiter, FixMemberConfigs was
unconditionally attempting to apply the hardcoded
{"external":"true"} tag, causing the operator to enter a
permanent error reconciliation loop.

Add !member.ArbiterOnly guard so tag reconciliation is skipped
for arbiters, matching MongoDB's own constraint.

Fixes: operator sets cluster status to error with message
"Cannot set tags on arbiters" when externalNodes contains
an arbiter member.

Made-with: Cursor
@CLAassistant
Copy link
Copy Markdown

CLAassistant commented Apr 9, 2026

CLA assistant check
Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you all sign our Contributor License Agreement before we can accept your contribution.
1 out of 2 committers have signed the CLA.

✅ mayankshah1607
❌ tariktunahanakan
You have signed the CLA already but the status is still pending? Let us recheck it.

Comment thread pkg/psmdb/mongo/mongo.go
member := []ConfigMember(*m)[i]
c, ok := cm[member.Host]
if ok && c.Tags != nil && !reflect.DeepEqual(c.Tags, member.Tags) {
if ok && !member.ArbiterOnly && c.Tags != nil && !reflect.DeepEqual(c.Tags, member.Tags) {
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this is not wrong actually but I'd like to skip adding tags in

func (r *ReconcilePerconaServerMongoDB) getConfigMemberForExternalNode(id int, extNode api.ExternalNode) mongo.ConfigMember {
as well which requires https://perconadev.atlassian.net/browse/K8SPSMDB-1031

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the feedback @egegunes!
Updated the PR to address the root cause in getConfigMemberForExternalNode:

  • Added arbiterOnly bool field to ExternalNode API type (relates to K8SPSMDB-1031)
  • In getConfigMemberForExternalNode, the {"external":"true"} tag is now only set when !extNode.ArbiterOnly, so arbiters never receive tags
  • ArbiterOnly is also propagated to ConfigMember for downstream consistency
  • Added nil-guard in the custom tags loop for the case where an arbiter has user-defined tags in the CR
    The fix in mongo.go (first commit) can be kept as defense-in-depth or dropped — up to you.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

thank you @tariktunahanakan.

i think the fix in mongo.go can be kept, yes.

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

rica ederim

…ters

Add `arbiterOnly` field to ExternalNode API type so users can explicitly
declare an external node as an arbiter in the CR spec.

In getConfigMemberForExternalNode, skip setting the {"external":"true"} tag
when the node is an arbiter — MongoDB does not allow any tags on arbiter
members and would reject the replSetReconfig with:
  (InvalidReplicaSetConfig) Cannot set tags on arbiters.

Also propagate ArbiterOnly to the ConfigMember so downstream reconciliation
has full context about the member role.

Relates-to: K8SPSMDB-1031
Made-with: Cursor
@pull-request-size pull-request-size Bot added size/S 10-29 lines and removed size/XS 0-9 lines labels Apr 10, 2026
Comment on lines +340 to 342
if !extNode.ArbiterOnly {
member.Tags = mongo.ReplsetTags{"external": "true"}
}
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

it'd be great to add unit tests for this function

Comment thread pkg/psmdb/mongo/mongo.go
member := []ConfigMember(*m)[i]
c, ok := cm[member.Host]
if ok && c.Tags != nil && !reflect.DeepEqual(c.Tags, member.Tags) {
if ok && !member.ArbiterOnly && c.Tags != nil && !reflect.DeepEqual(c.Tags, member.Tags) {
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

thank you @tariktunahanakan.

i think the fix in mongo.go can be kept, yes.

@egegunes egegunes changed the title fix(psmdb): skip tag update for arbiter-only external nodes K8SPSMDB-1031: Add the possibility to configure an external node as arbiter Apr 10, 2026
@egegunes egegunes added this to the v1.23.0 milestone Apr 10, 2026
@JNKPercona
Copy link
Copy Markdown
Collaborator

Test Name Result Time
arbiter passed 00:11:18
balancer passed 00:18:26
cross-site-sharded passed 00:18:50
custom-replset-name passed 00:10:24
custom-tls passed 00:14:30
custom-users-roles passed 00:10:23
custom-users-roles-sharded passed 00:11:35
data-at-rest-encryption passed 00:12:36
data-sharded passed 00:23:18
demand-backup passed 00:17:19
demand-backup-eks-credentials-irsa passed 00:00:08
demand-backup-fs passed 00:24:19
demand-backup-if-unhealthy failure 00:52:50
demand-backup-incremental-aws passed 00:12:26
demand-backup-incremental-azure passed 00:12:03
demand-backup-incremental-gcp-native passed 00:12:00
demand-backup-incremental-gcp-s3 passed 00:11:14
demand-backup-incremental-minio passed 00:26:26
demand-backup-incremental-sharded-aws passed 00:18:08
demand-backup-incremental-sharded-azure passed 00:18:04
demand-backup-incremental-sharded-gcp-native passed 00:17:52
demand-backup-incremental-sharded-gcp-s3 passed 00:18:12
demand-backup-incremental-sharded-minio passed 00:27:32
demand-backup-logical-minio-native-tls passed 00:08:38
demand-backup-physical-parallel passed 00:08:19
demand-backup-physical-aws passed 00:12:31
demand-backup-physical-azure passed 00:11:58
demand-backup-physical-gcp-s3 passed 00:12:20
demand-backup-physical-gcp-native passed 00:11:56
demand-backup-physical-minio passed 00:20:48
demand-backup-physical-minio-native passed 00:25:56
demand-backup-physical-minio-native-tls passed 00:20:14
demand-backup-physical-sharded-parallel passed 00:11:44
demand-backup-physical-sharded-aws passed 00:18:44
demand-backup-physical-sharded-azure passed 00:18:12
demand-backup-physical-sharded-gcp-native passed 00:17:52
demand-backup-physical-sharded-minio passed 00:17:24
demand-backup-physical-sharded-minio-native passed 00:18:20
demand-backup-sharded passed 00:26:10
demand-backup-snapshot passed 00:37:25
demand-backup-snapshot-vault passed 00:18:24
disabled-auth passed 00:16:34
expose-sharded passed 00:34:02
finalizer passed 00:10:29
ignore-labels-annotations passed 00:07:18
init-deploy passed 00:13:19
ldap passed 00:08:51
ldap-tls passed 00:13:08
limits passed 00:06:21
liveness passed 00:09:16
mongod-major-upgrade passed 00:12:19
mongod-major-upgrade-sharded passed 00:21:22
monitoring-2-0 passed 00:25:26
monitoring-pmm3 passed 00:29:03
multi-cluster-service passed 00:12:09
multi-storage passed 00:19:39
non-voting-and-hidden passed 00:17:00
one-pod passed 00:07:47
operator-self-healing-chaos passed 00:13:40
pitr passed 00:32:40
pitr-physical passed 01:06:17
pitr-sharded passed 00:21:08
pitr-to-new-cluster passed 00:25:23
pitr-physical-backup-source passed 00:55:42
preinit-updates passed 00:05:04
pvc-auto-resize passed 00:15:28
pvc-resize passed 00:17:28
recover-no-primary passed 00:29:38
replset-overrides passed 00:17:59
replset-remapping passed 00:17:11
replset-remapping-sharded passed 00:17:48
rs-shard-migration passed 00:14:38
scaling passed 00:12:57
scheduled-backup passed 00:17:21
security-context passed 00:07:51
self-healing-chaos passed 00:15:17
service-per-pod passed 00:18:43
serviceless-external-nodes passed 00:07:26
smart-update passed 00:08:08
split-horizon passed 00:14:15
stable-resource-version passed 00:05:00
storage passed 00:07:43
tls-issue-cert-manager passed 00:30:39
unsafe-psa passed 00:07:41
upgrade passed 00:09:59
upgrade-consistency passed 00:06:38
upgrade-consistency-sharded-tls passed 00:54:49
upgrade-sharded passed 00:20:02
upgrade-partial-backup passed 00:16:27
users passed 00:18:01
users-vault passed 00:13:30
version-service passed 00:27:16
Summary Value
Tests Run 92/92
Job Duration 02:59:28
Total Test Time 27:31:17

commit: 8c85daa
image: perconalab/percona-server-mongodb-operator:PR-2307-8c85daad

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

size/S 10-29 lines

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants