Fix possible race condition in swift_migration check - #1362
Conversation
|
Build failed (check pipeline). Post https://softwarefactory-project.io/zuul/t/rdoproject.org/buildset/fac7aa01feea42af93b72573e0fbe9bd ✔️ noop SUCCESS in 0s |
|
recheck |
Updated rings were only pushed to the first controller node, resulting in a data movement loop: controller 1 pushed data to OpenShift and removed it locally, while controller 2 and 3 were detecting missing data on controller 1 (because they were still using old rings) and replicating back to controller 1. Update both the docs and the test role to distribute rings to all three controller nodes and restart Swift services on each. Also add the -ignore_readdir_race option to the find command that checks for remaining .data/.db files on the standalone node. The replicator may remove empty directories while find is iterating, causing spurious errors. JIRA: OSPCIX-1318 Signed-off-by: Christian Schwede <cschwede@redhat.com>
|
Build failed (check pipeline). Post https://softwarefactory-project.io/zuul/t/rdoproject.org/buildset/cd52ec4bdd7842f0aed9b2ac905fbc44 ✔️ noop SUCCESS in 0s |
|
recheck |
|
Multinode job test-project started here: |
Output looks much better, the drain check succeeded after 50 seconds. |
|
/lgtm |
|
[APPROVALNOTIFIER] This PR is APPROVED This pull-request has been approved by: jistr The full list of commands accepted by this bot can be found here. The pull request process is described here DetailsNeeds approval from an approver in each of these files:
Approvers can indicate their approval by writing |
d3e6e17
into
openstack-k8s-operators:main
|
/cherry-pick 18.0-fr5 |
|
@ciecierski: new pull request created: #1402 DetailsIn response to this:
Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. |
4 participants
Updated rings were only pushed to the first controller node, resulting in a data movement loop: controller 1 pushed data to OpenShift and removed it locally, while controller 2 and 3 were detecting missing data
on controller 1 (because they were still using old rings) and replicating back to controller 1.
Update both the docs and the test role to distribute rings to all three controller nodes and restart Swift services on each.
Also add the -ignore_readdir_race option to the find command that checks for remaining .data/.db files on the standalone node. The replicator may remove empty directories while find is iterating, causing spurious errors.
JIRA: OSPCIX-1318