KONFLUX-14972: kubearchive-api-server OOMs on stone-prd-rh01#13104
Open
olegbet wants to merge 1 commit into
Open
KONFLUX-14972: kubearchive-api-server OOMs on stone-prd-rh01#13104olegbet wants to merge 1 commit into
olegbet wants to merge 1 commit into
Conversation
|
[APPROVALNOTIFIER] This PR is APPROVED This pull-request has been approved by: olegbet The full list of commands accepted by this bot can be found here. The pull request process is described here DetailsNeeds approval from an approver in each of these files:
Approvers can indicate their approval by writing |
Contributor
Kustomize Render DiffComparing
Total: 1 components, +3 -5 lines 📋 Full diff available in the workflow summary and as a downloadable artifact. |
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## main #13104 +/- ##
=======================================
Coverage 57.11% 57.11%
=======================================
Files 23 23
Lines 1455 1455
=======================================
Hits 831 831
Misses 548 548
Partials 76 76
Flags with carried forward coverage won't be shown. Click here to find out more. 🚀 New features to boost your workflow:
|
Signed-off-by: obetsun <obetsun@redhat.com>
olegbet
force-pushed
the
KONFLUX-14972_api-server_OOM_stone-prd-rh01
branch
from
July 20, 2026 07:25
b0a2258 to
d07ab6b
Compare
Code Review by Qodo
Context used✅ Compliance rules (platform):
3 rules 1. Invalid GOMEMLIMIT unit
|
![qodo-for-redhat-appstudio[bot]](https://avatars.githubusercontent.com/u/89927105?s=60&v=4){kind=small}
Comment on lines
1114
to
1116
| - name: GOMEMLIMIT | ||
| valueFrom: | ||
| resourceFieldRef: | ||
| resource: limits.memory | ||
| value: "440Mi" | ||
| - name: GOMAXPROCS |
There was a problem hiding this comment.
1. Invalid gomemlimit unit 🐞 Bug ≡ Correctness
kubearchive-api-server now sets GOMEMLIMIT to "440Mi", which is inconsistent with the repo’s existing GiB-formatted GOMEMLIMIT values and is likely to be ignored by the Go runtime. If ignored, the heap cap won’t be applied and the API server can continue to OOM despite the resource increases.
Agent Prompt
### Issue description
`GOMEMLIMIT` is set to a Kubernetes quantity (`"440Mi"`) in the api-server Deployment. This format is inconsistent with how `GOMEMLIMIT` is expressed elsewhere in this repo (e.g., `"7GiB"`, `10GiB`) and may not be parsed/used by the Go runtime, meaning the intended heap cap would not be applied.
### Issue Context
This PR is intended to mitigate OOMs by increasing memory and setting `GOMEMLIMIT`. If `GOMEMLIMIT` is ignored due to unit formatting, the change won’t achieve its purpose.
### Fix Focus Areas
- components/kubearchive/production/stone-prd-rh01/kubearchive.yaml[1114-1116]
### What to change
- Update `GOMEMLIMIT` to a Go-runtime-compatible value format (e.g., `"440MiB"` or a bytes integer), and ensure it remains below the container memory limit to leave headroom for non-heap memory.
- (Optional for consistency) align any other hard-coded `Mi` `GOMEMLIMIT` values introduced in this file to the same unit convention used elsewhere in the repo (e.g., `GiB`/`MiB`).
ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools
Comment on lines
1114
to
1116
| - name: GOMEMLIMIT | ||
| valueFrom: | ||
| resourceFieldRef: | ||
| resource: limits.memory | ||
| value: "440Mi" | ||
| - name: GOMAXPROCS |
There was a problem hiding this comment.
2. Gomemlimit decoupled resources 🐞 Bug ⚙ Maintainability
Switching kubearchive-api-server GOMEMLIMIT from resourceFieldRef to a hard-coded value makes it easy for future memory request/limit adjustments to drift out of sync with the Go heap cap. This increases ongoing operational risk because other kubearchive environments/templates still derive GOMEMLIMIT from container resources.
Agent Prompt
### Issue description
`kubearchive-api-server` changes `GOMEMLIMIT` from a resource-derived value (`resourceFieldRef`) to a hard-coded string. This creates a configuration drift hazard: future changes to memory requests/limits may not be accompanied by updates to `GOMEMLIMIT`.
### Issue Context
Most kubearchive manifests and templates still use `valueFrom.resourceFieldRef` for `GOMEMLIMIT`, and other components in this repo sometimes derive `GOMEMLIMIT` from `requests.memory`.
### Fix Focus Areas
- components/kubearchive/production/stone-prd-rh01/kubearchive.yaml[1114-1116]
### What to change
Choose one:
1) Restore a resource-derived `GOMEMLIMIT` (e.g., `valueFrom.resourceFieldRef.resource: requests.memory`) and set `requests.memory` to the desired heap cap value, keeping `limits.memory` higher for headroom.
2) If hard-coding is required to maintain headroom below the limit, add an explicit comment near `GOMEMLIMIT` documenting the intended relationship to `limits.memory` (e.g., `GOMEMLIMIT = limits.memory - safety_margin`) and ensure any future resource changes update both together.
ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
KONFLUX-14972(https://redhat.atlassian.net/browse/KONFLUX-14972)
Risk assessment
Validation
Staging validated with the configuration: #13102
Development validated with: #13105