feat: Enhance compactor resource configuration

[kayahk]

Until now, there is only a single resource definition for the compactor pod assigning the values to all containers in the deployment. The limits for the check-nats container are hard-coded in the template:

          image: {{ .Values.image.busybox.repository }}:{{ .Values.image.busybox.tag }}
          resources:
            limits:
              cpu: 50m
              memory: 50Mi

This prevents the allocation of resources beyond these limits for the main container of the compactor service, too. As a result of the low resource allocation, the compactor service gets OOMKilled frequently during regular operations.

This update adds separate resource requests and limits for the init containers and the main container in the compactor deployment to ensure better resource management and performance for the compactor component.

[kayahk]

@mmosarafO2 can you please also review this PR ?

[hengfeiyang]

i don't think check-nats pod itself will OOM, please help check the OOM pod is initContainer or compactor container.

[kayahk]

I checked this again. The intent of the PR is not specifically to fix an observed OOMKilled check-nats container. The issue is that check-nats resources are currently hard-coded in the template, so users can configure resources.compactor for the main compactor container but cannot fully control resources for all containers in the compactor pod.

Since init container resources also contribute to the pod’s effective resource profile, this PR moves the compactor init container resources into values so they can be configured by chart users.

[hengfeiyang]

Make sense, but i don't think it is necessary. Because the initContainer do nothing. only used to check if nats startted.