Skip to content

[Core] Add RBGStrategy workload strategy following OEP-0006#614

Open
diw-zw wants to merge 1 commit into
ome-projects:mainfrom
diw-zw:dev
Open

[Core] Add RBGStrategy workload strategy following OEP-0006#614
diw-zw wants to merge 1 commit into
ome-projects:mainfrom
diw-zw:dev

Conversation

@diw-zw
Copy link
Copy Markdown

@diw-zw diw-zw commented May 18, 2026

What this PR does

  • Introduce a new RoleBasedGroup deployment mode and the corresponding RBGStrategy workload strategy that packs all InferenceService components (engine, decoder, router) into a single RoleBasedGroup custom resource
  • Add RoleConfig / RoleConfigExtractor abstraction so each component (Engine, Decoder, Router) can produce its resolved pod spec and metadata without side effects, reusable by All-in-One workload strategies
  • Implement RBGReconciler for create/update of the RBG CR with HPA-safe replica preservation, per-role HPA targeting RoleBasedGroupScalingAdapter, and component status propagation from RBG role statuses

Why we need it

Refer to OEP: #505

Changes

New files:

  • pkg/controller/v1beta1/inferenceservice/components/role_config.go — RoleConfig struct, RoleConfigExtractor and ComponentStatusUpdater interfaces
  • pkg/controller/v1beta1/inferenceservice/reconcilers/rbg/rbg_reconciler.go — RBG CR reconciler (create/update with replica preservation and label merging)
  • pkg/controller/v1beta1/inferenceservice/workload/rbg_strategy.go — Full strategy: role config extraction, RBAC, RBG reconciliation, HPA, and status propagation
  • Unit tests for both the reconciler and strategy

Modified files:

  • constants.go — Add RoleBasedGroup deployment mode and RoleBasedGroupKind
  • decoder.go, engine.go, router.go — Implement ExtractRoleConfig() and UpdateStatus()
  • controller.go — Register RBG strategy (conditional on CRD availability), watch RoleBasedGroup resources
  • cleanup.go — Include RBG in orphan resource cleanup
  • hpa_reconciler.go — Add NewHPAReconcilerWithScaleTarget for custom scale target references
  • status_util.go — Export SetComponentCondition for strategy-level status propagation
  • main.go — Register rbgv1alpha2 scheme
  • RBAC role YAMLs — Grant permissions for workloads.x-k8s.io/rolebasedgroups
  • go.mod / go.sum — Add sigs.k8s.io/rbgs/api dependency (via github.com/sgl-project/rbg-api v0.7.0-alpha.3)

How to test

Checklist

  • Tests added/updated (if applicable)
  • Docs updated (if applicable)
  • make test passes locally

@github-actions github-actions Bot added helm Helm chart changes inferenceservice InferenceService controller changes rbac RBAC configuration changes autoscaling Autoscaling (HPA/KEDA) changes controller Controller changes tests Test changes config Configuration changes dependencies Dependency updates labels May 18, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

autoscaling Autoscaling (HPA/KEDA) changes config Configuration changes controller Controller changes dependencies Dependency updates helm Helm chart changes inferenceservice InferenceService controller changes rbac RBAC configuration changes tests Test changes

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant