Skip to content

feat(deploy): add Kustomize deployment option#245

Open
mkoushni wants to merge 3 commits into
llm-d:mainfrom
mkoushni:feat/kustomize-deployment
Open

feat(deploy): add Kustomize deployment option#245
mkoushni wants to merge 3 commits into
llm-d:mainfrom
mkoushni:feat/kustomize-deployment

Conversation

@mkoushni

@mkoushni mkoushni commented Jul 16, 2026

Copy link
Copy Markdown
Contributor

[Feature] Enable deployment using Kustomize

Adds a Kustomize deployment path for the Inference Payload Processor (IPP),
aligning with the llm-d community deployment standard and enabling IPP to be
consumed as a step in llm-d-benchmark
and GitOps workflows that prefer Kustomize over Helm.

The existing Helm chart (config/charts/payload-processor/) is untouched;
Kustomize is an additive alternative.


Fixes #125


Changes

New: config/kustomize/

config/kustomize/
├── README.md                              ← full usage guide
├── base/                                  ← provider-agnostic core resources
│   ├── kustomization.yaml
│   ├── deployment.yaml                    ← Deployment (1 replica, port 9004/9005)
│   ├── service.yaml                       ← ClusterIP Service (HTTP2, port 9004)
│   ├── serviceaccount.yaml
│   ├── rbac.yaml                          ← Role + RoleBinding (single-namespace)
│   └── configmap.yaml                     ← default PayloadProcessorConfig
└── overlays/
    ├── default/kustomization.yaml         ← no provider (base only)
    ├── istio/                             ← + EnvoyFilter + DestinationRule
    │   ├── kustomization.yaml
    │   ├── envoyfilter.yaml
    │   └── destinationrule.yaml
    └── gke/                               ← + GCPRoutingExtension + HealthCheckPolicy
        ├── kustomization.yaml
        ├── gcproutingextension.yaml
        └── healthcheckpolicy.yaml

Each overlay exposes a single namespace: field as the sole namespace control
point — the base layer intentionally omits metadata.namespace so that the
overlay drives it.

Modified: Makefile

New ##@ Deployment section:

Target Description
make kustomize Download kustomize binary to bin/ if absent
make kustomize-build [KUSTOMIZE_OVERLAY=default|istio|gke] Render manifests to stdout
make kustomize-deploy [KUSTOMIZE_OVERLAY=default|istio|gke] Apply to cluster
make kustomize-undeploy [KUSTOMIZE_OVERLAY=default|istio|gke] Remove from cluster

Modified: README.md

Added a Deployment section with side-by-side Helm and Kustomize quick-start
commands.

Fixed: pre-existing goconst lint issues

Extracted repeated string literals into named constants to clear golangci-lint
(4 issues, pre-existing):

File Constants added
pkg/config/loader/defaults.go configAPIVersion, configKind
test/integration/util.go headerContentLength, bodyFieldPrompt

Usage

# No provider (Deployment + Service only)
kubectl kustomize config/kustomize/overlays/default | kubectl apply -f -

# Istio (adds EnvoyFilter + DestinationRule)
make kustomize-deploy KUSTOMIZE_OVERLAY=istio

# GKE (adds GCPRoutingExtension + HealthCheckPolicy)
make kustomize-deploy KUSTOMIZE_OVERLAY=gke

# Custom namespace
cd config/kustomize/overlays/default
kustomize edit set namespace my-namespace

See config/kustomize/README.md for customization
options (namespace, image tag, custom IPP config, multi-namespace RBAC).


Testing

Kustomize build validation

All three overlays render without errors and produce the expected resource kinds:

Overlay Resources
default ConfigMap, Deployment, Role, RoleBinding, Service, ServiceAccount
istio + DestinationRule, EnvoyFilter
gke + GCPRoutingExtension, HealthCheckPolicy

Validated with:

kubectl kustomize config/kustomize/overlays/default
kubectl kustomize config/kustomize/overlays/istio
kubectl kustomize config/kustomize/overlays/gke

Go test suite

All 24 test packages pass with race detection enabled (go test -race -count=1 ./...):

Package Result Coverage
pkg/common ✅ PASS 81.8%
pkg/common/envoy ✅ PASS 87.5%
pkg/common/envoy/test ✅ PASS 100.0%
pkg/common/error ✅ PASS 100.0%
pkg/common/observability/logging ✅ PASS 69.4%
pkg/config/loader ✅ PASS 50.0%
pkg/datastore/inmemory ✅ PASS 100.0%
pkg/framework/interface/datalayer ✅ PASS 97.4%
pkg/framework/interface/requesthandling ✅ PASS 52.4%
pkg/framework/plugins/datalayer/notificationsource ✅ PASS 45.2%
pkg/framework/plugins/datalayer/requestmetadata ✅ PASS 92.3%
pkg/framework/plugins/modelselector/picker/maxscore ✅ PASS 66.7%
pkg/framework/plugins/modelselector/picker/random ✅ PASS 88.9%
pkg/framework/plugins/modelselector/picker/weightedrandom ✅ PASS 66.7%
pkg/framework/plugins/modelselector/scorer/costaware ✅ PASS 95.8%
pkg/framework/plugins/requesthandling/basemodelextractor ✅ PASS 81.4%
pkg/framework/plugins/requesthandling/bodyfieldtoheader ✅ PASS 100.0%
pkg/framework/plugins/requesthandling/profilepicker/single ✅ PASS 100.0%
pkg/handlers ✅ PASS 74.4%
pkg/metrics ✅ PASS 50.0%
pkg/modelselector ✅ PASS 82.4%
pkg/server ✅ PASS 61.1%
test/integration ✅ PASS 97.1%

Total coverage: 51.5% (statements)

Linting

golangci-lint run  →  0 issues.
go vet ./...       →  clean

Release note (write NONE if no user-facing change):

Add Kustomize deployment support with base manifests and overlays for default,
Istio, and GKE providers. New Makefile targets: kustomize-build, kustomize-deploy,
kustomize-undeploy (KUSTOMIZE_OVERLAY=default|istio|gke).

@github-actions github-actions Bot added the size/XL Denotes a PR that changes 500-999 lines, ignoring generated files. label Jul 16, 2026
mkoushni added 3 commits July 16, 2026 13:17
Signed-off-by: mkoushni <mkoushni@redhat.com>
v9 moves the action's runtime from Node 20 to Node 24, per the
upstream release notes.

Signed-off-by: mkoushni <mkoushni@redhat.com>
golangci-lint flagged plugin_test.go for improper gofmt formatting
and an unused waitForTokenPrices helper with no call sites.

Signed-off-by: mkoushni <mkoushni@redhat.com>
@mkoushni
mkoushni force-pushed the feat/kustomize-deployment branch from f54b42e to af5b45a Compare July 16, 2026 10:21
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

size/XL Denotes a PR that changes 500-999 lines, ignoring generated files.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Feature]: Enable deployment using Kustomize

1 participant