Skip to content

feature: add some field for OpenModel Playground Service #464

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
InftyAI-Agent merged 1 commit into from
Jun 19, 2025
Merged

feature: add some field for OpenModel Playground Service #464

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 5 additions & 0 deletions api/core/v1alpha1/model_types.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -228,6 +228,11 @@ type ModelStatus struct {
//+kubebuilder:object:root=true
//+kubebuilder:subresource:status
//+kubebuilder:resource:shortName=om,scope=Cluster
//+kubebuilder:printcolumn:name="OWNEDBY",type=string,JSONPath=`.spec.ownedBy`,description="Owner of the model"
//+kubebuilder:printcolumn:name="AGE",type=date,JSONPath=`.metadata.creationTimestamp`,description="Time since creation"
//+kubebuilder:printcolumn:name="MODELHUB",type=string,JSONPath=`.spec.source.modelHub.name`,description="Model hub name"
//+kubebuilder:printcolumn:name="MODELID",type=string,JSONPath=`.spec.source.modelHub.modelID`,description="Model ID on the model hub"
Copy link
Copy Markdown
Member

@kerthcet kerthcet Jun 18, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

what if the modelhub and modelid is not present, for example, we use s3 as the model source.

Copy link
Copy Markdown
Member Author

@googs1025 googs1025 Jun 18, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more. 

root@VM-0-3-ubuntu:/home/ubuntu/llmaz# kubectl get om
NAME          OWNEDBY   AGE     MODELHUB      MODELID             URI
opt-125m      llmaz     4h48m   Huggingface   facebook/opt-125m
qwen2-0--5b   llmaz     4h48m                                     ollama://qwen2:0.5b
qwen2-7b      llmaz     17m                                       oss://llmaz.oss-ap-southeast-1-internal.aliyuncs.com/models/Qwen2-7B

Indeed, I currently changed it to: as follows,
But I am not sure if it is a good way.

Copy link
Copy Markdown
Member

@kerthcet kerthcet Jun 18, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reasonable to me, maybe we should define it as hf://, but can't fully compatible with us today.

Can we make it N/A if no value?

Copy link
Copy Markdown
Member Author

@googs1025 googs1025 Jun 19, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Without changing the original code logic (for example, changing the unset field to N/A), it seems that printcolumn feature provided by kubebuilder can't work. 🤔

//+kubebuilder:printcolumn:name="URI",type=string,JSONPath=`.spec.source.uri`,description="URI of the model when using a custom source (e.g., s3://, ollama://)"

// OpenModel is the Schema for the open models API
type OpenModel struct {
Expand Down
4 changes: 4 additions & 0 deletions api/inference/v1alpha1/playground_types.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -93,6 +93,10 @@ type PlaygroundStatus struct {
//+kubebuilder:subresource:status
//+kubebuilder:resource:shortName={pl}
//+kubebuilder:subresource:scale:specpath=.spec.replicas,statuspath=.status.replicas,selectorpath=.status.selector
//+kubebuilder:printcolumn:name="MODEL",type=string,JSONPath=`.spec.modelClaim.modelName`,description="Model used in the Playground"
//+kubebuilder:printcolumn:name="REPLICAS",type=integer,JSONPath=`.status.replicas`,description="Current number of replicas"
//+kubebuilder:printcolumn:name="STATUS",type=string,JSONPath=`.status.conditions[?(@.type=='Available')].reason`,description="Current status (Available/Progressing)"
//+kubebuilder:printcolumn:name="AGE",type=date,JSONPath=`.metadata.creationTimestamp`,description="Time since creation"

// Playground is the Schema for the playgrounds API
type Playground struct {
Expand Down
4 changes: 4 additions & 0 deletions api/inference/v1alpha1/service_types.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -73,6 +73,10 @@ type ServiceStatus struct {
//+kubebuilder:subresource:status
//+kubebuilder:resource:shortName={isvc}
//+kubebuilder:subresource:scale:specpath=.spec.replicas,statuspath=.status.replicas,selectorpath=.status.selector
//+kubebuilder:printcolumn:name="NAME",type=string,JSONPath=`.metadata.name`,description="Name of the Inference Service"
//+kubebuilder:printcolumn:name="REPLICAS",type=integer,JSONPath=`.status.replicas`,description="Current number of replicas"
//+kubebuilder:printcolumn:name="STATUS",type=string,JSONPath=`.status.conditions[?(@.type=='Available')].reason`,description="Current status (Available/Progressing)"
//+kubebuilder:printcolumn:name="AGE",type=date,JSONPath=`.metadata.creationTimestamp`,description="Time since creation"

// Service is the Schema for the services API
type Service struct {
Expand Down
1 change: 0 additions & 1 deletion chart/crds/backendruntime-crd.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -1601,4 +1601,3 @@ spec:
storage: true
subresources:
status: {}

38 changes: 36 additions & 2 deletions chart/crds/openmodel-crd.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -25,7 +25,28 @@ spec:
singular: openmodel
scope: Cluster
versions:
- name: v1alpha1
- additionalPrinterColumns:
- description: Owner of the model
jsonPath: .spec.ownedBy
name: OWNEDBY
type: string
- description: Time since creation
jsonPath: .metadata.creationTimestamp
name: AGE
type: date
- description: Model hub name
jsonPath: .spec.source.modelHub.name
name: MODELHUB
type: string
- description: Model ID on the model hub
jsonPath: .spec.source.modelHub.modelID
name: MODELID
type: string
- description: URI of the model when using a custom source (e.g., s3://, ollama://)
jsonPath: .spec.source.uri
name: URI
type: string
name: v1alpha1
schema:
openAPIV3Schema:
description: OpenModel is the Schema for the open models API
Expand All @@ -50,6 +71,13 @@ spec:
spec:
description: ModelSpec defines the desired state of Model
properties:
createdAt:
description: |-
CreatedAt represents the creation timestamp of the running models serving by the backends,
which will be exported as the field of "Created" in openai-compatible API "/models".
It follows the format of RFC 3339, for example "2024-05-21T10:00:00Z".
format: date-time
type: string
familyName:
description: |-
FamilyName represents the model type, like llama2, which will be auto injected
Expand Down Expand Up @@ -115,6 +143,13 @@ spec:
maxItems: 8
type: array
type: object
ownedBy:
default: llmaz
description: |-
OwnedBy represents the owner of the running models serving by the backends,
which will be exported as the field of "OwnedBy" in openai-compatible API "/models".
Default to "llmaz" if not set.
type: string
source:
description: |-
Source represents the source of the model, there're several ways to load
Expand Down Expand Up @@ -240,4 +275,3 @@ spec:
storage: true
subresources:
status: {}

20 changes: 18 additions & 2 deletions chart/crds/playground-crd.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -15,7 +15,24 @@ spec:
singular: playground
scope: Namespaced
versions:
- name: v1alpha1
- additionalPrinterColumns:
- description: Model used in the Playground
jsonPath: .spec.modelClaim.modelName
name: MODEL
type: string
- description: Current number of replicas
jsonPath: .status.replicas
name: REPLICAS
type: integer
- description: Current status (Available/Progressing)
jsonPath: .status.conditions[?(@.type=='Available')].reason
name: STATUS
type: string
- description: Time since creation
jsonPath: .metadata.creationTimestamp
name: AGE
type: date
name: v1alpha1
schema:
openAPIV3Schema:
description: Playground is the Schema for the playgrounds API
Expand Down Expand Up @@ -1017,4 +1034,3 @@ spec:
specReplicasPath: .spec.replicas
statusReplicasPath: .status.replicas
status: {}

29 changes: 27 additions & 2 deletions chart/crds/service-crd.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -15,7 +15,24 @@ spec:
singular: service
scope: Namespaced
versions:
- name: v1alpha1
- additionalPrinterColumns:
- description: Name of the Inference Service
jsonPath: .metadata.name
name: NAME
type: string
- description: Current number of replicas
jsonPath: .status.replicas
name: REPLICAS
type: integer
- description: Current status (Available/Progressing)
jsonPath: .status.conditions[?(@.type=='Available')].reason
name: STATUS
type: string
- description: Time since creation
jsonPath: .metadata.creationTimestamp
name: AGE
type: date
name: v1alpha1
schema:
openAPIV3Schema:
description: Service is the Schema for the services API
Expand Down Expand Up @@ -8300,6 +8317,15 @@ spec:
SubGroupPolicy describes the policy that will be applied when creating subgroups
in each replica.
properties:
subGroupPolicyType:
default: LeaderWorker
description: |-
Defines what type of Subgroups to create. Defaults to
LeaderWorker
enum:
- LeaderWorker
- LeaderExcluded
type: string
subGroupSize:
description: |-
The number of pods per subgroup. This value is immutable,
Expand Down Expand Up @@ -16520,4 +16546,3 @@ spec:
specReplicasPath: .spec.replicas
statusReplicasPath: .status.replicas
status: {}

2 changes: 1 addition & 1 deletion chart/templates/deployment.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -67,4 +67,4 @@ spec:
- name: cert
secret:
defaultMode: 420
secretName: {{ include "chart.fullname" . }}-webhook-server-cert
secretName: {{ include "chart.fullname" . }}-webhook-server-cert
2 changes: 1 addition & 1 deletion chart/templates/global-config.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -5,4 +5,4 @@ metadata:
labels:
{{- include "chart.labels" . | nindent 4 }}
data:
config.data: {{ .Values.globalConfig.configData | toYaml | indent 1 }}
config.data: {{ .Values.globalConfig.configData | toYaml | indent 1 }}
2 changes: 1 addition & 1 deletion chart/templates/leader-election-rbac.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -56,4 +56,4 @@ roleRef:
subjects:
- kind: ServiceAccount
name: '{{ include "chart.fullname" . }}-controller-manager'
namespace: '{{ .Release.Namespace }}'
namespace: '{{ .Release.Namespace }}'
2 changes: 1 addition & 1 deletion chart/templates/manager-rbac.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -162,4 +162,4 @@ roleRef:
subjects:
- kind: ServiceAccount
name: '{{ include "chart.fullname" . }}-controller-manager'
namespace: '{{ .Release.Namespace }}'
namespace: '{{ .Release.Namespace }}'
2 changes: 1 addition & 1 deletion chart/templates/metrics-reader-rbac.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -25,4 +25,4 @@ roleRef:
subjects:
- kind: ServiceAccount
name: '{{ include "chart.fullname" . }}-controller-manager'
namespace: '{{ .Release.Namespace }}'
namespace: '{{ .Release.Namespace }}'
2 changes: 1 addition & 1 deletion chart/templates/metrics-service.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -12,4 +12,4 @@ spec:
control-plane: controller-manager
{{- include "chart.selectorLabels" . | nindent 4 }}
ports:
{{- .Values.metricsService.ports | toYaml | nindent 2 }}
{{- .Values.metricsService.ports | toYaml | nindent 2 }}
2 changes: 1 addition & 1 deletion chart/templates/mutating-webhook-configuration.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -86,4 +86,4 @@ webhooks:
- UPDATE
resources:
- services
sideEffects: None
sideEffects: None
2 changes: 1 addition & 1 deletion chart/templates/proxy-rbac.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -35,4 +35,4 @@ roleRef:
subjects:
- kind: ServiceAccount
name: '{{ include "chart.fullname" . }}-controller-manager'
namespace: '{{ .Release.Namespace }}'
namespace: '{{ .Release.Namespace }}'
2 changes: 1 addition & 1 deletion chart/templates/serviceaccount.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -8,4 +8,4 @@ metadata:
app.kubernetes.io/part-of: llmaz
{{- include "chart.labels" . | nindent 4 }}
annotations:
{{- toYaml .Values.controllerManager.serviceAccount.annotations | nindent 4 }}
{{- toYaml .Values.controllerManager.serviceAccount.annotations | nindent 4 }}
2 changes: 1 addition & 1 deletion chart/templates/validating-webhook-configuration.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -86,4 +86,4 @@ webhooks:
- UPDATE
resources:
- services
sideEffects: None
sideEffects: None
2 changes: 1 addition & 1 deletion chart/templates/webhook-server-cert.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -3,4 +3,4 @@ kind: Secret
metadata:
name: {{ include "chart.fullname" . }}-webhook-server-cert
labels:
{{- include "chart.labels" . | nindent 4 }}
{{- include "chart.labels" . | nindent 4 }}
2 changes: 1 addition & 1 deletion chart/templates/webhook-service.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -13,4 +13,4 @@ spec:
control-plane: controller-manager
{{- include "chart.selectorLabels" . | nindent 4 }}
ports:
{{- .Values.webhookService.ports | toYaml | nindent 2 }}
{{- .Values.webhookService.ports | toYaml | nindent 2 }}
19 changes: 18 additions & 1 deletion config/crd/bases/inference.llmaz.io_playgrounds.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -16,7 +16,24 @@ spec:
singular: playground
scope: Namespaced
versions:
- name: v1alpha1
- additionalPrinterColumns:
- description: Model used in the Playground
jsonPath: .spec.modelClaim.modelName
name: MODEL
type: string
- description: Current number of replicas
jsonPath: .status.replicas
name: REPLICAS
type: integer
- description: Current status (Available/Progressing)
jsonPath: .status.conditions[?(@.type=='Available')].reason
name: STATUS
type: string
- description: Time since creation
jsonPath: .metadata.creationTimestamp
name: AGE
type: date
name: v1alpha1
schema:
openAPIV3Schema:
description: Playground is the Schema for the playgrounds API
Expand Down
19 changes: 18 additions & 1 deletion config/crd/bases/inference.llmaz.io_services.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -16,7 +16,24 @@ spec:
singular: service
scope: Namespaced
versions:
- name: v1alpha1
- additionalPrinterColumns:
- description: Name of the Inference Service
jsonPath: .metadata.name
name: NAME
type: string
- description: Current number of replicas
jsonPath: .status.replicas
name: REPLICAS
type: integer
- description: Current status (Available/Progressing)
jsonPath: .status.conditions[?(@.type=='Available')].reason
name: STATUS
type: string
- description: Time since creation
jsonPath: .metadata.creationTimestamp
name: AGE
type: date
name: v1alpha1
schema:
openAPIV3Schema:
description: Service is the Schema for the services API
Expand Down
23 changes: 22 additions & 1 deletion config/crd/bases/llmaz.io_openmodels.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -16,7 +16,28 @@ spec:
singular: openmodel
scope: Cluster
versions:
- name: v1alpha1
- additionalPrinterColumns:
- description: Owner of the model
jsonPath: .spec.ownedBy
name: OWNEDBY
type: string
- description: Time since creation
jsonPath: .metadata.creationTimestamp
name: AGE
type: date
- description: Model hub name
jsonPath: .spec.source.modelHub.name
name: MODELHUB
type: string
- description: Model ID on the model hub
jsonPath: .spec.source.modelHub.modelID
name: MODELID
type: string
- description: URI of the model when using a custom source (e.g., s3://, ollama://)
jsonPath: .spec.source.uri
name: URI
type: string
name: v1alpha1
schema:
openAPIV3Schema:
description: OpenModel is the Schema for the open models API
Expand Down
Loading