Skip to content

Allow configuration through argument list (in addition to string)#95

Merged
xenoscopic merged 1 commit into
mainfrom
config-list
Jun 27, 2025
Merged

Allow configuration through argument list (in addition to string)#95
xenoscopic merged 1 commit into
mainfrom
config-list

Conversation

@xenoscopic
Copy link
Copy Markdown
Contributor

We'll allow runtime flag configuration via either an explicit, pre-parsed list (via runtime-flags) or via a string (via raw-runtime-flags). This will support the Compose models: spec as well as the Compose model provider, respectively.

Signed-off-by: Jacob Howard <jacob.howard@docker.com>
Comment thread pkg/inference/backend.go
Comment on lines +33 to +34
ContextSize int64 `json:"context-size,omitempty"`
RuntimeFlags []string `json:"runtime-flags,omitempty"`
Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't know if this JSON nomenclature is used anywhere (I couldn't find it), but I've tried to make it consistent before it becomes used anywhere.

Copy link
Copy Markdown

@p1-0tr p1-0tr left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

Copy link
Copy Markdown
Contributor

@doringeman doringeman left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM!
(Don't we usually use snake case for JSON keys?)

@xenoscopic
Copy link
Copy Markdown
Contributor Author

I don't think there's a strong Docker convention on the JSON keys (Docker API, Compose, et al. all look different). Right now we've already released provider support with config in Docker CE, so I think we have to leave the keys as is for now 😞

@xenoscopic xenoscopic merged commit 26a0a73 into main Jun 27, 2025
4 checks passed
@xenoscopic xenoscopic deleted the config-list branch June 27, 2025 14:29
doringeman pushed a commit to doringeman/model-runner that referenced this pull request Oct 2, 2025
Fix install race detection using more tolerant polling.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants