feat: unify serverless create on graphql saveEndpoint#1
Closed
TimPietruskyRunPod wants to merge 1 commit into
Closed
feat: unify serverless create on graphql saveEndpoint#1TimPietruskyRunPod wants to merge 1 commit into
TimPietruskyRunPod wants to merge 1 commit into
Conversation
route all serverless creates (template, hub, model-reference, cpu, gpu) through the saveEndpoint mutation instead of bifurcating between rest and graphql. this fixes flags being silently dropped on the model/hub paths and gives every flag the full capability set on one path (matching the web console). - map all flags to EndpointInput: idle-timeout, scale-by/threshold, execution-timeout, min-cuda-version, flash-boot (flashBootType enum), network-volume-ids (multi-region), model-reference - translate --gpu-id (gpu type id, e.g. 'NVIDIA A40') to the gpu pool id (e.g. AMPERE_48) saveEndpoint expects, via serverlessGpuPools - cpu via instanceIds (--instance-id, default cpu3g-4-16); no computeType - generate a default name when --name is omitted (saveEndpoint requires it) - echo instanceIds and networkVolumeIds in the create output - remove the now-unused rest CreateEndpoint path - validate conflicting flags instead of dropping them
Author
|
moving this to runpod/runpodctl |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
builds on top of runpod#276. unifies all of
serverless createonto the graphqlsaveEndpointmutation instead of routing some paths through rest and others through graphql.why
runpod#276 added
--model-reference, which forces the graphql path. but the bifurcation meant the path you hit (template → rest, model/hub → graphql) silently decided which flags actually worked. on the graphql paths,--idle-timeout,--scale-by/--scale-threshold,--execution-timeout,--min-cuda-version,--network-volume-ids, and--flash-bootwere silently dropped. the web console usessaveEndpointfor everything, andEndpointInputis a superset of the rest fields — so unifying on it is the way to get every capability on one path with nothing dropped.what changed
saveEndpoint. removed the now-unused restCreateEndpoint.EndpointInput. conflicting flags now error instead of being ignored (e.g.--network-volume-id+--network-volume-ids).saveEndpoint.gpuIdswants gpu pool ids (ADA_24), not the gpu type ids (NVIDIA A40) thatgpu list/--gpu-iduse.--gpu-idis now translated viaserverlessGpuPools(type id or pool id both work).instanceIds(new--instance-id, defaultcpu3g-4-16);saveEndpointhas nocomputeType.flashBootTypeenum, so--flash-boot=falseis honored without the old post-create rest patch.--network-volume-idssent as[{networkVolumeId}].saveEndpointrequires a name (String!, min 3) and never auto-generates one, so a default is generated client-side when--nameis omitted.instanceIdsandnetworkVolumeIdsso cpu/multi-region endpoints are visible.testing
go test ./...passes.NVIDIA GeForce RTX 4090→ADA_24,NVIDIA A40→AMPERE_48), cpu default + explicit instance, flags-take-effect verified server-side via graphql (idleTimeout/scaler/executionTimeoutMs/flashBootType=OFF), single + repeated model references, name auto-generation, data-center-ids, single + multi-region volumes, hub deploy, and all 6 validation/error paths.docs and AGENTS.md updated.