Deployment plugins currently don't receive rule-level resource constraints (e.g., threads, resources.mem_mb, resources.gpu).
It would be very useful to pass resource allocation for container tools like podman to enforce them. This would basically turn the scheduler into a userspace container orchestrator.
Proposed Solution
Expose the threads and resources objects to the deployment plugin interface so authors can translate them into native backend flags (including CPU, memory, and accelerators).
rule train_model:
threads: 8
resources:
mem_mb=32000,
gpu=1
So that this becomes:
podman run --cpus=8 --memory=32000m --device nvidia.com/gpu=1 <image>
Deployment plugins currently don't receive rule-level resource constraints (e.g., threads, resources.mem_mb, resources.gpu).
It would be very useful to pass resource allocation for container tools like
podmanto enforce them. This would basically turn the scheduler into a userspace container orchestrator.Proposed Solution
Expose the threads and resources objects to the deployment plugin interface so authors can translate them into native backend flags (including CPU, memory, and accelerators).
So that this becomes:
podman run --cpus=8 --memory=32000m --device nvidia.com/gpu=1 <image>