Feature request
Standardize the component name that decode workers register with in the /health endpoint across all framework adapters (dynamo.sglang, dynamo.trtllm, dynamo.vllm).
Currently, dynamo.sglang decode workers register as component: "backend", while dynamo.trtllm decode workers register as component: "tensorrt_llm". Prefill workers consistently use component: "prefill" across both frameworks.
A consistent component name (e.g. "backend" for all decode workers) would allow consumers of the /health endpoint to identify prefill vs decode workers without framework-specific logic.
Describe the problem you're encountering
When polling the Dynamo frontend /health endpoint to validate worker registration, the response uses different component values for decode workers depending on the framework:
SGLang decode worker response:
{"component": "backend", "endpoint": "generate", ...}TRT-LLM decode worker response:
{"component": "tensorrt_llm", "endpoint": "generate", ...}This means any orchestration or health-checking tool that needs to count registered decode workers must maintain a list of all possible component names per framework, rather than using a single generic filter.
Discovered while integrating dynamo.trtllm disaggregated serving into NVIDIA's Benchmarking Toolkit (BTK). Our health check plugin polls /health and filters by component name to validate worker registration.
Describe alternatives you've tried
We added an OR clause to our health check's jq filter to accept both "backend" and "tensorrt_llm" as decode components. This works but requires updating the filter every time a new framework adapter uses a different component name.
Feature request
Standardize the
componentname that decode workers register with in the/healthendpoint across all framework adapters (dynamo.sglang,dynamo.trtllm,dynamo.vllm).Currently,
dynamo.sglangdecode workers register ascomponent: "backend", whiledynamo.trtllmdecode workers register ascomponent: "tensorrt_llm". Prefill workers consistently usecomponent: "prefill"across both frameworks.A consistent component name (e.g.
"backend"for all decode workers) would allow consumers of the/healthendpoint to identify prefill vs decode workers without framework-specific logic.Describe the problem you're encountering
When polling the Dynamo frontend
/healthendpoint to validate worker registration, the response uses differentcomponentvalues for decode workers depending on the framework:SGLang decode worker response:
{"component": "backend", "endpoint": "generate", ...}TRT-LLM decode worker response:
{"component": "tensorrt_llm", "endpoint": "generate", ...}This means any orchestration or health-checking tool that needs to count registered decode workers must maintain a list of all possible component names per framework, rather than using a single generic filter.
Discovered while integrating
dynamo.trtllmdisaggregated serving into NVIDIA's Benchmarking Toolkit (BTK). Our health check plugin polls/healthand filters by component name to validate worker registration.Describe alternatives you've tried
We added an OR clause to our health check's jq filter to accept both
"backend"and"tensorrt_llm"as decode components. This works but requires updating the filter every time a new framework adapter uses a different component name.