Remove legacy proxy, update docs, and enhance scripts#1447
Merged
Conversation
The old `proxy.py` routing proxy is superseded by atomesh. Remove it along with the `_service_discovery_ping` heartbeat threads in both mooncake and moriio connectors that registered with it. The `proxy_ip` / `proxy_ping_port` config keys are still accepted but silently ignored, so existing kv-transfer-config payloads keep working.
Keep only the 7 scripts referenced by scripts/README.md (docker_start, start_prefill, start_decode, start_router, run_gsm8k, run_benchmark, ds_fp8_1p_tp4_1d_tp8_slurm). The rest are superseded by the weekly benchmark harness and per-model slurm scripts outside the repo.
…t.sh Auto-detect bnxt/ionic/mlx5 NICs and mount the correct host ibverbs provider libraries into the container. Also adds /dev/infiniband, --shm-size 128G, and post-start TCP backlog tuning. Verified on ionic (mia1-p02-g42).
Update kv-transfer-config examples to drop proxy_ip/proxy_ping_port and add atomesh router launch steps. Affected docs: - recipes/pd_disaggregation_guide.md - recipes/DeepSeek-V4.md - recipes/mesh/multi-node-atom.md - atom/kv_transfer/disaggregation/README.md
Remove the Mooncake build-from-source section (67 lines) and replace with docker pull + docker_start.sh which handles RDMA NIC detection and SO mounting automatically.
Single-node 1P+1D setup (GPU 0-3 prefill, GPU 4-7 decode, TP=4 each) with atomesh router. Covers base MXFP4, EAGLE3 variant, GSM8K accuracy validation, and serving benchmark — all via the router endpoint.
3-node setup: 2 prefill instances (TP=4, DPA + TBO) + 1 decode instance (TP=4, DPA, max-num-seqs=1024). Covers atomesh router with dual --prefill flags, GSM8K accuracy, and high-concurrency benchmark sweep.
Add --online_quant_config to all PD server commands (1P1D and 2P1D). Include a reference table documenting the exclude_layer differences between MXFP4/MXFP8 and TP-only/DPA modes.
…nnectors These config reads were kept for backward compatibility but the only consumer (_service_discovery_ping) was already removed. Old configs that pass proxy_ip still work — dict.get() silently discards the value.
…atom_mesh Replace mesh-sglang-latest with rocm/atom-dev:latest as the default docker image. List vllm-latest and sglang-latest as alternatives in comments. Rename default container from atom_sglang_mesh to atom_mesh.
The SLURM one-shot automation is now handled by per-model scripts outside the repo. Remove the generic ds_fp8_1p_tp4_1d_tp8_slurm.sh and its corresponding documentation from the scripts README.
Move all PD disaggregation sections (1P+1D, 2P+1D DPA, EAGLE3 variants, online_quant_config reference) from recipes/MiniMax-M3.md into a dedicated mesh recipe. The original file now has a cross-reference link. The new doc covers all 8 benchmark configurations (FP4/FP8 × 1P1D/2P1D × plain/EAGLE3), distilled from the weekly_mesh_benchmark scripts.
Distill the 5 weekly benchmark scripts (1P1D TP, 1P1D MTP, 2P1D DPA, 2P1D MTP DPA, 2P1D MTP3 DPA) into a single user-facing recipe at recipes/mesh/DeepSeek-V4.md covering all topologies with MTP as an add-on section.
Contributor
There was a problem hiding this comment.
Pull request overview
This PR removes the legacy Python-based P/D disaggregation proxy and updates the serving/disaggregation guides to route traffic via atomesh (Rust router), while also streamlining mesh-related scripts (notably improving Docker startup with RDMA NIC auto-detection).
Changes:
- Deleted the legacy
atom.kv_transfer.disaggregation.proxyimplementation and its unit tests. - Updated PD disaggregation documentation (and added new mesh recipes) to use
atomesh launch+handshake_port-based configs. - Simplified/removed a large set of legacy standalone + SLURM scripts; expanded
docker_start.shto auto-detect RDMA NIC type and mount appropriate host libraries.
Reviewed changes
Copilot reviewed 38 out of 38 changed files in this pull request and generated 3 comments.
Show a summary per file
| File | Description |
|---|---|
| tests/test_proxy.py | Removes unit tests tied to the deleted legacy proxy. |
| recipes/pd_disaggregation_guide.md | Rewrites PD guide to use atomesh instead of the Python proxy; updates ports and configs. |
| recipes/MiniMax-M3.md | Adds pointer to the new mesh-based PD disaggregation recipe. |
| recipes/mesh/multi-node-atom.md | Removes proxy-specific configuration/documentation from multi-node ATOM recipe. |
| recipes/mesh/MiniMax-M3.md | New end-to-end MiniMax-M3 PD disaggregation guide using atomesh. |
| recipes/mesh/DeepSeek-V4.md | New DeepSeek-V4-Pro PD disaggregation guide using atomesh. |
| recipes/DeepSeek-V4.md | Updates existing DeepSeek V4 doc to use atomesh instead of the Python proxy. |
| atom/mesh/scripts/start_standalone.sh | Removes legacy standalone SGLang launcher script. |
| atom/mesh/scripts/start_standalone_vllm.sh | Removes legacy standalone vLLM launcher script. |
| atom/mesh/scripts/README.md | Updates script docs (image/tag changes, removes one-shot SLURM automation section). |
| atom/mesh/scripts/ds_fp4_standalone_tp8_vllm_slurm.sh | Removes legacy standalone benchmark SLURM script. |
| atom/mesh/scripts/ds_fp4_standalone_tp8_sglang_slurm.sh | Removes legacy standalone benchmark SLURM script. |
| atom/mesh/scripts/ds_fp4_standalone_tp8_atom_slurm.sh | Removes legacy standalone benchmark SLURM script. |
| atom/mesh/scripts/ds_fp4_standalone_dp8ep8_sglang_slurm.sh | Removes legacy DP/EP standalone benchmark SLURM script. |
| atom/mesh/scripts/ds_fp4_standalone_dp8ep8_atom_slurm.sh | Removes legacy DP/EP standalone benchmark SLURM script. |
| atom/mesh/scripts/ds_fp4_1p_tp8_1d_tp8_vllm_slurm.sh | Removes legacy PD benchmark SLURM script. |
| atom/mesh/scripts/ds_fp4_1p_tp8_1d_tp8_atom_slurm.sh | Removes legacy PD benchmark SLURM script. |
| atom/mesh/scripts/ds_fp4_1p_tp4_2d_tp8_vllm_slurm.sh | Removes legacy PD benchmark SLURM script. |
| atom/mesh/scripts/docker_start.sh | Enhances Docker startup: NIC auto-detection + RDMA-related mounts + TCP backlog tuning. |
| atom/mesh/scripts/docker_start_vllm.sh | Removes legacy vLLM-specific docker start script (consolidation). |
| atom/mesh/scripts/atom_sgl_mesh_xpyd.sh | Removes legacy single-node orchestration script for SGLang mesh. |
| atom/kv_transfer/disaggregation/README.md | Updates disaggregation README to reference atomesh routing and new launch steps. |
| atom/kv_transfer/disaggregation/proxy.py | Deletes the legacy Python proxy service implementation. |
| atom/kv_transfer/disaggregation/moriio/moriio_connector.py | Removes legacy proxy service-discovery ping wiring from MoRI-IO connector. |
| atom/kv_transfer/disaggregation/moriio/moriio_common.py | Removes service discovery constants (no longer used). |
| atom/kv_transfer/disaggregation/mooncake/mooncake_connector.py | Removes legacy proxy service-discovery ping wiring from Mooncake connector. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
…eek-V4 recipe ATOM_CPU_AFFINITY was the old NUMA-blind linear CPU slice, replaced by the topology-aware ATOM_NUMA_BIND. The old var is not read by any code.
3 tasks
8 tasks
valarLip
approved these changes
Jul 7, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Motivation
Technical Details
Test Plan
Test Result
Submission Checklist