bugfix: fix padded decode cuda graph metadata.

[zhang-minchao]

Description

Native C++ decode CUDA graphs normally pad the token/query batch to reusable graph buckets. With max_seqs_per_batch=100, padding mode executes a 112-row graph. Before this fix, FlashInfer decode metadata still described only the 100 real sequences, so capture could complete but replay could access metadata out of bounds and surface a CUDA illegal-memory-access error (or a later NCCL failure under TP2).

This was previously hidden when enable_graph_mode_decode_no_padding=true: decode used the exact runtime batch size and never entered the 100 -> 112 padding path. This PR now supports both modes explicitly and aligns the CUDA bucket policy with AclGraphExecutorImpl:

• when decode no-padding mode is enabled, graph size and persistent sequence capacity remain at the actual batch size (100 -> 100);
• otherwise, persistent per-sequence capacity follows the graph bucket rule (100 -> 112);
• prefill continues to use padded graph buckets even when decode no-padding mode is enabled, preserving piecewise graph reuse;
• padded decode metadata is generated consistently in both the CUDA fast path and fallback path;
• padded sequences are represented as empty KV sequences by repeating cumulative KV offsets, setting KV-length deltas to 0, and setting paged_kv_last_page_len to 1;
• padded cache slots use -1, so reshape_paged_cache skips them instead of writing to real slot 0;
• graph-facing KV-length metadata stays in persistent buffers, keeping capture/replay addresses stable;
• tests cover fast/fallback metadata equivalence, padded decode 100 -> 112, no-padding decode 100 -> 100, and prefill piecewise graphs with the decode no-padding flag enabled.

The FlashInfer plan/decode invocation itself is unchanged. Padding and persistent graph inputs remain owned by CudaGraphExecutorImpl.

Related Issues

None.

Change Type

• Bug fix
• New feature
• Performance improvement
• Refactor
• Documentation
• Test
• Build or CI

Pull Request Checklist

Thank you for contributing to xLLM. Before requesting review, please make sure the following items are complete.

PR Title and Commit Messages

• The PR title and each commit message follow the xLLM commit format: <type>: <subject>.

Allowed types: feat, bugfix, docs, test, refactor, chore, style, revert, perf, model, build, release.
The subject should use clear English, start with a verb, include at least 4 words, and end with ..

Pre-commit Checks

• I have installed pre-commit by running pip install pre-commit or an equivalent command.
• I have installed the hooks with pre-commit install.
• I have run pre-commit run --all-files and fixed any reported issues.

If you are unsure how to set up pre-commit, see the pre-commit documentation.

Self Review

• I have self-reviewed the code according to .agents/skills/code-review/references/custom-code-style.md, especially code written or assisted by AI.
• I have rebased this PR onto the latest main branch.

Build and Test Coverage

• Tests have been added or updated as needed.
• CUDA: python setup.py build test has passed on a CUDA machine.
• NPU: python setup.py build test has passed on an NPU machine.
• MLU: python setup.py build test has passed on an MLU machine.

CUDA validation was run inside zmc-xllm-cuda-x86:

• python setup.py test --test-name cuda_graph_executor_test: 11/11 tests passed;
• python setup.py test: 684/684 executed tests passed (2 randomized fused-sampler tests skipped; disabled tests were not run).

Real C++ graph service sweeps also passed at TP1 and TP2 for batches 1 2 3 5 7 8 13 16 31 47 100, with input length 1024 and output length 256. Both service configurations enabled:

• enable_graph=true;
• enable_graph_vmm_pool=true;
• enable_prefill_piecewise_graph=true;
• max_tokens_for_graph_mode=1024.

The full sweep passed in both decode modes:

• padding mode: enable_graph_mode_decode_no_padding=false, including 100 -> 112;
• no-padding mode: enable_graph_mode_decode_no_padding=true, including exact batch 100.

No CUDA illegal-memory-access or in-request NCCL failure occurred. The TP2 no-padding log contains a TCPStore heartbeat warning only after the completed sweep, when the test harness intentionally stopped rank 0 before rank 1.

Reviewer Notes

Please focus review on:

• consistency between runtime graph bucket selection and persistent sequence capacity;
• decode-only scope of enable_graph_mode_decode_no_padding;
• metadata shape consistency and persistent-address stability in padding mode;
• prefill retaining bucket padding when decode no-padding mode is enabled.

The service runs used idle GPUs at test start, but latency measurements are not used as performance conclusions.