feat: add embedded-python model executor for qwen3 on cuda.#1891

Open

zhang-minchao wants to merge 48 commits into

xLLM-AI:mainfrom

zhang-minchao:feat/python-model-executor

Commits on Jul 14, 2026

refactor: rename max_memory_utilization to kv_cache_memory_fraction.
zhang-minchao
committed
feat: add embedded-python model executor for qwen3 on cuda.
zhang-minchao
committed
fix: reuse executor attn_metadata in PyCausalLM for cuda graph capture
zhang-minchao
committed
fix(graph): make prefill piecewise graph work across tp=1/2 and all input lengths
zhang-minchao
committed
revert: drop kv_cache_memory_fraction, restore max_memory_utilization
zhang-minchao
committed
revert: restore flashinfer workspace default to 128MB
zhang-minchao
committed
refactor: restructure python model package (xllm_models -> python)
zhang-minchao
committed
docs: add multi-hardware xllm_ops op registration design
zhang-minchao
committed
feat: reflect PROPERTY structs so the Python model executor gets full config
zhang-minchao
committed
docs: add todo for vmm piecewise graph-pool follow-up.
zhang-minchao
committed
style: address review feedback on the python model executor.
zhang-minchao
committed
refactor: move weight transform logic from C++ to Python
zhang-minchao
committed
fix: stage python model package into the correct wheel directory
zhang-minchao
committed
style: apply clang-format 20.1.6 to pass CI format check.
zhang-minchao
committed
feat: decode full CUDA graph + prefill piecewise for Python model executor
zhang-minchao
committed
fix(graph): fix cpp_graph decode padding crash and optimize py_graph prefill
zhang-minchao
committed
refactor: remove piecewise cuda graph dead code and optimize decode padding.
zhang-minchao
committed
refactor: restructure ops layer into compute/attention/collectives modules.
zhang-minchao
committed
docs: update multi-hardware ops design doc to reflect ops layer restructure.
zhang-minchao
committed
refactor: use flashinfer Python API directly, remove C++ attention ops bridge
zhang-minchao
committed
revert: restore Platform::is_support_sm90a() check in prefill plan
zhang-minchao
committed
revert: restore determine_attention_backend() in batch_prefill
zhang-minchao
committed
docs: simplify multi-hardware ops design doc
zhang-minchao
committed
docs: merge reshape_paged_cache into compute ops row
zhang-minchao
committed
docs: remove speculative NPU landing section
zhang-minchao
committed
docs: rewrite with C++/Python interaction details
zhang-minchao
committed
refactor: rename meta -> attn_meta in PyCausalLM::forward
zhang-minchao
committed
refactor: use full name attention_metadata_dict, no abbreviation
zhang-minchao
committed
refactor: pass model_impl via ModelContext instead of FLAGS
zhang-minchao
committed
bugfix: fix python executor tp and graph configuration.
zhang-minchao
committed
refactor: rename cuda torch ops library files.
zhang-minchao
committed
build: install flashinfer python runtime.
zhang-minchao
committed
docs: remove python executor design document.
zhang-minchao
committed
docs: add python model execution design documents.
zhang-minchao
committed
refactor: add dedicated python model executor.
zhang-minchao
committed
test: cover decode graph metadata updates.
zhang-minchao
committed
docs: simplify python executor refactor design.
zhang-minchao
committed
refactor: select flashinfer plan schema from backend parameter instead of hardware probe.
zhang-minchao
committed
feat: pass runtime batch limit and model length to python decode graph runner.
zhang-minchao
committed
test: cover padded decode metadata and shared buffer allocation in cuda graph.
zhang-minchao
committed
docs: explain backend-not-hardware flashinfer prefill abi selection.
zhang-minchao
committed
bugfix: fix sliding-window, linear bias, and position cast in python model executor.
zhang-minchao
committed
refactor: keep python executor changes scoped to python graph path
zhang-minchao
committed
bugfix: release python config while holding gil.
zhang-minchao
committed
refactor: narrow cuda graph metadata scope
zhang-minchao
committed
refactor: use empty sequences for graph padding
zhang-minchao
committed
docs: explain empty sequence graph padding
zhang-minchao
committed
docs: add python model architecture translation
zhang-minchao
committed