Skip to content

Commit 9ac89cf

Browse files
committed
update release v0.2.1
1 parent e65c501 commit 9ac89cf

341 files changed

Lines changed: 704755 additions & 61277 deletions

File tree

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

.gitignore

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -6,6 +6,9 @@ csrc/instantiations/*
66
mate/_version.py
77
mate/_build_meta.py
88
mate/data
9+
wrappers/*/_mate_wrapper_version.txt
10+
wrappers/*/_mate_wrapper_requirements.txt
11+
wrappers/*/*/_build_meta.py
912

1013
# vscode
1114
.vscode/

.pre-commit-config.yaml

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -53,6 +53,9 @@ repos:
5353
rev: 'v1.17.1' # Use the sha / tag you want to point at
5454
hooks:
5555
- id: mypy
56+
args: ["--config-file", "pyproject.toml"]
57+
files: ^mate/
58+
exclude: ^(wrappers/|3rdparty/|build/)
5659

5760
- repo: https://github.com/astral-sh/ruff-pre-commit
5861
# Ruff version.

AGENTS.md

Lines changed: 4 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -85,10 +85,13 @@ When working with MATE's dependencies and tools, refer to these official documen
8585
- **Working on FFI bindings** → Check TVM-FFI docs for export patterns and type marshaling
8686
- **Working on kernel bindings** → Check the TVM-FFI kernel library guide for output-allocation patterns and `TensorView` usage
8787

88-
8988
## Behavior Rules
9089

9190
- Always read relevant files before modifying code
9291
- For non-trivial tasks, create a plan first
9392
- Validate changes by running tests
9493
- Prefer minimal, localized changes
94+
- When running as `root` in the container, restore edited tracked files to the
95+
workspace owner/group and keep them user/group writable before handing off.
96+
A typical fix is `chown <workspace_uid>:<workspace_gid> <files>` followed by
97+
`chmod ug+rw <files>`, using the owner of `/workspace/mate` as the target.

README.md

Lines changed: 48 additions & 33 deletions
Original file line numberDiff line numberDiff line change
@@ -1,22 +1,13 @@
11
# MUSA AI Tensor Engine
22

3-
MATE (**M**USA **A**I **T**ensor **E**ngine) is a centralized library for Generative AI workloads on MUSA. It provides high-performance attention and GEMM operators, and compatibility wrappers for CUDA-oriented Python APIs.
3+
MATE (**M**USA **A**I **T**ensor **E**ngine) is a centralized library for Generative AI workloads on MUSA. It provides high-performance Attention and GEMM operators, and compatibility wrappers for CUDA-oriented Python APIs.
44

55
## Highlights
66

77
- High-performance attention and GEMM operators for MUSA
88
- Compatibility wrappers for `flash_attn_3`, `sageattention`, `flash_mla`, and `deep_gemm`
99
- CLI tools for environment checks, configuration inspection, and replay
1010

11-
## Quick Links
12-
13-
- CLI documentation: [docs/mate_cli.md](docs/mate_cli.md)
14-
- FlashAttention-3 compatibility summary: [docs/flash_attention.md](docs/flash_attention.md)
15-
- FlashAttention-3 wrapper: [wrappers/flash-attention/README.md](wrappers/flash-attention/README.md)
16-
- SageAttention wrapper: [wrappers/SageAttention/README.md](wrappers/SageAttention/README.md)
17-
- FlashMLA wrapper: [wrappers/FlashMLA/README.md](wrappers/FlashMLA/README.md)
18-
- DeepGEMM wrapper: [wrappers/DeepGEMM/README.md](wrappers/DeepGEMM/README.md)
19-
2011
## Requirements
2112

2213
| Component | Requirement |
@@ -25,53 +16,55 @@ MATE (**M**USA **A**I **T**ensor **E**ngine) is a centralized library for Genera
2516
| TorchMUSA | `2.7` or later |
2617
| Architecture | `Pinghu (MP31)` |
2718

28-
## Build From Source
19+
## Quick Start
2920

30-
```bash
31-
git clone https://github.com/MooreThreads/mate.git --recursive
32-
cd mate
33-
python -m build --wheel --no-isolation
34-
```
21+
Use these commands after the MUSA-enabled `torch` / `torch_musa` stack is
22+
installed. Keep dependency resolution disabled for local builds so pip does not
23+
replace that stack with upstream PyPI packages.
24+
25+
- Use `--no-build-isolation` for source installs.
26+
- Use `--no-isolation` for wheel builds.
27+
- Use `--no-deps` when installing local builds.
3528

36-
For local development, install MATE in editable mode:
29+
### Development Install
3730

3831
```bash
3932
git clone https://github.com/MooreThreads/mate.git --recursive
4033
cd mate
41-
pip install --no-build-isolation -e . -v
34+
pip install --no-build-isolation --no-deps -e . -v
4235
```
4336

44-
If you forgot `--recursive` when cloning, initialize submodules before building or
45-
installing:
37+
### Build a Wheel
4638

4739
```bash
48-
git submodule update --init --recursive
40+
git clone https://github.com/MooreThreads/mate.git --recursive
41+
cd mate
42+
python -m build --wheel --no-isolation
43+
python -m pip install --no-deps dist/mate-*.whl
4944
```
5045

51-
Pre-build AOT kernels before packaging the wheel:
46+
### Optional: Pre-Build AOT Kernels
5247

5348
```bash
54-
git clone https://github.com/MooreThreads/mate.git --recursive
55-
cd mate
5649
MATE_MUSA_ARCH_LIST=3.1 python -m mate.aot
5750
python -m build --wheel --no-isolation
5851
```
5952

60-
Customize AOT coverage by operator family when needed:
53+
Customize AOT coverage when needed:
6154

6255
```bash
6356
python -m mate.aot --attention-aot-level 0 --add-gemm true --add-moe false
6457
```
6558

66-
## Repository Layout
59+
### Notes
6760

68-
| Path | Purpose |
69-
| --- | --- |
70-
| `mate/` | Core Python package and public APIs |
71-
| `wrappers/` | Compatibility wrapper packages for existing Python ecosystems |
72-
| `docs/` | Markdown docs and Sphinx sources |
73-
| `tests/` | Correctness and integration tests |
74-
| `benchmarks/` | Performance and benchmarking scripts |
61+
- If the checkout was cloned without `--recursive`, run `git submodule update --init --recursive`.
62+
- Do not let pip resolve and replace the MUSA PyTorch dependencies unless that
63+
is intentional.
64+
- See [docs/mate_cli.md](docs/mate_cli.md) for CLI extras and local wheel
65+
installation details.
66+
- See [docs/environment_variables.md](docs/environment_variables.md) for build
67+
and runtime environment variables.
7568

7669
## MATE CLI
7770

@@ -96,6 +89,8 @@ mate list-dumps mate_dumps/
9689
```
9790

9891
See [docs/mate_cli.md](docs/mate_cli.md) for full CLI documentation.
92+
See [docs/environment_variables.md](docs/environment_variables.md) for the
93+
complete environment variable reference.
9994

10095
## Wrappers
10196

@@ -108,6 +103,16 @@ MATE uses the packages under `wrappers/` as a compatibility layer for CUDA-orien
108103
| `wrappers/FlashMLA` | `flash_mla` | `flash_mla` | FlashMLA-compatible MLA dense/sparse decode and sparse prefill APIs on top of MATE MLA operators on MUSA | [wrapper README](wrappers/FlashMLA/README.md) |
109104
| `wrappers/DeepGEMM` | `deep-gemm` | `deep_gemm` | DeepGEMM-compatible APIs on top of MATE GEMM operators on MUSA | [wrapper README](wrappers/DeepGEMM/README.md) |
110105

106+
## Repository Layout
107+
108+
| Path | Purpose |
109+
| --- | --- |
110+
| `mate/` | Core Python package and public APIs |
111+
| `wrappers/` | Compatibility wrapper packages for existing Python ecosystems |
112+
| `docs/` | Markdown docs and Sphinx sources |
113+
| `tests/` | Correctness and integration tests |
114+
| `benchmarks/` | Performance and benchmarking scripts |
115+
111116
## Build Documentation
112117

113118
After installing `mate`, build the Sphinx docs with:
@@ -118,6 +123,16 @@ cd docs
118123
make html
119124
```
120125

126+
## Quick Links
127+
128+
- CLI documentation: [docs/mate_cli.md](docs/mate_cli.md)
129+
- Environment variables: [docs/environment_variables.md](docs/environment_variables.md)
130+
- FlashAttention-3 compatibility summary: [docs/flash_attention.md](docs/flash_attention.md)
131+
- FlashAttention-3 wrapper: [wrappers/flash-attention/README.md](wrappers/flash-attention/README.md)
132+
- SageAttention wrapper: [wrappers/SageAttention/README.md](wrappers/SageAttention/README.md)
133+
- FlashMLA wrapper: [wrappers/FlashMLA/README.md](wrappers/FlashMLA/README.md)
134+
- DeepGEMM wrapper: [wrappers/DeepGEMM/README.md](wrappers/DeepGEMM/README.md)
135+
121136
## Acknowledgement
122137

123138
MATE is inspired by [FlashInfer](https://github.com/flashinfer-ai/flashinfer), [FlashAttention](https://github.com/Dao-AILab/flash-attention), [cutlass](https://github.com/NVIDIA/cutlass), [FlashMLA](https://github.com/deepseek-ai/FlashMLA), and [DeepGemm](https://github.com/deepseek-ai/DeepGEMM).

benchmarks/bench_gdn_decode.py

Lines changed: 216 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,216 @@
1+
from __future__ import annotations
2+
3+
import argparse
4+
5+
import torch
6+
7+
import mate
8+
9+
from mate.testing.utils import bench_kineto
10+
11+
12+
DEFAULT_BATCH_SIZES = (1, 2, 4, 8, 16, 32, 64, 128, 256, 512)
13+
DEFAULT_HEAD_CONFIGS = ("8,16", "16,32", "16,64")
14+
KERNEL_NAME = "gated_deltanet_decode_fp32_vk"
15+
16+
17+
def _dtype_size(dtype: torch.dtype) -> int:
18+
return torch.empty((), dtype=dtype).element_size()
19+
20+
21+
def _parse_head_config(spec: str) -> tuple[int, int]:
22+
parts = [part.strip() for part in spec.split(",") if part.strip()]
23+
if len(parts) != 2:
24+
raise ValueError(
25+
f"Invalid head config {spec!r}. Expected 'Hq,Hv', for example '16,32'."
26+
)
27+
num_q_heads, num_v_heads = (int(part) for part in parts)
28+
if num_q_heads <= 0 or num_v_heads <= 0:
29+
raise ValueError(f"Head counts must be positive, got {spec!r}.")
30+
return num_q_heads, num_v_heads
31+
32+
33+
def _gdn_decode_flops(
34+
batch_size: int, num_q_heads: int, num_v_heads: int, head_size: int
35+
) -> int:
36+
num_o_heads = max(num_q_heads, num_v_heads)
37+
return 6 * batch_size * num_o_heads * head_size * head_size
38+
39+
40+
def _gdn_decode_bytes(
41+
batch_size: int,
42+
num_q_heads: int,
43+
num_v_heads: int,
44+
head_size: int,
45+
input_dtype: torch.dtype,
46+
output_dtype: torch.dtype,
47+
) -> int:
48+
num_o_heads = max(num_q_heads, num_v_heads)
49+
elem_size = _dtype_size(input_dtype)
50+
51+
q_bytes = batch_size * num_q_heads * head_size * elem_size
52+
k_bytes = batch_size * num_q_heads * head_size * elem_size
53+
v_bytes = batch_size * num_v_heads * head_size * elem_size
54+
o_bytes = batch_size * num_o_heads * head_size * _dtype_size(output_dtype)
55+
state_bytes = 2 * batch_size * num_v_heads * head_size * head_size * 4
56+
A_log_bytes = num_v_heads * 4
57+
dt_bias_bytes = num_v_heads * 4
58+
a_bytes = batch_size * num_v_heads * elem_size
59+
b_bytes = batch_size * num_v_heads * elem_size
60+
61+
return (
62+
q_bytes
63+
+ k_bytes
64+
+ v_bytes
65+
+ o_bytes
66+
+ state_bytes
67+
+ A_log_bytes
68+
+ dt_bias_bytes
69+
+ a_bytes
70+
+ b_bytes
71+
)
72+
73+
74+
def _make_inputs(
75+
batch_size: int,
76+
num_q_heads: int,
77+
num_v_heads: int,
78+
head_size: int,
79+
dtype: torch.dtype,
80+
) -> tuple[torch.Tensor, ...]:
81+
device = torch.device("musa")
82+
q = torch.randn((batch_size, 1, num_q_heads, head_size), dtype=dtype, device=device)
83+
k = torch.randn((batch_size, 1, num_q_heads, head_size), dtype=dtype, device=device)
84+
v = torch.randn((batch_size, 1, num_v_heads, head_size), dtype=dtype, device=device)
85+
state = torch.randn(
86+
(batch_size, num_v_heads, head_size, head_size),
87+
dtype=torch.float32,
88+
device=device,
89+
)
90+
A_log = torch.randn((num_v_heads,), dtype=torch.float32, device=device) * 0.1
91+
dt_bias = torch.randn((num_v_heads,), dtype=torch.float32, device=device) * 0.1
92+
a = torch.randn((batch_size, 1, num_v_heads), dtype=dtype, device=device) * 0.1
93+
b = torch.randn((batch_size, 1, num_v_heads), dtype=dtype, device=device)
94+
return q, k, v, state, A_log, a, dt_bias, b
95+
96+
97+
def _bench_one_shape(
98+
batch_size: int,
99+
num_q_heads: int,
100+
num_v_heads: int,
101+
head_size: int,
102+
dtype: torch.dtype,
103+
num_tests: int,
104+
) -> float:
105+
q, k, v, state, A_log, a, dt_bias, b = _make_inputs(
106+
batch_size=batch_size,
107+
num_q_heads=num_q_heads,
108+
num_v_heads=num_v_heads,
109+
head_size=head_size,
110+
dtype=dtype,
111+
)
112+
113+
def _runner():
114+
mate.gated_delta_rule_decode(
115+
q=q,
116+
k=k,
117+
v=v,
118+
state=state,
119+
A_log=A_log,
120+
a=a,
121+
dt_bias=dt_bias,
122+
b=b,
123+
state_layout="VK",
124+
scale=None,
125+
use_qk_l2norm=True,
126+
)
127+
128+
seconds = bench_kineto(
129+
_runner,
130+
kernel_names=KERNEL_NAME,
131+
num_tests=num_tests,
132+
suppress_kineto_output=True,
133+
flush_l2=True,
134+
)
135+
if seconds <= 0:
136+
raise RuntimeError(f"Failed to capture kernel time for {KERNEL_NAME}.")
137+
return float(seconds)
138+
139+
140+
def main():
141+
parser = argparse.ArgumentParser(description="Benchmark MATE GDN decode kernel.")
142+
parser.add_argument("--dtype", choices=["fp16", "bf16"], default="bf16")
143+
parser.add_argument("--num-tests", type=int, default=10)
144+
parser.add_argument("--head-size", type=int, default=128)
145+
parser.add_argument(
146+
"--batch-sizes",
147+
type=int,
148+
nargs="+",
149+
default=list(DEFAULT_BATCH_SIZES),
150+
help="Batch sizes to benchmark. Default follows FlashInfer/SGLang decode sweeps.",
151+
)
152+
parser.add_argument(
153+
"--head-configs",
154+
nargs="+",
155+
default=list(DEFAULT_HEAD_CONFIGS),
156+
help="Head configs in 'Hq,Hv' form. Default covers 8,16 / 16,32 / 16,64.",
157+
)
158+
args = parser.parse_args()
159+
160+
if not (hasattr(torch, "musa") and torch.musa.is_available()):
161+
raise RuntimeError("MUSA device is not available.")
162+
163+
dtype = torch.float16 if args.dtype == "fp16" else torch.bfloat16
164+
head_configs = [_parse_head_config(spec) for spec in args.head_configs]
165+
166+
print(f"\nMUSA: {torch.musa.get_device_name(0)}")
167+
print(f"Kernel: {KERNEL_NAME}")
168+
print(
169+
f"Config: D={args.head_size}, dtype={args.dtype}, "
170+
f"head_configs={head_configs}, batches={tuple(args.batch_sizes)}"
171+
)
172+
print()
173+
174+
header = (
175+
f"{'Hq':>4s} {'Hv':>4s} {'B':>4s} "
176+
f"{'Latency(us)':>12s} {'TFLOPS':>8s} {'GB/s':>8s}"
177+
)
178+
print(header)
179+
print("-" * len(header))
180+
181+
for num_q_heads, num_v_heads in head_configs:
182+
for batch_size in args.batch_sizes:
183+
seconds = _bench_one_shape(
184+
batch_size=batch_size,
185+
num_q_heads=num_q_heads,
186+
num_v_heads=num_v_heads,
187+
head_size=args.head_size,
188+
dtype=dtype,
189+
num_tests=args.num_tests,
190+
)
191+
flops = _gdn_decode_flops(
192+
batch_size=batch_size,
193+
num_q_heads=num_q_heads,
194+
num_v_heads=num_v_heads,
195+
head_size=args.head_size,
196+
)
197+
io_bytes = _gdn_decode_bytes(
198+
batch_size=batch_size,
199+
num_q_heads=num_q_heads,
200+
num_v_heads=num_v_heads,
201+
head_size=args.head_size,
202+
input_dtype=dtype,
203+
output_dtype=dtype,
204+
)
205+
latency_us = seconds * 1e6
206+
tflops = flops / seconds / 1e12
207+
bandwidth = io_bytes / seconds / 1e9
208+
print(
209+
f"{num_q_heads:>4d} {num_v_heads:>4d} {batch_size:>4d} "
210+
f"{latency_us:>12.3f} {tflops:>8.3f} {bandwidth:>8.3f}"
211+
)
212+
torch.musa.empty_cache()
213+
214+
215+
if __name__ == "__main__":
216+
main()

0 commit comments

Comments
 (0)