|
1 | 1 | # MUSA AI Tensor Engine |
2 | 2 |
|
3 | | -MATE(**M**USA **A**I **T**ensor **E**ngine) is a centralized library for Generative AI. |
| 3 | +MATE (**M**USA **A**I **T**ensor **E**ngine) is a centralized library for Generative AI workloads on MUSA. It provides high-performance attention and GEMM operators, and compatibility wrappers for CUDA-oriented Python APIs. |
4 | 4 |
|
5 | | -## Installation |
| 5 | +## Highlights |
6 | 6 |
|
7 | | -### Requirements |
| 7 | +- High-performance attention and GEMM operators for MUSA |
| 8 | +- Compatibility wrappers for `flash_attn` and `deep_gemm` |
| 9 | +- CLI tools for environment checks, configuration inspection, and replay |
8 | 10 |
|
9 | | -MUSA Toolkits: 4.3.4 or later |
| 11 | +## Quick Links |
10 | 12 |
|
11 | | -TorchMUSA: 2.7.1 or later |
| 13 | +- CLI documentation: [docs/mate_cli.md](docs/mate_cli.md) |
| 14 | +- FlashAttention compatibility summary: [docs/flash_attention.md](docs/flash_attention.md) |
| 15 | +- FlashAttention wrapper: [wrappers/flash-attention/README.md](wrappers/flash-attention/README.md) |
| 16 | +- DeepGEMM wrapper: [wrappers/deep_gemm/README.md](wrappers/deep_gemm/README.md) |
12 | 17 |
|
13 | | -Architecutre: Pinghu(MP31) |
| 18 | +## Requirements |
14 | 19 |
|
15 | | -### Install From Source |
| 20 | +| Component | Requirement | |
| 21 | +| --- | --- | |
| 22 | +| MUSA Toolkit | `4.3.6` or later | |
| 23 | +| TorchMUSA | `2.7` or later | |
| 24 | +| Architecture | `Pinghu (MP31)` | |
| 25 | + |
| 26 | +## Build From Source |
16 | 27 |
|
17 | 28 | ```bash |
18 | | -git clone git@github.com:MooreThreads/mate.git --recursive |
| 29 | +git clone https://github.com/MooreThreads/mate.git --recursive |
19 | 30 | cd mate |
20 | | -pip install --no-build-isolation . -v |
| 31 | +bash build.sh |
21 | 32 | ``` |
22 | 33 |
|
23 | | -### Build From Source |
| 34 | +## Repository Layout |
| 35 | + |
| 36 | +| Path | Purpose | |
| 37 | +| --- | --- | |
| 38 | +| `mate/` | Core Python package and public APIs | |
| 39 | +| `wrappers/` | Compatibility wrapper packages for existing Python ecosystems | |
| 40 | +| `docs/` | Markdown docs and Sphinx sources | |
| 41 | +| `tests/` | Correctness and integration tests | |
| 42 | +| `benchmarks/` | Performance and benchmarking scripts | |
| 43 | + |
| 44 | +## MATE CLI |
| 45 | + |
| 46 | +MATE provides a command-line interface for configuration, debugging, diagnostics, and replay. |
| 47 | + |
| 48 | +| Command | Purpose | |
| 49 | +| --- | --- | |
| 50 | +| `mate check` | Validate the runtime environment | |
| 51 | +| `mate show-config` | Display installation and runtime configuration | |
| 52 | +| `mate env` | Show relevant environment variables | |
| 53 | +| `mate replay --dir PATH` | Replay API calls from Level 10 dumps | |
| 54 | +| `mate list-dumps PATH` | List recorded dump directories | |
| 55 | + |
| 56 | +Example: |
24 | 57 |
|
25 | 58 | ```bash |
26 | | -git clone git@github.com:MooreThreads/mate.git --recursive |
27 | | -cd mate |
28 | | -python -m build --wheel --no-isolation |
| 59 | +mate check |
| 60 | +mate show-config |
| 61 | +mate env |
| 62 | +mate replay --dir mate_dumps/ |
| 63 | +mate list-dumps mate_dumps/ |
29 | 64 | ``` |
30 | 65 |
|
31 | | -## Build Docs |
| 66 | +See [docs/mate_cli.md](docs/mate_cli.md) for full CLI documentation. |
| 67 | + |
| 68 | +## Wrappers |
| 69 | + |
| 70 | +MATE uses the packages under `wrappers/` as a compatibility layer for CUDA-oriented software stacks on MUSA. These wrappers preserve familiar package names and high-level APIs while routing execution to MATE operators and kernels on MUSA, which helps existing integrations migrate with smaller code changes. |
| 71 | + |
| 72 | +| Wrapper | Package | Import Path | Purpose | Documentation | |
| 73 | +| --- | --- | --- | --- | --- | |
| 74 | +| `wrappers/flash-attention` | `mate-flash-attention` | `flash_attn` | FlashAttention-compatible APIs on top of MATE attention operators on MUSA | [wrapper README](wrappers/flash-attention/README.md), [compatibility summary](docs/flash_attention.md) | |
| 75 | +| `wrappers/deep_gemm` | `mate-deep_gemm` | `deep_gemm` | DeepGEMM-compatible APIs on top of MATE GEMM operators on MUSA | [wrapper README](wrappers/deep_gemm/README.md) | |
| 76 | + |
| 77 | +## Build Documentation |
32 | 78 |
|
33 | | -**After installing mate**, run the following commands: |
| 79 | +After installing `mate`, build the Sphinx docs with: |
34 | 80 |
|
35 | 81 | ```bash |
36 | 82 | pip install sphinx furo |
37 | | -cd mate/docs |
| 83 | +cd docs |
38 | 84 | make html |
39 | 85 | ``` |
40 | 86 |
|
41 | 87 | ## Acknowledgement |
42 | 88 |
|
43 | | -Mate is inspired by [FlashInfer](https://github.com/flashinfer-ai/flashinfer), [FlashAttention](https://github.com/Dao-AILab/flash-attention), [cutlass](https://github.com/NVIDIA/cutlass), [FlashMLA](https://github.com/deepseek-ai/FlashMLA) and [DeepGemm](https://github.com/deepseek-ai/DeepGEMM) projects. |
| 89 | +MATE is inspired by [FlashInfer](https://github.com/flashinfer-ai/flashinfer), [FlashAttention](https://github.com/Dao-AILab/flash-attention), [cutlass](https://github.com/NVIDIA/cutlass), [FlashMLA](https://github.com/deepseek-ai/FlashMLA), and [DeepGemm](https://github.com/deepseek-ai/DeepGEMM). |
0 commit comments