Skip to content

Commit 0385302

Browse files
Cortexelus“Cortexelus”
andauthored
docs: MLX training/gradio + tflite gradio across the optimized READMEs (#75)
- optimized/README.md (new): index of the mlx / tflite / tensorRT runtimes with an inference / web-UI / training capability matrix + install one-liners. - optimized/mlx/README.md: call out underfit as the recommended (dashboard) way to train on a Mac; the CLI stays the lower-level path. - README.md: note the mlx/tflite web UIs + MLX LoRA training after the hardware table, plus a Docs row for MLX training. Co-authored-by: “Cortexelus” <“emperorcj@gmail.com”>
1 parent 84ebaae commit 0385302

3 files changed

Lines changed: 57 additions & 0 deletions

File tree

README.md

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -248,11 +248,14 @@ Optimized inference runtimes are available under [optimized/](optimized) — pic
248248
| [optimized/tensorRT](optimized/tensorRT) | Linux + NVIDIA GPU | CUDA/TensorRT | `curl -LsSf https://raw.githubusercontent.com/Stability-AI/stable-audio-3/main/optimized/tensorRT/bootstrap.sh \| bash` |
249249

250250

251+
Beyond inference, **[optimized/mlx](optimized/mlx)** and **[optimized/tflite](optimized/tflite)** each ship a **web UI** (`./sa3-gradio`), and **optimized/mlx** also does **LoRA training** on Apple Silicon (pure-MLX, no PyTorch) — see its [LoRA training](optimized/mlx/README.md#lora-training) section, or [underfit](https://github.com/dada-bots/underfit) for a full training dashboard built on it.
252+
251253
## Docs
252254

253255
| Guide | Description |
254256
|-------|-------------|
255257
| [Inference Methods](docs/workflows/inference.md) | Overview of inference modes (text-to-audio, inpainting, etc.) |
258+
| [MLX LoRA training](optimized/mlx/README.md#lora-training) | Finetune on Apple Silicon (pure-MLX); powers underfit's Mac backend |
256259
| [LoRA Training](docs/workflows/lora.md) | Fine-tune with LoRA: setup, training loop, and checkpointing |
257260
| [Autoencoder Workflows](docs/workflows/autoencoder.md) | Encode and decode audio with the VAE directly |
258261
| [Prompting Guide](docs/guides/prompting.md) | Prompt and control signal reference |

optimized/README.md

Lines changed: 46 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,46 @@
1+
# Optimized runtimes
2+
3+
Platform-specific runtimes for **Stable Audio 3** — each a self-contained,
4+
minimal-dependency way to run SA3 on a given accelerator. Pick by hardware:
5+
6+
| Runtime | Platforms | Accelerator | Inference | Web UI | Training |
7+
|---|---|---|---|---|---|
8+
| **[mlx](mlx)** | Apple Silicon Macs | Metal GPU (MLX) ||`./sa3-gradio` | ✅ LoRA (pure-MLX) |
9+
| **[tflite](tflite)** | macOS / Linux / Windows, x86 & ARM | CPU (LiteRT / XNNPACK) ||`./sa3-gradio` | LoRA *inference* only (weight-patching) |
10+
| **[tensorRT](tensorRT)** | Linux + NVIDIA GPU | CUDA / TensorRT ||||
11+
12+
All three read the same weights from
13+
[`stabilityai/stable-audio-3-optimized`](https://huggingface.co/stabilityai/stable-audio-3-optimized)
14+
(auto-downloaded on first use) and cover text-to-audio, audio-to-audio,
15+
inpainting, and CFG / negative-prompt guidance.
16+
17+
## One-liners
18+
19+
```bash
20+
# Apple Silicon (Metal GPU)
21+
curl -LsSf https://raw.githubusercontent.com/Stability-AI/stable-audio-3/main/optimized/mlx/bootstrap.sh | bash
22+
23+
# Any CPU (macOS / Linux / Windows — needs Git Bash or WSL on Windows for curl|bash)
24+
curl -LsSf https://raw.githubusercontent.com/Stability-AI/stable-audio-3/main/optimized/tflite/bootstrap.sh | bash
25+
26+
# Linux + NVIDIA
27+
curl -LsSf https://raw.githubusercontent.com/Stability-AI/stable-audio-3/main/optimized/tensorRT/bootstrap.sh | bash
28+
```
29+
30+
## Highlights
31+
32+
- **Web UI** — the **mlx** and **tflite** runtimes each ship a gradio app
33+
(`./sa3-gradio`) with every generation mode wired (text-to-audio, CFG /
34+
negative prompt, audio-to-audio, inpainting), tinted mel-spectrogram
35+
previews, model / precision hot-swap, and LoRA support.
36+
- **LoRA training** — the **mlx** runtime has a complete pure-MLX LoRA trainer
37+
(pre-encode → train → generate) that matches
38+
[underfit](https://github.com/dada-bots/underfit)'s conventions and
39+
checkpoint format, so it doubles as underfit's Apple-Silicon backend. See
40+
[mlx → LoRA training](mlx/README.md#lora-training). For a full training
41+
**dashboard** on a Mac, use underfit.
42+
- **LoRA inference** — every runtime loads `.safetensors` adapters (mlx/tflite
43+
merge or patch them into the graph; per-adapter strength + sampling-step
44+
gating on mlx).
45+
46+
See each runtime's `README.md` for install, usage, flags, and benchmarks.

optimized/mlx/README.md

Lines changed: 8 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -200,6 +200,14 @@ Apple-Silicon backend). Three steps: **pre-encode → train → generate**. See
200200
`TRAINING_CONVENTIONS.md` for the complete convention inventory and the
201201
torch(MPS)-vs-MLX forward+backward parity results.
202202

203+
> **Want a UI instead of the CLI? Use [underfit](https://github.com/dada-bots/underfit).**
204+
> It's a full LoRA-training dashboard that drives *this* MLX trainer as its
205+
> Apple-Silicon backend — dataset scanning/encoding, live loss + loss-by-timestep
206+
> curves, demo MP3s with spectrograms, per-checkpoint gradio launch, and
207+
> one-click model downloads. On a Mac it's the recommended way to train; follow
208+
> underfit's *Apple-Silicon quickstart*. The commands below are the lower-level
209+
> path — good for scripting, CI, or headless runs.
210+
203211
**1. Pre-encode** your audio to SAME latents (once, offline — torch-free):
204212

205213
```bash

0 commit comments

Comments
 (0)