|
| 1 | +# Unsloth Training Plugin |
| 2 | + |
| 3 | +Fine-tune LLMs with [Unsloth](https://unsloth.ai/) using GRPO (reinforcement learning) or SFT (supervised fine-tuning). |
| 4 | + |
| 5 | +## Install |
| 6 | + |
| 7 | +```bash |
| 8 | +/plugin install unsloth-training@duyet-claude-plugins |
| 9 | +``` |
| 10 | + |
| 11 | +Or via [skills.sh](https://skills.sh): |
| 12 | +```bash |
| 13 | +npx skills add duyet/claude-plugins/unsloth-training |
| 14 | +``` |
| 15 | + |
| 16 | +## Features |
| 17 | + |
| 18 | +- **GRPO** — RL with reward functions (no labeled outputs needed) |
| 19 | +- **SFT** — Supervised fine-tuning with input/output pairs |
| 20 | +- **Vision** — VLM fine-tuning (Qwen3-VL, Gemma3, Llama 3.2 Vision) |
| 21 | +- **FP8 Training** — 60% less VRAM, 1.4x faster (RTX 40+, H100) |
| 22 | +- **Packing** — 2-5x speedup for mixed-length data |
| 23 | +- **MLX** — Apple Silicon training via mlx-tune |
| 24 | +- **Export** — GGUF, Ollama, vLLM, LM Studio, SGLang |
| 25 | +- **Dataset Prep** — ChatML/ShareGPT/Alpaca formats, synthetic data generation |
| 26 | +- **Mobile** — QAT + ExecuTorch for iOS/Android deployment |
| 27 | + |
| 28 | +## Skills |
| 29 | + |
| 30 | +| Skill | Description | |
| 31 | +|-------|-------------| |
| 32 | +| `unsloth-training` | Core training guide with GRPO, SFT, reward design, model selection, export | |
| 33 | + |
| 34 | +## Reference Docs |
| 35 | + |
| 36 | +| Reference | Description | |
| 37 | +|-----------|-------------| |
| 38 | +| `installation.md` | pip/uv install, CUDA versions, venv, Colab | |
| 39 | +| `datasets-guide.md` | Dataset formats, chat templates, synthetic data | |
| 40 | +| `environment-flags.md` | Unsloth env flags (RETURN_LOGITS, COMPILE_DISABLE, etc.) | |
| 41 | +| `mlx-training.md` | Apple Silicon training with mlx-tune | |
| 42 | +| `fp8-training.md` | FP8 setup, VRAM savings | |
| 43 | +| `reward-design.md` | Reward function patterns for GRPO | |
| 44 | +| `domain-examples.md` | Voice AI, Sales Agent, Support examples | |
| 45 | +| `hyperparameters.md` | GRPOConfig/SFTConfig reference | |
| 46 | +| `export-formats.md` | GGUF, Ollama, vLLM, Dynamic 2.0 | |
| 47 | +| `advanced-training.md` | 500K context, packing, checkpoints | |
| 48 | +| `vision-training.md` | VLM fine-tuning | |
| 49 | +| `mobile-deployment.md` | QAT, ExecuTorch, iOS/Android | |
| 50 | +| `deployment.md` | Docker, vLLM, LoRA hot-swap, SGLang | |
| 51 | +| `troubleshooting.md` | Common fixes | |
| 52 | + |
| 53 | +## Usage Examples |
| 54 | + |
| 55 | +``` |
| 56 | +# Natural language triggers |
| 57 | +"fine-tune Qwen3.5 for classification" |
| 58 | +"train with GRPO and reward functions" |
| 59 | +"prepare dataset in ChatML format" |
| 60 | +"install unsloth on Colab" |
| 61 | +"train on Apple Silicon with MLX" |
| 62 | +"export model to GGUF for Ollama" |
| 63 | +``` |
| 64 | + |
| 65 | +## Quick Start |
| 66 | + |
| 67 | +### SFT |
| 68 | +```python |
| 69 | +from unsloth import FastLanguageModel |
| 70 | +from trl import SFTTrainer, SFTConfig |
| 71 | + |
| 72 | +model, tokenizer = FastLanguageModel.from_pretrained( |
| 73 | + model_name="unsloth/Qwen3.5-4B", |
| 74 | + max_seq_length=512, |
| 75 | + load_in_4bit=False, |
| 76 | + load_in_16bit=True, |
| 77 | +) |
| 78 | +model = FastLanguageModel.get_peft_model(model, r=16) |
| 79 | + |
| 80 | +trainer = SFTTrainer( |
| 81 | + model=model, |
| 82 | + train_dataset=dataset, |
| 83 | + processing_class=tokenizer, |
| 84 | + args=SFTConfig( |
| 85 | + per_device_train_batch_size=2, |
| 86 | + num_train_epochs=3, |
| 87 | + learning_rate=2e-4, |
| 88 | + packing=True, |
| 89 | + ), |
| 90 | +) |
| 91 | +trainer.train() |
| 92 | +model.save_pretrained_gguf("model", tokenizer, quantization_method="q4_k_m") |
| 93 | +``` |
| 94 | + |
| 95 | +### GRPO |
| 96 | +```python |
| 97 | +from trl import GRPOConfig, GRPOTrainer |
| 98 | + |
| 99 | +def correctness_reward(completions, answer, **kwargs): |
| 100 | + return [2.0 if extract_answer(c) == a else 0.0 |
| 101 | + for c, a in zip(completions, answer)] |
| 102 | + |
| 103 | +trainer = GRPOTrainer( |
| 104 | + model=model, |
| 105 | + args=GRPOConfig(num_generations=4, beta=0.04, learning_rate=5e-6), |
| 106 | + train_dataset=dataset, |
| 107 | + reward_funcs=[correctness_reward], |
| 108 | +) |
| 109 | +trainer.train() |
| 110 | +``` |
| 111 | + |
| 112 | +## License |
| 113 | + |
| 114 | +Apache-2.0 |
0 commit comments