|
8 | 8 | [](https://pytorch.org) |
9 | 9 |
|
10 | 10 | > **A second-order PyTorch optimizer that delivers Shampoo-quality preconditioned gradients at near-AdamW memory and throughput cost.** |
11 | | -> Drop-in replacement for `AdamW`. One-line change. Real gains. |
| 11 | +> Drop-in replacement for `AdamW`. One-line change. Real gains. |
| 12 | +> **Now available on PyPI:** `pip install scao` |
| 13 | +
|
| 14 | +--- |
| 15 | + |
| 16 | +## 🚀 Support the Research |
| 17 | + |
| 18 | +If you have endorsement rights on arXiv for **cs.LG** (Machine Learning), please consider endorsing our paper to help us share this work with the community: |
| 19 | + |
| 20 | +👉 **[Endorse SCAO on arXiv](https://arxiv.org/auth/endorse?x=X3VJ88)** |
| 21 | + |
12 | 22 |
|
13 | 23 | --- |
14 | 24 |
|
|
30 | 40 |
|
31 | 41 | ### Objection 3 — "It's lab code. Not suitable for the real world." |
32 | 42 |
|
33 | | -**Test:** Eliminated every dependency on PyTorch or Hugging Face internals ([`examples/scao.py`](examples/scao.py)) |
| 43 | +**Test:** SCAO is now a professional Python package, installable via PyPI ([`pip install scao`](https://pypi.org/project/scao/)). |
34 | 44 |
|
35 | | -**Result:** SCAO is now a **single file** — a true drop-in replacement. Running natively on Windows with no cloud setup, the loss dropped from **4.536 → 3.307 in under 4 minutes**. The model learned real-world context: *"The secret to a good software architecture is its openness."* |
| 45 | +**Result:** SCAO has moved from a research script to a production-ready package. It's a true drop-in replacement for AdamW. Running natively on Windows with no cloud setup, the loss dropped from **4.536 → 3.307 in under 4 minutes**. The model learned real-world context: *"The secret to a good software architecture is its openness."* |
36 | 46 |
|
37 | 47 | --- |
38 | 48 |
|
39 | 49 | ## Table of Contents |
40 | 50 |
|
| 51 | +- [🚀 Support the Research](#-support-the-research) |
41 | 52 | 1. [The Problem](#1-the-problem) |
42 | 53 | 2. [SCAO's Solution](#2-scaos-solution) |
43 | 54 | 3. [Algorithm](#3-algorithm) |
@@ -643,7 +654,6 @@ scao/ # Core library |
643 | 654 | └── setup.py # nvcc build (sm_70/75/80/86/89/90) |
644 | 655 |
|
645 | 656 | examples/ # Self-contained runnable examples |
646 | | -├── scao.py # Standalone single-file SCAO (no library install needed) |
647 | 657 | ├── train_local.py # Fine-tune GPT-2 125M with SCAO + LoRA (<8 GB VRAM) |
648 | 658 | ├── train_1m.py # Full fine-tuning throughput benchmark on TinyStories-1M |
649 | 659 | └── inference.py # Load LoRA checkpoint and generate text |
|
0 commit comments