Skip to content

Commit 120bd17

Browse files
author
Atome LM Team
committed
README: adapt pitch to honest framing
Replace "smallest LM that talks / no GPU no cloud no RAM / runs on a $2 MCU" hero with "reference implementation of a routed-ternary tiny LM with bit-exact Python <-> C99 inference, sized for MCU-class RAM budgets." Acknowledge prior art explicitly (BitNet b1.58 for ternary, Hymba/MossNet for hybrid SSM + sparse-attention routing, Super-Tiny-LM for byte tokenizer) and state directly that the contribution is integration, not architecture. Add explicit MCU status line: QEMU ARM parity passes; silicon bring-up is NOT done in this repository. Commercial integration (silicon bring-up, Secure Boot Pack, hardening) is sold separately. Rename "Frontier finding" -> "Reproducible result, narrow regime" and keep the 944K reversal up front instead of buried.
1 parent 2e60489 commit 120bd17

1 file changed

Lines changed: 34 additions & 21 deletions

File tree

README.md

Lines changed: 34 additions & 21 deletions
Original file line numberDiff line numberDiff line change
@@ -1,31 +1,44 @@
11
# Atome LM
22

3-
> **The smallest language model that talks.**
4-
> **No GPU. No cloud. No RAM.** Runs on a $2 microcontroller.
5-
6-
A tiny ternary language model designed for the Atome C99 inference engine. Three-pathway router architecture, byte-level tokenizer, integer-arithmetic forward pass, ships in under 100 KB of flash at default config — small enough for a Cortex-M0, an RP2040, an ESP32-C3.
7-
8-
**Try it now:** [live browser demo at atomelm.com](https://atomelm.com/demo.html) (no install, no signup).
9-
10-
**Get the kit:** training code, C engine, MCU firmware, benchmarks, paper — all in this repository, released under the [Apache 2.0 License](LICENSE). Train your own checkpoint with `scripts/train_demo.py` in ~30 min on a CPU.
11-
12-
Trained model weights are not part of this open-source release. Commercial integration support — MCU bring-up, the Atome Secure Boot Pack, per-platform hardening — is available separately at [atomelm.com](https://atomelm.com).
3+
> A reference implementation of a routed-ternary tiny language model with a
4+
> bit-exact Python ↔ C99 inference engine, sized for microcontroller-class RAM
5+
> budgets.
6+
7+
60K-default-parameter LM combining three known ideas into one open kit:
8+
ternary weights (after [BitNet b1.58](https://arxiv.org/abs/2402.17764)),
9+
a hybrid SSM + sparse-attention + local-conv block routed per token
10+
(after [Hymba](https://arxiv.org/abs/2411.13676) and
11+
[MossNet](https://arxiv.org/abs/2510.26182)),
12+
and a byte tokenizer at super-tiny scale
13+
(after [Guertler 2024](https://arxiv.org/abs/2405.14159)).
14+
**The contribution is integration, not architecture**: a complete train →
15+
ternary export → base-3 packing → C99 inference path, with bit-exact
16+
Python ↔ C parity enforced by tests.
17+
18+
**Get the kit:** training code, C engine, benchmarks, paper — all in this
19+
repository, released under the [Apache 2.0 License](LICENSE). Train your
20+
own checkpoint with `scripts/train_demo.py` in ~30 min on a CPU.
21+
22+
**MCU status:** QEMU ARM (Cortex-M3, MPS2-AN385) parity passes to FP32
23+
epsilon. **Silicon bring-up is not done in this repository.** For real
24+
device deployment we sell integration — silicon bring-up, the Atome Secure
25+
Boot Pack, per-platform hardening — at [atomelm.com](https://atomelm.com).
26+
Trained model weights are not part of this open-source release.
1327

1428
---
1529

16-
## Frontier finding
30+
## Reproducible result, narrow regime
1731

18-
At fixed parameter count, Atome's 3-pathway ternary architecture beats a
19-
vanilla GPT-FP32 baseline by **22 % on perplexity** (6.31 vs 8.12 ppl)
20-
while using **16× less disk** (15.1 KB vs 237.5 KB). At fixed flash
21-
budget, Atome wins by **52 %** (6.31 vs 13.10 ppl). 3,000 steps on
22-
TinyStories, single seed; full reproduction in [FRONTIER.md](FRONTIER.md).
32+
On TinyStories, 3000 steps, single seed: at fixed parameter count Atome's
33+
routed-ternary block reaches **6.31 ppl vs 8.12** for a vanilla GPT-FP32
34+
baseline (−22 %); at fixed flash budget **6.31 vs 13.10** (−52 %). Disk
35+
footprint is 16× smaller at param-match (15.1 KB vs 237.5 KB).
2336

24-
**This win holds at the ~60 K-parameter MCU regime and only there.** In a
25-
944 K-parameter scale-up A/B the vanilla FP32 baseline reverses it (wins by
26-
~11 %) — Atome's bet is deliberately the sub-1M, MCU-class regime. The full
27-
honest reading, reversal included, is in [`FRONTIER.md`](FRONTIER.md) and
28-
[`HONEST_RESULTS.md`](HONEST_RESULTS.md).
37+
**The result reverses at 944 K parameters**, where the vanilla FP32 baseline
38+
wins by ~11 %. Atome's bet is deliberately the sub-1M, MCU-class regime;
39+
above it ternary's capacity ceiling closes the gap and overruns it. Full
40+
reproduction in [`FRONTIER.md`](FRONTIER.md), full honest reading including
41+
the reversal in [`HONEST_RESULTS.md`](HONEST_RESULTS.md).
2942

3043
## Why
3144

0 commit comments

Comments
 (0)