|
7 | 7 |
|
8 | 8 | Python implementation of [GCF (Graph Compact Format)](https://gcformat.com/) — the most token-efficient wire format for LLMs. A drop-in alternative to JSON and TOON for any structured data. |
9 | 9 |
|
10 | | -**79% fewer input tokens than JSON. 75% fewer output tokens. 52% smaller than TOON. 100% LLM comprehension at 500 symbols, where JSON fails at 66.7%.** |
| 10 | +**79% fewer input tokens than JSON. 75% fewer output tokens. 52% smaller than TOON. 100% LLM comprehension at 500 symbols, where JSON scores 76.9% and TOON scores 92.3%.** |
11 | 11 |
|
12 | 12 | Docs: [gcformat.com](https://gcformat.com/) · [Playground](https://gcformat.com/playground.html) · [GCF vs TOON](https://gcformat.com/guide/vs-toon.html) |
13 | 13 |
|
@@ -164,27 +164,27 @@ Works on dicts, lists, and primitives. Lists of uniform dicts get tabular rows. |
164 | 164 |
|
165 | 165 | ## Comprehension Eval |
166 | 166 |
|
167 | | -Rigorous 3-way benchmark (GCF vs TOON vs JSON) at 500 symbols, 200 edges. Six structured extraction questions sent to an LLM: |
| 167 | +Rigorous 3-way benchmark (GCF vs TOON vs JSON) at 500 symbols, 200 edges. 13 structured extraction questions sent to an LLM: |
168 | 168 |
|
169 | 169 | | Format | Accuracy | Tokens | vs JSON | |
170 | 170 | |--------|----------|--------|---------| |
171 | | -| **GCF** | **100%** (6/6) | **11,090** | **79% fewer** | |
172 | | -| TOON | 100% (6/6) | 16,378 | 69% fewer | |
173 | | -| JSON | 66.7% (4/6) | 53,341 | baseline | |
| 171 | +| **GCF** | **100%** (13/13) | **11,090** | **79% fewer** | |
| 172 | +| TOON | 92.3% (12/13) | 16,378 | 69% fewer | |
| 173 | +| JSON | 76.9% (10/13) | 53,341 | baseline | |
174 | 174 |
|
175 | | -JSON failed on counting tasks. GCF and TOON both achieved perfect accuracy. GCF does it in 32% fewer tokens. |
| 175 | +GCF is the only format with perfect accuracy at scale, at 32% fewer tokens than TOON. |
176 | 176 |
|
177 | 177 | ## Token Efficiency (TOON's Own Benchmark) |
178 | 178 |
|
179 | 179 | Running [TOON's benchmark harness](https://github.com/blackwell-systems/toon/tree/gcf-comparison) with GCF inserted (their datasets, their tokenizer): |
180 | 180 |
|
181 | 181 | | Track | GCF | TOON | Result | |
182 | 182 | |-------|-----|------|--------| |
183 | | -| Mixed-structure (nested, semi-uniform) | 169,554 | 227,896 | **GCF 34% smaller** | |
| 183 | +| Mixed-structure (nested, semi-uniform) | 170,367 | 227,896 | **GCF 34% smaller** | |
184 | 184 | | Flat-only (tabular) | 66,026 | 67,837 | **GCF 3% smaller** | |
185 | | -| Semi-uniform event logs | 107,269 | 154,032 | **GCF 44% smaller** | |
| 185 | +| Semi-uniform event logs | 108,158 | 154,032 | **GCF 42% smaller** | |
186 | 186 |
|
187 | | -GCF wins on every dataset except deeply nested config (75 tokens on a 618-token payload). On semi-uniform data, GCF uses 44% fewer tokens than TOON. |
| 187 | +GCF wins all 6 datasets. On semi-uniform data (the most common real-world pattern), GCF uses 42% fewer tokens than TOON. |
188 | 188 |
|
189 | 189 | Reproducible: [blackwell-systems/toon@gcf-comparison](https://github.com/blackwell-systems/toon/tree/gcf-comparison) |
190 | 190 |
|
|
0 commit comments