Skip to content

Commit c4bf1a4

Browse files
blackwell-systemsblackwell-systems
committed
feat: initial GCF Python implementation
Pure Python encoder/decoder for Graph Compact Format (GCF), a token-optimized wire format for LLM tool responses. Includes: - encode/decode for full payloads - Session-based deduplication (thread-safe) - Delta encoding for incremental delivery - 43 tests covering encode, decode, round-trip, session, and delta - Zero runtime dependencies, Python 3.9+
0 parents  commit c4bf1a4

17 files changed

Lines changed: 1625 additions & 0 deletions

‎.gitignore‎

Lines changed: 10 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,10 @@
1+
__pycache__/
2+
*.py[cod]
3+
*$py.class
4+
*.egg-info/
5+
dist/
6+
build/
7+
.venv/
8+
.pytest_cache/
9+
*.egg
10+
.eggs/

‎LICENSE‎

Lines changed: 21 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,21 @@
1+
MIT License
2+
3+
Copyright (c) 2026 Blackwell Systems
4+
5+
Permission is hereby granted, free of charge, to any person obtaining a copy
6+
of this software and associated documentation files (the "Software"), to deal
7+
in the Software without restriction, including without limitation the rights
8+
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
9+
copies of the Software, and to permit persons to whom the Software is
10+
furnished to do so, subject to the following conditions:
11+
12+
The above copyright notice and this permission notice shall be included in all
13+
copies or substantial portions of the Software.
14+
15+
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
16+
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
17+
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
18+
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
19+
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
20+
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
21+
SOFTWARE.

‎README.md‎

Lines changed: 148 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,148 @@
1+
# gcf-python
2+
3+
Python implementation of [GCF (Graph Compact Format)](https://github.com/blackwell-systems/gcf).
4+
5+
**84% fewer tokens than JSON. 32% fewer than TOON. 100% LLM comprehension accuracy at 500 symbols, where JSON fails.**
6+
7+
## Install
8+
9+
```
10+
pip install gcf-format
11+
```
12+
13+
Zero dependencies. Pure Python. Python 3.9+.
14+
15+
## Quick Start
16+
17+
```python
18+
from gcf import encode, Payload, Symbol, Edge
19+
20+
p = Payload(
21+
tool="context_for_task",
22+
token_budget=5000,
23+
tokens_used=1847,
24+
symbols=[
25+
Symbol(qualified_name="pkg.AuthMiddleware", kind="function", score=0.78, provenance="lsp_resolved", distance=0),
26+
Symbol(qualified_name="pkg.NewServer", kind="function", score=0.54, provenance="lsp_resolved", distance=1),
27+
],
28+
edges=[
29+
Edge(source="pkg.NewServer", target="pkg.AuthMiddleware", edge_type="calls"),
30+
],
31+
)
32+
33+
output = encode(p)
34+
```
35+
36+
Output:
37+
```
38+
GCF tool=context_for_task budget=5000 tokens=1847 symbols=2
39+
## targets
40+
@0 fn pkg.AuthMiddleware 0.78 lsp_resolved
41+
## related
42+
@1 fn pkg.NewServer 0.54 lsp_resolved
43+
## edges
44+
@0<@1 calls
45+
```
46+
47+
## Decode
48+
49+
```python
50+
from gcf import decode
51+
52+
p = decode(input_text)
53+
print(p.tool, len(p.symbols), "symbols", len(p.edges), "edges")
54+
```
55+
56+
## Session Deduplication
57+
58+
Track transmitted symbols across multiple tool responses. Previously-sent symbols become bare references instead of full declarations:
59+
60+
```python
61+
from gcf import encode_with_session, Session, Payload, Symbol
62+
63+
sess = Session()
64+
65+
out1 = encode_with_session(payload1, sess) # full declarations
66+
out2 = encode_with_session(payload2, sess) # reused symbols as "@N # previously transmitted"
67+
```
68+
69+
By the 5th call in a session: 92.7% token savings vs JSON.
70+
71+
## Delta Encoding
72+
73+
When the consumer already has a prior context pack, send only what changed:
74+
75+
```python
76+
from gcf import encode_delta, DeltaPayload, Symbol, Edge
77+
78+
delta = DeltaPayload(
79+
tool="context_for_task",
80+
base_root="aaa111",
81+
new_root="bbb222",
82+
removed=[Symbol(qualified_name="pkg.OldFunc", kind="function")],
83+
added=[Symbol(qualified_name="pkg.NewFunc", kind="function", score=0.85, provenance="rwr")],
84+
delta_tokens=30,
85+
full_tokens=200,
86+
)
87+
88+
output = encode_delta(delta)
89+
```
90+
91+
81.2% savings on re-queries where the pack changed slightly.
92+
93+
## API
94+
95+
| Function | Description |
96+
|----------|-------------|
97+
| `encode(p: Payload) -> str` | Encode a payload to GCF text |
98+
| `decode(input_text: str) -> Payload` | Parse GCF text back to a Payload |
99+
| `encode_with_session(p: Payload, s: Session) -> str` | Encode with session deduplication |
100+
| `encode_delta(d: DeltaPayload) -> str` | Encode a delta (added/removed only) |
101+
| `Session()` | Create a new session tracker (thread-safe) |
102+
103+
## Types
104+
105+
| Type | Purpose |
106+
|------|---------|
107+
| `Payload` | Full GCF payload: tool, budget, symbols, edges, pack root |
108+
| `Symbol` | Graph node: qualified name, kind, score, provenance, distance |
109+
| `Edge` | Directed relationship: source, target, edge type |
110+
| `DeltaPayload` | Diff between two packs: added/removed symbols and edges |
111+
| `Session` | Thread-safe tracker for multi-call deduplication |
112+
| `KIND_ABBREV` / `KIND_EXPAND` | Bidirectional kind abbreviation dicts |
113+
114+
## Comprehension Eval
115+
116+
Rigorous 3-way benchmark (GCF vs TOON vs JSON) at 500 symbols, 200 edges. Six structured extraction questions sent to an LLM:
117+
118+
| Format | Accuracy | Tokens | vs JSON |
119+
|--------|----------|--------|---------|
120+
| **GCF** | **100%** (6/6) | **11,090** | **79% fewer** |
121+
| TOON | 100% (6/6) | 16,378 | 69% fewer |
122+
| JSON | 66.7% (4/6) | 53,341 | baseline |
123+
124+
JSON failed on counting tasks. GCF and TOON both achieved perfect accuracy. GCF does it in 32% fewer tokens.
125+
126+
## Token Efficiency (TOON's Own Benchmark)
127+
128+
Running [TOON's benchmark harness](https://github.com/blackwell-systems/toon/tree/gcf-comparison) with GCF inserted (their datasets, their tokenizer):
129+
130+
| Track | GCF | TOON | Result |
131+
|-------|-----|------|--------|
132+
| Mixed-structure (nested, semi-uniform) | 169,554 | 227,896 | **GCF 34% smaller** |
133+
| Flat-only (tabular) | 66,026 | 67,837 | **GCF 3% smaller** |
134+
| Semi-uniform event logs | 107,269 | 154,032 | **GCF 44% smaller** |
135+
136+
GCF wins on every dataset except deeply nested config (75 tokens on a 618-token payload). On semi-uniform data, GCF uses 44% fewer tokens than TOON.
137+
138+
Reproducible: [blackwell-systems/toon@gcf-comparison](https://github.com/blackwell-systems/toon/tree/gcf-comparison)
139+
140+
## Other Implementations
141+
142+
- **Go**: [github.com/blackwell-systems/gcf-go](https://github.com/blackwell-systems/gcf-go)
143+
- **TypeScript**: [github.com/blackwell-systems/gcf-typescript](https://github.com/blackwell-systems/gcf-typescript)
144+
- **Specification**: [github.com/blackwell-systems/gcf](https://github.com/blackwell-systems/gcf)
145+
146+
## License
147+
148+
MIT

‎pyproject.toml‎

Lines changed: 39 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,39 @@
1+
[build-system]
2+
requires = ["hatchling"]
3+
build-backend = "hatchling.build"
4+
5+
[project]
6+
name = "gcf-format"
7+
version = "0.1.0"
8+
description = "Python implementation of GCF (Graph Compact Format): token-optimized wire format for LLM tool responses"
9+
readme = "README.md"
10+
license = "MIT"
11+
requires-python = ">=3.9"
12+
authors = [
13+
{ name = "Blackwell Systems" },
14+
]
15+
keywords = ["gcf", "llm", "mcp", "token-efficient", "graph", "wire-format"]
16+
classifiers = [
17+
"Development Status :: 4 - Beta",
18+
"Intended Audience :: Developers",
19+
"License :: OSI Approved :: MIT License",
20+
"Programming Language :: Python :: 3",
21+
"Programming Language :: Python :: 3.9",
22+
"Programming Language :: Python :: 3.10",
23+
"Programming Language :: Python :: 3.11",
24+
"Programming Language :: Python :: 3.12",
25+
"Programming Language :: Python :: 3.13",
26+
"Topic :: Software Development :: Libraries",
27+
"Typing :: Typed",
28+
]
29+
30+
[project.urls]
31+
Homepage = "https://github.com/blackwell-systems/gcf-python"
32+
Specification = "https://github.com/blackwell-systems/gcf"
33+
"Go Implementation" = "https://github.com/blackwell-systems/gcf-go"
34+
35+
[tool.hatch.build.targets.wheel]
36+
packages = ["src/gcf"]
37+
38+
[tool.pytest.ini_options]
39+
testpaths = ["tests"]

‎src/gcf/__init__.py‎

Lines changed: 60 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,60 @@
1+
"""GCF (Graph Compact Format): token-optimized wire format for LLM tool responses.
2+
3+
84% fewer tokens than JSON. 32% fewer than TOON. 100% LLM comprehension accuracy.
4+
5+
Encode a payload:
6+
7+
from gcf import encode, Payload, Symbol
8+
9+
p = Payload(
10+
tool="context_for_task",
11+
token_budget=5000,
12+
tokens_used=1847,
13+
symbols=[Symbol(qualified_name="pkg.Func", kind="function", score=0.9, provenance="lsp_resolved")],
14+
)
15+
output = encode(p)
16+
17+
Decode a payload:
18+
19+
from gcf import decode
20+
p = decode(input_text)
21+
22+
Session deduplication:
23+
24+
from gcf import encode_with_session, Session
25+
sess = Session()
26+
out1 = encode_with_session(payload1, sess) # full declarations
27+
out2 = encode_with_session(payload2, sess) # reused symbols as bare refs
28+
29+
Delta encoding:
30+
31+
from gcf import encode_delta, DeltaPayload
32+
out = encode_delta(DeltaPayload(...))
33+
34+
Specification: https://github.com/blackwell-systems/gcf
35+
"""
36+
37+
from .constants import KIND_ABBREV, KIND_EXPAND
38+
from .decode import DecodeError, decode
39+
from .delta import encode_delta
40+
from .encode import encode
41+
from .session import Session, encode_with_session
42+
from .types import Components, DeltaPayload, Edge, Payload, Symbol
43+
44+
__all__ = [
45+
"Components",
46+
"DecodeError",
47+
"DeltaPayload",
48+
"Edge",
49+
"KIND_ABBREV",
50+
"KIND_EXPAND",
51+
"Payload",
52+
"Session",
53+
"Symbol",
54+
"decode",
55+
"encode",
56+
"encode_delta",
57+
"encode_with_session",
58+
]
59+
60+
__version__ = "0.1.0"

‎src/gcf/constants.py‎

Lines changed: 24 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,24 @@
1+
"""Kind abbreviation mappings for GCF encoding/decoding."""
2+
3+
# Maps full kind names to short GCF abbreviations.
4+
KIND_ABBREV: dict[str, str] = {
5+
"function": "fn",
6+
"type": "type",
7+
"method": "method",
8+
"interface": "iface",
9+
"var": "var",
10+
"const": "const",
11+
"resource": "resource",
12+
"table": "table",
13+
"class": "class",
14+
"selector": "selector",
15+
"field": "field",
16+
"route_handler": "route",
17+
"external": "ext",
18+
"file": "file",
19+
"package": "pkg",
20+
"service": "svc",
21+
}
22+
23+
# Maps short GCF abbreviations to full kind names.
24+
KIND_EXPAND: dict[str, str] = {v: k for k, v in KIND_ABBREV.items()}

0 commit comments

Comments
 (0)