Skip to content

Commit 6576300

Browse files
physicsrobclaude
andcommitted
README: adopt the substrate thesis and residual-stream paragraph
Two salvages from the pre-launch README: the opening now states the premise (a transformer is a fixed computational substrate that can be programmed by setting weights directly), and How-it-works gains a compressed residual-stream paragraph — column ownership and liveness, non-contiguous columns, logical concatenation, and the sublayers-can-only-add algebra (zeroed-column entry, negation cancel, additions on the skip connection). All claims verified against the current compiler. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
1 parent 5068efe commit 6576300

1 file changed

Lines changed: 17 additions & 5 deletions

File tree

README.md

Lines changed: 17 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -1,11 +1,13 @@
11
# torchwright
22

33
torchwright is a compiler that transforms computation graphs into the weights of a
4-
transformer. The output is a standard decoder-only transformer — causal softmax
5-
attention, rotary position embeddings, RMSNorm, a KV cache — and its weights are
6-
emitted by the compiler, not trained. Compiled models are packaged, by default,
7-
as ordinary `transformers` checkpoints in the Phi-3 architecture:
8-
`AutoModelForCausalLM` loads them with no custom code and no `trust_remote_code`.
4+
transformer. It treats a transformer as a fixed computational substrate that can
5+
be programmed: the compiler sets the weights directly, without any training, so
6+
that a standard decoder-only transformer — causal softmax attention, rotary
7+
position embeddings, RMSNorm, a KV cache — executes the source graph. Compiled
8+
models are packaged, by default, as ordinary `transformers` checkpoints in the
9+
Phi-3 architecture: `AutoModelForCausalLM` loads them with no custom code and no
10+
`trust_remote_code`.
911

1012
## Example
1113

@@ -55,6 +57,16 @@ constructs the same ops from gated SwiGLU sublayers, and is what the Phi-3
5557
output uses. A graph imports from exactly one — the compiler rejects a graph
5658
that mixes them.
5759

60+
The compiled computation lives in the residual stream — the fixed-width vector a
61+
transformer carries between layers. Every value the graph computes owns a set of
62+
columns there for as long as anything downstream still needs it; columns need not
63+
be contiguous, and concatenation is logical rather than physical. A sublayer can
64+
only *add* to the stream, and that one fact drives the compilation scheme: a new
65+
value lands in zeroed columns (0 + new = new), a value nothing needs anymore is
66+
cancelled by writing its negation (v + (−v) = 0), which frees its columns for
67+
reuse, and the graph's additions are implemented on the skip connection itself,
68+
costing no extra columns.
69+
5870
Every op compiles down to concrete weights — attention heads, rows of MLP
5971
sublayers. A scheduler assigns every node to a layer: `optimize=0`, the default,
6072
places nodes with a heuristic, and `optimize=1``3` hand placement to a

0 commit comments

Comments
 (0)