|
1 | 1 | # torchwright |
2 | 2 |
|
3 | 3 | torchwright is a compiler that transforms computation graphs into the weights of a |
4 | | -transformer. The output is a standard decoder-only transformer — causal softmax |
5 | | -attention, rotary position embeddings, RMSNorm, a KV cache — and its weights are |
6 | | -emitted by the compiler, not trained. Compiled models are packaged, by default, |
7 | | -as ordinary `transformers` checkpoints in the Phi-3 architecture: |
8 | | -`AutoModelForCausalLM` loads them with no custom code and no `trust_remote_code`. |
| 4 | +transformer. It treats a transformer as a fixed computational substrate that can |
| 5 | +be programmed: the compiler sets the weights directly, without any training, so |
| 6 | +that a standard decoder-only transformer — causal softmax attention, rotary |
| 7 | +position embeddings, RMSNorm, a KV cache — executes the source graph. Compiled |
| 8 | +models are packaged, by default, as ordinary `transformers` checkpoints in the |
| 9 | +Phi-3 architecture: `AutoModelForCausalLM` loads them with no custom code and no |
| 10 | +`trust_remote_code`. |
9 | 11 |
|
10 | 12 | ## Example |
11 | 13 |
|
@@ -55,6 +57,16 @@ constructs the same ops from gated SwiGLU sublayers, and is what the Phi-3 |
55 | 57 | output uses. A graph imports from exactly one — the compiler rejects a graph |
56 | 58 | that mixes them. |
57 | 59 |
|
| 60 | +The compiled computation lives in the residual stream — the fixed-width vector a |
| 61 | +transformer carries between layers. Every value the graph computes owns a set of |
| 62 | +columns there for as long as anything downstream still needs it; columns need not |
| 63 | +be contiguous, and concatenation is logical rather than physical. A sublayer can |
| 64 | +only *add* to the stream, and that one fact drives the compilation scheme: a new |
| 65 | +value lands in zeroed columns (0 + new = new), a value nothing needs anymore is |
| 66 | +cancelled by writing its negation (v + (−v) = 0), which frees its columns for |
| 67 | +reuse, and the graph's additions are implemented on the skip connection itself, |
| 68 | +costing no extra columns. |
| 69 | + |
58 | 70 | Every op compiles down to concrete weights — attention heads, rows of MLP |
59 | 71 | sublayers. A scheduler assigns every node to a layer: `optimize=0`, the default, |
60 | 72 | places nodes with a heuristic, and `optimize=1`–`3` hand placement to a |
|
0 commit comments