|
1 | 1 | # tinygrad Port Feedback |
2 | 2 |
|
3 | | -These items came from the tinygrad RSScript/modern-c port. They are not abstract |
4 | | -language wish-list items: each one removes a current source-port workaround or |
5 | | -manual wrapper. |
| 3 | +Remaining items from the tinygrad RSScript/modern-c port — each removes a real |
| 4 | +source-port workaround. Completed items are deleted as they land; see git history |
| 5 | +for their write-ups. Done so far: generated namespace isolation (+ `use … as` |
| 6 | +aliasing, qualified `module.fn`/`.Type`/value access, `use module.*` glob, |
| 7 | +qualified variant patterns), pattern matching (variants/options/constants/tuples/ |
| 8 | +lists), tuples & destructuring, `#lower_name` escape hatch, source-qualified |
| 9 | +symbol inventory, type-associated constants & static methods, value-semantics |
| 10 | +clone/derives, Option ergonomics (`?`-on-Option + combinators), and default |
| 11 | +parameters (Copy and non-Copy). |
6 | 12 |
|
7 | 13 | ## Must unblock awkward valid ports |
8 | 14 |
|
9 | | -- [x] **Generated namespace isolation.** Preserve source/module/type/member |
10 | | - identity through checking and lowering, then emit globally unique Rust symbols. |
11 | | - _Why:_ valid tinygrad names such as `helpers.count`, `device.count`, |
12 | | - `helpers.T`, and `tensor.T` currently collide because RSS lowers too many |
13 | | - things into one Rust item namespace. The port has to invent names like |
14 | | - `helpers_count` and map them back in tooling. |
15 | | - _Acceptance:_ two modules can define the same final name; a module-level value, |
16 | | - type alias, struct, method, and generated helper with related names cannot |
17 | | - collide after Rust lowering; diagnostics report the RSS source symbol, not only |
18 | | - the lowered Rust name. |
19 | | - _Done:_ a file's `module a.b` declaration gives its top-level symbols a module |
20 | | - identity. A scope-aware pass (`syntax::module_isolation`) runs on the assembled |
21 | | - program before HIR construction in every consumer (checker, register VM, Rust |
22 | | - backend) and: |
23 | | - (1) rewrites each module-scoped declaration (functions incl. `Type.method`, |
24 | | - types, sums, aliases, consts) to a globally unique `a_b__name` symbol — so two |
25 | | - modules can declare the same source name without colliding (no more RS0005 / |
26 | | - rustc E0428); the executable entry point `main` stays global; |
27 | | - (2) resolves every reference module-aware — local module, then `use` import, |
28 | | - then `module.fn` qualified — honoring shadowing by locals/params/fields, across |
29 | | - expressions, types, patterns, and protocol impls; |
30 | | - (3) is a complete no-op for files without a `module` declaration (the root |
31 | | - namespace), so single-module and module-less programs lower exactly as before; |
32 | | - (4) demangles diagnostics so they report the source symbol (`helpers.count`), |
33 | | - not the lowered Rust name. |
34 | | - Verified at VM/compiled parity with two modules sharing `count` and `Box` |
35 | | - (`module_isolation_lets_two_modules_share_a_symbol_name` plus the |
36 | | - `syntax::module_isolation` unit tests). The `#lower_name("...")` escape hatch |
37 | | - remains for pinning a specific backend symbol at FFI/autogen boundaries. |
38 | | - |
39 | | -- [ ] **Stable source-qualified symbol identity.** Store and expose a symbol's |
40 | | - module path, source qualified name, kind, visibility, and lowered backend name. |
41 | | - _Why:_ port tooling should compare `helpers.py::count` to |
42 | | - `helpers.rss::count`, not guess from bare names. |
43 | | - _Acceptance:_ RSS can emit a checked symbol inventory suitable for portman: |
44 | | - `module`, `qualname`, `kind`, `source_span`, `lowered_name`. |
45 | | - |
46 | | -- [ ] **Class/static member support.** Model class attributes, associated |
47 | | - constants, and static methods directly. |
48 | | - _Why:_ tinygrad uses symbols such as `Tensor.train`, `Device.DEFAULT`, |
49 | | - `dtypes.float`, `UOp.const`, and `UPat.var` as class/static members. Porting |
50 | | - them as free functions or dummy globals loses structure and creates name |
51 | | - conflicts. |
52 | | - _Acceptance:_ RSS supports declaring and lowering type-associated values and |
53 | | - methods without requiring separate global wrappers. |
54 | | - |
55 | | -- [ ] **Type aliases and generic aliases without value placeholders.** Add |
56 | | - source-visible type alias declarations that do not create value namespace |
57 | | - entries unless explicitly requested. |
58 | | - _Why:_ Python `TypeVar` and alias-like symbols currently become dummy constants |
59 | | - in the port just so coverage tooling can see them. |
60 | | - _Acceptance:_ aliases can be generic, can reference imported types, and appear |
61 | | - in the symbol inventory as type-level symbols. |
| 15 | +- [ ] **Generic type aliases.** Basic `type X = Y` aliases work; add *generic* |
| 16 | + aliases (`type Pair<T> = ...`) that can reference imported types and appear in |
| 17 | + the symbol inventory as type-level symbols, without creating value-namespace |
| 18 | + placeholders. |
| 19 | + _Why:_ Python `TypeVar`/generic-alias symbols otherwise become dummy constants. |
62 | 20 |
|
63 | 21 | - [ ] **Method/property lowering ergonomics.** Provide a simple way to model |
64 | 22 | Python-style getter properties and method-like computed fields. |
65 | | - _Why:_ tinygrad has many small property methods where the implementation is |
66 | | - simple but the RSS surface needs repetitive wrappers. |
| 23 | + _Why:_ tinygrad has many small property methods where the body is simple but the |
| 24 | + RSS surface needs repetitive wrappers. |
67 | 25 | _Acceptance:_ getter-style members lower predictably, can borrow `self`, and |
68 | 26 | preserve member identity for diagnostics and inventory. |
69 | 27 |
|
70 | 28 | ## Reduce manual translation volume |
71 | 29 |
|
72 | | -- [ ] **Container operation coverage.** Make common `List`, `Map`, `Set`, |
73 | | - `Bytes`, `Buffer`, tuple, and optional operations straightforward and |
74 | | - consistently checked. |
75 | | - _Why:_ tinygrad code is dense with `len`, `append`, `pop`, indexing, slicing, |
76 | | - membership, iteration, and clone/copy patterns. Every missing builtin creates a |
| 30 | +- [ ] **Container operation coverage.** Fill remaining `List`/`Map`/`Set`/ |
| 31 | + `Bytes`/`Buffer`/tuple/optional gaps (`len`, `append`, `pop`, index, slice, |
| 32 | + membership, iteration, clone/copy). |
| 33 | + _Why:_ tinygrad code is dense with these; every missing builtin becomes a |
77 | 34 | hand-written helper. |
78 | | - _Acceptance:_ supported operations either compile and lower to idiomatic Rust |
79 | | - or fail with an error that names the unsupported operation and type. |
80 | | - |
81 | | -- [ ] **Clone/copy consistency for builtin value types.** Continue the |
82 | | - `List`/`Bytes`/`Buffer` `.clone()` direction and make the checker/lowerer agree |
83 | | - exactly on every builtin clone target. |
84 | | - _Why:_ the tinygrad port previously hit Rust `E0425` because the checker |
85 | | - accepted a builtin clone that lowering emitted as a missing helper. |
86 | | - _Acceptance:_ checker-approved builtin clones always lower to valid Rust, and |
87 | | - checker-rejected clones state the supported alternatives. |
88 | | - |
89 | | -- [ ] **Option/null ergonomics.** Add concise checked forms for option tests, |
90 | | - unwrap-with-default, early return on none, and optional field access. |
91 | | - _Why:_ tinygrad has many maybe-present values; verbose option plumbing makes |
92 | | - direct ports drift away from the source. |
93 | | - _Acceptance:_ common option patterns require no custom helpers and preserve |
94 | | - ownership/borrow rules clearly. |
95 | | - |
96 | | -- [x] **Pattern matching over variants, options, constants, tuples, and lists.** |
97 | | - _Why:_ the UOp/UPat rewrite system is pattern-heavy. Lowering it as nested |
98 | | - conditionals would be slow to write and hard to audit. |
99 | | - _Acceptance:_ match arms can bind values, match enum/variant tags, constants, |
100 | | - tuples, and optional values; exhaustiveness or explicit fallback is checked. |
101 | | - _Done:_ variants, options, constants, tuple, and list patterns now match at |
102 | | - VM/compiled parity. List slice patterns support `[]`, fixed-length `[a, b]`, |
103 | | - head/rest `[first, ..rest]`, tail `[..init, last]`, and middle-rest |
104 | | - `[a, ..mid, z]` (element bindings come out owned; the rest binding is a |
105 | | - `List<T>`); the VM lowers them to a length test plus `ListGet`/`List.slice`, |
106 | | - the compiled backend to native Rust slice patterns with owned rebindings. |
107 | | - Exhaustiveness understands list lengths (`[]` + `[x, ..rest]` is exhaustive). |
108 | | - Plain (non-variant) generic struct patterns and generic field/binding types |
109 | | - resolve their element types from the scrutinee's arguments. See |
110 | | - `tests/fixtures/pass/list_patterns.rss` and the `parity_list_match` test. |
111 | | - |
112 | | -- [ ] **Callable and limited closure support.** Support function values and |
113 | | - simple captures well enough for callbacks and rewrite rules. |
114 | | - _Why:_ tinygrad uses lambdas/callback-like matchers. Without callable support, |
115 | | - the port must manually defunctionalize logic into many structs. |
| 35 | + _Acceptance:_ supported ops compile and lower to idiomatic Rust, or fail with an |
| 36 | + error naming the unsupported operation and type. |
| 37 | + |
| 38 | +- [ ] **Callable and limited closure support.** Beyond `noescape` callbacks: pass |
| 39 | + *named functions* as values and support richer checked captures for callbacks |
| 40 | + and rewrite rules. |
| 41 | + _Why:_ tinygrad uses lambdas/callback matchers; without this the port must |
| 42 | + defunctionalize logic into many structs. |
116 | 43 | _Acceptance:_ RSS can pass named functions and simple closures to higher-order |
117 | 44 | helpers with checked capture ownership. |
118 | 45 |
|
119 | | -- [ ] **Default arguments and construction helpers.** Provide a first-class way |
120 | | - to express common defaulted constructor/helper parameters. |
121 | | - _Why:_ tinygrad APIs use many Python defaults. Manual overload/wrapper sets |
122 | | - inflate the port. |
123 | | - _Acceptance:_ defaulted parameters lower deterministically and appear in docs |
124 | | - or symbol inventory without changing the call ABI unexpectedly. |
125 | | - |
126 | | -- [x] **Tuple and destructuring ergonomics.** Make multiple-return and unpacking |
127 | | - patterns concise. |
128 | | - _Why:_ tinygrad frequently returns and unpacks grouped values. |
129 | | - _Acceptance:_ tuple literals, tuple returns, and local destructuring work with |
130 | | - ownership and borrowing tracked normally. |
131 | | - _Done:_ tuples desugar to synthetic `__TupleN` generic structs at parse time — |
132 | | - literals `(a, b)`, types `(Int, String)`, `.itemN` access, tuple patterns in |
133 | | - `match`, and `let (a, b) = expr` destructuring (`_` skips an element), all at |
134 | | - VM/compiled parity and round-tripped by the formatter. See |
135 | | - `tests/fixtures/pass/tuples.rss` and the `parity_tuple_*` parity tests. |
| 46 | +- [ ] **Construction helpers.** (Default parameters — Copy and non-Copy — are |
| 47 | + done.) Provide first-class defaulted-constructor / builder-style helpers so |
| 48 | + Python-default-heavy constructor APIs don't inflate into overload/wrapper sets. |
| 49 | + _Acceptance:_ defaulted construction lowers deterministically and appears in the |
| 50 | + symbol inventory without changing the call ABI unexpectedly. |
136 | 51 |
|
137 | 52 | - [ ] **String and bytes utility coverage.** Fill gaps for split/join/search, |
138 | | - prefix/suffix checks, formatting, byte conversion, and cheap slicing where |
139 | | - supported. |
| 53 | + prefix/suffix checks, formatting, byte conversion, and cheap slicing. |
140 | 54 | _Why:_ device/runtime/autogen and file-path code are string/bytes-heavy. |
141 | | - _Acceptance:_ common string/bytes operations compile without local helper |
142 | | - shims, or fail with precise unsupported-operation diagnostics. |
| 55 | + _Acceptance:_ common string/bytes operations compile without local helper shims, |
| 56 | + or fail with precise unsupported-operation diagnostics. |
143 | 57 |
|
144 | 58 | ## Tooling and diagnostics |
145 | 59 |
|
146 | | -- [ ] **RSS-to-Rust diagnostic provenance.** Carry source spans and symbol |
147 | | - identity into generated Rust and back through compiler errors. |
| 60 | +- [ ] **RSS-to-Rust diagnostic provenance.** Source maps exist; carry the source |
| 61 | + span, source symbol, and lowered Rust symbol all the way through a *backend* |
| 62 | + Rust error. |
148 | 63 | _Why:_ when Rust fails, the port currently has to infer which RSS declaration |
149 | 64 | produced the bad item. |
150 | 65 | _Acceptance:_ a Rust backend error reports the RSS file, source span, source |
151 | 66 | symbol, and lowered Rust symbol. |
152 | 67 |
|
153 | | -- [x] **Explicit lowered-name escape hatch.** Add an attribute for rare cases |
154 | | - where a declaration needs a pinned backend name. |
155 | | - _Why:_ useful during compiler transition and for FFI/autogen boundaries. |
156 | | - _Acceptance:_ something like `#[lower_name = "helpers__count"]` is checked for |
157 | | - uniqueness and reflected in the symbol inventory. |
158 | | - _Done:_ functions accept `#lower_name("...")` (following the existing |
159 | | - `#deprecated("...")` attribute convention). The pin is validated (RS0035: must |
160 | | - be a valid Rust identifier and unique across functions' final backend names), |
161 | | - honored by the lowerer at the definition and every call site (via a per-run |
162 | | - override consulted by the canonical name-lowering helpers), reflected in the |
163 | | - symbol inventory's `lowered_name`, and round-tripped by the formatter. See |
164 | | - `tests/fixtures/pass/lower_name_attribute.rss`, `fixtures/fail/lower-name-*`, |
165 | | - and `lower_name_pin_renames_definition_and_call_sites`. |
166 | | - |
167 | 68 | - [ ] **Generated/internal helper namespace.** Keep compiler-generated helpers |
168 | | - separate from user declarations. |
| 69 | + compiler-reserved/mangled so they never collide with user module/type/member |
| 70 | + names. |
169 | 71 | _Why:_ helper names should never consume source-level names or force a port to |
170 | | - avoid otherwise valid identifiers. |
171 | | - _Acceptance:_ generated helper names are always compiler-reserved/mangled and |
172 | | - never collide with user-visible module/type/member names. |
| 72 | + avoid otherwise valid identifiers. (Module isolation + `#lower_name` cover |
| 73 | + user-symbol collisions; this is the generated-helper side.) |
173 | 74 |
|
174 | | -- [ ] **External/FFI declaration ergonomics.** Make copied runtime/autogen and |
175 | | - device boundaries easy to declare without large wrapper files. |
176 | | - _Why:_ tinygrad runtime/autogen should mostly be copied or bound, not manually |
177 | | - reimplemented. |
| 75 | +- [ ] **External/FFI declaration ergonomics.** Declare copied runtime/autogen and |
| 76 | + device boundaries compactly — `native fn` exists, but make whole boundaries easy |
| 77 | + to bind without large wrapper files. |
178 | 78 | _Acceptance:_ external functions/types can be declared compactly, checked, and |
179 | 79 | included in the symbol inventory with provenance. |
0 commit comments