Skip to content

Commit 5ccba16

Browse files
committed
docs: added depth to deep dive in README
1 parent 88cc280 commit 5ccba16

1 file changed

Lines changed: 160 additions & 20 deletions

File tree

README.md

Lines changed: 160 additions & 20 deletions
Original file line numberDiff line numberDiff line change
@@ -174,41 +174,181 @@ for _, r := range results {
174174
## Technical Deep Dive
175175

176176
<details>
177-
<summary><strong>Click to expand: SCEV & The Zipper</strong></summary>
177+
<summary><strong>Click to expand: Architecture & Algorithms</strong></summary>
178178

179-
### How It Works
179+
### Pipeline Overview
180180

181-
1. **Parse** — Load Go source into SSA (Static Single Assignment) form
182-
2. **Canonicalize** — Normalize variable names, branch ordering, loop structures
183-
3. **Fingerprint** — SHA-256 hash of the canonical IR
181+
```
182+
Source → SSA → Loop Analysis → SCEV → Canonicalization → SHA-256
183+
↓ ↓ ↓ ↓
184+
go/ssa Tarjan's Symbolic Virtual IR
185+
SCC Evaluation Normalization
186+
```
187+
188+
1. **SSA Construction** — `golang.org/x/tools/go/ssa` converts source to Static Single Assignment form with explicit control flow graphs
189+
2. **Loop Detection** — Natural loop identification via backedge detection (edge B→H where H dominates B)
190+
3. **SCEV Analysis** — Algebraic characterization of loop variables as closed-form recurrences
191+
4. **Canonicalization** — Deterministic IR transformation: register renaming, branch normalization, loop virtualization
192+
5. **Fingerprint** — SHA-256 of canonical IR string
193+
194+
### Scalar Evolution (SCEV) Engine
195+
196+
The SCEV framework (`scev.go`, 746 LOC) solves the "loop equivalence problem"—proving that syntactically different loops compute the same sequence of values.
197+
198+
**Core Abstraction: Add Recurrences**
199+
200+
An induction variable is represented as $\{Start, +, Step\}_L$, meaning at iteration $k$ the value is:
201+
202+
$$Val(k) = Start + (Step \times k)$$
203+
204+
This representation is closed under affine transformations:
205+
206+
| Operation | Result |
207+
|-----------|--------|
208+
| $\{S, +, T\} + C$ | $\{S+C, +, T\}$ |
209+
| $C \times \{S, +, T\}$ | $\{C \times S, +, C \times T\}$ |
210+
| $\{S_1, +, T_1\} + \{S_2, +, T_2\}$ | $\{S_1+S_2, +, T_1+T_2\}$ |
211+
212+
**IV Detection Algorithm (Tarjan's SCC)**
213+
214+
```
215+
1. Build dependency graph restricted to loop body
216+
2. Find SCCs via Tarjan's algorithm (O(V+E))
217+
3. For each SCC containing a header Phi:
218+
a. Extract cycle: Phi → BinOp → Phi
219+
b. Classify: Basic ({S,+,C}), Geometric ({S,*,C}), Polynomial
220+
c. Verify step is loop-invariant
221+
4. Propagate SCEV to derived expressions via recursive folding
222+
```
223+
224+
**Trip Count Derivation**
225+
226+
For a loop `for i := Start; i < Limit; i += Step`:
227+
228+
$$TripCount = \left\lceil \frac{Limit - Start}{Step} \right\rceil$$
229+
230+
Computed via ceiling division: `(Limit - Start + Step - 1) / Step`
231+
232+
The engine handles:
233+
- Up-counting (`i < N`) and down-counting (`i > N`) loops
234+
- Inclusive bounds (`i <= N` → add 1 to numerator)
235+
- Negative steps (normalized to absolute value)
236+
- Multi-predecessor loop headers (validates consistent start values)
184237
185-
The result: semantically equivalent code produces identical fingerprints.
238+
### Canonicalization Engine
186239
187-
### Scalar Evolution (SCEV) Analysis
240+
The canonicalizer (`canonicalizer.go`, 1162 LOC) transforms SSA into a deterministic string representation via five phases:
188241
189-
Standard hashing is brittle—changing `for i := 0` to `for range` breaks the hash. `sfw` solves this with an SCEV engine (`scev.go`) that algebraically solves loops:
242+
**Phase 1: Loop & SCEV Analysis**
243+
```go
244+
c.loopInfo = DetectLoops(fn)
245+
AnalyzeSCEV(c.loopInfo)
246+
```
247+
248+
**Phase 2: Semantic Normalization**
249+
- **Invariant Hoisting**: Pure calls like `len(s)` are virtually moved to pre-header
250+
- **IV Virtualization**: Phi nodes for IVs are replaced with SCEV notation `{0, +, 1}`
251+
- **Derived IV Propagation**: Expressions like `i*4` become `{0, +, 4}` in output
252+
253+
**Phase 3: Register Renaming**
254+
```
255+
Parameters: p0, p1, p2, ...
256+
Free Variables: fv0, fv1, ...
257+
Instructions: v0, v1, v2, ... (DFS order)
258+
```
259+
260+
**Phase 4: Deterministic Block Ordering**
261+
262+
Blocks are traversed in dominance-respecting DFS order, ensuring identical output regardless of SSA construction order. Successor ordering is normalized:
263+
- `>=` branches are rewritten to `<` with swapped successors
264+
- `>` branches are rewritten to `<=` with swapped successors
190265

191-
- **Induction Variable Detection:** Classifies loop variables as Add Recurrences: $\{Start, +, Step\}$
192-
- **Trip Count Derivation:** Proves that a `range` loop and an index loop iterate the same number of times
193-
- **Loop Invariant Hoisting:** Invariant expressions (e.g., `len(s)`) are virtually hoisted, so manual optimizations don't alter fingerprints
266+
**Phase 5: Virtual Control Flow**
194267

195-
**Result:** Refactor loop syntax freely. If the math is the same, the fingerprint is the same.
268+
Branch normalization is applied *virtually* (no SSA mutation) via lookup tables:
269+
```go
270+
virtualBlocks map[*ssa.BasicBlock]*virtualBlock // swapped successors
271+
virtualBinOps map[*ssa.BinOp]token.Token // normalized operators
272+
```
196273

197274
### The Semantic Zipper
198275

199-
When logic *does* change (e.g., architectural refactors), fingerprint comparison fails. The Zipper algorithm (`zipper.go`) takes two SSA graphs and "zips" them together starting from function parameters:
276+
The Zipper (`zipper.go`, 568 LOC) computes a semantic diff between two functions—what actually changed in behavior, ignoring cosmetic differences.
200277

201-
- **Anchor Alignment:** Parameters and free variables establish deterministic entry points
202-
- **Forward Propagation:** Traverses use-def chains to match semantically equivalent nodes
203-
- **Divergence Isolation:** Reports exactly what changed (e.g., "added `Call <net.Dial>`, preserved all assignments")
278+
**Algorithm: Parallel Graph Traversal**
279+
280+
```
281+
PHASE 0: Semantic Analysis
282+
- Run SCEV on both functions independently
283+
- Build canonicalizers for operand comparison
284+
285+
PHASE 1: Anchor Alignment
286+
- Map parameters positionally: oldFn.Params[i] ↔ newFn.Params[i]
287+
- Map free variables if counts match
288+
- Seed entry block via sequential matching (critical for main())
289+
290+
PHASE 2: Forward Propagation (BFS on Use-Def chains)
291+
while queue not empty:
292+
(vOld, vNew) = dequeue()
293+
for each user uOld of vOld:
294+
candidates = users of vNew with matching structural fingerprint
295+
for uNew in candidates:
296+
if areEquivalent(uOld, uNew):
297+
map(uOld, uNew)
298+
enqueue((uOld, uNew))
299+
break
300+
301+
PHASE 2.5: Terminator Scavenging
302+
- Explicitly match Return/Panic instructions via operand equivalence
303+
- Handles cases where terminators aren't reached via normal propagation
304+
305+
PHASE 3: Divergence Isolation
306+
- Added = newFn instructions not in reverse map
307+
- Removed = oldFn instructions not in forward map
308+
```
309+
310+
**Equivalence Checking**
311+
312+
Two instructions are equivalent iff:
313+
1. Same Go type (`reflect.TypeOf`)
314+
2. Same SSA value type (`types.Identical`)
315+
3. Same operation-specific properties (BinOp.Op, Field index, Alloc.Heap, etc.)
316+
4. All operands equivalent (recursive, with commutativity handling for ADD/MUL/AND/OR/XOR)
317+
318+
**Structural Fingerprinting (DoS Prevention)**
319+
320+
To prevent $O(N \times M)$ comparisons on high-fanout values, users are bucketed by structural fingerprint:
321+
```go
322+
fp := fmt.Sprintf("%T:%s", instr, op) // e.g., "*ssa.BinOp:+"
323+
candidates := newByOp[fp] // Only compare compatible types
324+
```
204325

205-
**Result:** A semantic changelog that ignores renaming, reordering, and helper extraction.
326+
Bucket size is capped at 100 to bound worst-case complexity.
206327

207328
### Security Hardening
208329

209-
- **Cycle Detection:** Prevents stack overflow DoS from malformed cyclic graphs
210-
- **IR Injection Prevention:** Sanitizes string literals and struct tags to prevent fake instruction injection
211-
- **NaN-Safe Comparisons:** Limits branch normalization to integer/string types to avoid floating-point edge cases
330+
| Threat | Mitigation |
331+
|--------|------------|
332+
| **Algorithmic DoS** (exponential SCEV) | Memoization cache per-loop: `loop.SCEVCache` |
333+
| **Quadratic Zipper** (5000 identical ADDs) | Fingerprint bucketing + `MaxCandidates=100` |
334+
| **RCE via CGO** | `CGO_ENABLED=0` during `packages.Load` |
335+
| **SSRF via module fetch** | `GOPROXY=off` prevents network calls |
336+
| **Stack overflow** (cyclic graphs) | Visited sets in all recursive traversals |
337+
| **NaN comparison instability** | Branch normalization restricted to `IsInteger \| IsString` types |
338+
| **IR injection** (fake instructions in strings) | Struct tags and literals sanitized before hashing |
339+
| **TypeParam edge cases** | Generic types excluded from branch swap (may hide floats) |
340+
341+
### Complexity Analysis
342+
343+
| Operation | Time | Space |
344+
|-----------|------|-------|
345+
| SSA Construction | $O(N)$ | $O(N)$ |
346+
| Loop Detection | $O(V+E)$ | $O(V)$ |
347+
| SCEV Analysis | $O(L \times I)$ amortized | $O(I)$ per loop |
348+
| Canonicalization | $O(I \times \log B)$ | $O(I + B)$ |
349+
| Zipper | $O(I^2)$ worst, $O(I)$ typical | $O(I)$ |
350+
351+
Where $N$ = source size, $V$ = blocks, $E$ = edges, $L$ = loops, $I$ = instructions, $B$ = blocks.
212352

213353
</details>
214354

0 commit comments

Comments
 (0)