Compiler

[bfarmer67]

Description

This PR introduces Hyperbee.Expressions.Compiler (HEC) — a high-performance, IR-based compiler for .NET
expression trees — and wires it into the existing Hyperbee.Expressions async/enumerable state machine
infrastructure via a pluggable interface.

Motivation

The System expression compiler (Expression.Compile()) is a general-purpose interpreter. It is correct
and portable, but it is slow. For applications that compile expression trees at runtime — rule engines,
query builders, dynamic pipelines, scripted workflows — compilation cost dominates. FastExpressionCompiler
(FEC) is a well-known improvement, but it has correctness gaps on complex patterns (nullable lifted ops,
certain try/catch shapes, RuntimeVariables, Dynamic expressions) and is a third-party dependency.

Hyperbee.Expressions already generates async and enumerable state machines as expression trees. Those
state machines need to be compiled. Before this PR, the only options were SEC (slow) or FEC (fast but
incomplete). Neither integrates cleanly with the state machine generator — each compiled MoveNext body
had to be compiled by whichever compiler happened to receive the outer lambda, which broke encapsulation.

Goals:

1. A first-party, high-performance compiler — close to FEC speed, full correctness, no third-party runtime dependency.
2. Pluggable compilation — AsyncBlockExpression should not be coupled to any specific compiler.
   The outer compiler (SEC, FEC, or HEC) should be able to compile inner state machine bodies automatically.
3. DI-friendly abstraction — IExpressionCompiler for injectable compiler selection.
4. Full expression coverage — every ExpressionType the System compiler handles, plus the patterns FEC
   does not.

What's New

Hyperbee.Expressions.Compiler (new package)

A four-stage IR-based compiler:

LambdaExpression
→ ExpressionLowerer (expression tree → flat IROp instruction stream)
→ StackSpillPass (eliminate merge-point locals at branch phi-points)
→ PeepholePass (constant folding, branch simplification, load/store elimination)
→ DeadCodePass (remove unreachable instructions after unconditional branches)
→ ILEmissionPass (IR → CIL → DynamicMethod)

Public API surface:

Type	Description
HyperbeeCompiler	Static façade — Compile, TryCompile, CompileWithFallback, CompileToMethod, CompileToInstanceMethod, UseAsDefault
HyperbeeExpressionCompiler	IExpressionCompiler singleton wrapper for DI
HyperbeeCompilerExtensions	lambda.CompileHyperbee() extension method
CompilerDiagnostics	IRCapture callback for IR listing inspection

Hyperbee.Expressions — pluggable coroutine builder (M1–M3)

Three milestones refactored the state machine generator to support compiler plug-in without changing
expression construction semantics:

Milestone	Change
M1	ICoroutineDelegateBuilder interface; HyperbeeCoroutineDelegateBuilder; CoroutineBuilderContext (AsyncLocal ambient + static default)
M2	HEC compiles all BlockAsync patterns correctly; fixes for short-branch overflow, struct field method calls, and extension-node constant scanning
M3	ExpressionRuntimeOptions.DelegateBuilder removed; IExpressionCompiler + SystemExpressionCompiler added; CoroutineBuilderContext.SetScope; HyperbeeExpressionCompiler; ambient set automatically during HyperbeeCompiler.Compile()

The net effect: BlockAsync options describe behavior only (Optimize, ModuleBuilderProvider, ExpressionCapture).
Compiler choice flows through ambient context — no options need to change when switching compilers.

---

Performance

Benchmarks run on .NET 9, BenchmarkDotNet, 3 iterations + 3 warmup, Intel Core i9-9980HK.

Compilation Speed

Tier	System	FEC	HEC	HEC vs System	HEC vs FEC
Simple	28.7 µs	3.1 µs	3.6 µs	8× faster	1.16×
Closure	27.4 µs	2.9 µs	3.4 µs	8× faster	1.17×
TryCatch	50.4 µs	3.9 µs	5.1 µs	10× faster	1.31×
Complex	136.7 µs	3.4 µs	4.4 µs	31× faster	1.29×
Loop	67.9 µs	4.2 µs	6.4 µs	11× faster	1.51×
Switch	60.4 µs	3.4 µs	5.2 µs	12× faster	1.53×

HEC compiles 9–34× faster than the System compiler and within 1.16–1.54× of FEC.

Memory Allocations (per compile call)

Tier	System	FEC	HEC	vs System
Simple	4,335 B	904 B	2,152 B	50% fewer
Closure	4,279 B	895 B	2,136 B	50% fewer
TryCatch	5,893 B	1,519 B	3,999 B	32% fewer
Complex	4,741 B	1,390 B	2,512 B	47% fewer
Loop	6,710 B	1,110 B	4,264 B	36% fewer
Switch	6,264 B	1,352 B	4,128 B	34% fewer

HEC allocates up to 50% less than the System compiler.

Execution Speed

Delegates produced by HEC execute at the same speed as SEC and FEC. At sub-nanosecond scale, differences
are within measurement noise. For CPU-bound workloads (Complex tier, ~25 ns), all three compilers are
within 10% of each other.

Performance Implementation Notes

• The dominant cost difference vs FEC is the constants-array closure: HEC captures non-embeddable
  constants (object refs, delegates, nested lambdas) in a object[] closure and indexes into it at
  runtime. FEC inlines constants directly via ldobj/stobj.
• AsyncLocal writes are expensive (~1 µs each). CoroutineBuilderContext.Exchange() is gated on
  ScanForNonEmbeddableConstants — for simple expressions (no extension nodes), the ambient write is
  skipped entirely, recovering ~2 µs per compile call for the common case.
• Three IL optimization passes (StackSpill, Peephole, DeadCode) reduce emitted instruction count and
  eliminate unnecessary locals, bringing HEC within striking distance of FEC's tight output.

---

Expression Coverage

HEC handles every ExpressionType the System compiler handles, plus patterns FEC does not:

Category	Notes
Arithmetic, bitwise, shift	All signed/unsigned checked variants
Comparison	Ceq, Clt, Cgt, un-variants
Conversion	Convert, ConvertChecked, ConvertCheckedUn, Box, UnboxAny, CastClass, IsInst
Nullable lifted ops	Full three-value-logic lifted arithmetic, bitwise, and comparison
Member access	Instance/static fields, properties, indexers
Method calls	Static, virtual, interface, constrained (value-type virtual calls)
Arrays	NewArray, LoadElement, StoreElement, LoadArrayLength, LoadElementAddress
Control flow	IfThen, IfThenElse, Switch, Loop, Goto, Label, Return
Exception handling	TryCatch, TryCatchFinally, TryFinally, TryFault, ExceptionFilter
Closures	Captured variables via StrongBox<T> hoisting; nested lambdas
RuntimeVariables	IRuntimeVariables via RuntimeVariablesHelper
Dynamic	DynamicExpression via CallSite
Quote	Expression.Quote for nested expression trees
CompileToMethod	Static MethodBuilder target (no closure support — all constants must be embeddable)
CompileToInstanceMethod	Instance/value-type MethodBuilder target (for IAsyncStateMachine.MoveNext)

---

Test Suite

2,610 test instances across 32 expression test files, 7 BlockAsync integration suites, and a
FEC issue regression suite. All tests pass (2,589 passed, 21 skipped).

Suite	Tests	Notes
Expression tests (32 files)	~2,340	3 compilers × all expression types
BlockAsync integration (7 files)	~210	Basic, conditional, core, loop, switch, try/catch, context
FEC known issues	10	Documents FEC failures; suppressed in main test runs
CompileToMethod / CompileToInstanceMethod	~40	Static and instance MethodBuilder patterns

Test policy:

• System fails → invalid pattern; test removed
• FEC fails → suppressed with Assert.Inconclusive + documented in FecKnownIssues.cs
• HEC fails → fix required (no suppressions for HEC)

Bugs Fixed in Hyperbee.Expressions

Several bugs were discovered during M2 integration testing and fixed in the core library:

Fix	Description
Short-branch overflow	ILEmissionPass now auto-upgrades short (br.s) branches to long (br) when the offset exceeds ±127 bytes. Required for the ConditionalAwait multi-state pattern.
Struct field method calls	ExpressionLowerer emits Constrained prefix + LoadFieldAddress for value-type field method calls (e.g., TaskAwaiter<T>.GetResult()). Fixes stack underflow in multi-state SequentialAwaits and TryCatch MoveNext patterns.
Extension-node constant scanning	ScanForNonEmbeddableConstants returns true for ExpressionType.Extension so AsyncBlockExpression reduction is handled correctly during HEC compilation.
Void-assign dup elimination	ExpressionLowerer.LowerBlock correctly suppresses the Dup emitted by LowerAssign when the assignment result is immediately discarded.
ldelema for struct-element field assignment	LoadElementAddress (ldelema) is now emitted instead of LoadElement + StoreElement when the target is a struct field element address.
Merge-point local elimination	StackSpillPass eliminates the temporary StoreLocal/LoadLocal pair inserted at branch merge-points when the value is immediately consumed, reducing unnecessary locals in if/else and switch patterns.

Documentation

Full just-the-docs documentation added under docs/ matching the structure of sibling projects
(hyperbee.pipeline, hyperbee.json, hyperbee.xs):

docs/
├── index.md Landing page
├── expressions/ 12 expression type pages
├── configuration/ Runtime options, module providers, DI
├── compiler/ Overview, API reference, diagnostics, performance
└── lab/ Fetch, JSON, Map/Reduce

Files Changed

• 37 commits, 117 files, +36,567 / −363 lines
• src/Hyperbee.Expressions.Compiler/ — new package (19 source files)
• src/Hyperbee.Expressions/ — 8 files modified (interfaces, ambient context, options)
• test/Hyperbee.Expressions.Compiler.Tests/ — new project (32 test files, ~2,340 test instances)
• test/Hyperbee.Expressions.Compiler.IssueTests/ — new project (FEC regression suite)
• test/Hyperbee.Expressions.Compiler.Benchmarks/ — new project (compilation + execution benchmarks)
• docs/ — 27 markdown pages + config

Checklist

• All 669 Hyperbee.Expressions.Tests pass (no regressions)
• All 2,589 Hyperbee.Expressions.Compiler.Tests pass (21 skipped — documented FEC limitations)
• All 10 Hyperbee.Expressions.Compiler.IssueTests pass
• Benchmarks measured and documented
• Documentation complete