feat(tco): self-recursive tail-call marker + LLVM tail hint

Adds a new `tco` pass that walks every function after the
optimisation fixed-point and flips `is_tail = true` on direct
self-recursive `Call`s whose result flows into the block's `Return`
terminator with no intervening work. The LLVM backend then emits
the `tail` flag on the corresponding `CallInst`, letting LLVM's
sibling-call optimisation collapse the recursion when the ABI and
stack layout allow it.

Scope is restricted to self-recursive direct calls. A broader
"every tail-position Call" scope was tried first and regressed the
fib bench ~45 % exec: the single `main → fib(40)` call was the only
mark in that program, and LLVM's IPA inliner reacted to the marker
in a way that slowed fib's hot loop body (the marker perturbed
inlining decisions for reasons unrelated to TCO). Self-recursion
also matches the case where a future HIR-level loop-rewrite pass
could fully eliminate the call frame, so the narrow scope is
where the next payoff naturally lands.

Cranelift's `return_call` lowering is half-wired (it'd
double-terminate the block alongside the HIR Return terminator) and
was unreachable before this commit. The three Cranelift call arms
that read `is_tail` now ignore it and fall back to standard `call`
until the terminator-skip plumbing lands separately. The HIR mark
is still useful — LLVM honours it today.

No bench delta on the current kernels (none use tail recursion); 7
new unit tests cover the marker's shape recognition.

Author: darmie

crates/compiler/src/cranelift_backend.rs

@@ -2126,7 +2126,7 @@ impl CraneliftBackend {
                             result,
                             callee,
                             args,
-                            is_tail,
+                            is_tail: _,
                             ..
                         } => {
                             let arg_values: Vec<Value> = args.iter()
@@ -6200,7 +6200,7 @@ impl CraneliftBackend {
                 callee,
                 args,
                 type_args,
-                is_tail,
+                is_tail: _,
                 ..
             } => {
                 let arg_vals: Vec<_> = args.iter().map(|arg| self.value_map[arg]).collect();
@@ -6212,11 +6212,17 @@ impl CraneliftBackend {
                             .module
                             .declare_func_in_func(cranelift_func, builder.func);
 
-                        let call = if *is_tail {
-                            builder.ins().return_call(func_ref, &arg_vals)
-                        } else {
-                            builder.ins().call(func_ref, &arg_vals)
-                        };
+                        // `return_call` is a Cranelift terminator. Our
+                        // block-level lowering still emits the HIR
+                        // block's Return terminator after the Call, so
+                        // a `return_call` here would leave the block
+                        // double-terminated and trip Cranelift's
+                        // verifier. The HIR tco marker still flips
+                        // `is_tail = true` for downstream backends
+                        // (LLVM gets the hint), but Cranelift falls
+                        // back to the standard call until the
+                        // terminator-skip plumbing lands.
+                        let call = builder.ins().call(func_ref, &arg_vals);
 
                         if let Some(result_id) = result {
                             let results = builder.inst_results(call);
@@ -6257,13 +6263,11 @@ impl CraneliftBackend {
                         // This is a simplified version - in reality we'd need to track function signatures
                         let sig_ref = builder.import_signature(self.module.make_signature());
 
-                        let call = if *is_tail {
-                            builder
-                                .ins()
-                                .return_call_indirect(sig_ref, ptr_val, &arg_vals)
-                        } else {
-                            builder.ins().call_indirect(sig_ref, ptr_val, &arg_vals)
-                        };
+                        // Same terminator-already-emitted concern as the
+                        // direct-call path above. Fall back to a
+                        // standard `call_indirect` until the lowering
+                        // can skip the trailing Return terminator.
+                        let call = builder.ins().call_indirect(sig_ref, ptr_val, &arg_vals);
 
                         if let Some(result_id) = result {
                             let results = builder.inst_results(call);
@@ -7068,11 +7072,10 @@ impl CraneliftBackend {
                             })?;
                         let func_ref = self.module.declare_func_in_func(func, builder.func);
 
-                        let call = if *is_tail {
-                            builder.ins().return_call(func_ref, &arg_vals)
-                        } else {
-                            builder.ins().call(func_ref, &arg_vals)
-                        };
+                        // Symbol calls share the same terminator-skip
+                        // concern as the direct-call arms above; fall
+                        // back to a standard `call` until that lands.
+                        let call = builder.ins().call(func_ref, &arg_vals);
 
                         if let Some(result_id) = result {
                             let results = builder.inst_results(call);

crates/compiler/src/lib.rs

@@ -58,6 +58,7 @@ pub mod runtime;
 pub mod scalar_replace_alloc; // Eliminate non-escaping Call(Intrinsic::Malloc) allocations (heap SROA)
 pub mod ssa;
 pub mod stdlib; // Standard library implementation using HIR Builder
+pub mod tco; // Tail-call optimisation marker
 pub mod trait_lowering; // Trait/interface lowering to HIR
 pub mod typed_cfg; // New: TypedAST-aware CFG builder
 pub mod value; // Unified runtime value type (used by interp + embed)
@@ -1681,6 +1682,7 @@ pub struct InterpOptStats {
     pub auto_vectorize: auto_vectorize::AutoVectorizeStats,
     pub cfg_simplify: cfg_simplify::CfgSimplifyStats,
     pub drop_insert: drop_insert::DropStats,
+    pub tco: tco::TcoStats,
 }
 
 /// Run the subset of HIR optimization passes that are safe for the
@@ -1886,6 +1888,23 @@ pub fn run_interp_safe_opts(module: &mut HirModule) -> InterpOptStats {
     stats.auto_vectorize.rejected_no_iv += av.rejected_no_iv;
     stats.auto_vectorize.rejected_trip_count += av.rejected_trip_count;
 
+    // Tail-call marker runs after the fixed-point sweep but before
+    // drop_insert. Two ordering reasons:
+    //   * After the fixed-point: inlining can convert a non-tail Call
+    //     into a tail-shape one (the inlined return becomes the
+    //     caller's return), and const_fold / cse / cfg_simplify may
+    //     also clear away intervening instructions that were blocking
+    //     a tail shape. Running tco after all of those settle catches
+    //     every shape they enable.
+    //   * Before drop_insert: drop_insert places `Free` calls between
+    //     Call and Return for any non-escaping malloc, breaking the
+    //     "Call is the last instruction" precondition. Running tco
+    //     first marks every valid shape; the drop_insert pass can
+    //     then add Free calls without those marks getting lost.
+    let tc = tco::run_module(module);
+    stats.tco.candidates_visited += tc.candidates_visited;
+    stats.tco.marked += tc.marked;
+
     // Drop-site insertion runs *after* the optimization fixed-point.
     // Order matters two ways:
     //   * Inserting Free calls earlier would create new memory

crates/compiler/src/llvm_backend.rs

@@ -1270,9 +1270,9 @@ impl<'ctx> LLVMBackend<'ctx> {
                 args,
                 type_args: _,
                 const_args: _,
-                is_tail: _,
+                is_tail,
             } => {
-                let result_val = self.compile_call(callee, args)?;
+                let result_val = self.compile_call(callee, args, *is_tail)?;
                 if let Some(res_id) = result {
                     self.value_map.insert(*res_id, result_val);
                 }
@@ -2991,6 +2991,7 @@ impl<'ctx> LLVMBackend<'ctx> {
         &mut self,
         callee: &HirCallable,
         args: &[HirId],
+        is_tail: bool,
     ) -> CompilerResult<BasicValueEnum<'ctx>> {
         match callee {
             HirCallable::Function(func_id) => {
@@ -3017,6 +3018,17 @@ impl<'ctx> LLVMBackend<'ctx> {
                     }
                 }
 
+                // Tail-call hint. The HIR tco marker restricts
+                // `is_tail = true` to self-recursive direct calls, so
+                // by the time we get here the call is structurally a
+                // candidate for LLVM's sibling-call optimisation. The
+                // flag is purely advisory — LLVM ignores it when it
+                // can't prove TCO is safe — so it never causes a
+                // miscompile, only enables one when applicable.
+                if is_tail {
+                    call_site.set_tail_call(true);
+                }
+
                 // Return value (or void)
                 match call_site.try_as_basic_value() {
                     ValueKind::Basic(val) => Ok(val),
@@ -3122,6 +3134,9 @@ impl<'ctx> LLVMBackend<'ctx> {
                     &arg_values,
                     "indirect_call",
                 )?;
+                if is_tail {
+                    call_site.set_tail_call(true);
+                }
 
                 // Return value (or void)
                 match call_site.try_as_basic_value() {