Reverse FFI foundation: register_builtin core API + tests

Adds Interpreter::register_builtin(name, Fn(&[Value]) -> Result<Value, String>)
— the foundation for OMC reaching out to host capabilities. Used as
the dispatch substrate for every "OMC calls X" feature: pyo3-embedded
Python (next), Godot signals, file pickers, sandboxed I/O, etc.

Dispatch precedence in call_function (tree-walk):
  1. user-defined fn (self.functions)
  2. host-registered builtin  ← NEW
  3. module-qualified call ("phi.fold")
  4. built-in stdlib

Same precedence in vm_call_builtin (VM path), so host builtins shadow
even hot stdlib names like str_len / now_ms — useful for sandboxing
and substitution. The HashMap lookup is one branch when not present.

Public API:
  fn register_builtin<F>(&mut self, name: &str, handler: F)
      where F: Fn(&[Value]) -> Result<Value, String> + 'static
  fn unregister_builtin(&mut self, name: &str) -> bool
  fn has_host_builtin(&self, name: &str) -> bool

Handlers:
  - Receive evaluated Values (not Expressions — they live outside
    OMC's eval context).
  - Return Value on success, String on error. Errors propagate
    through normal stack-trace machinery and are catchable via
    try/catch in OMC code.
  - Single-threaded (Rc<dyn Fn>) — matches OMC's runtime.
  - Stored as Rc<dyn Fn> so handlers can be cheaply cloned when
    the Interpreter itself is.

Test suite (omnimcode-core/tests/host_builtins.rs, 9 tests):
  - simple int double, both engines
  - side-effect captures (Python-callback pattern), both engines
  - return Vec<Value> as Array (NumPy → OMC array path)
  - error propagation via try/catch
  - shadowing stdlib names, both engines
  - register/unregister lifecycle

VM tests use a `__capture(value)` host fn pattern to retrieve the
final value, since the VM pops its top-level scope on exit and
doesn't preserve `h __result__` like tree-walk does. This pattern
will be reused by future PyO3 demos.

43/43 functional examples produce identical output under tree-walk
and VM. 9/9 new tests + 92 existing unit tests + ffi tests all pass.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

Author: RandomCoder-lab

omnimcode-core/src/interpreter.rs

@@ -41,6 +41,21 @@ pub struct Interpreter {
     /// sees in stack traces. The fn's own internal line numbers don't
     /// belong here; they'd need per-statement position tracking.
     call_stack: Vec<(String, crate::ast::Pos)>,
+    /// Reverse-FFI: builtins registered by the embedder (Python /
+    /// Godot / a Rust host). When OMC code calls a name not found
+    /// in user fns, modules, or the built-in stdlib, dispatch
+    /// falls through to this map. Lets an embedder expose host-side
+    /// capabilities (numpy, godot signals, file pickers, etc.) to
+    /// OMC programs without baking them into the interpreter.
+    ///
+    /// Stored as `Rc<dyn Fn>` so handlers can be cheaply cloned
+    /// when the Interpreter itself is cloned (rare, but FFI wrappers
+    /// occasionally do it). Single-threaded — handlers don't need
+    /// to be Send/Sync, matching the rest of OMC's runtime.
+    host_builtins: HashMap<
+        String,
+        std::rc::Rc<dyn Fn(&[Value]) -> Result<Value, String>>,
+    >,
 }
 
 impl Interpreter {
@@ -64,9 +79,56 @@ impl Interpreter {
             test_failures: std::cell::RefCell::new(Vec::new()),
             test_current_name: std::cell::RefCell::new(String::new()),
             call_stack: Vec::new(),
+            host_builtins: HashMap::new(),
         }
     }
 
+    /// Register a host-side builtin that OMC code can call by name.
+    /// The closure receives the evaluated argument values and returns
+    /// either a Value (success) or an error message that propagates
+    /// through OMC's normal Result chain (catchable via try/catch).
+    ///
+    /// Names registered here SHADOW user-defined functions of the
+    /// same name (so an embedder can hand OMC a custom `fetch_url`
+    /// that overrides any user `fn fetch_url(...)`). They're checked
+    /// AFTER user fns, BEFORE the built-in stdlib — same precedence
+    /// position the test runner's `test_*` overrides use.
+    ///
+    /// Type signatures are dynamic: the closure is responsible for
+    /// validating arg count and types. Use `args.len()` and
+    /// `matches!(args[0], Value::HInt(_))` etc. Errors are strings;
+    /// they appear in stack traces with the call site prefixed.
+    ///
+    /// Example:
+    /// ```ignore
+    /// let mut interp = Interpreter::new();
+    /// interp.register_builtin("double", |args| {
+    ///     if args.len() != 1 { return Err("double requires 1 arg".into()); }
+    ///     Ok(Value::HInt(HInt::new(args[0].to_int() * 2)))
+    /// });
+    /// // OMC code can now do `println(double(21));` and see "42".
+    /// ```
+    pub fn register_builtin<F>(&mut self, name: &str, handler: F)
+    where
+        F: Fn(&[Value]) -> Result<Value, String> + 'static,
+    {
+        self.host_builtins.insert(name.to_string(), std::rc::Rc::new(handler));
+    }
+
+    /// Remove a previously-registered host builtin. Returns true if
+    /// a handler was removed. Used by embedders that want to hand
+    /// OMC a temporary capability for a single call sequence.
+    pub fn unregister_builtin(&mut self, name: &str) -> bool {
+        self.host_builtins.remove(name).is_some()
+    }
+
+    /// True if a host builtin with this name is registered. Used by
+    /// the dispatch path; exposed publicly so embedders can check
+    /// before re-registering.
+    pub fn has_host_builtin(&self, name: &str) -> bool {
+        self.host_builtins.contains_key(name)
+    }
+
     /// xorshift64* — fast and tiny, sufficient for OMC scripting needs.
     /// Not cryptographic. Returns a non-zero u64.
     fn rng_next(&self) -> u64 {
@@ -1204,6 +1266,18 @@ impl Interpreter {
         if let Some((params, body)) = self.functions.get(name).cloned() {
             return self.invoke_user_function(name, &params, &body, args);
         }
+        // Reverse-FFI: host-registered builtins. Checked BEFORE module
+        // dispatch and the built-in stdlib so an embedder can shadow
+        // anything (including `read_file`, `now_ms`, etc.). Eval args
+        // here — the host fn receives Values, not Expressions, since
+        // it lives outside OMC's eval context.
+        if let Some(handler) = self.host_builtins.get(name).cloned() {
+            let mut argvals = Vec::with_capacity(args.len());
+            for a in args {
+                argvals.push(self.eval_expr(a)?);
+            }
+            return handler(&argvals);
+        }
         // Module-qualified calls (e.g., "phi.fold", "phi.res", "core.fib")
         if let Some((module, func)) = name.split_once('.') {
             return self.call_module_function(module, func, args);
@@ -3750,6 +3824,15 @@ impl Interpreter {
         name: &str,
         args: &[Value],
     ) -> Result<Value, String> {
+        // Reverse-FFI host builtins fire FIRST so they can shadow
+        // anything (including stdlib names like `read_file`). Lets an
+        // embedder hand OMC code a sandboxed `read_file` that only
+        // sees /tmp, etc. Skipped if the host hasn't registered the
+        // name — the no-op cost is one HashMap lookup.
+        if let Some(handler) = self.host_builtins.get(name).cloned() {
+            return handler(args);
+        }
+
         // Phase 4 fast-path: hot builtins handled directly on values,
         // bypassing the synthetic-arg shim. Each one shaved ~50% off
         // its per-call time on the benchmark suite (str_concat went

omnimcode-core/tests/host_builtins.rs

@@ -0,0 +1,244 @@
+//! Reverse-FFI host-builtin tests.
+//!
+//! Verifies that an embedder can register Rust closures as OMC-callable
+//! builtins, and that the dispatch works for both the tree-walk
+//! interpreter and the bytecode VM. Uses a shared cell to capture
+//! side effects so we can assert the host code actually ran.
+
+use omnimcode_core::bytecode_opt::optimize_module;
+use omnimcode_core::compiler::compile_program;
+use omnimcode_core::interpreter::Interpreter;
+use omnimcode_core::parser::Parser;
+use omnimcode_core::value::{HArray, HInt, Value};
+use omnimcode_core::vm::Vm;
+use std::cell::Cell;
+use std::rc::Rc;
+
+/// Run `source` through the tree-walk interpreter with `setup_host`
+/// called on the interpreter before execution. Returns the final
+/// value of `__result__`.
+fn run_treewalk(
+    source: &str,
+    setup_host: impl FnOnce(&mut Interpreter),
+) -> Result<Value, String> {
+    let mut parser = Parser::new(source);
+    let stmts = parser.parse()?;
+    let mut interp = Interpreter::new();
+    setup_host(&mut interp);
+    interp.execute(stmts)?;
+    interp
+        .get_var_for_testing("__result__")
+        .ok_or_else(|| "no __result__ variable".to_string())
+}
+
+/// Run `source` through the bytecode VM with `setup_host` called
+/// before execution. The Vm's internal Interpreter is what the host
+/// fns end up registered on.
+///
+/// The VM pops its top-level scope on exit so we can't read
+/// `__result__` from it the way tree-walk can. Instead, the source
+/// must end with `__capture(<expr>);` where `__capture` is a host
+/// fn we register to stash the result in a returned Cell.
+fn run_vm_with_capture(
+    source: &str,
+    setup_host: impl FnOnce(&mut Interpreter),
+) -> Result<Value, String> {
+    let mut parser = Parser::new(source);
+    let stmts = parser.parse()?;
+    let module = compile_program(&stmts)?;
+    let mut module = module;
+    optimize_module(&mut module);
+    let mut vm = Vm::new();
+    let captured: Rc<std::cell::RefCell<Option<Value>>> =
+        Rc::new(std::cell::RefCell::new(None));
+    let captured_clone = Rc::clone(&captured);
+    {
+        let interp = vm.interp_mut();
+        setup_host(interp);
+        interp.register_builtin("__capture", move |args| {
+            *captured_clone.borrow_mut() = args.first().cloned();
+            Ok(Value::Null)
+        });
+        interp.process_imports(&stmts)?;
+        interp.register_user_functions(&stmts);
+        for (lname, lparams, lbody) in &module.lambda_asts {
+            interp.register_lambda(lname, lparams.clone(), lbody.clone());
+        }
+    }
+    vm.run_module(&module)?;
+    let result = captured.borrow().clone();
+    result.ok_or_else(|| "no __capture(...) call in source".to_string())
+}
+
+#[test]
+fn host_builtin_simple_int_double_treewalk() {
+    let v = run_treewalk(
+        r#"
+        h __result__ = double(21);
+        "#,
+        |interp| {
+            interp.register_builtin("double", |args| {
+                Ok(Value::HInt(HInt::new(args[0].to_int() * 2)))
+            });
+        },
+    )
+    .unwrap();
+    assert_eq!(v.to_int(), 42);
+}
+
+#[test]
+fn host_builtin_simple_int_double_vm() {
+    let v = run_vm_with_capture(
+        r#"
+        __capture(double(21));
+        "#,
+        |interp| {
+            interp.register_builtin("double", |args| {
+                Ok(Value::HInt(HInt::new(args[0].to_int() * 2)))
+            });
+        },
+    )
+    .unwrap();
+    assert_eq!(v.to_int(), 42);
+}
+
+/// Confirm side effects propagate Rust-side. The host fn writes to a
+/// shared Cell; we read it after OMC execution. This is the pattern
+/// PyO3 will use for round-tripping data.
+#[test]
+fn host_builtin_side_effect_treewalk() {
+    let captured: Rc<Cell<i64>> = Rc::new(Cell::new(0));
+    let captured_clone = Rc::clone(&captured);
+    let _ = run_treewalk(
+        r#"
+        capture(89);
+        h __result__ = 1;
+        "#,
+        move |interp| {
+            interp.register_builtin("capture", move |args| {
+                captured_clone.set(args[0].to_int());
+                Ok(Value::Null)
+            });
+        },
+    )
+    .unwrap();
+    assert_eq!(captured.get(), 89);
+}
+
+#[test]
+fn host_builtin_side_effect_vm() {
+    let captured: Rc<Cell<i64>> = Rc::new(Cell::new(0));
+    let captured_clone = Rc::clone(&captured);
+    let _ = run_vm_with_capture(
+        r#"
+        capture(89);
+        __capture(1);
+        "#,
+        move |interp| {
+            interp.register_builtin("capture", move |args| {
+                captured_clone.set(args[0].to_int());
+                Ok(Value::Null)
+            });
+        },
+    )
+    .unwrap();
+    assert_eq!(captured.get(), 89);
+}
+
+/// Host fn returns an array — the value flows back into OMC normally.
+/// Tests the "I want my Python list to look like an OMC array" path.
+#[test]
+fn host_builtin_returns_array() {
+    let v = run_treewalk(
+        r#"
+        h xs = numpy_arange(5);
+        h __result__ = arr_len(xs);
+        "#,
+        |interp| {
+            interp.register_builtin("numpy_arange", |args| {
+                let n = args[0].to_int().max(0) as usize;
+                let items: Vec<Value> = (0..n)
+                    .map(|i| Value::HInt(HInt::new(i as i64)))
+                    .collect();
+                Ok(Value::Array(HArray::from_vec(items)))
+            });
+        },
+    )
+    .unwrap();
+    assert_eq!(v.to_int(), 5);
+}
+
+/// Host fn errors propagate as OMC errors — catchable via try/catch.
+#[test]
+fn host_builtin_error_is_catchable() {
+    let v = run_treewalk(
+        r#"
+        try {
+            broken();
+            h __result__ = 0;
+        } catch e {
+            h __result__ = e;
+        }
+        "#,
+        |interp| {
+            interp.register_builtin("broken", |_args| {
+                Err("intentional host failure".to_string())
+            });
+        },
+    )
+    .unwrap();
+    match v {
+        Value::String(s) => assert!(s.contains("intentional host failure")),
+        other => panic!("expected error string, got {:?}", other),
+    }
+}
+
+/// Host fn shadows a stdlib name. Used for sandboxing — embedder hands
+/// OMC a custom `read_file` that only sees a whitelisted directory.
+#[test]
+fn host_builtin_shadows_stdlib() {
+    let v = run_treewalk(
+        r#"
+        h __result__ = now_ms();
+        "#,
+        |interp| {
+            interp.register_builtin("now_ms", |_args| {
+                Ok(Value::HInt(HInt::new(12345)))
+            });
+        },
+    )
+    .unwrap();
+    assert_eq!(v.to_int(), 12345);
+}
+
+/// Same shadowing test under the VM — verifies vm_call_builtin checks
+/// host_builtins BEFORE vm_fast_dispatch, which would otherwise win
+/// for hot stdlib names.
+#[test]
+fn host_builtin_shadows_stdlib_under_vm() {
+    let v = run_vm_with_capture(
+        r#"
+        __capture(str_len("ignored"));
+        "#,
+        |interp| {
+            interp.register_builtin("str_len", |_args| {
+                Ok(Value::HInt(HInt::new(999)))
+            });
+        },
+    )
+    .unwrap();
+    assert_eq!(v.to_int(), 999);
+}
+
+/// unregister_builtin removes a previously-registered handler. The
+/// next call resolves to the underlying stdlib (or fails if no
+/// stdlib match).
+#[test]
+fn host_builtin_unregister() {
+    let mut interp = Interpreter::new();
+    interp.register_builtin("custom", |_args| Ok(Value::HInt(HInt::new(7))));
+    assert!(interp.has_host_builtin("custom"));
+    assert!(interp.unregister_builtin("custom"));
+    assert!(!interp.has_host_builtin("custom"));
+    assert!(!interp.unregister_builtin("custom"));
+}