Skip to content

Commit 8db4dc2

Browse files
Path A.4: read-only array support in dual-band JIT
OMC arrays now JIT. NewArray, ArrayLen, and ArrayIndex are implemented in the dual-band lowerer; any pure-int OMC fn that builds an array, reads from it, and returns a scalar will JIT. Layout: each NewArray emits an `alloca [N+1 x i64]` in the fn's entry block. Slot 0 holds the length (so ArrayLen needs no side-channel — arrays are self-describing). Slots 1..=N hold the elements, populated in source order from the operand stack. On the operand stack, an array is its pointer cast to i64 (ptrtoint at NewArray, inttoptr at use). This fits the existing Vec<VectorValue> stack convention without needing a typed enum — same pragmatic encoding as Path A.2's float bitcast trick. The caller-facing fn signature stays scalar i64; arrays are an internal-only representation. ArrayIndex's slot calc is `idx + 1` — skip the length prefix and GEP to the real element. ArrayLen is `gep idx 0, load i64`. Implementation surface (all in omnimcode-codegen/src/dual_band.rs): - Op::NewArray(N) handler -> emit_new_array helper - Op::ArrayLen handler -> emit_array_len helper - Op::ArrayIndex handler -> emit_array_index helper Tests (4 new, all passing): - jit_array_len_returns_correct_length: arr_len of [10,20,30,40,50] = 5 - jit_array_index_reads_correct_element: arr_get at each index returns the expected value - jit_array_sum_in_loop: sum [1..10] via ArrayLen + ArrayIndex inside a while loop, returns 55. Exercises the full pattern. - jit_array_via_dispatch_hook: end-to-end through Interpreter + dispatch hook (the OMC_HBIT_JIT=1 CLI path). sum [100,200,300] through JIT'd fn returns 600. Confirms arrays survive the round trip through the CLI dispatch layer. Out of scope for Path A.4 MVP (deferred): - Op::ArrayIndexAssign / ArrSetNamed: mutable writes to array slots. Needs careful thinking about α/β consistency for the written value (does β diverge or stay matched?). - Dynamic resize / arr_push: would need heap allocation via libc malloc + a header word for capacity. Stack alloca only supports fixed-size at NewArray time. - Array-typed parameters / returns: caller-facing signature is still scalar i64 only. - Multi-dimensional arrays: nested NewArray should already work via pointer-as-i64 representation (untested, but same scheme). These are all separate sessions when an actual workload needs them. The MVP unlocks every "fn that builds a constant table and sums/searches it" pattern, which is exactly the shape of most substrate-aligned harmonic libs. Workspace: 38 codegen tests pass (1 IR + 4 cross-fn + 4 arrays + 5 dual-band + 5 dispatch + 3 floats + 3 harmony + 5 phi_shadow + 8 scalar). Smoke + harmonic-lib + 149 core tests still green. This wraps Path A: - A.1: harmony-gated branch elision benched (95.2% reduction on high-harmony, break-even at 5-8% input fraction) - A.2: float arithmetic in lowerers - A.3: bytecode VM bench (VM is 2.1x over tree-walk; JIT is 119x over VM) - A.4: array reads (this commit) Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
1 parent fb61a88 commit 8db4dc2

2 files changed

Lines changed: 359 additions & 0 deletions

File tree

omnimcode-codegen/src/dual_band.rs

Lines changed: 210 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -361,6 +361,38 @@ impl<'ctx, 'a> DualBandLowerer<'ctx, 'a> {
361361
Op::Shl => self.bin_vec(&mut stack, i, |b, l, r| b.build_left_shift(l, r, "shl"))?,
362362
Op::Shr => self.bin_vec(&mut stack, i, |b, l, r| b.build_right_shift(l, r, true, "shr"))?,
363363

364+
// Path A.4: read-only array support.
365+
//
366+
// Layout: `alloca [N+1 x i64]`. Slot 0 holds the
367+
// length; slots 1..=N hold the elements. Self-describing
368+
// so ArrayLen needs no side-channel.
369+
//
370+
// Operand-stack convention: arrays live as
371+
// pointer-cast-to-i64 on the stack. ptrtoint at push;
372+
// inttoptr at use. The bit pattern survives storage in
373+
// user-level h-variables (which are <2 x i64> in
374+
// dual-band) because lane 0 carries the pointer and
375+
// matches what ArrayIndex / ArrayLen extract.
376+
//
377+
// Arrays live in the fn's stack frame. ArrayIndexAssign
378+
// (mutable writes) and dynamic resize are out of scope
379+
// for Path A.4 MVP — see Sessions later for those.
380+
Op::NewArray(n_elems) => {
381+
let v = self.emit_new_array(&mut stack, i, *n_elems)?;
382+
stack.push(v);
383+
}
384+
Op::ArrayLen => {
385+
let arr_v = self.pop(&mut stack, i, "ArrayLen ptr")?;
386+
let len = self.emit_array_len(arr_v, i)?;
387+
stack.push(self.splat(len, "alen_v")?);
388+
}
389+
Op::ArrayIndex => {
390+
let idx_v = self.pop(&mut stack, i, "ArrayIndex idx")?;
391+
let arr_v = self.pop(&mut stack, i, "ArrayIndex ptr")?;
392+
let val = self.emit_array_index(arr_v, idx_v, i)?;
393+
stack.push(self.splat(val, "aidx_v")?);
394+
}
395+
364396
Op::Eq => self.cmp_vec(&mut stack, i, IntPredicate::EQ)?,
365397
Op::Ne => self.cmp_vec(&mut stack, i, IntPredicate::NE)?,
366398
Op::Lt => self.cmp_vec(&mut stack, i, IntPredicate::SLT)?,
@@ -681,6 +713,184 @@ impl<'ctx, 'a> DualBandLowerer<'ctx, 'a> {
681713
}
682714
}
683715

716+
/// Path A.4: NewArray — pop N values from the operand stack, build
717+
/// a length-prefixed `[N+1 x i64]` alloca in the entry block, store
718+
/// the popped values into slots 1..=N (in source order — bytecode
719+
/// pushes elements left-to-right so popping gives reverse order),
720+
/// store length N at slot 0, and return the pointer as a splat'd
721+
/// `<2 x i64>` (lane 0 = ptr-as-i64, lane 1 = same).
722+
fn emit_new_array(
723+
&mut self,
724+
stack: &mut Vec<VectorValue<'ctx>>,
725+
op_idx: usize,
726+
n: usize,
727+
) -> Result<VectorValue<'ctx>, CodegenError> {
728+
let i64_type = self.ctx.i64_type();
729+
// Pop N values (each is a <2 x i64>; we extract α as the
730+
// user-visible scalar). Reverse to get source order.
731+
let mut elems: Vec<IntValue<'ctx>> = Vec::with_capacity(n);
732+
for k in 0..n {
733+
let v_v = self
734+
.pop(stack, op_idx, &format!("NewArray elem {}", k))?;
735+
let alpha = self
736+
.builder
737+
.build_extract_element(v_v, i64_type.const_int(0, false), "narr_a")
738+
.map_err(|e| format!("NewArray extract α at op{}: {}", op_idx, e))?;
739+
let alpha_iv = match alpha {
740+
BasicValueEnum::IntValue(iv) => iv,
741+
_ => return Err(format!("NewArray elem {} not int at op{}", k, op_idx)),
742+
};
743+
elems.push(alpha_iv);
744+
}
745+
elems.reverse();
746+
747+
// Allocate [N+1 x i64] in the entry block so the alloca
748+
// dominates all uses, regardless of which CFG block the
749+
// NewArray op was emitted from.
750+
let arr_ty = i64_type.array_type((n as u32) + 1);
751+
let current_block = self
752+
.builder
753+
.get_insert_block()
754+
.ok_or_else(|| format!("NewArray no insert block at op{}", op_idx))?;
755+
let entry = self.function.get_first_basic_block().unwrap();
756+
match entry.get_first_instruction() {
757+
Some(first) => self.builder.position_before(&first),
758+
None => self.builder.position_at_end(entry),
759+
}
760+
let arr_ptr = self
761+
.builder
762+
.build_alloca(arr_ty, &format!("arr_op{}", op_idx))
763+
.map_err(|e| format!("NewArray alloca at op{}: {}", op_idx, e))?;
764+
self.builder.position_at_end(current_block);
765+
766+
// Store length at slot 0.
767+
let zero32 = self.ctx.i32_type().const_int(0, false);
768+
let len_gep = unsafe {
769+
self.builder
770+
.build_in_bounds_gep(arr_ty, arr_ptr, &[zero32, zero32], "narr_len_gep")
771+
.map_err(|e| format!("NewArray len gep at op{}: {}", op_idx, e))?
772+
};
773+
self.builder
774+
.build_store(len_gep, i64_type.const_int(n as u64, false))
775+
.map_err(|e| format!("NewArray len store at op{}: {}", op_idx, e))?;
776+
777+
// Store elements at slots 1..=N.
778+
for (k, val) in elems.iter().enumerate() {
779+
let idx32 = self.ctx.i32_type().const_int((k + 1) as u64, false);
780+
let elem_gep = unsafe {
781+
self.builder
782+
.build_in_bounds_gep(arr_ty, arr_ptr, &[zero32, idx32], "narr_e_gep")
783+
.map_err(|e| format!("NewArray elem{} gep at op{}: {}", k, op_idx, e))?
784+
};
785+
self.builder
786+
.build_store(elem_gep, *val)
787+
.map_err(|e| format!("NewArray elem{} store at op{}: {}", k, op_idx, e))?;
788+
}
789+
790+
// Cast the pointer to i64 and splat into <2 x i64>.
791+
let ptr_as_i64 = self
792+
.builder
793+
.build_ptr_to_int(arr_ptr, i64_type, "narr_ptr_i64")
794+
.map_err(|e| format!("NewArray ptrtoint at op{}: {}", op_idx, e))?;
795+
self.splat(ptr_as_i64, "narr_v")
796+
}
797+
798+
/// Path A.4: ArrayLen — extract α (pointer-as-i64) from the
799+
/// vector, inttoptr to a [N+1 x i64] pointer, GEP slot 0, load.
800+
/// Returns the length as a scalar i64 (caller will splat it).
801+
fn emit_array_len(
802+
&self,
803+
arr_v: VectorValue<'ctx>,
804+
op_idx: usize,
805+
) -> Result<IntValue<'ctx>, CodegenError> {
806+
let i64_type = self.ctx.i64_type();
807+
let alpha = self
808+
.builder
809+
.build_extract_element(arr_v, i64_type.const_int(0, false), "alen_a")
810+
.map_err(|e| format!("ArrayLen extract α at op{}: {}", op_idx, e))?;
811+
let alpha_iv = match alpha {
812+
BasicValueEnum::IntValue(iv) => iv,
813+
_ => return Err(format!("ArrayLen ptr not int at op{}", op_idx)),
814+
};
815+
// For opaque pointers, GEP needs the element type. We use a
816+
// single-element pointee `[1 x i64]` to GEP slot 0; the load
817+
// returns the length we wrote at NewArray time.
818+
let one_i64 = i64_type.array_type(1);
819+
let ptr_ty = self.ctx.ptr_type(inkwell::AddressSpace::default());
820+
let ptr = self
821+
.builder
822+
.build_int_to_ptr(alpha_iv, ptr_ty, "alen_ptr")
823+
.map_err(|e| format!("ArrayLen inttoptr at op{}: {}", op_idx, e))?;
824+
let zero32 = self.ctx.i32_type().const_int(0, false);
825+
let len_gep = unsafe {
826+
self.builder
827+
.build_in_bounds_gep(one_i64, ptr, &[zero32, zero32], "alen_gep")
828+
.map_err(|e| format!("ArrayLen gep at op{}: {}", op_idx, e))?
829+
};
830+
let len = self
831+
.builder
832+
.build_load(i64_type, len_gep, "alen_load")
833+
.map_err(|e| format!("ArrayLen load at op{}: {}", op_idx, e))?;
834+
match len {
835+
BasicValueEnum::IntValue(iv) => Ok(iv),
836+
_ => Err(format!("ArrayLen load not int at op{}", op_idx)),
837+
}
838+
}
839+
840+
/// Path A.4: ArrayIndex — extract α (pointer) and the user-given
841+
/// scalar index, GEP to slot `idx + 1` (skipping the length
842+
/// prefix), load the element. Returns the element as a scalar i64.
843+
fn emit_array_index(
844+
&self,
845+
arr_v: VectorValue<'ctx>,
846+
idx_v: VectorValue<'ctx>,
847+
op_idx: usize,
848+
) -> Result<IntValue<'ctx>, CodegenError> {
849+
let i64_type = self.ctx.i64_type();
850+
let arr_alpha = self
851+
.builder
852+
.build_extract_element(arr_v, i64_type.const_int(0, false), "aidx_aptr")
853+
.map_err(|e| format!("ArrayIndex extract α at op{}: {}", op_idx, e))?;
854+
let idx_alpha = self
855+
.builder
856+
.build_extract_element(idx_v, i64_type.const_int(0, false), "aidx_aix")
857+
.map_err(|e| format!("ArrayIndex extract idx α at op{}: {}", op_idx, e))?;
858+
let arr_iv = match arr_alpha {
859+
BasicValueEnum::IntValue(iv) => iv,
860+
_ => return Err(format!("ArrayIndex ptr not int at op{}", op_idx)),
861+
};
862+
let idx_iv = match idx_alpha {
863+
BasicValueEnum::IntValue(iv) => iv,
864+
_ => return Err(format!("ArrayIndex idx not int at op{}", op_idx)),
865+
};
866+
let ptr_ty = self.ctx.ptr_type(inkwell::AddressSpace::default());
867+
let ptr = self
868+
.builder
869+
.build_int_to_ptr(arr_iv, ptr_ty, "aidx_ptr")
870+
.map_err(|e| format!("ArrayIndex inttoptr at op{}: {}", op_idx, e))?;
871+
// Compute slot index = user_idx + 1 (skip the length prefix).
872+
let one = i64_type.const_int(1, false);
873+
let slot = self
874+
.builder
875+
.build_int_add(idx_iv, one, "aidx_slot")
876+
.map_err(|e| format!("ArrayIndex slot calc at op{}: {}", op_idx, e))?;
877+
// Use `i64` as the GEP element type — equivalent to "i64*"
878+
// arithmetic. Each step is sizeof(i64) = 8 bytes.
879+
let elem_gep = unsafe {
880+
self.builder
881+
.build_in_bounds_gep(i64_type, ptr, &[slot], "aidx_gep")
882+
.map_err(|e| format!("ArrayIndex gep at op{}: {}", op_idx, e))?
883+
};
884+
let val = self
885+
.builder
886+
.build_load(i64_type, elem_gep, "aidx_load")
887+
.map_err(|e| format!("ArrayIndex load at op{}: {}", op_idx, e))?;
888+
match val {
889+
BasicValueEnum::IntValue(iv) => Ok(iv),
890+
_ => Err(format!("ArrayIndex load not int at op{}", op_idx)),
891+
}
892+
}
893+
684894
/// Session F intrinsic: replace the β lane of a `<2 x i64>`
685895
/// vector value with the phi-shadow of α.
686896
///
Lines changed: 149 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,149 @@
1+
//! Path A.4 — read-only array support in the dual-band JIT.
2+
//!
3+
//! Arrays are represented as `alloca [N+1 x i64]` allocations in the
4+
//! fn's stack frame. Slot 0 holds the length; slots 1..=N hold the
5+
//! elements. Self-describing — ArrayLen needs no side-channel.
6+
//!
7+
//! On the operand stack, an array is the pointer cast to i64
8+
//! (ptrtoint at NewArray, inttoptr at use). This fits the existing
9+
//! Vec<VectorValue> stack convention without needing a typed enum.
10+
//!
11+
//! Out of scope for Path A.4 MVP:
12+
//! - ArrayIndexAssign (mutable writes)
13+
//! - Dynamic resize
14+
//! - Returning arrays from JIT'd fns (caller-facing signature is i64)
15+
//! - Multi-dimensional / nested arrays
16+
//!
17+
//! These are the next sessions' work. The MVP unlocks any pure-int OMC
18+
//! fn that builds an array, reads from it, and returns a scalar.
19+
20+
#![cfg(feature = "llvm-jit")]
21+
22+
use inkwell::context::Context;
23+
use omnimcode_codegen::JitContext;
24+
use omnimcode_core::parser::Parser;
25+
26+
#[test]
27+
fn jit_array_len_returns_correct_length() {
28+
let source = r#"
29+
fn arr5_len(unused) {
30+
h arr = [10, 20, 30, 40, 50];
31+
return arr_len(arr);
32+
}
33+
"#;
34+
let mut parser = Parser::new(source);
35+
let statements = parser.parse().expect("parse");
36+
let module = omnimcode_core::compiler::compile_program(&statements).expect("compile");
37+
let ctx = Context::create();
38+
let jit = JitContext::new(&ctx).expect("jit");
39+
let jitted = jit.jit_module(&module).expect("jit_module");
40+
let f = jitted.get("arr5_len").expect("arr5_len JIT'd");
41+
assert_eq!(f.call(&[0]).expect("call"), 5);
42+
}
43+
44+
#[test]
45+
fn jit_array_index_reads_correct_element() {
46+
let source = r#"
47+
fn arr5_at(idx) {
48+
h arr = [10, 20, 30, 40, 50];
49+
return arr_get(arr, idx);
50+
}
51+
"#;
52+
let mut parser = Parser::new(source);
53+
let statements = parser.parse().expect("parse");
54+
let module = omnimcode_core::compiler::compile_program(&statements).expect("compile");
55+
let ctx = Context::create();
56+
let jit = JitContext::new(&ctx).expect("jit");
57+
let jitted = jit.jit_module(&module).expect("jit_module");
58+
let f = jitted.get("arr5_at").expect("arr5_at JIT'd");
59+
assert_eq!(f.call(&[0]).expect("call"), 10);
60+
assert_eq!(f.call(&[1]).expect("call"), 20);
61+
assert_eq!(f.call(&[2]).expect("call"), 30);
62+
assert_eq!(f.call(&[3]).expect("call"), 40);
63+
assert_eq!(f.call(&[4]).expect("call"), 50);
64+
}
65+
66+
#[test]
67+
fn jit_array_sum_in_loop() {
68+
// The headline workload: sum the elements of a small array.
69+
// Exercises NewArray + ArrayLen + ArrayIndex inside a while loop.
70+
let source = r#"
71+
fn sum_arr(unused) {
72+
h arr = [1, 2, 3, 4, 5, 6, 7, 8, 9, 10];
73+
h sum = 0;
74+
h k = 0;
75+
while k < arr_len(arr) {
76+
sum = sum + arr_get(arr, k);
77+
k = k + 1;
78+
}
79+
return sum;
80+
}
81+
"#;
82+
let mut parser = Parser::new(source);
83+
let statements = parser.parse().expect("parse");
84+
let module = omnimcode_core::compiler::compile_program(&statements).expect("compile");
85+
let ctx = Context::create();
86+
let jit = JitContext::new(&ctx).expect("jit");
87+
let jitted = jit.jit_module(&module).expect("jit_module");
88+
let f = jitted.get("sum_arr").expect("sum_arr JIT'd");
89+
assert_eq!(f.call(&[0]).expect("call"), 55); // 1+2+...+10
90+
}
91+
92+
#[test]
93+
fn jit_array_via_dispatch_hook() {
94+
// End-to-end through Interpreter dispatch (matches CLI's
95+
// OMC_HBIT_JIT=1 path). Verifies arrays survive the JIT round-
96+
// trip when called from the user-facing tree-walk.
97+
use omnimcode_codegen::JittedFn;
98+
use omnimcode_core::interpreter::Interpreter;
99+
use omnimcode_core::value::{HInt, Value};
100+
use std::collections::HashMap;
101+
use std::rc::Rc;
102+
103+
let source = r#"
104+
fn sum_arr(unused) {
105+
h arr = [100, 200, 300];
106+
h sum = 0;
107+
h k = 0;
108+
while k < arr_len(arr) {
109+
sum = sum + arr_get(arr, k);
110+
k = k + 1;
111+
}
112+
return sum;
113+
}
114+
h result = sum_arr(0);
115+
"#;
116+
let mut parser = Parser::new(source);
117+
let statements = parser.parse().expect("parse");
118+
let module = omnimcode_core::compiler::compile_program(&statements).expect("compile");
119+
let ctx = Context::create();
120+
let jit = JitContext::new(&ctx).expect("jit");
121+
let jitted_map = jit.jit_module(&module).expect("jit_module");
122+
assert!(
123+
jitted_map.contains_key("sum_arr"),
124+
"sum_arr should JIT (uses NewArray, ArrayLen, ArrayIndex)"
125+
);
126+
let jitted_for_hook: HashMap<String, JittedFn> = jitted_map.clone();
127+
let dispatch: omnimcode_core::interpreter::JitDispatch = Rc::new(
128+
move |name: &str, args: &[Value]| {
129+
let jf = jitted_for_hook.get(name)?;
130+
if args.len() != jf.arity {
131+
return None;
132+
}
133+
let mut int_args = Vec::with_capacity(args.len());
134+
for a in args {
135+
match a {
136+
Value::HInt(h) => int_args.push(h.value),
137+
Value::Bool(b) => int_args.push(if *b { 1 } else { 0 }),
138+
_ => return None,
139+
}
140+
}
141+
jf.call(&int_args).map(|r| Ok(Value::HInt(HInt::new(r))))
142+
},
143+
);
144+
let mut interp = Interpreter::new();
145+
interp.set_jit_dispatch(Some(dispatch));
146+
interp.execute(statements).expect("exec");
147+
let r = interp.get_var_for_testing("result").expect("result");
148+
assert_eq!(r.to_int(), 600);
149+
}

0 commit comments

Comments
 (0)