Skip to content

Commit 12f63c5

Browse files
committed
Assert proposal number
Signed-off-by: Nicholas Gates <nick@nickgates.com>
1 parent 69485dc commit 12f63c5

1 file changed

Lines changed: 31 additions & 31 deletions

File tree

proposals/0019-array-execution.md

Lines changed: 31 additions & 31 deletions
Original file line numberDiff line numberDiff line change
@@ -36,11 +36,9 @@ fn reduce_parent(
3636
) -> VortexResult<Option<ArrayRef>>;
3737

3838
// Execution. May read data buffers.
39-
fn execute(
40-
array: &Self::Array, ctx: &mut ExecutionCtx,
41-
) -> VortexResult<ExecutionStep>;
39+
fn execute(array: &Self::Array) -> VortexResult<ExecutionStep>;
4240
fn execute_parent(
43-
array: &Self::Array, parent: &ArrayRef, child_idx: usize, ctx: &mut ExecutionCtx,
41+
array: &Self::Array, parent: &ArrayRef, child_idx: usize,
4442
) -> VortexResult<Option<ArrayRef>>;
4543
```
4644

@@ -50,8 +48,13 @@ pub enum ExecutionStep {
5048
/// then call execute on this array again.
5149
ExecuteChild(usize),
5250

51+
/// Columnarize the child at this index (canonicalize without
52+
/// cross-step optimization or execute_parent), replace it,
53+
/// then call execute on this array again.
54+
ColumnarizeChild(usize),
55+
5356
/// Execution is complete.
54-
Done(Columnar),
57+
Done(ArrayRef),
5558
}
5659
```
5760

@@ -65,7 +68,9 @@ control to the scheduler.
6568

6669
- `ExecuteChild(i)` asks the scheduler to execute child `i` to columnar, replace it, and call
6770
`execute` again.
68-
- `Done(columnar)` returns the final columnar result.
71+
- `ColumnarizeChild(i)` asks the scheduler to columnarize child `i` (canonicalize without
72+
cross-step optimization or execute_parent), replace it, and call `execute` again.
73+
- `Done(result)` returns the final result.
6974

7075
**execute_parent** returns `Option<ArrayRef>`. `None` means the child can't handle this parent.
7176
`Some(result)` means it handled the parent — the result can be in **any encoding**, not just
@@ -115,14 +120,19 @@ fn execute_to_columnar(root: ArrayRef, ctx: &mut ExecutionCtx) -> VortexResult<C
115120
}
116121

117122
// Execute
118-
match current.vtable().execute(&current, ctx)? {
123+
match current.vtable().execute(&current)? {
119124
ExecutionStep::ExecuteChild(i) => {
120125
let child = current.child(i);
121126
stack.push((current, i));
122127
current = optimize_recursive(child, ctx)?;
123128
}
129+
ExecutionStep::ColumnarizeChild(i) => {
130+
let child = current.child(i);
131+
stack.push((current, i));
132+
current = child;
133+
}
124134
ExecutionStep::Done(result) => {
125-
current = result.into_array();
135+
current = result;
126136
}
127137
}
128138
}
@@ -179,11 +189,11 @@ We use option 3. The cache is dropped when the `ExecutionCtx` is dropped.
179189
**DictArray** — execute codes into Primitive, then gather:
180190

181191
```rust
182-
fn execute(dict: &DictArray, ctx: &mut ExecutionCtx) -> VortexResult<ExecutionStep> {
192+
fn execute(dict: &DictArray) -> VortexResult<ExecutionStep> {
183193
let Some(codes) = dict.codes().as_opt::<PrimitiveVTable>() else {
184194
return Ok(ExecutionStep::ExecuteChild(0));
185195
};
186-
let gathered = gather(dict.values(), codes, ctx)?;
196+
let gathered = gather(dict.values(), codes)?;
187197
Ok(ExecutionStep::Done(gathered))
188198
}
189199
```
@@ -195,13 +205,13 @@ runs.
195205
**ScalarFnArray** — columnarize children left-to-right, then evaluate:
196206

197207
```rust
198-
fn execute(sfn: &ScalarFnArray, ctx: &mut ExecutionCtx) -> VortexResult<ExecutionStep> {
208+
fn execute(sfn: &ScalarFnArray) -> VortexResult<ExecutionStep> {
199209
for (i, child) in sfn.children().iter().enumerate() {
200210
if child.as_opt::<AnyColumnar>().is_none() {
201211
return Ok(ExecutionStep::ExecuteChild(i));
202212
}
203213
}
204-
let result = sfn.scalar_fn().execute(sfn.columnar_children(), ctx)?;
214+
let result = sfn.scalar_fn().execute(sfn.columnar_children())?;
205215
Ok(ExecutionStep::Done(result))
206216
}
207217
```
@@ -212,20 +222,20 @@ when choosing input order — the first input is executed first.
212222
**FilterArray** — columnarize child, then apply mask:
213223

214224
```rust
215-
fn execute(filter: &FilterArray, ctx: &mut ExecutionCtx) -> VortexResult<ExecutionStep> {
225+
fn execute(filter: &FilterArray) -> VortexResult<ExecutionStep> {
216226
let Some(child) = filter.child().as_opt::<AnyCanonical>() else {
217227
return Ok(ExecutionStep::ExecuteChild(0));
218228
};
219229
let filtered = filter.mask().apply_to(child.into())?;
220-
Ok(ExecutionStep::Done(Columnar::Canonical(filtered)))
230+
Ok(ExecutionStep::Done(filtered.into_array()))
221231
}
222232
```
223233

224234
**BitPacked** — leaf, decompresses directly:
225235

226236
```rust
227-
fn execute(bp: &BitPackedArray, ctx: &mut ExecutionCtx) -> VortexResult<ExecutionStep> {
228-
Ok(ExecutionStep::Done(Columnar::Canonical(Canonical::Primitive(unpack(bp)?))))
237+
fn execute(bp: &BitPackedArray) -> VortexResult<ExecutionStep> {
238+
Ok(ExecutionStep::Done(unpack(bp)?.into_array()))
229239
}
230240
```
231241

@@ -246,23 +256,13 @@ inspect the tree after each step. If an encoding the exporter cares about become
246256
(DictArray for DuckDB dictionary vectors, FSST for DuckDB FSST vectors), the exporter
247257
intercepts it without decompressing.
248258

249-
### Removing ExecutionCtx from VTable methods
250-
251-
The `execute` and `execute_parent` signatures shown above accept `&mut ExecutionCtx`. This gives
252-
encodings the ability to recursively execute children, bypassing the scheduler's caching and
253-
cross-step optimization. Nothing in the type system prevents it.
254-
255-
A stronger design: remove `ExecutionCtx` from the VTable method signatures entirely. The
256-
scheduler owns the execution state (cache, tracing). `execute` receives no context. The method signature itself
257-
communicates "return a step, don't execute anything."
259+
### No ExecutionCtx in VTable methods
258260

259-
This also eliminates the current ergonomic friction of
260-
`let ctx = session.create_execution_ctx(); array.execute(&mut ctx)` — callers just call the
261-
scheduler directly.
261+
`execute` and `execute_parent` do not receive `ExecutionCtx`. The scheduler owns all execution
262+
state (cache, tracing). The method signature itself communicates "return a step, don't execute
263+
anything" — encodings cannot recursively execute children because they have no mechanism to do so.
262264

263-
If `execute_parent` also yields `ExecutionStep` (see unresolved questions), the same argument
264-
applies: it gets resource access but not execution power. The scheduler is the only code that
265-
drives execution.
265+
Callers invoke the scheduler directly rather than calling `array.execute(&mut ctx)`.
266266

267267
### Decompress-into-buffer
268268

0 commit comments

Comments
 (0)