Skip to content

BE-615: HashQL: Introduce the concept of synthetic closures in the MIR#8894

Open
indietyp wants to merge 8 commits into
bm/be-607-hashql-remove-smallvec-from-the-standardlibrary-for-modulefrom
bm/be-615-hashql-introduce-synthetic-closures-and-trivial-closure
Open

BE-615: HashQL: Introduce the concept of synthetic closures in the MIR#8894
indietyp wants to merge 8 commits into
bm/be-607-hashql-remove-smallvec-from-the-standardlibrary-for-modulefrom
bm/be-615-hashql-introduce-synthetic-closures-and-trivial-closure

Conversation

@indietyp

Copy link
Copy Markdown
Member

🌟 What is the purpose of this PR?

Introduces Source::Synthetic for compiler-generated wrapper bodies and extends administrative reduction to handle single-operation function bodies (TrivialClosure).

This is groundwork for making intrinsics usable in value position (e.g. passing + as an argument to a higher-order function). The Synthetic source variant will identify wrapper bodies that bridge non-first-class operations into callable closures, and TrivialClosure ensures they are always inlined away.

🔍 What does this change?

  • Adds Source::Synthetic(Symbol) to the MIR Source enum for compiler-synthesized operation wrappers
  • Adds ReductionKind::TrivialClosure to administrative reduction, recognizing single-basic-block bodies where a trivial prelude leads to a single non-call operation (Binary, Unary, Aggregate, Input, Load) whose result is returned
  • Synthetic bodies get InlineDirective::Always (same as Ctor)
  • All downstream Source matches updated: execution pipeline, statement placement, eval context, pretty printer
  • Adds [synthetic sym::path] source to the body! test macro

The TrivialClosure classification is intentionally broader than just synthetic bodies. Any function matching the shape is reducible, which unlocks optimization cascades: single-operation closures get inlined, exposing constant-foldable expressions, enabling branch elimination downstream.

Pre-Merge Checklist 🚀

🚢 Has this modified a publishable library?

This PR:

  • does not modify any publishable blocks or libraries, or modifications do not need publishing

📜 Does this require a change to the docs?

The changes in this PR:

  • are internal and do not require a docs change

🕸️ Does this require a change to the Turbo Graph?

The changes in this PR:

  • do not affect the execution graph

🐾 Next steps

  • Implement qualified_path in reification to generate synthetic wrapper bodies for intrinsics in value position
  • Intrinsic classification (first-classable vs syntactic forms)
  • Call handler optimization to recognize administrative thunk-force patterns and skip throwaway thunk body generation

🛡 What tests cover this?

  • Classification unit tests: classify_trivial_closure_binary, classify_trivial_closure_unary, classify_non_reducible_non_trivial_prelude
  • Inlining snapshot tests: inline_trivial_closure_binary, inline_trivial_closure_transitive
  • Existing compiletest suite (e.g. nested-branch-elimination now demonstrates the full optimization cascade)

❓ How to test this?

cargo test --package hashql-mir --lib -- administrative_reduction::tests

📹 Demo

The nested-branch-elimination compiletest shows the cascade effect: closures containing single comparisons (> 100, < 0) are now inlined by AR, exposing constant-foldable comparisons (50 > 100, 50 < 0), which enables the entire branch structure to collapse to return "in range".

Copilot AI review requested due to automatic review settings June 22, 2026 13:47
@vercel

vercel Bot commented Jun 22, 2026

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated (UTC)
hash Ready Ready Preview, Comment Jul 3, 2026 4:36pm
petrinaut Ready Ready Preview Jul 3, 2026 4:36pm
1 Skipped Deployment
Project Deployment Actions Updated (UTC)
hashdotdesign-tokens Ignored Ignored Preview Jul 3, 2026 4:36pm

@vercel vercel Bot temporarily deployed to Preview – petrinaut June 22, 2026 13:47 Inactive
@cursor

cursor Bot commented Jun 22, 2026

Copy link
Copy Markdown

PR Summary

Medium Risk
Touches MIR classification, inlining, and Postgres codegen paths; behavior is largely covered by snapshots but wrong reduction or join shape could change query results or performance.

Overview
Adds Source::Synthetic(Symbol) for compiler-generated wrappers that turn non-first-class ops into callable closures, with InlineDirective::Always like ctors. Source::Intrinsic now carries an Intrinsic value (IntrinsicId + optimize flag) instead of a bare DefId; built-in DefId dict/list op constants are removed.

Administrative reduction gains ReductionKind::TrivialClosure: single-block bodies whose last statement is a primitive op (Binary, Unary, Aggregate, Input, Load) and return that result are inlined at call sites—not only forwarding Apply wrappers. That drives updated inline/post-inline snapshots (e.g. branch collapse in nested-branch-elimination).

Pipeline Source matches include Synthetic alongside intrinsics in execution placement, eval live-out, and pretty-printing; body! adds [synthetic sym::path] and intrinsic test sources.

Smaller related changes: Pool is allocator-parameterized; core::json types are shared via a public types module; graph::entity::property is typed as unknown() (not Option) pending BE-62; Postgres golden SQL switches entity_editions to CROSS JOIN LATERAL subqueries; eval/postgres .aux.mir fixtures refresh local ordering from lowering.

Reviewed by Cursor Bugbot for commit 489dd1a. Bugbot is set up for automated code reviews on this repo. Configure here.

indietyp commented Jun 22, 2026

Copy link
Copy Markdown
Member Author

Warning

This pull request is not mergeable via GitHub because a downstack PR is open. Once all requirements are satisfied, merge this PR as a stack on Graphite.
Learn more

This stack of pull requests is managed by Graphite. Learn more about stacking.

@codecov

codecov Bot commented Jun 22, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 3.62%. Comparing base (4abefab) to head (489dd1a).

Additional details and impacted files
@@                                         Coverage Diff                                          @@
##           bm/be-607-hashql-remove-smallvec-from-the-standardlibrary-for-module   #8894   +/-   ##
====================================================================================================
  Coverage                                                                  3.62%   3.62%           
====================================================================================================
  Files                                                                       478     478           
  Lines                                                                     14463   14463           
  Branches                                                                   3020    3020           
====================================================================================================
  Hits                                                                        524     524           
  Misses                                                                    13894   13894           
  Partials                                                                     45      45           
Flag Coverage Δ
apps.hash-ai-worker-ts 1.39% <ø> (ø)
apps.hash-api 6.39% <ø> (ø)
local.hash-backend-utils 2.81% <ø> (ø)
local.hash-graph-sdk 10.00% <ø> (ø)
local.hash-isomorphic-utils 0.18% <ø> (ø)

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@indietyp indietyp force-pushed the bm/be-607-hashql-remove-smallvec-from-the-standardlibrary-for-module branch from 5d51d92 to f62be24 Compare July 3, 2026 11:39
@indietyp indietyp force-pushed the bm/be-615-hashql-introduce-synthetic-closures-and-trivial-closure branch from 4ad5098 to 74017c2 Compare July 3, 2026 11:39

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 51 out of 52 changed files in this pull request and generated 6 comments.

Comment thread libs/@local/hashql/core/src/module/std_lib/core/mod.rs
Comment on lines +140 to +142
// In this case it may either be a forwarding closure **or** a synthetic closure, this
// depends on the final statement, in either case the last statement must be an assignment.
let StatementKind::Assign(Assign { lhs, ref rhs }) = final_stmt.kind else {
| RValue::Load(_) => Self::TrivialClosure,
};

// Terminator must return the call's result.
Comment thread libs/@local/hashql/mir/src/builder/body.rs
Comment thread libs/@local/hashql/mir/src/body/mod.rs
Comment thread libs/@local/hashql/mir/src/intrinsic.rs
@indietyp indietyp force-pushed the bm/be-607-hashql-remove-smallvec-from-the-standardlibrary-for-module branch from f62be24 to 558d30c Compare July 3, 2026 14:27
@indietyp indietyp force-pushed the bm/be-615-hashql-introduce-synthetic-closures-and-trivial-closure branch from 74017c2 to 410fa1b Compare July 3, 2026 14:27
Copilot AI review requested due to automatic review settings July 3, 2026 14:28
@indietyp indietyp force-pushed the bm/be-615-hashql-introduce-synthetic-closures-and-trivial-closure branch from 410fa1b to ac8ed9e Compare July 3, 2026 14:28
@indietyp indietyp force-pushed the bm/be-607-hashql-remove-smallvec-from-the-standardlibrary-for-module branch from 558d30c to d08cc92 Compare July 3, 2026 14:28

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 51 out of 52 changed files in this pull request and generated 2 comments.

Comment on lines 27 to 29
impl DefId {
/// Built-in dictionary insert operation (immutable).
///
/// This operation inserts a key-value pair into a dictionary,
/// returning a new dictionary with the added pair. The original
/// dictionary remains unchanged.
pub const DICT_INSERT: Self = Self::new(0xFFFF_FE00);
/// Built-in dictionary insert operation (mutable).
///
/// This operation inserts a key-value pair into a dictionary in-place,
/// modifying the original dictionary. Used for efficient dictionary
/// construction and updates.
pub const DICT_INSERT_MUT: Self = Self::new(0xFFFF_FE01);
/// Built-in dictionary remove operation (immutable).
///
/// This operation removes a key-value pair from a dictionary,
/// returning a new dictionary without the specified key. The
/// original dictionary remains unchanged.
pub const DICT_REMOVE: Self = Self::new(0xFFFF_FE02);
/// Built-in dictionary remove operation (mutable).
///
/// This operation removes a key-value pair from a dictionary in-place,
/// modifying the original dictionary and returning the removed value
/// if the key existed.
pub const DICT_REMOVE_MUT: Self = Self::new(0xFFFF_FE03);
/// Built-in list pop operation (immutable).
///
/// This operation removes the last element from a list, returning
/// both the element and a new list without the element. The original
/// list remains unchanged.
pub const LIST_POP: Self = Self::new(0xFFFF_FE04);
/// Built-in list pop operation (mutable).
///
/// This operation removes the last element from a list in-place,
/// returning the removed element while modifying the original list.
pub const LIST_POP_MUT: Self = Self::new(0xFFFF_FE05);
/// Built-in list push operation (immutable).
///
/// This operation appends an element to a list, returning a new list
/// without modifying the original. Used for functional-style list
/// manipulation where immutability is preferred.
pub const LIST_PUSH: Self = Self::new(0xFFFF_FE06);
/// Built-in list push operation (mutable).
///
/// This operation appends an element to a list in-place, modifying
/// the original list. Used for imperative-style list manipulation
/// where performance is critical.
pub const LIST_PUSH_MUT: Self = Self::new(0xFFFF_FE07);
pub const PLACEHOLDER: Self = Self::MAX;
}
Comment on lines 11 to 17
pub(in crate::module::std_lib) mod bits;
pub(in crate::module::std_lib) mod bool;
pub(in crate::module::std_lib) mod cmp;
pub(in crate::module::std_lib) mod json;
pub mod json;
pub(in crate::module::std_lib) mod math;
pub mod option;
pub(in crate::module::std_lib) mod result;
@indietyp indietyp force-pushed the bm/be-607-hashql-remove-smallvec-from-the-standardlibrary-for-module branch from d08cc92 to 5014ae0 Compare July 3, 2026 14:36
@indietyp indietyp force-pushed the bm/be-615-hashql-introduce-synthetic-closures-and-trivial-closure branch from ac8ed9e to 4aacfd8 Compare July 3, 2026 14:36

@cursor cursor Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cursor Bugbot has reviewed your changes and found 1 potential issue.

Fix All in Cursor

❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.

Reviewed by Cursor Bugbot for commit 4aacfd8. Configure here.

Copilot AI review requested due to automatic review settings July 3, 2026 15:21
@indietyp indietyp force-pushed the bm/be-607-hashql-remove-smallvec-from-the-standardlibrary-for-module branch from 5014ae0 to 8c7419a Compare July 3, 2026 15:21
@indietyp indietyp force-pushed the bm/be-615-hashql-introduce-synthetic-closures-and-trivial-closure branch from 4aacfd8 to f2b45cb Compare July 3, 2026 15:21

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 51 out of 52 changed files in this pull request and generated 3 comments.

Comment on lines +140 to +164
// In this case it may either be a forwarding closure **or** a synthetic closure, this
// depends on the final statement, in either case the last statement must be an assignment.
let StatementKind::Assign(Assign { lhs, ref rhs }) = final_stmt.kind else {
return None;
};

// The kind depends on the final statement. Both cases are beta-reduction
// (substituting arguments into a function body at a call site):
//
// Apply: the body forwards to another call. The wrapper (lambda x. f(x)) is an
// eta-expansion of the inner function; inlining at the call site is beta-reduction
// that exposes the inner call directly.
//
// Binary, Unary, and other primitive operations: the body computes a single result
// from its arguments. Inlining substitutes the arguments and exposes the primitive
// operation. A subsequent constant folding pass may then perform delta-reduction
// (evaluating the primitive when operands are known).
let kind = match rhs {
RValue::Apply(_) => Self::ForwardingClosure,
RValue::Aggregate(_)
| RValue::Binary(_)
| RValue::Unary(_)
| RValue::Input(_)
| RValue::Load(_) => Self::TrivialClosure,
};
Comment on lines 167 to 171
if bb.terminator.kind
== TerminatorKind::Return(Return {
value: Operand::Place(lhs),
})
{
Comment on lines +66 to 73
// `property<T>(entity: Entity<T>, path: JsonPath) -> ?`
// TODO(BE-62): return `Option<?>` once pattern matching allows for option destructuring, to
// allow for proper comparison
let decl = decl!(context;
<T>(entity: context.ty.apply([(entity_ty.arguments[0].id, T)], entity_ty.id),
path: json_path_ty.id
) -> option(&context.ty, context.ty.unknown())
) -> context.ty.unknown()
);
@indietyp indietyp force-pushed the bm/be-615-hashql-introduce-synthetic-closures-and-trivial-closure branch from f2b45cb to 489dd1a Compare July 3, 2026 16:24
@indietyp indietyp force-pushed the bm/be-607-hashql-remove-smallvec-from-the-standardlibrary-for-module branch from 8c7419a to 4abefab Compare July 3, 2026 16:24
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area/libs Relates to first-party libraries/crates/packages (area) area/tests New or updated tests type/eng > backend Owned by the @backend team

Development

Successfully merging this pull request may close these issues.

2 participants