Skip to content

feat!: Support generic comptime functions#1630

Open
mark-koch wants to merge 5 commits into
mainfrom
mk/generic-comptime
Open

feat!: Support generic comptime functions#1630
mark-koch wants to merge 5 commits into
mainfrom
mk/generic-comptime

Conversation

@mark-koch
Copy link
Copy Markdown
Collaborator

@mark-koch mark-koch commented Apr 2, 2026

Closes #896

BREAKING CHANGE: TracedFunctionDef is no longer a CompilableDef, but now a CheckableGenericDef that returns a monomorphic TracedMonoFunctionDef after checking
BREAKING CHANGE: trace_function now requires the generic instantiation as an extra argument

@mark-koch mark-koch requested a review from a team as a code owner April 2, 2026 14:13
@mark-koch mark-koch requested a review from acl-cqc April 2, 2026 14:13
@github-actions
Copy link
Copy Markdown
Contributor

github-actions Bot commented Apr 2, 2026

🐰 Bencher Report

Branchmk/generic-comptime
TestbedLinux
Click to view all benchmark results
BenchmarkLatencyBenchmark Result
microseconds (µs)
(Result Δ%)
Upper Boundary
microseconds (µs)
(Limit %)
tests/benchmarks/test_big_array.py::test_big_array_check📈 view plot
🚷 view threshold
498,346.28 µs
(-38.39%)Baseline: 808,929.89 µs
849,376.38 µs
(58.67%)
tests/benchmarks/test_big_array.py::test_big_array_compile📈 view plot
🚷 view threshold
1,357,856.57 µs
(-13.27%)Baseline: 1,565,544.88 µs
1,643,822.13 µs
(82.60%)
tests/benchmarks/test_big_array.py::test_big_array_executable📈 view plot
🚷 view threshold
6,267,170.48 µs
(-10.07%)Baseline: 6,969,038.29 µs
7,317,490.20 µs
(85.65%)
tests/benchmarks/test_ctrl_flow.py::test_many_ctrl_flow_check📈 view plot
🚷 view threshold
76,847.48 µs
(-22.15%)Baseline: 98,714.66 µs
103,650.39 µs
(74.14%)
tests/benchmarks/test_ctrl_flow.py::test_many_ctrl_flow_compile📈 view plot
🚷 view threshold
142,670.06 µs
(-22.44%)Baseline: 183,939.09 µs
193,136.05 µs
(73.87%)
tests/benchmarks/test_ctrl_flow.py::test_many_ctrl_flow_executable📈 view plot
🚷 view threshold
723,402.41 µs
(-33.61%)Baseline: 1,089,615.34 µs
1,144,096.11 µs
(63.23%)
tests/benchmarks/test_prelude.py::test_import_guppy📈 view plot
🚷 view threshold
34.76 µs
(-17.11%)Baseline: 41.94 µs
44.03 µs
(78.95%)
🐰 View full continuous benchmarking report in Bencher

@hugrbot
Copy link
Copy Markdown
Collaborator

hugrbot commented Apr 2, 2026

This PR contains breaking changes to the public Python API.

Breaking changes summary
guppylang-internals/src/guppylang_internals/tracing/function.py:71: trace_function(node):
Positional parameter was moved
Details: position: from 4 to 5 (+1)

guppylang-internals/src/guppylang_internals/tracing/function.py:71: trace_function(func_def):
Positional parameter was moved
Details: position: from 5 to 6 (+1)

guppylang-internals/src/guppylang_internals/tracing/function.py:71: trace_function(generic_args):
Parameter was added as required

guppylang-internals/src/guppylang_internals/definition/traced.py:0: TracedFunctionDef.compile_outer:
Public object was removed

guppylang-internals/src/guppylang_internals/definition/traced.py:0: CompiledTracedFunctionDef.__init__(func_def):
Positional parameter was moved
Details: position: from 6 to 7 (+1)

guppylang-internals/src/guppylang_internals/definition/traced.py:0: CompiledTracedFunctionDef.__init__(generic_args):
Parameter was added as required


@codecov-commenter
Copy link
Copy Markdown

codecov-commenter commented Apr 2, 2026

Codecov Report

❌ Patch coverage is 83.01887% with 9 lines in your changes missing coverage. Please review.
✅ Project coverage is 92.82%. Comparing base (85e42d1) to head (a31c69e).

Files with missing lines Patch % Lines
...ernals/src/guppylang_internals/tracing/function.py 76.92% 9 Missing ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##             main    #1630      +/-   ##
==========================================
- Coverage   92.87%   92.82%   -0.05%     
==========================================
  Files         136      136              
  Lines       13167    13206      +39     
==========================================
+ Hits        12229    12259      +30     
- Misses        938      947       +9     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@github-actions
Copy link
Copy Markdown
Contributor

github-actions Bot commented Apr 2, 2026

🐰 Bencher Report

Branchmk/generic-comptime
TestbedLinux
Click to view all benchmark results
Benchmarkhugr_bytesBenchmark Result
bytes x 1e3
(Result Δ%)
Upper Boundary
bytes x 1e3
(Limit %)
hugr_nodesBenchmark Result
nodes
(Result Δ%)
Upper Boundary
nodes
(Limit %)
tests/benchmarks/test_big_array.py::test_big_array_compile📈 view plot
🚷 view threshold
158.77 x 1e3
(0.00%)Baseline: 158.77 x 1e3
160.36 x 1e3
(99.01%)
📈 view plot
🚷 view threshold
6,641.00
(0.00%)Baseline: 6,641.00
6,707.41
(99.01%)
tests/benchmarks/test_ctrl_flow.py::test_many_ctrl_flow_compile📈 view plot
🚷 view threshold
27.54 x 1e3
(0.00%)Baseline: 27.54 x 1e3
27.81 x 1e3
(99.01%)
📈 view plot
🚷 view threshold
1,074.00
(0.00%)Baseline: 1,074.00
1,084.74
(99.01%)
tests/benchmarks/test_queue_push_pop.py::test_queue_push_benchmark_compile📈 view plot
🚷 view threshold
10.91 x 1e3
(0.00%)Baseline: 10.91 x 1e3
11.02 x 1e3
(99.01%)
📈 view plot
🚷 view threshold
308.00
(0.00%)Baseline: 308.00
311.08
(99.01%)
tests/benchmarks/test_queue_push_pop.py::test_queue_push_pop_benchmark_compile📈 view plot
🚷 view threshold
14.84 x 1e3
(0.00%)Baseline: 14.84 x 1e3
14.99 x 1e3
(99.01%)
📈 view plot
🚷 view threshold
435.00
(0.00%)Baseline: 435.00
439.35
(99.01%)
🐰 View full continuous benchmarking report in Bencher

@mark-koch mark-koch changed the title feat: Support generic comptime functions feat!: Support generic comptime functions Jun 8, 2026
@codspeed-hq
Copy link
Copy Markdown

codspeed-hq Bot commented Jun 8, 2026

Merging this PR will not alter performance

✅ 9 untouched benchmarks


Comparing mk/generic-comptime (a31c69e) with main (85e42d1)

Open in CodSpeed

Copy link
Copy Markdown
Contributor

@acl-cqc acl-cqc left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks great, Mark, lots of nice examples in tests :). Comments relate only to comments/docs in the code really although I am nervous about the __closure__, __code__ and co_freevars bit....is there any alternative there?

T = guppy.type_var("T", copyable=True, droppable=True)

@guppy.comptime
def foo(x: T @ comptime) -> T:
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I admit I'm a bit confused here as to whether the @ comptime applied to the T here or the x or indeed whether it's necessary at all?

Copy link
Copy Markdown
Collaborator Author

@mark-koch mark-koch Jun 8, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It applies to the x. It says that foo takes an input x of type T whose value must be known at compile-time. This in turn is syntactic sugar for a generic function

@guppy.comptime
def foo[T: (Copy, Drop), x: T]() -> T: ...


@guppy.comptime
def foo(self) -> int:
assert isinstance(n, int)
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Presumably this is not in any sense "required" but we are just checking that it works? (Which is good to check - i.e. that we are able to access n during comptime execution)

assert isinstance(n, int)
assert isinstance(m, int)
assert isinstance(k, int)
return n + m + k
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
return n + m + k
return xs[k] + ys[k]

generic_values = {
x: const_argument_to_python_value(arg)
for x, arg in generic_args.items()
# TODO: We don't have a comptime representation of types yet, so we can only
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This "representation of types" that we need is not a Hugr representation, right, just a python representation to provide at comptime?

In which case, we do have one: the same representation as in the rest of the compiler, namely guppylang_internals.tys...... So we could expose that. I'm not saying we should**, but maybe clarify the comment ("public representation" perhaps, or say something about not wanting to expose guppylang-internals stuff)

** I mean, how much guppylang-internals to expose, is a subjective question, but I'm not going to push you one way or the other. We do in a few places I think but not many.

return self.ty.params

def check(self, type_args: Inst, globals: Globals) -> "TracedMonoFunctionDef":
"""Type checks the body of the function."""
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
"""Type checks the body of the function."""
"""Type checks (and thus traces/executes) the body of the function."""

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actually I'm not sure it does trace/execute?? That only happens in compile? In which case are we really doing "check the body" at all?

call = dfg.builder.call(
self.func_def, *args, instantiation=func_ty, type_args=hugr_type_args
)
call = dfg.builder.call(self.func_def, *args)
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So...just checking...the ht.FunctionType has been compiled into self.func_def, right?

val = generic_values.pop(inp.name)
else:
# Function inputs are only allowed to be mutable if they are borrowed.
# For owned arguments, mutation wouldn't be observable by the caller,
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

(driveby?) Not sure I follow this. The mutation wouldn't be observable by the caller but only because the caller has lost/handed over the object that was mutated, so that respects python semantics (in that we have statically disallowed programs in which they'd be able to see the mutations!)

if x in generic_values:
f.__closure__[i].cell_contents = generic_values.pop(x)

# If all values have been assigned, we can stop here
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: I think all the code below is fine (does nothing) if generic_values are empty... is it worth this? (It's nice to have only one yield)



@contextmanager
def insert_generic_args(
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

how about add_generic_args_to_globals or ...to_function_globals? That'd make the callsite a lot clearer (I was thinking it was gonna prepend them to an existing list of generic args or something)

instantiation of generic parameters."""
# The values of Python 3.12+ style generics are generated using an annotation scope
# (see https://docs.python.org/3/reference/compound_stmts.html#generic-functions).
# Thus, their values will be bound in the `__closure__` table of the function and
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Argh. This is by far the nastiest bit. I note there is no python version check, either, which might make me feel slightly happier. I didn't see this from the link you give just above, is that the best reference?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Support generic comptime functions

4 participants