Skip to content

Add nightly #935

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
yihozhang merged 5 commits into from
Jul 6, 2026
Merged

Add nightly #935

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
5 commits
Select commit Hold shift + click to select a range
5bb42af
nightly
yihozhang Jun 25, 2026
fe240e6
Render nightly benchmark report with eval-live
oflatt Jun 25, 2026
f99482a
Merge pull request #936 from oflatt-claude/nightly-eval-live
yihozhang Jun 25, 2026
15888b8
nightly: install deps into a venv to satisfy PEP 668 (#939)
oflatt-claude Jun 29, 2026
7f11137
nightly: prefer the rustup cargo shim over the system cargo (#942)
oflatt-claude Jun 29, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 4 additions & 0 deletions .gitignore
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -9,6 +9,8 @@ pkg/
flamegraph.svg
perf.data*
profile.json
nightly/output/
nightly/.venv/
/*.json
/*.txt

Expand All @@ -24,3 +26,5 @@ scripts/compiled
tests/*.json

lcov.info

__pycache__/
1 change: 1 addition & 0 deletions CHANGELOG.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,7 @@

## [Unreleased] - ReleaseDate

- Add `make nightly` and `scripts/nightly_bench.py`, a hyperfine-based benchmark harness that measures every `tests/**/*.egg` program at 1/2/4/8 threads and (where supported) in proof-testing mode, caps each run at a 2-minute timeout, skips sub-50ms programs, and emits an HTML dashboard (one row per benchmark, one column per configuration) for nightly.cs.washington.edu. The dashboard uses [eval-live](https://github.com/oflatt/eval-live) for interactive filtering and sorting.
- Add typed `EGraph` extension state that clones with `EGraph` and is restored by `push`/`pop`.
- Report full source file paths in egglog span and error messages.
- Fix seminaive matching after nested containers rebuild in place by propagating dirty container ids through parent containers.
Expand Down
12 changes: 11 additions & 1 deletion Makefile
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
.PHONY: all test nits docs graphs rm-graphs doctest coverage insta-test fixnits
.PHONY: all test nits docs graphs rm-graphs doctest coverage insta-test fixnits nightly

RUST_SRC=$(shell find . -type f -wholename '*/src/*.rs' -or -name 'Cargo.toml')
TESTS=$(shell find tests/ -type f -name '*.egg' -not -name '*repro-*')
Expand All @@ -7,6 +7,16 @@ WWW=${PWD}/target/www

all: test nits docs

# Build egglog and benchmark every tests/*.egg file with hyperfine, writing an
# HTML dashboard to nightly/output/ (matching `report=` in nightly-conf).
# Run nightly on nightly.cs.washington.edu. Dependencies install into a venv so
# this works on PEP 668 externally-managed systems; eval_live must be importable
# by the script's interpreter, so run it with the venv's python.
nightly:
python3 -m venv nightly/.venv
nightly/.venv/bin/pip install -q -r scripts/requirements.txt
nightly/.venv/bin/python scripts/nightly_bench.py

test: doctest
cargo insta test --test-runner nextest --release --workspace --unreferenced reject

Expand Down
Loading
Loading