Skip to content

Ci benchmark workflows#28

Merged
brnrdog merged 4 commits into
mainfrom
ci-benchmark-workflows
Apr 21, 2026
Merged

Ci benchmark workflows#28
brnrdog merged 4 commits into
mainfrom
ci-benchmark-workflows

Conversation

@brnrdog

@brnrdog brnrdog commented Apr 21, 2026

Copy link
Copy Markdown
Owner

No description provided.

@brnrdog brnrdog marked this pull request as ready for review April 21, 2026 17:32
@brnrdog brnrdog merged commit e2bee28 into main Apr 21, 2026
4 checks passed
@brnrdog brnrdog deleted the ci-benchmark-workflows branch April 21, 2026 17:32
@github-actions

Copy link
Copy Markdown

ReScript Signals benchmark: PR vs main

Compared implementations:

  • ReScript Signals (main)
  • ReScript Signals (PR)

Overall:

Version Total ms Avg ms/test
ReScript Signals (main) 17575.19 878.76
ReScript Signals (PR) 18012.50 900.62
Delta (ReScript Signals (PR) - ReScript Signals (main)) 437.31 2.49%

Per-test delta (lower is better):

Test Main ms PR ms Diff ms Diff %
2-10x5 - lazy80% 708.02 769.48 61.46 8.68%
25-1000x5 4811.98 4973.60 161.63 3.36%
3-5x500 358.14 387.26 29.12 8.13%
4-1000x12 - dyn5% 7275.85 7364.27 88.42 1.22%
6-100x15 - dyn50% 515.37 539.13 23.76 4.61%
6-10x10 - dyn25% - lazy80% 434.64 452.80 18.15 4.18%
avoidablePropagation 386.97 386.37 -0.59 -0.15%
broadPropagation 332.05 342.18 10.13 3.05%
cellx1000 22.69 21.95 -0.74 -3.27%
cellx2500 58.43 59.16 0.73 1.25%
createComputations 371.51 307.89 -63.61 -17.12%
createSignals 5.17 4.95 -0.22 -4.22%
deepPropagation 126.93 131.16 4.23 3.34%
diamond 232.27 246.75 14.48 6.23%
molBench 39.67 39.81 0.15 0.37%
mux 416.67 432.81 16.13 3.87%
repeatedObservers 36.90 37.85 0.95 2.57%
triangle 82.87 78.47 -4.40 -5.31%
unstable 48.73 52.34 3.61 7.41%
updateSignals 1310.32 1384.24 73.91 5.64%

Note: single-machine run in CI. Numbers can vary with runner load and Node/V8 version.

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 4886daa71c

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

cat benchmark-results/ci/pr-vs-frameworks/pr-comment-latest.md >> "$GITHUB_STEP_SUMMARY"

- name: Upsert PR benchmark comment
if: github.event_name == 'pull_request'

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Badge Guard PR comment upsert against fork pull requests

This step runs for every pull_request event, but forked PRs get a read-only GITHUB_TOKEN; in that context, the github.rest.issues.*Comment calls fail with 403 and the benchmark job is marked failed even though benchmarking succeeded. Add a fork-aware condition (for example, skip when github.event.pull_request.head.repo.fork is true) or move comment writing to a trusted workflow context; the same pattern is also present in .github/workflows/benchmark-pr-vs-main.yml.

Useful? React with 👍 / 👎.

run: npm run build

- name: Clone benchmark suite
run: git clone --depth=1 https://github.com/milomg/js-reactivity-benchmark.git .tmp/js-reactivity-benchmark

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Pin benchmark suite revision before cloning

The workflow clones milomg/js-reactivity-benchmark from its moving default branch, so upstream changes can break this repo’s PR checks without any change here. That risk is concrete in this commit because scripts/ci/benchmark-pr-vs-frameworks.mjs expects specific framework names and throws if they change. Cloning a fixed tag or commit SHA will make CI results reproducible and prevent flaky external breakages.

Useful? React with 👍 / 👎.

@github-actions

Copy link
Copy Markdown

Reactivity benchmark: PR vs top frameworks

Compared implementations:

  • Alien Signals
  • Preact Signals
  • SolidJS
  • Svelte v5
  • Vue
  • ReScript Signals (PR)

Overall ranking (lower total ms is better):

Rank Framework Total ms Avg ms/test
1 Alien Signals 3602.86 180.14
2 Preact Signals 4060.88 203.04
3 Vue 5608.95 280.45
4 Svelte v5 8098.11 404.91
5 SolidJS 10007.48 500.37
6 ReScript Signals (PR) 17293.06 864.65

Per-test runtime (ms):

Framework 2-10x5 - lazy80% 25-1000x5 3-5x500 4-1000x12 - dyn5% 6-100x15 - dyn50% 6-10x10 - dyn25% - lazy80% avoidablePropagation broadPropagation cellx1000 cellx2500 createComputations createSignals deepPropagation diamond molBench mux repeatedObservers triangle unstable updateSignals
Alien Signals 335.52 656.32 185.93 503.61 282.56 241.10 95.64 200.85 13.51 34.85 97.64 4.13 62.08 127.90 28.38 167.70 32.48 39.31 47.04 446.33
Preact Signals 426.11 894.63 203.57 573.27 296.43 240.51 112.34 211.34 11.57 31.43 94.81 3.72 86.74 141.17 28.29 163.18 22.10 46.53 36.75 436.38
Vue 620.78 985.86 292.99 714.88 396.66 344.56 244.75 254.19 29.30 84.53 147.26 4.97 114.93 211.25 61.78 243.71 28.81 65.42 51.65 710.68
Svelte v5 1135.07 1015.27 338.85 900.57 405.46 373.95 570.40 420.31 18.37 51.07 169.96 3.50 175.72 434.81 30.17 249.66 76.91 124.02 110.25 1493.78
SolidJS 2099.56 1387.72 488.42 1106.77 652.93 682.88 332.67 554.44 27.16 75.31 204.59 5.21 205.18 400.76 35.37 298.09 94.02 137.08 134.08 1085.24
ReScript Signals (PR) 666.95 4824.42 319.91 7265.32 499.54 417.48 331.24 317.33 24.10 64.02 253.79 5.51 94.37 215.95 40.16 396.10 31.00 64.29 41.21 1420.38

Note: single-machine run in CI. Numbers can vary with runner load and Node/V8 version.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant