Skip to content

Commit a554d83

Browse files
committed
ci: harden release gate required check matching
1 parent 7c60dab commit a554d83

4 files changed

Lines changed: 517 additions & 109 deletions

File tree

.claude/hooks/main-ci-status.sh

Lines changed: 23 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -158,32 +158,49 @@ checks_jsonl=$(gh api \
158158
# that each define a `detect-changes` job). Mirrors the Ruby dedup in
159159
# `validate_main_ci_status!` (rakelib/release.rake). Keep the two in sync.
160160
checks_json=$(echo "${checks_jsonl}" | jq -s '
161-
[.[] | {id, name, status, conclusion, html_url, suite_id: (.check_suite.id // .id)}]
162-
| group_by([.suite_id, .name])
161+
[.[] | {id, name, status, conclusion, html_url, suite_id: (.check_suite.id // .id), app_id: (.app.id // null)}]
162+
| group_by([.suite_id, .name, .app_id])
163163
| map(max_by(.id))
164164
' 2>/dev/null) || fail_open "jq slurp failed"
165165

166166
required_json=$(gh api \
167167
"repos/${repo_slug}/branches/main/protection/required_status_checks" \
168-
--jq '(.contexts // []) + (.checks // [] | map(.context)) | unique' \
168+
--jq '{contexts: (.contexts // []), checks: (.checks // [] | map({context, app_id}))}' \
169169
2>/dev/null || echo "null")
170170
case "${required_json}" in
171-
\[*\]) ;;
171+
\{*\}) ;;
172172
*) required_json="null" ;;
173173
esac
174174

175175
# Aggregate counts with jq. `success`, `skipped`, `neutral` are all "passing".
176176
# Anything completed with another conclusion is a failure. Anything not yet
177177
# completed is in_progress.
178178
summary=$(echo "${checks_json}" | jq -r --argjson required_names "${required_json}" '
179+
def app_wildcard($app_id): $app_id == null or $app_id == -1;
180+
def check_matches($required):
181+
.name == $required.context and (app_wildcard($required.app_id) or .app_id == $required.app_id);
182+
def required_check_label:
183+
if app_wildcard(.app_id) then .context else "\(.context) (app_id: \(.app_id))" end;
184+
179185
. as $all
180-
| ($all | map(.name)) as $observed_names
181186
| {
182187
total: length,
183188
passed: [.[] | select(.status == "completed" and (.conclusion | IN("success", "skipped", "neutral")))] | length,
184189
failed: [.[] | select(.status == "completed" and (.conclusion | IN("success", "skipped", "neutral") | not))],
185190
in_progress: [.[] | select(.status != "completed")],
186-
missing_required: (if $required_names == null then [] else ($required_names - $observed_names) end)
191+
missing_required: (
192+
if $required_names == null then []
193+
else
194+
# NOTE: commit statuses are not fetched here; this hook is a fail-open
195+
# display tool. The Ruby release gate owns legacy status enforcement.
196+
(($required_names.contexts // [])
197+
| map(select(. as $context | ($all | any(.name == $context) | not))))
198+
+
199+
(($required_names.checks // [])
200+
| map(select(. as $required | ($all | any(. as $run | ($run | check_matches($required))) | not)))
201+
| map(required_check_label))
202+
end
203+
)
187204
}
188205
| "TOTAL=\(.total)",
189206
"PASSED=\(.passed)",

analysis/vm-script-caching-investigation-2026-06-07.md

Lines changed: 44 additions & 37 deletions
Original file line numberDiff line numberDiff line change
@@ -4,11 +4,11 @@ Related issue: [#3282](https://github.com/shakacode/react_on_rails/issues/3282)
44

55
## Verdict
66

7-
| Area | Status | Finding |
8-
|------|--------|---------|
9-
| vm.Script caching benefit | **Not needed** | Execution time dominates; compile overhead is negligible for real workloads |
10-
| Hypothesis from #3282 | Refuted | Expected ~3ms savings not achievable — real compile cost is <15μs even for 500KB code |
11-
| Implementation effort | Not justified | Adds complexity for <0.01% improvement on actual render paths |
7+
| Area | Status | Finding |
8+
| ------------------------- | -------------- | ------------------------------------------------------------------------------------- |
9+
| vm.Script caching benefit | **Not needed** | Execution time dominates; compile overhead is negligible for real workloads |
10+
| Hypothesis from #3282 | Refuted | Expected ~3ms savings not achievable — real compile cost is <15μs even for 500KB code |
11+
| Implementation effort | Not justified | Adds complexity for <0.01% improvement on actual render paths |
1212

1313
## Executive Summary
1414

@@ -19,6 +19,7 @@ Issue #3282 proposed caching pre-compiled `vm.Script` objects to save the ~3ms "
1919
3. **The 3ms observed** was likely execution time, not parse time
2020

2121
Caching `vm.Script` is only beneficial for scripts that:
22+
2223
- Execute in <1μs (trivial expressions)
2324
- Run 100+ times with identical code
2425
- Have minimal actual computation
@@ -36,26 +37,26 @@ The render template in react_on_rails_pro does real work (React rendering, props
3637

3738
### 1. Compilation Cost vs Code Size
3839

39-
| Code Size | Compile Time | Cached Exec | Uncached Exec | Speedup |
40-
|-----------|--------------|-------------|---------------|---------|
41-
| 60 chars | 0.54μs | 0.62μs | 1.37μs | **2.2x** |
42-
| 8 KB | 0.54μs | 38.04μs | 38.54μs | **1.01x** |
43-
| 69 KB | 2.75μs | 400.12μs | 404.00μs | **1.01x** |
44-
| 650 KB | 0.83μs | 2016.46μs | 2016.25μs | **1.00x** |
40+
| Code Size | Compile Time | Cached Exec | Uncached Exec | Speedup |
41+
| --------- | ------------ | ----------- | ------------- | --------- |
42+
| 60 chars | 0.54μs | 0.62μs | 1.37μs | **2.2x** |
43+
| 8 KB | 0.54μs | 38.04μs | 38.54μs | **1.01x** |
44+
| 69 KB | 2.75μs | 400.12μs | 404.00μs | **1.01x** |
45+
| 650 KB | 0.83μs | 2016.46μs | 2016.25μs | **1.00x** |
4546

4647
**Observation**: For scripts with real execution time (38μs+), caching provides <1% improvement because compilation (0.5-3μs) is noise compared to execution.
4748

4849
### 2. Parse Complexity Analysis
4950

5051
Different AST patterns were tested to find maximum parse cost:
5152

52-
| Pattern | Code Size | Compile Time | Size Needed for 1ms Compile |
53-
|---------|-----------|--------------|----------------------------|
54-
| Nested objects | 2.7 KB | 1.0μs | **2.6 MB** |
55-
| Generators | 9 KB | 1.2μs | 7.3 MB |
56-
| Class definitions | 16 KB | 1.4μs | 11.2 MB |
57-
| Regex literals | 29 KB | 1.8μs | 15.4 MB |
58-
| String literals | 111 KB | 3.8μs | 27.6 MB |
53+
| Pattern | Code Size | Compile Time | Size Needed for 1ms Compile |
54+
| ----------------- | --------- | ------------ | --------------------------- |
55+
| Nested objects | 2.7 KB | 1.0μs | **2.6 MB** |
56+
| Generators | 9 KB | 1.2μs | 7.3 MB |
57+
| Class definitions | 16 KB | 1.4μs | 11.2 MB |
58+
| Regex literals | 29 KB | 1.8μs | 15.4 MB |
59+
| String literals | 111 KB | 3.8μs | 27.6 MB |
5960

6061
**Observation**: Even the most expensive parsing pattern (deeply nested objects) requires 2.6 MB of code to reach 1ms compile time. This is unrealistic for render templates.
6162

@@ -64,33 +65,35 @@ Different AST patterns were tested to find maximum parse cost:
6465
How much time is saved across N executions of a trivial script (32 chars)?
6566

6667
| Executions | Cached Total | Uncached Total | Time Saved |
67-
|------------|--------------|----------------|------------|
68-
| 10 | 15μs | 33μs | 18μs |
69-
| 100 | 112μs | 183μs | 71μs |
70-
| 1,000 | 627μs | 1,410μs | 784μs |
71-
| 10,000 | 3,844μs | 11,043μs | **7.2ms** |
68+
| ---------- | ------------ | -------------- | ---------- |
69+
| 10 | 15μs | 33μs | 18μs |
70+
| 100 | 112μs | 183μs | 71μs |
71+
| 1,000 | 627μs | 1,410μs | 784μs |
72+
| 10,000 | 3,844μs | 11,043μs | **7.2ms** |
7273

7374
**Observation**: Need ~10,000 executions of trivial scripts to save 7ms total. The render template runs once per request — no amortization possible.
7475

7576
### 4. Heavy Computation Scripts
7677

7778
Scripts that do real work (object creation, loops, array operations):
7879

79-
| Script Type | Cached/exec | Uncached/exec | Speedup |
80-
|-------------|-------------|---------------|---------|
81-
| Trivial (return 1+1) | 0.37μs | 1.05μs | 2.8x |
82-
| Light (loop 10x) | 0.47μs | 1.17μs | 2.5x |
83-
| Medium (100 sqrt calls) | 12.87μs | 13.80μs | **1.07x** |
84-
| Heavy (1000 objects) | 39.15μs | 39.99μs | **1.02x** |
80+
| Script Type | Cached/exec | Uncached/exec | Speedup |
81+
| ----------------------- | ----------- | ------------- | --------- |
82+
| Trivial (return 1+1) | 0.37μs | 1.05μs | 2.8x |
83+
| Light (loop 10x) | 0.47μs | 1.17μs | 2.5x |
84+
| Medium (100 sqrt calls) | 12.87μs | 13.80μs | **1.07x** |
85+
| Heavy (1000 objects) | 39.15μs | 39.99μs | **1.02x** |
8586

8687
**Observation**: The moment execution time exceeds ~5μs, caching benefit drops to noise levels.
8788

8889
## Why the Original 3ms Estimate Was Wrong
8990

9091
Issue #3282 stated:
92+
9193
> Per-request JS-parse cost: ~3 ms
9294
9395
This measurement likely included:
96+
9497
1. **React component evaluation** (function execution, not parsing)
9598
2. **Props serialization** (JSON stringify/parse overhead)
9699
3. **Context creation** (`vm.createContext` cost, not `vm.Script` compile)
@@ -101,18 +104,20 @@ Our isolated benchmark shows V8 parses 1MB of JavaScript in ~26μs. A 200-byte t
101104
## Scripts That Would Benefit (Not Our Use Case)
102105

103106
Caching helps when:
107+
104108
```js
105109
// Config literals — 2-3x faster
106110
const config = new vm.Script(`({ apiUrl: "...", timeout: 5000 })`);
107111

108-
// Simple expressions — 2-3x faster
112+
// Simple expressions — 2-3x faster
109113
const formula = new vm.Script(`price * quantity * (1 - discount)`);
110114

111115
// Feature flags — 2x faster
112116
const check = new vm.Script(`user.role === "admin"`);
113117
```
114118

115119
Caching does NOT help when:
120+
116121
```js
117122
// Data transformation — 1.05x (no real gain)
118123
const transform = new vm.Script(`
@@ -127,13 +132,13 @@ const render = new vm.Script(`
127132

128133
## Decision Framework
129134

130-
| Condition | Cache vm.Script? |
131-
|-----------|------------------|
132-
| Script execution < 1μs | Yes |
133-
| Script execution > 10μs | No |
134-
| Script runs 1x per request | No |
135-
| Script is pure config/expression | Yes |
136-
| Script does React rendering | **No** |
135+
| Condition | Cache vm.Script? |
136+
| -------------------------------- | ---------------- |
137+
| Script execution < 1μs | Yes |
138+
| Script execution > 10μs | No |
139+
| Script runs 1x per request | No |
140+
| Script is pure config/expression | Yes |
141+
| Script does React rendering | **No** |
137142

138143
## Recommendation
139144

@@ -147,13 +152,15 @@ The optimization targets the wrong layer. V8's JIT compiler is not the bottlenec
147152
4. Reducing React reconciliation work
148153

149154
Adding `vm.Script` caching would:
155+
150156
- Add code complexity (cache invalidation, memory management)
151157
- Provide <0.01% improvement on real workloads
152158
- Create false confidence that "we optimized the VM layer"
153159

154160
## Artifacts
155161

156162
Benchmark scripts and raw data available at:
163+
157164
- Original experiment: `vm-script-cache/tmp/experiments/20260607T083551Z-vm-script-cache/`
158165

159166
### Raw Benchmark Output (Summary)

0 commit comments

Comments
 (0)