feat(ci): add entity-registry determinism check — fixes #758 by rafaelscosta · Pull Request #765 · SynkraAI/aiox-core

rafaelscosta · 2026-05-18T11:59:09Z

Summary

New CI step that re-runs `populate-entity-registry.js` on every PR touching the IDS sources and fails if the committed `entity-registry.yaml` diverges from a clean regen output. Closes the category of drift that produced the #754 false alarm.

Why this exists

PR #752 inadvertently committed a registry generated against a partially-edited working tree, with `dependencies: []` for 5 agents and 4 tasks whose markdown sources still declared those deps. The drift was invisible until issue #754 was filed (and then refuted — see #754 closure comment), demonstrating that regen-vs-committed drift can corrupt the registry without anyone noticing until impact analysis goes wrong.

This gate catches that mechanically before merge.

Components

`scripts/validate-registry-determinism.js`: snapshots committed registry → runs populator → reads regen output → ALWAYS restores snapshot in `finally` (working tree preserved regardless of outcome) → strips volatile fields (`lastUpdated`, `lastVerified`, `checksum`) → deep-compares via sorted recursive stringify (so map ordering doesn't yield false positives) → emits actionable per-entry diff on failure.
`package.json`: `npm run validate:registry-determinism` wires the script.
`.github/workflows/ci.yml`: new `registry-determinism` job gated by `changes.outputs.code` (matches `.aiox-core/`, `scripts/`, `bin/`, `packages/`, `src/`, `tools/`). Runtime: ~5 min via existing ubuntu-latest runner. Blocks merge on drift.

Tested locally (both directions)

Clean main (d877231): `npm run validate:registry-determinism` exits 0 with "820 entities, structure matches regen output."
Induced drift: manually zeroed `dependencies` of one task, ran validator. Exits 1 with: `[tasks] analyze-brownfield: dependencies differ` and remediation steps.

The remediation hint output ends with copy-paste-ready commands to fix locally:

```
node .aiox-core/development/scripts/populate-entity-registry.js
git add .aiox-core/data/entity-registry.yaml
git commit -m "chore(ids): regen entity-registry"
```

What this gate does NOT catch

Drift in other generated files (`install-manifest.yaml`, IDE projections) — already covered by `validate:manifest` and `IDE Command Sync Validation`.
Algorithmic regressions in the populator itself — those land via the regular dev/test loop. This gate only verifies deterministic output given the committed source state.

Test plan

Clean local main passes
Induced-drift local failure surfaces specific entry name
Lint clean
Typecheck clean
After merge: confirm new `Entity Registry Determinism` check appears in PR checks UI on next code-touching PR

Issue

Closes #758

🤖 Generated with Claude Code

Summary by CodeRabbit

Tests
- Added an automated registry determinism check to CI that runs during validation, reports results in the job summary, and can fail the overall validation if discrepancies are found.
Chores
- Added a local validation command to package scripts and a CLI validation utility to compare committed and freshly generated registry output for consistency.

Re-runs `populate-entity-registry.js` against the current source tree and diffs the output against the committed `.aiox-core/data/entity-registry.yaml`. Fails the build if any non-volatile fields (dependencies, usedBy, structural) differ between the committed registry and a clean regen. Catches the category of drift that produced #754 (false alarm): a regen run against a partially-edited working tree silently producing a registry that doesn't match what the algorithm would produce on the same source. ## Implementation - `scripts/validate-registry-determinism.js` — snapshots the committed registry, runs the populator (which mutates the file in place), reads back, always restores the snapshot in `finally` so the working tree is preserved regardless of outcome, strips volatile fields (lastUpdated, lastVerified, checksum) before deep-comparing, and emits an actionable error listing the specific entries that diverged on dependencies or usedBy. - `package.json` — `validate:registry-determinism` npm script wired to the new script. - `.github/workflows/ci.yml` — new `registry-determinism` job in the existing `changes.outputs.code` gate (.aiox-core/**, scripts/**, etc), runs in ~5 min, blocks merge on drift. ## Tested locally - Clean main: PASSES (820 entities, structure matches). - Induced drift (manually zeroed deps of one task): FAILS with "[tasks] analyze-brownfield: dependencies differ" and remediation hint. ## What this does NOT catch - Drift inside other generated files (install-manifest.yaml, IDE projections) — covered by `validate:manifest` and `validate:ide-sync` already. - Algorithmic regressions in the populator itself — those should land via the regular dev/test loop; this gate only verifies deterministic output given the committed source state. Closes #758 Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

vercel · 2026-05-18T11:59:11Z

The latest updates on your projects. Learn more about Vercel for GitHub.

Project	Deployment	Actions	Updated (UTC)
aiox-core	Ready	Preview, Comment	May 18, 2026 0:06am

coderabbitai · 2026-05-18T11:59:23Z

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro

Run ID: 7ad63a1d-fb84-4239-8fdb-cef2a9d95148

📥 Commits

Reviewing files that changed from the base of the PR and between 4cfe49f and 1506dd4.

📒 Files selected for processing (1)

.github/workflows/ci.yml

🚧 Files skipped from review as they are similar to previous changes (1)

.github/workflows/ci.yml

Walkthrough

Adds a Node.js validator that re-runs the entity registry populator, strips volatile metadata, and deterministically compares the regenerated registry to the committed .aiox-core/data/entity-registry.yaml; wires the validator into CI via an npm script and a conditional GitHub Actions job that gates validation-summary.

Changes

Registry Determinism Validation

Layer / File(s)	Summary
Registry determinism validator script `scripts/validate-registry-determinism.js`	Implements path constants and volatile field set; helpers to recursively strip volatile metadata and emit standardized failure messages; snapshots and restores the committed registry around an exec of `populate-entity-registry.js`; deterministically compares structures (sorted keys) and reports capped drift details with remediation commands.
CI and npm script integration `package.json`, `.github/workflows/ci.yml`	Adds `validate:registry-determinism` npm script and a new conditionally-run `registry-determinism` GitHub Actions job that installs deps and runs the validator; integrates the job into `validation-summary` and fails the summary when the job result is `failure`.

Estimated code review effort

🎯 3 (Moderate) | ⏱️ ~20 minutes

Possibly related PRs

SynkraAI/aiox-core#753: Overlaps because this PR's validator runs populate-entity-registry.js, which #753 modifies (SCAN_CONFIG changes).
SynkraAI/aiox-core#748: Related because changes to metadata generation (extractPurpose, lastUpdated/lastVerified/checksum) affect generated registry content the validator compares.

Suggested labels

type: test

Suggested reviewers

oalanicolas
Pedrovaleriolopez

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name	Status	Explanation	Resolution
Docstring Coverage	⚠️ Warning	Docstring coverage is 33.33% which is insufficient. The required threshold is 80.00%.	Write docstrings for the functions missing them to satisfy the coverage threshold.

✅ Passed checks (4 passed)

Check name	Status	Explanation
Description Check	✅ Passed	Check skipped - CodeRabbit’s high-level summary is enabled.
Title check	✅ Passed	The title clearly and concisely describes the main change: adding an entity-registry determinism check to CI, and references the fixed issue `#758`.
Linked Issues check	✅ Passed	All code requirements from `#758` are met: the validation script strips volatile fields, compares structures, fails on drift, and provides remediation commands.
Out of Scope Changes check	✅ Passed	All changes are directly in scope: CI workflow setup, npm script entry, and the validation implementation script, with no unrelated modifications.

_{✏️ Tip: You can configure your own custom pre-merge checks in the settings.}

✨ Finishing Touches

🧪 Generate unit tests (beta)

Create PR with unit tests
Commit unit tests in branch feat/registry-determinism-ci-758

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

_{Comment @coderabbitai help to get the list of available commands and usage tips.}

github-actions · 2026-05-18T12:01:06Z

📊 Coverage Report

Coverage report not available

📈 Full coverage report available in Codecov

Generated by PR Automation (Story 6.1)

coderabbitai

Actionable comments posted: 1

🤖 Prompt for all review comments with AI agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In @.github/workflows/ci.yml:
- Around line 321-347: The new job "registry-determinism" must be added to the
overall validation/merge gate by making "validation-summary" depend on it and
include its failure in the summary; update the "validation-summary" job's needs
list to include "registry-determinism" (or include it among the jobs whose
results are aggregated) so that any failure in registry-determinism blocks the
merge gate, and ensure the summary aggregation logic references
"registry-determinism" when computing pass/fail state.

🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

Push a commit to this branch (recommended)
Create a new PR with the fixes

ℹ️ Review info

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro

Run ID: c04e6058-773c-4129-8361-072d5f23c934

📥 Commits

Reviewing files that changed from the base of the PR and between 6216cda and 4cfe49f.

📒 Files selected for processing (3)

.github/workflows/ci.yml
package.json
scripts/validate-registry-determinism.js

CodeRabbit pointed out that the new `registry-determinism` job was running but not aggregated by `validation-summary`, so a determinism failure wouldn't block the merge gate. Added to `needs` list and to the FAILED check logic with a clear error message. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

Addressed in commit 1506dd4 — registry-determinism is now wired into validation-summary needs list and FAILED-check logic. Superseded by APPROVED review 4309972894.

…ynkraAI#765) * feat(ci): add entity-registry determinism check [SynkraAI#758] Re-runs `populate-entity-registry.js` against the current source tree and diffs the output against the committed `.aiox-core/data/entity-registry.yaml`. Fails the build if any non-volatile fields (dependencies, usedBy, structural) differ between the committed registry and a clean regen. Catches the category of drift that produced SynkraAI#754 (false alarm): a regen run against a partially-edited working tree silently producing a registry that doesn't match what the algorithm would produce on the same source. ## Implementation - `scripts/validate-registry-determinism.js` — snapshots the committed registry, runs the populator (which mutates the file in place), reads back, always restores the snapshot in `finally` so the working tree is preserved regardless of outcome, strips volatile fields (lastUpdated, lastVerified, checksum) before deep-comparing, and emits an actionable error listing the specific entries that diverged on dependencies or usedBy. - `package.json` — `validate:registry-determinism` npm script wired to the new script. - `.github/workflows/ci.yml` — new `registry-determinism` job in the existing `changes.outputs.code` gate (.aiox-core/**, scripts/**, etc), runs in ~5 min, blocks merge on drift. ## Tested locally - Clean main: PASSES (820 entities, structure matches). - Induced drift (manually zeroed deps of one task): FAILS with "[tasks] analyze-brownfield: dependencies differ" and remediation hint. ## What this does NOT catch - Drift inside other generated files (install-manifest.yaml, IDE projections) — covered by `validate:manifest` and `validate:ide-sync` already. - Algorithmic regressions in the populator itself — those should land via the regular dev/test loop; this gate only verifies deterministic output given the committed source state. Closes SynkraAI#758 Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * fix(ci): wire registry-determinism into validation-summary gate [SynkraAI#758] CodeRabbit pointed out that the new `registry-determinism` job was running but not aggregated by `validation-summary`, so a determinism failure wouldn't block the merge gate. Added to `needs` list and to the FAILED check logic with a clear error message. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

rafaelscosta requested review from Pedrovaleriolopez and oalanicolas as code owners May 18, 2026 11:59

vercel Bot deployed to Preview May 18, 2026 11:59 View deployment

github-actions Bot added area: docs Documentation (docs/) area: devops CI/CD, GitHub Actions (.github/) labels May 18, 2026

coderabbitai Bot previously requested changes May 18, 2026

View reviewed changes

Comment thread .github/workflows/ci.yml

vercel Bot deployed to Preview May 18, 2026 12:06 View deployment

coderabbitai Bot approved these changes May 18, 2026

View reviewed changes

rafaelscosta merged commit f31254d into main May 18, 2026
41 checks passed

rafaelscosta deleted the feat/registry-determinism-ci-758 branch May 18, 2026 12:11

coderabbitai Bot mentioned this pull request Jul 14, 2026

fix: isolate CodeRabbit unit tests from WSL probe on Windows #809

Open

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Uh oh!

feat(ci): add entity-registry determinism check — fixes #758 - #765

feat(ci): add entity-registry determinism check — fixes #758#765
rafaelscosta merged 2 commits into
mainfrom
feat/registry-determinism-ci-758

rafaelscosta commented May 18, 2026 •

edited by coderabbitai Bot

Loading

Uh oh!

vercel Bot commented May 18, 2026 •

edited

Loading

Uh oh!

coderabbitai Bot commented May 18, 2026 •

edited

Loading

❌ Failed checks (1 warning)

Uh oh!

github-actions Bot commented May 18, 2026 •

edited

Loading

Uh oh!

coderabbitai Bot left a comment

Uh oh!

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

Uh oh!

Uh oh!

Conversation

rafaelscosta commented May 18, 2026 • edited by coderabbitai Bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

Why this exists

Components

Tested locally (both directions)

What this gate does NOT catch

Test plan

Issue

Summary by CodeRabbit

Uh oh!

vercel Bot commented May 18, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

coderabbitai Bot commented May 18, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Walkthrough

Changes

Estimated code review effort

Possibly related PRs

Suggested labels

Suggested reviewers

❌ Failed checks (1 warning)

Uh oh!

github-actions Bot commented May 18, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

📊 Coverage Report

Uh oh!

coderabbitai Bot left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

rafaelscosta commented May 18, 2026 •

edited by coderabbitai Bot

Loading

vercel Bot commented May 18, 2026 •

edited

Loading

coderabbitai Bot commented May 18, 2026 •

edited

Loading

github-actions Bot commented May 18, 2026 •

edited

Loading