✨ Align local TDD dynamic content filtering#274
Merged
Conversation
Match local confirmed-region filtering to cloud behavior by using center-based region matching, pixel-weighted coverage, and SSIM gates. Persist bundled and fallback dynamic metadata during baseline downloads so local TDD can use the same context as cloud review.
This comment has been minimized.
This comment has been minimized.
Use the latest released honeydiff package for local TDD comparisons so the CLI runs against the newest diff metadata behavior.
This comment has been minimized.
This comment has been minimized.
Keep React and React DOM in the published dependency set because the packed CLI loads the SSR report renderer at runtime.
Vizzly - Visual Test ResultsCLI Reporter - 2 changes need review
Meaningful diffs needing review (2)dashboard-mixed-state · Firefox · 375×892 · 28.7% diff fullscreen-viewer · Firefox · 375×667 · 78.5% diff
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.


Why
Local TDD was using a much thinner view of dynamic content than cloud review.
In the cloud, Vizzly has access to honeydiff metadata, historical hot spot analysis, and user-confirmed dynamic regions. That lets cloud review treat things like timestamps, generated copy bands, avatar slots, and other known noisy areas differently from real layout changes.
The local TDD workflow did not have that full context. Even when the cloud knew a region was dynamic, the local server could still surface the change as a normal diff. The result was a broken-feeling loop: you would run TDD locally, get a wall of diffs, and then see cloud review behave differently because it had better metadata.
This PR makes the CLI side consume and apply that dynamic-content context so local TDD and cloud review are making the same kind of decision.
What Changed
Baseline downloads now persist dynamic metadata
When the app returns bundled TDD baseline context, the CLI now saves:
.vizzly/hotspots.json.vizzly/regions.jsonThat comparison-id fallback matters because older comparison payloads may not include the new
dynamic_contentfield directly. In that case, the CLI asks the comparison context endpoint for the history payload and persists the dynamic regions/hotspots from there.Local honeydiff runs now request the needed metadata
The CLI comparison path now asks honeydiff for:
The important one for this PR is SSIM. Confirmed dynamic regions should not become a blanket escape hatch. They are safe only when the changed pixels are in expected regions and the overall image remains structurally similar.
Confirmed-region filtering now mirrors cloud semantics
Before this PR, the initial local implementation checked whether a diff cluster intersected a confirmed region. During review, that turned out to be too permissive.
Cloud confirmed-region matching uses center proximity, then weights coverage by changed pixels. This PR updates the CLI to follow that same shape:
pixelCountThis means a huge unmatched change will not be hidden just because a small piece of it touches a confirmed region. It also means a structural layout shift inside or near a dynamic area still fails locally, matching the cloud behavior we actually want.
Honeydiff is upgraded to the latest release
This branch also upgrades
@vizzly-testing/honeydifffrom the locked0.10.1install to0.10.3, the currentlatestdist-tag on npm.Keeping the CLI on the newest honeydiff release matters here because this local TDD path now depends directly on honeydiff's richer metadata: merged clusters, SSIM/perceptual score, and GMSD. The app PR upgrades honeydiff to the same version so local and cloud are comparing with the same package release.
User Impact
Local TDD should stop producing wildly different results from cloud review for known dynamic content.
The intended workflow becomes:
That makes local TDD useful again for pages with dynamic text, generated areas, or recurring noisy regions.
Safety Notes
This is intentionally conservative:
The tests cover the important failure mode from review: a cluster can intersect a confirmed region but still fail if its center does not match the confirmed region the way cloud expects.
Verification
npm view @vizzly-testing/honeydiff version dist-tags --jsonconfirmedlatestis0.10.3npm ls @vizzly-testing/honeydiff --depth=0npx biome check src/tdd/tdd-service.js src/tdd/services/comparison-service.js src/tdd/core/region-coverage.js tests/tdd/tdd-service.test.js tests/tdd/services/comparison-service.test.js tests/tdd/core/region-coverage.test.jsnode --test --test-concurrency=1 --test-reporter=spec tests/tdd/core/region-coverage.test.js tests/tdd/services/comparison-service.test.js tests/tdd/tdd-service.test.jsnpm run build