Added Docs for the new visual tests #7640
Conversation
davepagurek
left a comment
davepagurek
left a comment
There was a problem hiding this comment.
This is great @Vaivaswat2244, thanks!
In addition to this explanation of how the new diff check works, would you also be interested in adding a section that explains some best practices? Off the top of my head, this probably means:
- keeping the images as close as possible to 50x50, since we resize for efficiency before doing a diff
- trying to keep the level of detail in the test to something that can be seen easily at that small size
- calling
screenshot()multiple times in a test if you need to test a bunch of variants rather than trying to make them all visible at the same time in one screenshot - ...and anything else you can think of!
| * Minor differences in curve smoothness | ||
|
|
||
| For example, text rendered on macOS might appear slightly different from text rendered in a Linux CI environment. The same applies to thin lines, curves, and other graphical elements with anti-aliasing. | ||
| An example of this can be the below image which earlier caused tests to fail in CI because of different rendering environments. |
There was a problem hiding this comment.
We might want an additional sentence to reiterate that although this failed before, this is an example of something that should pass, and will pass using the method you describe below.
This reverts commit 67d98c0.
davepagurek
left a comment
davepagurek
left a comment
There was a problem hiding this comment.
Sorry for the delay in reviewing! This looks just about good to go, I just left some minor formatting comments and then we can merge!
| p5.stroke(0); | ||
| p5.strokeWeight(1); | ||
| p5.line(10, 25, 40, 25); | ||
| screenshot('thin-line'); |
There was a problem hiding this comment.
I think screenshot doesn't do anything with arguments, maybe this and the other screenshot call could be done with a comment instead? e.g.
| screenshot('thin-line'); | |
| screenshot(); // Screenshot with thin lines |
| It's important to note that the improved algorithm described above allows tests with acceptable platform-specific variations to pass correctly. Tests that previously failed due to minor rendering differences (like anti-aliasing variations or subtle text rendering differences) will now pass as they should, while still detecting actual rendering bugs. | ||
| For example, a test showing text rendering that previously failed on CI (despite looking correct visually) will now pass with the improved algorithm, as it can distinguish between meaningful differences and acceptable platform-specific rendering variations. This makes the test suite more reliable and reduces false failures that require manual investigation. | ||
|
|
||
| SOME BEST PRACTISES FOR WRITING VISUAL TESTS: |
There was a problem hiding this comment.
Maybe we can use a markdown subheading for this to fit stylistically with the rest of the document?
| SOME BEST PRACTISES FOR WRITING VISUAL TESTS: | |
| ### Some best practices for writing visual tests: |
4 participants
Resolves #7617
Changes:
Added docs and explained the new algorithm.
Added visual examples for better understanding.
Screenshots of the change:
PR Checklist
npm run lintpasses