You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
* Revise instructions for UI screenshot validation
Updated UI validation instructions for clarity and detail.
* Add version 1.3.0 release notes to CHANGES.txt
* Add project URLs to setup.py
* Update assertions in test_library.py for accuracy
* Add coverage for AIVision in pytest command
* Update assertion message in test for AI response
* Update assertions in test_lib.py for accuracy
* Update test_lib.py
* Fix assertion message in test_library.py
* Update assertion for instructions in test_library.py
* Update assertion for instructions in test_lib.py
logger.warn(f"Could not create combined image: {e}")
164
164
165
-
instructions="""First image is showing actual application view.
166
-
Second image is reference design template.
167
-
Verify screenshot matches look and feel template. Pay attention to details, design is important.
168
-
Make sure to check also all the visible logos, titles, labels, spelling, texts, links, menus, banners
169
-
and any available graphics. Always doublecheck the reference image in case you think some
170
-
text, label, logo or element is overlapping or containing typo.
165
+
instructions="""
166
+
First image is the actual application screenshot.
167
+
Second image is the reference design/template screenshot.
168
+
169
+
Compare the actual screenshot against the reference screenshot as a visual UI/template validation.
170
+
171
+
Primary goal:
172
+
Verify that the actual page matches the reference page in layout, visual structure, branding, and look-and-feel.
173
+
174
+
Check carefully:
175
+
- Overall page structure and visual hierarchy
176
+
- Header, navigation, menu icons, logos, brand marks, and banners
177
+
- Sections, cards, panels, containers, borders, backgrounds, shadows, and spacing
178
+
- Buttons, links, icons, badges, dropdowns, input fields, and other interactive elements
179
+
- Element positions, alignment, relative sizing, padding, margins, and grouping
180
+
- Font style, font weight, approximate font size, text alignment, and color usage
181
+
- Overlapping, clipped, truncated, hidden, misplaced, or visually broken elements
182
+
- Unexpected wrapping, excessive spacing, missing spacing, or layout shifts
183
+
- Whether all expected visible UI elements are present
184
+
185
+
Important text comparison rules:
186
+
Do not compare dynamic text values literally. Values such as phone numbers, account numbers, names, balances, prices, dates, times, counts, statuses, identifiers, and user-specific data may be different and must not cause failure.
187
+
188
+
Treat dynamic text as visual text blocks:
189
+
- The exact value may differ.
190
+
- The text block should still appear in the expected location.
191
+
- It should have similar styling, size, alignment, color, and visual weight.
192
+
- It should not break the layout, overlap other elements, be clipped, or cause unexpected wrapping.
193
+
194
+
Static/template text rules:
195
+
Static labels, headings, menu names, fixed links, fixed button labels, fixed section names, and fixed instructional texts should be checked only when they are clearly part of the template.
196
+
However, do not fail only because the actual text content differs if the difference appears to be dynamic or user-specific.
197
+
198
+
Ignore:
199
+
- Browser chrome, OS status bars, emulator/device frames, address bars, scroll bars, timestamps, debug overlays, and comparison labels such as "Actual" or "Expected", unless they are part of the application UI.
200
+
- Minor anti-aliasing differences, screenshot compression artifacts, tiny pixel-level shifts, and insignificant rendering differences.
201
+
- Small differences caused by platform/browser font rendering if the layout and visual hierarchy remain correct.
202
+
203
+
Fail the comparison when:
204
+
- A required UI element is missing or an unexpected major UI element is present.
205
+
- Logo, header, menu, navigation, main section, card, button, link, or icon is visually incorrect or misplaced.
206
+
- Layout structure differs significantly from the reference.
207
+
- Elements overlap, are clipped, truncated, hidden, or visually broken.
208
+
- Text blocks are present but their placement, style, size, color, or wrapping materially differs from the reference.
209
+
- Spacing, alignment, sizing, or visual hierarchy is noticeably inconsistent with the reference.
210
+
- The page looks like a different template, broken responsive layout, or incorrect design version.
211
+
212
+
Pass the comparison when:
213
+
- The actual screenshot preserves the same layout, structure, visual hierarchy, branding, and front-end appearance as the reference.
214
+
- Differences are limited to dynamic text values or insignificant rendering variations.
215
+
- All expected elements are present and visually usable.
0 commit comments