Skip to content

Commit c56e1ae

Browse files
Ws/fix 1146 fixed bidi elements (#1147)
* fix: fix 11146 - added a new arg `biDiOrigin?: 'document' | 'viewport'` - added UT's * chore: log line * chore: add changeset * chore: fix edge screenshot * chore: new images * chore: new android images * chore: new set of images * chore: exclude some tests * chore: change config for LT * :chore: revert store all image --------- Co-authored-by: Cursor <cursoragent@cursor.com>
1 parent 9bb4e8e commit c56e1ae

68 files changed

Lines changed: 323 additions & 8 deletions

File tree

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.
Lines changed: 58 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,58 @@
1+
---
2+
"@wdio/image-comparison-core": patch
3+
"@wdio/visual-service": patch
4+
---
5+
6+
## #1146 Fix BiDi element screenshots missing composited layers (scrollbars, fixed/sticky overlays)
7+
8+
### Root cause
9+
10+
When `checkElement` / `saveElement` is used with the WebDriver BiDi protocol, the screenshot was taken with `browsingContext.captureScreenshot` using `origin: 'document'`. This renders the document layout independently of the browser's compositor, which means **composited layers are never included** — element-level scrollbars, `position: fixed` / `position: sticky` overlays, and elements with a `will-change` CSS property all render as invisible or without their correct visual state.
11+
12+
The switch to `origin: 'document'` was introduced in an earlier fix (commit `227f10a`) to avoid a `zero dimensions` error that occurred when `origin: 'viewport'` was used for elements that were outside the visible viewport. That fix was correct for out-of-viewport elements, but it also silently broke composited-layer capture for all elements.
13+
14+
### Fix: new `biDiOrigin` method option
15+
16+
A new **method-level** option `biDiOrigin` has been added to `saveElement` / `checkElement`. It is BiDi-only and ignored for the legacy WebDriver screenshot path.
17+
18+
| Value | Behaviour |
19+
|---|---|
20+
| `'document'` *(default)* | Previous behaviour — works for any element position but composited layers (scrollbars, overlays, `will-change`) are not captured |
21+
| `'viewport'` | Captures the composited frame as the browser painted it — scrollbars, fixed/sticky overlays and `will-change` layers are included. The element must be visible in the viewport; descriptive errors are thrown when it is not |
22+
23+
#### Usage
24+
25+
```ts
26+
// Capture an element with its scrollbar / overlay visible:
27+
await browser.checkElement(element, 'myTag', { biDiOrigin: 'viewport' })
28+
await browser.saveElement(element, 'myTag', { biDiOrigin: 'viewport' })
29+
```
30+
31+
#### Error messages when `biDiOrigin: 'viewport'` cannot produce a valid screenshot
32+
33+
**Element larger than the viewport** — must fall back to `'document'`:
34+
```
35+
[BiDi viewport screenshot] The element dimensions (1400x800px) exceed the viewport (1280x720px).
36+
You must use the default `biDiOrigin: 'document'` for this element.
37+
Note: with `'document'` origin, composited layers such as scrollbars, fixed/sticky overlays,
38+
and elements using `will-change` may not appear in the screenshot.
39+
```
40+
41+
**Element not in the viewport at all** — needs scrolling:
42+
```
43+
[BiDi viewport screenshot] The element is not in the viewport
44+
(element: x=0, y=900, 300x200px; viewport: 1280x720px).
45+
Call `element.scrollIntoView()` before taking the screenshot, or set `autoElementScroll: true`.
46+
```
47+
48+
**Element partially outside the viewport but fits** — needs to be scrolled fully into view:
49+
```
50+
[BiDi viewport screenshot] The element is not fully visible in the viewport
51+
(element: x=-20, y=100, 300x200px; viewport: 1280x720px).
52+
The element fits within the viewport — scroll it fully into view by calling
53+
`element.scrollIntoView()` or setting `autoElementScroll: true`.
54+
```
55+
56+
### Committers: 1
57+
58+
- Wim Selles ([@wswebcreation](https://github.com/wswebcreation))

packages/image-comparison-core/src/commands/__snapshots__/checkWebElement.test.ts.snap

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -92,6 +92,7 @@ exports[`checkWebElement > should execute checkWebElement with basic options 2`]
9292
},
9393
"saveElementOptions": {
9494
"method": {
95+
"biDiOrigin": undefined,
9596
"disableBlinkingCursor": false,
9697
"disableCSSAnimation": false,
9798
"enableLayoutTesting": false,
@@ -494,6 +495,7 @@ exports[`checkWebElement > should handle custom element options 1`] = `
494495
},
495496
"saveElementOptions": {
496497
"method": {
498+
"biDiOrigin": undefined,
497499
"disableBlinkingCursor": true,
498500
"disableCSSAnimation": true,
499501
"enableLayoutTesting": true,
@@ -835,6 +837,7 @@ exports[`checkWebElement > should handle undefined method options with fallbacks
835837
},
836838
"saveElementOptions": {
837839
"method": {
840+
"biDiOrigin": undefined,
838841
"disableBlinkingCursor": false,
839842
"disableCSSAnimation": false,
840843
"enableLayoutTesting": false,

packages/image-comparison-core/src/commands/__snapshots__/saveWebElement.test.ts.snap

Lines changed: 14 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -16,6 +16,7 @@ exports[`saveWebElement > should call takeElementScreenshot with BiDi disabled w
1616
{
1717
"addressBarShadowPadding": 6,
1818
"autoElementScroll": true,
19+
"biDiOrigin": "document",
1920
"deviceName": "desktop",
2021
"devicePixelRatio": 2,
2122
"deviceRectangles": {
@@ -71,6 +72,7 @@ exports[`saveWebElement > should call takeElementScreenshot with BiDi disabled w
7172
},
7273
"initialDevicePixelRatio": 2,
7374
"innerHeight": 900,
75+
"innerWidth": undefined,
7476
"isAndroid": false,
7577
"isAndroidChromeDriverScreenshot": false,
7678
"isAndroidNativeWebScreenshot": false,
@@ -106,6 +108,7 @@ exports[`saveWebElement > should call takeElementScreenshot with BiDi disabled w
106108
{
107109
"addressBarShadowPadding": 6,
108110
"autoElementScroll": true,
111+
"biDiOrigin": "document",
109112
"deviceName": "desktop",
110113
"devicePixelRatio": 2,
111114
"deviceRectangles": {
@@ -161,6 +164,7 @@ exports[`saveWebElement > should call takeElementScreenshot with BiDi disabled w
161164
},
162165
"initialDevicePixelRatio": 2,
163166
"innerHeight": 900,
167+
"innerWidth": 1200,
164168
"isAndroid": false,
165169
"isAndroidChromeDriverScreenshot": false,
166170
"isAndroidNativeWebScreenshot": false,
@@ -432,6 +436,7 @@ exports[`saveWebElement > should call takeElementScreenshot with BiDi disabled w
432436
{
433437
"addressBarShadowPadding": 6,
434438
"autoElementScroll": true,
439+
"biDiOrigin": "document",
435440
"deviceName": "desktop",
436441
"devicePixelRatio": 2,
437442
"deviceRectangles": {
@@ -487,6 +492,7 @@ exports[`saveWebElement > should call takeElementScreenshot with BiDi disabled w
487492
},
488493
"initialDevicePixelRatio": 2,
489494
"innerHeight": 900,
495+
"innerWidth": undefined,
490496
"isAndroid": false,
491497
"isAndroidChromeDriverScreenshot": false,
492498
"isAndroidNativeWebScreenshot": false,
@@ -866,6 +872,7 @@ exports[`saveWebElement > should call takeElementScreenshot with correct options
866872
{
867873
"addressBarShadowPadding": 6,
868874
"autoElementScroll": true,
875+
"biDiOrigin": "document",
869876
"deviceName": "desktop",
870877
"devicePixelRatio": 2,
871878
"deviceRectangles": {
@@ -921,6 +928,7 @@ exports[`saveWebElement > should call takeElementScreenshot with correct options
921928
},
922929
"initialDevicePixelRatio": 2,
923930
"innerHeight": 900,
931+
"innerWidth": undefined,
924932
"isAndroid": false,
925933
"isAndroidChromeDriverScreenshot": false,
926934
"isAndroidNativeWebScreenshot": false,
@@ -1170,6 +1178,7 @@ exports[`saveWebElement > should handle NaN dimension values correctly 2`] = `
11701178
{
11711179
"addressBarShadowPadding": 6,
11721180
"autoElementScroll": true,
1181+
"biDiOrigin": "document",
11731182
"deviceName": "desktop",
11741183
"devicePixelRatio": 1,
11751184
"deviceRectangles": {
@@ -1225,6 +1234,7 @@ exports[`saveWebElement > should handle NaN dimension values correctly 2`] = `
12251234
},
12261235
"initialDevicePixelRatio": 1,
12271236
"innerHeight": NaN,
1237+
"innerWidth": undefined,
12281238
"isAndroid": false,
12291239
"isAndroidChromeDriverScreenshot": false,
12301240
"isAndroidNativeWebScreenshot": false,
@@ -1524,6 +1534,7 @@ exports[`saveWebElement > should pass autoElementScroll option correctly 2`] = `
15241534
{
15251535
"addressBarShadowPadding": 6,
15261536
"autoElementScroll": true,
1537+
"biDiOrigin": "document",
15271538
"deviceName": "desktop",
15281539
"devicePixelRatio": 2,
15291540
"deviceRectangles": {
@@ -1579,6 +1590,7 @@ exports[`saveWebElement > should pass autoElementScroll option correctly 2`] = `
15791590
},
15801591
"initialDevicePixelRatio": 2,
15811592
"innerHeight": 900,
1593+
"innerWidth": undefined,
15821594
"isAndroid": false,
15831595
"isAndroidChromeDriverScreenshot": false,
15841596
"isAndroidNativeWebScreenshot": false,
@@ -1614,6 +1626,7 @@ exports[`saveWebElement > should pass resizeDimensions option correctly 2`] = `
16141626
{
16151627
"addressBarShadowPadding": 6,
16161628
"autoElementScroll": true,
1629+
"biDiOrigin": "document",
16171630
"deviceName": "desktop",
16181631
"devicePixelRatio": 2,
16191632
"deviceRectangles": {
@@ -1669,6 +1682,7 @@ exports[`saveWebElement > should pass resizeDimensions option correctly 2`] = `
16691682
},
16701683
"initialDevicePixelRatio": 2,
16711684
"innerHeight": 900,
1685+
"innerWidth": undefined,
16721686
"isAndroid": false,
16731687
"isAndroidChromeDriverScreenshot": false,
16741688
"isAndroidNativeWebScreenshot": false,

packages/image-comparison-core/src/commands/checkWebElement.ts

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -24,6 +24,7 @@ export default async function checkWebElement(
2424
// 1. Extract common variables
2525
const commonCheckVariables = extractCommonCheckVariables({ folders, instanceData, wicOptions: checkElementOptions.wic })
2626
const {
27+
biDiOrigin,
2728
disableBlinkingCursor,
2829
disableCSSAnimation,
2930
enableLayoutTesting,
@@ -40,6 +41,7 @@ export default async function checkWebElement(
4041
const saveElementOptions: SaveElementOptions = {
4142
wic: checkElementOptions.wic,
4243
method: {
44+
biDiOrigin,
4345
disableBlinkingCursor,
4446
disableCSSAnimation,
4547
enableLayoutTesting,

packages/image-comparison-core/src/commands/element.interfaces.ts

Lines changed: 10 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -16,6 +16,16 @@ export interface SaveElementMethodOptions extends Partial<Folders>, BaseWebScree
1616
* @default undefined
1717
*/
1818
resizeDimensions?: ResizeDimensions;
19+
/**
20+
* BiDi-only: which coordinate origin to use when capturing element screenshots via the BiDi protocol.
21+
* - `'document'` (default): renders the document layout, works for any element position but
22+
* does NOT capture composited layers (scrollbars, fixed/sticky overlays, `will-change` elements).
23+
* - `'viewport'`: captures the composited frame as painted, which includes scrollbars and overlays,
24+
* but requires the element to be fully visible in the viewport. Throws a descriptive error when the
25+
* element is outside, partially outside, or larger than the viewport.
26+
* @default 'document'
27+
*/
28+
biDiOrigin?: 'document' | 'viewport';
1929
}
2030

2131
export interface CheckElementMethodOptions extends SaveElementMethodOptions, CheckMethodOptions { }

packages/image-comparison-core/src/commands/saveWebElement.ts

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -39,6 +39,7 @@ export default async function saveWebElement(
3939
window: {
4040
devicePixelRatio,
4141
innerHeight,
42+
innerWidth,
4243
isEmulated,
4344
isLandscape,
4445
},
@@ -52,16 +53,19 @@ export default async function saveWebElement(
5253
} = enrichedInstanceData
5354

5455
// 3. Take the screenshot
56+
const biDiOrigin = saveElementOptions.method.biDiOrigin ?? 'document'
5557
const elementScreenshotOptions: ElementScreenshotDataOptions = {
5658
addressBarShadowPadding,
5759
autoElementScroll,
60+
biDiOrigin,
5861
deviceName,
5962
devicePixelRatio: devicePixelRatio || 1,
6063
deviceRectangles: instanceData.deviceRectangles,
6164
element,
6265
isEmulated,
6366
initialDevicePixelRatio: initialDevicePixelRatio || 1,
6467
innerHeight,
68+
innerWidth,
6569
isAndroidNativeWebScreenshot,
6670
isAndroidChromeDriverScreenshot,
6771
isAndroid,

packages/image-comparison-core/src/methods/screenshots.interfaces.ts

Lines changed: 5 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -250,10 +250,14 @@ export interface ElementScreenshotDataOptions extends
250250
MobileCroppingOptions {
251251
/** Whether to automatically scroll the element into view. */
252252
autoElementScroll: boolean;
253+
/** BiDi-only: coordinate origin for element screenshots ('document' | 'viewport'). */
254+
biDiOrigin?: 'document' | 'viewport';
253255
/** The element to take a screenshot of. */
254256
element: HTMLElement | WebdriverIO.Element | ChainablePromiseElement;
255-
/** The inner height. */
257+
/** The inner height of the viewport. */
256258
innerHeight?: number;
259+
/** The inner width of the viewport. */
260+
innerWidth?: number;
257261
/** Resize dimensions for the screenshot. */
258262
resizeDimensions: any;
259263
}

packages/image-comparison-core/src/methods/takeElementScreenshots.test.ts

Lines changed: 87 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -149,6 +149,93 @@ describe('takeElementScreenshot', () => {
149149
})
150150
})
151151

152+
describe('BiDi viewport screenshots', () => {
153+
const vpOptions: ElementScreenshotDataOptions = {
154+
...baseOptions,
155+
biDiOrigin: 'viewport',
156+
innerWidth: 1280,
157+
innerHeight: 720,
158+
}
159+
160+
it('should take viewport screenshot when element is fully inside the viewport', async () => {
161+
// element at (10, 20, 100x200) — fits in 1280x720 viewport
162+
const result = await takeElementScreenshot(browserInstance, vpOptions, true)
163+
164+
expect(result).toEqual({
165+
base64Image: 'bidi-screenshot-data',
166+
isWebDriverElementScreenshot: false,
167+
})
168+
expect(takeBase64BiDiScreenshotSpy).toHaveBeenCalledWith({
169+
browserInstance,
170+
origin: 'viewport',
171+
clip: { x: 10, y: 20, width: 100, height: 200 },
172+
})
173+
})
174+
175+
it('should throw when element dimensions exceed the viewport', async () => {
176+
getElementRectMock.mockResolvedValueOnce({ x: 0, y: 0, width: 1400, height: 800 })
177+
178+
const err = await takeElementScreenshot(browserInstance, vpOptions, true).catch(e => e) as Error
179+
expect(err.message).toMatch(/element dimensions \(1400x800px\) exceed the viewport \(1280x720px\)/)
180+
expect(err.message).toMatch(/biDiOrigin: 'document'/)
181+
expect(err.message).toMatch(/composited layers/)
182+
})
183+
184+
it('should throw when element is completely outside the viewport', async () => {
185+
// element below the fold
186+
getElementRectMock.mockResolvedValueOnce({ x: 0, y: 800, width: 100, height: 200 })
187+
188+
const err = await takeElementScreenshot(browserInstance, vpOptions, true).catch(e => e) as Error
189+
expect(err.message).toMatch(/element is not in the viewport/)
190+
expect(err.message).toMatch(/scrollIntoView/)
191+
expect(err.message).toMatch(/autoElementScroll: true/)
192+
})
193+
194+
it('should throw when element is partially outside the viewport but fits', async () => {
195+
// element starts at x=-10, so it bleeds left of the viewport
196+
getElementRectMock.mockResolvedValueOnce({ x: -10, y: 0, width: 100, height: 200 })
197+
198+
const err = await takeElementScreenshot(browserInstance, vpOptions, true).catch(e => e) as Error
199+
expect(err.message).toMatch(/not fully visible in the viewport/)
200+
expect(err.message).toMatch(/fits within the viewport/)
201+
expect(err.message).toMatch(/autoElementScroll: true/)
202+
})
203+
204+
it('should include element and viewport dimensions in the error messages', async () => {
205+
getElementRectMock.mockResolvedValueOnce({ x: 0, y: 900, width: 200, height: 100 })
206+
207+
const err = await takeElementScreenshot(browserInstance, vpOptions, true).catch(e => e) as Error
208+
expect(err.message).toMatch(/x=0, y=900, 200x100px/)
209+
expect(err.message).toMatch(/viewport: 1280x720px/)
210+
})
211+
212+
it('should scroll element into view before validating when autoElementScroll is true', async () => {
213+
const vpScrollOptions: ElementScreenshotDataOptions = { ...vpOptions, autoElementScroll: true }
214+
// After scroll, element is fully in viewport
215+
executeMock.mockResolvedValueOnce(300) // previous scroll position
216+
217+
const result = await takeElementScreenshot(browserInstance, vpScrollOptions, true)
218+
219+
expect(result.base64Image).toBe('bidi-screenshot-data')
220+
// scrollElementIntoView + scrollToPosition (restore)
221+
expect(executeMock).toHaveBeenCalledTimes(2)
222+
expect(takeBase64BiDiScreenshotSpy).toHaveBeenCalledWith(
223+
expect.objectContaining({ origin: 'viewport' })
224+
)
225+
})
226+
227+
it('should use origin: document for the default (no biDiOrigin set)', async () => {
228+
const defaultOptions = { ...vpOptions, biDiOrigin: undefined }
229+
230+
const result = await takeElementScreenshot(browserInstance, defaultOptions, true)
231+
232+
expect(result.base64Image).toBe('bidi-screenshot-data')
233+
expect(takeBase64BiDiScreenshotSpy).toHaveBeenCalledWith(
234+
expect.objectContaining({ origin: 'document' })
235+
)
236+
})
237+
})
238+
152239
describe('Legacy screenshots', () => {
153240
let logErrorSpy: ReturnType<typeof vi.spyOn>
154241

0 commit comments

Comments
 (0)