Skip to content

fix: relax selenium exact pin to allow Appium 3 compatibility (#1312)#1319

Merged
mikahanninen merged 18 commits into
masterfrom
fix/issue-1312-selenium-version-pin
Mar 8, 2026
Merged

fix: relax selenium exact pin to allow Appium 3 compatibility (#1312)#1319
mikahanninen merged 18 commits into
masterfrom
fix/issue-1312-selenium-version-pin

Conversation

@mikahanninen

@mikahanninen mikahanninen commented Mar 7, 2026

Copy link
Copy Markdown
Collaborator

Summary

Changes

Bug fixes / compatibility

File Change
packages/main/pyproject.toml ==4.15.2>=4.16.0,<5.0.0, version bump to 31.2.0
packages/main/uv.lock + root uv.lock selenium 4.15.2 → 4.41.0
Selenium.py imports Switched to concrete submodule paths to fix pylint no-name-in-module
Selenium.py _send_command_and_get_result Replaced removed _url HTTP hack with execute_cdp_cmd; added Chromium guard
Selenium.py FirefoxOptions Removed subclass workaround for binary._start_cmd (obsolete since 4.16)
Selenium.py add_virtual_authenticator Use constructor kwargs instead of post-construction attribute assignment

EdgeDriver UTF-16 BOM fix (rpaframework-core 12.1.1)

Root cause: Microsoft's LATEST_RELEASE_<major>_WINDOWS endpoint returns the version string as UTF-16 LE with a BOM (\xff\xfe bytes). requests decodes this to a Python string starting with \ufeff. webdriver-manager 4.0.2 only calls .rstrip() which strips trailing whitespace but leaves the leading \ufeff intact. That BOM gets URL-encoded as %EF%BB%BF in the download URL → 404.

Reproducer:

import requests
r = requests.get("https://msedgedriver.microsoft.com/LATEST_RELEASE_145_WINDOWS")
print(repr(r.text[:20]))  # '\ufeff145.0.3800.97\r\n'

Fix: EdgeChromiumDriver subclass in RPA.core.webdriver overrides get_latest_release_version() and get_stable_release_version() to strip \ufeff before returning. Registered via a matching EdgeChromiumDriverManager subclass.

This is a bug in webdriver-manager 4.0.2 (latest). The fix lives in rpaframework-core so users are not blocked waiting for an upstream release.

Release order after merge:

  1. Publish rpaframework-core 12.1.1
  2. Bump rpaframework-core>=12.1.1 in packages/main/pyproject.toml, regenerate lock, publish rpaframework 31.2.0

New keywords (selenium 4.x)

Keyword Description
Find Element Above/Below/To Left Of/To Right Of/Near Relative locators (selenium 4.x)
Get Browser Logs Chromium-only; returns CDP log entries
Block URLs / Unblock URLs CDP network interception
Wait For Network Request Polls window.performance resources; raises TimeoutException
Add Virtual Authenticator / Remove Virtual Authenticator WebAuthn testing (Chromium-only)

All Chromium-only keywords raise NotImplementedError with a clear message on non-Chromium drivers.

Tests & tooling

File Change
test_browser.py TestRelativeLocators, TestBrowserLogs, TestNetworkInterception, TestVirtualAuthenticator classes; updated import guard
tests/resources/relative_locator_test.html New HTML fixture for relative locator tests
packages/core/tests/python/test_edge_driver.py Standalone EdgeDriver integration test (run on Windows); verifies BOM strip + download + headless session
.github/workflows/main.yaml build-wheel PR job: builds wheel, uploads artifact, posts comment with download link; pinned to Python 3.11
packages/main/scripts/pre-push.sh Pre-push hook: pylint + fast tests + AI review before every push
packages/main/scripts/ai-review.sh Pipes branch diff through claude --print for local Cursor-bot-style review (advisory, never blocks push)

Test plan

  • CI passes on all matrix (Windows/Ubuntu/macOS × Python 3.9/3.10/3.11)
  • test_selenium_api_imports — all production selenium imports remain resolvable
  • test_selenium_constraint_is_not_exact_pin — guards against re-introducing an exact pin
  • test_edge_driver.py — run manually on Windows where Edge is installed; verifies BOM strip, EdgeDriver download, and headless session

🤖 Generated with Claude Code

mikahanninen and others added 3 commits March 7, 2026 20:03
- Change `selenium==4.15.2` to `selenium>=4.15.2,<5.0.0` in packages/main
- Bump rpaframework version 31.1.2 -> 31.2.0
- Upgrade selenium in main's lock file: 4.15.2 -> 4.41.0
- Add standalone EdgeDriver integration test (packages/core) that exercises
  the azureedge.net -> microsoft.com redirect fix from PR #1227
- Add two pytest regression tests: API import check and constraint pin guard

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
`ChromiumRemoteConnection._url` was removed in newer selenium 4.x versions,
breaking `print_to_pdf` and `execute_devtools_protocol_command` with:
  AttributeError: 'ChromiumRemoteConnection' object has no attribute '_url'

Replace the manual HTTP POST to the chromium endpoint with the public
`driver.execute_cdp_cmd()` API available since selenium 4.0.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Comment thread packages/main/src/RPA/Browser/Selenium.py
Relative Locators:
- Find Element Above
- Find Element Below
- Find Element To Left Of
- Find Element To Right Of
- Find Element Near

Browser Logs:
- Get Browser Logs (browser, performance, driver, client log types)

Network Interception (Chromium CDP):
- Block URLs (wildcard pattern matching via CDP Network.setBlockedURLs)
- Unblock URLs
- Wait For Network Request (polls window.performance resource timeline)

Virtual Authenticator (WebAuthn / Passkey testing):
- Add Virtual Authenticator (ctap2/u2f, usb/nfc/ble/hybrid/internal)
- Remove Virtual Authenticator

All keywords are covered by tests in TestRelativeLocators,
TestBrowserLogs, TestNetworkInterception, and TestVirtualAuthenticator.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Comment thread packages/main/tests/python/test_browser.py
Adds a build-wheel job to the rpaframework CI workflow that:
- Builds the .whl with uv build
- Uploads it as a GitHub Actions artifact (retained 14 days)
- Posts/updates a PR comment linking to the workflow run's Artifacts section

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
@github-actions

github-actions Bot commented Mar 7, 2026

Copy link
Copy Markdown

Wheel artifact

Built from commit d156ddd — download from the workflow run (Artifacts section at the bottom of the page).

Artifact name: rpaframework-wheel (retained 14 days)

Comment thread .github/workflows/main.yaml
mikahanninen and others added 2 commits March 7, 2026 20:44
- get_browser_logs: raise NotImplementedError for 'performance'/'client'
  log types on non-Chromium browsers with actionable message
- add_virtual_authenticator: raise NotImplementedError on non-Chromium,
  raise ValueError for invalid protocol or transport values
- remove_virtual_authenticator: raise NotImplementedError on non-Chromium
- unblock_urls: add match= to existing test for consistency

Tests added for all new guards (invalid inputs + non-Chromium browser).

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- Import ChromeOptions, EdgeOptions, FirefoxOptions, FirefoxProfile, IeOptions
  from their concrete submodules instead of selenium.webdriver (re-exports
  no longer traceable by pylint in newer selenium)
- Remove FirefoxOptions wrapper class; binary_location is now a native string
  property on selenium.webdriver.firefox.options.Options (the binary._start_cmd
  workaround was specific to selenium <4.16)

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Comment thread packages/main/src/RPA/Browser/Selenium.py
Comment thread packages/main/tests/python/test_browser.py
test_selenium_api_imports now mirrors the exact imports in Selenium.py:
- Use concrete submodule paths (e.g. selenium.webdriver.chrome.options)
  instead of the old selenium.webdriver shorthand imports
- Add missing new imports: VirtualAuthenticatorOptions, locate_with,
  selenium_webdriver (top-level), FirefoxOptions concrete path

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Comment thread packages/main/pyproject.toml Outdated
Explicit working-directory on each run step removes reliance on the
global defaults section — makes it clear the wheel is built inside
packages/main/ and the artifact path packages/main/dist/*.whl is correct.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Comment thread packages/main/src/RPA/Browser/Selenium.py
…for_network_request

Built-in TimeoutError is not caught by Robot Framework's standard timeout
handling patterns (Run Keyword And Expect Error, Wait Until Keyword Succeeds).
Use selenium.common.exceptions.TimeoutException, consistent with how all
other timeout paths in the library behave (e.g. WebDriverWait).

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Comment thread packages/main/src/RPA/Browser/Selenium.py
…ush hook

- Guard get_browser_logs with is_chromium for all log types (not just
  performance/client). Since selenium 4.32 get_log() moved to ChromiumWebDriver
  and is not available on Firefox or other non-Chromium drivers at all.
  Remove misleading suggestion to use 'browser'/'driver' on non-Chromium.
- Parametrize the non-Chromium guard test to cover all four log types.
- Add packages/main/scripts/pre-push.sh: runs pylint + fast tests locally
  before push to catch CI failures early. Install with:
    cp packages/main/scripts/pre-push.sh .git/hooks/pre-push
    chmod +x .git/hooks/pre-push
  Hook is already installed in this repo's .git/hooks/.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
)
with patch.object(type(library), "is_chromium", new_callable=PropertyMock, return_value=False):
with pytest.raises(NotImplementedError, match="Chromium"):
library.get_browser_logs(log_type=log_type)

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Browser logs test misplaced in network interception class

Low Severity

The parametrized test_get_browser_logs_raises_for_non_chromium test is placed inside TestNetworkInterception instead of TestBrowserLogs, where it logically belongs. This makes the test organization misleading since there's already a dedicated TestBrowserLogs class for browser log tests.

Fix in Cursor Fix in Web

mikahanninen and others added 3 commits March 8, 2026 09:51
ai-review.sh pipes the branch diff through `claude --print` for a
Cursor-bot-style review before every push. Issues are printed but
advisory only — the push is never blocked by AI feedback.

Integrates into pre-push.sh automatically; skip with AI_REVIEW=0.
Run standalone: ./packages/main/scripts/ai-review.sh

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- Bump selenium minimum to >=4.16.0 so FirefoxOptions.binary_location
  is always the native string property (no _start_cmd workaround needed)
- Add Chromium guard to _send_command_and_get_result so any future
  direct caller gets NotImplementedError instead of WebDriverException
- Pin test_find_element_below to expect 'inp-password' (not a broad set)
- Use VirtualAuthenticatorOptions constructor params instead of
  post-construction attribute assignment
- Validate AI_REVIEW_BASE against [A-Za-z0-9/_.-]+ before use in git diff

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- Regenerate root uv.lock with selenium specifier >=4.16.0,<5.0.0
- pre-push.sh: use set -euo pipefail (was set -e)
- build-wheel CI job: pin Python 3.11 via actions/setup-python

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Comment thread packages/main/pyproject.toml
Comment thread packages/main/src/RPA/Browser/Selenium.py
Comment thread packages/main/uv.lock Outdated
mikahanninen and others added 3 commits March 8, 2026 19:40
…er #BOM)

webdriver-manager 4.0.2 calls resp.text.rstrip() on the response from
Microsoft's LATEST_RELEASE_<major>_WINDOWS endpoint. That endpoint
returns UTF-16 LE with a BOM, which requests decodes as a string starting
with \ufeff. The leading BOM is not stripped by rstrip(), so it gets
URL-encoded as %EF%BB%BF in the download URL, causing a 404.

Add EdgeChromiumDriver subclass in RPA.core.webdriver that overrides
get_latest_release_version() and get_stable_release_version() to strip
\ufeff before returning. Add EdgeChromiumDriverManager that injects the
patched driver, and register it in AVAILABLE_DRIVERS.

Reproducer:
    import requests
    r = requests.get("https://msedgedriver.microsoft.com/LATEST_RELEASE_145_WINDOWS")
    print(repr(r.text[:20]))  # '\ufeff145.0.3800.97\r\n'

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
…=4.16.0

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
msedge.exe is installed in Program Files but not added to PATH on Windows.
Check PROGRAMFILES(X86), PROGRAMFILES, and LOCALAPPDATA locations.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

@cursor cursor Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cursor Bugbot has reviewed your changes and found 1 potential issue.

Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.

Group by severity: [HIGH], [MEDIUM], [LOW].

GIT DIFF:
$DIFF"

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Shell expansion mangles diff content in review script

Low Severity

The PROMPT variable is assigned using double quotes, and $DIFF is interpolated inside it at line 58. Any $, backtick, or $(...) sequences in the git diff content will be interpreted by the shell — $HOME becomes the home directory path, $(cmd) gets executed, etc. Python code diffs frequently contain $ (regex, string templates). The PROMPT assignment or the echo "$PROMPT" pipe needs to avoid shell expansion of the diff content.

Fix in Cursor Fix in Web

rpaframework-core 12.1.1 contains the UTF-16 BOM strip fix for EdgeDriver.
packages/main still requires >=12.1.0 in this commit — after 12.1.1 is
published to PyPI, bump the minimum in main to >=12.1.1 and regenerate
the lock file before releasing rpaframework 31.2.0.

Release order:
  1. rpaframework-core 12.1.1
  2. rpaframework 31.2.0 (bump core dep to >=12.1.1, uv lock, release)

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
@mikahanninen mikahanninen merged commit 5a78d0c into master Mar 8, 2026
22 checks passed
@mikahanninen mikahanninen deleted the fix/issue-1312-selenium-version-pin branch March 8, 2026 19:08
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

RPA Framework 31.1.0 pins selenium==4.15.2, breaking Edge auto-driver download due to deprecated msedgedriver.azureedge.net

1 participant