[enhancement] Plan 38: Cookie Banner Auto-Dismiss

[Delqhi]

Problem

Many survey sites show GDPR/cookie consent banners that block the survey flow. Bot tries to click survey elements behind the banner → wrong targets → stuck.

Browser-use's LLM agent handles consent implicitly. Our deterministic solver needs explicit banner detection + dismissal.

Plan

→ plans/38-cookie-banner-dismiss.md

Key Changes

1. Banner detection patterns — Selector lists for OneTrust, CookieYes, CookieConsent, generic; text-based fallbacks (Akzeptieren, Accept All)
2. Auto-dismiss before DOM scan — In solve_page(), check for + dismiss banner BEFORE question detection
3. CDP click — isTrusted=true click on accept button
4. Cookie strategy config — --cookie-strategy accept|reject|ignore (default: accept — matches real user behavior)
5. Banner state cache — Remember which selector worked per domain

Validation

• Survey site with OneTrust banner → auto-dismissed in <2s
• No false positives (real survey buttons not dismissed)
• Cached selector works on revisit

Files

• survey-cli/survey/skills/cookie_banner.py — NEW
• survey-cli/survey/skills/smart_page_solver.py

Source

browser-use handles consent naturally via LLM; we need deterministic approach

[Delqhi]

Update: Deep browser-use Docs Research (2026-05-24)

Missing from initial plan — NOW ADDED:

§6 Network-Level Banner Blocking (Advanced):

browser-use blocks banners at the Chromium C++ level (never loaded).
We can approximate with CDP Network.setBlockedURLs:

BANNER_SCRIPT_URLS = ["*/onetrust/*", "*/cookieyes/*", "*/cookieconsent/*", "*/quantcast/*", "*/iubenda/*"]
await sws.send("Network.setBlockedURLs", {"urls": BANNER_SCRIPT_URLS})

Trade-off: Some sites break if banner script blocked (JS errors).
Start with dismiss-after-load (reliable), add network blocking as optimization.

§7 Ad Blocking (Parallel):

Cookie banners often co-occur with ads. Block ad scripts to reduce page noise:

AD_BLOCK_URLS = ["*/doubleclick/*", "*/googlesyndication/*", "*/googleadservices/*"]
await sws.send("Network.setBlockedURLs", {"urls": AD_BLOCK_URLS + BANNER_SCRIPT_URLS})

Key browser-use insight:
"Ad and cookie banner blocking" is listed as a CORE STEALTH FEATURE, not a separate module.
"Banners are dismissed automatically so the agent sees clean pages and executes faster."
If we had their C++ fork, banners would be blocked at network level. Since we use JS injection, we dismiss after load. Goal is the same: agent sees clean pages.

[Delqhi]

BONUS: Stealth includes banner blocking (2026-05-24)

From browser-use Stealth docs:

"Ad and cookie banner blocking — Banners are dismissed automatically so the agent sees clean pages and executes faster."

Key insight:

Cookie banner dismissal is listed as a CORE STEALTH FEATURE, not a separate module.
If stealth is good enough (Plan 32 + Plan 33 + Plan 37), banners should rarely appear.

Our approach:

1. Network-level blocking (CDP setBlockedURLs) — Plan 38 §6
2. Dismiss-after-load as safety net — Plan 38 §1-5

Browser-use's implementation:

They block at C++ Chromium level (never loaded). We approximate with CDP.
Same goal: agent sees clean pages.