[enhancement] Plan 40: Human-in-the-Loop for Account Setup

[Delqhi]

Problem

New Heypiggy profiles require manual Google OAuth login. CAPTCHAs during login need human solve. Bot either crashes or loops forever when hitting these walls.

Browser-use has a formal HITL pattern: pause agent → human interacts in live browser → agent continues.

Plan

→ plans/40-human-in-the-loop.md

Key Changes

1. HITL pause points — login_required, captcha_blocked, account_setup, 2fa_required
2. Pause for human — Print message + wait for Enter (with configurable timeout, default 300s)
3. Login status detection — Heypiggy-specific check for avatar/user-menu/logged-in indicator
4. Screenshot on pause — Save screenshot so human can see what's happening
5. Auto-continue detection — Poll for login completion instead of only waiting for Enter
6. CLI flag — --hitl (enable) / --no-hitl (disable, current default)

Validation

• New profile → bot pauses for login → human logs in → bot continues
• CAPTCHA appears → HITL pause → human solves → bot continues
• Auto-detect login after human action within 10s

Files

• survey-cli/survey/skills/hitl.py — NEW
• survey-cli/survey/skills/deterministic_solver.py
• survey-cli/survey_cli_entry.py

Source

browser-use "Human in the Loop" (pause agent, human acts, agent continues)

[Delqhi]

Update: Deep browser-use Docs Research (2026-05-24)

Missing from initial plan — NOW ADDED:

§7 2FA/TOTP Support (from browser-use 2FA docs):

browser-use handles 4 types of 2FA. We should support at least 2:

Type A: TOTP via pyotp (fully autonomous)

import pyotp
def generate_totp(secret: str) -> str:
    return pyotp.TOTP(secret).now()

CLI: --totp-secret JBSWY3DPEHPK3PXP (base32 secret from authenticator setup)

Type B: HITL for complex auth (fallback) — when TOTP not available (SMS 2FA, push notification)

Type C: Email verification (future) — browser-use has "Agent Mail" (unique email per session, receives verification codes). Needs IMAP integration. Out of scope for now.

Type D: Profile reuse (Plan 35) — login once, save profile, reuse cookies. Primary strategy; HITL/TOTP only when cookies expire.

§8 Session Keepalive (from browser-use HITL docs):

"Sessions time out after 15 minutes of inactivity."

Our HITL pauses must send periodic CDP pings to prevent Chrome disconnection:

async def _keepalive_loop(self, interval=120):
    while self._hitl_paused:
        await self._sws.eval("1+1")  # minimal CDP command
        await asyncio.sleep(interval)

§9 Screenshot + URL Print on HITL Pause:

⏸️  HITL PAUSE: login_required
   Chrome is running — switch to it to complete login.
   Current URL: https://www.heypiggy.com/login
   Screenshot: /tmp/hitl_pause_1716508800.png
   Press Enter when done (timeout: 300s)

[Delqhi]

BONUS: Blog "How to Authenticate AI Web Agents" (2026-03-08)

https://browser-use.com/posts/web-agent-authentication

sensitive_data placeholder pattern (agent NEVER sees credentials):

agent = Agent(
    task='Go to example.com/login, enter x_user, x_pass, and x_bu_2fa_code',
    sensitive_data={
        'x_user': 'myusername',
        'x_pass': 'mypassword',
        'x_bu_2fa_code': 'JBSWY3DPEHPK3PXP',
    },
)

• Agent only sees placeholder names (x_user)
• Values injected DIRECTLY into page, not via LLM
• MORE SECURE than passing credentials through prompts
• Implement sensitive_data placeholder pattern in our solver

1Password domain-scoped tool:

@tools.action('Get 2FA code', domains=['*.google.com'])
async def get_1password_2fa() -> ActionResult:
    client = await Client.authenticate(auth=OP_TOKEN, ...)
    code = await client.secrets.resolve(f'op://Private/{ITEM}/One-time passcode')
    return ActionResult(extracted_content=code)

Domain-scoped: only activates on matching domains.

AgentMail (email-2FA reference):

inbox = await email_client.inboxes.create()
@tools.registry.action('Get verification code from email')
async def get_verification_code():
    emails = await email_client.inboxes.messages.list(inbox_id=inbox.inbox_id)
    if emails.messages:
        return ActionResult(extracted_content=emails.messages[0].text)

Summary: our bot vs browser-use auth features

Feature	Browser Use	Our Bot
TOTP	x_bu_2fa_code placeholder	Plan 40 (pyotp)
HITL	input("Press Enter...")	Plan 40 (same)
1Password	Custom tool + domains	Future
Profile reuse	storage_state	Plan 35
Agent Mail	API + polling	Future