Skip to content

Commit 7220fc9

Browse files
author
SIN-Agent
committed
feat: heypiggy-login → label-based dynamic (no hardcoded indices)
CRITICAL ROBUSTNESS FIX (2026-05-03): - heypiggy-login: replaced all hardcoded element indices (25, 35, 62, 41) with dynamic find_element_by_label() using tree_markdown regex - supports: Weiter + Continue, Fortfahren + Continue (multi-lang) - retry-with-backoff helper for every step - After every click: get_window_state cache (new screen = new indices!) DOCS UPDATED: - commands.md: dynamic flow, robustness warning (no fixed indices!) - learn.md: label-based table, robustness rules (5 rules) - brain.md: ROBUSTNESS-PRINZIP section with anti-pattern vs SOTA comparison - ADOPTION.md: popup-flow with label-matching, 5 robustness rules - fix.md: new section on consent-flow (2026-05-03) - issues.md: SR-10 partial E2E (Login→Dashboard verified, 2FA blocking balance) E2E VERIFIED (PID 89465): - Consent-Flow: Google(43) → Email → Weiter → Fortfahren → Weiter → Dashboard ✅ - Earnings shown: money bag 0.65 € ✅ - Script handles label-changes, multi-language, retries
1 parent 42fd740 commit 7220fc9

6 files changed

Lines changed: 234 additions & 251 deletions

File tree

ADOPTION.md

Lines changed: 17 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -67,22 +67,32 @@ python -m stealth_runner.autonomous_daemon stop
6767

6868
### Google OAuth Popup Flow (CRITICAL)
6969

70-
**Problem**: `skylight-cli` sees only the main window. Popup buttons (Google OAuth, consent dialogs) have DIFFERENT element indices in the popup AX-tree.
70+
**Problem**: `skylight-cli` sieht nur das Hauptfenster. Popup-Buttons (Google OAuth, consent dialogs) haben DIFFERENT element indices in the popup AX-tree.
7171

72-
**Solution**: Always use `cua-driver` for popup windows:
72+
**⚠️ KEINE FIXEN INDICES!** Google kann UI jederzeit ändern. Immer label-basiert suchen!
73+
74+
**Solution**: Always use `cua-driver` with label-matching for popup windows:
7375

7476
```bash
7577
# Find the popup window
7678
POPUP_WID=$(cua-driver call list_windows '{}' | python3 -c "import sys,json; ...")
7779

78-
# Load popup elements
79-
cua-driver call get_window_state '{"pid":12345,"window_id":"'$POPUP_WID'"}'
80+
# Load popup elements (CRITICAL — cache before every action!)
81+
cua-driver call get_window_state "{\"pid\":12345,\"window_id\":\"$POPUP_WID\"}"
8082

81-
# Click in popup (NOT skylight-cli!)
82-
cua-driver call click '{"pid":12345,"window_id":"'$POPUP_WID'","element_index":35,"action":"press"}'
83+
# Click in popup via LABEL (robust against UI changes)
84+
# NOT: element_index=35 — find "Weiter" by label instead!
85+
# Use: find_element_by_label("Weiter", "AXButton", tree)
8386
```
8487

85-
See `docs/cua-driver-popup-pattern.md` for the complete pattern.
88+
See `cli/heypiggy-login` for the complete dynamic label-based pattern.
89+
90+
**⚠️ ROBUSTNESS RULES:**
91+
1. Cache tree with `get_window_state` BEFORE every action
92+
2. Parse labels from `tree_markdown`, not from fixed indices
93+
3. Multi-language: "Weiter" + "Continue", "Fortfahren" + "Continue"
94+
4. Retry 3× with exponential backoff
95+
5. "Fixe Indices" = technischer Schuldenberg — NIEMALS hardcodieren!
8696

8797
---
8898

brain.md

Lines changed: 31 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -18,6 +18,37 @@ cua-driver call set_value '{"pid":PID,"window_id":WID,"element_index":N,"value":
1818

1919
Siehe `docs/cua-driver-popup-pattern.md` für vollständige Doku.
2020

21+
## 🔑 Google Login: Consent-Flow (E2E-verifiziert 2026-05-03, PID 89465)
22+
23+
**Kein Passwort, kein Passkey — nur Consent mit bestehenden Google-Cookies!**
24+
25+
⚠️ **KEINE FIXEN INDICES!** Google ändert UI ständig. Script nutzt `find_element_by_label()`:
26+
- Sucht nach Label im `tree_markdown` via regex
27+
- Multi-Language: "Weiter" + "Continue", "Fortfahren" + "Continue"
28+
- Retry mit exponential backoff
29+
30+
Flow: Google(43) → Email(Label) → Weiter(Label) → Fortfahren(Label) → Weiter(Label) → Dashboard ✅
31+
32+
## 🔑 ROBUSTNESS-PRINZIP (2026-05-03): Immer dynamisch, nie fix!
33+
34+
**Jede UI kann sich jederzeit ändern.** Fixe Element-Indizes sind technischer Schuldenberg.
35+
36+
| Anti-Pattern | SOTA-Pattern |
37+
|---|---|
38+
| `element_index=35` hardcodiert | `find_element_by_label("Weiter", "AXButton", tree)` |
39+
| `skylight list-elements` + Index 35 | `cua get_window_state` + Label-Matching |
40+
| 1 Versuch, dann Error | Retry 3× + exponential backoff |
41+
| Annahme: "UI ändert sich nie" | "UI ändert sich JEDEN TAG" |
42+
43+
**Always:**
44+
1. Cache tree mit `get_window_state` VOR jeder Aktion
45+
2. Parse tree mit Label/Role regex
46+
3. Fallback-Suche (generisch nach spezifisch)
47+
4. Retry-Logik mit backoff
48+
5. Fallback-URL / Alternative-Flow wenn möglich
49+
50+
Siehe `cli/heypiggy-login` (dynamischer Label-basierter Flow).
51+
2152
## AKTIVES MODELL
2253
- **Name**: `nvidia/nemotron-3-nano-omni-30b-a3b-reasoning`
2354
- **API**: `POST https://integrate.api.nvidia.com/v1/chat/completions`

0 commit comments

Comments
 (0)