fix(opera): support modern Chromium profiles layout and zero-kill cookie extraction#556
fix(opera): support modern Chromium profiles layout and zero-kill cookie extraction#556alztrk wants to merge 8 commits into
Conversation
…kie extraction Closes stickerdaniel#555 Modern Opera (Opera One, 2023+) uses the standard Chromium profiles layout with a Default/ subdirectory, not the flat layout used by old Opera and Opera GX. The import code only searched the flat layout and silently found zero profiles. Also fixes reading the locked Cookies database without killing browser processes, using DuplicateHandle instead of Windows Restart Manager.
Greptile SummaryThis PR fixes Opera cookie import on modern Chromium-based profile layouts. The main changes are:
Confidence Score: 4/5This is close, but the cookie snapshot path should be fixed before merging.
linkedin_mcp_server/browser_import/extract.py Important Files Changed
Prompt To Fix All With AIFix the following 1 code review issue. Work through them one at a time, proposing concise fixes.
---
### Issue 1 of 1
linkedin_mcp_server/browser_import/extract.py:833-841
**Snapshot still splits** When `Cookies` copies successfully in this phase but `Cookies-wal` or `Cookies-shm` is locked, the main DB is captured before the sidecar is copied through DuplicateHandle or Restart Manager. If Chromium commits or checkpoints between those phases, the temp directory contains SQLite files from different moments. The later read opens the copy with `mode=ro`, so it can replay a WAL that does not match the copied DB and return stale cookies, miss the current `li_at`, or fail with a corrupt snapshot. The DB and sidecars need to be captured as one consistency unit instead of batching only the files that remain locked.
Reviews (7): Last reviewed commit: "Merge remote-tracking branch 'origin/mai..." | Re-trigger Greptile |
ctypes.windll, WinDLL, WINFUNCTYPE and os.O_BINARY are Windows-only. Without ty: ignore, the Linux CI reports them as errors.
Greptile identified a race: after RmEndSession the browser restarts its network process and reacquires the lock. The copy must happen during the unlock window, before the session ends.
Also passes copy_to for sidecar Restart Manager fallback.
Previously each locked file (DB, WAL, SHM) was handled individually through separate Restart Manager sessions or DuplicateHandle calls, risking a mixed-snapshot where DB and sidecars come from different database moments. Now all files that still need unlocking after DuplicateHandle are registered with one RmStartSession, released atomically via one RmShutdown, and copied within a single unlock window.
| need_duphandle: list[tuple[Path, Path]] = [] | ||
| for src, dst in copies: | ||
| try: | ||
| shutil.copy2(src, dst) | ||
| os.chmod(dst, 0o600) | ||
| except PermissionError: | ||
| if os.name == "nt": | ||
| need_duphandle.append((src, dst)) | ||
| else: |
There was a problem hiding this comment.
Snapshot still splits When
Cookies copies successfully in this phase but Cookies-wal or Cookies-shm is locked, the main DB is captured before the sidecar is copied through DuplicateHandle or Restart Manager. If Chromium commits or checkpoints between those phases, the temp directory contains SQLite files from different moments. The later read opens the copy with mode=ro, so it can replay a WAL that does not match the copied DB and return stale cookies, miss the current li_at, or fail with a corrupt snapshot. The DB and sidecars need to be captured as one consistency unit instead of batching only the files that remain locked.
Prompt To Fix With AI
This is a comment left during a code review.
Path: linkedin_mcp_server/browser_import/extract.py
Line: 833-841
Comment:
**Snapshot still splits** When `Cookies` copies successfully in this phase but `Cookies-wal` or `Cookies-shm` is locked, the main DB is captured before the sidecar is copied through DuplicateHandle or Restart Manager. If Chromium commits or checkpoints between those phases, the temp directory contains SQLite files from different moments. The later read opens the copy with `mode=ro`, so it can replay a WAL that does not match the copied DB and return stale cookies, miss the current `li_at`, or fail with a corrupt snapshot. The DB and sidecars need to be captured as one consistency unit instead of batching only the files that remain locked.
How can I resolve this? If you propose a fix, please make it concise.Port get_saved_jobs (stickerdaniel#523), job output_mode/file export (stickerdaniel#519), Opera modern profile layout and Windows zero-kill cookie copy (stickerdaniel#556), and comment reading plus analytics tools (stickerdaniel#571) onto the Tier B branch.
Port get_saved_jobs (stickerdaniel#523), job output_mode/file export (stickerdaniel#519), Opera modern profile layout and Windows zero-kill cookie copy (stickerdaniel#556), and comment reading plus analytics tools (stickerdaniel#571) onto the Tier B branch.
Closes #XXX
--import-from-browser operawas broken on two fronts: the profile layout discovery was wrong for modern Opera, and the locked Cookies database could not be read without killing browser processes.Opera profile layout
Old Opera and Opera GX store the default profile directly in the user data folder (a "flat" layout). Opera One (2023+) uses the standard Chromium layout with a
Defaultsubdirectory. The original code only looked for the flat layout, so it found zero profiles on any modern Opera installation.The fix tries the
profileslayout first, then falls back toflat. This covers all three cases: modern Opera One (profiles), legacy Opera and Opera GX (flat), and migrated users who have both — the active cookies inDefaultare preferred over any stale copies lingering at the root level.Locked Cookies database
Chromium locks its Cookies SQLite database with a mandatory byte-range lock while the browser is running, so it cannot be opened or copied by normal means. The existing code used Windows Restart Manager to kill the browser's network process, release the lock, copy the file, then let the process restart. This has two problems: it kills browser processes (the zero-kill requirement), and the network process restarts so quickly that the lock is often reacquired before the copy completes.
The primary path now uses
DuplicateHandleinstead. It enumerates all system handles viaNtQuerySystemInformation, locates the handle locking the target file, duplicates it from the lock-holding process (with onlyPROCESS_DUP_HANDLEaccess), verifies it is a disk file viaGetFileType, resolves the path withGetFinalPathNameByHandleW, and reads the content. A thread-based timeout prevents hangs on pipe or socket handles. The Restart Manager fallback remains but its race condition is fixed: the copy now happens immediately afterRmShutdownwith tight retries, before the network process can restart.Other fixes
FILE_TYPE_INDEX = 42, which was specific to one Windows build, in favor of aGetFileTypecall that works across all versions.restype/argtypeson all Win32 API calls to prevent 64-bit handle truncation in ctypes.Verification
Tested end-to-end on Windows 11 with Opera One open: 34 LinkedIn cookies extracted, session validated against
/feed/. No processes killed.Synthetic prompt