Promote synthetic STARTUP session for Playwright connectOverCDP

[staylor]

Summary

Makes Lightpanda's CDP server work with playwright-core's chromium.connectOverCDP. Previously, Playwright would auto-attach to the synthetic STARTUP target Lightpanda advertises in Target.setAutoAttach, drive it with Page.enable / Page.navigate / etc. on sessionId="STARTUP", and time out forever waiting for Page.frameNavigated events that never came — the dispatcher silently {}-acked every STARTUP-tagged command except Page.getFrameTree.

Reproduction

import { chromium } from 'playwright-core';
const browser = await chromium.connectOverCDP('http://127.0.0.1:9222');
const ctx = browser.contexts()[0] ?? await browser.newContext();
const page = ctx.pages()[0] ?? await ctx.newPage();
await page.goto('https://www.allbirds.com/products/mens-wool-runners', { waitUntil: 'load' });
// → TimeoutError: page.goto: Timeout exceeded

puppeteer-core was unaffected because it uses a different protocol shape: puppeteer.connect() → browser.createBrowserContext() → context.newPage(), which sends Target.createBrowserContext + Target.createTarget (no sessionId) and then drives the new target on a real session_id assigned by doAttachtoTarget. It never sends a STARTUP-tagged work command.

Root cause

Lightpanda's CDP server is built around the assumption "browser starts empty, driver creates everything." Target.setAutoAttach advertises a synthetic placeholder target with sessionId="STARTUP" so drivers don't block waiting for a real target to appear. The comment in setAutoAttach makes the assumption explicit: "Hopefully, the first thing they'll do is create a real BrowserContext and progress from there."

Playwright's chromium.connectOverCDP does the opposite. It assumes a real Chrome on the other end and auto-attaches to whatever target is advertised, then immediately drives that target. The dispatcher's dispatchStartupCommand:

fn dispatchStartupCommand(command: *Command, method: []const u8) !void {
    if (std.mem.eql(u8, method, "Page.getFrameTree")) {
        return dispatchCommand(command, method);
    }
    return command.sendResult(null, .{});  // ← {}-acks EVERYTHING else
}

silently drops Page.enable, Page.navigate, Network.enable, Runtime.enable, Page.setLifecycleEventsEnabled, Page.addScriptToEvaluateOnNewDocument, Emulation.*, etc. After Page.navigate, no Page.frameNavigated / lifecycleEvent / loadEventFired events ever fire and page.goto times out.

Fix

Three coordinated changes:

1. Lazy-promote on first STARTUP work command. dispatchStartupCommand now creates a real BrowserContext + Target whose session_id is the literal string "STARTUP" the first time a STARTUP-tagged command needs real page state, then routes through the normal dispatcher. After promotion, isValidSessionId accepts "STARTUP" and every subsequent command flows through the standard handlers.

Target.* and Runtime.runIfWaitingForDebugger are explicitly held out: Puppeteer sends Target.setAutoAttach and Runtime.runIfWaitingForDebugger with sessionId="STARTUP" between puppeteer.connect() and its own Target.createBrowserContext. Promoting on those would steal the bc out from under Puppeteer's createBrowserContext (which would then error with "Cannot have more than one browser context at a time"). Silently {}-acking them, as before, keeps Puppeteer's flow intact.

Once a bc exists with session_id != "STARTUP" (i.e. a real session_id was assigned by createTarget + doAttachtoTarget), further STARTUP-tagged commands are rejected with -32001 "Unknown sessionId" rather than silently no-oped, since they're now stale.

2. Synthetic IDs match the real first frame / context. The synthetic targetId / browserContextId in setAutoAttach were "TID-STARTUP" / "BID-STARTUP". They are now "FID-0000000001" / "BID-1" — the same strings the first real Target / BrowserContext will be assigned (Session.nextFrameId returns 1 first; the bc id generator returns BID-1 first). After promotion, every event Lightpanda emits (Page.frameNavigated, Runtime.executionContextCreated, etc.) carries IDs that line up with what the driver already recorded from the synthetic event. Page.getFrameTree's synthetic placeholder was updated to match (LID-STARTUP → LID-0000000001). Without this, Playwright's first getFrameTree response carried a different frame.id than the targetId it had just learned from Target.attachedToTarget, and Playwright marked the original main frame as detached — page.goto then errored synchronously with "Frame has been detached. (after 1ms)".

3. doAttachtoTarget reuses the literal "STARTUP" session_id when the synthetic STARTUP session was already advertised. A new flag cdp.startup_session_advertised is set when setAutoAttach emits the synthetic event, and consumed by doAttachtoTarget the next time it would otherwise generate a fresh session_id. Without this, Puppeteer's flow ended up with two Target.attachedToTarget events for the same targetId (one with sessionId="STARTUP", then another with sessionId="SID-1" after createTarget ran). Puppeteer treats those as two separate sessions and tries to initialize a Page over each; the STARTUP one then fails every command with "Unknown sessionId" because bc.session_id had been overwritten to SID-1.

A new helper promoteStartupSession in domains/target.zig mirrors the bootstrap portion of createTarget but skips the Target.targetCreated / Target.attachedToTarget events (the driver already received Target.attachedToTarget in setAutoAttach and a duplicate would re-trigger Playwright's confusion).

Verification

• playwright-core 1.59.1 chromium.connectOverCDP + page.goto on https://www.allbirds.com/products/mens-wool-runners now returns status=200 and fires framenavigated / domcontentloaded / load events. Server stays alive across 10 sequential runs.
• puppeteer-core 24.42.0 puppeteer.connect() + createBrowserContext + newPage().goto() still works end-to-end and extracts the expected <title> and ~922 KB body.
• 523/523 unit tests pass. The existing cdp: STARTUP sessionId test was rewritten to assert the new lazy-promote behavior, and two new tests cover the rejection paths (bc with non-STARTUP session_id, bc with no session_id at all).
• Combined with Defer page teardown while worker scripts are evaluating #2398 (the worker re-entrancy crash fix), 10 sequential mixed Puppeteer / Playwright runs against the same lightpanda serve process: 9 successful (status=200), 1 Playwright timeout (network flake on the 10th run), server stayed alive throughout.

Notes / out of scope

Playwright now navigates successfully but page.title() returns "" and page.evaluate(() => document.title) errors with "Execution context was destroyed, most likely because of a navigation." That's Lightpanda's Page.createIsolatedWorld / Runtime.executionContextCreated flow not re-binding Playwright's utility-world context after navigation — a separate gap, not fixed here. The user-facing page.goto path works.

Independent of #2398. Either order of landing is fine.

[staylor]

repro: https://gist.github.com/staylor/af9ab08ad850646dc2ca63bf4001df23

[krichprollsch]

Hello @staylor, first thank you for the PR.

Do you have a concrete example where the current behavior blocks you? ie. you can't create the first browser context by yourself?

While I agree w/ your initial analysis and the issue w/ clients trying to use the default browser context/target, I'm not sure about supporting lazy load on auto-attach.

As you mentioned, our idea is to offer an empty browser first and delegate the responsibility to create new browser context and new page itself.

Lazy load is smart, but complex since you have different paths depending on the client (which can change in the future).

And it doesn't really work in the case of a Playwright script using connectOverCDP + creating new browser context. It results in trying to create 2 successive BCs which is not currently possible. That's why e2e tests are currently failing.

So the main problem is we don't have clear paths between Playwright scripts using the default BC and the ones creating a clean new BC.

That's why we have to explain to Stagehand's users to create a new page while the official doc reuses the existing one.

    // Important: in the official documentation, Stagehand uses the default
    // existing page. But Lightpanda requires explicit page creation
    // instead.
    const page = await stagehand.context.newPage();

Even if we could create a default BC, we don't want to penalize all Playwright users by creating two if they don't use the default one.

That's why I'm in the position where I think it's better to ask users to create context manually instead of trying to use the default one.

But at least 1 thing could be improved: it would be better to return an explicit error than silently accept the command on STARTUP. But we have to choose the commands carefully, I don't want to break current support. Maybe we could safely display server-side warnings.

WDYT?

[staylor]

this is less important for-me-personally now, since I just shifted to using Puppeteer instead - I'll come back to this if someone needs to confirm Playwright behavior