AI WS: Investigate Stagehand caching

[nikitachapovskii-dev]

https://docs.stagehand.dev/v3/basics/extract#server-side-caching

see if we can implement this.

possible blockers:

• if we switch to env: BROWSERBASE, server-side caching becomes possible (typically enabled by default), but that requires infrastructure changes.
• Also, per-call cache control (serverCache: false/true) may not be available in our current version combo, even if baseline Browserbase caching works.

Other notes:

• Find out what exactly is cached. If it's the data, then we probably can't save it. If it's metadata or just some page selectors (css, html), then we possibly can enable it.

[nikitachapovskii-dev]

I would not do that

Even if we self-host Browserbase, I still wouldn’t enable Stagehand server-side caching in the public actor.

Self-hosting adds shared server-side state to a multi-user public actor. At that point we’re no longer just scraping a page and returning a result, we’re maintaining a cache layer with its own isolation, invalidation, and debugging story. For a store actor, that’s a pretty big tradeoff for a speed optimization.

It also isn’t a tiny switch. To use this cache we’d have to move from the current local Stagehand setup to env: "BROWSERBASE", which changes auth wiring, browser behavior, and likely network/proxy behavior too. That means more infra, more edge cases, and more ways for runs to behave differently.

And even if the cache is “safe enough” in terms of secrets, it still stores automation artifacts like prompts/selectors/config. In a shared actor, I’d rather avoid introducing any reusable cross-run server-side state unless we really need it and can scope it per tenant.

So yeah, I’d skip it.

cc @nicklamonov

[nicklamonov]

What if we move Stagehand to a separate standby actor and use it as a service within AI WS? It will have its own infra ready for requests and can possibly store cache when active. AI WS would be the main (if not the only) source of requests for it.
Do you think it could work better like that?

[nikitachapovskii-dev]

If we split Stagehand into its own standby actor/service, the main upside is architectural: we isolate the heavy browser + Stagehand runtime from AI WS, keep warm infra ready for requests, and make AI WS a thinner orchestrator. That could help with cold starts, browser reuse, retries, observability, and operational ownership, but for caching we need also browserbase.
With browserbase that unlocks actual Stagehand server-side caching.

Browserbase is clearly available as an external paid SaaS. Their official plans page lists Free, Developer ($20/mo), Startup ($99/mo), and Scale (custom)
https://docs.browserbase.com/account/billing/plans

But I couldn't find a public self-serve self-hosted Browserbase offering. Their website does say “Self-hosted available for ultimate control”, but I did not find public installation docs or a normal self-serve path for that. What I did find is an enterprise section and enterprise-only features like BYOS, which suggests that any self-hosted / special deployment path is likely an enterprise conversation
https://www.browserbase.com/enterprise