Feat/wb ab testing

[SvenAlHamad]

Summary

Adds end-to-end A/B testing to the Website Builder: editors define an experiment on a page, create full-page variants, set a traffic split, activate/deactivate, and edit each variant on the normal editor canvas. Variants are served server-side by the SDK with deterministic bucketing, a stable split, an instant kill-switch, and a provider seam for an exposure event (PostHog wiring is a follow-up). Built into core (not an extension).

---

1. Storage & serving mechanics

Data model — two private Headless CMS models (never in the page's revision timeline):

• wbyWbExperiment — belongs to a page (pageEntryId, baselineRevisionId), holds status (draft → running → stopped), name, trafficSplit ({ control, variants: { : weight } }), targeting, analytics, timestamps, winningVariantId.
• wbyWbVariant — belongs to an experiment; a full page-like content snapshot (properties/elements/bindings/metadata/extensions) copied from the baseline, with its own draft/ready status.

Approval / governance — variants never serve unpublished. Serving reads published experiment/variant content by entryId; a PageAfterPublish handler cascades a publish to the page's running experiment + ready variants, so the page's existing approval workflow is the single gate.

How a variant is chosen (per request, in the SDK) — getPageWithExperiment(path):

1. Fetch the control page + the active experiment for the path (cached per path). No running experiment → control.
2. Kill-switch: an uncached getExperimentPaused check — a paused experiment instantly reverts to control (content stays cached).
3. Resolve a visitor context (visitor id from cookie, country from CDN geo headers, device from UA).
4. Bucketing (assignVariant): targeting gate → inclusion gate (hash(visitorId:experimentId:inclusion) vs trafficPercentage) → weighted pick over control + ready variants using cyrb53(visitorId:experimentId).
5. Overlay the chosen variant's content onto the control; forced ?wb-variant= bypasses bucketing for QA.

Persistence & stickiness — no assignment is stored. Only a wb_ab_vid cookie (random UUID, 1-year, no PII, set in middleware) is persisted; the same visitor id always hashes to the same bucket, so assignment is sticky and stateless.

Split correctness — cyrb53 is uniform across visitor ids, so weight configured split; inclusion and bucket selection use independenthash seeds.

Caching — reads cache and key only on path / variant id, never on visitor id, so the CDN cache stays shareable; only the kill-switch read is no-store.

Permissions — experiments/variants have no separate permission entities; every read/write/publish/delete checks the corresponding wb.page action. The read-only frontend key (wb.page:r / $wb.readonly) can serve and preview them
---
2. SDK changes (website-builder-sdk / -react / -nextjs)

• ContentSdk exposes getPageExperiment, getVariantContent, getExperimentPaused, and a bound getPageWithExperiment(path, options) — bucketing, targeting, overlay, exposure, caching, and the kill-switch are all encapsulated. optiones the forced-variant id) or an explicit forcedVariantId, plusvisitor overrides.
• Kill-switch added: getExperimentPaused on IDataProvider/DefaultDaRIMENT_PAUSED query + a noStore path on ApiClient.query;getPageWithExperiment reverts to control for a non-running or paused experiment.
• @webiny/website-builder-react re-exports the experiment surface (eVisitorContext, assignVariant, analytics-provider registration,FORCED_VARIANT_PARAM, DEFAULT_VISITOR_COOKIE, and all experiment types).
• @webiny/website-builder-nextjs ships a middleware helper at @webidleware (createWebsiteBuilderMiddleware + default middleware/config)handling preview/draft mode and the wb_ab_vid cookie.

---

3. Implementing the SDK in a frontend (to test this PR)

▎ These packages are unpublished on the branch, so link/build them n link the four @webiny/website-builder-* packages, or point the appat the built dist). The demo app in this workspace is already refactored to the shape below.

middleware.ts
export { middleware, config } from "@webiny/website-builder-nextjs/

app/[[...slug]]/page.tsx — the whole A/B path is one call:
import { contentSdk } from "@webiny/website-builder-nextjs";
import { initializeContentSdk, getTenant } from "@/src/contentSdk";

const { isEnabled: isPreview } = await draftMode();
initializeContentSdk({ preview: isPreview, tenantId: await getTenant() });

const { page } = await contentSdk.getPageWithExperiment(path, { searchParams: search });
return ;

Verify:

1. In Admin, open a page → Experiments → create one with two variants and a split → edit each variant on the canvas → Activate → publish the page.
2. Split/stickiness: load the page across several fresh sessions (cn tracks the weights; within one session it stays put.
3. Forced preview: ?wb-variant= renders that variant (uncached, QA only).
4. Kill-switch: pause the experiment → the page reverts to control

---

Known limitations / follow-ups

• Exposure event not emitted yet — the provider seam exists; wiring the PostHog exposure call is the next step.
• Draft-variant "preview in a new tab" stays app-side (reads an unph public serving deliberately doesn't expose).
• Variant add/remove is locked when editing an existing experiment; no analytics/results UI yet.
• Feature should be only available behind a feature flag, later this should be behind a WCP license

[Pavel910]

I've reviewed the api-* part of the PR. In general the only important thing to change is that we never return null from use cases. It's a general rule - use cases either return a record, or throw an error. The errors are nicely typed, and can be easily used for flow control. Adding more edge cases is easy, you simply add more error types.

[SvenAlHamad]

(c/p from my Claude Code :) )
Thanks Pavel — all addressed in 8d3f9a8:

• No null from use cases/repositories: GetActiveExperimentForPath and GetActiveExperimentForRevision now return a record or Result.fail(...). Added NoActiveExperimentError and ExperimentPausedError; the two "get active" repositories fail with NoActiveExperimentError instead of Result.ok(null).
• execute naming: renamed the repository method to execute.
• Paused error class: the use case now returns Result.fail(new ExperimentPausedError(...)).
• Split file: ExperimentPause is now one file per use case (Pause/Resume/IsPaused).
• Top-level features: added ExperimentFeature and VariantFeature; index.ts registers only those two.

One thing to flag: the serving query getPageExperiment legitimately has a "no experiment on this page" outcome. To keep that from being an error the SDK logs on every plain page, the resolver maps NoActiveExperiment/Paused to a null data response (still typed failures inside the use case, just translated at the GraphQL boundary). Happy to adjust if you'd prefer a different convention there.

[brunozoric]

I wrote few comments, but these stand out:

• Most abstractions.ts have same problem - more than one abstraction defined.
• Too much useEffect in your components. Move to presenters.

[SvenAlHamad]

Thanks @brunozoric — all addressed. Rundown:

Models & keys — 783ace2

• Dropped the WEBINY_API_LEGACY_MODELS conditional; models are declared directly as wbyWbExperiment / wbyWbVariant.
• Keys now go through the repo's slugify package rather than the hand-rolled helper.
• On .compress() / .encrypt(): left off deliberately — the stored fields are small config JSON (traffic split, targeting rules, analytics id + key), so there's nothing large enough to warrant compression and no secrets/credentials to encrypt. Happy to add them if you'd still prefer.

One abstraction per file — d547cbc

• Every experiment/variant abstractions.ts is split into abstractions/.ts (one interface/type per file), matching the api-opensearch layout you pointed to. Each feature's index.ts stays the public barrel.

Presenters (webhooks pattern) — ff6d7ed, refined in cf58418

• Rewrote the admin experiments UI to the webhooks/src/admin shape: MobX Presenter + ViewModel + DataSource + feature (resolve in singleton scope) + observer views. The React context and useEffect-based data loading are gone — a hub presenter plus a mobx reaction handle experiments/selection and the per-card variant summaries, with init() / dispose() on the view lifecycle.
• Child components take the presenter and drive it directly (no split callback props), and all display data (bucket colours, current bucket, switcher items) is derived in the ViewModel rather than in the components. The only remaining effects are editor selectors (isReadOnly) and the autosave subscription, mirroring the framework's own PageAutoSave.

One intentional deviation — presenter scoping. The ExperimentsManager (drawer) and ExperimentForm presenters use the standard child-container + DiContainerProvider scoping. The ExperimentsEditor hub presenter, though, is an app-wide singleton (registered in Extension.tsx). Unlike a webhooks page — where one WebhookListView hosts the whole feature under a single provider — the experiments UI mounts in disjoint page-editor slots with no common React ancestor: the switcher/indicator in a top-bar Ui.TopBar.Action, the toolbar in a Ui.Content.Element, and the surface swap in PageEditor. They all have to observe the same selection/paused state, so a shared singleton is the only way to bridge them. It's initialised per page via presenter.init(page.id), and only one editor is open at a time, so it's effectively page-scoped. Happy to wrap the whole editor in an editor-level DiContainerProvider instead if you'd rather keep it uniform.