feat(project-engine-client): add GET /v1/url/resolve to the vendored client + mock (#1771)

<!-- mysticat-pr-skill -->

## 1. Abstract
Adds the Semrush Project Engine `GET /v1/url/resolve` endpoint to the
vendored `spacecat-shared-project-engine-client` — its generated types,
the spec overlay, and the stateful Counterfact mock. This is §1 of the
cross-repo brand-URL normalization spec.

## 2. Reasoning
When we add a brand URL to a Semrush AIO project we currently send the
raw URL verbatim (e.g. `https://www.lovesac.com`). Semrush stores brand
URLs in a canonical, scheme-less form (`lovesac.com`) and its own
frontend normalizes first by calling the Project Engine URL-resolve
endpoint before writing. Sending the raw URL produces a
duplicate/mismatched brand-URL entry that does not line up with the
canonical benchmark Semrush already holds. Fixing that requires the
endpoint to exist in the vendored typed client first — this PR is that
prerequisite. Reported by the Semrush team in the Elements-wrappers
review thread (2026-07-02/03); the live contract was captured directly
against prod.

## 3. High-level overview of the changes
Models a live endpoint that is absent from the vendored swagger,
following the established overlay + regenerate + mock pattern (precedent
CR1/CR8). No consumer-facing client code changes — the client is a thin
generic `openapi-fetch` wrapper, so once the path is in the generated
types a caller just does `client.GET('/v1/url/resolve', { params: {
query: { primary_url } } })`.

- **Spec overlay (CR16):** adds a `model.UrlResolveResponse` schema (`{
domain, primary_url, is_valid }`, all required) and the `GET
/v1/url/resolve` path (query param `primary_url` required,
`200`/`400`/`401` responses). Regenerating from the overlay updates
`src/generated/types.ts` and the pydantic models; the regen also catches
up a previously-stale pydantic artifact (a CR14 field that had only been
regenerated into the TS types).
- **Mock behaviour (before → after):** the endpoint did not exist → the
mock now canonicalizes `primary_url`. It strips the scheme and a leading
`www.` while preserving any other subdomain and the path
(`http://www.lovesac.com/products` → `lovesac.com/products`;
`blog.hubspot.com` stays `blog.hubspot.com`); `domain` is the
registrable apex (subdomain stripped too). Unresolvable/garbage input
returns `{ domain: '', primary_url: '', is_valid: false }` at HTTP `200`
— never an error — matching the live trap that consumers must check
`is_valid`.
- The canonicalization is a pure, unit-tested helper
(`mock/url-resolve.js`) exposed on the mock context like the existing
`tagId`/`parentIdField` helpers, rather than logic buried in the
coverage-excluded route handler.
- Endpoint inventory in `docs/mock-usage.md` updated to list the new
route.

Two intentional, documented mock simplifications: the apex is the last
two dot-labels (so a multi-part public suffix like `example.co.uk`
collapses to `co.uk`, unlike live's PSL), and an empty `primary_url=`
returns `200 is_valid:false` rather than `400` (Counterfact enforces the
`required` presence — a missing param still `400`s — but not the string
`minLength`).

## 4. Required information
- Spec: adobe/serenity-docs#25
- Other: brand-URL normalization tracked alongside the Serenity
mapping-table work adobe/serenity-docs#23

## 5. Affected / used mysticat-workspace projects
- `spacecat-api-service` — will consume this endpoint (§2 of the spec):
a new `resolveUrl` transport method + resolve-before-write on the
brand-URL CREATE/EDIT paths. Blocked on this package being released
(contract dependency).
- `mysticat-data-service` — the Serenity migration CLI (§3, on the
`feat/customer_onboarding` branch / #737) will add the same
`resolve_url` call. Also blocked on this release.

## 6. Additional information outside the code
- Booted the stateful Counterfact mock (`npm run test:e2e`) and drove
the new `/v1/url/resolve` route end-to-end through the real
`createSerenityProjectEngineApiClient` — verified the canonical case,
the subdomain/path-preserving case, the `is_valid:false` garbage case
(HTTP 200), and the missing-vs-empty `primary_url` request-validation
behaviour.
- The live contract behind the mock was captured directly against prod
Project Engine (`https://adobe-hackathon.semrush.com`) with a real IMS
bearer (serenity-docs#25 §0) — method, param, response shape, error
codes, and the `is_valid:false` empty-string trap are ground truth, not
assumptions.

## 7. Test plan
- (a) Local: unit suite plus the offline overlay freshness gate (asserts
CR16 both applies and is necessary against the vendored swagger), and
the `MOCK_E2E=1` end-to-end suite driving the booted mock through the
typed client — all exercised locally.
- (b) Per-env: this package ships no runtime service, so there is no
eph/dev/stage/prod deploy to verify. After merge, semantic-release
publishes a new npm version and the GHCR mock Docker image republishes
on the release tag; consumers verify by bumping
`@adobe/spacecat-shared-project-engine-client` and calling
`client.GET('/v1/url/resolve', { params: { query: { primary_url } } })`
against the published mock image in their integration tests.

## 8. Deployment & merge order
- This PR is the hard prerequisite for the rest of serenity-docs#25 — it
must merge and release (npm + GHCR mock image) first.
- After release: the api-service change (§2) bumps the dependency and
adds the resolve-before-write; the migration CLI change (§3) does the
same. Both are `related` and neither can build/test until this version
publishes.

Ordered sequence: merge + release this PR → api-service §2 + migration
§3 bump the dep and proceed in parallel.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

---------

Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

Author: rainer-friederich

packages/spacecat-shared-project-engine-client/docs/mock-usage.md

@@ -135,6 +135,7 @@ method that calls it.
 | Method + path | Consumer | Behaviour |
 | --- | --- | --- |
 | `GET /v1/languages` | `listLanguages` | language catalog → `{ page, total, items }` |
+| `GET /v1/url/resolve` | brand-URL write path (api-service `resolveUrl`, migration CLI — serenity-docs#25) | canonicalize a raw URL → `{ domain, primary_url, is_valid }` (overlay CR16). `primary_url` strips scheme + `www.` (subdomain + path preserved), `domain` is the registrable apex; garbage input → empty strings + `is_valid:false` at HTTP `200` (never an error). `primary_url` is `required` — a MISSING param `400`s; an EMPTY value falls through to `200 is_valid:false` (Counterfact enforces presence, not the spec's `minLength`). Mock apex = last-two-labels (a multi-part suffix like `example.co.uk` → `co.uk` is a documented simplification vs live's PSL) |
 | `GET /v1/workspaces/{id}/brand-topics` | `getBrandTopics` | top-level array `[{ topic, volume, prompts }]`; `domain` + `country` are `required` query params — omitting either → `400` (enforced by request validation, matching live) |
 | `PUT /v1/workspaces/{id}/projects/{project_id}/ci/competitors` | `updateCiCompetitors` | full replace → `{ ci_competitors }` |
 | `GET /v2/workspaces/{id}/projects/{project_id}/aio/init_status` | `getInitStatus` | `{ initialized }`. **Live route is `/v2`** — the vendored swagger's `/v1` path 404s (overlay CR8, verified live across 4 projects). The api-service consumer still calls `/v1` today (a pre-existing bug — it degrades to `initialized: null`). |

packages/spacecat-shared-project-engine-client/mock/context.js

@@ -47,6 +47,7 @@ import { LANGUAGE_CATALOG } from './language-catalog.js';
 import { AI_MODEL_CATALOG } from './ai-model-catalog.js';
 import { tagId } from './tag-id.js';
 import { parentIdField } from './parent-id.js';
+import { resolveUrl } from './url-resolve.js';
 import { SEEDS, DEFAULT_SEED } from './seeds.js';
 
 /**
@@ -104,6 +105,11 @@ export class Context {
     // `PATCH /aio/tags/{tag_id}` — share one coerce-then-spread definition and can't drift, the
     // same `$.context` lib-helper convention as `tagId` above.
     this.parentIdField = parentIdField;
+    // The URL canonicalizer (mock/url-resolve.js). Exposed so the `GET /v1/url/resolve` route
+    // computes the normalized `{ domain, primary_url, is_valid }` through one pure, unit-tested
+    // function rather than inline in the coverage-excluded handler — same `$.context` lib-helper
+    // convention as `tagId`. The consumer resolves a raw brand URL through this before writing it.
+    this.resolveUrl = resolveUrl;
   }
 
   /**

packages/spacecat-shared-project-engine-client/mock/counterfact/routes/v1/url/resolve.js

@@ -0,0 +1,32 @@
+/*
+ * Copyright 2026 Adobe. All rights reserved.
+ * This file is licensed to you under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License. You may obtain a copy
+ * of the License at http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software distributed under
+ * the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR REPRESENTATIONS
+ * OF ANY KIND, either express or implied. See the License for the specific language
+ * governing permissions and limitations under the License.
+ */
+
+/**
+ * Static handler for GET /v1/url/resolve — the URL-canonicalization endpoint the consumer
+ * (spacecat-api-service brand-URL write path + the Serenity migration CLI) calls before writing a
+ * brand URL, so the stored value matches the canonical benchmark Semrush already holds. Added to
+ * the spec by overlay CR16. Live shape: `{ domain, primary_url, is_valid }` (verified 2026-07-03
+ * against prod — serenity-docs#25 §0). The canonicalization lives in `mock/url-resolve.js` (exposed
+ * as `$.context.resolveUrl`) so it is unit-tested on its own; the empty/invalid default lives in
+ * the factory (`createUrlResolveMock`), so a garbage input returns `{ domain: '', primary_url: '',
+ * is_valid: false }` with HTTP 200 — never an error. A MISSING `primary_url` 400s at request
+ * validation (the required query param) before this handler runs; an EMPTY value falls through here
+ * and returns 200 is_valid:false (Counterfact enforces presence but not the spec's minLength:1, so
+ * this one edge diverges from live's empty-400 — benign, the consumer keys off is_valid).
+ * Materialized into `.counterfact/routes/` by the mock runner; excluded from coverage.
+ */
+
+/** GET — canonicalize `primary_url` to `{ domain, primary_url, is_valid }` via the factory. */
+export function GET($) {
+  const overrides = $.context.resolveUrl($.query.primary_url);
+  return $.response[200].json($.context.factories.createUrlResolveMock(overrides));
+}

packages/spacecat-shared-project-engine-client/mock/factories.js

@@ -41,6 +41,7 @@
 /** @typedef {Schemas['http_server.BasicResponse']} BasicResponse */
 /** @typedef {Schemas['model.AIOProjectInitializedResponse']} InitStatus */
 /** @typedef {Schemas['model.CICompetitor']} CiCompetitor */
+/** @typedef {Schemas['model.UrlResolveResponse']} UrlResolve */
 
 import { isoForLanguageId } from './language-catalog.js';
 import { findCatalogEntryByKey } from './ai-model-catalog.js';
@@ -428,3 +429,18 @@ export const createCiCompetitorMock = (overrides = {}) => ({
   color: '',
   ...overrides,
 });
+
+/**
+ * A URL-resolve result (`UrlResolveResponse`) — the `GET /v1/url/resolve` body. The DEFAULT is the
+ * live invalid/unresolvable shape (`{ domain: '', primary_url: '', is_valid: false }`, HTTP 200),
+ * so the route handler passes the {@link resolveUrl} overrides for a valid input and nothing (→
+ * this empty default) for an invalid one. All three fields are required by the schema (CR16).
+ * @param {Partial<UrlResolve>} [overrides]
+ * @returns {UrlResolve}
+ */
+export const createUrlResolveMock = (overrides = {}) => ({
+  domain: '',
+  primary_url: '',
+  is_valid: false,
+  ...overrides,
+});

packages/spacecat-shared-project-engine-client/mock/url-resolve.js

@@ -0,0 +1,79 @@
+/*
+ * Copyright 2026 Adobe. All rights reserved.
+ * This file is licensed to you under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License. You may obtain a copy
+ * of the License at http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software distributed under
+ * the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR REPRESENTATIONS
+ * OF ANY KIND, either express or implied. See the License for the specific language
+ * governing permissions and limitations under the License.
+ */
+
+// @ts-check
+
+/**
+ * The URL-canonicalization the live Semrush Project Engine `GET /v1/url/resolve` performs — the
+ * single source of truth behind the `mock/counterfact/routes/v1/url/resolve.js` handler, exposed on
+ * the per-request context as `context.resolveUrl` (every route reads its lib helpers through
+ * `$.context`, never an import — see {@link Context}). Kept a pure function so it is unit-tested on
+ * its own (the route handler is coverage-excluded), the same convention as {@link tagId} /
+ * `parentIdField`.
+ *
+ * Live contract (verified 2026-07-03 against prod `adobe-hackathon.semrush.com` — serenity-docs#25
+ * §0): `primary_url` strips the scheme and a leading `www.` but PRESERVES any other subdomain and
+ * the path (`http://www.lovesac.com/products` → `lovesac.com/products`; `https://blog.hubspot.com`
+ * → `blog.hubspot.com`); `domain` is the registrable apex, stripping the subdomain too
+ * (`blog.hubspot.com` → `hubspot.com`). For unresolvable/garbage input the live API returns
+ * `{ domain: '', primary_url: '', is_valid: false }` with HTTP **200** (not an error) — the factory
+ * default {@link createUrlResolveMock} IS that empty/invalid shape, so this returns only the
+ * OVERRIDES for a valid input and an empty object (→ defaults) for an invalid one.
+ *
+ * MOCK SIMPLIFICATION: the apex is the last two dot-labels, so a multi-part public suffix
+ * (`example.co.uk`) collapses to `co.uk` rather than `example.co.uk`. The live API uses the real
+ * Public Suffix List. Faithful for the single-suffix domains the consumer resolves in practice
+ * (`lovesac.com`, `hubspot.com`); documented here rather than pulling a PSL dependency into it.
+ *
+ * @param {unknown} primaryUrl the raw `primary_url` query value
+ * @returns {{ domain: string, primary_url: string, is_valid: boolean } | {}} the resolve overrides
+ *   for a valid URL, or `{}` (→ the empty/invalid factory default) for unresolvable input
+ */
+export const resolveUrl = (primaryUrl) => {
+  const raw = typeof primaryUrl === 'string' ? primaryUrl.trim() : '';
+  if (!raw) {
+    return {};
+  }
+
+  let parsed;
+  try {
+    // A scheme-less input (`www.lovesac.com`) is not a valid URL on its own, so prepend `https://`
+    // for parsing; an input that already carries a scheme is parsed as-is (its scheme is dropped
+    // from the output regardless).
+    const hasScheme = /^[a-z][a-z0-9+.-]*:\/\//i.test(raw);
+    parsed = new URL(hasScheme ? raw : `https://${raw}`);
+  } catch {
+    return {};
+  }
+
+  let host = parsed.hostname.toLowerCase();
+  if (host.startsWith('www.')) {
+    host = host.slice(4);
+  }
+
+  const labels = host.split('.');
+  const tld = labels[labels.length - 1];
+  // Require a dotted hostname whose last label is an alphabetic TLD (>=2 letters). This rejects a
+  // bare single-label host, an empty label (`foo..com`), and an IP literal (`192.168.0.1`, whose
+  // last label is numeric) — all of which the live API returns `is_valid: false` for.
+  if (labels.length < 2 || labels.some((label) => label === '') || !/^[a-z]{2,}$/.test(tld)) {
+    return {};
+  }
+
+  // Live preserves the path but drops a bare `/`; query/hash are not part of a brand URL.
+  const path = parsed.pathname === '/' ? '' : parsed.pathname;
+  return {
+    domain: labels.slice(-2).join('.'),
+    primary_url: `${host}${path}`,
+    is_valid: true,
+  };
+};

packages/spacecat-shared-project-engine-client/python/serenity_project_engine/__init__.py

@@ -81,6 +81,15 @@ info:
 #      `parent_id: null` to promote a child tag to a root (verified 2026-07-02 against prod). The
 #      vendored swagger types it as a plain string, so request validation rejected the literal
 #      null. Remove if Semrush documents the nullability upstream.
+# CR16 Add GET /v1/url/resolve — the URL-canonicalization endpoint the live Semrush frontend calls
+#      before writing a brand URL, absent from the vendored spec. Live: GET
+#      /v1/url/resolve?primary_url=<raw> -> 200 { domain, primary_url, is_valid } (verified
+#      2026-07-03 against prod adobe-hackathon.semrush.com — serenity-docs#25 §0). `primary_url` is
+#      the canonical brand-URL value (scheme + `www.` stripped, subdomain + path PRESERVED);
+#      `domain` is the registrable apex (subdomain stripped too); `is_valid` is false — with EMPTY
+#      `domain`/`primary_url` strings, still HTTP 200 — for unresolvable/garbage input. Consumers
+#      resolve a raw URL through this before writing it as a brand URL so the stored value matches
+#      the canonical benchmark Semrush already holds. Remove when Semrush ships the path upstream.
 #
 # Freshness gate: test/overlay.test.js asserts every action above still APPLIES (matches >0
 # nodes) and is still NECESSARY (changes the converted vendored swagger) — so a re-vendored
@@ -415,3 +424,63 @@ actions:
     update:
       properties:
         parent_id: { type: string, nullable: true }
+
+  # ── CR16: Add GET /v1/url/resolve ────────────────────────────────────────
+  # Live: 200 -> { domain, primary_url, is_valid }. Verified 2026-07-03 against prod
+  # (serenity-docs#25 §0). All three fields are ALWAYS present — even the is_valid:false case returns
+  # empty `domain`/`primary_url` strings rather than omitting them — so all three are marked required
+  # (the same faithful-`required` philosophy as CR5: the mock's handler always populates them and the
+  # live API always returns them). The query param is `required` + `minLength: 1` to document the live
+  # contract (missing OR empty → 400). Counterfact's request validation enforces the `required`
+  # presence (a MISSING param 400s, as with the brand-topics params) but not the string `minLength`,
+  # so an EMPTY value falls through to the handler, which returns 200 is_valid:false — the one benign
+  # mock divergence from live's empty-400, documented on the route handler.
+  - target: "$.components.schemas"
+    description: "CR16: add the UrlResolveResponse schema"
+    update:
+      "model.UrlResolveResponse":
+        type: object
+        required: [domain, primary_url, is_valid]
+        properties:
+          domain: { type: string }
+          primary_url: { type: string }
+          is_valid: { type: boolean }
+
+  - target: "$.paths"
+    description: "CR16: add the GET /v1/url/resolve path"
+    update:
+      "/v1/url/resolve":
+        get:
+          summary: "Resolve a URL to its canonical brand-URL form"
+          # Folded scalar (>-): authored multi-line for review, emitted as one line.
+          description: >-
+            Canonicalize a raw URL to the form Semrush stores as a brand URL. `primary_url` strips
+            the scheme and a leading `www.` but preserves any other subdomain and the path;
+            `domain` is the registrable apex (subdomain stripped too). For unresolvable or garbage
+            input `is_valid` is false and `domain`/`primary_url` are empty strings — still HTTP 200,
+            so consumers MUST check `is_valid` and never write the empty value.
+          operationId: "resolve-url"
+          tags: ["AIO"]
+          parameters:
+            - { name: primary_url, in: query, required: true, schema: { type: string, minLength: 1 } }
+          responses:
+            "200":
+              description: "OK"
+              content:
+                application/json:
+                  schema:
+                    $ref: "#/components/schemas/model.UrlResolveResponse"
+            "400":
+              description: "Bad Request (missing or empty primary_url)"
+              content:
+                application/json:
+                  schema:
+                    $ref: "#/components/schemas/http_server.BasicResponse"
+            "401":
+              description: "Unauthorized"
+              content:
+                application/json:
+                  schema:
+                    $ref: "#/components/schemas/http_server.BasicResponse"
+          security:
+            - imsBearer: []