Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Original file line number Diff line number Diff line change
Expand Up @@ -77,7 +77,31 @@ change, the live proof.
relies on it — those cases provision an allocation and write prompts/publish under freshly
minted, never-created project ids to exercise metering in isolation. If a future consumer flow
needs unknown-project 404s, add a shared `requireProject(scope)` guard to the child writers and
seed the parent projects the quota cases use.
seed the parent projects the quota cases use. (Live evidence, 2026-06-29: GET / DELETE / publish /
add-prompt / by_tags / list-ai_models on a non-existent `project_id` all `404 { message: "not
found" }`. The mock now mirrors this for **DELETE project** — see the read-view section — but child
writes still succeed against a bogus project id, on purpose.)
- **No tenant / ownership model.** PE `GET projects` returns `200 { items: [] }` for *any* workspace
id; live `403 { invalid access attempt }`s an unknown/unowned workspace (and `200 { items: [] }`
only for an owned-but-empty one) — the mock has no notion of ownership. Likewise the **bearer gate
is presence-only**: a garbage `Bearer xxx` passes (live `401`s it). Both are deliberate — the
consumer always operates on workspaces it owns, with a real IMS token. (verified 2026-06-29)
- **Two live error layers collapsed into one.** Live has a gateway/auth layer (`application/json`,
`{detail}`/`{message}`) and a PE app/handler layer (**`text/plain`**, Go-validator/handler bodies:
`400` field-validation, `404 {message:"not found"}`, `500 {message:"internal server error"}`,
`409` conflict). The mock returns every error as `application/json` with a single envelope, and
does **not** reproduce the unguarded **500s** live throws for a bad input it never validates
(unknown `language_id`, `type:"seo"`, `GET projects` with no params / `?live=true`). The consumer
reads status, not error bodies (`mapError` redacts upstream), so this is shape-only. (2026-06-29)
- **Pagination is asymmetric across catalogs.** `GET /v1/languages` ignores `page`/`limit` (always
all 38, `page:1`) — the mock matches; `GET /v1/ai_models` and `GET projects` *do* paginate live —
the mock does not honor `page`/`limit` on those (returns the full list). No consumer paginates
these, so it is unmodelled rather than wrong. (2026-06-29)
- **`brand-topics` `prompts` is empty in the mock.** Live `GET brand-topics` returns each topic with
a large multilingual keyword array under `prompts`; the mock's two static topics carry
`prompts: []`, so serenity's volume-ranked `generateAndAttachPrompts` (which reads per-topic
`prompts`) gets nothing from the mock — a fidelity gap for the prompt-generation flow only. Seed
richer topics if a cross-repo e2e drives that path. (2026-06-29)
- **`listBrandUrls` always returns `200 { brand_urls }`; live can `404`.** Live (verified
2026-06-25) `GET …/aio/benchmarks/{bid}/brand_urls` returns `404 { message: "not found" }` when
`{bid}` is not the project's listable (auto-created main-brand) benchmark — even though a `POST`
Expand All @@ -93,18 +117,72 @@ change, the live proof.
immediately readable on purpose (deterministic test double); to exercise the consumer's
absent-benchmark / `404`-skip branches, model them **deterministically via seed/control state**,
never via time-based delays — see "Replicating live async behaviour" below.
- **Create ops report `existing_count: 0` unconditionally.** `POST .../aio/prompts/tagged` and
`POST .../ai_models/benchmarks` always return `existing_count: 0` — the mock models no dedup
against already-present rows, so the consumer's "some already present" branch
(`existing_count > 0`) cannot be exercised against this mock. Deliberate: the confirmed consumer
flows create into freshly scoped collections, and dedup fidelity adds store complexity no flow
reads. Add a name/domain-keyed existing-count if a future flow depends on it.
- **Create ops now model live dedup/conflict (#1745).** `POST .../aio/prompts/tagged` dedups by
prompt **text** — a text already present is not re-created, it is counted in `existing_count` and
gets no new id (live, 2026-06-29). `POST .../ai_models/benchmarks` instead treats a duplicate
brand name / alias / domain as a **hard `409`** `{ message: "ai benchmark conflict: duplicate
brand name or alias" }` (benchmarks and prompts dedup *differently* live) and creates nothing for
the batch — including two conflicting entries **within the same batch**, not just against
already-stored rows. Not modelled: **slice-uniqueness** (two US/en projects both succeed with distinct ids —
PE does not enforce it; that invariant lives only in serenity's `findBySlice`/DB), matching live.

## Project read-view fidelity (#1745, live-pinned 2026-06-29)

The created/published project read-view is reconstructed by the consumer into an addressable
**market** (`spacecat-api-service` `subworkspace-projects.js`), so its `settings.ai` shape is
load-bearing. Two fields are resolved by the mock to match live, and the publish action mutates the
stored project:

- **`settings.ai.language.name` = the ISO code, NOT the English display name.** Live returns the
ISO code (`"en"`) here on both the create response and every read-view, while `language.id` is the
catalog UUID sent on create (the SAME UUID `GET /v1/languages` returns under the English display
name `"English"` — only `name` differs between the two views). The consumer's `langOf` reads
`settings.ai.language.name` directly and lowercases it as the slice code, so the create factory
resolves `language_id` → ISO via the shared **`mock/language-catalog.js`** (`isoForLanguageId`),
which carries an `iso` column alongside the live `id`/`name`. Only `en` is live-verified; the
other ISO codes are standard ISO 639-1, best-effort so any market a downstream e2e exercises
round-trips. (`GET /v1/languages` still serves just `{ id, name }` — the `iso` column is
mock-internal.)
- **`settings.ai.country.name`** is populated from the country code via `Intl.DisplayNames` (region).
**Documented divergence:** live returns the short informal name (`"USA"` for `us`); Intl returns
`"United States"`. Accepted because **no consumer reads `country.name`** — `geoOf` resolves geo
from `country.code` → `resolveLocation` — so it is fidelity-only, and a hand-maintained
code→informal-name catalog would be unread maintenance. `location.id`/`name` are echoed from the
request verbatim (matches live for the consumer-driven create path).
- **Publish moves the stored read-view to live.** `POST .../publish` flips the stored project's
`publish_status` `draft` → `live` and stamps `published_at`, so a later `GET`/list reports the
published status (the slice's `mapPublishStatus(publish_status)` reads `live`). **`is_draft` is
left as-is** — live keeps `is_draft: true` after publish (only `publish_status`/`published_at`
change; live's full quirk is `true` on create → `false` on GET-as-draft → `true` post-publish),
and the consumer ignores `is_draft` entirely. The publish update is a no-op for an unknown project
id, so the metering cases (which publish under never-created ids) are unaffected.

## Endpoint status-code quirks the mock does NOT reproduce (live errors; mock is permissive)

Live returns errors for several call *shapes* the consumer never sends; the mock answers normally.
None affects the consumer, but they are documented so a future caller doesn't assume mock parity
(live captured 2026-06-29):

| call | live | mock | why it's safe |
| --- | --- | --- | --- |
| `GET /v1/.../projects` (no params) | **500** | 200 | the consumer always sends `?type=ai` |
| `GET /v1/.../projects?live=true` | **500** | 200 | the consumer never sends `live=` |
| `GET .../aio/prompts` | **405** | (DELETE only) | reads go via `by_tags` (POST) |
| `GET .../aio/prompts/tagged` | **405** | (POST only) | create is POST |
| `GET .../ci/competitors` | **405** | (PUT only) | update is PUT |
| `POST .../publish` | 202, `content-length: 0`, **no `Content-Type`** | 202 + `Content-Type: application/json` | consumer keys off status/`response.ok`; body empty either way (Counterfact limit, see responses.js) |

## Replicating live async behaviour (don't use timers)

Live Semrush is **eventually consistent**: a just-created prompt/brand-URL isn't listed yet, and a
project's main-brand benchmark is generated asynchronously (it didn't appear within ~60s of create,
even after a publish — verified 2026-06-25). The mock is deliberately **immediately consistent**.
even after a publish — verified 2026-06-25). A further, distinct case (live-pinned 2026-06-29):
the project **list** (`GET projects?type=ai`) lags a new project by ~4 s, and **prompt reads**
(`POST aio/prompts/by_tags`) are **publish-gated** — a created prompt stays invisible until the
project is published, then appears (this is a published-snapshot read, not lag). The end-to-end
consumer flow closes anyway because `POST /prompts` runs `publishAffected`; a test that lists
prompts *before* that publish would diverge from the immediately-consistent mock. The mock is
deliberately **immediately consistent**.

**Do NOT make the mock time-based** (delays, "appears after N seconds / N reads"). A test double's
value is determinism; introducing wall-clock async makes consumer tests flaky and slow, and the
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -43,6 +43,7 @@ import { createQuota } from './quota.js';
import { authError } from './auth.js';
import { emptyAck } from './responses.js';
import * as factories from './factories.js';
import { LANGUAGE_CATALOG } from './language-catalog.js';
import { SEEDS, DEFAULT_SEED } from './seeds.js';

/**
Expand Down Expand Up @@ -82,6 +83,10 @@ export class Context {
// literals — the factory is the single, tsc-checked source of truth for each shape, so the
// handlers can't drift from the spec the way duplicated literals would.
this.factories = factories;
// The canonical language catalog (mock/language-catalog.js). Exposed so the `GET /v1/languages`
// route serves it without duplicating the 38-entry list, mirroring how `factories` is shared —
// every route reads its lib data through `$.context`, never an import.
this.languageCatalog = LANGUAGE_CATALOG;
}

/**
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -14,54 +14,16 @@
* Static handler for GET /v1/languages — the language catalog the consumer
* (spacecat-api-service `listLanguages`) reads to resolve an ISO 639-1 code to a Semrush language
* UUID. The live shape is `{ page, total, items: [{ id, name }] }` — items carry NO key/icon (that
* is the ai_models shape). This is the FULL live catalog (38 languages, real UUIDs + names),
* captured verbatim 2026-06-25 so the mock returns the same taxonomy the consumer resolves against.
* is the ai_models shape) and NO `iso` (that field is mock-only, for the read-view resolver). The
* catalog (38 languages, real UUIDs + names, captured verbatim 2026-06-25) lives in
* `mock/language-catalog.js` so it is shared with the project read-view factory rather than
* duplicated here; it is exposed on the per-request context as `$.context.languageCatalog`.
* Materialized into `.counterfact/routes/` by the mock runner; excluded from coverage.
*/

const LANGUAGES = [
{ id: '76b01191-fa8b-4c1d-a71f-55ca8996669c', name: 'Afrikaans' },
{ id: '7d3efff9-f6b6-4818-ac6b-0d1d2c047803', name: 'Arabic' },
{ id: 'cfcdee66-f9b3-4f39-9066-141b8de1a52d', name: 'Armenian' },
{ id: 'c5d84e73-66f9-4f17-989f-9a421d17cfe3', name: 'Azerbaijani' },
{ id: 'ac1ace1e-22a1-40e9-b869-4fe2d0467603', name: 'Bulgarian' },
{ id: '728bef4c-94cf-4e14-bc06-56534751c71a', name: 'Chinese Simplified' },
{ id: '4fecf249-03fa-430a-b0b1-42421c5c1f7d', name: 'Chinese Traditional' },
{ id: 'e64e7f7d-11d9-4830-a022-f370e0a74bf7', name: 'Czech' },
{ id: '4f407a00-600e-4bdc-bf18-e5d8c7f758e3', name: 'Danish' },
{ id: '9090239f-5a3e-4101-a828-9bde09ad378b', name: 'Dutch' },
{ id: '5a0a33ed-7f5c-4901-befd-a042c0350da1', name: 'English' },
{ id: 'f8b95d67-6ac5-48c2-9ed1-73b9e52d7f92', name: 'Filipino' },
{ id: '0a28a154-72ac-421c-86cf-1c4d240f3cda', name: 'Finnish' },
{ id: '471aba35-ee1f-4207-b6f6-8c6550bbbab8', name: 'French' },
{ id: '5227c7cd-4136-4fca-919c-42e666787319', name: 'Georgian' },
{ id: 'e5282ae9-83a6-4ea3-b3cf-5e99d8f51eca', name: 'German' },
{ id: 'd7db513e-b5f6-4b11-9309-eda861451008', name: 'Greek' },
{ id: 'a2c8acc6-be1f-47de-94aa-3c2587922922', name: 'Hebrew' },
{ id: 'a3cd47fe-dc60-4cca-89e3-16984a1755af', name: 'Hindi' },
{ id: '495ccfcb-d61f-4544-9dc1-24d6393c9d18', name: 'Hungarian' },
{ id: 'd5376885-d3d8-4e6c-92ea-61d5ff9a7369', name: 'Indonesian' },
{ id: '3ea61bf2-ebf4-40b5-8a59-900e701d966d', name: 'Italian' },
{ id: '54bb0cca-b095-444b-be88-818a02e9beeb', name: 'Japanese' },
{ id: 'ad6bf224-42b0-46b6-b59a-21ea9d021fb8', name: 'Kazakh' },
{ id: 'c181d356-b65e-4ffa-b706-f61f96503d58', name: 'Korean' },
{ id: '1de63814-323e-4b2f-bd36-20fe15de14ae', name: 'Kurdish' },
{ id: '9bb46132-a85a-41c8-8d56-38bf5a050e76', name: 'Malay' },
{ id: 'a3b6e3f2-82f1-4b0b-8223-ae224d54bd63', name: 'Norwegian' },
{ id: 'd53c0677-c451-46b2-9650-83dbca109f25', name: 'Polish' },
{ id: 'a4e931c1-22c8-45d4-897e-a653acbc8126', name: 'Portuguese' },
{ id: 'd64b3fb0-62eb-4d7e-947b-952194314b97', name: 'Romanian' },
{ id: '19933246-6952-4929-ad0d-6cee874ea553', name: 'Russian' },
{ id: '6abdf315-f969-4643-894f-aefabe7ed057', name: 'Slovak' },
{ id: '29047b68-fe6f-4525-8ef7-8b1c2349af5d', name: 'Spanish' },
{ id: '5c400ede-f822-402e-be32-4a88049e74e4', name: 'Swedish' },
{ id: 'b8208c02-3e73-433b-b6c2-2784d0ba9bfa', name: 'Thai' },
{ id: '71630579-fd14-4357-81b3-0ae4ecd54975', name: 'Turkish' },
{ id: '12aa4de5-3526-4b96-909a-c033c1f4e32c', name: 'Vietnamese' },
];

/** GET — list the language catalog (each row shaped via the factory). */
/** GET — list the language catalog `{ id, name }` (each row shaped via the factory). */
export function GET($) {
const items = LANGUAGES.map((l) => $.context.factories.createLanguageMock(l));
const items = $.context.languageCatalog
.map(({ id, name }) => $.context.factories.createLanguageMock({ id, name }));
return $.response[200].json({ page: 1, total: items.length, items });
}
Original file line number Diff line number Diff line change
Expand Up @@ -47,9 +47,17 @@ export function PATCH($) {
return $.response[200].json(updated);
}

/** DELETE — remove a project. */
/**
* DELETE — remove a project. Live returns 404 `{ message: 'not found' }` for a non-existent project
* (verified 2026-06-29); the mock mirrors the status (bare 404, like the sibling GET — the
* text/plain error body is a documented simplification). The consumer treats a delete-404 as
* success (idempotent decommission), so the status change is consumer-equivalent.
*/
export function DELETE($) {
const { path, context } = $;
context.ops.projects.remove({ workspaceId: path.id }, path.project_id);
const removed = context.ops.projects.remove({ workspaceId: path.id }, path.project_id);
if (!removed) {
return { status: 404 };
}
return { status: 204 };
}
Original file line number Diff line number Diff line change
Expand Up @@ -18,6 +18,15 @@
* returns the disguised quota 405 — exactly the "publishing an empty-units child 405s" behaviour
* the consumer's `republishBestEffort` swallows (see mock/quota.js). Excluded from coverage
* (materialized handler).
*
* Stateful side-effect: on a successful publish the stored project's read-view is moved to the
* live state — `publish_status` flips `draft` → `live` and `published_at` is set — so a later
* `GET`/list reflects the publish (the slice's reported status reads `live`, not `draft`; #1745).
* `is_draft` is deliberately LEFT as-is: live keeps `is_draft: true` after publish (only
* `publish_status`/`published_at` change), and the consumer ignores `is_draft` anyway. The update
* is a no-op for an unknown project id (the quota cases publish under never-created ids), so the
* 202 ack is unaffected — child writes do not assert the parent project exists (see
* docs/mock-statefulness.md).
*/

/** POST — publish the draft → 202 (empty body); 405 when the workspace has 0 prompt units. */
Expand All @@ -30,6 +39,11 @@ export function POST($) {
contentType: 'application/json',
};
}
// Reflect the publish in the stored read-view: draft → live, stamp published_at, keep is_draft.
context.ops.projects.update({ workspaceId: path.id }, path.project_id, {
publish_status: 'live',
published_at: new Date().toISOString(),
});
// Empty body (content-length 0) like live. The explicit content type (via emptyAck) bypasses
// Counterfact's response negotiation, which would otherwise 406 under `Accept: application/json`.
return context.emptyAck(202);
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -17,15 +17,50 @@
* always a competitor (`main_brand: false` — the API cannot set the system-managed own brand).
* Writes to the SAME store key (`benchmarks:{ws}:{pid}`) the v1 list/delete use, so a subsequent
* list reflects it. Live: 200 `IDsWithStatsResponse` `{ ids, existing_count }` (verified
* 2026-06-25). Excluded from coverage (materialized handler).
* 2026-06-25). Unlike prompts (which dedup into `existing_count`), a benchmark whose brand name,
* alias, or domain collides with an existing one is a HARD **409** conflict
* `{ message: 'ai benchmark conflict: duplicate brand name or alias' }` (verified live 2026-06-29);
* the whole batch is rejected, nothing is created. Excluded from coverage (materialized handler).
*/
Comment thread
rainer-friederich marked this conversation as resolved.

/** POST — batch-create benchmarks (body: array) → 200 { ids, existing_count }. */
/** Case-insensitive identity tokens (brand name + aliases + domain) a benchmark conflicts on. */
const identityTokens = (b) => [
b?.brand_name,
b?.domain,
// Guard the spread: a raw request entry's `brand_aliases` may not be an array (spreading a
// string would iterate its characters into spurious tokens).
...(Array.isArray(b?.brand_aliases) ? b.brand_aliases : []),
].filter((t) => typeof t === 'string' && t.length > 0).map((t) => t.toLowerCase());

/** POST — batch-create benchmarks (body: array) → 200 { ids, existing_count }; 409 on conflict. */
export function POST($) {
const { path, body, context } = $;
const scope = { workspaceId: path.id, projectId: path.project_id };
const entries = Array.isArray(body) ? body : [];

// Reject if any incoming benchmark collides (brand name / alias / domain, case-insensitive) with
// one already in the project — the live hard-409, not a silent duplicate (#1745 second sweep).
// Each entry's tokens are folded into `taken` as we iterate, so two conflicting entries in the
// SAME batch collide too, not just against already-stored benchmarks.
const taken = new Set(context.ops.benchmarks.list(scope).flatMap(identityTokens));
const conflicts = entries.some((b) => {
const tokens = identityTokens(b);
const hit = tokens.some((t) => taken.has(t));
tokens.forEach((t) => taken.add(t));
return hit;
});
if (conflicts) {
return {
status: 409,
body: context.factories.createBasicResponseMock({
message: 'ai benchmark conflict: duplicate brand name or alias',
}),
contentType: 'application/json',
};
}

const created = context.ops.benchmarks.createMany(
{ workspaceId: path.id, projectId: path.project_id },
scope,
entries.map((b) => context.factories.createBenchmarkMock({
brand_name: b?.brand_name ?? '',
domain: b?.domain ?? '',
Expand Down
Loading
Loading