fix(project-engine-client): bypass Counterfact 406 on empty-body 2xx mock acks (#1744)

<!-- mysticat-pr-skill -->

## 1. Abstract
Makes the Project Engine Counterfact mock return its intended empty-body
2xx ack (publish, batch-delete, update-benchmark) when the request
carries `Accept: application/json`, instead of `406 Not Acceptable`.

## 2. Reasoning
The mock 406'd on every empty-body 2xx handler whenever the caller sent
`Accept: application/json` — which the serenity transport in
spacecat-api-service does on every call. A bare `{ status, body }`
return goes through Counterfact's response content-negotiation, and an
empty-body `202` declares no `application/json` representation, so
Counterfact answered `406` rather than the `202`. This is a
mock-fidelity bug, not a production bug: the real Semrush gateway
returns the empty ack regardless of `Accept`. The 406 blocked the
serenity create/activate API-level IT increment — `createMarket` /
`activate` create a draft then `publish`, and the publish step failed
upstream with a `502 serenityUpstreamError`.

## 3. High-level overview of the changes
A new `emptyAck()` helper returns a raw `{ status, body: '', contentType
}` envelope. The explicit content type is the signal that makes
Counterfact skip response negotiation/validation — the same bypass the
mock's auth 401 and quota 405 already rely on. The helper is wired onto
the per-request Context as `context.emptyAck`, mirroring how the auth
guard is exposed.

Behaviour delta:

- Before: the four empty-body `202` ack handlers (publish, benchmarks
batch-delete, benchmark update, brand-urls batch-delete) returned `406`
under `Accept: application/json`; `*/*` or no `Accept` got `202`.
- After: all four return `202` with an empty body regardless of
`Accept`, matching the live gateway's `content-length: 0` ack.

The three `204 No Content` handlers (project delete, ai_models
batch-delete, prompts batch-delete) are intentionally left as bare `{
status: 204 }`: Counterfact special-cases `204` and serves it without
negotiating, so they never 406. A `204` carries no body and no
`Content-Type`, which the bare return preserves — keeping them closer to
the live response than routing them through the helper would.

## 4. Required information
- Jira / issue:  #1742
- Other: blocked IT increment
adobe/spacecat-api-service#2709 · PE mock image
#1732

## 5. Affected / used mysticat-workspace projects
- spacecat-api-service — consumer (contract). Its serenity API-level IT
suite boots this mock as a GHCR image; the create/activate lifecycle
increment is unblocked once the mock image carrying this fix is
published and the tag is bumped there.

## 6. Additional information outside the code
Booted the mock locally (`MOCK_SEED=workspace-with-data`) and replayed
the issue's reproduction against the seeded workspace/project under
`Accept: application/json`:

- publish, benchmarks batch-delete, benchmark PUT, brand-urls
batch-delete — each now responds `202` (was `406`); the publish response
body is `0` bytes, matching the live `content-length: 0` ack.
- The three `204` delete handlers still respond `204` under the same
`Accept: application/json`, confirming they were correctly left
untouched.

Validated against the real prod gateway with an IMS prod token
(workspace `bfe3c7eb-469f-4331-9b10-7b1319bb6460`), probing the
empty-2xx ack path non-destructively via a no-op batch delete (`{"ids":
[]}`):

- Prod returns `202` with `content-length: 0` and **no `Content-Type`
header**.
- The mock matches prod on status and (empty) body, but adds one
`Content-Type: application/json` header prod does not send. This is a
known, unavoidable Counterfact limitation — a truthy content type is the
only way to get a 0-byte body past Counterfact's negotiation; without it
the response either `406`s or falls back to a `text/plain` "Accepted"
8-byte body (verified across both the raw-return and typed
`$.response[202]` paths). The extra header is invisible to the consumer,
which keys off `response.ok`/status. Documented inline in
`mock/responses.js`.

## 7. Test plan
- Local: booted the mock and ran the issue's repro `curl` for all four
ack endpoints with `Accept: application/json` (results in section 6);
also ran the package e2e suite, which drives the real typed client
against a live mock and now sends `Accept: application/json` on the
raw-fetch ack assertions to pin the regression.
- To verify in the consuming repo: build/publish the mock image from
this branch (or the released tag), bump the mock image tag in
adobe/spacecat-api-service#2709 , and run the
serenity create/activate IT lifecycle — `POST .../markets`
(createMarket) and `POST .../activate` should reach `201` / `200`
instead of `502`.

## 8. Deployment & merge order
- Related: adobe/spacecat-api-service#2709
(related) — consumes the fixed mock image; its create/activate increment
depends on this.
- Related: #1732 (related)
— the release-tag workflow that publishes the mock GHCR image.

Order: merge this PR → semantic-release publishes a new package version
and the mock-image workflow publishes a new GHCR tag → bump that tag in
the api-service PR to unblock its IT.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

---------

Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

Author: rainer-friederich

packages/spacecat-shared-project-engine-client/CLAUDE.md

@@ -70,10 +70,17 @@ entity through them — `context.factories.createBenchmarkMock({ brand_name, dom
 catalog handlers map their data rows through `createLanguageMock` / `createAiModelMock` /
 `createBrandTopicMock` — rather than emitting an inline literal that the untyped handler (the
 `@ts-check` exception) couldn't catch drifting. The factory is the single, tsc-checked source of
-truth for each shape. This holds for EVERY handler, including the trivial envelope shapes: the
-202 acks build a `createBasicResponseMock`, `getInitStatus` a `createInitStatusMock`, and
-`updateCiCompetitors` maps through `createCiCompetitorMock` — there are no inline-literal
-exceptions left.
+truth for each shape. This holds for EVERY handler that returns an entity, including the trivial
+envelope shapes: `getInitStatus` builds a `createInitStatusMock` and `updateCiCompetitors` maps
+through `createCiCompetitorMock` — there are no inline-literal entity exceptions left.
+
+The empty-body action acks (publish, batch-delete, update-benchmark) are NOT entities — they
+mirror the live gateway's `content-length: 0` ack, so they return `context.emptyAck(202)` from
+`mock/responses.js` rather than a factory. `emptyAck` is a raw `{ status, body: '', contentType }`
+envelope (like `auth.js`'s 401 and `quota.js`'s 405): the explicit `contentType` makes Counterfact
+skip response content-negotiation, which otherwise **406**s an empty-body 2xx under `Accept:
+application/json` (the header the real serenity transport always sends). A bare `{ status: 204 }`
+delete needs no helper — Counterfact serves 204 No Content without negotiating.
 
 ## Spec corrections: the overlay is the single source of truth
 

packages/spacecat-shared-project-engine-client/mock/context.js

@@ -41,6 +41,7 @@ import { InMemoryStore } from './store.js';
 import { createStatefulOps } from './stateful.js';
 import { createQuota } from './quota.js';
 import { authError } from './auth.js';
+import { emptyAck } from './responses.js';
 import * as factories from './factories.js';
 import { SEEDS, DEFAULT_SEED } from './seeds.js';
 
@@ -70,6 +71,12 @@ export class Context {
     // Bearer-auth gate. Stateless, so it is a plain reference to the pure guard; every real route
     // calls `context.authError($.headers)`, the `__*` control routes do not (see mock/auth.js).
     this.authError = authError;
+    // Empty-body 2xx ack shape (mock/responses.js). Stateless, so a plain reference to the pure
+    // helper. Action-ack handlers (publish, batch-delete, update-benchmark) return
+    // `context.emptyAck(202)` so the empty body carries an explicit content type and bypasses
+    // Counterfact's response negotiation, which otherwise 406s under `Accept: application/json`.
+    // 204 No Content handlers don't need it — Counterfact serves 204 raw (see mock/responses.js).
+    this.emptyAck = emptyAck;
     // The typed entity factories (mock/factories.js), exposed so route handlers build every
     // response entity through them (`context.factories.createXMock(...)`) instead of inline
     // literals — the factory is the single, tsc-checked source of truth for each shape, so the

packages/spacecat-shared-project-engine-client/mock/counterfact/routes/v1/workspaces/{id}/projects/{project_id}/ai_models/benchmarks.js

@@ -48,6 +48,7 @@ export function DELETE($) {
     { workspaceId: path.id, projectId: path.project_id },
     body?.ids ?? [],
   );
-  // Empty body (content-length 0) like live, not Counterfact's default "Accepted" reason.
-  return { status: 202, body: '' };
+  // Empty body (content-length 0) like live. The explicit content type (via emptyAck) bypasses
+  // Counterfact's response negotiation, which would otherwise 406 under `Accept: application/json`.
+  return context.emptyAck(202);
 }

packages/spacecat-shared-project-engine-client/mock/counterfact/routes/v1/workspaces/{id}/projects/{project_id}/ai_models/benchmarks/{benchmark_id}.js

@@ -27,6 +27,7 @@ export function PUT($) {
     path.benchmark_id,
     { ...body },
   );
-  // Empty body (content-length 0) like live, not Counterfact's default "Accepted" reason.
-  return { status: 202, body: '' };
+  // Empty body (content-length 0) like live. The explicit content type (via emptyAck) bypasses
+  // Counterfact's response negotiation, which would otherwise 406 under `Accept: application/json`.
+  return context.emptyAck(202);
 }

packages/spacecat-shared-project-engine-client/mock/counterfact/routes/v1/workspaces/{id}/projects/{project_id}/publish.js

@@ -30,6 +30,7 @@ export function POST($) {
       contentType: 'application/json',
     };
   }
-  // Empty body (content-length 0) like live, not Counterfact's default "Accepted" reason.
-  return { status: 202, body: '' };
+  // Empty body (content-length 0) like live. The explicit content type (via emptyAck) bypasses
+  // Counterfact's response negotiation, which would otherwise 406 under `Accept: application/json`.
+  return context.emptyAck(202);
 }

packages/spacecat-shared-project-engine-client/mock/counterfact/routes/v2/workspaces/{id}/projects/{project_id}/aio/benchmarks/{benchmark_id}/brand_urls.js

@@ -53,6 +53,7 @@ export function DELETE($) {
     { workspaceId: path.id, projectId: path.project_id, benchmarkId: path.benchmark_id },
     body?.ids ?? [],
   );
-  // Empty body (content-length 0) like live, not Counterfact's default "Accepted" reason.
-  return { status: 202, body: '' };
+  // Empty body (content-length 0) like live. The explicit content type (via emptyAck) bypasses
+  // Counterfact's response negotiation, which would otherwise 406 under `Accept: application/json`.
+  return context.emptyAck(202);
 }

packages/spacecat-shared-project-engine-client/mock/responses.js

@@ -0,0 +1,61 @@
+/*
+ * Copyright 2025 Adobe. All rights reserved.
+ * This file is licensed to you under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License. You may obtain a copy
+ * of the License at http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software distributed under
+ * the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR REPRESENTATIONS
+ * OF ANY KIND, either express or implied. See the License for the specific language
+ * governing permissions and limitations under the License.
+ */
+
+// @ts-check
+
+/**
+ * Shared raw-response shapes for the Project Engine mock.
+ *
+ * Counterfact runs a bare `{ status, body }` return through response content-negotiation: it
+ * picks a representation for the request's `Accept` header. An empty-body 2xx (the live "action
+ * ack" — publish / batch-delete / update-benchmark, all `content-length: 0` live) declares no
+ * `application/json` representation, so when the caller sends `Accept: application/json` — which
+ * the real serenity transport in `spacecat-api-service` does on EVERY call — Counterfact has
+ * nothing to serve and answers **406 Not Acceptable** instead of the intended 2xx. The real
+ * Semrush gateway never does this: a no-content response is returned regardless of `Accept`.
+ *
+ * The fix is the same raw-return bypass `auth.js` (the 401) and `quota.js` (the disguised 405)
+ * already rely on: a `{ status, body, contentType }` literal carries an explicit content type, so
+ * Counterfact skips negotiation/validation and serves the response verbatim. {@link emptyAck}
+ * centralizes that shape so empty-body ack handlers can't reintroduce the negotiation 406.
+ *
+ * KNOWN, UNAVOIDABLE DIVERGENCE — the mock's empty 202 carries a `Content-Type: application/json`
+ * header; live prod sends the empty ack with `content-length: 0` and NO `Content-Type` at all
+ * (captured 2026-06-29 against the real gateway: `202`, `content-length: 0`, no content type). This
+ * is a Counterfact framework limit, not a choice: a truthy `contentType` is the ONLY way to get a
+ * 0-byte body past negotiation — without it the body either 406s or falls back to Counterfact's
+ * default `text/plain` "Accepted" 8-byte body (both verified across the raw-return and typed
+ * `$.response[202]` paths). The divergence is invisible to the only consumer — the serenity
+ * transport keys off `response.ok`/status, and the body is empty either way — so status + body
+ * match prod exactly; only this one extra response header differs. `application/json` is chosen
+ * because it matches the caller's `Accept`.
+ *
+ * NOTE — this is for `2xx` acks that carry an EMPTY body but are NOT `204`. A bare
+ * `{ status: 204 }` is already safe: Counterfact special-cases `204 No Content` and serves it
+ * without negotiating (verified 2026-06-29 — `204` returns `204` under `Accept: application/json`,
+ * while `202, body: ''` 406s), so the `204` delete handlers deliberately do NOT go through here —
+ * a 204 carries no body and (per HTTP) no `Content-Type`, which the bare return preserves.
+ */
+
+/**
+ * An empty-body 2xx ack shaped to bypass Counterfact's response content-negotiation, mirroring
+ * the live gateway's `content-length: 0` action acks. The `contentType` is what makes Counterfact
+ * skip negotiation; the body stays empty and the consumer keys off `response.ok`/status only, so
+ * `application/json` (matching the caller's `Accept`) is the natural, negotiation-satisfying
+ * choice. Use for empty-body `202`-class acks; `204 No Content` needs no helper (see module note).
+ * @param {200 | 201 | 202} [status] the empty-body 2xx ack status (defaults to the publish/
+ *   action-ack `202`); the literal union keeps the "empty-body 2xx ack only" contract tsc-enforced
+ * @returns {{ status: 200 | 201 | 202, body: string, contentType: string }}
+ */
+export function emptyAck(status = 202) {
+  return { status, body: '', contentType: 'application/json' };
+}

packages/spacecat-shared-project-engine-client/test/e2e/project-engine-mock.e2e.js

@@ -585,11 +585,17 @@ async function waitForReady(baseUrl, deadline, getStderr) {
 
     // Live ack: 202 with an EMPTY body (verified 2026-06-25), not a BasicResponse. Raw fetch so the
     // empty body is asserted — the typed client swallows it, so a regression to a JSON body passes.
+    // `Accept: application/json` (what the real serenity transport sends) pins the negotiation-
+    // bypass fix: without the handler's content type this empty 202 would 406 (issue 1742).
     const benchUrl = `${baseUrl}/v1/workspaces/${SEED_WORKSPACE}`
       + `/projects/${SEED_PROJECT}/ai_models/benchmarks`;
     const rawBenchDel = await fetch(benchUrl, {
       method: 'DELETE',
-      headers: { Authorization: 'Bearer e2e-token', 'content-type': 'application/json' },
+      headers: {
+        Authorization: 'Bearer e2e-token',
+        'content-type': 'application/json',
+        Accept: 'application/json',
+      },
       body: JSON.stringify({ ids: created.ids }),
     });
     expect(rawBenchDel.status).to.equal(202);
@@ -604,11 +610,16 @@ async function waitForReady(baseUrl, deadline, getStderr) {
   // Mirrors the consumer's updateBenchmark: PUT a brand_aliases re-sync, list reflects it.
   it('updates a benchmark in place (PUT v1) and the list reflects the change', async () => {
     // Live ack: 202 with an EMPTY body (verified 2026-06-25) — raw fetch asserts the empty body.
+    // `Accept: application/json` pins the negotiation-bypass fix (406 otherwise — issue 1742).
     const benchPutUrl = `${baseUrl}/v1/workspaces/${SEED_WORKSPACE}`
       + `/projects/${SEED_PROJECT}/ai_models/benchmarks/${SEED_IDS.benchmarkId}`;
     const rawBenchPut = await fetch(benchPutUrl, {
       method: 'PUT',
-      headers: { Authorization: 'Bearer e2e-token', 'content-type': 'application/json' },
+      headers: {
+        Authorization: 'Bearer e2e-token',
+        'content-type': 'application/json',
+        Accept: 'application/json',
+      },
       body: JSON.stringify({ brand_aliases: ['Adobe Inc', 'Adobe Systems'] }),
     });
     expect(rawBenchPut.status).to.equal(202);
@@ -656,11 +667,16 @@ async function waitForReady(baseUrl, deadline, getStderr) {
     expect(listed.brand_urls.map((u) => u.id)).to.include(created.ids[0]);
 
     // Live ack: 202 with an EMPTY body (verified 2026-06-25) — raw fetch asserts the empty body.
+    // `Accept: application/json` pins the negotiation-bypass fix (406 otherwise — issue 1742).
     const buUrl = `${baseUrl}/v2/workspaces/${SEED_WORKSPACE}`
       + `/projects/${SEED_PROJECT}/aio/benchmarks/${benchmarkId}/brand_urls`;
     const rawBuDel = await fetch(buUrl, {
       method: 'DELETE',
-      headers: { Authorization: 'Bearer e2e-token', 'content-type': 'application/json' },
+      headers: {
+        Authorization: 'Bearer e2e-token',
+        'content-type': 'application/json',
+        Accept: 'application/json',
+      },
       body: JSON.stringify({ ids: created.ids }),
     });
     expect(rawBuDel.status).to.equal(202);
@@ -677,9 +693,12 @@ async function waitForReady(baseUrl, deadline, getStderr) {
     expect(pubRes.status).to.equal(202);
     // Live action acks (publish, delete/update-benchmark, delete-brand-urls) return a 202 with an
     // EMPTY body (verified 2026-06-25), not a BasicResponse — a raw fetch confirms no body.
+    // `Accept: application/json` (the serenity transport's header) pins the negotiation-bypass fix:
+    // this is the exact request shape that 406'd and blocked the create/activate IT (issue 1742).
     const pubUrl = `${baseUrl}/v1/workspaces/${SEED_WORKSPACE}/projects/${SEED_PROJECT}/publish`;
     const rawPub = await fetch(pubUrl, {
-      method: 'POST', headers: { Authorization: 'Bearer e2e-token' },
+      method: 'POST',
+      headers: { Authorization: 'Bearer e2e-token', Accept: 'application/json' },
     });
     expect(rawPub.status).to.equal(202);
     expect(await rawPub.text()).to.equal('');

packages/spacecat-shared-project-engine-client/test/mock/responses.test.js

@@ -0,0 +1,32 @@
+/*
+ * Copyright 2025 Adobe. All rights reserved.
+ * This file is licensed to you under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License. You may obtain a copy
+ * of the License at http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software distributed under
+ * the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR REPRESENTATIONS
+ * OF ANY KIND, either express or implied. See the License for the specific language
+ * governing permissions and limitations under the License.
+ */
+
+import { expect } from 'chai';
+import { emptyAck } from '../../mock/responses.js';
+
+describe('mock responses', () => {
+  describe('emptyAck', () => {
+    it('returns the raw empty-202 ack with an explicit content type (the negotiation bypass)', () => {
+      expect(emptyAck()).to.deep.equal({
+        status: 202,
+        body: '',
+        contentType: 'application/json',
+      });
+    });
+
+    it('acks with the given 2xx status (and still bypasses negotiation)', () => {
+      // The non-default arg exercises the other branch of the `status = 202` default, so the
+      // no-arg test above plus this one cover both; an empty body + contentType stay invariant.
+      expect(emptyAck(200)).to.deep.equal({ status: 200, body: '', contentType: 'application/json' });
+    });
+  });
+});