Skip to content

fix(llmo-customer-analysis): route v2 brandalf dispatches by changeKind (LLMO-4744)#2480

Open
rainer-friederich wants to merge 3 commits into
mainfrom
fix/LLMO-4744
Open

fix(llmo-customer-analysis): route v2 brandalf dispatches by changeKind (LLMO-4744)#2480
rainer-friederich wants to merge 3 commits into
mainfrom
fix/LLMO-4744

Conversation

@rainer-friederich

@rainer-friederich rainer-friederich commented May 6, 2026

Copy link
Copy Markdown
Contributor

Pre-cutover blocker for the Adobe brandalf_migration. Sibling of LLMO-4743 (Reader 4).

Problem

Under brandalf, the v1 LLMO config S3 mirror is frozen. The existing snapshot diff in runLlmoCustomerAnalysis (handler.js:439, 448) reads two versioned S3 snapshots and compares them — both return identical data, so the diff is always empty and the three downstream triggers silently never fire on real user edits made via brandalf-UI / elmo-UI:

  • triggerCdnLogsReport
  • drsClient.triggerBrandDetection
  • triggerGeoBrandPresenceRefresh

Audit reports show status: completed, configChangesDetected: false regardless of what the user actually changed.

There is also a second, deeper root cause confirmed during investigation: the audit is not even being dispatched on subsequent v2 edits. BrandsController v2 endpoints (the ones elmo-UI / brandalf-UI hits) write to normalized Postgres tables but never enqueue an llmo-customer-analysis audit. The legacy v1 PUT path was the only dispatcher and is bypassed under brandalf. The companion api-service PR adobe/spacecat-api-service #2337 fixes the dispatch side.

Investigation confirmed:

  • triggerBrandDetection and triggerGeoBrandPresenceRefresh have NO scheduled equivalent — llmo-customer-analysis change-detection is the sole firing path.
  • triggerCdnLogsReport has scheduled equivalents but only this code path sets categoriesUpdated: true (forces URL→category pattern rebuild).

Architecture choice

Three options were considered (full notes in chat thread):

  • Option A — diff the v2 normalized Postgres tables, store a previous-snapshot somewhere. Rejected: the llmo_customer_config JSONB is seeded once at onboarding and never refreshed by BrandsController, so it has no live data to diff against. A snapshot mechanism (new column, new table, S3, DDB) is its own design problem and races on concurrent edits.
  • Option B — keep the v2 JSONB synced on every BrandsController write, version it via S3, diff two S3 versions through a v2→v1 shape adapter. Rejected: three repos including a spacecat-shared-utils bug fix; full JSONB rebuild on every write; fragile schema-shape adapter.
  • Option C (this PR) — the writer already knows what changed. Pass that signal as changeKind from BrandsController and route triggers from it. No diff, no snapshot, no migration. Smallest blast radius.

Fix

Add a v2 dispatch path that short-circuits the snapshot diff when an audit message carries auditContext.changeKind. The companion spacecat-api-service PR (adobe/spacecat-api-service #2337) dispatches one such message per site after every customer-config mutation.

Routing:

  • brands, competitorstriggerGeoBrandPresenceRefresh
  • categories, topics, entitiesdrsClient.triggerBrandDetection
  • prompts → both triggers (prompts feed BP queries directly and affect topic/category attribution)
  • cdn-logs-report is intentionally not fired for v2 (out of scope under brandalf — owned by a separate path)

Why triggerGeoBrandPresenceRefresh is correct under brandalf

triggerGeoBrandPresenceRefresh ultimately calls drsClient.publishBrandPresenceAnalyze for each existing BP collection sheet. That method publishes an SNS JOB_COMPLETED event with reanalysis: true and result_location pointing at the uploaded sheet — same shape DRS sees when an external provider job (BrightData / Google AI / OpenAI) completes. DRS picks up the event, runs BP analysis against the current Postgres v2 customer config, and the brand_presence_executions DB sync runs as the natural tail of that pipeline. The DB ends up eventually-consistent with the user's edit. We don't run the analysis ourselves — we just signal DRS to re-run it with the new config.

The case this does not cover is adding a brand entity whose name was never collected from providers; for that you'd need a new DRS schedule run, which is an additive follow-up, not a regression fix.

Defensive checks (each warns + returns no-op completed result):

  • Unknown changeKind
  • Missing organizationId in both site and auditContext
  • Brandalf flag disabled for the org (audit-worker safety net — api-service can dispatch freely)
  • DRS not configured (skips brand-detection but still fires BP refresh for prompts)

The branch sits before the first-onboarding logic so subsequent v2 edits do not re-create BP schedules (createAndTriggerBrandPresenceSchedule is not idempotent).

The v1 path is untouched: messages without changeKind fall through to the existing snapshot-diff flow unchanged.

Files

  • src/llmo-customer-analysis/handler.js — new handleV2ChangeKindDispatch function + branch in runLlmoCustomerAnalysis
  • test/audits/llmo-customer-analysis.test.js — 14 new test cases under describe('v2 changeKind dispatch')

Test plan

  • All 13 changeKind values dispatch the correct triggers
  • CDN logs trigger never fires for v2 dispatches
  • No v1 S3 reads, no first-onboarding BP schedule creation for v2 dispatches
  • Defensive paths: unknown changeKind, missing orgId, brandalf disabled, DRS unconfigured
  • v1 path regression: messages without changeKind still diff S3 snapshots and trigger as before
  • npm test (affected file) — 65 tests pass (51 existing + 14 new)
  • 100% branch coverage on handler.js
  • Verified on a brandalf_migration test org: real user edit via brandalf-UI → appropriate downstream trigger fires (post-deploy validation)

Deferred follow-ups (not in this PR)

  • Bump dispatch-failure logs from warnerror so silent-no-op cases page on alerts. The api-service dispatcher and the audit-worker brandalf-disabled path both currently log at warn.
  • Tri-state isBrandalfEnabled (true / false / unknown) so transient feature-flag-fetch failures don't masquerade as "brandalf off" and silently drop triggers. Bigger refactor — own ticket.
  • Trigger a fresh DRS schedule run on brand/competitor additions (not just alias edits), so newly-added brand entities get queried against providers without waiting for the next scheduled cadence.

Related

  • Jira: https://jira.corp.adobe.com/browse/LLMO-4744
  • Parent: LLMO-4715 (full audit-worker JS reader/writer scope)
  • Sibling: LLMO-4743 (Reader 4 — v1-write ban enforcement)
  • Epic: LLMO-4587 (Brand Presence brandalf migration)

Deployment ordering

This PR is safe to deploy first. The new branch is dead until a v2 message arrives (i.e., until the api-service PR ships). No existing behavior changes.

Related Issues

  • LLMO-4744

…nd (LLMO-4744)

Under brandalf, the v1 LLMO config S3 mirror is frozen, so the existing
snapshot diff in `runLlmoCustomerAnalysis` always returns no-change and the
three downstream triggers (cdn-logs-report, drs-brand-detection,
geo-brand-presence-trigger-refresh) silently never fire on real user edits
made via brandalf-UI / elmo-UI.

This change adds a v2 dispatch path that short-circuits the snapshot diff
when an audit message carries `auditContext.changeKind`. spacecat-api-service
BrandsController will dispatch one such message per site after every
customer-config mutation, with `changeKind` identifying what changed
(brands, competitors, categories, topics, entities, prompts).

Routing:
- brands, competitors -> triggerGeoBrandPresenceRefresh
- categories, topics, entities -> drsClient.triggerBrandDetection
- prompts -> both triggers
- cdn-logs-report is intentionally not fired for v2 (out of scope under
  brandalf — owned by a separate path)

Defensive checks: unknown changeKind, missing organizationId, brandalf flag
disabled, and DRS not configured all return a no-op completed result with
the appropriate warn log. The branch sits before the first-onboarding logic
so subsequent v2 edits do not re-create BP schedules
(createAndTriggerBrandPresenceSchedule is not idempotent).

The v1 path is untouched: messages without changeKind fall through to the
existing snapshot-diff flow unchanged.

13 new test cases cover all changeKind values, defensive paths, and v1
backward compatibility.
@rainer-friederich

Copy link
Copy Markdown
Contributor Author

Self-review pass via local code-review agent. No actionable findings — reviewer reported two lower-confidence (~50%) design notes:

  • isBrandalfEnabled latency / rate-limit risk — every v2 dispatch makes a feature-flags round-trip before any trigger fires. Inherited from the existing brandalf-aware first-onboarding code path; not a regression.
  • Trigger retry idempotency under partial failure — for changeKind: 'prompts', if triggerBrandDetection succeeds and triggerGeoBrandPresenceRefresh then rejects, the message is NACK'd and DRS brand detection re-fires on retry. DRS brand detection is idempotent (same as v1 path semantics). Not a regression.

Reviewer summary: "The PR is in good shape. The fix is correct, well-tested, and does not regress the v1 path."

…ts kind)

Branch coverage on handler.js fell from 100% to 99.18% on the
'triggeredSteps.join(', ') || 'none'' fallback at line 392 — the '\u2018none\u2019'
side never fired in tests because every existing changeKind that
reaches the trigger fan-out either fires brand-detection (when DRS is
configured) or BP refresh (for brands/competitors/prompts).

Add one case where DRS is unconfigured AND changeKind is 'categories'
(BP refresh not in scope), so triggeredSteps stays empty and the
'triggered: none' log path executes. Restores 100% branch coverage.
@github-actions

github-actions Bot commented May 6, 2026

Copy link
Copy Markdown
Contributor

This PR will trigger a patch release when merged.

@codecov

codecov Bot commented May 6, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.

📢 Thoughts on this report? Let us know!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant