Skip to content

Destination topics: duplicates and redundant wildcard states are persisted #990

Description

@alexluong

Affects: Topics validation/storage (internal/models), destination create/update API; portal TopicPicker in wildcard mode. Relevant since #912 (TOPICS_ALLOW_WILDCARDS).

Backend: the topics array is unnormalized data

Topics.Validate checks each entry independently — is it an available topic, or a pattern matching ≥1 available topic. It never looks at the set as a whole, so all of these are accepted and persisted verbatim:

["user.created", "user.created"]          // exact duplicates
["user.*", "user.created"]                // wildcard + entry it covers
["*.created", "user.created", "user.*"]   // overlapping patterns + covered entry

Delivery is unaffected (matching is any-entry-wins), but the stored array is ambiguous as data:

  • Duplicates come back verbatim on every read — API responses, SDK results, portal display, tenant.subscriptions.updated operator event payloads.
  • There's no defined semantics for what a redundant set means for editing: if ["user.*", "user.created"] is stored and the wildcard is later removed, is the destination still subscribed to user.created? Today the answer depends on which client edits it and how.
  • Since feat(api): support wildcard topic subscriptions #912 these states aren't just API-reachable — the portal creates them (see below), so redundant sets will occur organically in production data.

Portal: wildcard editing corrupts subscriptions in these states

Setup: TOPICS=user.created,user.updated,user.deleted, wildcards enabled.

The portal creates the mixed state itself: check user.created, then add user.* via the search field — the now-covered explicit entry is left in place, giving ["user.*", "user.created"].

From there, unchecking a single topic while its group wildcard is selected (which converts the wildcard into explicit topics) goes wrong three ways:

Start state Action Portal submits Expected
1. Writes duplicates ["user.*", "user.created"] uncheck user.updated ["user.created", "user.created", "user.deleted"] ["user.created", "user.deleted"]
2. Can't unsubscribe ["user.*", "user.created"] uncheck user.created ["user.created", "user.updated", "user.deleted"] — checkbox snaps back to checked ["user.updated", "user.deleted"]
3. Filter wipes hidden topics ["user.*"] search created, uncheck user.created [] — one unchecked box empties the subscription ["user.updated", "user.deleted"]

All three live in the same code path: the hasWildcard branch of the child checkbox onChange (TopicPicker.tsx, ~line 329). It removes only the exact wildcardId from the current selection (rows 1–2: covered entries survive the conversion) and rebuilds the explicit list from the search-filtered visibleTopics (row 3).

Suggested direction

  • Normalize on write: dedupe exact duplicates and fold entries covered by a sibling pattern (["user.*", "user.created"]["user.*"]). One consideration for the folding case: it discards the explicit entry, so removing user.* later also removes the user.created subscription — acceptable, but worth being deliberate about.
  • Overlapping patterns (["*.created", "user.*"]) are a state to be aware of: neither covers the other, so normalization leaves both — pattern-vs-pattern redundancy is out of scope here.
  • The portal conversion needs fixing regardless: redundant states are already persistable, so the editor must handle encountering them.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Fields

    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions