Content pack export produces dangling stream references when a referenced stream is unresolvable

[patrickmann]

Creating a new revision of a content pack (re-exporting) can produce a dangling stream reference: a dashboard widget (or search type / query filter) references a stream by a content-pack id that has no matching stream or stream_title entity in the exported pack. Installing such a pack then fails with a missing stream exception.

This is the export-side root cause behind Graylog2/graylog-plugin-enterprise#12987. The import side is being hardened separately in #26581 (which makes such packs installable by dropping the unresolvable reference); this issue tracks stopping the broken pack from being produced in the first place.

Confirmed root cause

The failure is the product of two decisions that combine badly during export:

1. StreamReferenceFacade.exportEntity loads the stream (streamService.load()) to emit the stream_title (STREAM_REF_V1) entity. If the load fails it returns Optional.empty(), so no entity is emitted.
2. The widget / search type / query filter that references the stream still serializes with the stream's content-pack UUID, because the stream is a node in the export dependency graph and therefore received a UUID in EntityDescriptorIds regardless of whether its entity was emitted.

Result: the exported pack contains a reference to a UUID that no entity in the pack defines → dangling reference → install fails.

catch (Exception) in exportEntity was broadened from catch (NotFoundException) in 7.0.1 (#24124, backport bd5885f), which widened which load failures silently drop the entity. Note, however, that the pre-7.0.1 code already dropped the entity for NotFoundException (a genuinely deleted stream), so the dangling-reference bug predates that change — #24124 only made it easier to hit.

Reproduction (confirmed on Graylog 7.1.4, single node)

1. Create a stream titled Evaluation: Self Monitoring Logs.
2. Install the pack from Graylog2/graylog-plugin-enterprise#12987 (rev 1) — succeeds.
3. Delete (or otherwise make unloadable) a stream that a dashboard widget references.
4. Re-export a new revision via POST /api/system/catalog with the installed native entity descriptors.
5. The exported rev 2 contains a widget/search-type reference to a UUID with no matching stream/stream_title entity.
6. Install rev 2 → HTTP 500, missing stream exception.

Possible fixes

A. Narrow the catch back to NotFoundException

• Pro: Small; restores fail-loud for non-not-found errors.
• Con: Insufficient — the NotFoundException path still produces a dangling reference; also risks re-introducing whatever Update error handling for Stream titles in content packs #24124/Validate ObjectId of StreamEntity before searching Mongo (7.0) #24167 fixed. Not viable alone.

B. Fail export loudly when a referenced stream can't be resolved

Throw a clear ContentPackException naming the missing stream instead of emitting nothing.

• Pro: No silent data loss and no broken pack; the user learns at export time, the right moment to fix the source dashboard.
• Con: One stale reference blocks the entire revision, even for entities the user does not care about.

C. Make the export self-consistent (recommended)

Ensure a stream that cannot be emitted does not receive a content-pack id, so the referencing widget/search-type/query naturally drops it (with a WARN). Cleanest seam: filter unresolvable stream descriptors at resolve time in ContentPackEntityResolver (it already has StreamService injected) before minting EntityDescriptorIds; also make the export-side mappers tolerant (WidgetDTO.toContentPackEntity already drops via filter(Optional::isPresent); SearchType.mappedStreams currently throws and would need to drop instead).

• Pro: Produces an installable, self-consistent pack; revision creation is not blocked; symmetric with the import-side fix.
• Con: Silent semantic loss (widget loses a stream binding); touches the two-phase resolve/collect flow and requires aligning the widget vs. search-type mapper behavior.

D. Import-side tolerance only (already in #26581)

• Pro: Makes any dangling pack installable regardless of source; zero export-side risk.
• Con: Broken packs are still produced; the binding is silently lost on install.

Recommendation

Keep the import-side tolerance from #26581 as a safety net and add C so both export and import degrade gracefully and stay symmetric — this restores the invariant (the export never references an entity it did not emit) whose absence causes the bug. B is a reasonable alternative if the product view is that a content pack must be complete or refuse to export.

References

• Import-side hardening: Tolerate unresolvable stream references during content pack install #26581
• Original report: Graylog2/graylog-plugin-enterprise#12987
• Related history: Update error handling for Stream titles in content packs #24124 (broadened error handling), Validate ObjectId of StreamEntity before searching Mongo (7.0) #24167 (stream ObjectId validation)