perf(tag inheritance): batch_mode + per-batch bulk during import + reorganize

[valentijnscholten]

Summary

Wraps the import / reimport hot loop in tag_inheritance.suppress_tag_inheritance() and bulk-applies inherited Product tags per batch, before post_process_findings_batch dispatches, so rules / deduplication see inherited tags on finding.tags. Pairs with the per-batch import-tag work landed in #14839.

This is the Phase B Stage 2 follow-up to #14811 / #14812 / #14827.

Changes

• process_findings in DefaultImporter and DefaultReImporter now runs its finding-creation loop inside tag_inheritance.suppress_tag_inheritance(). Per batch — after apply_import_tags_for_batch and before dojo_dispatch_task(post_process_findings_batch, ...) — calls a new helper apply_inherited_tags_for_findings(batch_findings) that bulk-syncs inherited tags on the batch's Findings + the Endpoints (V2) / Locations (V3) reachable from them via FK chain.
• DefaultReImporter now tracks newly-created findings separately in new_findings_in_batch and passes only those to apply_inherited_tags_for_findings. Matched/existing findings already have inheritance from their original creation, so re-running through-table reads on them was pure overhead on no-change reimports.
• LocationManager._persist_locations skips apply_inherited_tags_for_locations when no new LocationProductReference rows were created. New finding refs only link findings already in self._product, so they can't change a Location's Product membership; only a new product ref can. When the product-ref list is empty the Location's inherited target set is unchanged.
• The signal-driven auto-inheritance entrypoint is renamed inherit_instance_tags → auto_inherit_product_tags and moved from dojo.tags.inheritance to dojo.tags.signals, colocated with the receivers that call it. It is the only inheritance helper that uses is_suppressed() as a gate (others use suppress_tag_inheritance() to wrap their writes), so the gate semantic is exclusive to the signal path.
• The post_save handlers for child models (Endpoint/Engagement/Test/Finding/Location) and ref models (LocationFindingReference/LocationProductReference) are combined into a single inherit_tags_on_instance handler with sender-based target dispatch + created=True gate. Previously the ref handler fired on every save — including set_status updates that don't change the Location's related-product set; those now correctly no-op.
• EndpointManager.get_or_create_endpoints replaces its per-row inherit_instance_tags(ep) cleanup loop with a single apply_inherited_tags_for_endpoints(created) bulk call. Inside the importer the per-batch helper already covers these endpoints via Endpoint.status_endpoint.finding; the bulk call is retained as a defensive hook for non-importer callers.
• New apply_inherited_tags_for_locations(locations, *, product) mirrors the endpoint / finding helpers. LocationManager._bulk_inherit_tags is removed; the persist path delegates to the new helper. Drops a redundant outer suppress_tag_inheritance() (writes inside _sync_inherited_tags are already wrapped) and a redundant bulk pre-fetch of existing inherited tags (the through-table primitive does that read once already).
• _inherited_tag_names_for_location (the per-Location callback for the bulk sync primitive) now filters contributing products by their own enable_product_tag_inheritance flag. Previously, tags from flag-off products linked to the same Location could leak in via propagate_tags_on_product_sync and apply_inherited_tags_for_findings.
• The bulk sync primitive is restructured: _sync_inheritance_for_qs(queryset, target_names_per_child=callable) → _sync_inheritance_for_ids(model_class, child_ids, target_tag_names: set[str] | Callable[[int], set[str]]). Callers pass values_list("pk", flat=True) directly so the primitive never materializes children as full model instances; the bulk helpers only need instance.pk and instance.__class__, so model_class(pk=pid) stubs (created lazily for rows whose tag set actually changes) are enough. Constant-target callers pass a plain set; the per-pk callable form is reserved for Locations (multi-product target). Eliminates the Finding-row hydration that dominated wall-clock time on large products (~22s for ~14000 findings → expected to drop substantially; the query-count baseline can't see this since it measures SQL, not Python time).
• propagate_tags_on_product_sync (used by the product-tag-toggle Celery task) gains an early-exit when neither system-wide nor per-product inheritance is enabled, eliminating ~9 wasted reads per call on inheritance-off products. State transitions (toggling either flag) still trigger a full sweep via their existing signal handlers.
• Location gains iter_related_products() — a related-manager (self.products + self.findings) implementation returning list[Product]. Callers that pre-issue prefetch_related("products__product__tags", "findings__finding__test__engagement__product__tags") get zero extra queries per Location. The existing JOIN'd all_related_products() is kept for the per-instance signal path where prefetching is not possible.
• Change the "managing" of inherited tags from a "rebuild every time" to a "diff" based approach where added inherited tags on products are cascaded into the object's tags field, and removed inherited tags on products are also removed from the object's tag. No need anymore to reconstruct user_tags or do complex passing of potential tags. Behavior is still the same, guarded by pre-existing tests.
• Centralize tag (inheritance) logic into dojo.tags.inheritance. Move the propagate_tags_on_product Celery task into that module alongside its _sync counterpart and delete dojo/product/helpers.py. A propagate_tags_on_product_deprecated alias is kept under the old task name so in-flight tasks complete after upgrade.

Query-count impact

Pins in unittests/test_tag_inheritance_perf.py updated to match. "Before" numbers for the per-100-children scenarios are the pre-Phase-A baselines recorded in source comments next to each constant. The two propagate_tags_on_product_sync rows and the reimport-with-new-findings rows are newly added baselines so there is no Before number.

Hot path	Before	After	Δ
ZAP scan import V2 (19 findings)	1378	420	-958
ZAP scan import V3	1256	444	-812
ZAP reimport no-change V2	69	69	0
ZAP reimport no-change V3	87	81	-6
ZAP reimport subset → full V2 (9 new)	—	169	(new)
ZAP reimport subset → full V3 (9 new)	—	198	(new)
Product tag add → 100 findings (V2)	4758	91	-4667
Product tag add → 100 findings (V3)	4759	91	-4668
Product tag remove → 100 findings (V2)	4540	53	-4487
Product tag remove → 100 findings (V3)	4541	53	-4488
Product tag add → 100 endpoints (V2)	3958	91	-3867
Product tag remove → 100 endpoints (V2)	3740	53	-3687
Product tag add → 100 locations (V3)	4532	125	-4407
Product tag remove → 100 locations (V3)	4307	75	-4232
propagate_tags_on_product_sync no-op V2	—	9	(new)
propagate_tags_on_product_sync no-op V3	—	18	(new)

[github-actions]

This pull request has conflicts, please resolve those before we can evaluate the pull request.

[github-actions]

Conflicts have been resolved. A maintainer will review the pull request shortly.