- 0285_vulnerability_id_entity_tables: CreateModel x2 (empty tables => index DDL
  instant; >50M-row GIN-strip fallback documented in the header).
- 0286_backfill_vulnerability_id_entities: atomic=False, Postgres-guarded,
  RunPython forward -> dojo.vulnerability_id.backfill.run_backfill (noop reverse).
- backfill.run_backfill: idempotent, insert-only, set-based SQL shared with the
  WP5 migrate_vulnerability_ids command — (1) entities from legacy table
  (MAX type carry), (2) cve-only entities + Python type-stamp, (3) references
  windowed by finding_id (ROW_NUMBER cve-first then legacy-PK; order 0-based),
  (4) order-0 refs for cve-only findings. ON CONFLICT DO NOTHING everywhere.
- test_migrations.TestVulnerabilityIdBackfill: seeds ordered / cve-drift /
  case-variant / cve-only / null-cve findings, asserts entity extraction, type,
  per-finding order (cve at 0), pair parity, and idempotent re-run.
  Uses a plain TestCase against run_backfill rather than MigratorTestCase:
  django_test_migrations replays the full squashed chain from a wiped schema in
  setUp and collides on legacy tables (dojo_cred_user) — the same reason the only
  other MigratorTestCase in this file is skipped. Migration application itself is
  covered by the create-db and dev-DB migrate runs.

Verified: 0285/0286 apply cleanly on a fresh create-db build and on the dev DB;
backfill test green; re-running run_backfill is a no-op.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

Every vulnerability-id writer now persists the legacy Vulnerability_Id rows
byte-for-byte AND the new VulnerabilityId entity + FindingVulnerabilityReference
rows, through a single seam. The flag gates reads only (WP4), so the two stores
never drift. Legacy-write blocks are marked "TODO: Delete after Vulnerability_Id
retirement".

- dojo/vulnerability_id/manager.py: bulk_get_or_create_entities (race-safe
  bulk_create(ignore_conflicts)+refetch), persist_for_finding (single-finding
  core; does NOT touch finding.cve — caller keeps the cve sync), and
  VulnerabilityIdManager (importer accumulate/flush; record / record_reconcile /
  flush in one transaction). order == enumerate position => order 0 is the cve.
- base_importer: store_vulnerability_ids -> manager.record; flush_vulnerability_ids
  -> manager.flush() then the unchanged Finding_CWE flush (CWE buffers are NOT
  owned by the manager and keep riding this flush boundary).
- default_reimporter.reconcile_vulnerability_ids: keep the prefetched set-equality
  early-exit (legacy store authoritative) + cve sync; on change -> record_reconcile.
- finding/helper.save_vulnerability_ids: dedupe/sanitize + cve sync unchanged;
  storage delegates to persist_for_finding. (migrate_cve stays legacy-only.)

Tests: test_importers_importer updated to assert the manager's buffers (46 green);
new fixture-free test_vulnerability_id asserts the entity-side dual-write, order,
reconcile, orphan retention, and empty-clears for persist_for_finding, the manager,
and save_vulnerability_ids; test_finding_helper.test_save_vulnerability_ids now
asserts delegation to persist_for_finding + cve sync.

Verified locally: test_importers_importer + test_vulnerability_id +
TestSaveVulnerabilityIds green; manual import/reimport smoke on the dev DB shows
both stores mirrored with correct order and orphan entities retained. The
fixture-based suites (import_reimport / rest_framework / deduplication_logic /
finding_helper API) run in OSS CI (dojo_testdata.json can't load in the Pro
container: watson.searchentry).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

Reads switch to the VulnerabilityId entity/reference store when
V3_FEATURE_VULNERABILITY_IDS is on (default True); off keeps today's legacy-table
reads. The flag is consulted ONLY in dojo/vulnerability_id/queries.py and the two
Finding methods. Writes stay unconditionally dual (WP3), so both stores hold
identical data and the flip is reversible with no drift.

queries.py seam: use_entity_reads, finding_ids_with_vulnerability_ids (in/exact),
vulnerability_id_prefetch (with nested prefix), first_vulnerability_id_subquery
(order==0), finding_vulnerability_id_strings.

Re-pointed (identical output flag-off; entity read flag-on):
- Finding.vulnerability_ids property + get_vulnerability_ids() (cve prepend /
  sorted-join kept byte-for-byte).
- dojo/filters.py both filter fns; risk_acceptance exact-match mixin.
- The 7 prefetch sites (deduplication, commands/dedupe, finding/ui + test/ui
  views, finding/queries x2) -> vulnerability_id_prefetch().
- v2 serializer: VulnerabilityIdsField reads [{"vulnerability_id": s}] from the
  flag-appropriate store (schema pinned via extend_schema_field); create/update
  pop `parsed_vulnerability_ids` and still funnel writes to save_vulnerability_ids.
- Finding.copy() now dual-writes copied ids through persist_for_finding (was a
  legacy-only .objects.create — would have been invisible flag-on).

Deliberately NOT re-pointed this PR (kept on the dual-written legacy table, which
stays correct in both flag states; entity migration deferred to the legacy-drop
phase): watson registration + the classic OSS search view. Rationale: watson is
disabled in the Pro runtime (untestable locally), the classic search view/template
are coupled to watson indexing Vulnerability_Id, and the classic UI is being
retired. This also removes the buildwatson-on-flip requirement. The Pro Vue global
search IS re-pointed to the entity in the Pro PR.

Tests (fixture-free, run locally + CI): test_vulnerability_id.TestReadSeamFlagParity
asserts the v2 wire shape, hash input (get_vulnerability_ids), the property, and the
filter seam are byte-identical flag-off vs flag-on; test_importers_importer reconcile
seeds via the dual-write seam so entity reads resolve. Verified green in BOTH flag
states locally; the fixture-based suites (rest_framework schema, import_reimport,
deduplication_logic, finding_helper API, watson trio) run in OSS CI.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

- migrate_vulnerability_ids: wraps dojo.vulnerability_id.backfill.run_backfill
  (the same routine 0286 runs); --window-size; PostgreSQL-guarded; prints per-step
  row counts; idempotent/resumable so big tenants pre-run it before the deploy.
- verify_vulnerability_ids: read-only reconciliation, nonzero exit on any drift so
  it can gate a ring flip. Per sampled finding (--sample, default 1000): every
  legacy (finding, id) pair has a reference; extra references allowed only when
  cve-only-derived (string == finding.cve); the order-0 reference == cve. Plus
  legacy-row / entity / reference / orphan-entity counts, both table names printed
  with roles. --json for machine output.

Tests (fixture-free): migrate backfills legacy-only findings to entities/refs with
order 0 == cve and is a no-op on re-run; verify passes on consistent data and
raises SystemExit on a legacy row with no matching reference.

Verified on the dev DB: migrate_vulnerability_ids runs clean and idempotent;
verify_vulnerability_ids exits 0.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

… backfill -> flag parity)

TestReadSeamFlagParity seeds via the dual-write seam, so both stores are trivially in
sync. These new tests cover the actual UPGRADE scenario a deployment hits: legacy
Vulnerability_Id rows written the old way (no entity refs), then backfilled by
migrate_vulnerability_ids, then read flag-off (legacy) vs flag-on (entity).

- test_backfilled_normal_data_reads_identically: for seam-shaped data (cve == first/
  lowest-PK row) get_vulnerability_ids, the vulnerability_ids property, and the raw v2
  wire shape are byte-identical across the flag.
- test_backfilled_cve_not_first_row_preserves_hash_and_property: anomalous data where cve
  is not the first-created legacy row keeps hash + property byte-identical (both are
  order-insensitive by construction); only the RAW v2 wire ORDER differs (legacy PK-asc
  vs entity cve-first, same set) — asserted so the bounded divergence is deliberate.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

The unconditional dual-write of VulnerabilityId entities + FindingVulnerabilityReference
rows adds a small constant per-import query delta (batched, not per-finding). Counts
updated to match a pure-OSS CI run: dedup first_import +6 / second_import +4; reimport
import1 +6 / reimport1 +12 / reimport2 +5, across TestDojoImporterPerformanceSmall and
...SmallLocations. Async-task counts unchanged (dual-write is synchronous). reimport3
(queries4) unaffected.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

Three test-only fixes surfaced by V3_FEATURE_VULNERABILITY_IDS defaulting True in OSS
(reads hit the entity store):

- test_bulk_risk_acceptance_api + test_pdf_report_rendering seeded vuln ids via the
  legacy Vulnerability_Id table directly, so entity reads returned empty flag-on. Seed
  via save_vulnerability_ids (unconditional dual-write) instead — production code is
  unchanged and correct. bulk_risk verified locally (3 tests green flag-on); the pdf
  report test loads dojo_testdata (can't run in the Pro container) and relies on OSS CI.
- test_tag_inheritance_perf: re-baseline the 6 EXPECTED_ZAP_* assertNumQueries constants
  for the dual-write delta (+2 import / +12 reimport-no-change / +6 reimport-with-new),
  from a pure-OSS CI run.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

The rest-framework CI matrix runs a V3_FEATURE_LOCATIONS=True leg where base save()
calls full_clean(). The new vuln-id test classes created Product/Test without the
required description / scan_type / environment, so all 7 setUpClass hooks errored with
ValidationError under that leg (they passed the False leg). Provide the required fields
(Finding creation already skips validation via skip_validation=True, so it's unaffected).
Reproduced locally with V3_FEATURE_LOCATIONS=True: 14 tests green.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

…s, add cutover-safety tests

Review response (#15331) plus prep for the one-way legacy retirement.

Rename (per review): VulnerabilityId -> Vulnerability; module dojo/vulnerability_id/ -> dojo/vulnerability/;
table dojo_vulnerabilityid -> dojo_vulnerability (migration 0285 amended in place). String column stays
vulnerability_id; reference FK stays `vulnerability`. Added vocabulary + porting-trap docstrings.

Must-fixes:
- MF4: UniqueConstraint(finding, order) on FindingVulnerabilityReference (model + migration 0285).
- MF2: verify_vulnerability_ids downgrades order-0 != cve to a non-fatal warning when cve is not among
  the finding's ids (unreachable without breaking hash parity); stays a hard gate for genuine ordering drift.

Bug fixes surfaced by the new tests:
- cve-only hash-parity break: backfill step 4 fabricated a cve-only reference the live dual-write never
  creates, so entity reads diverged from legacy (dedup churn on deploy). Removed; the reference set now
  mirrors the legacy row set exactly. cve stays an orphan registry entity.
- reimporter N+1: reconcile read the non-prefetched legacy relation under flag-on. Now reads through the
  seam (matches the prefetch). Perf re-baselined: -5 queries per reimport across 24 assertions.

Reader ports toward the cutover (route through the flag seam, reversible): Finding.copy, reimporter reconcile.

Tests (fixture-free, both flag states): legacy-independence (poison), dedup differential, real-scanner
corpus (8 parsers), enumerable shapes (cve-only/null-cve/unicode/quotes/copy), MF2 warning, MF4 constraint.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

Read vulnerability ids exclusively from the Vulnerability entity +
FindingVulnerabilityReference through-table. Remove the read flag and all
flag-off branches, delete the legacy Vulnerability_Id model, and add migration
0287 dropping dojo_vulnerability_id (0286 backfills the entities first).

- MF2: backfill no longer inserts cve-only references (preserves hash parity)
- MF4: UniqueConstraint(finding, order) on the reference table
- Reimporter reads finding_vulnerability_id_strings (fixes a latent N+1)
- Repoint tests to the entity store; remove legacy-model coverage; delete the
  migrate_cve / verify_vulnerability_ids commands

NOTE: test_tag_inheritance_perf EXPECTED_* query counts need recalibration
under watson-enabled CI after the dual-write removal.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

dev added 0285_cicd_infrastructure, which collided with the vuln-id 0285.
Renumber the three vuln-id migrations to chain linearly after it:
  0285_vulnerability_id_entity_tables     -> 0286 (dep: 0285_cicd_infrastructure)
  0286_backfill_vulnerability_id_entities -> 0287
  0287_delete_vulnerability_id            -> 0288
makemigrations --check clean; single leaf (0288).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

…n-id lane

Second CI pass after the rebase onto newer dev:

- test_importers_performance: rebased dev shifted the baseline again (-1 on
  import/dedup steps, -2 more on reimport1). Reset all 22 affected counts to the
  current CI actuals; the warm-up primes FindingVulnerabilityReference.
- dojo/search/views.py: the classic /simple_search vuln-id lane queried the
  deleted Vulnerability_Id via watson. An earlier stub (authorized_vulnerability_ids
  = []) 500'd on watson.filter/.prefetch_related ('list' has no attribute ...),
  breaking search_test and any UI test that visits /simple_search. Remove the lane
  entirely (model is gone and unregistered from watson); Vue global search covers
  vuln ids. The dev DD_WATSON_SEARCH_ENABLED toggle still 410s when watson is off.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

The committed scanner corpus yields 49 distinct real ids; the >=50 bound was an
arbitrary over-tight threshold. findings_checked>=20 already guards against a
vacuous run, and 40 keeps a strong diversity guard while being robust to the
corpus size.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

Addresses review of the entity-only cutover (rollout machinery):

- backfill.py: the reference build is now convergent — per finding_id window it
  DELETEs that window's references and re-inserts from the current legacy rows,
  inside one transaction per window. A pre-run + deploy-time catch-up where a
  finding's ids changed no longer (a) leaves stale references behind, nor (b)
  raises on the unique_finding_order constraint by re-inserting at occupied
  order slots. Still idempotent and resumable. Window bounds span both the legacy
  and reference tables so stale refs past the legacy max are still reconciled.
- Remove the silent non-PostgreSQL vendor guard from the backfill migration (0287)
  and the migrate_vulnerability_ids command. DefectDojo is PostgreSQL-only, so the
  backfill runs unconditionally and fails loudly rather than skipping and letting
  the unconditional 0288 drop destroy un-migrated data.
- 0288: add a one-query safety invariant before DeleteModel — abort the drop if
  dojo_vulnerability_id still has rows but the reference table is empty (the useful
  residue of the removed verify command).
- Fix stale "frozen until dropped in a follow-up migration" docs in manager.py /
  queries.py — 0288 in this same change drops the legacy table.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>