Skip to content

feat(finding): pluggable false-positive-history candidate filter#15155

Closed
valentijnscholten wants to merge 8 commits into
DefectDojo:feat/cwe-vuln-id-consolidationfrom
valentijnscholten:feat/false-positive-history-hook
Closed

feat(finding): pluggable false-positive-history candidate filter#15155
valentijnscholten wants to merge 8 commits into
DefectDojo:feat/cwe-vuln-id-consolidationfrom
valentijnscholten:feat/false-positive-history-hook

Conversation

@valentijnscholten

@valentijnscholten valentijnscholten commented Jul 4, 2026

Copy link
Copy Markdown
Member

🔗 PR stack — CWE / vulnerability-ID consolidation (OSS)

Merge bottom-up:

  • #15145 — autodetected vulnerability-ID type + uniqueness constraint
    • #15143 — multiple CWEs per finding
      • #15154 — docs: Pro set-based dedup hash-code fields
      • #15155 — pluggable false-positive-history candidate filter

👉 This PR: #15155


…aint

Adds Vulnerability_Id.vulnerability_id_type, autodetected from the id's
leading prefix (CVE-2024-1234 -> CVE, GHSA-... -> GHSA), stored and indexed
so identifiers can be filtered/grouped by type. Populated on import (bulk)
and on save(); existing rows backfilled by migration.

Also de-duplicates (finding, vulnerability_id) rows and adds a unique
constraint on the pair. CWE is a weakness class and is intentionally NOT
part of this change; vulnerability_id_type does not participate in
hash_code, so existing hash codes and deduplication are unaffected.

Migrations: 0276 (type column + lookup index), 0277 (dedupe + backfill,
data), 0278 (unique constraint).
Stacked on top of the vulnerability_id-type change (feat/vulnerability-id-type).

Adds a Finding_CWE relationship so a finding can carry multiple CWEs, mirroring
vulnerability ids: the primary CWE stays on Finding.cwe; additional CWEs live in
the relationship and are exposed via finding.cwes. Wired through the UI, the API
(cwes field), and parsers (finding.unsaved_cwes). CWE is a weakness class, kept
out of hash_code and the cve field, so existing hash codes/dedup are unaffected.

Migrations 0279_finding_cwe (create table) and 0280_backfill_finding_cwe (seed
from legacy Finding.cwe), chained after the vulnerability_id migrations (0278).
Backfill existing findings with: manage.py migrate_cwe
The generic JSON and CSV importers emitted only a single `cwe` per finding,
so a finding could not be imported with a CWE *set*. Mirror the existing
`vulnerability_ids` handling to accept multiple CWEs:

- JSON: accept an optional `cwes` list (popped like `vulnerability_ids`,
  since it is not a Finding field). The first entry becomes the primary
  `finding.cwe` when `cwe` is not already given; the full set is kept on
  `finding.unsaved_cwes`. Mixed int / "CWE-<n>" forms are supported.
- CSV: accept a delimited `CweIds` column alongside the existing single
  `CweId`, with the same primary-plus-set behavior; merge CWE sets across
  internally-deduplicated rows like vulnerability ids.

The import pipeline persists the set via `Finding_CWE`; normalization and
deduplication happen in `finding_cwe_labels()`. The legacy single `cwe`
path is unchanged.

Adds a fabricated multi-CWE fixture and parser tests (JSON + CSV).
Add 'cwes' to HASHCODE_ALLOWED_FIELDS and hash it via a new get_cwes(), which
mirrors get_vulnerability_ids. Because the primary cwe is a scalar that is always
set (so Finding.cwes is non-empty even before the extra Finding_CWE rows are
written during import), get_cwes prefers unsaved_cwes whenever present, so the
pre-save and post-save import hashes agree.

Tests: get_cwes prefers unsaved_cwes, is stable across save, empty with no cwe,
and cwes participates in compute_hash_code.
Mirror the existing cve/vulnerability_ids deprecation note: the single primary
cwe field is merged into the cwes set for backward compatibility and can be
removed once no longer needed.
… cached property

get_cwes() used the cwes @cached_property for the saved path; if finding.cwes was
accessed before the Finding_CWE rows were written (serializer/signal/log), it cached
an empty/partial set and the hash then used that stale value -> nondeterministic
hash_code (flaky dedup under parallel test runs). Mirror get_vulnerability_ids: read
the finding_cwe_set reverse relation directly (uncached, prefetch-honoring), falling
back to the unsaved values before save. save_cwes stores the primary cwe as a row too,
so finding_cwe_set carries the full set.
@valentijnscholten
valentijnscholten force-pushed the feat/false-positive-history-hook branch from 6fa344f to bfe43b2 Compare July 5, 2026 18:04
Move cwe_number, cwe_label, parse_cwes, finding_cwe_labels out of
finding/vulnerability_id.py into a dedicated finding/cwe.py (CWEs are a weakness
class, distinct from vulnerability identifiers). resolve_vulnerability_id_type
stays in vulnerability_id.py. All importers updated.
Add FINDING_FALSE_POSITIVE_HISTORY_CANDIDATE_FILTER_METHOD hook to
do_false_positive_history_batch. When configured, the per-finding candidate
list resolved by the deduplication algorithm is passed through the plugin
method before the FP decision, letting a plugin (Pro) narrow candidates by
fields excluded from the hash string but compared per pair (set-match tokens
on vulnerability_ids / CWEs). Resolved once via get_custom_method; a no-op
when unset, so default behavior is unchanged.

Tests: hook can suppress matches (drops candidates -> no FP replication) and
passthrough matches the default path.
@valentijnscholten
valentijnscholten force-pushed the feat/false-positive-history-hook branch from bfe43b2 to 75a1390 Compare July 6, 2026 13:54
@Maffooch
Maffooch force-pushed the feat/cwe-vuln-id-consolidation branch from 354058e to 518df6c Compare July 14, 2026 01:07
@github-actions

Copy link
Copy Markdown
Contributor

This pull request has conflicts, please resolve those before we can evaluate the pull request.

@Maffooch
Maffooch force-pushed the feat/cwe-vuln-id-consolidation branch 2 times, most recently from 2f3a611 to 0b291ea Compare July 14, 2026 03:15
@Maffooch
Maffooch deleted the branch DefectDojo:feat/cwe-vuln-id-consolidation July 14, 2026 03:54
@Maffooch Maffooch closed this Jul 14, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants