feat!(detector): route VulnCheck and JVN to vuls2 by MaineK00n · Pull Request #2590 · future-architect/vuls

MaineK00n · 2026-06-29T16:49:12Z

Summary

Migrates VulnCheck (vulncheck-nist-nvd2) and JVN CPE detection from
go-cve-dictionary to vuls2 — the last two CPE sources — then removes
go-cve-dictionary entirely, completing the vuls2 migration. Builds on the prior
NVD / MITRE CVE v5 (#2586) / EUVD and Cisco (#2581) / Palo Alto (#2589) /
Fortinet (#2591) vuls2 migrations, all now on master.

Breaking change

go-cve-dictionary is removed entirely: its CLI/config (cveDict,
GoCveDictConf), DB validation and the go-cve detection paths are gone. CPE
detection — and the JVN/VulnCheck content and JP-CERT / US-CERT alerts that came
with it — now comes solely from the vuls2 DB.

Commits

detect and enrich VulnCheck NVD++ via vuls2 — detect on the source's
whole CPE criteria and suppress a matched scanned CPE whose part:vendor:product
is in the verified-defined set (NVD/Fortinet/Cisco/PaloAlto), keyed by CVE-ID
(GetVulnerabilityDataByVulnerabilityID); enrich VulnCheck content.
detect and enrich JVN via vuls2 — the same suppression for JVN (JVN sits
under JVNDB-* roots, so the verified set is fetched by CVE-ID). The CPE
detection path produces the per-CVE JVN CveContent; enrichJVN lifts JP-CERT
alerts and adds a minimal {SourceLink} JVNDB pointer for CVEs that have JVN
data but were not detected via JVN (e.g. package / RedHat-detected), keeping the
detection-built content when JVN was the detector. JVN's note-level title / CVSS
/ CWE are deliberately NOT distributed onto each per-CVE content (a single JVNDB
note often covers several CVEs); they live in the JVNDB DistroAdvisory.
remove go-cve-dictionary — delete the cve-client, config plumbing,
converters and dead detection paths; CPE detection is now vuls2-only.
honour per-CPE UseJVN in vuls2 and detect macOS through it — vuls2.DetectCPEs
takes []vuls2.CPE{URI, UseJVN}; walkCPECriteria suppresses a JVN-source match
whose scanned CPE is UseJVN:false, at the same match-time granularity as the
verified-product suppression. macOS apple CPEs (vuls2.MacOSCPEs) are detected
here as UseJVN:false, restoring the go-cve macOS behaviour; the dead detector-side
Cpe struct is dropped for vuls2.CPE.
error on an incomplete recognized-family scan, skip per-server — a recognized
OS family reaching DetectPkgCves with no release / no packages now errors
instead of silently reporting zero CVEs; Detect records it on r.Errors and
skips just that server.
add JVN/VulnCheck detection and enrichment unit tests — Test_postConvert
and Test_enrich cases for the migrated sources (NVD was previously the only
CPE source with such tests), with fixtures under testdata/fixtures/enrich.

Verification

build (normal / -tags scanner / GOOS=windows), go vet, gofmt, unit tests green.
Smoke (test DB): macOS apple-CPE detection suppresses JVN; the same openssl CPE
with UseJVN=true → 66 CVEs (15 with JVN) vs UseJVN=false → 57 (0 JVN).

Notes

Cost optimisation deferred to a separate PR. Verified-product suppression
(collectVerifiedProducts) re-fetches each detected VulnCheck/JVN CVE's
verified-source data with a per-CVE GetVulnerabilityDataByVulnerabilityID. On a
pathological kernel-as-CPE scan against an 11.5 GB DB
(cpe:/o:linux:linux_kernel:6.1 → 7268 CVEs; 1162 with VulnCheck / 109 with JVN
content) this dominates DetectCPEs — measured ~42s warm cache (cold cache: did
not finish within 2 min):

phase	time (warm)
`detect`	~5.8s
`collectVerifiedProducts`	~36s (~85%)
`postConvert`	~0.4s

The re-fetch is redundant: cpe.Detect → util.Detect already returns, for every
candidate root, all sources' full criteria (the CPE index is keyed per
part:vendor:product / version-agnostic, and conditions pass through
unconditionally), so the verified sources' defined products are already present in
the in-memory detection result. The follow-up will derive the suppression set from
that result instead of re-fetching, removing the per-CVE bolt I/O.

🤖 Generated with Claude Code

Move VulnCheck (vulncheck-nist-nvd2) detection and enrichment from go-cve-dictionary to the vuls2 DB, mirroring the earlier NVD/MITRE migrations. vuls2 now surfaces VulnCheck CPE matches (models.Vulncheck content, Vulncheck{ExactVersion,VendorProduct}Match tiers, console.vulncheck.com source link) and enriches CVSS/refs/CWE via enrichVulnCheck. To preserve go-cve-dictionary's isCpeURIAlsoDefinedInVerifiedDataSource behavior, a matched VulnCheck CPE is suppressed when the same part:vendor:product is DEFINED by a verified source (NVD/Fortinet/Cisco/ PaloAlto) for that CVE. The verified-defined set is gathered per CVE via GetVulnerabilityDataByVulnerabilityID (keyed by CVE ID, not RootID, and cached) so it composes with sources rooted elsewhere; isSuppressedCPESource gates which sources this applies to. go-cve-dictionary no longer detects or fills VulnCheck (dropped from the CPE admission gate and FillCvesWithGoCVEDictionary; detail.Vulnchecks cleared alongside detail.Nvds). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

Copilot

Pull request overview

This PR completes the migration of CPE-driven sources (VulnCheck NVD++ and JVN) from go-cve-dictionary to vuls2, and removes go-cve-dictionary integration/configuration from vuls. It also introduces per-CPE JVN gating (UseJVN) and verified-source suppression to preserve legacy detection semantics while unifying detection/enrichment under vuls2.

Changes:

Route CPE detection (including VulnCheck and JVN) through vuls2 with per-CPE UseJVN and “verified product” suppression logic.
Add vuls2 enrich support for VulnCheck and JVN (including JP-CERT alert extraction) and update/extend vuls2 fixtures/tests accordingly.
Remove go-cve-dictionary client/config/converters/tests and associated CLI/server plumbing.

Reviewed changes

Copilot reviewed 22 out of 23 changed files in this pull request and generated 2 comments.

Show a summary per file

File	Description
subcmds/server.go	Removes go-cve-dictionary init/DB validation from server subcommand startup.
subcmds/report.go	Removes go-cve-dictionary init path from report command.
subcmds/report_windows.go	Same as report.go for Windows build.
server/server.go	Removes go-cve-dictionary fill; relies on vuls2 detection/enrich paths in server mode.
models/utils.go (deleted)	Removes go-cve-dictionary-to-model converters (JVN/NVD/VulnCheck/Fortinet/PaloAlto/Cisco).
models/utils_test.go (deleted)	Removes tests for the deleted conversion helpers.
go.mod	Drops go-cve-dictionary and related indirect deps.
go.sum	Prunes transitive deps no longer needed after go-cve-dictionary removal.
detector/vuls2/vuls2.go	Adds `CPE{URI, UseJVN}`, no-JVN suppression, verified-product suppression, and verified-product collection via vuls2 DB.
detector/vuls2/vuls2_test.go	Updates/extends tests for new CPE conversion/suppression/enrich behavior and adds fixtures coverage.
detector/vuls2/vendor.go	Adds source-link cases (JVN, VulnCheck), adds JVN/VulnCheck enrichers, and adds `MacOSCPEs`.
detector/vuls2/vendor_test.go	Adds unit tests for `MacOSCPEs`.
detector/vuls2/testdata/fixtures/enrich/vulncheck-nist-nvd2/datasource.json	Adds VulnCheck enrich datasource fixture.
detector/vuls2/testdata/fixtures/enrich/vulncheck-nist-nvd2/data/2024/CVE-2024-3400.json	Adds VulnCheck vulnerability fixture.
detector/vuls2/testdata/fixtures/enrich/jvn-feed-rss/datasource.json	Adds JVN RSS enrich datasource fixture.
detector/vuls2/testdata/fixtures/enrich/jvn-feed-rss/data/2024/JVNDB-2024-000123.json	Adds JVN advisory/vulnerability fixture used by enrich tests.
detector/util.go	Removes go-cve-dictionary DB validation helper.
detector/detector.go	Switches CPE detection to vuls2-only; adds “skip per-server on incomplete recognized-family scan” behavior and moves macOS CPE detection into pkg detection phase.
detector/detector_test.go (deleted)	Removes tests that were only validating go-cve-dictionary-based confidence selection.
detector/cve_client.go (deleted)	Removes go-cve-dictionary HTTP/DB client implementation.
config/vulnDictConf.go	Removes vuln dict config/types; leaves vuls2 config.
config/tomlloader.go	Removes vuln dict init during TOML load.
config/config.go	Removes `CveDict` from config and its validation on report.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Copilot

Pull request overview

Copilot reviewed 22 out of 23 changed files in this pull request and generated 1 comment.

Copilot

Pull request overview

Copilot reviewed 22 out of 23 changed files in this pull request and generated 1 comment.

Copilot

Pull request overview

Copilot reviewed 22 out of 23 changed files in this pull request and generated 1 comment.

Copilot

Pull request overview

Copilot reviewed 22 out of 23 changed files in this pull request and generated no new comments.

Move JVN (jvn-feed-rss / jvn-feed-detail) detection and enrichment from go-cve-dictionary to the vuls2 DB, applying the same verified-source suppression as VulnCheck — go-cve-dictionary runs the identical isCpeURIAlsoDefinedInVerifiedDataSource gate over JVN and VulnCheck, so JVN joins isSuppressedCPESource. Because JVN's version-ANY CPEs over-match, the verified-defined suppression removes many NVD-bounded false positives (e.g. linux_kernel 5.10: JVN matches drop from ~13.5k to ~108, with no CVE co-detected by both JVN and NVD). JVN's rich fields (Japanese title/summary, CVSS, CWE) live in advisory content, so enrichJVN reads them from there and replaces the sparse, vulnerability-derived Jvn content the CPE detection path produces. JP-CERT alerts are reproduced from JVN advisory references: go-cve-dictionary selected references whose source was "JPCERT-AT", which the vuls2 jvn extractor flattens to "jvndb.jvn.jp", but every such reference is a JPCERT/CC alert URL (www.jpcert.or.jp/at/...), so the URL identifies them equivalently (verified 15184/15191 against the raw feed). go-cve-dictionary no longer detects or fills JVN (dropped from the CPE admission gate, the JVN advisory synthesis, getMaxConfidence, and FillCvesWithGoCVEDictionary; fillCertAlerts removed; detail.Jvns cleared alongside detail.Nvds / detail.Vulnchecks). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

With NVD, VulnCheck, JVN, Fortinet, Cisco and PaloAlto all detected and enriched from the vuls2 DB, go-cve-dictionary is no longer a detection or enrichment source, so drop it entirely: - detector: remove the go-cve-dictionary client (cve_client.go), detectCpeURIsCvesWithGoCVEDictionary, FillCvesWithGoCVEDictionary, getMaxConfidence and ValidateDBs; DetectCpeURIsCves now only drives vuls2 and no longer takes a GoCveDictConf / LogOpts. - models: remove the go-cve-dictionary model converters (utils.go). - config: remove GoCveDictConf, the cveDict config and the VulnDict validate/init plumbing that existed only for it. - subcmds / server: drop the cveDict init, DB validation and fill calls. - go.mod: drop github.com/vulsio/go-cve-dictionary and now-unused deps. This assumes Fortinet/Cisco/PaloAlto are already migrated to vuls2 (their migrations land before this is rebased); until then this branch has no go-cve-dictionary fallback for them. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

Copilot

Pull request overview

Copilot reviewed 22 out of 23 changed files in this pull request and generated 1 comment.

…gh it CPE detection now carries a per-CPE UseJVN flag down into vuls2 instead of the detector-side flag that go-cve-dictionary removal had left dead. vuls2.DetectCPEs takes []vuls2.CPE{URI, UseJVN}; preConvertCPEs collects the FS forms of the UseJVN:false CPEs, and walkCPECriteria suppresses a JVN-source match whose scanned CPE is in that set — the same match-time granularity as the verified-product suppression beside it. So a JVN-only detection on such a CPE never registers and a CPE also detected by NVD/VulnCheck keeps only those. Use it for macOS. macOS has no package security database, so its installed applications and OS are synthesised as Apple CPEs (vuls2.MacOSCPEs, in vendor.go) and detected through the CPE path — moved out of Detect()'s generic CPE-collection block into the macOS case of DetectPkgCves. MacOSCPEs marks its CPEs UseJVN:false, restoring the go-cve-dictionary behaviour where Apple CPEs do not consult JVN; user / OWASP CPEs stay UseJVN:true. The detector-side Cpe struct, identical to vuls2.CPE, is dropped in favour of the latter (DetectCpeURIsCves now takes []vuls2.CPE). isPkgCvesDetactable is folded into DetectPkgCves's r.Family switch so every family is a single inline case; OS-family behaviour is preserved (the error-on-incomplete-scan change is a separate commit). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

…Errors A recognized OS family (RedHat/Debian/...) that reaches DetectPkgCves with no release or no packages is an incomplete scan result. It used to be skipped silently, reporting zero CVEs as if the host were clean. Record the failure on r.Errors instead; trivy scans (which detect on their own) are still skipped, and an unknown family is still only an error when it actually carries scan data. DetectPkgCves records this on r.Errors and returns without error, so only that server's package detection is skipped — its CPE/WordPress detection and the other servers still run. A genuine detection failure (e.g. the vuls2 DB) still returns an error and aborts the run, consistent with the other detection steps. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

Copilot

Pull request overview

Copilot reviewed 22 out of 23 changed files in this pull request and generated no new comments.

shino

🏁 🥅 Great!! 👋 👋

Cover the two CPE sources migrated to vuls2 whose source-specific logic the NVD-based cases did not exercise. Test_postConvert (inline DetectResult): - VulnCheck exact CPE accept -> VulncheckExactVersionMatch + models.Vulncheck content with the console.vulncheck.com SourceLink. - JVN exact CPE accept -> demoted to JvnVendorProductMatch, sparse models.Jvn content (SourceLink ""), and the JVNDB DistroAdvisory. Test_enrich (testdata/fixtures/enrich/{vulncheck-nist-nvd2,jvn-feed-rss}): - enrichVulnCheck fills models.Vulncheck content (CVSS/CWE/refs/SourceLink). - enrichJVN rebuilds the rich Jvn content from the advisory, folds the sparse detection content's references forward (carriedRefs), and emits the JP-CERT alert from a jpcert.or.jp/at/ reference. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

collectVerifiedProducts keyed the verified part:vendor:product set by RootID, unioning it across every CVE under the root. For multi-CVE JVN roots (a JVNDB note covers several CVEs) this over-suppressed: a VulnCheck / JVN match for product P was dropped for every CVE in the note when P was verified for any one of them, hiding sibling CVEs the verified source does not actually cover. Key the verified set per (RootID, CVE) and walk each suppressed source's CPE criteria once per constituent CVE with that CVE's own verified set, so suppression is scoped to the CVE and AND/OR criteria satisfaction stays correct. (Empirically JVN/VulnCheck criteria are all single/OR with no AND of multiple CPEs, so the result matches a simpler post-walk filter, but keeping suppression inside walkCPECriteria stays correct even if AND criteria appear.) Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

shino

🏁 🥅 Great!! 👋 👋

MaineK00n force-pushed the MaineK00n/vulncheck-jvn-vuls2 branch 4 times, most recently from 1cbaa98 to 9bd685f Compare June 30, 2026 06:37

MaineK00n self-assigned this Jun 30, 2026

MaineK00n force-pushed the MaineK00n/vulncheck-jvn-vuls2 branch 3 times, most recently from a6cac72 to a2be586 Compare June 30, 2026 10:30

MaineK00n changed the base branch from master to shino/cpe-detect-fortinet June 30, 2026 10:30

MaineK00n force-pushed the MaineK00n/vulncheck-jvn-vuls2 branch from a2be586 to ec112ea Compare June 30, 2026 10:47

MaineK00n changed the title ~~feat: migrate VulnCheck/JVN detection to vuls2 and remove go-cve-dictionary~~ feat!(detector): route VulnCheck and JVN to vuls2 Jun 30, 2026

Base automatically changed from shino/cpe-detect-fortinet to master June 30, 2026 12:57

MaineK00n force-pushed the MaineK00n/vulncheck-jvn-vuls2 branch from ec112ea to 493b2d2 Compare June 30, 2026 13:07

MaineK00n marked this pull request as ready for review June 30, 2026 13:07

MaineK00n requested a review from Copilot June 30, 2026 13:08

Copilot started reviewing on behalf of MaineK00n June 30, 2026 13:08 View session

Copilot AI reviewed Jun 30, 2026

View reviewed changes

Comment thread detector/detector.go Outdated

Comment thread detector/vuls2/vuls2_test.go Outdated

MaineK00n force-pushed the MaineK00n/vulncheck-jvn-vuls2 branch from 493b2d2 to a754b6b Compare June 30, 2026 13:25

MaineK00n requested a review from Copilot June 30, 2026 13:27

Copilot started reviewing on behalf of MaineK00n June 30, 2026 13:27 View session

Copilot AI reviewed Jun 30, 2026

View reviewed changes

Comment thread detector/detector.go

MaineK00n requested a review from Copilot June 30, 2026 14:28

Copilot started reviewing on behalf of MaineK00n June 30, 2026 14:28 View session

Copilot AI reviewed Jun 30, 2026

View reviewed changes

Comment thread detector/vuls2/vuls2.go Outdated

MaineK00n force-pushed the MaineK00n/vulncheck-jvn-vuls2 branch from a754b6b to dd5767d Compare June 30, 2026 14:43

MaineK00n requested a review from Copilot June 30, 2026 16:07

Copilot started reviewing on behalf of MaineK00n June 30, 2026 16:07 View session

Copilot AI reviewed Jun 30, 2026

View reviewed changes

Comment thread detector/detector.go

MaineK00n force-pushed the MaineK00n/vulncheck-jvn-vuls2 branch from e1e6dc9 to b1d095e Compare June 30, 2026 16:20

MaineK00n force-pushed the MaineK00n/vulncheck-jvn-vuls2 branch from b1d095e to 4906fd7 Compare June 30, 2026 16:22

MaineK00n requested a review from Copilot June 30, 2026 16:23

Copilot started reviewing on behalf of MaineK00n June 30, 2026 16:24 View session

Copilot AI reviewed Jun 30, 2026

View reviewed changes

MaineK00n force-pushed the MaineK00n/vulncheck-jvn-vuls2 branch from 4906fd7 to 021cdec Compare June 30, 2026 16:29

MaineK00n and others added 2 commits July 1, 2026 01:50

MaineK00n force-pushed the MaineK00n/vulncheck-jvn-vuls2 branch 2 times, most recently from 77a8bdd to 411ea72 Compare June 30, 2026 17:17

MaineK00n requested a review from Copilot June 30, 2026 17:19

Copilot started reviewing on behalf of MaineK00n June 30, 2026 17:19 View session

Copilot AI reviewed Jun 30, 2026

View reviewed changes

Comment thread detector/detector.go Outdated

MaineK00n force-pushed the MaineK00n/vulncheck-jvn-vuls2 branch from 411ea72 to 035115e Compare June 30, 2026 17:26

MaineK00n and others added 2 commits July 1, 2026 02:28

MaineK00n force-pushed the MaineK00n/vulncheck-jvn-vuls2 branch from 035115e to adaa539 Compare June 30, 2026 17:29

MaineK00n requested a review from Copilot June 30, 2026 17:29

Copilot started reviewing on behalf of MaineK00n June 30, 2026 17:30 View session

Copilot AI reviewed Jun 30, 2026

View reviewed changes

MaineK00n requested a review from shino June 30, 2026 17:34

MaineK00n mentioned this pull request Jul 1, 2026

perf(detector): derive verified-product suppression from the detection result #2592

Merged

shino previously approved these changes Jul 1, 2026

View reviewed changes

MaineK00n and others added 2 commits July 1, 2026 13:13

MaineK00n dismissed shino’s stale review via 28d61cb July 1, 2026 04:13

MaineK00n force-pushed the MaineK00n/vulncheck-jvn-vuls2 branch from adaa539 to 28d61cb Compare July 1, 2026 04:13

MaineK00n requested a review from shino July 1, 2026 04:16

shino approved these changes Jul 1, 2026

View reviewed changes

MaineK00n merged commit f621477 into master Jul 1, 2026
8 checks passed

MaineK00n deleted the MaineK00n/vulncheck-jvn-vuls2 branch July 1, 2026 04:32

Uh oh!

Uh oh!

Conversation

MaineK00n commented Jun 29, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

Breaking change

Commits

Verification

Notes

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

shino left a comment

Choose a reason for hiding this comment

Uh oh!

shino left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

MaineK00n commented Jun 29, 2026 •

edited

Loading