Add JVN (Japan Vulnerability Notes) vulnerability data source#6640
Open
mitsukado-shinagawa wants to merge 2 commits into
Open
Add JVN (Japan Vulnerability Notes) vulnerability data source#6640mitsukado-shinagawa wants to merge 2 commits into
mitsukado-shinagawa wants to merge 2 commits into
Conversation
Proposes mirroring JVN (Japan Vulnerability Notes / JVN iPedia) as a new vuln-data-source module via the ADR-010 extension point: yearly detail feeds (jvndb_detail_YYYY.rdf) with checksum-based skipping, CPE 2.2 to 2.3 normalisation, Japanese version-expression parsing into vers ranges, and all advisories stored under a new JVN source. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Signed-off-by: mitsukado-shinagawa <mitsukado-shinagawa@users.noreply.github.com>
✅ Snyk checks have passed. No issues have been found so far.
💻 Catch issues earlier using the plugins for VS Code, JetBrains IDEs, Visual Studio, and Eclipse. |
Up to standards ✅🟢 Issues
|
| Metric | Results |
|---|---|
| Complexity | 258 |
NEW Get contextual insights on your PRs based on Codacy's metrics, along with PR and Jira context, without leaving GitHub. Enable AI reviewer
TIP This summary will be updated as you push new changes.
cb39a1e to
7cde1da
Compare
Implements ADR-031. Adds a vuln-data-source/jvn module (disabled by
default) that mirrors JVN iPedia from the yearly detail feeds
(jvndb_detail_YYYY.rdf, 1998-present), skipping years whose sha256 in
checksum.txt is unchanged since the last successful run.
Advisories are parsed from the VULDEF schema, converted to CycloneDX BOVs
and ingested through the existing mirror pipeline; no changes to
ingestion, storage, or matching. JVN-specific handling:
- CPE 2.2 URIs are normalised to CPE 2.3 formatted strings.
- The free-text Japanese VersionNumber field is parsed into vers ranges
(e.g. "0.15.0 以上 0.15.2 未満" -> vers:generic/>=0.15.0|<0.15.2);
products without a parseable version fall back to an all-versions range.
- Platform/edition qualifiers that JVN appends to version texts
("5.0 (client)", "Version 1809 for x64-based Systems", "12.04 LTS",
"2013 SP1") are stripped in a salvage pass — NVD encodes the same
qualifiers in separate CPE attributes (update/edition/target hardware),
so the stripped version value matches NVD's representation. Cisco-style
parenthesized build versions ("11.1(15)ca") are kept verbatim, as NVD
does. Measured against all yearly feeds (813k VersionNumber entries),
this halves the share of affected products whose version information
would otherwise be lost (0.95% -> 0.49%).
- CWE ids are extracted from <RelatedItem type="cwe"> entries; their
glossary URLs are skipped rather than emitted as advisory references.
- The Impact and Solution section descriptions are mapped to the
CycloneDX detail and recommendation fields.
- Every advisory is stored under the JVN source, keyed by its JVNDB id.
Registers JVN in Vulnerability.Source, as the mirror validates source
names via Source.ofName.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Signed-off-by: mitsukado-shinagawa <mitsukado-shinagawa@users.noreply.github.com>
7cde1da to
42f9b22
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
Adds a new vulnerability data source module for JVN / JVN iPedia (Japan Vulnerability Notes, operated by JPCERT/CC and IPA), implementing the
VulnDataSourceextension point of ADR-010. Disabled by default.The module mirrors the complete JVN history from the yearly detail feeds (
jvndb_detail_YYYY.rdf, 1998–present), skipping years whosesha256inchecksum.txtis unchanged since the last successful run (feed-digest checkpoint in the plugin KV store, mirroring the NVD module's checkpointing). Advisories are converted to CycloneDX BOVs and flow through the existing mirror pipeline — no changes to ingestion, storage, or matching.JVN-specific handling:
affectscomponents."0.15.0 以上 0.15.2 未満"). These are parsed intoversranges (以上→>=,未満→<, …) so CPE matching is version-precise. Products with no parseable version fall back to an all-versions range.JVNsource, keyed by itsJVNDB-*id (no CVE→NVD routing; rationale and the considered alternative are in the ADR). This requires registeringJVNinVulnerability.Source, since the mirror validates source names viaSource.ofName.Addressed Issue
Closes #6626
Additional Details
docs/adr/031-jvn-vulnerability-data-source.md(first commit of this PR, status Proposed) documents the PoC measurements that drove the design — notably that the MyJVN query API caps at the ~747 most recent advisories and cannot backfill history (hence the yearly feeds, the same channel vulsio/go-cve-dictionary uses), and that ~96% of JVN entries overlap the NVD, making the JVN-only ~4% (mostly Japanese domestic-vendor products), Japanese descriptions, and timeliness the durable value.InternalVulnAnalyzerproduced findings for in-range CPE components and correctly did not match out-of-range versions.PluginManagerto support loading of plugins from JARs #6187, this is contributed in-tree for now. Once external plugin loading matures, I'd be happy to use this module as a real-world test case for migrating a vulnerability data source out-of-tree.Checklist
This PR fixes a defect, and I have provided tests to verify that the fix is effectiveThis PR introduces changes to the database model, and I have updated the migration changelog accordinglydocs/adr/