Skip to content

Commit 1d01485

Browse files
nscuroCopilot
andauthored
Add vuln data source docs and initial air-gap guide (#97)
* Add vuln data source docs and initial air-gap guide Signed-off-by: nscuro <nscuro@protonmail.com> * Potential fix for pull request finding Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com> Signed-off-by: Niklas <nscuro@protonmail.com> * Potential fix for pull request finding Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com> Signed-off-by: Niklas <nscuro@protonmail.com> --------- Signed-off-by: nscuro <nscuro@protonmail.com> Signed-off-by: Niklas <nscuro@protonmail.com> Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
1 parent f85fe99 commit 1d01485

21 files changed

Lines changed: 405 additions & 236 deletions
35.2 KB
Loading
20.7 KB
Loading
49.4 KB
Loading

docs/concepts/.pages

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -2,6 +2,7 @@ title: Concepts
22
nav:
33
- index.md
44
- About access control: access-control.md
5+
- About vulnerability data sources: about-vulnerability-data-sources.md
56
- About notifications: notifications.md
67
- About time series metrics: time-series-metrics.md
78
- About component policies: component-policies.md
Lines changed: 145 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,145 @@
1+
# About vulnerability data sources
2+
3+
A vulnerability data source is an upstream feed of known security defects that Dependency-Track uses to find
4+
vulnerabilities in your components. No single feed covers every ecosystem, identifier scheme, or disclosure path, so
5+
Dependency-Track integrates with several. You pick which ones to enable based on what you ship and where the relevant
6+
advisories are published. None of them are required, and enabling all of them is rarely the right answer.
7+
8+
This page explains what each source contributes, how the data reaches your findings, and the trade-offs you accept when
9+
you enable a source or turn one off. For the procedure to enable sources, see [Configuring vulnerability
10+
sources](../guides/administration/configuring-vulnerability-sources.md).
11+
12+
## Mirrored sources
13+
14+
Dependency-Track can mirror three public data sources into its own database:
15+
16+
- **[National Vulnerability Database (NVD)](https://nvd.nist.gov/)** is the canonical CVE feed maintained by NIST. It
17+
contains over 200,000 CVE records spanning the early nineties to the present and describes affected products as
18+
Common Platform Enumeration (CPE) names. NVD is the broadest source for non-package software (operating systems,
19+
firmware, network equipment), but its CPE-based product model is a poor fit for open source packages.
20+
- **[GitHub advisories](https://github.com/advisories)** is GitHub's curated advisory database (GHSA). It covers open
21+
source packages on npm, Maven, PyPI, NuGet, Go, RubyGems, and others. Advisories use Package URL (PURL) keys, often
22+
appear before the corresponding NVD entry, and frequently carry more ecosystem-specific detail than the NVD record.
23+
- **[OSV](https://osv.dev/)** is Google's open source vulnerability database. It aggregates advisories from GitHub,
24+
PyPA, RustSec, the Go team, distribution security teams, and others into one PURL-keyed dataset, and it covers
25+
ecosystems and distributions that NVD and GHSA do not (for example crates.io, Hex, Pub, Alpine, and Debian).
26+
27+
### Picking sources
28+
29+
The three feeds overlap heavily for open source ecosystems. Most operators do not need all three:
30+
31+
- **GHSA and OSV cover much of the same ground.** OSV ingests GitHub advisories and adds advisories from other curators
32+
(PyPA, RustSec, Go, distribution security teams). OSV also lets you mirror per ecosystem, which is finer-grained
33+
control than GHSA offers. If you want one source for open source packages, prefer OSV.
34+
- **GHSA** is useful on its own if you stay close to the GitHub ecosystem or want the data exactly as GitHub publishes
35+
it, without the aggregation layer that OSV adds.
36+
- **NVD** is the only mirrored source that meaningfully covers non-package software identified by CPE. Enable it when
37+
your BOMs include such components, or when you specifically need CVE coverage independent of curator-specific feeds.
38+
39+
## How Dependency-Track uses the data
40+
41+
Mirroring is a background task. On a configurable schedule and at instance startup, Dependency-Track downloads new and
42+
changed records from each enabled source and writes them into its own database.
43+
44+
Vulnerability analysis is a separate process. When you upload a Bill of Materials or trigger analysis, the [internal
45+
analyzer](../reference/analyzers.md#internal) matches the project's components against the mirrored data already stored
46+
locally. The internal analyzer makes no outbound calls during analysis, which makes findings reproducible and keeps
47+
analysis fast.
48+
49+
For the full analysis workflow, see [Vulnerability analysis](architecture/design/vulnerability-analysis.md).
50+
51+
### Turning a source off
52+
53+
Disabling a source stops further mirroring from that source, but Dependency-Track does not delete the records it has
54+
already written. Vulnerabilities mirrored before the source was turned off remain in the database, and the internal
55+
analyzer keeps matching components against them on subsequent analysis runs. Findings already raised from that source
56+
are not retroactively removed either.
57+
58+
The practical consequence is that turning a source off is reversible without re-downloading, but it does not clean up
59+
after itself. If you intend to drop a source entirely and have its findings disappear, expect to clear the data
60+
manually. Dependency-Track does not yet provide a built-in workflow for this.
61+
62+
## How component matching works
63+
64+
Each advisory in a mirrored source names the products it affects, using either a CPE or a PURL, and specifies the
65+
affected versions. The internal analyzer matches a component to an advisory by identifier scheme first and version
66+
second.
67+
68+
| Source | Identifier in records | What components need |
69+
|:-------|:----------------------|:---------------------|
70+
| NVD | CPE | A valid CPE on the component |
71+
| GitHub advisories | PURL | A valid PURL on the component |
72+
| OSV | PURL | A valid PURL on the component |
73+
74+
A component with neither a CPE nor a PURL is not evaluated.
75+
76+
### CPE matching
77+
78+
CPE matching follows the [NIST CPE name matching
79+
specification](https://nvlpubs.nist.gov/nistpubs/Legacy/IR/nistir7696.pdf): the analyzer compares all eleven CPE
80+
attributes (`part`, `vendor`, `product`, `version`, `update`, `edition`, `language`, `sw_edition`, `target_sw`,
81+
`target_hw`, `other`) and rejects the match if any attribute is disjoint.
82+
83+
Two adjustments reduce false positives compared to a literal reading of the spec:
84+
85+
- A SUBSET vendor combined with a SUPERSET product (or the reverse) is rejected, because these combinations otherwise
86+
produce noisy matches across unrelated products (see [issue
87+
#3178](https://github.com/DependencyTrack/dependency-track/issues/3178)).
88+
- Component versions of `*` (ANY) and `-` (NA) are handled per the spec, not run through range comparison.
89+
90+
When the advisory specifies a version range, the analyzer evaluates it using the
91+
[vers](https://www.packageurl.org/docs/vers/introduction) range specification, with a versioning scheme inferred from
92+
the component's PURL when present and the `generic` scheme otherwise.
93+
94+
### PURL matching
95+
96+
PURL matching requires the advisory's PURL `type`, `namespace`, and `name` to match the component's exactly. For
97+
OS-package PURLs (`deb`, `rpm`), if both sides carry a `distro` qualifier they must agree, with semantic equivalences
98+
honored (for example, `debian-13` matches `trixie`).
99+
100+
Version comparison uses [vers](https://www.packageurl.org/docs/vers/introduction) with the versioning scheme derived
101+
from the PURL type. If the strict scheme fails to parse a version, the analyzer falls back to the `generic` scheme to
102+
avoid false negatives. For `deb` and `rpm` PURLs, an `epoch` qualifier is folded into the version as `epoch:version`
103+
before comparison when the version does not already encode one.
104+
105+
### Practical consequence
106+
107+
Most modern BOM generators emit PURLs but not CPEs. As a result, components from package ecosystems (npm, Maven, PyPI,
108+
and so on) match GHSA and OSV well but produce few or no findings from NVD data, even when CVE records exist for those
109+
packages. This is the most common reason operators see fewer NVD findings than expected.
110+
111+
If you need NVD coverage for ecosystem packages, your BOM generator must also produce CPEs for those components.
112+
Otherwise, rely on GHSA or OSV for open source coverage and on NVD primarily for non-package software identified by CPE.
113+
The OWASP SBOM Forum's [recommendations to improve the
114+
NVD](https://owasp.org/blog/2022/09/13/sbom-forum-recommends-improvements-to-nvd) document this CPE-vs-PURL mismatch and
115+
the industry effort to close it.
116+
117+
## Aliases across sources
118+
119+
A single vulnerability often has different identifiers in different sources. Log4Shell, for example, is `CVE-2021-44228`
120+
in NVD and `GHSA-jfh8-c2jp-5v3q` in GHSA.
121+
122+
Dependency-Track records aliases when sources publish explicit links between identifiers and surfaces them in the UI.
123+
Aliases do not de-duplicate findings: if two enabled sources both report Log4Shell for the same component, you see two
124+
findings, one per source, with the alias relationship visible on each.
125+
126+
## Mirrored sources vs. external analyzers
127+
128+
The mirrored sources are not the only way Dependency-Track gets vulnerability data. It also ships analyzers that call
129+
external services at analysis time:
130+
131+
- **OSS Index** (Sonatype)
132+
- **Snyk**
133+
- **Trivy** (against a Trivy server you operate)
134+
- **VulnDB** (Flashpoint, commercial)
135+
136+
These analyzers send component identifiers to a third party on every analysis run, return findings inline, and store
137+
nothing locally beyond the result. They complement the mirrored sources but introduce an outbound dependency and, in
138+
some cases, a commercial contract. See [Vulnerability analyzers](../reference/analyzers.md) for the full list and their
139+
requirements.
140+
141+
## Vulnerabilities you define yourself
142+
143+
The [private vulnerability repository](../reference/datasources/private-vulnerability-repository.md) holds
144+
vulnerabilities you define yourself. Use it for vulnerabilities in internal code or for pre-disclosure tracking.
145+
Findings from private vulnerabilities behave the same as findings from public sources.

docs/concepts/changes-in-v5.md

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -168,6 +168,7 @@ remediation steps, lives in the
168168
use for them), and its file storage abstracts over backends like S3
169169
rather than assuming a local filesystem to serve from. Dependency-Track
170170
no longer serves this endpoint. Fetch feeds directly from NIST or run a dedicated mirror.
171+
See [Running air-gapped](../guides/administration/running-air-gapped.md) for hosting an internal NVD or OSV mirror.
171172

172173
[CEL]: https://cel.dev/
173174
[Protobuf]: https://protobuf.dev/

docs/guides/administration/.pages

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -7,6 +7,8 @@ nav:
77
- configuring-secret-management.md
88
- configuring-internal-ca.md
99
- configuring-http-proxy.md
10+
- configuring-vulnerability-sources.md
11+
- running-air-gapped.md
1012
- configuring-observability.md
1113
- configuring-ldap.md
1214
- configuring-oidc.md

docs/guides/administration/configuring-http-proxy.md

Lines changed: 4 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -1,9 +1,9 @@
11
# Configuring an HTTP proxy
22

3-
The Dependency-Track API server makes outbound HTTP and HTTPS calls to mirror vulnerability data sources, fetch
4-
package metadata from repositories, perform OIDC discovery, deliver webhooks, and reach other integrations. In
5-
environments where outbound traffic must traverse a corporate proxy, configure the API server to route those calls
6-
through it.
3+
The Dependency-Track API server makes outbound HTTP and HTTPS calls to mirror
4+
[vulnerability data sources](configuring-vulnerability-sources.md), fetch package metadata from repositories, perform
5+
OIDC discovery, deliver webhooks, and reach other integrations. In environments where outbound traffic must traverse a
6+
corporate proxy, configure the API server to route those calls through it.
77

88
Proxy configuration applies to the API server only. The frontend is a static single-page app served to the
99
user's browser; any requests it appears to make actually originate from the browser itself.

docs/guides/administration/configuring-internal-ca.md

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -15,7 +15,8 @@ Common scenarios where this occurs:
1515
- **LDAPS**: connecting to an LDAP server using TLS ([Configuring LDAP](configuring-ldap.md))
1616
- **OIDC**: the identity provider uses a privately signed certificate ([Configuring OIDC](configuring-oidc.md))
1717
- **HTTP proxy**: an intercepting TLS proxy sits between Dependency-Track and external services
18-
- **Internal vulnerability sources**: a private vulnerability repository or API server with an internal certificate
18+
- **Internal vulnerability sources**: an internal mirror of NVD or OSV behind a privately signed certificate (see
19+
[Configuring vulnerability sources](configuring-vulnerability-sources.md) and [Running air-gapped](running-air-gapped.md))
1920

2021
The solution is to import your internal CA certificate into the JVM truststore and
2122
mount the modified truststore into the container.
Lines changed: 103 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,103 @@
1+
# Configuring vulnerability sources
2+
3+
Dependency-Track can mirror three public vulnerability data sources into its internal database: the National
4+
Vulnerability Database (NVD), GitHub advisories, and OSV. You pick which ones to enable, configure them through the web
5+
UI, and trigger an initial mirror so findings appear without waiting for the next scheduled run.
6+
7+
For background on what each source provides, when one is enough, and how the internal analyzer turns mirrored data into
8+
findings, see [About vulnerability data sources](../../concepts/about-vulnerability-data-sources.md).
9+
10+
## Prerequisites
11+
12+
For each source you plan to enable, allow outbound HTTPS access from the API server to the corresponding host:
13+
14+
| Source | Host |
15+
|:-------|:-----|
16+
| NVD | `nvd.nist.gov` |
17+
| GitHub advisories | `api.github.com` |
18+
| OSV | `storage.googleapis.com` |
19+
20+
If outbound traffic must go through a proxy, see [Configuring an HTTP proxy](configuring-http-proxy.md). If the proxy
21+
intercepts TLS, see [Configuring internal CA trust](configuring-internal-ca.md).
22+
23+
The GitHub advisories source also requires a GitHub personal access token (PAT). The token needs no scopes, but the
24+
GitHub GraphQL API rejects unauthenticated requests. Create one at
25+
[github.com/settings/tokens](https://github.com/settings/tokens). NVD and OSV do not require credentials.
26+
27+
## Enabling sources
28+
29+
Open **Administration > Vulnerability Sources** in the web UI. Each source has its own configuration panel. The following steps
30+
cover the least configuration needed for findings.
31+
32+
### NVD
33+
34+
1. Open **Administration > Vulnerability Sources > NVD**.
35+
2. Enable the source.
36+
3. Select **Test** to verify the configured endpoint is reachable and that the feed file format matches what
37+
Dependency-Track expects.
38+
4. Select **Mirror now** to download the feed immediately. The first mirror downloads the entire dataset and can take a
39+
while.
40+
41+
![NVD configuration](../../assets/images/guides/administration/configuring-vulnerability-sources/nvd.png)
42+
43+
NVD records describe affected products as CPEs. The internal analyzer skips components that lack a valid CPE when
44+
evaluating NVD data. If you expect findings for open source packages identified by PURL and see none from NVD, that is
45+
the reason. See [How component matching
46+
works](../../concepts/about-vulnerability-data-sources.md#how-component-matching-works).
47+
48+
### GitHub advisories
49+
50+
1. Open **Administration > Vulnerability Sources > GitHub**.
51+
2. Enable the source.
52+
3. Enter the GitHub PAT from the prerequisites.
53+
4. Select **Mirror now** to download advisories immediately.
54+
55+
![GitHub Advisories configuration](../../assets/images/guides/administration/configuring-vulnerability-sources/github.png)
56+
57+
### OSV
58+
59+
1. Open **Administration > Vulnerability Sources > OSV**.
60+
2. Enable the source.
61+
3. Select the ecosystems you want to mirror. Dependency-Track downloads only the ecosystems you select.
62+
4. Select **Mirror now** to download the selected ecosystems immediately.
63+
64+
![OSV configuration](../../assets/images/guides/administration/configuring-vulnerability-sources/osv.png)
65+
66+
!!! tip
67+
For Debian, select the **Debian** ecosystem rather than individual Debian version ecosystems. The Debian package is
68+
a superset of all version-specific ones.
69+
70+
## Triggering an initial mirror
71+
72+
After enabling a source for the first time, use **Mirror now** rather than waiting for the next scheduled run. The first
73+
NVD mirror in particular can take a long time, and you want it underway before users start uploading BOMs and expecting
74+
findings.
75+
76+
Mirror progress and errors appear in the API server logs, so tail the logs during initial setup if you need to follow
77+
what each mirror is doing.
78+
79+
## Scheduling mirror runs
80+
81+
Each source has its own cron property. Mirrors also run on instance startup. To change the schedule, set the
82+
corresponding property:
83+
84+
- NVD: [`dt.task.nist.mirror.cron`](../../reference/configuration/properties.md#dttasknistmirrorcron)
85+
- GitHub advisories:
86+
[`dt.task.git.hub.advisory.mirror.cron`](../../reference/configuration/properties.md#dttaskgithubadvisorymirrorcron)
87+
- OSV: [`dt.task.osv.mirror.cron`](../../reference/configuration/properties.md#dttaskosvmirrorcron)
88+
89+
## Verifying findings
90+
91+
Once a mirror completes, upload a BOM for a project that contains components you know to be vulnerable, or trigger
92+
analysis on an existing project. Findings should appear within seconds of analysis completing. If they do not, check the
93+
API server logs for mirror errors and confirm the components carry the identifier the source uses for matching (CPE for
94+
NVD, PURL for GitHub advisories and OSV).
95+
96+
## See also
97+
98+
- [About vulnerability data sources](../../concepts/about-vulnerability-data-sources.md)
99+
- [Vulnerability datasources reference](../../reference/datasources/index.md)
100+
- [Vulnerability analyzers reference](../../reference/analyzers.md)
101+
- [Running air-gapped](running-air-gapped.md)
102+
- [Configuring an HTTP proxy](configuring-http-proxy.md)
103+
- [Configuring internal CA trust](configuring-internal-ca.md)

0 commit comments

Comments
 (0)