Skip to content

Commit b5d2458

Browse files
nscuronscuro
committed
Add reference docs for Cache, Task scheduler, and Telemetry
Signed-off-by: nscuro <nscuro@protonmail.com>
1 parent de86020 commit b5d2458

7 files changed

Lines changed: 235 additions & 3 deletions

File tree

‎docs/reference/configuration/.pages‎

Lines changed: 4 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,7 +1,10 @@
11
nav:
22
- Application: application.md
3+
- Cache: cache.md
34
- Data sources: datasources.md
4-
- File storage: file-storage.md
55
- Database: database.md
66
- Durable execution engine: dex-engine.md
7+
- File storage: file-storage.md
8+
- Task scheduler: task-scheduler.md
9+
- Telemetry: telemetry.md
710
- All properties: properties.md

‎docs/reference/configuration/cache.md‎

Lines changed: 77 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,77 @@
1+
# Cache
2+
3+
Dependency-Track caches results of expensive operations to reduce load on
4+
external services and speed up repeated lookups. Each cache has a TTL after
5+
which entries expire and the cache provider drops them.
6+
7+
Select the provider via [`dt.cache.provider`](properties.md#dtcacheprovider).
8+
9+
## Providers
10+
11+
### Database
12+
13+
The `database` provider stores cache entries in the `CACHE_ENTRY` table on a
14+
configured data source. This is the only built-in provider.
15+
16+
!!! warning
17+
The provider does not manage its own schema. The configured data source
18+
must point at the same schema as the `default` data source, where Dependency-Track's
19+
schema migrations create the `CACHE_ENTRY` table.
20+
21+
!!! tip
22+
Pointing the cache at a [secondary data source](datasources.md) that
23+
targets the same schema can still be useful as a way to isolate cache
24+
traffic from the main connection pool, so that cache activity doesn't
25+
compete with the API server for database connections.
26+
27+
A background maintenance worker periodically deletes expired rows and
28+
refreshes the per-cache size counters that feed the cache metrics.
29+
30+
Configuration:
31+
32+
- [`dt.cache.provider.database.datasource.name`](properties.md#dtcacheproviderdatabasedatasourcename)
33+
- [`dt.cache.provider.database.maintenance.initial-delay-ms`](properties.md#dtcacheproviderdatabasemaintenanceinitial-delay-ms)
34+
- [`dt.cache.provider.database.maintenance.interval-ms`](properties.md#dtcacheproviderdatabasemaintenanceinterval-ms)
35+
36+
## Entry TTLs
37+
38+
Each named cache has its own TTL property of the form
39+
`dt.cache."<name>".ttl-ms`.
40+
41+
### Vulnerability analyzer results
42+
43+
Cached results of remote vulnerability analyzer lookups, keyed by component
44+
identifier. Reduces API calls to upstream services across analyses of the same
45+
components.
46+
47+
Configuration:
48+
49+
- [`dt.cache."vuln-analyzer.oss-index.results".ttl-ms`](properties.md#dtcachevuln-analyzeross-indexresultsttl-ms)
50+
- [`dt.cache."vuln-analyzer.snyk.results".ttl-ms`](properties.md#dtcachevuln-analyzersnykresultsttl-ms)
51+
52+
### Package metadata resolver responses
53+
54+
Cached HTTP responses from package registry metadata endpoints, used to detect
55+
outdated components.
56+
57+
!!! tip
58+
Entries include the response body together with `ETag` and `Last-Modified` validators
59+
so that refreshes after the freshness window revalidate with a conditional request and
60+
receive a 304 when the upstream response hasn't changed. The TTL must exceed the 12 h
61+
freshness window for conditional requests to fire.
62+
63+
Configuration:
64+
65+
- [`dt.cache."package-metadata-resolver.cargo.responses".ttl-ms`](properties.md#dtcachepackage-metadata-resolvercargoresponsesttl-ms)
66+
- [`dt.cache."package-metadata-resolver.composer.responses".ttl-ms`](properties.md#dtcachepackage-metadata-resolvercomposerresponsesttl-ms)
67+
- [`dt.cache."package-metadata-resolver.cpan.responses".ttl-ms`](properties.md#dtcachepackage-metadata-resolvercpanresponsesttl-ms)
68+
- [`dt.cache."package-metadata-resolver.gem.responses".ttl-ms`](properties.md#dtcachepackage-metadata-resolvergemresponsesttl-ms)
69+
- [`dt.cache."package-metadata-resolver.github.responses".ttl-ms`](properties.md#dtcachepackage-metadata-resolvergithubresponsesttl-ms)
70+
- [`dt.cache."package-metadata-resolver.gomodules.responses".ttl-ms`](properties.md#dtcachepackage-metadata-resolvergomodulesresponsesttl-ms)
71+
- [`dt.cache."package-metadata-resolver.hackage.responses".ttl-ms`](properties.md#dtcachepackage-metadata-resolverhackageresponsesttl-ms)
72+
- [`dt.cache."package-metadata-resolver.hex.responses".ttl-ms`](properties.md#dtcachepackage-metadata-resolverhexresponsesttl-ms)
73+
- [`dt.cache."package-metadata-resolver.maven.responses".ttl-ms`](properties.md#dtcachepackage-metadata-resolvermavenresponsesttl-ms)
74+
- [`dt.cache."package-metadata-resolver.nixpkgs.responses".ttl-ms`](properties.md#dtcachepackage-metadata-resolvernixpkgsresponsesttl-ms)
75+
- [`dt.cache."package-metadata-resolver.npm.responses".ttl-ms`](properties.md#dtcachepackage-metadata-resolvernpmresponsesttl-ms)
76+
- [`dt.cache."package-metadata-resolver.nuget.responses".ttl-ms`](properties.md#dtcachepackage-metadata-resolvernugetresponsesttl-ms)
77+
- [`dt.cache."package-metadata-resolver.pypi.responses".ttl-ms`](properties.md#dtcachepackage-metadata-resolverpypiresponsesttl-ms)

‎docs/reference/configuration/datasources.md‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,4 +1,4 @@
1-
# Data Sources
1+
# Data sources
22

33
Data sources are logical objects through which database connections can be acquired.
44

‎docs/reference/configuration/file-storage.md‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,4 +1,4 @@
1-
# File Storage
1+
# File storage
22

33
Dependency-Track uses file storage for intermediate data during background processing,
44
including uploaded BOMs, vulnerability analysis results, and large notifications.

‎docs/reference/configuration/task-scheduler.md‎

Lines changed: 73 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,73 @@
1+
# Task scheduler
2+
3+
The task scheduler runs or triggers recurring background activities like vulnerability
4+
data source mirroring, integration uploads, metric updates, and maintenance
5+
sweeps. The [durable execution engine](dex-engine.md) handles the bulk of
6+
asynchronous work and is a separate subsystem.
7+
8+
Nodes with the scheduler enabled coexist safely. The scheduler coordinates
9+
execution through the database, so each due task runs on exactly one node
10+
without requiring leader election.
11+
12+
## Configuration
13+
14+
You can turn the scheduler off on a per-node basis, for example to dedicate
15+
specific nodes to serving web traffic. The `threads` setting caps how many
16+
scheduled tasks may run concurrently on a single node. Because the
17+
[durable execution engine](dex-engine.md) performs most asynchronous work,
18+
this is rarely the right knob to tune.
19+
20+
Configuration:
21+
22+
- [`dt.task-scheduler.enabled`](properties.md#dttask-schedulerenabled)
23+
- [`dt.task-scheduler.threads`](properties.md#dttask-schedulerthreads)
24+
- [`dt.task-scheduler.poll-interval-ms`](properties.md#dttask-schedulerpoll-interval-ms)
25+
- [`dt.task-scheduler.shutdown-max-wait-ms`](properties.md#dttask-schedulershutdown-max-wait-ms)
26+
27+
## Cron expressions
28+
29+
Each scheduled task takes its schedule from a `dt.task.<name>.cron` property,
30+
which holds a five-field UNIX cron expression:
31+
32+
```text
33+
minute hour day-of-month month day-of-week
34+
```
35+
36+
All expressions use **UTC**.
37+
38+
Example, run every day at 03:30 UTC:
39+
40+
```ini
41+
dt.task.nvd-vuln-data-source-mirror.cron=30 3 * * *
42+
```
43+
44+
## Scheduled tasks
45+
46+
The scheduler ships with the following recurring tasks. Tasks marked with
47+
[^1] also run once shortly after startup, with a random delay of up to one
48+
minute.
49+
50+
| Task | Property | Default cron | Purpose |
51+
|-----------------------------------|-------------------------------------------------------------------------------------------------------------------------|------------------|-----------------------------------------------------------------------------------------------|
52+
| Portfolio metrics update | [`dt.task.portfolio-metrics-update.cron`](properties.md#dttaskportfolio-metrics-updatecron) | `10 * * * *` | Refreshes per-project and portfolio time series metrics. |
53+
| Vulnerability metrics update | [`dt.task.vuln-metrics-update.cron`](properties.md#dttaskvuln-metrics-updatecron) | `0 */6 * * *` | Refreshes counters used by the vulnerability dashboard. |
54+
| Portfolio vulnerability analysis | [`dt.task.portfolio-analysis.cron`](properties.md#dttaskportfolio-analysiscron) | `0 6 * * *` | Re-analyzes every component in the portfolio against current vulnerability data. |
55+
| NVD mirror[^1] | [`dt.task.nvd-vuln-data-source-mirror.cron`](properties.md#dttasknvd-vuln-data-source-mirrorcron) | `0 4 * * *` | Mirrors the NIST National Vulnerability Database. |
56+
| GitHub Advisories mirror[^1] | [`dt.task.github-advisory-vuln-data-source-mirror.cron`](properties.md#dttaskgithub-advisory-vuln-data-source-mirrorcron) | `0 2 * * *` | Mirrors the GitHub Advisory Database. |
57+
| OSV mirror[^1] | [`dt.task.osv-vuln-data-source-mirror.cron`](properties.md#dttaskosv-vuln-data-source-mirrorcron) | `0 3 * * *` | Mirrors the OSV vulnerability database. |
58+
| EPSS mirror[^1] | [`dt.task.epss-mirror.cron`](properties.md#dttaskepss-mirrorcron) | `0 1 * * *` | Mirrors the FIRST EPSS scores feed. |
59+
| Vulnerability database maintenance | [`dt.task.vuln-database-maintenance.cron`](properties.md#dttaskvuln-database-maintenancecron) | `0 0 * * *` | Removes orphaned vulnerability records and reconciles indexes. |
60+
| Vulnerability policy bundle sync[^1] | [`dt.task.vuln-policy-bundle-sync.cron`](properties.md#dttaskvuln-policy-bundle-synccron) | `*/15 * * * *` | Pulls the configured vulnerability policy bundle. No-op when `dt.vuln-policy-bundle.url` is empty. |
61+
| Package metadata resolution | [`dt.task.package-metadata-resolution.cron`](properties.md#dttaskpackage-metadata-resolutioncron) | `0 1 * * *` | Refreshes outdated-component status for portfolio components. |
62+
| Package metadata maintenance | [`dt.task.package-metadata-maintenance.cron`](properties.md#dttaskpackage-metadata-maintenancecron) | `0 */12 * * *` | Removes stale and orphaned package metadata records. |
63+
| Metrics maintenance | [`dt.task.metrics-maintenance.cron`](properties.md#dttaskmetrics-maintenancecron) | `1 * * * *` | Compacts time series metric history. |
64+
| Tag maintenance | [`dt.task.tag-maintenance.cron`](properties.md#dttasktag-maintenancecron) | `0 */12 * * *` | Removes unused tags. |
65+
| Project maintenance | [`dt.task.project-maintenance.cron`](properties.md#dttaskproject-maintenancecron) | `0 */4 * * *` | Applies project retention and cleanup rules. |
66+
| Fortify SSC upload | [`dt.task.fortify-ssc-upload.cron`](properties.md#dttaskfortify-ssc-uploadcron) | `0 2 * * *` | Uploads finding reports to Fortify SSC. |
67+
| DefectDojo upload | [`dt.task.defect-dojo-upload.cron`](properties.md#dttaskdefect-dojo-uploadcron) | `0 2 * * *` | Uploads finding reports to DefectDojo. |
68+
| Kenna Security upload | [`dt.task.kenna-security-upload.cron`](properties.md#dttaskkenna-security-uploadcron) | `0 2 * * *` | Uploads finding reports to Kenna Security. |
69+
| Expired session cleanup | [`dt.task.expired-session-cleanup.cron`](properties.md#dttaskexpired-session-cleanupcron) | `0 * * * *` | Deletes expired user session tokens. |
70+
| Scheduled notification dispatch | [`dt.task.scheduled-notification-dispatch.cron`](properties.md#dttaskscheduled-notification-dispatchcron) | `* * * * *` | Polls for due scheduled notification rules and dispatches them. |
71+
| Telemetry submission[^1] | [`dt.task.telemetry-submission.cron`](properties.md#dttasktelemetry-submissioncron) | `0 */1 * * *` | Submits anonymous usage data. See [Telemetry](telemetry.md). |
72+
73+
[^1]: Triggered once on startup with a random delay of up to one minute, then on the configured schedule.

‎docs/reference/configuration/telemetry.md‎

Lines changed: 78 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,78 @@
1+
# Telemetry
2+
3+
Dependency-Track periodically submits a small, anonymous data point to
4+
`https://metrics.dependencytrack.org`. This helps the maintainers understand
5+
adoption, plan version support, and decide which database systems to keep
6+
testing against.
7+
8+
The submission does not include information that could tie data back to a
9+
specific organization, such as IP addresses, hostnames, or project content.
10+
11+
## Data collected
12+
13+
Each submission is a single JSON object with four fields:
14+
15+
| Field | Description |
16+
|--------------|-----------------------------------------------------------------------------|
17+
| `system_id` | Random cluster identifier generated at first startup. Stable across restarts and node replacements. |
18+
| `dt_version` | Dependency-Track release version. |
19+
| `db_type` | Database product name reported by JDBC (for example, `PostgreSQL`). |
20+
| `db_version` | Database product version reported by JDBC. |
21+
22+
The `system_id` lets the maintainers correlate submissions from the same
23+
deployment over time without identifying who runs it.
24+
25+
## Submission schedule
26+
27+
The [task scheduler](task-scheduler.md) runs the telemetry submission task
28+
once shortly after startup with a random delay of up to one minute, and then
29+
on the schedule defined by
30+
[`dt.task.telemetry-submission.cron`](properties.md#dttasktelemetry-submissioncron)
31+
(default `0 */1 * * *`, hourly).
32+
33+
The task itself enforces a 24-hour interval between submissions, so the cron
34+
expression controls how often the task checks whether a new submission is
35+
due, not how often data is actually sent.
36+
37+
## Opting out
38+
39+
You can opt out at any time through any of the following methods.
40+
41+
### Administration UI
42+
43+
Navigate to **Administration** > **Configuration** > **Telemetry** and turn
44+
off submission.
45+
46+
### Configuration API
47+
48+
```shell linenums="1"
49+
curl -X POST \
50+
-H 'X-Api-Key: odt_...' \
51+
-H 'Content-Type: application/json' \
52+
-d '{
53+
"groupName": "telemetry",
54+
"propertyName": "submission.enabled",
55+
"propertyValue": "false"
56+
}' \
57+
https://dtrack.example.com/api/v1/configProperty
58+
```
59+
60+
### Before first startup
61+
62+
To opt out before the first startup populates the runtime setting, set
63+
[`dt.telemetry.submission.default-enabled`](properties.md#dttelemetrysubmissiondefault-enabled)
64+
to `false`:
65+
66+
```ini
67+
dt.telemetry.submission.default-enabled=false
68+
```
69+
70+
As an environment variable:
71+
72+
```shell
73+
DT_TELEMETRY_SUBMISSION_DEFAULT_ENABLED=false
74+
```
75+
76+
This property only seeds the initial value of the runtime setting. Once the
77+
setting exists in the database, changes via the UI or REST API override the
78+
initial value.

‎mkdocs.yml‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -81,6 +81,7 @@ markdown_extensions:
8181
- abbr
8282
- admonition
8383
- attr_list
84+
- footnotes
8485
- md_in_html
8586
- toc:
8687
permalink: true

0 commit comments

Comments
 (0)