You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: docs/Developer and Contributor Corner/Database Engine.mdx
+3-7Lines changed: 3 additions & 7 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -31,7 +31,7 @@ Using the **timestamp** and **duration**, Netdata calculates for each point its
31
31
32
32
For incremental metrics (counters), Netdata interpolates the collected values to align them to the expected **end time** at the microsecond level, absorbing data collection micro-latencies.
33
33
34
-
When data points are stored in higher tiers (time aggregations - see [Tiers](#Tiers) below), each data point has:
34
+
When data points are stored in higher tiers (time aggregations - see [Tiers](#tiers) below), each data point has:
35
35
36
36
1. The **sum** of the original values that have been aggregated
37
37
2. The **count** of all the original values aggregated,
@@ -93,7 +93,7 @@ This collection of 64 pages that is packed and compressed together is called an
93
93
94
94
#### Datafiles
95
95
96
-
Multiple **extents** are appended to **datafiles** (filename suffix `.ndf`), until these **datafiles** become full. The size of each **datafile** is determined automatically by Netdata. The minimum for each **datafile** is 4MB and the maximum 512MB. Depending on the amount of disk space configured for each tier, Netdata will decide a **datafile** size trying to maintain about 50 datafiles for the whole database, within the limits mentioned (4MB min, 512MB max per file). The maximum number of datafiles supported is 65536, and therefore the maximum database size (per tier) that Netdata can support is 32TB.
96
+
Multiple **extents** are appended to **datafiles** (filename suffix `.ndf`) until these **datafiles** become full. The size of each **datafile** is determined automatically by Netdata. Depending on the amount of disk space configured for each tier, Netdata chooses a target **datafile** size and clamps it between the compiled minimum and maximum datafile sizes. The current target is to keep about 100 datafiles for the tier.
97
97
98
98
#### Journal Files
99
99
@@ -179,7 +179,7 @@ Its primary use is to index information about the open datafile, the one that st
179
179
180
180
The clean queue is an LRU for reducing the journal v2 scans during querying.
181
181
182
-
Open cache uses memory ballooning too, like the main cache, based on its own hot pages. Open cache hot size is mainly controlled by the size of the open datafile. This is why on netdata versions with journal files v2, we decreased the maximum datafile size from 1GB to 512MB, and we increased the target number of datafiles from 20 to 50.
182
+
Open cache uses memory ballooning too, like the main cache, based on its own hot pages. Open cache hot size is mainly controlled by the size of the open datafile. This is why Netdata keeps datafile sizes bounded and targets many datafiles per tier.
183
183
184
184
On bigger setups open cache will get a bigger LRU by automatically sizing it (the whole open cache) to 5% to the size of (the whole) main cache.
185
185
@@ -196,7 +196,3 @@ The time-ranges of the queries running control the amount of shared memory requi
196
196
## Metrics Registry
197
197
198
198
DBENGINE uses 150 bytes of memory for every metric for which retention is maintained but is not currently being collected.
Copy file name to clipboardExpand all lines: docs/Logs/Systemd Journal Logs/Systemd Journal Plugin Reference.mdx
+29Lines changed: 29 additions & 0 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -47,6 +47,35 @@ This plugin is a Netdata Function Plugin. A free Netdata Cloud account is requir
47
47
48
48
The plugin is designed for native package installations, source installations, and Docker installations (Debian-based). If using Docker, make sure you're using the Debian-based containers.
49
49
50
+
## Offline Function test mode
51
+
52
+
Fixture harnesses can execute the Function query path directly against an existing
-`<journal-dir>` is scanned recursively for systemd journal files.
62
+
- stdin is the JSON Function request body (non-empty, maximum 16 MiB).
63
+
-`--request` is not supported and fails with usage output.
64
+
-`--timeout <seconds>` controls the offline Function execution timeout. It
65
+
defaults to `60`; use `--timeout 0` to map to a very large finite timeout for
66
+
long-running fixture comparisons.
67
+
- stdout contains only the raw JSON Function response.
68
+
- errors are written to stderr and return non-zero.
69
+
70
+
For example, an info request can use this payload:
71
+
72
+
```json
73
+
{"info":true}
74
+
```
75
+
76
+
Function output includes volatile fields such as versions and timing-derived values.
77
+
Test harnesses should normalize those fields before comparing fixture outputs.
78
+
50
79
## Journal sources
51
80
52
81
The plugin automatically detects available journal sources based on the journal files in `/var/log/journal` (persistent logs) and `/run/log/journal` (volatile logs).
Copy file name to clipboardExpand all lines: docs/Netdata Agent/Configuration/Database.mdx
+14-1Lines changed: 14 additions & 1 deletion
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -29,7 +29,14 @@ Use [`edit-config`](/docs/netdata-agent/configuration#edit-configuration-files)
29
29
30
30
:::note
31
31
32
-
In a Parent-Child setup, these settings manage the entire storage space used by the Parent for storing metrics collected both by itself and its Children.
32
+
In a Parent-Child setup, these settings control the Parent's total storage for metrics collected locally and metrics received from Children.
33
+
34
+
Child and Parent storage are independent:
35
+
36
+
- A Child can keep local history based on its own `[db].mode`.
37
+
- Streamed metrics can also be persisted on the Parent, in the Parent's own dbengine files.
38
+
39
+
Retention size is enforced **per-tier**, not per Child, so all streaming Children share the Parent's tier quota. For Parent sizing guidance, see [Parent Retention Sizing](/docs/netdata-agent/resource-utilization/disk-&-retention#parent-retention-sizing).
33
40
34
41
:::
35
42
@@ -41,6 +48,12 @@ You can fine-tune retention for each tier by setting a time limit or size limit.
41
48
| Time Limit = 0, Size Limit > 0 |**Space based:** data is stored with a disk space limit, regardless of time |
42
49
| Time Limit > 0, Size Limit > 0 |**Combined time and space limits:** data is deleted once it reaches either the time limit or the disk space limit, whichever comes first |
43
50
51
+
:::note
52
+
53
+
Retention size limits are soft targets, not hard caps enforced at write time. Actual disk usage can temporarily exceed the configured limit. For the detailed enforcement behavior, see [Retention Size Enforcement](/docs/netdata-agent/database#retention-size-enforcement).
54
+
55
+
:::
56
+
44
57
You can change these limits using [`edit-config`](/docs/netdata-agent/configuration#edit-configuration-files) to open `netdata.conf`:
|`dbengine`| The high performance multi-tiered time-series database of Netdata, providing superior storage efficiency (~0.5 bytes per sample on disk for high resolution per-second data), and fast long term data queries (typically 20+ times faster) by transparently utilizing all available database tiers. For details, see [Database Engine](/docs/developer-and-contributor-corner/database-engine). |
20
-
|`ram`| Stores data entirely in memory without disk persistence. This is typically used in IoT deviced or children that stream their metrics to Netdata parents, to avoid having any disk dependency on Netdata|
21
-
|`none`| Operates without storage (metrics can only be streamed to a Netdata parent). |
20
+
|`ram`| Stores data entirely in memory without disk persistence. This is typically used in IoT devices or children that stream their metrics to Netdata parents, to avoid having any disk dependency on Netdata |
21
+
|`none`| Operates without storage (metrics can only be streamed to a Netdata parent). |
22
22
23
23
## Tiers
24
24
@@ -48,10 +48,27 @@ These limits are fully configurable. See [Changing how long Netdata stores metri
48
48
49
49
Netdata provides a visual representation of storage utilization for both the time and space limits across all Tiers under "Netdata" -> "dbengine retention" on the dashboard. This chart shows exactly how your storage space (disk space limits) and time (time limits) are used for metric retention.
50
50
51
+
### Retention Size Enforcement
52
+
53
+
Retention size limits are **soft caps**, not hard caps. Netdata writes data unconditionally and checks limits afterwards — it does not block or reject writes when a cap is approaching.
54
+
55
+
How enforcement works:
56
+
57
+
1.**Periodic and asynchronous checks**: Netdata checks whether a tier has exceeded its configured size limit periodically and also after normal database activity (such as extent writes and rotation completions). Data continues to be written to disk without restriction between these checks.
58
+
59
+
2.**Whole-file deletion**: When the configured size limit is exceeded, Netdata schedules deletion of the oldest complete datafiles until the retention check no longer reports the tier over its limit. Multiple files may be deleted across one or more rotation passes. Datafile size is determined automatically (see [Database Engine datafiles](/docs/developer-and-contributor-corner/database-engine#datafiles)). Because entire files are removed and cannot be partially deleted, actual disk usage can overshoot the configured limit before enforcement catches up.
60
+
61
+
3.**Why tier 0 overshoots more**: Tier 0 collects per-second data, producing the highest write volume. More data accumulates between enforcement checks, and data files fill faster. Tier 1 and tier 2 have lower write volumes and their disk usage grows more predictably.
62
+
63
+
**Practical guidance**:
64
+
65
+
- Provision storage with workload-specific headroom, especially on parent nodes streaming from many children. The required headroom depends on ingestion rate, compression, storage throughput, and how quickly rotation catches up.
66
+
- Setting both **retention size** and **retention time** for the same tier can reduce retained history when either threshold is reached, but it does not create a hard disk cap.
67
+
- There is currently no mechanism to enforce a true hard cap on dbengine disk usage. To reduce the risk of disk-full conditions, validate retention sizing under expected production load and ensure adequate storage headroom.
68
+
51
69
## Cache sizes
52
70
53
71
There are two cache sizes that can be used to optimize the Database:
54
72
55
73
1.**Page cache size**: The main cache that keeps metrics data into memory. When data is not found in it, the extent cache is consulted, and if not found in that too, they are loaded from the disk.
56
74
2.**Extent cache size**: The compressed extent cache. It keeps in memory compressed data blocks, as they appear on disk, to avoid reading them again. Data found in the extent cache but not in the main cache have to be uncompressed to be queried.
Copy file name to clipboardExpand all lines: docs/Netdata Agent/Installation/Windows/Windows.mdx
+68-1Lines changed: 68 additions & 1 deletion
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -143,12 +143,20 @@ Automatic updates require internet access and are not possible on air-gapped sys
143
143
144
144
## Access Netdata Dashboard
145
145
146
+
After installation, verify the Netdata service is running:
147
+
148
+
```powershell
149
+
Get-Service netdata
150
+
```
151
+
146
152
After installation, open your browser and go to:
147
153
148
154
```
149
155
http://localhost:19999
150
156
```
151
157
158
+
If your subscription and installation mode allow local monitoring, you should see the Netdata Dashboard — a real-time metrics overview with system CPU, memory, disk, and network charts updating every second.
159
+
152
160
## License Information
153
161
154
162
By using silent installation, you agree to:
@@ -161,7 +169,7 @@ By using silent installation, you agree to:
161
169
When Netdata is installed on Windows, it automatically registers as a Windows Service and appears in
162
170
**Add or remove programs** (also known as **Programs and Features** or **Apps & features** in newer Windows versions).
163
171
The service can be monitored through the [Netdata Dashboard](http://localhost:19999).
164
-
To start, stop, or restart the service, use Windows Services (services.msc) or command-line tools
172
+
To start, stop, or restart the service, use the [PowerShell commands described in Service Control](/docs/netdata-agent/maintenance/service-control#windows).
165
173
166
174
## Automatic Updates
167
175
@@ -244,3 +252,62 @@ Instead of daily updates, you might prefer:
244
252
- Only on specific days of the week
245
253
246
254
:::
255
+
256
+
## Working with Netdata on Windows
257
+
258
+
Netdata on Windows includes a bundled MSYS2 environment for working with Netdata configuration files and other internals.
259
+
260
+
### Open the MSYS2 environment
261
+
262
+
Launch the bundled MSYS2 shell using one of these methods (the paths below assume Netdata is installed in the default location):
263
+
264
+
-**Windows Run dialog**: Press `Win + R`, enter `"C:\Program Files\Netdata\msys2.exe"`, and press `Enter`.
Inside the MSYS2 environment, use the `edit-config` helper to edit Netdata configuration files:
290
+
291
+
```bash
292
+
cd /etc/netdata
293
+
./edit-config netdata.conf
294
+
```
295
+
296
+
For the complete `edit-config` workflow and configuration directory layout, see [Netdata Agent Configuration](/docs/netdata-agent/configuration#edit-configuration-files).
297
+
298
+
On Windows, `edit-config` opens files with the `nano` editor.
![github-actions[bot]](https://avatars.githubusercontent.com/in/15368?v=4&size=40){kind=avatar}
0 commit comments