Skip to content

Commit e40683d

Browse files
renetapopovamauvo
andauthored
Clarify import memory controls (#2980) (#3072)
## Summary - clarify on the generic admin configuration page that `neo4j-admin database import` uses `--max-off-heap-memory` rather than `server.memory.pagecache.size` for import memory sizing - add an `Import memory controls` subsection to the import page - explain that `Configured max memory` in import output is the effective off-heap memory budget - update the full and incremental option tables to make `--max-off-heap-memory` the explicit tuning knob for import memory ## Validation - `git diff --check` - `npm run build:preview` --------- Co-authored-by: David Pond <davidpond@gmail.com>
1 parent fa67261 commit e40683d

5 files changed

Lines changed: 23 additions & 10 deletions

File tree

modules/ROOT/pages/backup-restore/consistency-checker.adoc

Lines changed: 1 addition & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -98,8 +98,7 @@ The `neo4j-admin database check` command has the following options:
9898
| .
9999

100100
|--max-off-heap-memory=<size>
101-
| Maximum memory that the command can use for page cache and various caching data structures to improve performance.
102-
Value can be plain numbers, such as 10000000 or, for example, 20G for 20 gigabytes, or 70%, which will amount to 70% of currently free memory on the machine.
101+
| Maximum off-heap memory that the command can use for page cache and various caching data structures to improve performance. Use this option to tune the command memory usage; the command does not use the server.memory.pagecache.size configuration setting for this purpose. Values can be plain numbers, such as 10000000, or, for example, 20G for 20 gigabytes, or 70%, which will amount to 70% of currently free memory on the machine.
103102
|90%
104103

105104
|--threads=<number of threads>

modules/ROOT/pages/backup-restore/copy-database.adoc

Lines changed: 1 addition & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -133,8 +133,7 @@ The indexes will be built the first time the database is started.
133133
|
134134

135135
|--from-pagecache, --max-off-heap-memory=<size>
136-
|label:new[Introduced in 2025.01] Maximum memory that the command can use for page cache and various caching data structures to improve performance.
137-
Values can be plain numbers, such as 10000000, or, for example, 20G for 20 gigabytes, or 70%, which will amount to 70% of currently free memory on the machine.
136+
|label:new[Introduced in 2025.01] Maximum off-heap memory that the command can use for page cache and various caching data structures to improve performance. Use this option to tune the command memory usage; the command does not use the server.memory.pagecache.size configuration setting for this purpose. Values can be plain numbers, such as 10000000, or, for example, 20G for 20 gigabytes, or 70%, which will amount to 70% of currently free memory on the machine.
138137
|90%
139138

140139
|--from-path-data=<path>

modules/ROOT/pages/database-administration/standard-databases/migrate-database.adoc

Lines changed: 1 addition & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -97,8 +97,7 @@ This option is only relevant when migrating from Neo4j 4.4.
9797
|
9898

9999
|--max-off-heap-memory=<size> label:new[Introduced in Neo4j 2025.04]
100-
|Maximum memory that the command can use for page cache and various caching data structures to improve performance.
101-
Values can be plain numbers, such as 10000000, or, for example, 20G for 20 gigabytes, or 70%, which will amount to 70% of currently free memory on the machine.
100+
|Maximum off-heap memory that the command can use for page cache and various caching data structures to improve performance. Use this option to tune the command memory usage; the command does not use the server.memory.pagecache.size configuration setting for this purpose. Values can be plain numbers, such as 10000000, or, for example, 20G for 20 gigabytes, or 70%, which will amount to 70% of currently free memory on the machine.
102101
|90%
103102

104103
|--pagecache=<size> label:deprecated[Deprecated in Neo4j 2025.04]

modules/ROOT/pages/import.adoc

Lines changed: 17 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -61,6 +61,21 @@ For information on `LOAD CSV`, see the link:{neo4j-docs-base-uri}/cypher-manual/
6161
For in-depth examples of using the command `neo4j-admin database import`, refer to the xref:tutorial/neo4j-admin-import.adoc[Tutorials -> Importing data].
6262
====
6363

64+
[[import-memory-controls]]
65+
=== Import memory controls
66+
67+
`neo4j-admin database import` uses two main memory controls:
68+
69+
* JVM heap, configured for the admin process through the usual configuration sources described in xref:neo4j-admin-neo4j-cli.adoc#_configuration[Neo4j Admin and Neo4j CLI -> Configuration].
70+
* Off-heap working memory, controlled by `--max-off-heap-memory`.
71+
If this option is not provided, the importer derives a budget from the available system resources.
72+
73+
The importer does not use the `server.memory.pagecache.size` configuration setting to size import memory.
74+
The off-heap budget is shared across page cache and other off-heap data structures used during import.
75+
In dry-run and verbose output, `Configured max memory` shows the effective off-heap memory budget available to the import command.
76+
77+
[[import-file-considerations]]
78+
=== Input file considerations
6479
These are some things you need to keep in mind when creating your input files:
6580

6681
* Fields are comma-separated by default but a different delimiter can be specified.
@@ -303,8 +318,7 @@ Possible values are:
303318
| {check-mark}
304319

305320
|--max-off-heap-memory=<size>
306-
|Maximum memory that the command can use for page cache and various caching data structures to improve performance.
307-
Values can be plain numbers, such as 10000000, or, for example, 20G for 20 gigabytes, or 70%, which will amount to 70% of currently free memory on the machine.
321+
|Maximum off-heap memory that the command can use for page cache and various caching data structures to improve performance. Use this option to tune the command memory usage; the command does not use the server.memory.pagecache.size configuration setting for this purpose. Values can be plain numbers, such as 10000000, or, for example, 20G for 20 gigabytes, or 70%, which will amount to 70% of currently free memory on the machine.
308322
|90%
309323
| {check-mark}
310324
| {check-mark}
@@ -1001,8 +1015,7 @@ Possible values are:
10011015
| {check-mark}
10021016

10031017
|--max-off-heap-memory=<size>
1004-
|Maximum memory that the command can use for page cache and various caching data structures to improve performance.
1005-
Values can be plain numbers, such as 10000000, or, for example, 20G for 20 gigabytes, or 70%, which will amount to 70% of currently free memory on the machine.
1018+
|Maximum off-heap memory that the command can use for page cache and various caching data structures to improve performance. Use this option to tune the command memory usage; the command does not use the server.memory.pagecache.size configuration setting for this purpose. Values can be plain numbers, such as 10000000, or, for example, 20G for 20 gigabytes, or 70%, which will amount to 70% of currently free memory on the machine.
10061019
|90%
10071020
| {check-mark}
10081021
| {check-mark}

modules/ROOT/pages/neo4j-admin-neo4j-cli.adoc

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -273,6 +273,9 @@ Administration operations use the configuration specified in the _neo4j.conf_ fi
273273
Sharing configuration between the DBMS and its administration tasks makes sense as most settings are the same.
274274
In some cases, however, it is better to override some settings specified in _neo4j.conf_ by configuring the tasks (instead of updating the config settings in the _neo4j.conf_ file) because administration tasks generally use fewer resources than the DBMS.
275275
For instance, if the page cache of your DBMS is configured to a very high value in _neo4j.conf_, and you want to override this because the admin tasks like backup do not need so much memory, you provide configuration for the admin tasks instead of updating the page cache setting in the _neo4j.conf_ file.
276+
`neo4j-admin database import` does not use `server.memory.pagecache.size` for import memory sizing.
277+
Instead, use `--max-off-heap-memory` to tune the command's off-heap working memory.
278+
See xref:import.adoc#import-memory-controls[Import -> Import memory controls] for details.
276279

277280
There are several options for overriding settings specified in the _neo4j.conf_ file using administration tasks:
278281

0 commit comments

Comments
 (0)