You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: docs/source/details/backendconfig.rst
+33Lines changed: 33 additions & 0 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -94,6 +94,39 @@ Using the Streaming API (i.e. ``SeriesInterface::readIteration()``) will do this
94
94
Parsing eagerly might be very expensive for a Series with many iterations, but will avoid bugs by forgotten calls to ``Iteration::open()``.
95
95
In complex environments, calling ``Iteration::open()`` on an already open environment does no harm (and does not incur additional runtime cost for additional ``open()`` calls).
96
96
97
+
By default, the library will print a warning to suggest using deferred Iteration parsing when opening a Series takes long.
98
+
The timeout can be tuned by the JSON/TOML key ``hint_lazy_parsing_timeout`` (integer, seconds):
99
+
if set to a positive value, the library will print periodic warnings to stderr when eager parsing of Iterations takes longer than the specified number of seconds (default: ``20``). Setting this option to ``0`` disables the warnings.
100
+
101
+
Environment variables may alternatively be used for options concerning deferred iteration parsing:
102
+
103
+
* Environment variable ``OPENPMD_DEFER_ITERATION_PARSING``: if set to a truthy value (e.g. ``1``), the Series will be opened with deferred iteration parsing as if ``{"defer_iteration_parsing": true}`` had been supplied.
104
+
* Environment variable ``OPENPMD_HINT_LAZY_PARSING_TIMEOUT``: accepts integral values equivalent to the ``hint_lazy_parsing_timeout`` key.
105
+
106
+
Examples:
107
+
108
+
.. code-block:: bash
109
+
110
+
# enable lazy parsing via env var
111
+
export OPENPMD_DEFER_ITERATION_PARSING=1
112
+
113
+
# disable the parsing hint/warning
114
+
export OPENPMD_HINT_LAZY_PARSING_TIMEOUT=0
115
+
116
+
Or in a Series constructor JSON/TOML configuration:
117
+
118
+
.. code-block:: json
119
+
120
+
{
121
+
"defer_iteration_parsing": true,
122
+
"hint_lazy_parsing_timeout": 20
123
+
}
124
+
125
+
As of openPMD-api 0.17.0, the parser verifies that all records within a mesh or within a particle species have consistent shapes / extents.
126
+
This is used for filling in the shape for constant components that do not define it.
127
+
In order to skip this check in the error case, the key ``{"verify_homogeneous_extents": false}`` may be set (alternatively ``export OPENPMD_VERIFY_HOMOGENEOUS_EXTENTS=0`` will do the same).
128
+
This will help read datasets with inconsistent metadata definitions.
129
+
97
130
The key ``resizable`` can be passed to ``Dataset`` options.
98
131
It if set to ``{"resizable": true}``, this declares that it shall be allowed to increased the ``Extent`` of a ``Dataset`` via ``resetDataset()`` at a later time, i.e., after it has been first declared (and potentially written).
99
132
For HDF5, resizable Datasets come with a performance penalty.
0 commit comments