Skip to content

Commit 9607c83

Browse files
committed
Cleanup
1 parent 1cfea7b commit 9607c83

2 files changed

Lines changed: 17 additions & 14 deletions

File tree

docs/source/details/backendconfig.rst

Lines changed: 15 additions & 13 deletions
Original file line numberDiff line numberDiff line change
@@ -292,8 +292,8 @@ Dataset-specific configuration
292292
------------------------------
293293

294294
Some configuration options should be applicable on a per-dataset basis.
295-
All dataset-specific configuration options supported by the openPMD-api are additionally backend-specific, being format-specific serialization instructions such as compression or chunking.
296-
Dataset-specific configuration is hence specified under the key path ``<backend>.dataset``, e.g.:
295+
Most dataset-specific configuration options supported by the openPMD-api are additionally backend-specific, being format-specific serialization instructions such as compression or chunking.
296+
Such dataset-specific and backend-specific configuration is hence specified under the key path ``<backend>.dataset``, e.g.:
297297

298298
.. code-block:: json
299299
@@ -322,39 +322,41 @@ As a separate JSON/TOML configuration during dataset initialization
322322

323323
Similarly to the ``Series`` constructor, the ``Dataset`` constructor optionally receives a JSON/TOML configuration, used for setting options specifically only for those datasets initialized with this ``Dataset`` specification. The default given in the ``Series`` constructor will be overridden.
324324

325+
This is the preferred way for configuring dataset-specific options that are *not* backend-specific (currently only ``{"resizable": true}``).
326+
325327
By pattern-matching the dataset names
326328
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
327329

328-
The above approach has the disadvantage that it has to be supported explicitly at the level of the downstream application, e.g. a simulation or data reader. As an alternative, the the dataset configuration under ``<backend>.dataset`` can also be given as a list of alternatives that are matched against the dataset name in sequence, e.g. ``hdf5.dataset = [<pattern_1>, <pattern_2>, ...]``.
330+
The above approach has the disadvantage that it has to be supported explicitly at the level of the downstream application, e.g. a simulation or data reader. As an alternative, the the backend-specific dataset configuration under ``<backend>.dataset`` can also be given as a list of alternatives that are matched against the dataset name in sequence, e.g. ``hdf5.dataset = [<pattern_1>, <pattern_2>, ...]``.
329331

330-
Each such pattern ``<pattern_i>`` is a JSON object with key ``cfg`` and optional key ``select``: ``{"select": <pattern>, "cfg": <cfg>}``.
332+
Each such pattern ``<pattern_i>`` is a JSON object with key ``cfg`` and optional key ``select``: ``{"select": <regex>, "cfg": <cfg>}``.
331333

332-
In here, ``<pattern>`` is a regex or a list of regexes, as of type egrep as defined by the `C++ standard library <https://en.cppreference.com/w/cpp/regex/basic_regex/constants>`__.
333-
``<cfg>`` is a configuration that will be forwarded as-is to openPMD.
334+
In here, ``<regex>`` is a regex or a list of regexes, of type egrep as defined by the `C++ standard library <https://en.cppreference.com/w/cpp/regex/basic_regex/constants>`__.
335+
``<cfg>`` is a configuration that will be forwarded as a "regular" dataset configuration to the backend.
334336

335337
.. note::
336338

337-
To match lists of regular expressions ``select = [PATTERN_1, PATTERN_2, ..., PATTERN_n]``, the list is internally transformed into a single regular expression ``($^)|(PATTERN_1)|(PATTERN_2)|...|(PATTERN_n)``.
339+
To match lists of regular expressions ``select = [REGEX_1, REGEX_2, ..., REGEX_n]``, the list is internally transformed into a single regular expression ``($^)|(REGEX_1)|(REGEX_2)|...|(REGEX_n)``.
338340

339-
The single patterns will be processed in top-down manner, selecting the first matching pattern found in the list.
340-
The regexes will be matched against the openPMD dataset path either within the Iteration (e.g. ``meshes/E/x`` or ``particles/.*/position/.*``) or within the Series (e.g. ``/data/1/meshes/E/x`` or ``/data/.*/particles/.*/position/.*``), considering full matches only.
341+
In a configuration such as ``hdf5.dataset = [<pattern_1>, <pattern_2>, ...]``, the single patterns will be processed in top-down manner, selecting the first matching pattern found in the list.
342+
The specified regexes will be matched against the openPMD dataset path either within the Iteration (e.g. ``meshes/E/x`` or ``particles/.*/position/.*``) or within the Series (e.g. ``/data/1/meshes/E/x`` or ``/data/.*/particles/.*/position/.*``), considering full matches only.
341343

342344
.. note::
343345

344346
The dataset name is determined by the result of ``attributable.myPath().openPMDPath()`` where ``attributable`` is an object in the openPMD hierarchy.
345347

346348
.. note::
347349

348-
To match against the path within the containing Iteration or within the containing Series, the specified regular expression is internally transformed into ``(/data/[0-9]+/)?(PATTERN)`` where ``PATTERN`` is the specified pattern, and then matched against the full dataset path.
350+
To match against the path within the containing Iteration or within the containing Series, the specified regular expression is internally transformed into ``(/data/[0-9]+/)?(REGEX)`` where ``REGEX`` is the specified pattern, and then matched against the full dataset path.
349351

350352
The **default configuration** is specified by omitting the ``select`` key.
351353
Specifying more than one default is an error.
352354
If no pattern matches a dataset, the default configuration is chosen if specified, or an empty JSON object ``{}`` otherwise.
353355

354356
A full example:
355357

356-
.. literalinclude:: openpmd_extended_config.json
357-
:language: json
358-
359358
.. literalinclude:: openpmd_extended_config.toml
360359
:language: toml
360+
361+
.. literalinclude:: openpmd_extended_config.json
362+
:language: json

examples/13_write_dynamic_configuration.cpp

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -186,7 +186,8 @@ CFG.CHUNKS = [10]
186186

187187
/*
188188
* We want different compression settings for this dataset, so we pass
189-
* a dataset-specific configuration.
189+
* a dataset-specific configuration. This will override any definition
190+
* specified above.
190191
* Also showcase how to define an resizable dataset.
191192
* This time in JSON.
192193
*/

0 commit comments

Comments
 (0)