|
| 1 | +.. |
| 2 | + Copyright (c) 2024-2026 CRS4 |
| 3 | +
|
| 4 | + Licensed under the Apache License, Version 2.0 (the "License"); |
| 5 | + you may not use this file except in compliance with the License. |
| 6 | + You may obtain a copy of the License at |
| 7 | + |
| 8 | + http://www.apache.org/licenses/LICENSE-2.0 |
| 9 | + |
| 10 | + Unless required by applicable law or agreed to in writing, software |
| 11 | + distributed under the License is distributed on an "AS IS" BASIS, |
| 12 | + WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. |
| 13 | + See the License for the specific language governing permissions and |
| 14 | + limitations under the License. |
| 15 | + |
| 16 | + |
| 17 | +.. _offline_mode: |
| 18 | + |
| 19 | +Offline Mode and HTTP Caching |
| 20 | +============================= |
| 21 | + |
| 22 | +To resolve remote resources — JSON-LD ``@context`` documents, profile artifacts |
| 23 | +and, optionally, remote RO-Crates — the validator performs HTTP requests. These |
| 24 | +requests go through a **persistent HTTP cache**, which makes validation faster |
| 25 | +and reproducible and enables an **offline mode** where requests are served |
| 26 | +exclusively from the cache. |
| 27 | + |
| 28 | +This page covers how to use offline mode and manage the cache both from the |
| 29 | +:ref:`command line <offline_mode_cli>` and through the |
| 30 | +:ref:`Python API <offline_mode_api>`. |
| 31 | + |
| 32 | + |
| 33 | +How caching works |
| 34 | +----------------- |
| 35 | + |
| 36 | +Every HTTP-backed resource fetched during validation is stored in a persistent |
| 37 | +cache (by default under the user cache directory, shared across runs). On the |
| 38 | +first online validation against a profile, the resources it declares are cached |
| 39 | +automatically, so a later run can reuse the same cache without any network |
| 40 | +access. |
| 41 | + |
| 42 | +Offline mode (``--offline`` / ``offline=True``) forbids network access |
| 43 | +altogether: every request must be satisfied by the cache, otherwise the affected |
| 44 | +resource is reported as a cache miss. For this reason offline mode requires the |
| 45 | +cache to be enabled and cannot be combined with the cache-disabling options. |
| 46 | + |
| 47 | + |
| 48 | +.. _offline_mode_cli: |
| 49 | + |
| 50 | +Command-line usage |
| 51 | +------------------ |
| 52 | + |
| 53 | +Offline validation |
| 54 | +~~~~~~~~~~~~~~~~~~~ |
| 55 | + |
| 56 | +Pass ``--offline`` to the ``validate`` command to forbid any network access: |
| 57 | +every HTTP request must then be satisfied by the cache. |
| 58 | + |
| 59 | +.. code-block:: bash |
| 60 | +
|
| 61 | + rocrate-validator validate --offline path/to/ro-crate |
| 62 | +
|
| 63 | +Related options: |
| 64 | + |
| 65 | +- ``--cache-path PATH`` — use a specific cache directory. By default a persistent |
| 66 | + directory under the user cache dir is used, so entries are shared across runs. |
| 67 | +- ``--cache-max-age SECONDS`` — maximum age of cached entries; ``-1`` (the |
| 68 | + default) means entries never expire. |
| 69 | +- ``--no-cache`` / ``-nc`` — disable the cache entirely: every request hits the |
| 70 | + network and nothing is persisted. This flag is **mutually exclusive** with |
| 71 | + ``--offline``, since offline mode needs the cache to serve requests. |
| 72 | + |
| 73 | +Managing the cache |
| 74 | +~~~~~~~~~~~~~~~~~~ |
| 75 | + |
| 76 | +The ``cache`` subcommand inspects and manages the HTTP cache: |
| 77 | + |
| 78 | +.. code-block:: bash |
| 79 | +
|
| 80 | + # Show the cache location, backend, size and offline status |
| 81 | + rocrate-validator cache info |
| 82 | +
|
| 83 | + # List cached entries (alias: `ls`); filter, sort or emit JSON |
| 84 | + rocrate-validator cache list |
| 85 | + rocrate-validator cache list --url w3id.org --sort size |
| 86 | + rocrate-validator cache list --json |
| 87 | +
|
| 88 | + # Remove every cached entry |
| 89 | + rocrate-validator cache reset --yes |
| 90 | +
|
| 91 | +Pre-populating the cache (warm-up) |
| 92 | +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ |
| 93 | + |
| 94 | +Before going offline you can pre-fetch everything you will need with |
| 95 | +``cache warm``: |
| 96 | + |
| 97 | +.. code-block:: bash |
| 98 | +
|
| 99 | + # Warm the resources declared by every installed profile |
| 100 | + rocrate-validator cache warm --all-profiles |
| 101 | +
|
| 102 | + # Warm only specific profiles |
| 103 | + rocrate-validator cache warm -p ro-crate-1.1 -p workflow-ro-crate-1.0 |
| 104 | +
|
| 105 | + # Also fetch and cache remote RO-Crates or arbitrary URLs |
| 106 | + rocrate-validator cache warm --crate https://example.org/crate.zip |
| 107 | + rocrate-validator cache warm -u https://w3id.org/ro/crate/1.1/context |
| 108 | +
|
| 109 | +When invoked without any source option, ``cache warm`` defaults to warming all |
| 110 | +installed profiles. A summary table reports which URLs were cached, skipped or |
| 111 | +failed; the command exits with a non-zero status if any URL fails. |
| 112 | + |
| 113 | + |
| 114 | +.. _offline_mode_api: |
| 115 | + |
| 116 | +Programmatic usage |
| 117 | +------------------ |
| 118 | + |
| 119 | +The same offline behaviour can be enabled programmatically through |
| 120 | +``ValidationSettings``: |
| 121 | + |
| 122 | +.. code-block:: python |
| 123 | +
|
| 124 | + from rocrate_validator import services, models |
| 125 | +
|
| 126 | + settings = services.ValidationSettings( |
| 127 | + rocrate_uri='/path/to/ro-crate', |
| 128 | + profile_identifier='ro-crate-1.1', |
| 129 | + # Serve every HTTP request from the cache; uncached resources fail. |
| 130 | + offline=True, |
| 131 | + # Optional: use a dedicated cache directory (defaults to the user cache). |
| 132 | + # cache_path='/tmp/rocv-cache', |
| 133 | + # Optional: maximum age of cached entries; -1 (default) = never expire. |
| 134 | + # cache_max_age=-1, |
| 135 | + ) |
| 136 | +
|
| 137 | + result = services.validate(settings) |
| 138 | +
|
| 139 | +The cache-related settings are: |
| 140 | + |
| 141 | +- ``offline`` (``bool``, default ``False``) — when ``True``, HTTP requests are |
| 142 | + served only from the cache; uncached resources raise a cache-miss error. |
| 143 | +- ``no_cache`` (``bool``, default ``False``) — disable the cache entirely. It is |
| 144 | + **incompatible** with ``offline=True`` and raises ``ValueError`` if combined. |
| 145 | +- ``cache_path`` (``Path``, optional) — cache directory; defaults to the |
| 146 | + persistent user cache so online and offline runs share the same entries. |
| 147 | +- ``cache_max_age`` (``int``, optional) — maximum entry age in seconds; ``-1`` |
| 148 | + means entries never expire. |
| 149 | + |
| 150 | +When ``offline`` is ``False``, the resources declared by the selected profiles |
| 151 | +are warmed up automatically before validation, so that a later offline run |
| 152 | +reusing the same cache succeeds without network access. To pre-populate the cache |
| 153 | +explicitly (e.g. in a CI pipeline), use the ``rocrate-validator cache warm`` |
| 154 | +command described in :ref:`offline_mode_cli`. |
0 commit comments