Skip to content

Commit 20ef694

Browse files
merge with master
2 parents 20a685e + 016fdd9 commit 20ef694

13 files changed

Lines changed: 427 additions & 67 deletions

File tree

docs/src/admin/thesauri/thesauri.md

Lines changed: 62 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -92,6 +92,7 @@ GeoNode provides a single command (``thesaurus``) with multiple actions:
9292
* ``list``: list existing thesauri
9393
* ``load``: load a RDF file
9494
* ``dump``: dump a thesaurus into a file
95+
* ``autoload``: automatically discover and load all thesauri shipped by installed apps
9596

9697
.. code-block::
9798

@@ -102,12 +103,13 @@ GeoNode provides a single command (``thesaurus``) with multiple actions:
102103
[--format {json-ld,n3,nt,pretty-xml,sorted-xml,trig,ttl,xml}] [--default-lang LANG] [--version]
103104
[-v {0,1,2,3}] [--settings SETTINGS] [--pythonpath PYTHONPATH] [--traceback] [--no-color]
104105
[--force-color] [--skip-checks]
105-
[{list,load,dump}]
106+
[{list,load,dump,autoload}]
106107

107-
Handles thesaurus commands ['list', 'load', 'dump']
108+
Handles thesaurus commands ['list', 'load', 'dump', 'autoload']
108109

109110
positional arguments:
110-
{list,load,dump} thesaurus operation to run
111+
{list,load,dump,autoload}
112+
thesaurus operation to run
111113

112114
options:
113115
-h, --help show this help message and exit
@@ -227,6 +229,63 @@ In order to only export the entries we edited, we'll issue the command::
227229
python manage.py thesaurus dump -i labels-i18n --include "proj1_*" --include "*_ovr" -f labels-i18n.proj1.rdf
228230

229231

232+
### Auto-loading thesauri: ``thesaurus autoload``
233+
234+
The ``autoload`` subcommand scans every installed Django app for a ``thesauri/`` directory
235+
at the top level of the app package, then loads all ``.rdf`` files it finds there.
236+
This is how GeoNode and third-party apps can ship thesauri that are loaded automatically at start-up.
237+
238+
```bash
239+
python manage.py thesaurus autoload
240+
```
241+
242+
For each ``.rdf`` file discovered, the command runs the equivalent of ``thesaurus load --action update``,
243+
so the operation is **idempotent**: running it multiple times will not create duplicates; instead,
244+
existing records are updated and missing ones are created.
245+
246+
**Convention for app-provided thesauri**
247+
248+
Place one or more ``.rdf`` files inside a ``thesauri/`` directory at the root of your app package:
249+
250+
```
251+
my_geonode_app/
252+
thesauri/
253+
my_vocabulary.rdf
254+
another_vocab.rdf
255+
models.py
256+
...
257+
```
258+
259+
All ``.rdf`` files in that directory are picked up automatically whenever ``thesaurus autoload``
260+
(or ``invoke loadthesauri``) is executed.
261+
262+
!!! note
263+
The ``autoload`` command is automatically run during GeoNode's Docker container start-up sequence (see [Initialization at boot](#initialization-at-boot)).
264+
265+
266+
## Initialization at boot { #initialization-at-boot }
267+
268+
When GeoNode starts (e.g. via the Docker entrypoint), the following initialization steps are executed in order:
269+
270+
1. **Database migrations** – applies any pending schema migrations.
271+
2. **Fixtures** – loads default OAuth2 apps, admin user, and site data (only on first boot or when ``FORCE_REINIT=true``).
272+
3. **Static files** – collects static assets.
273+
4. **Thesauri autoload** – runs ``thesaurus autoload`` to load or update all ``.rdf`` files found in any installed app's ``thesauri/`` directory. This step runs on **every** boot so that thesaurus updates shipped with an upgraded app are applied automatically.
274+
275+
To run the thesaurus autoload step manually:
276+
277+
```bash
278+
# Inside the GeoNode container
279+
python manage.py thesaurus autoload
280+
```
281+
282+
Or using the invoke task:
283+
284+
```bash
285+
invoke loadthesauri
286+
```
287+
288+
230289
## Configuring a Thesaurus
231290

232291

docs/src/setup/docker/vanilla-docker-installation.md

Lines changed: 9 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -81,6 +81,15 @@ Executing UWSGI server uwsgi --ini /usr/src/app/uwsgi.ini for Production
8181
[uWSGI] getting INI configuration from /usr/src/app/uwsgi.ini
8282
```
8383

84+
The container performs these initialization steps before starting the application server:
85+
86+
1. **Database migrations** – applies any pending schema migrations.
87+
2. **Fixtures** – loads default OAuth2 apps, admin user and site data (only on first boot or when ``FORCE_REINIT=true``).
88+
3. **Static files** – collects static assets.
89+
4. **Thesauri autoload** – scans all installed apps for a ``thesauri/`` directory and loads (or updates) any ``.rdf`` files found there. This makes sure thesauri shipped by GeoNode apps are always up-to-date.
90+
91+
See [Thesauri – Initialization at boot](../../../admin/thesauri/thesauri.md#initialization-at-boot) for more details on the thesaurus autoload step.
92+
8493
To exit just hit `CTRL+C`.
8594

8695
This message means that the GeoNode containers have been started. Browsing to `http://localhost/` will show the GeoNode home page. You should be able to successfully log with the credentials of admin user which are defined in the .env file and start using it right away.

entrypoint.sh

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -56,6 +56,7 @@ else
5656
fi
5757

5858
invoke statics
59+
invoke loadthesauri
5960

6061
echo "Executing UWSGI server $cmd for Production"
6162
fi

geonode/base/api/exceptions.py

Lines changed: 12 additions & 8 deletions
Original file line numberDiff line numberDiff line change
@@ -32,14 +32,18 @@ def geonode_exception_handler(exc, context):
3232
# to get the standard error response.
3333
response = exception_handler(exc, context)
3434

35-
if response is not None and isinstance(exc, APIException):
36-
# for the upload exception we need a custom response
37-
detail = _extract_detail(exc)
38-
response.data = {
39-
"success": False,
40-
"errors": [str(detail)],
41-
"code": exc.code if hasattr(exc, "code") else exc.default_code,
42-
}
35+
if response is not None:
36+
if response.status_code == 401:
37+
response.headers.pop("WWW-Authenticate", None)
38+
39+
if isinstance(exc, APIException):
40+
# for the upload exception we need a custom response
41+
detail = _extract_detail(exc)
42+
response.data = {
43+
"success": False,
44+
"errors": [str(detail)],
45+
"code": exc.code if hasattr(exc, "code") else exc.default_code,
46+
}
4347
return response
4448

4549

geonode/base/api/tests.py

Lines changed: 15 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -3742,6 +3742,21 @@ def test_metadata_uploaded_preserve_can_be_updated(self):
37423742
self.assertTrue(doc.metadata_uploaded_preserve)
37433743
self.assertTrue(response.json()["resource"]["metadata_uploaded_preserve"])
37443744

3745+
def test_www_authenticate_header_is_removed_for_401_responses(self):
3746+
"""
3747+
Ensure WWW-Authenticate header is removed for 401 responses to prevent browsers from showing a login prompt
3748+
"""
3749+
try:
3750+
user = get_user_model().objects.create_user(
3751+
username="user_test_delete", email="user_test_delete@geonode.org", password="user"
3752+
)
3753+
url = reverse("users-detail", kwargs={"pk": user.pk})
3754+
# Anonymous can't read
3755+
response = self.client.get(url, format="json")
3756+
self.assertNotIn("WWW-Authenticate", response.headers)
3757+
finally:
3758+
user.delete()
3759+
37453760

37463761
class TestExtraMetadataBaseApi(GeoNodeBaseTestSupport):
37473762
def setUp(self):

geonode/base/management/commands/thesaurus.py

Lines changed: 15 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -2,6 +2,7 @@
22
from django.core.management.base import BaseCommand, CommandError
33

44
from geonode.base.management.command_utils import setup_logger
5+
from geonode.base.management.commands.thesaurus_subcommands.autoload import autoload_thesauri
56
from geonode.base.management.commands.thesaurus_subcommands.dump import (
67
dump_thesaurus,
78
DUMP_FORMATS,
@@ -16,7 +17,8 @@
1617
COMMAND_LIST = "list"
1718
COMMAND_DUMP = "dump"
1819
COMMAND_LOAD = "load"
19-
COMMANDS = [COMMAND_LIST, COMMAND_LOAD, COMMAND_DUMP]
20+
COMMAND_AUTOLOAD = "autoload"
21+
COMMANDS = [COMMAND_LIST, COMMAND_LOAD, COMMAND_DUMP, COMMAND_AUTOLOAD]
2022

2123

2224
class Command(BaseCommand):
@@ -41,6 +43,12 @@ def add_arguments(self, parser):
4143
choices=ACTIONS,
4244
help="Actions to run upon data loading (default: create)",
4345
)
46+
load_group.add_argument(
47+
"--langs",
48+
dest="langs",
49+
action="append",
50+
help="Only import labels for the requested languages; can be repeated",
51+
)
4452

4553
dump_group = parser.add_argument_group('Params for "dump" subcommand')
4654
dump_group.add_argument("-o", "--out", nargs="?", help="Full path to the output file to be created")
@@ -99,6 +107,8 @@ def handle(self, *args, **options):
99107
input_file = options.get("file")
100108
action = options.get("action")
101109
identifier = options.get("identifier")
110+
lang = options.get("lang")
111+
langs = options.get("langs") or []
102112

103113
if not input_file:
104114
raise CommandError("'load' command requires the <file> parameter.")
@@ -107,7 +117,10 @@ def handle(self, *args, **options):
107117
action = ACTION_CREATE
108118
logger.info(f"Missing action param: setting actions as '{action}'")
109119

110-
load_thesaurus(input_file, identifier, action)
120+
load_thesaurus(input_file, identifier, action, default_lang=lang, langs=langs)
121+
122+
elif subcommand == COMMAND_AUTOLOAD:
123+
autoload_thesauri()
111124

112125
else:
113126
raise CommandError(f"Unknown subcommand: {subcommand}")
Lines changed: 39 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,39 @@
1+
import os
2+
3+
from django.apps import apps
4+
5+
from geonode.base.management.command_utils import setup_logger
6+
from geonode.base.management.commands.thesaurus_subcommands.load import load_thesaurus, ACTION_UPDATE
7+
8+
logger = setup_logger()
9+
10+
11+
def autoload_thesauri():
12+
"""
13+
Discover and load all thesauri (.rdf files) found in a `thesauri/` directory
14+
within each installed Django app. Uses the `update` action so existing entries
15+
are updated and new ones are created without duplicates.
16+
"""
17+
loaded = 0
18+
for app_config in apps.get_app_configs():
19+
thesauri_dir = os.path.join(app_config.path, "thesauri")
20+
logger.debug(f"Looking for auto thesaurus in app '{app_config.name}' path: {thesauri_dir}")
21+
if not os.path.isdir(thesauri_dir):
22+
continue
23+
try:
24+
rdf_files = [f for f in os.listdir(thesauri_dir) if f.lower().endswith(".rdf")]
25+
except OSError as e:
26+
logger.error(
27+
f"Failed to scan thesauri directory for app '{app_config.name}' at '{thesauri_dir}': {e}",
28+
exc_info=True,
29+
)
30+
continue
31+
for rdf_file in sorted(rdf_files):
32+
rdf_path = os.path.join(thesauri_dir, rdf_file)
33+
logger.info(f"Autoloading thesaurus from app '{app_config.name}': {rdf_path}")
34+
try:
35+
load_thesaurus(rdf_path, identifier=None, action=ACTION_UPDATE, log_details=False)
36+
loaded += 1
37+
except Exception as e:
38+
logger.error(f"Failed to load thesaurus '{rdf_path}': {e}", exc_info=True)
39+
logger.info(f"Autoload complete: {loaded} thesaurus file(s) loaded.")

0 commit comments

Comments
 (0)