You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: docs/installation.md
+25-14Lines changed: 25 additions & 14 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -1,12 +1,12 @@
1
1
# Installation & Setup
2
2
3
-
The service runs with Docker Compose. The default setup validates metadata documents only; enabling object storage adds validation of complete stored crates. If you are upgrading from a 1.\* release, see the [upgrade guide](upgrading.md) first.
3
+
The RO-Crate Validation Service works in two ways: a metadata-only mode, in which the contents of an `ro-crate-metadata.json` file are assessed, and storage-backed validation, where complete RO-Crates (zip or directory) are evaluated.
4
4
5
5
## Quick start
6
6
7
7
You will need Docker with Docker Compose.
8
8
9
-
First, clone the repository, copy the example environment file, and start the stack:
9
+
To start, clone the repository, copy the example environment file, and start the stack:
You should update the default `.env` values when running the object store in production.
19
+
Remember to update the default `.env` values when running the object store in production.
20
20
21
-
The API is served at `http://localhost:5001`. Redis and a Celery worker are also started, but are only used once storage is enabled. To check the service is up:
21
+
The API is served at `http://localhost:5001`. Redis and a Celery worker are also started, but these are only used once storage is enabled.
22
+
23
+
To check the service is up, run:
22
24
23
25
```bash
24
26
curl http://localhost:5001/healthz
25
27
```
26
28
27
29
This returns `{"status": "ok"}`.
28
30
29
-
To validate some metadata, post the contents of an `ro-crate-metadata.json` file to the metadata endpoint. The [running example](https://www.researchobject.org/ro-crate/specification/1.2/introduction.html#running-example) from the RO-Crate specification is a good test document. The file needs to be sent as an escaped JSON string, which `jq` can do:
31
+
To validate the contents of an `ro-crate-metadata.json` file, post to the metadata endpoint. The [running example](https://www.researchobject.org/ro-crate/specification/1.2/introduction.html#running-example) from the RO-Crate specification is a good test document.
32
+
33
+
The file needs to be sent as an escaped JSON string, which `jq` can do:
The response contains a `status` of `valid`, `invalid` or `error`, along with the detailed findings. The[API reference](api.md) describes the endpoints and result format in full.
39
+
The response contains a `status` of `valid`, `invalid` or `error`, along with the detailed validation. For more information, the[API reference](api.md) describes the endpoints and result format in full.
36
40
37
41
## Enabling object storage
38
42
39
-
To validate a complete RO-Crate (zip or directory) held in an object store, set `STORAGE_ENABLED=true` in `.env`. Storage mode requires six settings, [described below](#configuration-reference): `S3_ENDPOINT`, `S3_ACCESS_KEY`, `S3_SECRET_KEY`, `S3_BUCKET`, `CELERY_BROKER_URL` and `CELERY_RESULT_BACKEND`. The service will fail at startup if any are missing. The Compose stack already sets the Celery variables to the bundled Redis, so in practice only the S3 settings in `.env` matter here.
43
+
To validate a complete RO-Crate (zip or directory) held in an object store, set `STORAGE_ENABLED=true` in `.env`.
44
+
45
+
The storage-backed validation mode requires six settings, [described below](#configuration-reference): `S3_ENDPOINT`, `S3_ACCESS_KEY`, `S3_SECRET_KEY`, `S3_BUCKET`, `CELERY_BROKER_URL` and `CELERY_RESULT_BACKEND`.
40
46
41
-
Then start the stack with the bundled development object store (RustFS):
47
+
The service will fail at startup if any are missing. The Compose stack already sets the Celery variables to the bundled Redis, so in practice only the S3 settings in `.env` matter here.
48
+
49
+
Then start the stack with the bundled development object store (RustFS), run:
42
50
43
51
```bash
44
52
docker compose --profile objectstore up --build
@@ -47,9 +55,9 @@ docker compose --profile objectstore up --build
47
55
RustFS serves the S3 API on port 9000 and a web console at `http://localhost:9001`. Development credentials are set in `example.env`.
48
56
49
57
!!! warning
50
-
The service does not create the bucket itself. Create the bucket in the console or with any S3 client. The bucket name needs to match `S3_BUCKET` (`ro-crates` by default).
58
+
The service does not create the bucket itself. Create the bucket in the console or with an S3 client. The bucket name needs to match `S3_BUCKET` (`ro-crates` by default).
51
59
52
-
Upload an RO-Crate under the crate prefix: `crates/<id>.zip` for a zipped RO-Crate, or `crates/<id>/` for a directory. Note that for a zipped RO-Crate, `ro-crate-metadata.json` must be at the root of the archive.
60
+
Upload an RO-Crate under the prefix: `crates/<id>.zip` for a zipped RO-Crate, or `crates/<id>/` for a directory. Note that for a zipped RO-Crate, `ro-crate-metadata.json` must be at the root of the archive.
53
61
54
62
The readiness endpoint checks the object store and broker connections:
Any S3-compatible store can be used in place of RustFS, including AWS S3, MinIO and Ceph: set `S3_ENDPOINT`, the credentials and `S3_BUCKET` for your store, and skip the `objectstore` profile. If you already run a 1.\* release against MinIO, the [upgrade guide](upgrading.md) maps the old settings to the new ones.
70
+
Any S3-compatible store can be used in place of RustFS, including AWS S3, MinIO and Ceph: set `S3_ENDPOINT`, the credentials and `S3_BUCKET` for your store. In this case, do not run the `objectstore` profile.
71
+
72
+
If you already use a 1.\* release against MinIO, the [upgrade guide](upgrading.md) maps the old settings to the new ones.
63
73
64
74
## Configuration reference
65
75
@@ -84,11 +94,12 @@ Any S3-compatible store can be used in place of RustFS, including AWS S3, MinIO
84
94
85
95
## Custom profiles
86
96
87
-
The validator comes with the base RO-Crate profiles. For Five Safes validation, the prebuilt `ghcr.io/esciencelab/ro-crate-validation-service-fivesafes-profile` image has the `five-safes-crate` profile already included; see [Five Safes validation](five-safes.md).
97
+
The validator comes with several RO-Crate profiles, and for the Five Safes RO-Crate, the prebuilt `ghcr.io/esciencelab/ro-crate-validation-service-fivesafes-profile` image has the `five-safes-crate` profile already included; see [Five Safes validation](five-safes.md).
88
98
89
-
Adding other profiles works by mounting a directory of profile definitions into both the `flask` and `celery_worker` containers. You set `EXTRA_PROFILES_PATH` to the mounted path. Both containers need the mount as metadata-only validation runs in the API process and stored-crate validation runs in the worker. There is a working example in `docker-compose-develop.yml`.
99
+
To add other profiles, mount a directory into both the `flask` and `celery_worker` containers, and set `EXTRA_PROFILES_PATH` to the mounted path. Note that both containers need the mount as metadata-only validation runs in the API process and stored-crate validation runs in the worker. There is a working example in `docker-compose-develop.yml`.
90
100
91
-
`EXTRA_PROFILES_PATH` adds the directory to the bundled profiles, whereas `PROFILES_PATH` replaces them entirely.
101
+
!!! note
102
+
`EXTRA_PROFILES_PATH` adds the directory to the bundled profiles, whereas `PROFILES_PATH` replaces them entirely.
0 commit comments