Skip to content

Commit f157091

Browse files
committed
improve docs
1 parent ac34d48 commit f157091

1 file changed

Lines changed: 25 additions & 6 deletions

File tree

docs/persistentStorage.md

Lines changed: 25 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -6,9 +6,10 @@ This document describes Ocean Node **Persistent Storage** at a high level: what
66

77
## What it is
88

9-
Persistent Storage is a simple bucket + file store intended for **long-lived artifacts** that Ocean Node needs to keep across requests (and potentially across restarts), and to reference later (e.g. as file objects for compute).
9+
Persistent Storage is a simple bucket + file store intended for **long-lived artifacts** that Ocean Node needs to keep across requests and across restarts, and to reference later (e.g. as file objects for compute).
1010

1111
Key primitives:
12+
1213
- **Bucket**: a logical container for files.
1314
- **File**: binary content stored inside a bucket.
1415
- **Bucket registry**: a local SQLite table that stores bucket metadata (owner, access lists, createdAt).
@@ -35,7 +36,8 @@ Key primitives:
3536

3637
Persistent Storage uses two stores:
3738

38-
1) **Bucket registry (SQLite)**
39+
1. **Bucket registry (SQLite)**
40+
3941
- File: `databases/persistentStorage.sqlite`
4042
- Table: `persistent_storage_buckets`
4143
- Columns:
@@ -44,7 +46,8 @@ Persistent Storage uses two stores:
4446
- `accessListJson` (JSON-encoded access list array)
4547
- `createdAt` (unix timestamp)
4648

47-
2) **Backend data**
49+
2. **Backend data**
50+
4851
- `localfs`: writes file bytes to the configured folder under `buckets/<bucketId>/<fileName>`.
4952
- `s3`: not implemented yet.
5053

@@ -75,11 +78,13 @@ This access list is used to decide whether a given `consumerAddress` is allowed
7578

7679
Access checks happen at two levels:
7780

78-
1) **Backend enforcement** (required)
81+
1. **Backend enforcement** (required)
82+
7983
- Backend operations `listFiles`, `uploadFile`, `deleteFile`, and `getFileObject` all require `consumerAddress`.
8084
- The base class helper `assertConsumerAllowedForBucket(consumerAddress, bucketId)` loads the bucket ACL and throws `PersistentStorageAccessDeniedError` if the consumer is not allowed.
8185

82-
2) **Handler enforcement** (command-specific)
86+
2. **Handler enforcement** (command-specific)
87+
8388
- `createBucket`: additionally checks the node-level allow list `config.persistentStorage.accessLists` (who can create buckets at all).
8489
- `getBuckets`: queries registry rows filtered by `owner` and then:
8590
- if `consumerAddress === owner`: returns all buckets for that owner
@@ -115,6 +120,10 @@ Access checks happen at two levels:
115120
- Deletes the named file from the bucket.
116121
- Enforces bucket ACL.
117122

123+
- **getFileObject**
124+
- Returns fileObject format for c2d use
125+
- Enforces bucket ACL.
126+
118127
### Not implemented yet
119128

120129
- **S3 backend**
@@ -127,6 +136,7 @@ Access checks happen at two levels:
127136
Persistent storage is controlled by `persistentStorage` in node config.
128137

129138
Key fields:
139+
130140
- `enabled`: boolean
131141
- `type`: `"localfs"` or `"s3"`
132142
- `accessLists`: AccessList[] — node-level allow list to create buckets
@@ -138,9 +148,18 @@ Key fields:
138148

139149
## Usage
140150

151+
Flow is:
152+
153+
- create bucket (or use existing bucket)
154+
- list files
155+
- upload file if needed
156+
- GetFileObject to get object needed for c2d reference
157+
- start c2d job using fileObject for datasets
158+
141159
### P2P commands
142160

143161
All persistent storage operations are implemented as protocol commands in the handler:
162+
144163
- `persistentStorageCreateBucket`
145164
- `persistentStorageGetBuckets`
146165
- `persistentStorageListFiles`
@@ -155,6 +174,7 @@ Each command requires authentication (token or signature) based on Ocean Node’
155174
HTTP routes are available under `/api/services/persistentStorage/...` and call the same handlers. See `docs/API.md` for the full parameter lists and examples.
156175

157176
At a glance:
177+
158178
- `POST /api/services/persistentStorage/buckets`
159179
- `GET /api/services/persistentStorage/buckets`
160180
- `GET /api/services/persistentStorage/buckets/:bucketId/files`
@@ -171,4 +191,3 @@ Upload uses the raw request body as bytes and forwards it to the handler as a st
171191
- The bucket registry is local to the node (SQLite file). If you run multiple nodes, each node’s registry is independent unless you externalize/replicate it.
172192
- `listBuckets(owner)` requires `owner` and only returns buckets that were created with that owner recorded.
173193
- Filenames in `localfs` are constrained (no path separators) to avoid path traversal.
174-

0 commit comments

Comments
 (0)