Skip to content

Commit 0cca789

Browse files
alexcos20alexcos20
authored
document publish flow & isolated markets (#1273)
1 parent 039257c commit 0cca789

1 file changed

Lines changed: 158 additions & 0 deletions

File tree

‎docs/Publishing.md‎

Lines changed: 158 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,158 @@
1+
# Publishing assets and isolated markets
2+
3+
This document describes how assets (DDOs) move from a publisher’s machine into the Ocean network, how Ocean Nodes index them, and how you can restrict **who indexes what** to build **isolated markets**—logical subsets of the public network that share the same chain but apply stricter rules on the node side.
4+
5+
---
6+
7+
## The default (public) flow
8+
9+
In the usual setup, every Ocean Node that follows the network is free to index every asset that appears on chain, subject to normal validation (factory deployment, hashes, optional Policy Server hooks on the indexer, etc.).
10+
11+
1. **Build the DDO locally**
12+
You construct a valid DDO with [ocean.js](https://github.com/oceanprotocol/ocean.js) (or equivalent tooling).
13+
14+
2. **Optional: validate against one or more nodes**
15+
Before publishing, you can call the **`validateDDO`** command on a node (HTTP `POST /directCommand` or P2P). The node checks the DDO shape and rules; if [Policy Server](./PolicyServer.md) is configured, it can run extra authorization checks. On success, the node may return a **validation signature** payload (see [API](./API.md#validate-ddo)) used when you publish so the chain can record **which validator** approved the metadata.
16+
17+
3. **Publish on chain**
18+
You submit the metadata transaction (create/update) so the asset is anchored on the target chain.
19+
20+
4. **Indexing**
21+
Every node that indexes that chain processes the relevant events and stores the DDO in its local index (e.g. Typesense/Elasticsearch), so the asset becomes discoverable through those nodes’ APIs.
22+
23+
Nothing in this flow *requires* an allowlist: if your nodes do not set the isolation options below, they behave as part of the open network.
24+
25+
```mermaid
26+
flowchart LR
27+
A[Build DDO] --> B[Optional validateDDO]
28+
B --> C[Publish on chain]
29+
C --> D[Nodes index events]
30+
D --> E[Query / consume via nodes]
31+
```
32+
33+
---
34+
35+
## Isolated markets
36+
37+
An **isolated market** is not a separate blockchain; it is a **policy** you enforce by configuring groups of Ocean Nodes so they only index (and thus only advertise) assets that satisfy your rules. You can combine several mechanisms.
38+
39+
| Mechanism | What it restricts | Typical use |
40+
|-----------|-------------------|-------------|
41+
| `ALLOWED_VALIDATORS` / `ALLOWED_VALIDATORS_LIST` | Only assets whose publishing tx includes approval from **allowed validator addresses** | Org-approved catalog |
42+
| `AUTHORIZED_PUBLISHERS` / `AUTHORIZED_PUBLISHERS_LIST` | Only assets whose **on-chain owner** is allowlisted | Known publishers only |
43+
| `AUTHORIZED_DECRYPTERS` / `AUTHORIZED_DECRYPTERS_LIST` | Which **nodes** may call **`decryptDDO`** on the node that encrypted metadata | Encrypted DDO + indexer allowlist |
44+
45+
Environment variable names and JSON shapes are documented in [env.md](./env.md).
46+
47+
---
48+
49+
## Scenario 1 — Validator-gated indexing (`ALLOWED_VALIDATORS`)
50+
51+
**Goal:** Only index assets that were **validated by a specific node** (or set of nodes), e.g. an organization that runs a “gatekeeper” node with extra checks.
52+
53+
**How it works**
54+
55+
1. **Gatekeeper node**
56+
Run a node (`node1`) that performs `validateDDO`. Optionally connect it to a **Policy Server** so enterprise rules (SSO, LDAP, VC checks, etc.) run before the node signs—see [PolicyServer.md](./PolicyServer.md).
57+
58+
2. **Validation signature**
59+
When validation succeeds, the node returns signing material (the node signs a hash derived from the DDO using its **provider wallet**). Your client must pass this through **ocean.js** when publishing so the protocol emits **`MetadataValidated`** in the **same transaction** as metadata creation/update. Indexers look for those events and read **validator addresses** from them.
60+
61+
3. **Isolated indexers**
62+
On every node that should **only** follow that gatekeeper, set:
63+
64+
```bash
65+
ALLOWED_VALIDATORS='["0xGatekeeperNodeProviderAddress"]'
66+
```
67+
68+
Use the **Ethereum address** of the gatekeeper node’s signing key (the same key used when producing the validation signature). You can list multiple addresses.
69+
70+
4. **Access lists (optional)**
71+
`ALLOWED_VALIDATORS_LIST` maps **chain ID → AccessList contract addresses**. If set, at least one on-chain validator must appear on those lists (see implementation and [env.md](./env.md)).
72+
73+
**Outcome:** Nodes with `ALLOWED_VALIDATORS` set **skip** metadata events whose transaction does not include a `MetadataValidated` proof, or where none of the validators match your allowlist. Other nodes on the network—without this setting—still index everything as usual.
74+
75+
---
76+
77+
## Scenario 2 — Publisher allowlists (`AUTHORIZED_PUBLISHERS`)
78+
79+
**Goal:** Index only assets published by **specific wallets**, regardless of which validator signed (or use this together with Scenario 1).
80+
81+
**How it works**
82+
83+
- The indexer checks the **Data NFT owner** (`owner` from the metadata event) against your configuration.
84+
- If `AUTHORIZED_PUBLISHERS` is non-empty, the owner must correspond to **one of the listed addresses** (compared case-insensitively against the configured list).
85+
- If `AUTHORIZED_PUBLISHERS_LIST` is set, the owner must satisfy the **AccessList** contracts for that chain.
86+
87+
**Example (conceptual)**
88+
89+
```bash
90+
# Only these publishers’ assets are indexed
91+
AUTHORIZED_PUBLISHERS='["0xPublisherA...","0xPublisherB..."]'
92+
```
93+
94+
**Combining with validators:** Set both `ALLOWED_VALIDATORS` and `AUTHORIZED_PUBLISHERS`. An asset is indexed only if it passes **both** checks.
95+
96+
---
97+
98+
## Scenario 3 — Encrypted DDO and decrypt allowlists (`AUTHORIZED_DECRYPTERS`)
99+
100+
**Goal:** At publish time, pick **one node** to encrypt the DDO; only **authorized** other nodes can decrypt it during indexing, so only those nodes can build a full index entry.
101+
102+
**How it works**
103+
104+
1. **Publishing**
105+
When encrypting metadata for chain storage, you choose a node (often via its HTTP URL or peer id, depending on client flow). That node’s **`encrypt`** path prepares ciphertext that other parties cannot read without going through **`decryptDDO`**.
106+
107+
2. **On-chain reference**
108+
The encrypted payload is stored/ referenced such that indexers know **which node** can decrypt (HTTP URL of the encrypting node, or the node’s peer id for local decrypt).
109+
110+
3. **Indexing and `decryptDDO`**
111+
When another node indexes the asset, it must call the encrypting node’s **`POST /api/services/decrypt`** (with a nonce and signature proving the caller). The decrypting node checks whether the requester’s **`decrypterAddress`** (the indexing node’s own Ethereum address) is allowed.
112+
113+
4. **`AUTHORIZED_DECRYPTERS` on the encrypting node**
114+
On the node that holds the decryption keys, set:
115+
116+
```bash
117+
AUTHORIZED_DECRYPTERS='["0xIndexerNode1...","0xIndexerNode2..."]'
118+
```
119+
120+
If this list is **non-empty**, only those addresses **or** the encrypting node itself may decrypt. Everyone else receives **403** and **cannot** complete indexing for that ciphertext.
121+
122+
5. **Access lists (optional)**
123+
`AUTHORIZED_DECRYPTERS_LIST` restricts callers via **AccessList** contracts per chain (see [env.md](./env.md)).
124+
125+
**Outcome:** The asset may exist on chain for everyone, but **only nodes you list** can successfully decrypt and index it. Others fail at decrypt and leave the asset out of their index.
126+
127+
---
128+
129+
## Combining scenarios (example)
130+
131+
A private catalog might use:
132+
133+
- **`ALLOWED_VALIDATORS`:** only your org’s gatekeeper node address.
134+
- **`AUTHORIZED_PUBLISHERS`:** only approved data-owner wallets.
135+
- **Encrypted DDO + `AUTHORIZED_DECRYPTERS`:** only your federation’s indexer node addresses can call `decrypt` on the encryption node.
136+
137+
Tune each layer to match how much you trust **validators**, **publishers**, and **indexer machines**.
138+
139+
---
140+
141+
## Other useful knobs
142+
143+
- **`INDEXER_NETWORKS`:** Limit which chains a node indexes (see [env.md](./env.md)).
144+
- **Policy Server on the indexer:** Even in public mode, your indexer can call Policy Server on **`newDDO`** / **`updateDDO`** to reject indexing—orthogonal to the allowlists above but complementary for org-wide policy.
145+
- **`VALIDATE_UNSIGNED_DDO`:** Controls whether `validateDDO` requires publisher proofs before signing; relevant when hardening validation flows (see [env.md](./env.md)).
146+
147+
---
148+
149+
## Where to read next
150+
151+
| Topic | Doc |
152+
|--------|-----|
153+
| All env vars (`ALLOWED_*`, `AUTHORIZED_*`, lists) | [env.md](./env.md) |
154+
| Policy Server actions (`validateDDO`, `newDDO`, …) | [PolicyServer.md](./PolicyServer.md) |
155+
| `validateDDO` command shape | [API.md](./API.md#validate-ddo) |
156+
| Key / signing model | [KeyManager.md](./KeyManager.md) |
157+
158+
For client-side steps (exact ocean.js calls to pass validation into `MetadataValidated` and to choose an encrypting node), refer to the **ocean.js** documentation and examples for your stack version; the on-chain requirement from the node’s perspective is **`MetadataValidated` in the publishing transaction** when using `ALLOWED_VALIDATORS`, and successful **`/api/services/decrypt`** when metadata is encrypted.

0 commit comments

Comments
 (0)