Skip to content

Commit ffd4ed5

Browse files
zeljkoXzeljkoX
authored
feat: plat-6873 add storage documentation (#395)
* chore: initial * feat: add repos logic and config processing logic * chore: fixes * chore: imprve env sync_nonce and add service info log * chore: lint * chore: add notifications crud endpoints * chore: add notification request validations * chore: initial work for notificaitons crud * chore: fix openapi generation * chore: improvements * chore: improvements * chore: improvements * chore: impr * chore: intial signers crud impl * chore: improvements * chore: refactor * chore: impr * refactor + remove vault cloud due to deprecation of service * chore: impr * chore: fix clippy * chore: fixes * chore: add noboost comments * chore: typo * chore: stellar * chore: noboost * feat: plat-6845 implement relayer models * chore: improvements * chore: improvements * chore: impr * chore: implement merge patch logic for relayer * chore: improvements * chore: improvements * chore: impr * chore: clippy * chore: cleanup * chore: add controller tests * chore: more contrller tests * chore: resolve bug * chore: tests * chore: more tests * chore: fixes * chore: impr * chore: add nosemgrep rule * feat: securely store secrets in storage * chore: allow starting service with empty config * chore: improvements * chore: clippy * chore: pr suggestions * chore: impr * chore: improvements * chore: format * chore: improvements * feat: plat-6864 impre response schema and revert some model name changes to avoid sdk breaking changes * feat: add support for transaction expiration * chore: clippy * feat: plat-6873 add storage documentation * chore: PR suggestion * chore: leftover * chore: PR suggestion
1 parent cb5b2e7 commit ffd4ed5

10 files changed

Lines changed: 525 additions & 21 deletions

File tree

‎README.md‎

Lines changed: 6 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -53,11 +53,12 @@ The repository includes several ready-to-use examples to help you get started wi
5353
| Example | Description |
5454
| ------------------------------------------------------------ | ----------------------------------------- |
5555
| [`basic-example`](./examples/basic-example/) | Simple setup with Redis |
56+
| [`redis-storage`](./examples/redis-storage/) | Simple setup with Redis for storage |
5657
| [`basic-example-logging`](./examples/basic-example-logging/) | Configuration with file-based logging |
5758
| [`basic-example-metrics`](./examples/basic-example-metrics/) | Setup with Prometheus and Grafana metrics |
58-
| [`vault-secret-signer`](./examples/vault-secret-signer/) | Using HashiCorp Vault for key management |
59-
| [`vault-transit-signer`](./examples/vault-transit-signer/) | Using Vault Transit for secure signing |
60-
| [`evm-turnkey-signer`](./examples/evm-turnkey-signer/) | Using Turnkey Signer for EVM secure signing |
59+
| [`vault-secret-signer`](./examples/vault-secret-signer/) | Using HashiCorp Vault for key management |
60+
| [`vault-transit-signer`](./examples/vault-transit-signer/) | Using Vault Transit for secure signing |
61+
| [`evm-turnkey-signer`](./examples/evm-turnkey-signer/) | Using Turnkey Signer for EVM secure signing |
6162
| [`solana-turnkey-signer`](./examples/solana-turnkey-signer/) | Using Turnkey Signer for Solana secure signing |
6263
| [`solana-google-cloud-kms-signer`](./examples/solana-google-cloud-kms-signer/) | Using Google Cloud KMS Signer for Solana secure signing |
6364
| [`network-configuration-config-file`](./examples/network-configuration-config-file/) | Using Custom network configuration via config file |
@@ -286,6 +287,8 @@ Create `.env` with correct values according to your needs from `.env.example` fi
286287
cp .env.example .env
287288
```
288289

290+
> **Note**: After the service is running, all configuration components (relayers, signers, notifications) can also be managed via REST API endpoints for runtime changes. See the [Configuration Guide](https://docs.openzeppelin.com/relayer/configuration) for details on API-based configuration management.
291+
289292
### Creating a Signer
290293

291294
To create a new signer keystore, use the provided key generation tool:

‎docs/modules/ROOT/nav.adoc‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -3,6 +3,7 @@
33
* xref:configuration.adoc[Configuration]
44
** xref:signers.adoc[Signers]
55
** xref:network_configuration.adoc[Network Configuration]
6+
** xref:storage.adoc[Storage Configuration]
67
* xref:evm.adoc[EVM Integration]
78
* xref:solana.adoc[Solana Integration]
89
* xref:stellar.adoc[Stellar Integration]

‎docs/modules/ROOT/pages/configuration.adoc‎

Lines changed: 84 additions & 15 deletions
Original file line numberDiff line numberDiff line change
@@ -8,12 +8,18 @@ Please ensure appropriate access permissions on all configuration files (for `./
88

99
[IMPORTANT]
1010
====
11-
The configuration system consists of two main components:
11+
The OpenZeppelin Relayer supports two configuration approaches:
1212
13+
**File-based Configuration:**
1314
1. **`config.json`**: Contains relayer definitions, signer configurations, and network policies
1415
2. **`.env`** file: Contains environment variables like API keys and connection strings
1516
16-
Both files must be properly configured before starting the application. Changes to either file require restarting the container to take effect.
17+
**API-based Configuration:**
18+
- Full CRUD operations for relayers, signers, and notifications via REST API
19+
- Changes take effect immediately (no container restart required)
20+
- See the **API Reference** page for detailed endpoints documentation
21+
22+
See xref:storage.adoc[Storage Configuration] for detailed information about how file-based and API-based configurations work together, storage behavior, and best practices.
1723
1824
For quick setup examples with pre-configured files, see the https://github.com/OpenZeppelin/openzeppelin-relayer/tree/main/examples[examples directory] in our GitHub repository.
1925
====
@@ -40,6 +46,21 @@ This table lists the environment variables and their default values.
4046
| `info, debug, warn, error, trace`
4147
| Log level.
4248

49+
| `REPOSITORY_STORAGE_TYPE`
50+
| `in-memory`
51+
| `in-memory, redis`
52+
| Type of storage used for storing repository config and resources. See xref:storage.adoc[Storage Configuration] for detailed information.
53+
54+
| `RESET_STORAGE_ON_START`
55+
| `false`
56+
| `bool`
57+
| Clears all resources from storage on startup and reloads entries from the config file. See xref:storage.adoc[Storage Configuration] for usage details.
58+
59+
| `TRANSACTION_EXPIRATION_HOURS`
60+
| `4`
61+
| `number`
62+
| Number of hours after which transactions in a final state are removed from storage. See xref:storage.adoc[Storage Configuration] for more information.
63+
4364
| `CONFIG_DIR`
4465
| `./config`
4566
| `<any relative file path where config.json is located>`
@@ -98,18 +119,22 @@ This table lists the environment variables and their default values.
98119
| `REDIS_URL`
99120
| `redis://localhost:6379`
100121
| `<redis connection string>`
101-
| Redis connection URL for the relayer.
122+
| Redis connection URL for the relayer. See xref:storage.adoc[Storage Configuration] for Redis setup details.
102123

103124
| `REDIS_CONNECTION_TIMEOUT_MS`
104125
| `10000`
105126
| `<timeout in milliseconds>`
106-
| Connection timeout for Redis in milliseconds.
127+
| Connection timeout for Redis in milliseconds. See xref:storage.adoc[Storage Configuration] for Redis configuration.
107128

108129
| `REDIS_KEY_PREFIX`
109130
| `oz-relayer`
110131
| `string`
111-
| Redis key prefix
132+
| Redis key prefix for namespacing. See xref:storage.adoc[Storage Configuration] for more information.
112133

134+
| `STORAGE_ENCRYPTION_KEY`
135+
| ``
136+
| `string`
137+
| Encryption key used to encrypt data at rest in Redis storage. See xref:storage.adoc[Storage Configuration] for security details.
113138

114139
| `RPC_TIMEOUT_MS`
115140
| `10000`
@@ -173,18 +198,20 @@ PROVIDER_RETRY_MAX_DELAY_MS=2000
173198
PROVIDER_MAX_FAILOVERS=3
174199
ENABLE_SWAGGER=false
175200
KEYSTORE_PASSPHRASE=your_keystore_passphrase
201+
STORAGE_ENCRYPTION_KEY=X67aXacJB+krEldv9i2w7NCSFwwOzVV/1ELM2KJJjQw=
202+
REPOSITORY_STORAGE_TYPE=redis
203+
RESET_STORAGE_ON_START=false
204+
TRANSACTION_EXPIRATION_HOURS=8
176205
```
177206

178207
== Main configuration file (config.json)
179208

180209
This file can exist in any directory, but the default location is `./config/config.json`.
181210

182-
Copy the example config file and update values according to your needs
183-
184-
[source,bash]
185-
----
186-
cp config/config.example.json config/config.json
187-
----
211+
[NOTE]
212+
====
213+
All components defined in `config.json` can also be managed via REST API endpoints. This provides runtime flexibility for adding, updating, or removing relayers, signers, and notifications without restarting the service. See the **API Reference** page for detailed endpoints documentation.
214+
====
188215

189216
Key sections in this file include:
190217

@@ -196,22 +223,28 @@ Key sections in this file include:
196223

197224
=== 1. Signers
198225

199-
Transaction signers are responsible for cryptographically signing transactions before they are submitted to blockchain networks. The `signers` array must contain at least one valid signer configuration.
226+
Transaction signers are responsible for cryptographically signing transactions before they are submitted to blockchain networks.
200227

201228
For comprehensive details on configuring all supported signer types including:
202229

203230
- Local keystore file signers
204-
- HashiCorp Vault (secret, cloud, and transit)
231+
- HashiCorp Vault (secret and transit)
205232
- Cloud KMS providers (Google Cloud, AWS)
206233
- Turnkey signers
207234
- Security best practices and troubleshooting
208235

209236
See the dedicated xref:signers.adoc[Signers Configuration] guide.
210237

238+
[TIP]
239+
====
240+
Signers can also be managed via API endpoints.
241+
242+
See the **API Reference** page for detailed endpoints documentation.
243+
====
211244

212245
=== 2. Notifications
213246

214-
* `notifications` array, which should contain, at least, one valid configuration:
247+
* `notifications` array containing notification entries:
215248

216249
[source,json]
217250
----
@@ -253,9 +286,16 @@ Available configuration fields
253286
|Signing key value, env variable name, ...
254287
|===
255288

289+
[TIP]
290+
====
291+
Notifications can also be managed via API endpoints.
292+
293+
See the **API Reference** page for detailed endpoints documentation.
294+
====
295+
256296
=== 3. Relayers
257297

258-
* `relayers` array, containing at least one valid relayer configuration:
298+
* `relayers` array, containing relayer entries:
259299

260300
[source,json]
261301
----
@@ -469,6 +509,14 @@ When using custom RPC URLs:
469509
* If a weight is not specified for an endpoint, it defaults to 100.
470510
====
471511

512+
[TIP]
513+
====
514+
Relayers could also be managed via API endpoints.
515+
516+
See the **API Reference** page for detailed endpoints documentation.
517+
====
518+
519+
472520
=== 4. Plugins
473521

474522
For more information on how to write a plugin, please refer to the xref:plugins.adoc[Plugins] page.
@@ -684,3 +732,24 @@ Full `config/config.json` example with evm and solana relayers definitions using
684732
]
685733
}
686734
----
735+
736+
== Configuration Management Approaches
737+
738+
The OpenZeppelin Relayer supports two complementary approaches for configuration management:
739+
740+
=== File-based Configuration
741+
- Ideal for initial setup and deployment
742+
- Configuration persists across restarts
743+
- Requires container restart for changes to take effect
744+
- Suitable for infrastructure-as-code workflows
745+
746+
=== API-based Configuration
747+
- Enables runtime configuration changes
748+
- No service restarts required
749+
- Perfect for dynamic environments
750+
- Supports automated configuration management
751+
752+
[NOTE]
753+
====
754+
See xref:storage.adoc[Storage Configuration] for detailed information about how file-based and API-based configurations work together, storage behavior, and best practices.
755+
====

‎docs/modules/ROOT/pages/index.adoc‎

Lines changed: 11 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -231,6 +231,7 @@ For quick setup with various configurations, check the https://github.com/OpenZe
231231
* `evm-gcp-kms-signer`: Using Google Cloud KMS for EVM secure signing
232232
* `evm-turnkey-signer`: Using Turnkey for EVM secure signing
233233
* `solana-turnkey-signer`: Using Turnkey for Solana secure signing
234+
* `redis-storage`: Using Redis for Storage
234235
* `network-configuration-config-file`: Using Custom network configuration via config file
235236
* `network-configuration-json-file`: Using Custom network configuration via JSON file
236237
@@ -311,14 +312,22 @@ DOCKERFILE=Dockerfile.production docker compose up -d
311312

312313
== Configuration
313314

314-
OpenZeppelin Relayer requires proper configuration before starting. The configuration system uses two main files:
315+
OpenZeppelin Relayer supports two configuration approaches:
315316

317+
**File-based Configuration:**
316318
- **`config.json`**: Contains relayer definitions, signer configurations, and network policies
317319
- **`.env`**: Contains environment variables like API keys and connection strings
318320

321+
**API-based Configuration:**
322+
- Runtime configuration management via REST API
323+
- No service restarts required for configuration changes
324+
- Full CRUD operations for relayers, signers, and notifications
325+
319326
[IMPORTANT]
320327
====
321-
Both configuration files must be properly set up before starting the application. Changes to either file require restarting the container to take effect.
328+
Both approaches can be used together. File-based configuration is loaded on startup, while API changes provide runtime flexibility. Changes to environment variables (`.env`) always require restarting the container.
329+
330+
When used together, API changes are not synced to file-based configuration. File-based configuration is loaded only once when using persistent storage mode.
322331
323332
For quick setup examples with pre-configured files, see the https://github.com/OpenZeppelin/openzeppelin-relayer/tree/main/examples[examples directory] in our GitHub repository.
324333
====

‎docs/modules/ROOT/pages/signers.adoc‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -5,7 +5,7 @@
55

66
Signers are responsible for cryptographically signing transactions before they are submitted to blockchain networks. OpenZeppelin Relayer supports multiple signer types to accommodate different security requirements and infrastructure setups.
77

8-
The `signers` array in your configuration must contain at least one valid signer configuration. Each signer is referenced by its `id` in relayer configurations.
8+
Each signer is referenced by its `id` in relayer configurations.
99

1010
== Configuration Structure
1111

0 commit comments

Comments
 (0)