Skip to content

Commit 27df441

Browse files
andreogleandreogle
committed
Add cache server, push targets, requester-specified encoding
Cache server: standalone process that receives signed beacon data from airnodes and serves it to clients with configurable delay tiers and per-endpoint auth. New CLI command `airnode cache-server`. Push targets: airnode push loop can POST signed beacons to external cache servers with retry logic (2 retries, 1s delay via go() options). Requester-specified encoding: clients can pass _type, _path, _times in request parameters to control encoding on endpoints without a fixed encoding block. Supports partial encoding where the operator fixes some fields and the requester provides the rest. Also includes: - RPC retry for NFT ownership and payment verification (go() retries) - Rate limiting extracted to src/rate-limit.ts for reuse - Signature verification (verifySignedBeacon) for cache server ingestion - BoundedMap values() method and bounded beacon ingestion store - Schema validation rejecting required params with default values - Documentation for cache server, push targets, requester encoding - Integration tests: async lifecycle, beacon routes, stable JSON signatures, upstream error handling, requester encoding - Unit tests: rate-limit, bounded-map, cache-server, push targets
1 parent 8676b2b commit 27df441

37 files changed

Lines changed: 2103 additions & 109 deletions

‎book/docs/cli.md‎

Lines changed: 19 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -36,6 +36,25 @@ airnode start -c config.yaml -e .env.production
3636

3737
Shut down gracefully with `Ctrl+C` (`SIGINT`) or `SIGTERM`.
3838

39+
## `airnode cache-server`
40+
41+
Start the cache server for signed beacon data. See [Cache Server](/docs/operators/cache-server) for full documentation.
42+
43+
```bash
44+
airnode cache-server [options]
45+
```
46+
47+
| Option | Alias | Default | Description |
48+
| ----------------- | ----- | ------------------- | ---------------------------- |
49+
| `--config <path>` | `-c` | `cache-server.yaml` | Path to the config file |
50+
| `--env <path>` | `-e` | `.env` | Path to the environment file |
51+
52+
```bash
53+
airnode cache-server -c cache-server.yaml -e .env.production
54+
```
55+
56+
No `AIRNODE_PRIVATE_KEY` is needed — the cache server verifies signatures but does not sign.
57+
3958
## `airnode config validate`
4059

4160
Validate a config file without starting the server. Checks YAML syntax, schema validation, and cross-field consistency.

‎book/docs/config/apis.md‎

Lines changed: 61 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -374,10 +374,14 @@ encoding:
374374

375375
| Field | Type | Required | Description |
376376
| ------- | -------- | -------- | ------------------------------------------------------------------------------- |
377-
| `type` | `string` | Yes | Solidity type(s) for ABI encoding: `int256`, `uint256`, `bool`, `bytes32`, etc. |
378-
| `path` | `string` | Yes | JSONPath expression to extract the value from the API response. |
377+
| `type` | `string` | No | Solidity type(s) for ABI encoding: `int256`, `uint256`, `bool`, `bytes32`, etc. |
378+
| `path` | `string` | No | JSONPath expression to extract the value from the API response. |
379379
| `times` | `string` | No | Multiplier applied before encoding. Converts decimals to integers for Solidity. |
380380

381+
All fields are optional individually. The encoding is complete when both `type` and `path` are present — either from the
382+
config, from the requester's request parameters, or a combination of both. See
383+
[requester-specified encoding](#requester-specified-encoding) below.
384+
381385
### Multi-value encoding
382386

383387
Encode multiple values from a single API response using comma-separated `type`, `path`, and `times`:
@@ -407,6 +411,42 @@ endpoints:
407411
# No encoding -- raw JSON response is signed
408412
```
409413

414+
### Requester-specified encoding
415+
416+
Clients can control encoding by passing reserved parameters in their request body: `_type`, `_path`, and optionally
417+
`_times`. These parameters are consumed by the pipeline and never sent to the upstream API.
418+
419+
**Three modes:**
420+
421+
1. **Operator-fixed** — the endpoint has a complete `encoding` block (`type` + `path`). Client reserved parameters are
422+
ignored. The endpoint ID commits to this encoding.
423+
2. **Partial** — the endpoint has an `encoding` block with some fields (e.g. `type` only). The client fills in the
424+
missing fields via `_path` or `_times`. Operator fields take precedence.
425+
3. **Requester-only** — no `encoding` block. The client provides `_type` and `_path`. If neither is provided, raw JSON
426+
mode is used.
427+
428+
```bash
429+
# Requester chooses what to extract from a raw endpoint
430+
curl -X POST http://localhost:3000/endpoints/{endpointId} \
431+
-H "Content-Type: application/json" \
432+
-d '{"parameters":{"ids":"ethereum","vs_currencies":"usd","_type":"int256","_path":"$.ethereum.usd","_times":"1e18"}}'
433+
```
434+
435+
```yaml
436+
# Partial: operator fixes the type, requester chooses the path
437+
endpoints:
438+
- name: flexiblePrice
439+
path: /simple/price
440+
encoding:
441+
type: int256
442+
# path comes from the requester's _path parameter
443+
```
444+
445+
If the merged result has `_type` without `_path` (or vice versa), the server returns 400.
446+
447+
:::note Push endpoints require a complete encoding in the config (`type` + `path`). The push loop has no client
448+
parameters, so requester-specified encoding does not apply to push. :::
449+
410450
## Push
411451

412452
The `push` field enables a background loop that calls the upstream API on a fixed interval and stores signed data for
@@ -415,11 +455,25 @@ relayers.
415455
```yaml
416456
push:
417457
interval: 10000 # call API every 10 seconds
458+
targets:
459+
- url: http://cache.example.com/beacons/0xYourAirnodeAddress
460+
authToken: ${CACHE_SERVER_AUTH_TOKEN}
418461
```
419462

420-
| Field | Type | Required | Description |
421-
| ---------- | -------- | -------- | ------------------------------------------------ |
422-
| `interval` | `number` | Yes | Loop interval in milliseconds. Positive integer. |
463+
| Field | Type | Required | Description |
464+
| ---------- | -------- | -------- | ------------------------------------------------------------------- |
465+
| `interval` | `number` | Yes | Loop interval in milliseconds. Positive integer. |
466+
| `targets` | `array` | No | Cache server URLs to push signed beacons to. See [Cache Server][1]. |
467+
468+
Each target has:
469+
470+
| Field | Type | Required | Description |
471+
| ----------- | -------- | -------- | ------------------------------------------------------ |
472+
| `url` | `string` | Yes | Cache server ingestion URL (includes airnode address). |
473+
| `authToken` | `string` | Yes | Bearer token for push authentication. |
474+
475+
Pushed data is available locally via `GET /beacons` and `GET /beacons/{beaconId}`. If `targets` are configured, the
476+
signed data is also POSTed to each cache server after every update (retried up to 2 times on failure). If the endpoint
477+
also has a `cache.delay`, the local beacon data is not served until the delay has elapsed.
423478

424-
Pushed data is available via `GET /beacons` and `GET /beacons/{beaconId}`. If the endpoint also has a `cache.delay`, the
425-
beacon data is not served until the delay has elapsed.
479+
[1]: /docs/operators/cache-server

‎book/docs/operators/cache-server.md‎

Lines changed: 159 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,159 @@
1+
---
2+
slug: /operators/cache-server
3+
sidebar_position: 3
4+
---
5+
6+
# Cache Server
7+
8+
The cache server is a standalone process that receives signed beacon data from one or more airnodes and serves it to
9+
clients. It has no private key, no API credentials, and no business logic — it only stores and serves pre-signed data.
10+
11+
## When to use it
12+
13+
The built-in `/beacons` routes on the airnode are sufficient for single-operator setups. A separate cache server is
14+
useful when:
15+
16+
- **Multiple airnodes** push signed data to a shared read endpoint. Clients query one URL instead of N airnode URLs.
17+
- **Security isolation** — the airnode holds the private key and API credentials. A cache server has neither, reducing
18+
attack surface for the public-facing read layer.
19+
- **Delay tiers** — serve real-time data to paying clients and delayed data to free-tier clients, using the same signed
20+
data. The airnode signs once; the cache server enforces delay policies.
21+
- **Independent scaling** — beacon reads don't compete with the airnode's API call and signing workload.
22+
23+
## Architecture
24+
25+
```
26+
Airnode A ──push──→ ┌──────────────┐ ←── GET /realtime/{beaconId} ── Paying client
27+
Airnode B ──push──→ │ Cache Server │ ←── GET /delayed/{beaconId} ── Free client
28+
Airnode C ──push──→ └──────────────┘
29+
```
30+
31+
Each airnode POSTs signed beacons to the cache server. The cache server verifies signatures on ingestion (recovering the
32+
signer address via EIP-191) and rejects data that doesn't match the claimed airnode address.
33+
34+
## Configuration
35+
36+
The cache server uses a separate config file from the airnode:
37+
38+
```yaml
39+
version: '1.0'
40+
41+
server:
42+
port: 8090
43+
host: '0.0.0.0'
44+
cors:
45+
origins:
46+
- '*'
47+
rateLimit:
48+
window: 60000
49+
max: 1000
50+
51+
allowedAirnodes:
52+
- address: '0xf39Fd6e51aad88F6F4ce6aB8827279cffFb92266'
53+
authToken: ${CACHE_SERVER_AUTH_TOKEN}
54+
55+
endpoints:
56+
- path: /realtime
57+
delaySeconds: 0
58+
auth:
59+
type: apiKey
60+
keys:
61+
- ${REALTIME_CLIENT_KEY}
62+
- path: /delayed
63+
delaySeconds: 60
64+
auth:
65+
type: free
66+
```
67+
68+
### `allowedAirnodes`
69+
70+
Either a list of allowed airnode addresses with their push auth tokens, or `'*'` to accept any airnode (still requires a
71+
non-empty bearer token).
72+
73+
### `endpoints`
74+
75+
Each endpoint defines a read path with a delay and optional auth:
76+
77+
| Field | Type | Required | Description |
78+
| -------------- | -------- | -------- | ---------------------------------------------------------- |
79+
| `path` | `string` | Yes | URL path prefix (e.g. `/realtime`). Must start with `/`. |
80+
| `delaySeconds` | `number` | Yes | Minimum age of data in seconds. `0` for real-time. |
81+
| `auth` | `object` | No | Client-facing auth (same format as airnode endpoint auth). |
82+
83+
The same signed beacon data is served through all endpoints. The delay filter only returns beacons whose timestamp is at
84+
least `delaySeconds` old.
85+
86+
## Running
87+
88+
```bash
89+
airnode cache-server -c cache-server.yaml -e .env
90+
```
91+
92+
| Option | Alias | Default | Description |
93+
| ----------------- | ----- | ------------------- | ---------------------------- |
94+
| `--config <path>` | `-c` | `cache-server.yaml` | Path to the config file |
95+
| `--env <path>` | `-e` | `.env` | Path to the environment file |
96+
97+
No `AIRNODE_PRIVATE_KEY` is needed — the cache server only verifies signatures, it does not sign.
98+
99+
## API routes
100+
101+
### `POST /beacons/{airnodeAddress}`
102+
103+
Ingest signed beacon data. Requires `Authorization: Bearer <token>` matching the `authToken` for the airnode address in
104+
`allowedAirnodes`.
105+
106+
**Request body** — a single beacon or an array of beacons:
107+
108+
```json
109+
{
110+
"airnode": "0x...",
111+
"endpointId": "0x...",
112+
"beaconId": "0x...",
113+
"timestamp": 1711234567,
114+
"data": "0x...",
115+
"signature": "0x..."
116+
}
117+
```
118+
119+
**Response:**
120+
121+
```json
122+
{ "count": 1, "skipped": 0, "errors": 0 }
123+
```
124+
125+
- `count` — beacons successfully stored (newer than existing)
126+
- `skipped` — beacons with a timestamp equal to or older than the existing value
127+
- `errors` — beacons with invalid signatures or missing fields
128+
129+
### `GET /{endpointPath}/{beaconId}`
130+
131+
Serve a single beacon by ID. Returns 425 if the beacon is newer than the endpoint's `delaySeconds` allows.
132+
133+
### `GET /{endpointPath}`
134+
135+
List all beacons available at this endpoint, filtered by the delay.
136+
137+
### `GET /health`
138+
139+
Health check. Returns `{ "status": "ok", "version": "..." }`.
140+
141+
## Push targets (airnode side)
142+
143+
To push signed data from the airnode to a cache server, add `targets` to the endpoint's `push` configuration:
144+
145+
```yaml
146+
push:
147+
interval: 10000
148+
targets:
149+
- url: http://cache.example.com/beacons/0xYourAirnodeAddress
150+
authToken: ${CACHE_SERVER_AUTH_TOKEN}
151+
```
152+
153+
| Field | Type | Required | Description |
154+
| ----------- | -------- | -------- | ------------------------------------------------------ |
155+
| `url` | `string` | Yes | Cache server ingestion URL (includes airnode address). |
156+
| `authToken` | `string` | Yes | Bearer token matching the cache server's config. |
157+
158+
Push requests are retried up to 2 times with a 1-second delay on failure. The push is fire-and-forget — failures don't
159+
block the push loop.

‎book/docs/roadmap.md‎

Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -21,7 +21,8 @@ clients.
2121
- Plugin system with 6 hooks and per-request time budgets
2222
- AirnodeVerifier contract (pull path -- verify + forward to callback)
2323
- AirnodeDataFeed contract (push path -- verify + store + read, beacon sets with median aggregation)
24-
- Push loop with beacon store for data feed relayers
24+
- Push loop with beacon store and push targets for external cache servers
25+
- Cache server (standalone process for receiving, verifying, and serving signed beacon data)
2526
- DNS identity verification (ERC-7529)
2627
- In-memory response cache with TTL
2728
- Cache delay for OEV windows
@@ -74,8 +75,7 @@ Developer experience and ecosystem tooling.
7475
- **Config builder**: Visual interface for building Airnode configs from OpenAPI specs. Replaces manual YAML editing.
7576
- **Endpoint directory**: Public registry of available airnode endpoints with documentation, pricing, and availability
7677
metrics. Endpoint IDs are the common identifier -- operators serving the same API produce the same ID.
77-
- **Cache server**: Thin standalone process that receives signed push data from the airnode and serves it to relayers.
78-
Decouples the airnode from serving beacon data directly. Supports delayed endpoints for OEV tiers.
78+
- **Operator dashboard**: Request volume, revenue, uptime metrics.
7979

8080
## Future
8181

‎book/sidebars.ts‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -17,7 +17,7 @@ const sidebars: SidebarsConfig = {
1717
{
1818
type: 'category',
1919
label: 'Airnode Operators',
20-
items: ['operators/index', 'operators/deployment'],
20+
items: ['operators/index', 'operators/deployment', 'operators/cache-server'],
2121
},
2222
{
2323
type: 'category',

‎examples/configs/cache-server/config.yaml‎

Lines changed: 27 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,27 @@
1+
version: '1.0'
2+
3+
server:
4+
port: 8090
5+
host: '0.0.0.0'
6+
cors:
7+
origins:
8+
- '*'
9+
rateLimit:
10+
window: 60000
11+
max: 1000
12+
13+
allowedAirnodes:
14+
- address: '0xf39Fd6e51aad88F6F4ce6aB8827279cffFb92266'
15+
authToken: ${CACHE_SERVER_AUTH_TOKEN}
16+
17+
endpoints:
18+
- path: /realtime
19+
delaySeconds: 0
20+
auth:
21+
type: apiKey
22+
keys:
23+
- ${REALTIME_CLIENT_KEY}
24+
- path: /delayed
25+
delaySeconds: 60
26+
auth:
27+
type: free

‎examples/configs/complete/.env.example‎

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -17,3 +17,7 @@ COINGECKO_API_KEY=test-coingecko-key
1717
WEATHER_API_KEY=test-weather-key
1818
RANDOM_ORG_TOKEN=test-random-token
1919
CLIENT_API_KEY=test-client-key
20+
21+
# Push target configuration — for pushing signed data to a cache server
22+
AIRNODE_ADDRESS=0x70997970C51812dc3A010C7d01b50e0d17dc79C8
23+
CACHE_SERVER_AUTH_TOKEN=test-cache-token

‎examples/configs/complete/config.yaml‎

Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -40,7 +40,6 @@ apis:
4040
description: Coin ID (e.g. ethereum, bitcoin)
4141
- name: vs_currencies
4242
in: query
43-
required: true
4443
default: usd
4544
encoding:
4645
type: int256
@@ -51,6 +50,9 @@ apis:
5150
delay: 60000 # public responses delayed by 60s (OEV window)
5251
push:
5352
interval: 10000 # call API every 10 seconds, store signed data for relayers
53+
targets:
54+
- url: http://localhost:8090/beacons/${AIRNODE_ADDRESS}
55+
authToken: ${CACHE_SERVER_AUTH_TOKEN}
5456
description: Get the current price of a coin
5557

5658
- name: coinMarketData
@@ -150,11 +152,9 @@ apis:
150152
fixed: generateIntegers
151153
- name: min
152154
in: body
153-
required: true
154155
default: 0
155156
- name: max
156157
in: body
157-
required: true
158158
default: 100
159159
encoding:
160160
type: uint256

0 commit comments

Comments
 (0)