Skip to content

Commit a268cb0

Browse files
committed
Update doc with tests
1 parent 6da9ad1 commit a268cb0

File tree

1 file changed

+60
-1
lines changed

1 file changed

+60
-1
lines changed

pkg/beholder/DurableEmitterDesign.md

Lines changed: 60 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -227,7 +227,7 @@ This replaces the global beholder emitter, covering **all** emission paths:
227227
| chainlink-common | `pkg/beholder/durable_emitter_integration_test.go` | Integration tests (mock gRPC server) |
228228
| chainlink | `core/services/beholder/durable_event_store_orm.go` | `PgDurableEventStore` (Postgres ORM) |
229229
| chainlink | `core/services/beholder/durable_event_store_orm_test.go` | ORM tests + Postgres benchmarks + load tests |
230-
| chainlink | `core/services/beholder/durable_emitter_load_test.go` | Full-stack load tests (Postgres + mock gRPC) |
230+
| chainlink | `core/services/beholder/durable_emitter_load_test.go` | TPS ramp/sustained/payload tests (Postgres + mock or external Chip via `CHIP_INGRESS_TEST_ADDR`) |
231231
| chainlink | `core/store/migrate/migrations/0295_chip_durable_events.sql` | Postgres migration |
232232
| chainlink | `core/config/telemetry_config.go` | `DurableEmitterEnabled()` on Telemetry interface |
233233
| chainlink | `core/config/toml/types.go` | TOML field + setFrom merge |
@@ -287,6 +287,65 @@ Real gRPC server with controllable failure injection:
287287
| `Benchmark_FullStack_EmitThroughput` | Upper bound events/sec through full pipeline |
288288
| `Benchmark_FullStack_EmitPayloadSizes` | Full emit at 64B, 256B, 1KB, 4KB |
289289

290+
### Durable emitter TPS load tests (`chainlink/core/services/beholder/durable_emitter_load_test.go`)
291+
292+
These tests exercise **Postgres + `DurableEmitter` + Chip Ingress** (in-process mock **or** a real gateway). They are heavier than the ORM benchmarks and require a **real Postgres** (not `txdb`).
293+
294+
#### Prerequisites
295+
296+
- **`CL_DATABASE_URL`** — must point at a Postgres instance where migration **`0295_chip_durable_events`** has been applied (`cre.chip_durable_events` exists). Same URL pattern as other chainlink DB tests.
297+
- **Short tests skipped** — if your test runner uses `-short`, these tests are skipped (`SkipShortDB`); run **without** `-short`.
298+
299+
#### Mock Chip vs real Chip Ingress
300+
301+
| Mode | How | Notes |
302+
|------|-----|--------|
303+
| **Mock** (default) | Do **not** set `CHIP_INGRESS_TEST_ADDR` | In-process gRPC server; tests can count **Server recv** events and inject failures (outage, slow Chip). |
304+
| **Real Chip** | Set `CHIP_INGRESS_TEST_ADDR=host:port` | Dials external Chip Ingress. Optional: `CHIP_INGRESS_TEST_TLS`, `CHIP_INGRESS_TEST_BASIC_AUTH_*`, `CHIP_INGRESS_TEST_SKIP_BASIC_AUTH`, `CHIP_INGRESS_TEST_SKIP_SCHEMA_REGISTRATION`. You need Kafka/Redpanda, topic **`chip-demo`**, and schema subject **`chip-demo-pb.DemoClientPayload`** (e.g. Atlas `make create-topic-and-schema` under `atlas/chip-ingress`). |
305+
306+
Tests that **inject** Chip failures or rely on **in-process** receive counts are **skipped** when `CHIP_INGRESS_TEST_ADDR` is set.
307+
308+
#### How to run
309+
310+
From the `chainlink` repo root (examples):
311+
312+
```bash
313+
# All beholder tests including TPS (requires CL_DATABASE_URL)
314+
export CL_DATABASE_URL='postgres://...'
315+
go test -v -count=1 ./core/services/beholder/ -run 'TestTPS_|TestChipIngressExternalPing'
316+
317+
# Ramp-up only (100 → 500 → 1k → 2k TPS levels)
318+
go test -v -count=1 ./core/services/beholder/ -run TestTPS_RampUp
319+
320+
# Sustained 1k TPS for 60s + drain check
321+
go test -v -count=1 ./core/services/beholder/ -run TestTPS_Sustained1k
322+
323+
# Payload size scaling (fixed duration per size)
324+
go test -v -count=1 ./core/services/beholder/ -run TestTPS_PayloadSizeScaling
325+
326+
# External Chip smoke (with addr set)
327+
export CHIP_INGRESS_TEST_ADDR='localhost:50051'
328+
go test -v -count=1 ./core/services/beholder/ -run TestChipIngressExternalPing
329+
```
330+
331+
After a full package run, **`TestMain`** prints a **TPS LOAD TEST SUMMARY** block aggregating result blocks from **`TestTPS_RampUp`**, **`TestTPS_Sustained1k`**, **`TestTPS_1k_WithChipOutage`** (mock only; skipped with external Chip), and **`TestTPS_PayloadSizeScaling`**.
332+
333+
#### Reading the tables (column glossary)
334+
335+
| Column | Meaning |
336+
|--------|---------|
337+
| **Target TPS** | Requested emit rate (token-bucket style scheduling across workers). |
338+
| **Achieved TPS** | `Total emits ÷ window duration` — realized successful `Emit()` throughput. |
339+
| **Total emits** | Count of **`Emit()` calls that returned `nil`** in the measurement window (successful Postgres insert path). Does not count failures. |
340+
| **Emit p50 / p99** | Latency of successful `Emit()` calls (dominated by DB insert). |
341+
| **Failures** | `Emit()` calls that returned an error (e.g. DB failure). |
342+
| **Server recv** | **Mock only:** number of events observed by the in-process gRPC server (`Publish` / `PublishBatch`). |
343+
| **Queue depth** | Rows remaining in `cre.chip_durable_events` after the emit phase (+ short settle), i.e. backlog not yet deleted after successful publish. |
344+
345+
#### Why **Server recv** shows **N/A** with real Chip
346+
347+
The **Server recv** column is implemented by counting events on the **in-process mock** `ChipIngress` server. When you use **`CHIP_INGRESS_TEST_ADDR`**, there is no mock — the client talks to a **real** gateway — so the test **cannot** count server-side receives in-process. Use **Kafka / Chip / gateway metrics** (or consumer verification) to validate end-to-end delivery instead. **Total emits** and **Achieved TPS** still reflect client-side durable insert success; they are not replaced by N/A.
348+
290349
### CRE Smoke Tests (live Docker environment)
291350

292351
Tests connect to the node's Postgres and query `cre.chip_durable_events` directly, using `pg_stat_user_tables` for insert/delete statistics — the same pattern used by the EVM LogTrigger test for `trigger_pending_events`.

0 commit comments

Comments
 (0)