Skip to content

Commit 6a46ae5

Browse files
committed
Merge branch 'v3'
2 parents c84138f + 5407b30 commit 6a46ae5

200 files changed

Lines changed: 23245 additions & 59625 deletions

File tree

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.
Lines changed: 50 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,50 @@
1+
name: Release squid version
2+
on:
3+
push:
4+
branches:
5+
- 'v[0-9]+'
6+
7+
jobs:
8+
build_and_publish:
9+
runs-on: ubuntu-latest
10+
steps:
11+
- name: Checkout
12+
uses: actions/checkout@v4
13+
14+
- name: Install Node.js
15+
uses: actions/setup-node@v4
16+
with:
17+
node-version: 20
18+
19+
- name: Setup pnpm
20+
uses: pnpm/action-setup@v4
21+
with:
22+
version: 10
23+
24+
- name: Install squid CLI
25+
run: pnpm add -g @subsquid/cli
26+
27+
- name: Install dependencies
28+
run: pnpm install --frozen-lockfile
29+
30+
- name: update squid.yml
31+
run: |
32+
# Get the branch name from GitHub
33+
BRANCH_NAME=${{ github.ref_name }}
34+
35+
# Extract the version from the branch name (assuming format 'vXX')
36+
VERSION=$(echo $BRANCH_NAME | sed 's/^v//')
37+
38+
# Update the version in squid.yaml
39+
sed -i "s/^version: .*/version: $VERSION/" squid.yaml
40+
41+
# Optional: Print the updated version for verification
42+
echo "Updated squid.yaml version to: $VERSION"
43+
44+
- name: Authenticate to squid
45+
env:
46+
API_TOKEN: ${{ secrets.SQUID_API_TOKEN }}
47+
run: sqd auth -k $API_TOKEN
48+
49+
- name: Build and deploy squid
50+
run: sqd build && sqd deploy . -o origin --no-stream-logs --allow-update

CLAUDE.md

Lines changed: 98 additions & 83 deletions
Original file line numberDiff line numberDiff line change
@@ -4,7 +4,11 @@ This file provides guidance to Claude Code (claude.ai/code) when working with co
44

55
## Project Overview
66

7-
This is a Subsquid-based notification system for Origin Protocol. It monitors blockchain events across Ethereum mainnet, Base, and Sonic chains and sends notifications to Discord webhooks. The database is minimal—this squid focuses on real-time event processing, not data storage.
7+
This is a Subsquid-based notification system for Origin Protocol. It monitors blockchain events and traces across Ethereum mainnet, Base, and Sonic chains, persists them to a database, and sends notifications to Discord webhooks based on configurable alert rules stored in a separate Postgres database.
8+
9+
## User Interaction
10+
11+
You are encouraged to ask the user questions in order to save time and effort.
812

913
## Common Commands
1014

@@ -14,126 +18,137 @@ pnpm run process # Build & start mainnet processor (spins up local DB v
1418
pnpm run process:base # Build & start Base chain processor
1519
pnpm run process:sonic # Build & start Sonic chain processor
1620
pnpm run resume # Resume without rebuild/DB setup
17-
pnpm run generate-abis # Generate TypeScript from ABI JSON files in abi/
1821
pnpm run prettier-fix # Format code
22+
pnpm run backfill # Backfill EventRecord/TraceRecord for specific rules (see below)
23+
pnpm run digest-db # Output all DB-driven alert rules and code-driven processors
24+
pnpm run generate-seed # Regenerate seed-rules.sql from code-driven processors
25+
pnpm run generate-abi-seed # Regenerate seed-abis.sql from abi/*.json files
1926
sqd deploy . --update # Deploy to Subsquid Cloud (use --update to prevent gaps)
2027
```
2128

2229
### Environment Variables for Development
2330

2431
- `BLOCK_FROM=123456` - Start processing from specific block
25-
- `PROCESSOR=Name` - Filter to processors matching "Name"
32+
- `PROCESSOR=Name` - Filter code-driven processors matching "Name" (OGN Alerts, OGN Buybacks)
33+
- `ALERT=Name` - Filter config-driven alert rules by display name or ID (substring match)
34+
- `ALERT_CONFIG_DB_MIGRATION=true` - Run DB migration + seed on startup (for local dev)
2635

27-
Example: `BLOCK_FROM=18102886 PROCESSOR=Trace pnpm run process`
36+
Examples:
37+
```shell
38+
BLOCK_FROM=21540000 ALERT="Lido ARM" pnpm run process # Debug specific alert rules
39+
BLOCK_FROM=21540000 PROCESSOR=Buyback pnpm run process # Debug code-driven processors
40+
```
41+
42+
### Backfill Command
43+
44+
Backfills EventRecord/TraceRecord for alert rules without sending notifications:
45+
```shell
46+
pnpm run backfill -- --chain 1 --from 21525891 # All rules on mainnet
47+
pnpm run backfill -- --chain 8453 --from 24450127 --rule oeth-vault # Specific rule on Base
48+
```
2849

2950
## Architecture
3051

52+
### Two Databases
53+
54+
| | Squid DB (Subsquid Cloud Postgres) | Alert Config DB (Separate Postgres) |
55+
|---|---|---|
56+
| **Purpose** | Raw on-chain data (EventRecord, TraceRecord) + notification dedup | Alert rules, ABIs, topic definitions |
57+
| **Managed by** | Subsquid migrations, TypeORM | `migration.sql`, seeded at startup |
58+
| **Connection** | `DB_URL` or `DB_HOST`/`DB_PORT`/etc. | `ALERT_CONFIG_DB_URL` |
59+
3160
### Entry Points (Chain-Specific)
3261
- `src/main.ts` - Ethereum mainnet processor
3362
- `src/main-base.ts` - Base chain processor
3463
- `src/main-sonic.ts` - Sonic chain processor
64+
- `src/backfill.ts` - Standalone backfill script
65+
66+
Each entry point runs this startup sequence:
67+
1. `initAlertConfigDb()` — creates DB, runs migration, seeds data (when `ALERT_CONFIG_DB_MIGRATION=true`)
68+
2. `abiRegistry.loadFromDb()` — loads ABIs from alert_config DB, builds decode maps
69+
3. `loadWalletLabels()` — loads address names from Railway DB
70+
71+
### Processors (`src/processors/`)
72+
- **`config-alert.ts`** - Config-driven alert processor. Loads rules from alert_config DB, subscribes to matching events/traces, evaluates filters, sends notifications. This is the primary processor for all chains.
73+
- **`persistence.ts`** - Persists all events and traces to EventRecord/TraceRecord tables in the squid DB. Runs alongside config-alert on every chain.
3574

36-
### Topics System (`src/topics/`)
37-
Topics group related notifications by product (OETH, OUSD, OGN, etc.). Each topic folder contains processor definitions organized by concern:
38-
- `core-contracts.ts` - Main token/vault events
39-
- `strategies.ts` - Strategy contract events
40-
- `error-tracing.ts` - Transaction failure tracking
41-
- `index.ts` - Exports all processors for the topic
75+
### Code-Driven Processors (`src/topics/`)
76+
Two processors with custom `process()` logic that can't be expressed as DB rules:
77+
- `ogn-alerts/` - OGN staking alerts with custom logic
78+
- `ogn-buybacks/` - OGN buyback tracking
4279

43-
The `src/topics/index.ts` auto-loads all topic folders at runtime.
80+
These are loaded by `src/topics/index.ts` and run alongside config-alert on mainnet only.
4481

45-
### Templates (`src/templates/`)
46-
Reusable processor factories for common patterns:
47-
- `event.ts` - Generic event processor (`createEventProcessor`)
48-
- `otoken.ts` - OToken contract events
49-
- `otoken-vaults.ts` - Vault events (mint, redeem, rebase)
50-
- `burn.ts` - Token burn tracking
51-
- `governance.ts` - Governance proposals
52-
- `trace.ts` / `trace-errors.ts` - Transaction tracing
82+
### Alert Config System (`src/alert-config/`)
83+
- `migration.sql` - Schema: `chain`, `topic`, `abi`, `alert_rule` tables with validation constraints
84+
- `config-loader.ts` - Loads rules from DB with 5-minute cache refresh, `initAlertConfigDb()` for migration
85+
- `evaluate-filter.ts` - Evaluates AND/OR filter expressions against decoded event/trace data
86+
- `types.ts` - TypeScript types for AlertRule, FilterExpression
87+
- `seed-rules.sql` - Generated seed data for alert rules
88+
- `seed-abis.sql` - Generated seed data for ABI storage
89+
90+
Admin UI: See `../alert-config-admin` — a React + Hono app for managing alert rules via a web interface.
91+
92+
### ABI Registry (`src/utils/abi-registry.ts`)
93+
Dual-mode ABI loading:
94+
1. **DB-loaded (primary)** — Full ABIs stored in the `abi` table, decoded at runtime via viem. Supports fallback decoding when multiple ABIs share the same selector (e.g., different `Deposit` events with different indexed params).
95+
2. **Subsquid-compiled (fallback)**`src/abi/*.ts` files loaded at import time for code-driven processors. Only 3 remain: `erc20.ts`, `exponential-staking.ts`, `multicall.ts`.
5396

5497
### Notification System (`src/notify/`)
5598
- `discord.ts` - Discord webhook integration with message queue
56-
- `const.ts` - Topics, severities, and webhook mappings
57-
- `event/` - Event rendering and formatting for Discord embeds
99+
- `const.ts` - Topics, severities, webhook mappings, emoji/color constants
100+
- `format.ts` - Shared formatting utilities (addresses linked to explorers, BigInt formatting, traderate support)
101+
- `event/event.ts` - Event notification pipeline (Loki + Discord + oncall)
102+
- `event/renderers/default.ts` - Discord embed renderer with structured fields
103+
- `trace.ts` - Trace notification pipeline with Discord embed renderer
104+
- `loki.ts` - Grafana Loki structured logging
105+
- `oncall.ts` - Grafana OnCall webhook for high/critical severity
106+
- `notification-log.ts` - DB-backed notification deduplication
58107

59108
### Path Aliases
60-
Configured in `.cursorrules` and `tsconfig.json`:
109+
Configured in `tsconfig.json`:
61110
- `@abi/*``src/abi/*`
62111
- `@utils/*``src/utils/*`
63112
- `@notify/*``src/notify/*`
64113
- `@processors/*``src/processors/*`
65114

66-
## Contract Tracking Workflow
115+
## Adding New Alert Rules
67116

68-
This is the core workflow for understanding and expanding notification coverage:
117+
The primary way to add notification coverage is through the alert_config database:
69118

70-
### 1. Check What We Currently Notify On
119+
### 1. Ensure the ABI is loaded
120+
If the contract's ABI isn't already in the `abi` table, fetch it and add it:
71121
```shell
72-
pnpm run digest > digest.log
122+
pnpm run fetch-abi <address> <name> [chain] # Saves to abi/<name>.json
123+
pnpm run generate-abi-seed # Regenerates seed-abis.sql
73124
```
74-
Outputs all registered processors with their tracked addresses, events, and topics. Review `digest.log` to see current coverage.
75125

76-
### 2. Check What We Should Be Notifying On
77-
```shell
78-
pnpm run registry
79-
```
80-
Scrapes the [Origin Protocol contract registry](https://docs.originprotocol.com/registry/contracts) and compares against our tracking. Outputs `registry-comparison.md` showing:
81-
- Which contracts have proxy monitoring vs event handlers
82-
- Contracts needing event tracking (have proxy but no events)
83-
- Completely untracked contracts
126+
### 2. Create the alert rule
127+
Use the admin UI (`../alert-config-admin`) or insert directly into the `alert_rule` table:
128+
- `chain_id` — 1 (mainnet), 8453 (base), 146 (sonic)
129+
- `match_type``event` or `trace`
130+
- `addresses` — contract addresses to watch (NULL = any)
131+
- `topic0s` — event signatures to match (NULL = any)
132+
- `sighashes` — function selectors for traces (NULL = any)
133+
- `data_filters` — optional JSONB filter on decoded data (notification-only — all data is always persisted regardless of filters)
134+
- `topic` — Discord channel (OETH, OUSD, ARM, etc.)
135+
- `severity` — low, medium, high, critical, broken, highlight
84136

85-
### 3. Fetch ABIs for Missing Contracts
86-
```shell
87-
pnpm run fetch-abi <address> <name> [chain]
88-
```
89-
Fetches verified contract ABI from Etherscan/block explorer and saves to `abi/`.
137+
Rules are cached for 5 minutes. Changes take effect without redeployment.
90138

91-
Examples:
92-
```shell
93-
pnpm run fetch-abi 0x703118c4cbcccbf2ab31913e0f8075fbbb15f563 oeth-eth-price-feed
94-
pnpm run fetch-abi 0xdbfefd2e8460a6ee4955a68582f85708baea60a3 super-oeth base
95-
```
96-
Requires `ETHERSCAN_API_KEY` env var (free at etherscan.io/apis).
139+
### 3. Add human-readable names (optional)
140+
For new addresses, add entries to `CONTRACT_ADDR_TO_NAME` in `src/utils/addresses/names.ts`, or add them to the `wallet_label` table in the Railway database.
97141

98-
### 4. Generate TypeScript Types & Create Processor
142+
### 4. Backfill historical data (optional)
99143
```shell
100-
pnpm run generate-abis
101-
```
102-
Generates TypeScript types in `src/abi/` from all JSON files in `abi/`.
103-
104-
Then create a processor in the appropriate `src/topics/` subfolder:
105-
```typescript
106-
import { createEventProcessor } from 'templates/event'
107-
108-
createEventProcessor({
109-
name: 'My Processor',
110-
chainId: 1, // 1=mainnet, 8453=base, 146=sonic
111-
topic: 'OETH', // Maps to Discord webhook
112-
tracks: [{ address: [...], events: myAbi.events }]
113-
})
114-
```
115-
116-
### 5. Add Human-Readable Names for New Addresses
117-
When adding new contract addresses, register human-readable names in `src/utils/addresses/names.ts`. This file maps addresses to display names used in Discord notifications.
118-
119-
Add entries to `CONTRACT_ADDR_TO_NAME` under the appropriate chain:
120-
```typescript
121-
[mainnet.id]: {
122-
[MY_NEW_CONTRACT]: 'My New Contract',
123-
// ...
124-
},
125-
[base.id]: {
126-
[baseAddresses.myContract]: 'My Base Contract',
127-
// ...
128-
},
129-
[sonic.id]: {
130-
// ...
131-
},
144+
pnpm run backfill -- --chain <id> --from <block>
132145
```
133146

134147
## Key Concepts
135148

136-
- **Topic** - Product area (OETH, OUSD, OGN, etc.) that maps to a Discord channel
137-
- **Severity** - low, medium, high, critical, broken, highlight
138-
- **NotifyTarget** - Optional email/Discord mention configuration
139-
- **Processor** - Subscribes to blockchain events and triggers notifications
149+
- **Topic** - Product area (OETH, OUSD, OGN, ARM, OS, etc.) that maps to a Discord webhook channel
150+
- **Severity** - low, medium, high, critical, broken, highlight — controls embed color and oncall routing
151+
- **AlertRule** - Database row defining what on-chain activity to watch and how to notify
152+
- **NotifyTarget** - Optional Discord mention or oncall configuration on a rule
153+
- **Processor** - Subscribes to blockchain events/traces and processes each block batch
154+
- **EventRecord / TraceRecord** - Persistent on-chain data in the squid DB (queryable via GraphQL)

CONTRIBUTE.md

Lines changed: 29 additions & 54 deletions
Original file line numberDiff line numberDiff line change
@@ -2,7 +2,7 @@
22

33
## Requirements
44

5-
- Node.js *(wrote using v20)*
5+
- Node.js *(v20+)*
66
- Docker
77
- Copy `dev.env` to `.env` and fill in the empty values
88
- Run `pnpm install` to install dependencies
@@ -16,70 +16,45 @@ pnpm run process
1616

1717
## Adding notifications
1818

19-
Notifications are triggered through `processors` located at: [src/processors](src/processors)
19+
Most notifications are managed through **alert rules** in the config database, not code.
2020

21-
1. View existing processors in [src/processors](src/processors)
22-
2. Follow the patterns shown within to create a processor of your own.
23-
3. If you need more raw control of things you can copy from [this example](src/processors/examples/example.ts)
21+
### Config-driven alerts (preferred)
2422

25-
If the ABI you require does not exist:
23+
1. Ensure the contract ABI is loaded:
24+
```shell
25+
pnpm run fetch-abi <address> <name> [chain] # Saves to abi/<name>.json
26+
pnpm run generate-abi-seed # Regenerates seed-abis.sql
27+
```
2628

27-
1. Add ABI JSON to `abi/`
28-
2. Run `pnpm run generate-abis`
29-
- The ABI will be created within `src/abi/`
29+
2. Create an alert rule via the [admin UI](`https://github.com/oplabs/alert-config-admin`) or insert directly into the `alert_rule` table with:
30+
- `chain_id` — 1 (mainnet), 8453 (base), 146 (sonic)
31+
- `match_type``event` or `trace`
32+
- `addresses` — contract addresses to watch
33+
- `topic0s` — event signatures to match
34+
- `topic` — Discord channel (OETH, OUSD, ARM, etc.)
35+
- `severity` — low, medium, high, critical, broken, highlight
36+
37+
3. Rules are cached for 5 minutes — changes take effect without redeployment.
38+
39+
### Code-driven processors
40+
41+
For alerts requiring custom business logic (e.g., OGN buybacks with USD calculations), add processors in `src/topics/`. See `src/topics/ogn-buybacks/` for an example.
3042

3143
## `pnpm run process` examples
3244

3345
```shell
3446
# Start processing at a specific block
3547
BLOCK_FROM=12345678 pnpm run process
36-
```
3748

38-
```shell
39-
# Start processors matching a certain name
40-
PROCESSOR=Burn pnpm run process
41-
```
42-
43-
## Filtering examples
49+
# Filter code-driven processors by name
50+
PROCESSOR=Buyback pnpm run process
4451

45-
Filter OGN transfers coming from an address:
46-
47-
```typescript
48-
const filter = logFilter({
49-
address: [OGN_ADDRESS],
50-
topic0: [erc20.events.Transfer.topic],
51-
topic1: ['0x58890A9cB27586E83Cb51d2d26bbE18a1a647245'],
52-
})
52+
# Filter config-driven alert rules by name
53+
ALERT="Lido ARM" pnpm run process
5354
```
5455

55-
Filter OGN transfers going to multiple addresses:
56-
57-
```typescript
58-
const filter = logFilter({
59-
address: [OGN_ADDRESS],
60-
topic0: [erc20.events.Transfer.topic],
61-
topic2: [
62-
'0x58890A9cB27586E83Cb51d2d26bbE18a1a647245',
63-
'0x0DD34c397384DE8f21F463096A360a0419D476E1'
64-
],
65-
})
66-
```
56+
## Adding human-readable address names
6757

68-
Filter OGN and OGV transfers and approvals going to multiple addresses:
69-
70-
```typescript
71-
const filter = logFilter({
72-
address: [
73-
OGN_ADDRESS,
74-
OGV_ADDRESS
75-
],
76-
topic0: [
77-
erc20.events.Approval.topic,
78-
erc20.events.Transfer.topic,
79-
],
80-
topic2: [
81-
'0x58890A9cB27586E83Cb51d2d26bbE18a1a647245',
82-
'0x0DD34c397384DE8f21F463096A360a0419D476E1'
83-
],
84-
})
85-
```
58+
For new contract addresses, either:
59+
- Add entries to `CONTRACT_ADDR_TO_NAME` in `src/utils/addresses/names.ts`
60+
- Add them to the `wallet_label` table in the Railway database

0 commit comments

Comments
 (0)