Skip to content
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
78 changes: 78 additions & 0 deletions .claude/skills/ea-pr-reviewer/SKILL.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,78 @@
---
name: ea-pr-reviewer
description: Review Chainlink External Adapter (EA) changes in external-adapters-js using the EAF—production code quality, unit tests (isolated business logic only), and integration tests (TestAdapter end-to-end). Use whenever the user asks for EA PR review, adapter code review, EAF correctness, or to validate unit/integration tests under packages/sources. Also use for pre-merge review, diff-scoped review, or “is this adapter production-ready?” even if they do not say “skill” or “reviewer.”
---

# EA PR reviewer

Merged rubric for **code review**, **unit test validation**, and **integration test validation** for source adapters in the **external-adapters-js** monorepo.

## Repository assumptions

- Root = monorepo root. Adapters live under `packages/sources/<adapter>/` (`src/`, `test/unit/`, `test/integration/`, etc.).
- EAF sources: after `yarn install`, under `.yarn/unplugged/@chainlink-external-adapter-framework-npm-*/node_modules/@chainlink/external-adapter-framework/` (details in `references/eaf-framework-paths.md`).

## Scoping

- If the user names **paths**, a **diff**, or an **adapter**, infer `packages/sources/<adapter>/` from paths when possible.
- Review **only what is in scope** for the **change under review**:
- If **no files** under `test/integration/**` are in the stated PR/diff scope, or the user explicitly says **integration tests were not modified**, then in **`full` mode** set `integration_tests` to **`approved: true`** and **`rationale: ""`**. Do **not** run the full integration rubric to fail that dimension for pre-existing suite gaps. You may add **one optional sentence** in `## Summary` (not in `rationale`) if you want to note technical debt in unchanged tests.
- For **unit tests:** if the user says unit tests were **untouched** and no `test/unit/**` paths are in the stated PR/diff scope, set `unit_tests` to **`approved: true`** and **`rationale: ""`**. If **any** `test/unit/**` path **is** in scope, apply the **full** unit rubric for the adapter (including completeness of isolated business logic such as other exports in `src/` modules that lack unit coverage)—do **not** set `approved: true` only because the edited test file itself is reasonable.
- If the user asks for a **standalone** integration or unit review (single mode), always apply the corresponding reference fully—even if files were unchanged—because the question is about the suite quality, not PR scope.
- When given a file list or diff, prioritize **those files plus minimal surrounding context**. When asked for a **full** adapter review **with no PR scope stated**, read the whole package areas that matter (`src/`, `config/`, `transport/`, `endpoint/`, tests) and apply all rubrics.

### Code review depth (`full` mode, transport-related PRs)

If the user says **`src/`** changed, **transport** changed, or the diff touches `packages/sources/<adapter>/src/transport/**`, you **must** read **every** `*.ts` file under `packages/sources/<adapter>/src/transport/` (recursively) before returning `code_review.approved: true`. While reading, actively check: optional chaining / guards on nested fields from external messages (WS/JSON), and safe indexing on provider arrays (e.g. `[0]` on possibly empty arrays). If any transport file was not read, set `code_review.approved: false` and say which paths were skipped.

## Routing

Determine **mode** from the user message:

| Mode | When | Load |
| ------------- | --------------------------------------------------------- | --------------------------------------------------------- |
| `code` | Code/production review only, no test validation requested | `references/code-review.md` |
| `unit` | Unit tests only | `references/unit-tests.md` |
| `integration` | Integration tests only | `references/integration-tests.md` |
| `full` | PR / branch review, or scope unclear | All three references, in order: code → integration → unit |

Read the relevant reference file(s) before judging. Use `references/eaf-framework-paths.md` whenever you need to look up framework file locations.

## Outputs

### Single mode (`code`, `unit`, or `integration`)

Return:

- `approved`: boolean
- `rationale`: string — **empty string when `approved` is true** for `unit` and `integration` (save tokens). For `code`, when `approved` is true, `rationale` may be empty or a brief note; when false, use detailed feedback with `[file:line] category: description. Fix: suggestion` per issue.

### Full mode (`full`)

Return a single structured result:

- `code_review`: `{ "approved": boolean, "rationale": string }`
- `integration_tests`: `{ "approved": boolean, "rationale": string }` — empty `rationale` when approved
- `unit_tests`: `{ "approved": boolean, "rationale": string }` — empty `rationale` when approved
- `overall_approved`: boolean — `true` only if all three `approved` values are `true`

Apply **unchanged** dimensions as above (approved with empty rationale, or a one-line “no integration test changes in scope”).

### Optional human summary

After structured output, you may add a short **Markdown** block titled `## Summary` suitable for pasting into a GitHub PR comment: verdict, blockers, and next steps—no more than a few bullets.

## Conduct

- **Validators:** never edit files; analysis only (unit and integration modes, and test portions of full mode).
- **Code review mode:** still read-only unless the user explicitly asks you to apply fixes.
- Do **not** produce long meta-narratives about your process, tool usage, or internal checklists. Deliver the verdict and structured fields (and optional short summary) only.

## Reference index

| File | Content |
| ----------------------------------- | ------------------------- |
| `references/eaf-framework-paths.md` | Unplugged EAF paths |
| `references/code-review.md` | Production code checklist |
| `references/integration-tests.md` | Integration test rubric |
| `references/unit-tests.md` | Unit test rubric |
23 changes: 23 additions & 0 deletions .claude/skills/ea-pr-reviewer/evals/evals.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,23 @@
{
"skill_name": "ea-pr-reviewer",
"evals": [
{
"id": 1,
"prompt": "Review the full EA package at packages/sources/tiingo: code, integration tests, and unit tests. The PR only changed src/ (transport-related) and test/unit/transport-utils.test.ts—integration tests under test/integration were not modified.",
"expected_output": "Full mode: code_review + unit_tests judged; integration_tests approved with empty rationale (unchanged in PR). Optional Summary may mention pre-existing integration debt. overall_approved reflects code+unit only.",
"files": []
},
{
"id": 2,
"prompt": "Validate only the integration tests in packages/sources/tiingo/test/integration for EAF quality (TestAdapter, mocks, coverage). Do not review application source code beyond what is needed to understand test intent.",
"expected_output": "Single-mode integration: approved + rationale; empty rationale if pass; integration-focused, not a full code-review dump.",
"files": []
},
{
"id": 3,
"prompt": "Are the unit tests in packages/sources/tiingo/test/unit meaningful business-logic tests, or are they testing framework behavior?",
"expected_output": "Single-mode unit validation; distinguishes framework vs business logic; structured approved/rationale per skill.",
"files": []
}
]
}
89 changes: 89 additions & 0 deletions .claude/skills/ea-pr-reviewer/references/code-review.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,89 @@
# Code review rubric (EA production readiness)

## Goal

Review EA **source** code for quality, correctness, and proper External Adapter Framework (EAF) usage. Be strict: the adapter should be production-ready.

**Framework location:** See `eaf-framework-paths.md` in this folder for unplugged paths under `.yarn/unplugged/.../external-adapter-framework/`.

## Review checklist

### 1. Framework component selection

**Principle:** Use the most specific framework component available for the use case.

- Transport matches data source (REST → `HttpTransport`, WebSocket → `WebSocketTransport`, SSE → `SseTransport`).
- Endpoint type matches data (e.g. price → `PriceEndpoint`, bid/ask → `LwbaEndpoint`, stocks → `StockEndpoint`, PoR → PoR endpoints).
- Adapter type matches endpoints (`PriceAdapter`, `PoRAdapter`, etc.).
- Use `EmptyInputParameters` when there are no input params (not `new InputParameters({}, [{}])`).
- Use framework response types (`SingleNumberResultResponse`, etc.).
- Configuration uses `AdapterConfig` with proper types.

### 2. Code quality

**DRY:** No duplicated logic; shared constants/types extracted; no copy-paste blocks.

**KISS:** No unnecessary abstractions; no over-engineering; direct readable code; avoid shallow wrappers around native/framework APIs.

**Single responsibility:** Transport fetches data; endpoint routes; config holds settings.

**Readability:** Clear names; consistent style; sensible structure; avoid overly dense one-liners.

### 3. Correctness

**Types:** Solid TypeScript; no `any` without justification; I/O types match framework expectations; generics constrained.

**Numbers:** Large integers via `BigInt`, `bignumber.js`, or `ethers.toBigInt()`; decimals via `decimal.js`; no unsafe `parseInt`/`parseFloat` on large values; preserve precision appropriate for on-chain feeds (e.g. 18 decimals).

**Errors:** Framework error types (`AdapterError`, `AdapterInputError`, etc.); actionable messages; no silent swallow without logging; provider failures handled gracefully.

### 4. Performance

- `prepareRequests` batches efficiently (not N identical upstream calls for N params).
- Minimal API usage; no redundant fetching.
- No obvious leaks; parsing stays lean.

### 5. Configuration

URLs and secrets from config, not hardcoded. Required vs optional clear. Sensible defaults. Types match reality (string, number, boolean, enum).

### 6. Libraries vs raw HTTP

| Use case | Library | Instead of |
| -------------- | ------------------------- | ----------------- |
| EVM calls | `ethers` | Raw JSON-RPC HTTP |
| Solana | `@solana/web3.js` | Raw HTTP |
| Large integers | `bignumber.js` / `BigInt` | `Number` |
| Decimal math | `decimal.js` | Native floats |

### 7. Transports

**HttpTransport:** Minimal `prepareRequests`; `parseResponse` maps to params; errors handled; response types match endpoints.

**WebSocketTransport:** Correct `url`; `handlers.message` parses correctly; subscribe/unsubscribe builders if needed.

**Custom:** Extends correct base (e.g. `SubscriptionTransport`); background execution correct.

### 8. Input parameters

Typed params; required vs optional clear; useful aliases; clear descriptions.

## Review process

1. Read relevant files under the EA package (`src/`, `config/`, `transport/`, `endpoint/`).
2. Evaluate against this checklist.
3. Allow justified exceptions when requirements demand them.
4. Decide if the result is production-ready.

## Severity

**Must fix (blocks approval):** Wrong framework component; numeric precision loss; missing critical error handling; hardcoded secrets/URLs; pathological request patterns (N calls where one batch suffices).

**Should fix:** Minor style noise; verbose but correct code; missing optional optimizations.

**Context-dependent:** Custom transports, justified raw HTTP, unusual requirements—flag for humans with context.

## Output for this dimension

- `approved`: boolean
- `rationale`: string — if not approved, issues with paths and fixes. Prefer lines like: `[file:line] category: description. Fix: suggestion`
22 changes: 22 additions & 0 deletions .claude/skills/ea-pr-reviewer/references/eaf-framework-paths.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,22 @@
# EAF framework source paths (Yarn unplugged)

After `yarn install` at the **external-adapters-js** repo root, framework sources live under:

**Base:** `.yarn/unplugged/@chainlink-external-adapter-framework-npm-*/node_modules/@chainlink/external-adapter-framework/`

Use the resolved path when reading files (glob may match one version directory).

## By concern

| Concern | Path (relative to base) |
| ------------------------------------------------------------------- | --------------------------------------------- |
| Package root / discovering components | `.` (browse `package.json`, exports) |
| Testing utils (`TestAdapter`, `MockWebsocketServer`, timer helpers) | `util/testing-utils.d.ts` |
| Response / adapter types | `util/types.d.ts` |
| HTTP transport | `transports/http.d.ts` |
| Subscription / background execution | `transports/abstract/subscription.d.ts` |
| All transports | `transports/` |
| Input / schema validation | `validation/`, `validation/input-params.d.ts` |
| Adapter lifecycle, basic adapter | `adapter/`, `adapter/basic.d.ts` |

When instructions say “read framework source,” use these paths to confirm patterns instead of guessing from memory.
67 changes: 67 additions & 0 deletions .claude/skills/ea-pr-reviewer/references/integration-tests.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,67 @@
# Integration test validation (EAF)

**Scope only:** integration tests. **Do not edit** test or source files.

**Framework paths:** See `eaf-framework-paths.md` for `TestAdapter`, `MockWebsocketServer`, `HttpTransport`, `SubscriptionTransport`, and response types.

Read framework sources to understand what the harness provides, expected transport patterns, and standard response shapes.

## Task

- Review **only** integration tests for an adapter under `packages/sources/<adapter>/test/integration/**`.
- Question: **Are these integration tests high quality, meaningful, effective, efficient, and complete end-to-end tests for this adapter?**
- **If yes:** `approved: true`, `rationale: ""` (empty — do not explain pass).
- **If no:** `approved: false` and concise rationale (what to fix).

## What counts as an integration test

A real integration test:

1. **Goes through the adapter entrypoint** — uses framework `TestAdapter` with realistic requests (e.g. `{ base, quote, endpoint }`).
2. **Realistic env + provider mocks** — only needed env vars; `BACKGROUND_EXECUTE_MS` around **10s (10000ms)** for background cases; upstream mocked with `nock`, `MockWebsocketServer`, `SocketServerMock`, etc., not the adapter under test.
3. **Asserts user-visible behavior** — status codes, JSON (often snapshots), key fields (`result`, `data.result`, symbols, ripcord, etc.); validates `AdapterResponse`-shaped results.

Tests that only hit pure helpers or internals without the adapter entrypoint are **not** integration tests—call that out.

## Quality signals for the suite

- **Happy paths:** Major endpoints covered; distinct request shapes that change behavior each have coverage where it matters.
- **Validation:** `{}` / missing required fields → `400` with sensible errors; bad combinations tested where relevant.
- **Upstream failures:** Important paths cover 4xx/5xx and malformed data; adapter maps to appropriate errors (e.g. 502/504) with useful JSON.
- **Edge cases:** Non-trivial branches (zeros, empty arrays, inverse pairs, staleness, mappings).
- **Transports:** REST exercises real request build + nocked HTTP. WS/streaming: cache warmup (`testAdapter.request`, `waitForCache`), `FakeTimers` / `runAllUntilTime` when TTL/heartbeat matters; success, failure/invariant paths, deduplication where relevant.
- **Env / cache / rate limits:** Explicit minimal env; cache/TTL tests for heavy adapters; rate limiting often disabled in tests without hiding real behavior.

If an entire dimension is missing (e.g. no validation errors anywhere), lean **no**.

## Low-value patterns

Flag as redundant for integration:

- No adapter entrypoint.
- Language/framework trivia.
- Only pure utilities without the adapter.
- Mock-wiring focus instead of final adapter output.
- Many near-duplicates with no new behavioral insight.

## Answer shape

- **Approved:** `approved: true`, `rationale: ""`.
- **Rejected:** `approved: false`, short actionable bullets (2–3 gaps max).

## How tests should run (sanity)

```bash
cd external-adapters-js && yarn install && yarn setup
cd external-adapters-js && yarn clean && yarn build
export adapter=[adapter-name]
timeout 30 yarn test $adapter/test/integration
```

Flaky tests, real network, or undocumented setup are quality issues.

## Output discipline

Focus on EA code and tests. Do **not** add meta-reports, “how I followed these rules,” tool logs, or extra checklists in the reply.

When validating: confirm `TestAdapter` usage, transport pattern matches adapter type (from framework source), mocking uses framework utilities where appropriate, and assertions match framework response types.
Loading
Loading