Skip to content

Commit 29ca941

Browse files
feat: implement getTransactions and getLedgers
Serve the two history methods from a new per-ledger index. When a transaction closes a ledger, the server writes ledgers/ledger_<seq>.json carrying the ledger sequence, the closing transaction's hash, the close time, and the ledger-header artifacts (hash, headerXdr, metadataXdr) -- built in Python because K cannot construct XDR. The K semantics read the index and the stored receipts to collect and format both responses, keeping response formatting on the K side. Parameter validation (limit 1-200, startLedger bounds, cursor/startLedger exclusivity, xdrFormat) lives in the server next to the other methods' parameter checks and rejects with -32602. Cursors follow real stellar-rpc: a TOID-style stringified integer for getTransactions (one transaction per ledger here, applicationOrder always 1) and the plain ledger sequence for getLedgers, returned only when a page is full. Serialization matches the Go protocol structs, including their quirks: per-transaction createdAt is a JSON number (unlike the singular getTransaction), per-ledger ledgerCloseTime is a decimal string, the top-level close-time keys differ between the two methods, and empty resultXdr/resultMetaXdr stubs are omitted. Failed transactions close no ledger and therefore do not appear in the history.
1 parent 074fbe3 commit 29ca941

7 files changed

Lines changed: 439 additions & 13 deletions

File tree

docs/architecture.md

Lines changed: 3 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -51,7 +51,7 @@ flowchart TB
5151

5252
**[Detailed documentation](server.md)**
5353

54-
The server implements six RPC methods — `getHealth`, `getNetwork`, `getLatestLedger`, `sendTransaction`, `getTransaction`, and `traceTransaction` — and the K semantics answer all of them.
54+
The server implements eight RPC methods — `getHealth`, `getNetwork`, `getLatestLedger`, `sendTransaction`, `getTransaction`, `getTransactions`, `getLedgers`, and `traceTransaction` — and the K semantics answer all of them.
5555

5656
`sendTransaction` always returns `PENDING` and clients poll `getTransaction` for the result — matching the Stellar RPC async pattern even though the transaction executes synchronously. See [server.md](server.md) for details.
5757

@@ -91,11 +91,12 @@ All of the server's input and output artifacts live in one directory, the *io di
9191
| `metadata.json` | persistent | the K semantics | `{"latest_ledger": N}` — the server ledger counter, bumped by 1 per committed transaction. |
9292
| `receipts/receipt_<hash>.json` | persistent | the semantics (on success) or the server (on failure) | one stored receipt per transaction, keyed by tx hash, answering `getTransaction`. Each is `{status, ledger, createdAt, envelopeXdr, resultXdr, resultMetaXdr}`. |
9393
| `traces/trace_<hash>.jsonl` | persistent | the semantics | one execution trace per transaction, keyed by tx hash — the instruction-level records, one JSON object per line. `traceTransaction` returns this file's contents. |
94+
| `ledgers/ledger_<seq>.json` | persistent | the server (on success) | one record per closed ledger — `{sequence, txHash, closedAt, hash, headerXdr, metadataXdr}` — the ledger→transaction index behind `getTransactions` and `getLedgers`. Written in Python because the header artifacts are XDR, which K cannot construct. |
9495
| `requests/request_<n>.json` | persistent | the server | an archive of each incoming JSON-RPC request, numbered by a monotonic counter, kept for debugging. |
9596
| `request.json` | transient | the server | the request envelope for the call in flight (`method`, `id`, `now`, and method-specific fields). The semantics remove it once they respond. |
9697
| `response.json` | transient | the semantics | the JSON-RPC response (`{jsonrpc, id, result}`) for the most recent call. The server reads it back; it is absent when a transaction gets stuck. |
9798

98-
Receipts, traces, and request archives are split into one file per item — keyed by tx hash, or numbered — so that no single file grows without bound as the chain advances. The server creates the `receipts/`, `traces/`, and `requests/` directories before the semantics run, because the K file-system hooks open files with POSIX `open()`, which does not create parent directories.
99+
Receipts, traces, ledger records, and request archives are split into one file per item — keyed by tx hash, ledger sequence, or a counter — so that no single file grows without bound as the chain advances. The server creates the `receipts/`, `traces/`, `ledgers/`, and `requests/` directories before the semantics run, because the K file-system hooks open files with POSIX `open()`, which does not create parent directories.
99100

100101
The world state stays in KORE (rather than a JSON snapshot) because an uploaded wasm module is a `ModuleDecl` that the semantics cannot reconstruct from bytes — only `wasm2kast` (Python) can produce it. The receipts and the ledger counter, by contrast, are plain data and live in JSON files, which the semantics read and write directly via the file-system hooks.
101102

docs/node-semantics.md

Lines changed: 6 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -17,6 +17,7 @@ The semantics communicate with the Python process through files in the working d
1717
| `metadata.json` | K ↔ K | `{"latest_ledger": N}` — the ledger counter |
1818
| `receipts/receipt_<hash>.json` | K → Python | one stored receipt per transaction, keyed by tx hash |
1919
| `traces/trace_<hash>.jsonl` | K → Python | one execution trace per transaction (per-instruction records), keyed by tx hash |
20+
| `ledgers/ledger_<seq>.json` | Python → K | one record per closed ledger (`sequence`, `txHash`, `closedAt`, and the ledger-header XDR artifacts), written by the server and read back to serve `getTransactions`/`getLedgers` |
2021

2122
---
2223

@@ -39,7 +40,8 @@ insert-handleRequestFile → handleRequestFile
3940
4041
#dispatchMethod(method, request) ← routes on the "method" field
4142
42-
├─ getHealth / getNetwork / getLatestLedger / getTransaction / traceTransaction → #respond(...)
43+
├─ getHealth / getNetwork / getLatestLedger / getTransaction /
44+
│ getTransactions / getLedgers / traceTransaction → #respond(...)
4345
4446
└─ sendTransaction → #runTx → run steps
4547
→ #finalizeTx → record receipt + bump ledger → #respond(...)
@@ -62,6 +64,7 @@ If `request.json` is absent, `insert-handleRequestFile` does not fire and K halt
6264
- `getNetwork``{ "friendbotUrl": null, "passphrase": ..., "protocolVersion": ... }` (passphrase/version come from the request, keeping the semantics network-agnostic)
6365
- `getLatestLedger` → reads `metadata.json` and returns `{ "id": <64 zeros>, "protocolVersion": ..., "sequence": <latest_ledger> }`
6466
- `getTransaction` → reads the hash's `receipts/receipt_<hash>.json` file; returns the stored receipt merged with the current `latestLedger`/`latestLedgerCloseTime`, or `{ "status": "NOT_FOUND", ... }` when the file is absent
67+
- `getTransactions` / `getLedgers` → walk the per-ledger index files `ledgers/ledger_<seq>.json` from the envelope's `startSeq` up to the latest ledger, taking at most `limit` records (`#txInfos` / `#ledgerInfos`), and format the history page (`#txHistoryPage` / `#ledgerHistoryPage`). Parameter validation and cursor resolution happen in the server; the response `cursor` (`#pageCursor`) names the page's last record when the page is full — a TOID for transactions, the plain sequence for ledgers — and is empty otherwise. Serialization matches real stellar-rpc, including its quirks: per-transaction `createdAt` is a JSON number (unlike singular `getTransaction`), per-ledger `ledgerCloseTime` is a decimal string, and empty `resultXdr`/`resultMetaXdr` stubs are omitted (`#optXdrEntry`)
6568

6669
`#respond(ID, RESULT)` is the shared terminal: it writes the JSON-RPC envelope to `response.json`, removes `request.json`, and sets the exit code to 0.
6770

@@ -105,7 +108,8 @@ The trace is not part of the receipt — the executing steps already appended it
105108

106109
- `#getJSON(key, obj[, default])`, `#getString(key, obj)`, `#getInt(key, obj)` — read a field
107110
- `#concatJSONs(a, b)` — append object entries (used to merge `latestLedger` fields into a stored receipt)
108-
- `#receiptFile(hash)`, `#traceFile(hash)` — build the per-transaction file paths (`receipts/receipt_<hash>.json`, `traces/trace_<hash>.jsonl`)
111+
- `#receiptFile(hash)`, `#traceFile(hash)`, `#ledgerFile(seq)` — build the per-item file paths (`receipts/receipt_<hash>.json`, `traces/trace_<hash>.jsonl`, `ledgers/ledger_<seq>.json`)
112+
- `#readJSONFile(path)`, `#asInt(json)`, `#lengthJSONs(list)`, `#lastIntIn(key, list)` — small conveniences for the history methods
109113

110114
These complement the **order-sensitive** step decoders below.
111115

docs/notes.md

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -22,7 +22,7 @@ State lives in the io dir as `state.kore` (KORE world state) and `metadata.json`
2222

2323
## Tests (`src/tests/integration/`)
2424

25-
- `test_server.py` drives the running HTTP server end-to-end. It exercises the read-only methods, `sendTransaction` + `getTransaction`, ledger increments, the full lifecycle (create → upload wasm → deploy → invoke), and the `traceTransaction` flows. `test_call_tx_with_args` deploys `args.wat` and calls functions with `bool`, `u32`, `i32`, `u64`, `i64`, `u128`, `i128`, and `symbol` arguments, exercising the `scval_to_json` / `#decodeArg` pipeline.
25+
- `test_server.py` drives the running HTTP server end-to-end. It exercises the read-only methods, `sendTransaction` + `getTransaction`, the history methods (`getTransactions`/`getLedgers` response shapes, pagination, and parameter validation against the official spec and the Go protocol structs), ledger increments, the full lifecycle (create → upload wasm → deploy → invoke), and the `traceTransaction` flows. `test_call_tx_with_args` deploys `args.wat` and calls functions with `bool`, `u32`, `i32`, `u64`, `i64`, `u128`, `i128`, and `symbol` arguments, exercising the `scval_to_json` / `#decodeArg` pipeline.
2626
- `test_integration.py` and `test_unit.py` hold small sanity checks.
2727

2828
Run with `make test` (requires `make kdist-build` first).
@@ -36,3 +36,4 @@ The tests do not yet cover `bytes` / `address` SCVal arguments or `SCVec` / `SCM
3636
- `resultXdr` / `resultMetaXdr` are empty stubs (contract return values not surfaced).
3737
- `SCVec` / `SCMap` contract arguments are not yet encoded.
3838
- `simulateTransaction`, `getEvents`, `getLedgerEntries`, `getFeeStats`, and TTL/footprint operations are not implemented.
39+
- `getTransactions` / `getLedgers` serve only ledgers with an index file under `ledgers/`; io-dirs created before the ledger index existed resume fine, but their earlier ledgers do not appear in the history.

docs/server.md

Lines changed: 42 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -14,6 +14,7 @@ class StellarRpcServer:
1414
state_file: Path # io_dir / 'state.kore'
1515
receipts_dir: Path # io_dir / 'receipts' — receipt_<hash>.json per transaction
1616
traces_dir: Path # io_dir / 'traces' — trace_<hash>.jsonl per transaction
17+
ledgers_dir: Path # io_dir / 'ledgers' — ledger_<seq>.json per closed ledger
1718
requests_dir: Path # io_dir / 'requests' — request_<n>.json archive
1819
```
1920

@@ -28,7 +29,7 @@ server = StellarRpcServer(io_dir=Path('out'))
2829
server.handle_rpc('sendTransaction', {'transaction': xdr})
2930
```
3031

31-
For `sendTransaction` it builds the request envelope with `encoder.build_tx_request` and runs it with `interpreter.run`; for the read-only methods (`getHealth`, `getNetwork`, `getLatestLedger`, `getTransaction`, `traceTransaction`) it builds a small envelope and runs it. In every case the *content* of the response is produced by the semantics (`node.md`), not by Python — the one exception is the failure fallback (below). Each call is logged to stderr.
32+
For `sendTransaction` it builds the request envelope with `encoder.build_tx_request` and runs it with `interpreter.run`; for the read-only methods (`getHealth`, `getNetwork`, `getLatestLedger`, `getTransaction`, `getTransactions`, `getLedgers`, `traceTransaction`) it builds a small envelope and runs it. For the history methods (`getTransactions`, `getLedgers`) the server also validates the pagination parameters — limit range, `startLedger` bounds, `cursor`/`startLedger` exclusivity — and resolves the cursor to the first ledger sequence to serve, because the JSON-RPC parameter-error path lives here. In every case the *content* of the response is produced by the semantics (`node.md`), not by Python — the exceptions are the failure fallback (below) and the per-ledger XDR artifacts (`_record_closed_ledger`), which K cannot construct. Each call is logged to stderr.
3233

3334
---
3435

@@ -51,7 +52,7 @@ At construction the server prepares the *io dir*, where `state.kore` lives at `i
5152
- **`state.kore` absent**`interpreter.empty_config()` produces the initial idle K configuration (a blank-slate state with no accounts, contracts, or storage) and writes it; `metadata.json` is seeded with `{"latest_ledger": 0}`.
5253
- **`state.kore` present** — it is used as-is, and `metadata.json` is seeded only if missing. This lets you resume a previous session (ledger counter and stored receipts included) or start against a pre-built state.
5354

54-
In both cases the server creates the `receipts/`, `traces/`, and `requests/` directories if they do not already exist, because the K file-system hooks write into them but cannot create them.
55+
In both cases the server creates the `receipts/`, `traces/`, `ledgers/`, and `requests/` directories if they do not already exist, because the K file-system hooks write into them but cannot create them.
5556

5657
Once the socket is bound, `serve` logs three lines to stderr: whether it is starting from a fresh state (an empty io-dir) or resuming an existing one (with the latest ledger), the io-dir path, and the listening address. Instruction tracing is always on, so every transaction the semantics run produces a trace. (Tracing only produces records for contract invocations.)
5758

@@ -71,6 +72,8 @@ per successful transaction:
7172
write receipts/receipt_<hash>.json, bump latest_ledger in metadata.json,
7273
and write response.json
7374
→ NodeInterpreter persists the new state.kore
75+
→ the server writes ledgers/ledger_<seq>.json (the ledger→tx index entry,
76+
with the ledger-header XDR artifacts)
7477
7578
per failed (stuck) transaction:
7679
→ no response.json is produced; state.kore and metadata.json are left unchanged
@@ -143,6 +146,42 @@ All methods are answered by the K semantics and follow the [Stellar RPC specific
143146

144147
`resultXdr` and `resultMetaXdr` are currently empty stubs. The receipt carries no trace — use `traceTransaction` with the same hash to fetch it.
145148

149+
### `getTransactions`
150+
151+
`getTransactions` returns the transactions in a ledger range, in chain order. Params: `startLedger` (number; mutually exclusive with a cursor), `pagination` `{cursor, limit}` (limit 1–200, default 50), and `xdrFormat` (`base64` only; `json` is rejected with `-32602`). The records are joined from the per-ledger index files (`ledgers/ledger_<seq>.json`) and the stored receipts. Failed transactions never close a ledger, so they do not appear in the history.
152+
153+
```json
154+
{
155+
"transactions": [
156+
{ "status": "SUCCESS", "txHash": "<64-char hex>", "applicationOrder": 1, "feeBump": false,
157+
"envelopeXdr": "<base64 XDR>", "ledger": 5, "createdAt": 1716000000 }
158+
],
159+
"latestLedger": 5, "latestLedgerCloseTimestamp": 1716000000,
160+
"oldestLedger": 0, "oldestLedgerCloseTimestamp": 0,
161+
"cursor": ""
162+
}
163+
```
164+
165+
Serialization follows real stellar-rpc: ledger sequences and the top-level close times are JSON numbers, and per-transaction `createdAt` is a number too (unlike the singular `getTransaction`, where it is a string — an upstream quirk). `resultXdr`/`resultMetaXdr` are omitted while the receipts carry empty stubs. Each ledger holds exactly one transaction on this node, so `applicationOrder` is always `1`. The `cursor` is a TOID-style stringified integer (`ledger << 32 | applicationOrder << 12`) naming the page's last transaction when the page is full, and empty otherwise; resume by passing it as `pagination.cursor` (without `startLedger`).
166+
167+
### `getLedgers`
168+
169+
`getLedgers` returns the closed ledgers in a range and takes the same parameters as `getTransactions`. Each record comes straight from the ledger's index file; `headerXdr` (a `LedgerHeaderHistoryEntry`) and `metadataXdr` (a `LedgerCloseMeta`) are built by the server when the ledger closes, since K cannot construct XDR, and the ledger `hash` is the SHA-256 of the header XDR — unique per ledger and chained through `previousLedgerHash`.
170+
171+
```json
172+
{
173+
"ledgers": [
174+
{ "hash": "<64-char hex>", "sequence": 5, "ledgerCloseTime": "1716000000",
175+
"headerXdr": "<base64 XDR>", "metadataXdr": "<base64 XDR>" }
176+
],
177+
"latestLedger": 5, "latestLedgerCloseTime": 1716000000,
178+
"oldestLedger": 0, "oldestLedgerCloseTime": 0,
179+
"cursor": ""
180+
}
181+
```
182+
183+
Per-ledger `ledgerCloseTime` is a *string* holding a decimal number (matching real stellar-rpc's Go `,string` encoding), while the top-level close times are numbers. The `cursor` is the last returned ledger sequence, stringified, under the same full-page rule as `getTransactions`.
184+
146185
---
147186

148187
## Failure fallback
@@ -161,4 +200,4 @@ komet-node [--host HOST] [--port PORT] [--io-dir DIR]
161200
|---|---|---|
162201
| `--host` | `localhost` | Bind address |
163202
| `--port` | `8000` | Port |
164-
| `--io-dir` | a fresh temp dir | Directory holding every artifact (`state.kore`, `metadata.json`, `receipts/`, `traces/`, `requests/`) |
203+
| `--io-dir` | a fresh temp dir | Directory holding every artifact (`state.kore`, `metadata.json`, `receipts/`, `traces/`, `ledgers/`, `requests/`) |

0 commit comments

Comments
 (0)