Merge pull request #32 from runtimeverification/fix/rpc-spec-shape

Align RPC response shapes with the Stellar RPC spec

Author: RaoulSchaffranek

docs/node-semantics.md

@@ -56,14 +56,14 @@ If `request.json` is absent, `insert-handleRequestFile` does not fire and K halt
 
 ## Dispatch and the read-only methods
 
-`#dispatch` reads the `method` field and routes to a per-method rule. The read-only methods answer directly from constants and the bookkeeping files:
+`#dispatch` reads the `method` field and routes to a per-method rule. The read-only methods answer directly from constants and the bookkeeping files. Response shapes follow real stellar-rpc's serialization: ledger sequences and `protocolVersion` are JSON numbers; close-time fields are int64-as-string (JSON strings holding a decimal integer).
 
-- `getHealth` → `{ "status": "healthy" }`
-- `getNetwork` → `{ "friendbotUrl": null, "passphrase": ..., "protocolVersion": ... }` (passphrase/version come from the request, keeping the semantics network-agnostic)
-- `getLatestLedger` → reads `metadata.json` and returns `{ "id": <64 zeros>, "protocolVersion": ..., "sequence": <latest_ledger> }`
-- `getTransaction` → reads the hash's `receipts/receipt_<hash>.json` file; returns the stored receipt merged with the current `latestLedger`/`latestLedgerCloseTime`, or `{ "status": "NOT_FOUND", ... }` when the file is absent
+- `getHealth` → `{ "status": "healthy", "latestLedger": N, "latestLedgerCloseTime": <now>, "oldestLedger": 0, "oldestLedgerCloseTime": "0", "ledgerRetentionWindow": N+1 }` (everything since genesis is retained)
+- `getNetwork` → `{ "passphrase": ..., "protocolVersion": ... }` (passphrase/version come from the request, keeping the semantics network-agnostic; `friendbotUrl` is omitted — no friendbot)
+- `getLatestLedger` → reads `metadata.json` and returns `{ "id": #ledgerId(seq), "protocolVersion": ..., "sequence": <latest_ledger> }`; `#ledgerId` derives a deterministic per-ledger 64-hex id from the sequence (the semantics have no hash hooks, so it is a fixed odd-constant multiply mod 2^256, which is injective)
+- `getTransaction` → reads the hash's `receipts/receipt_<hash>.json` file; returns the stored receipt merged with the ledger-range fields (`latestLedger`/`latestLedgerCloseTime`/`oldestLedger`/`oldestLedgerCloseTime`), or `{ "status": "NOT_FOUND", ... }` (with the same ledger-range fields) when the file is absent
 
-`#respond(ID, RESULT)` is the shared terminal: it writes the JSON-RPC envelope to `response.json`, removes `request.json`, and sets the exit code to 0.
+`#respond(ID, RESULT)` is the shared terminal: it writes the JSON-RPC envelope to `response.json`, removes `request.json`, and sets the exit code to 0. `#respondError(ID, CODE, MSG)` is its error counterpart, writing `{jsonrpc, id, error: {code, message}}` instead; the unknown-method fallback uses it to answer `-32601 Method not found` (a safety net — the Python server filters unknown methods before they reach K).
 
 ---
 
@@ -83,9 +83,11 @@ If `request.json` is absent, `insert-handleRequestFile` does not fire and K halt
 
 1. writes `metadata.json` with `latest_ledger + 1`,
 2. writes the receipt to `receipts/receipt_<hash>.json`:
-   `{ status: "SUCCESS", ledger, createdAt, envelopeXdr, resultXdr: "", resultMetaXdr: "" }`,
+   `{ status: "SUCCESS", applicationOrder: 1, feeBump: false, envelopeXdr, resultXdr: "", resultMetaXdr: "", ledger, createdAt }`,
 3. responds with `{hash, status: "PENDING", latestLedger, latestLedgerCloseTime}`.
 
+A `sendTransaction` whose hash already has a receipt never reaches `#runTx`: dispatch answers `{hash, status: "DUPLICATE", latestLedger, latestLedgerCloseTime}` directly, without executing anything or advancing the ledger. (For wasm uploads the Python server also suppresses the `<program>` injection on duplicates, since those steps would run before dispatch.)
+
 The trace is not part of the receipt — the executing steps already appended it to `traces/trace_<hash>.jsonl`. Reaching `#finalizeTx` means the steps completed without getting stuck, so the status is `SUCCESS`. A failed transaction gets stuck before this point, `response.json` is never written, and the Python server records the `FAILED` receipt instead.
 
 ### traceTransaction

docs/notes.md

@@ -34,5 +34,7 @@ The tests do not yet cover `bytes` / `address` SCVal arguments or `SCVec` / `SCM
 ## Known gaps
 
 - `resultXdr` / `resultMetaXdr` are empty stubs (contract return values not surfaced).
+- `sendTransaction`'s `errorResultXdr` is always a generic `txMALFORMED` result (no per-cause codes such as `txBAD_SEQ` — sequence numbers, fees, and signatures are not modelled), and `diagnosticEventsXdr` is never populated (both fields are optional in the spec). `TRY_AGAIN_LATER` is never returned by design: it signals mempool backpressure, which cannot arise in a synchronous node without a mempool.
 - `SCVec` / `SCMap` contract arguments are not yet encoded.
 - `simulateTransaction`, `getEvents`, `getLedgerEntries`, `getFeeStats`, and TTL/footprint operations are not implemented.
+- Receipts are not a stable format: older io-dirs store `ledger`/`createdAt` with pre-spec types and lack `applicationOrder`/`feeBump`. Resume only io-dirs written by the same version, or start fresh.

docs/server.md

@@ -28,7 +28,7 @@ server = StellarRpcServer(io_dir=Path('out'))
 server.handle_rpc('sendTransaction', {'transaction': xdr})
 ```
 
-For `sendTransaction` it builds the request envelope with `encoder.build_tx_request` and runs it with `interpreter.run`; for the read-only methods (`getHealth`, `getNetwork`, `getLatestLedger`, `getTransaction`, `traceTransaction`) it builds a small envelope and runs it. In every case the *content* of the response is produced by the semantics (`node.md`), not by Python — the one exception is the failure fallback (below). Each call is logged to stderr.
+For `sendTransaction` it builds the request envelope with `encoder.build_tx_request` and runs it with `interpreter.run` (skipping the `<program>` wasm injection when the hash already has a receipt, so a duplicate is not re-executed — the semantics answer `DUPLICATE`); for the read-only methods (`getHealth`, `getNetwork`, `getLatestLedger`, `getTransaction`, `traceTransaction`) it builds a small envelope and runs it. In every case the *content* of the response is produced by the semantics (`node.md`), not by Python — the one exception is the failure fallback (below). Each call is logged to stderr.
 
 ---
 
@@ -83,24 +83,30 @@ Because these artifacts live on disk, the server can be stopped and restarted wi
 
 ## RPC methods
 
-All methods are answered by the K semantics and follow the [Stellar RPC specification](https://developers.stellar.org/docs/data/apis/rpc/methods).
+All methods are answered by the K semantics and follow the [Stellar RPC specification](https://developers.stellar.org/docs/data/apis/rpc/methods), including its serialization quirks: ledger sequence numbers and `protocolVersion` are JSON numbers, while close-time fields (`latestLedgerCloseTime`, `oldestLedgerCloseTime`, `createdAt`) are int64s serialized as JSON strings holding a decimal integer, matching what real stellar-rpc emits.
 
 ### `getHealth`
 
-`getHealth` returns `{"status": "healthy"}`.
+```json
+{ "status": "healthy", "latestLedger": 4, "latestLedgerCloseTime": "1716000000", "oldestLedger": 0, "oldestLedgerCloseTime": "0", "ledgerRetentionWindow": 5 }
+```
+
+The node retains every ledger since genesis, so `oldestLedger` is always 0 and the retention window covers all ledgers so far. Per-ledger close times are not recorded: the latest close time is approximated by the request time and the oldest by the epoch.
 
 ### `getNetwork`
 
 ```json
-{ "friendbotUrl": null, "passphrase": "Test SDF Network ; September 2015", "protocolVersion": "22" }
+{ "passphrase": "Test SDF Network ; September 2015", "protocolVersion": 22 }
 ```
 
+`friendbotUrl` is omitted (not `null`) because the node runs no friendbot — matching real stellar-rpc's `omitempty` serialization.
+
 ### `getLatestLedger`
 
-`getLatestLedger` returns the current ledger sequence (the `latest_ledger` from `metadata.json`), which increments by 1 per successfully committed transaction.
+`getLatestLedger` returns the current ledger sequence (the `latest_ledger` from `metadata.json`), which increments by 1 per successfully committed transaction. The `id` is a deterministic per-ledger stand-in for the ledger-header hash, derived from the sequence (see `#ledgerId` in [node-semantics.md](node-semantics.md)).
 
 ```json
-{ "id": "0000...0000", "protocolVersion": "22", "sequence": 4 }
+{ "id": "1715609f7c74...", "protocolVersion": 22, "sequence": 4 }
 ```
 
 ### `sendTransaction`
@@ -111,9 +117,13 @@ All methods are answered by the K semantics and follow the [Stellar RPC specific
 
 **Response**:
 ```json
-{ "hash": "<64-char hex>", "status": "PENDING", "latestLedger": "5", "latestLedgerCloseTime": "1716000000" }
+{ "hash": "<64-char hex>", "status": "PENDING", "latestLedger": 5, "latestLedgerCloseTime": "1716000000" }
 ```
 
+Resubmitting a transaction whose hash already has a receipt returns status `DUPLICATE` without re-executing it: the ledger does not advance and the stored receipt is untouched.
+
+A transaction that decodes as XDR but cannot be processed (e.g. it contains an operation the semantics do not support) is rejected at admission time with status `ERROR` and an `errorResultXdr` carrying a `txMALFORMED` `TransactionResult` — mirroring how real stellar-rpc reports core's admission rejections. Such a transaction never reaches the ledger: no receipt is stored (`getTransaction` stays `NOT_FOUND`) and the ledger does not advance. Undecodable XDR remains a plain `-32602 Invalid params` error, as in real stellar-rpc. `TRY_AGAIN_LATER` is never returned: it signals mempool backpressure, and komet-node executes synchronously without a mempool, so the condition it reports cannot arise.
+
 ### `traceTransaction`
 
 `traceTransaction` retrieves the instruction trace of a previously submitted transaction. It takes a `hash` parameter (the same one `getTransaction` takes) and returns the trace that `sendTransaction` stored on that transaction's receipt. The result is the trace itself: a JSONL string with one record per executed WebAssembly instruction, or `null` when no transaction with that hash exists.
@@ -124,7 +134,7 @@ All methods are answered by the K semantics and follow the [Stellar RPC specific
 
 ### `getTransaction`
 
-`getTransaction` reads the hash's `receipts/receipt_<hash>.json` file.
+`getTransaction` reads the hash's `receipts/receipt_<hash>.json` file. The `hash` parameter must be a 64-character hex string; anything else is rejected with `-32602 Invalid params` (this and `traceTransaction` share the validation).
 
 | Status | Meaning |
 |---|---|
@@ -135,13 +145,17 @@ All methods are answered by the K semantics and follow the [Stellar RPC specific
 **`SUCCESS` response**:
 ```json
 {
-  "status": "SUCCESS", "ledger": "5", "createdAt": "1716000000",
+  "status": "SUCCESS", "applicationOrder": 1, "feeBump": false,
   "envelopeXdr": "<base64 XDR>", "resultXdr": "", "resultMetaXdr": "",
-  "latestLedger": "5", "latestLedgerCloseTime": "1716000000"
+  "ledger": 5, "createdAt": "1716000000",
+  "latestLedger": 5, "latestLedgerCloseTime": "1716000000",
+  "oldestLedger": 0, "oldestLedgerCloseTime": "0"
 }
 ```
 
-`resultXdr` and `resultMetaXdr` are currently empty stubs. The receipt carries no trace — use `traceTransaction` with the same hash to fetch it.
+The ledger-range fields (`latestLedger` through `oldestLedgerCloseTime`) are present on every response, `NOT_FOUND` included. Every ledger contains exactly one transaction, so `applicationOrder` is always 1; fee-bump envelopes are not supported, so `feeBump` is always false. `resultXdr` and `resultMetaXdr` are currently empty stubs. The receipt carries no trace — use `traceTransaction` with the same hash to fetch it.
+
+Receipts are an internal format, not a stable one: receipts written by older komet-node versions stored `ledger` as a string and lack `applicationOrder`/`feeBump`, and `getTransaction` returns stored receipts verbatim. Start from a fresh io-dir after upgrading rather than resuming one with old receipts.
 
 ---