Merge branch 'main' into dependabot/uv/setuptools-83.0.0

Author: RaoulSchaffranek

README.md

@@ -123,7 +123,7 @@ curl -s http://localhost:8000 -H 'Content-Type: application/json' \
   -d '{"jsonrpc":"2.0","id":1,"method":"traceTransaction","params":{"hash":"c7099cbe10a9bfa1cdf9c9d368e1e1c932f535a70e4403b7aa409ce19fc36805"}}'
 ```
 
-`traceTransaction` returns the stored trace as its result. The trace is a JSONL string (one JSON record per executed WebAssembly instruction); it is shown decoded here for readability:
+`traceTransaction` returns the stored trace as its result: a JSON array with one record per executed WebAssembly instruction.
 
 ```jsonc
 {

docs/architecture.md

@@ -51,7 +51,7 @@ flowchart TB
 
 → **[Detailed documentation](server.md)**
 
-The server implements eight RPC methods — `getHealth`, `getNetwork`, `getLatestLedger`, `getVersionInfo`, `getFeeStats`, `sendTransaction`, `getTransaction`, and the komet-specific `traceTransaction` extension — and the K semantics answer all of them. JSON-RPC framing (single calls, batch arrays, notifications) is handled in Python; the semantics see one request envelope per invocation.
+The server implements eleven RPC methods — `getHealth`, `getNetwork`, `getLatestLedger`, `getVersionInfo`, `getFeeStats`, `sendTransaction`, `getTransaction`, `getTransactions`, `getLedgers`, `getLedgerEntries`, and the komet-specific `traceTransaction` extension — and the K semantics answer all of them. JSON-RPC framing (single calls, batch arrays, notifications) is handled in Python; the semantics see one request envelope per invocation. For `getLedgerEntries` the semantics' answer is an intermediate shape: K performs the state lookups, and `ledger_entries.py` translates between the base64 XDR wire format (`LedgerKey` in, `LedgerEntryData` out) and the JSON the semantics exchange, since K cannot parse or produce XDR.
 
 `sendTransaction` always returns `PENDING` and clients poll `getTransaction` for the result — matching the Stellar RPC async pattern even though the transaction executes synchronously. See [server.md](server.md) for details.
 
@@ -89,20 +89,24 @@ All of the server's input and output artifacts live in one directory, the *io di
 |---|---|---|---|
 | `state.kore` | persistent | `NodeInterpreter` | the full K world-state configuration — accounts, contract code (including uploaded wasm `ModuleDecl`s), contract storage, ledger metadata — serialized in KORE. Read before each run and rewritten after a successful one. |
 | `metadata.json` | persistent | the K semantics | `{"latest_ledger": N}` — the server ledger counter, bumped by 1 per committed transaction. |
-| `receipts/receipt_<hash>.json` | persistent | the semantics (on success) or the server (on failure) | one stored receipt per transaction, keyed by tx hash, answering `getTransaction`. Each is `{status, ledger, createdAt, envelopeXdr, resultXdr, resultMetaXdr}`. |
-| `traces/trace_<hash>.jsonl` | persistent | the semantics | one execution trace per transaction, keyed by tx hash — the instruction-level records, one JSON object per line. `traceTransaction` returns this file's contents. |
+| `receipts/receipt_<hash>.json` | persistent | the semantics and the server (on success — the server rewrites the semantics' internal `returnValue` field into `resultXdr`/`resultMetaXdr`), or the server alone (on failure) | one stored receipt per transaction, keyed by tx hash, answering `getTransaction`. Each is `{status, ledger, createdAt, envelopeXdr, resultXdr, resultMetaXdr?}`. |
+| `traces/trace_<hash>.jsonl` | persistent | the semantics | one execution trace per transaction, keyed by tx hash — the instruction-level records, one JSON object per line. `traceTransaction` returns these records as a JSON array. |
+| `ledgers/ledger_<seq>.json` | persistent | the server (on success) | one record per closed ledger — `{sequence, txHash, closedAt, hash, headerXdr, metadataXdr}` — the ledger→transaction index behind `getTransactions` and `getLedgers`. Written in Python because the header artifacts are XDR, which K cannot construct. |
 | `requests/request_<n>.json` | persistent | the server | an archive of each incoming JSON-RPC request, numbered by a monotonic counter, kept for debugging. |
+| `wasms/<hash>.wasm` | persistent | the server | the raw bytes of each successfully uploaded wasm module, keyed by hex sha256. The K state stores modules parsed (`ModuleDecl`), so `getLedgerEntries` CONTRACT_CODE entries read the original bytes from here. |
+| `events/events_<ledger>.json` | persistent | the server | one JSON array per ledger with the contract events emitted in it, already in the spec's Event shape. Built by the server from the staged records after a successful transaction; read back by the K `getEvents` rules. |
 | `request.json` | transient | the server | the request envelope for the call in flight (`method`, `id`, `now`, and method-specific fields). The semantics remove it once they respond. |
 | `response.json` | transient | the semantics | the JSON-RPC response (`{jsonrpc, id, result}`) for the most recent call. The server reads it back; it is absent when a transaction gets stuck. |
+| `events_staged.jsonl` | transient | the K semantics | the contract events of the transaction in flight, one JSON record per `contract_event` call. The server converts them into `events/events_<ledger>.json` after a successful run and removes the file. |
 
-Receipts, traces, and request archives are split into one file per item — keyed by tx hash, or numbered — so that no single file grows without bound as the chain advances. The server creates the `receipts/`, `traces/`, and `requests/` directories before the semantics run, because the K file-system hooks open files with POSIX `open()`, which does not create parent directories.
+Receipts, traces, ledger records, events, and request archives are split into one file per item — keyed by tx hash, ledger sequence, or a counter — so that no single file grows without bound as the chain advances. The server creates the `receipts/`, `traces/`, `ledgers/`, `requests/`, `wasms/`, and `events/` directories before the semantics run, because the K file-system hooks open files with POSIX `open()`, which does not create parent directories.
 
 The world state stays in KORE (rather than a JSON snapshot) because an uploaded wasm module is a `ModuleDecl` that the semantics cannot reconstruct from bytes — only `wasm2kast` (Python) can produce it. The receipts and the ledger counter, by contrast, are plain data and live in JSON files, which the semantics read and write directly via the file-system hooks.
 
 ```mermaid
 flowchart TB
     boot(["server start"]) --> exists{"state.kore exists?"}
-    exists -->|"no"| init["empty_config() builds the idle K config in KORE<br/>write state.kore · seed metadata.json {latest_ledger: 0}<br/>create receipts/ traces/ requests/"]
+    exists -->|"no"| init["empty_config() builds the idle K config in KORE<br/>write state.kore · seed metadata.json {latest_ledger: 0}<br/>create receipts/ traces/ requests/ wasms/"]
     exists -->|"yes"| reuse["use existing state.kore<br/>seed metadata.json if missing · ensure artifact dirs exist"]
     init --> ready(["ready for requests"])
     reuse --> ready
@@ -175,8 +179,7 @@ sequenceDiagram
 
 ## What's not yet implemented
 
-- `resultXdr` / `resultMetaXdr` in `getTransaction` responses (contract return values)
-- `simulateTransaction` (dry-run without state mutation)
-- `getEvents`, `getLedgerEntries`, `getTransactions`, `getLedgers` and other read-only RPC methods
+- `simulateTransaction` of upload/deploy host functions, auth recording, resource metering, `events`/`restorePreamble`/`stateChanges` (only invoke-contract dry runs with the return value are supported)
+- `system` events in `getEvents` (the semantics have no source of them; contract events are supported)
 - `ExtendFootprintTTL` and `RestoreFootprint` operations
 - `SCVec` / `SCMap` contract-argument types in the request encoder (`scval_to_json`)

docs/node-semantics.md

@@ -17,6 +17,9 @@ The semantics communicate with the Python process through files in the working d
 | `metadata.json` | K ↔ K | `{"latest_ledger": N}` — the ledger counter |
 | `receipts/receipt_<hash>.json` | K → Python | one stored receipt per transaction, keyed by tx hash |
 | `traces/trace_<hash>.jsonl` | K → Python | one execution trace per transaction (per-instruction records), keyed by tx hash |
+| `ledgers/ledger_<seq>.json` | Python → K | one record per closed ledger (`sequence`, `txHash`, `closedAt`, and the ledger-header XDR artifacts), written by the server and read back to serve `getTransactions`/`getLedgers` |
+| `events_staged.jsonl` | K → Python | the contract events of the transaction in flight, one JSON record per `contract_event` call |
+| `events/events_<ledger>.json` | Python → K | one finished JSON array of Event objects per ledger, served by `getEvents` |
 
 ---
 
@@ -40,7 +43,7 @@ insert-handleRequestFile → handleRequestFile
 #dispatchMethod(method, request)        ← routes on the "method" field
          │
          ├─ getHealth / getNetwork / getLatestLedger / getVersionInfo / getFeeStats
-         │    / getTransaction / traceTransaction → #respond(...)
+         │    / getTransaction / getTransactions / getLedgers / getLedgerEntries / getEvents / traceTransaction → #respond(...)
          │
          └─ sendTransaction → #runTx → run steps
                 → #finalizeTx → record receipt + bump ledger → #respond(...)
@@ -65,6 +68,9 @@ If `request.json` is absent, `insert-handleRequestFile` does not fire and K halt
 - `getVersionInfo` → echoes the version fields the server put into the envelope (`version`, `commitHash`, `buildTimestamp`, `captiveCoreVersion` — package metadata lives on the Python side); `protocolVersion` is a JSON number
 - `getFeeStats` → constant `#feeDistribution` objects (komet-node has no fee market) for `sorobanInclusionFee`/`inclusionFee`, plus a live `latestLedger` number from `metadata.json`; all distribution fields except `ledgerCount` are decimal strings, matching real stellar-rpc's Go `,string` encoding
 - `getTransaction` → reads the hash's `receipts/receipt_<hash>.json` file; returns the stored receipt merged with the ledger-range fields (`latestLedger`/`latestLedgerCloseTime`/`oldestLedger`/`oldestLedgerCloseTime`), or `{ "status": "NOT_FOUND", ... }` (with the same ledger-range fields) when the file is absent
+- `getTransactions` / `getLedgers` → walk the per-ledger index files `ledgers/ledger_<seq>.json` from the envelope's `startSeq` up to the latest ledger, taking at most `limit` records (`#txInfos` / `#ledgerInfos`), and format the history page (`#txHistoryPage` / `#ledgerHistoryPage`). Parameter validation and cursor resolution happen in the server; the response `cursor` (`#pageCursor`) names the page's last record when the page is full — a TOID for transactions, the plain sequence for ledgers — and is empty otherwise. Serialization matches real stellar-rpc, including its quirks: per-transaction `createdAt` is a JSON number (unlike singular `getTransaction`), per-ledger `ledgerCloseTime` is a decimal string, and empty `resultXdr`/`resultMetaXdr` stubs are omitted (`#optXdrEntry`)
+- `getLedgerEntries` → looks each *key descriptor* of the request up in the world-state cells (`<accounts>`, `<contracts>`, `<contractData>`, `<contractCodes>`) and responds with `{ "entries": [...], "latestLedger": <int> }`. The server decodes the base64 `LedgerKey` XDR into the descriptors beforehand and re-encodes the found entries as `LedgerEntryData` XDR afterwards (`ledger_entries.py`) — the entries in K's response are an intermediate JSON shape (per-kind payloads such as `balance`, `wasmHash`, or an ScVal value serialised by `#scVal2JSON`, the inverse of `#decodeArg`). Keys that match nothing are skipped via an `[owise]` rule — per the spec they are not an error
+- `getEvents` → scans the `events/events_<ledger>.json` files of the requested window, applies the request's filters (type, contract ids, topic matchers with `*`/`**`), and paginates; returns `{ "latestLedger": ..., "events": [...], "cursor": ... }`. The events themselves were captured during `sendTransaction`: a rule in `node.md` shadows the upstream no-op `contract_event` host function, resolves the topics/data host objects, and stages them in `events_staged.jsonl`; the Python server then produces the finished per-ledger files (base64 SCVal XDR and strkey ids are XDR work K cannot do). A `startLedger` beyond the chain tip is answered with `#respondError` (JSON-RPC `-32600`)
 
 `#respond(ID, RESULT)` is the shared terminal: it writes the JSON-RPC envelope to `response.json`, removes `request.json`, and sets the exit code to 0. `#respondError(ID, CODE, MSG)` is its error counterpart, writing `{jsonrpc, id, error: {code, message}}` instead; the unknown-method fallback uses it to answer `-32601 Method not found` (a safety net — the Python server filters unknown methods before they reach K).
 
@@ -86,16 +92,20 @@ If `request.json` is absent, `insert-handleRequestFile` does not fire and K halt
 
 1. writes `metadata.json` with `latest_ledger + 1`,
 2. writes the receipt to `receipts/receipt_<hash>.json`:
-   `{ status: "SUCCESS", applicationOrder: 1, feeBump: false, envelopeXdr, resultXdr: "", resultMetaXdr: "", ledger, createdAt }`,
+   `{ status: "SUCCESS", applicationOrder: 1, feeBump: false, envelopeXdr, ledger, createdAt, returnValue }` — `returnValue` is
+   the contract call's return `ScVal`, JSON-encoded by `#scValToJSON` (or `null` when the
+   transaction made no contract call), read off the `<hostStack>` before the host is reset,
 3. responds with `{hash, status: "PENDING", latestLedger, latestLedgerCloseTime}`.
 
-A `sendTransaction` whose hash already has a receipt never reaches `#runTx`: dispatch answers `{hash, status: "DUPLICATE", latestLedger, latestLedgerCloseTime}` directly, without executing anything or advancing the ledger. (For wasm uploads the Python server also suppresses the `<program>` injection on duplicates, since those steps would run before dispatch.)
+The internal `returnValue` field never reaches an RPC client: the Python server immediately rewrites it into the spec-mandated `resultXdr`/`resultMetaXdr` base64 XDR fields (`_attach_result_xdr` in `server.py`), because K cannot construct XDR. To keep the return value observable here, `uncheckedCallTx` (unlike komet's `callTx`) does not `#resetHost` after the call; `#recordAndRespond` serialises the stack top and resets the host instead.
+
+A `sendTransaction` whose hash already has a receipt never reaches `#runTx`: dispatch answers `{hash, status: "DUPLICATE", latestLedger, latestLedgerCloseTime}` directly, without executing anything or advancing the ledger. (For wasm uploads the Python server also suppresses the `<program>` injection on duplicates, since those steps would run before dispatch.) The Python `_attach_result_xdr` rewrite is skipped for duplicates, so a resubmission never disturbs the stored receipt.
 
 The trace is not part of the receipt — the executing steps already appended it to `traces/trace_<hash>.jsonl`. Reaching `#finalizeTx` means the steps completed without getting stuck, so the status is `SUCCESS`. A failed transaction gets stuck before this point, `response.json` is never written, and the Python server records the `FAILED` receipt instead.
 
 ### traceTransaction
 
-`traceTransaction` is a komet-specific extension, not part of the Stellar RPC specification (see [server.md](server.md#tracetransaction-komet-specific-extension)). It is a read-only lookup: it takes a `hash` (the same parameter `getTransaction` takes) and responds with the contents of `traces/trace_<hash>.jsonl`, or `null` when no trace file exists for that hash. Because tracing is always on, every `sendTransaction` writes this file.
+`traceTransaction` is a komet-specific extension, not part of the Stellar RPC specification (see [server.md](server.md#tracetransaction-komet-specific-extension)). It is a read-only lookup: it takes a `hash` (the same parameter `getTransaction` takes) and responds with the contents of `traces/trace_<hash>.jsonl` as a JSON array (one record per executed instruction), or `null` when no trace file exists for that hash. Rather than decoding and re-encoding each record — a full round-trip whose cost scales with the (potentially very large) trace — the semantics trust that the file the tracer wrote is already valid JSON and build the array by string concatenation, joining the lines with commas inside `[` … `]`. Because tracing is always on, every `sendTransaction` writes this file.
 
 ### Two ways steps are delivered
 
@@ -110,7 +120,8 @@ The trace is not part of the receipt — the executing steps already appended it
 
 - `#getJSON(key, obj[, default])`, `#getString(key, obj)`, `#getInt(key, obj)` — read a field
 - `#concatJSONs(a, b)` — append object entries (used to merge `latestLedger` fields into a stored receipt)
-- `#receiptFile(hash)`, `#traceFile(hash)` — build the per-transaction file paths (`receipts/receipt_<hash>.json`, `traces/trace_<hash>.jsonl`)
+- `#receiptFile(hash)`, `#traceFile(hash)`, `#ledgerFile(seq)` — build the per-item file paths (`receipts/receipt_<hash>.json`, `traces/trace_<hash>.jsonl`, `ledgers/ledger_<seq>.json`)
+- `#readJSONFile(path)`, `#asInt(json)`, `#lengthJSONs(list)`, `#lastIntIn(key, list)` — small conveniences for the history methods
 
 These complement the **order-sensitive** step decoders below.
 
@@ -146,6 +157,10 @@ rule HexBytes(S)  => Int2Bytes(lengthString(S) /Int 2, String2Base(S, 16), BE)
   requires lengthString(S) >Int 0
 ```
 
+### `Bytes2Hex(Bytes) → String`
+
+The inverse direction is K's built-in `Bytes2Hex` (hook `BYTES.bytes2hex`): it encodes `Bytes` as a lowercase, zero-padded hex string. The `getLedgerEntries` rules use it to report hashes, addresses, and `ScBytes` values to the server.
+
 ### `string2WasmToken(String) → WasmStringToken`
 
 `string2WasmToken` wraps a K `String` into a `WasmStringToken` (`hook(STRING.string2token)`). It is required because `callTx` expects a `WasmString` for the function name.