Merge pull request #31 from runtimeverification/feat/rpc-plumbing

feat: xdrFormat param, JSON-RPC batch support, protocol docs

Author: RaoulSchaffranek

docs/architecture.md

@@ -51,7 +51,7 @@ flowchart TB
 
 → **[Detailed documentation](server.md)**
 
-The server implements six RPC methods — `getHealth`, `getNetwork`, `getLatestLedger`, `sendTransaction`, `getTransaction`, and `traceTransaction` — and the K semantics answer all of them.
+The server implements six RPC methods — `getHealth`, `getNetwork`, `getLatestLedger`, `sendTransaction`, `getTransaction`, and the komet-specific `traceTransaction` extension — and the K semantics answer all of them. JSON-RPC framing (single calls, batch arrays, notifications) is handled in Python; the semantics see one request envelope per invocation.
 
 `sendTransaction` always returns `PENDING` and clients poll `getTransaction` for the result — matching the Stellar RPC async pattern even though the transaction executes synchronously. See [server.md](server.md) for details.
 

docs/node-semantics.md

@@ -92,7 +92,7 @@ The trace is not part of the receipt — the executing steps already appended it
 
 ### traceTransaction
 
-`traceTransaction` is a read-only lookup. It takes a `hash` (the same parameter `getTransaction` takes) and responds with the contents of `traces/trace_<hash>.jsonl`, or `null` when no trace file exists for that hash. Because tracing is always on, every `sendTransaction` writes this file.
+`traceTransaction` is a komet-specific extension, not part of the Stellar RPC specification (see [server.md](server.md#tracetransaction-komet-specific-extension)). It is a read-only lookup: it takes a `hash` (the same parameter `getTransaction` takes) and responds with the contents of `traces/trace_<hash>.jsonl`, or `null` when no trace file exists for that hash. Because tracing is always on, every `sendTransaction` writes this file.
 
 ### Two ways steps are delivered
 

docs/notes.md

@@ -36,5 +36,6 @@ The tests do not yet cover `bytes` / `address` SCVal arguments or `SCVec` / `SCM
 - `resultXdr` / `resultMetaXdr` are empty stubs (contract return values not surfaced).
 - `sendTransaction`'s `errorResultXdr` is always a generic `txMALFORMED` result (no per-cause codes such as `txBAD_SEQ` — sequence numbers, fees, and signatures are not modelled), and `diagnosticEventsXdr` is never populated (both fields are optional in the spec). `TRY_AGAIN_LATER` is never returned by design: it signals mempool backpressure, which cannot arise in a synchronous node without a mempool.
 - `SCVec` / `SCMap` contract arguments are not yet encoded.
+- The `xdrFormat` parameter is accepted on `getTransaction` and `sendTransaction`, but only the default `'base64'` is supported; `'json'` is rejected with `-32602`.
 - `simulateTransaction`, `getEvents`, `getLedgerEntries`, `getFeeStats`, and TTL/footprint operations are not implemented.
 - Receipts are not a stable format: older io-dirs store `ledger`/`createdAt` with pre-spec types and lack `applicationOrder`/`feeBump`. Resume only io-dirs written by the same version, or start fresh.

docs/server.md

@@ -19,6 +19,20 @@ class StellarRpcServer:
 
 The server is a plain `http.server.HTTPServer` (not pyk's `JsonRpcServer`). A `BaseHTTPRequestHandler` reads each POST body and calls `_handle`, which parses the JSON-RPC frame and delegates to `handle_rpc`.
 
+### JSON-RPC framing
+
+The server implements the JSON-RPC 2.0 framing rules, including batch calls:
+
+- A single request object gets a single response object.
+- An **array** body is a batch: each element is validated and dispatched on its own, and the response is an array with one entry per answered element (matched by `id`; invalid elements each get an Invalid Request error with `id: null`). Batch elements run sequentially — the server is single-threaded by design, so a batch is equivalent to sending its elements one at a time. An empty array is answered with a single Invalid Request error object, per the spec.
+- A request without an `id` member is a **notification**: it is executed but never answered, not even with an error. A batch of only notifications (or a single notification) produces an empty response body.
+
+Framing happens entirely in Python (`_handle` / `_handle_batch` / `_handle_single`); the K semantics see one request envelope per invocation regardless of how requests were framed on the wire.
+
+### The `xdrFormat` parameter
+
+`getTransaction` and `sendTransaction` accept the spec's optional `xdrFormat` parameter. Only `'base64'`, the spec default, is supported: XDR fields in responses are always base64 strings. The alternative `'json'` format (XDR rendered as JSON objects) is not implemented and is rejected with error `-32602` and a message saying so; any other value is rejected with `-32602` as well. The check runs before anything else, so a `sendTransaction` with a bad `xdrFormat` is rejected without executing the transaction.
+
 ### `handle_rpc(method, params, request_id) -> str`
 
 `handle_rpc` is the dispatch entry point; it returns the JSON-RPC response envelope as a string. You can call it **without** the HTTP layer, which is convenient for scripts and tests:
@@ -83,7 +97,7 @@ Because these artifacts live on disk, the server can be stopped and restarted wi
 
 ## RPC methods
 
-All methods are answered by the K semantics and follow the [Stellar RPC specification](https://developers.stellar.org/docs/data/apis/rpc/methods), including its serialization quirks: ledger sequence numbers and `protocolVersion` are JSON numbers, while close-time fields (`latestLedgerCloseTime`, `oldestLedgerCloseTime`, `createdAt`) are int64s serialized as JSON strings holding a decimal integer, matching what real stellar-rpc emits.
+All methods are answered by the K semantics and follow the [Stellar RPC specification](https://developers.stellar.org/docs/data/apis/rpc/methods) — except `traceTransaction`, which is a komet-specific extension (see below) — including its serialization quirks: ledger sequence numbers and `protocolVersion` are JSON numbers, while close-time fields (`latestLedgerCloseTime`, `oldestLedgerCloseTime`, `createdAt`) are int64s serialized as JSON strings holding a decimal integer, matching what real stellar-rpc emits.
 
 ### `getHealth`
 
@@ -124,7 +138,9 @@ Resubmitting a transaction whose hash already has a receipt returns status `DUPL
 
 A transaction that decodes as XDR but cannot be processed (e.g. it contains an operation the semantics do not support) is rejected at admission time with status `ERROR` and an `errorResultXdr` carrying a `txMALFORMED` `TransactionResult` — mirroring how real stellar-rpc reports core's admission rejections. Such a transaction never reaches the ledger: no receipt is stored (`getTransaction` stays `NOT_FOUND`) and the ledger does not advance. Undecodable XDR remains a plain `-32602 Invalid params` error, as in real stellar-rpc. `TRY_AGAIN_LATER` is never returned: it signals mempool backpressure, and komet-node executes synchronously without a mempool, so the condition it reports cannot arise.
 
-### `traceTransaction`
+### `traceTransaction` (komet-specific extension)
+
+`traceTransaction` is **not part of the Stellar RPC specification** — it exists only on komet-node, and clients must not expect it from real Stellar RPC endpoints. It keeps its plain name rather than a vendor-prefixed one (`komet_traceTransaction`): the official spec has no method of that name and none is announced, so there is no collision to avoid, and renaming would break every existing client for no gain. If stellar-rpc ever claims the name, the method will be renamed with a prefix.
 
 `traceTransaction` retrieves the instruction trace of a previously submitted transaction. It takes a `hash` parameter (the same one `getTransaction` takes) and returns the trace that `sendTransaction` stored on that transaction's receipt. The result is the trace itself: a JSONL string with one record per executed WebAssembly instruction, or `null` when no transaction with that hash exists.
 

src/komet_node/server.py

@@ -29,6 +29,11 @@
 # trace stored on a previously executed transaction's receipt (see _read_only_envelope).
 _TX_METHODS: Final = ('sendTransaction',)
 
+# Methods whose spec accepts an optional `xdrFormat` param (protocols/rpc:
+# GetTransactionRequest.Format, SendTransactionRequest.Format). komet-node supports only
+# the default 'base64' format; see _check_xdr_format.
+_XDR_FORMAT_METHODS: Final = ('getTransaction', 'sendTransaction')
+
 _log = logging.getLogger('komet_node')
 
 
@@ -151,33 +156,66 @@ def shutdown(self) -> None:
     # ------------------------------------------------------------------
 
     def _handle(self, body: bytes) -> bytes:
-        """Parse a raw JSON-RPC body and return the response bytes (the HTTP entry point)."""
+        """Parse a raw JSON-RPC body and return the response bytes (the HTTP entry point).
+
+        An array body is a JSON-RPC 2.0 batch; anything else is a single call. The result
+        may be empty (no response body) when every request was a notification.
+        """
         try:
             req = json.loads(body.decode('utf-8'))
         except (json.JSONDecodeError, UnicodeDecodeError):
             return _error_bytes(None, -32700, 'Parse error')
-        if not isinstance(req, dict):
+        if isinstance(req, list):
+            return self._handle_batch(req)
+        response = self._handle_single(req)
+        return b'' if response is None else response.encode('utf-8')
+
+    def _handle_batch(self, batch: list[Any]) -> bytes:
+        """Answer a JSON-RPC 2.0 batch call: one response per element, in an array.
+
+        Per section 6 of the spec: an empty array is itself a single Invalid Request error;
+        invalid elements each get their own error response; notifications get no response
+        entry, and a batch of only notifications gets no response body at all. Elements run
+        sequentially (the server is single-threaded by design, see serve()).
+        """
+        if not batch:
             return _error_bytes(None, -32600, 'Invalid Request')
+        responses = [response for element in batch if (response := self._handle_single(element)) is not None]
+        if not responses:
+            return b''
+        return ('[' + ','.join(responses) + ']').encode('utf-8')
+
+    def _handle_single(self, req: Any) -> str | None:
+        """Validate one JSON-RPC request frame and dispatch it.
+
+        Returns the response as a JSON string, or ``None`` for a notification — a valid
+        request frame without an ``id`` member, which per JSON-RPC 2.0 is executed but
+        never answered, not even with an error.
+        """
+        if not isinstance(req, dict):
+            return _error_str(None, -32600, 'Invalid Request')
         request_id = req.get('id')
 
         # Validate the JSON-RPC frame before dispatch (JSON-RPC 2.0):
         #   - wrong/missing protocol version or a non-string method => Invalid Request
         #   - params, if present, must be a structured (object) value => else Invalid params
         if req.get('jsonrpc') != '2.0' or not isinstance(req.get('method'), str):
-            return _error_bytes(request_id, -32600, 'Invalid Request')
+            return _error_str(request_id, -32600, 'Invalid Request')
         params = req.get('params')
         if params is None:
             params = {}
-        elif not isinstance(params, dict):
-            return _error_bytes(request_id, -32602, 'Invalid params')
 
-        try:
-            return self.handle_rpc(req['method'], params, request_id).encode('utf-8')
-        except Exception:
-            # An unexpected error must never take down the server thread, but it must not
-            # vanish silently either — log the traceback before returning Internal error.
-            traceback.print_exc()
-            return _error_bytes(request_id, -32603, 'Internal error')
+        if not isinstance(params, dict):
+            response = _error_str(request_id, -32602, 'Invalid params')
+        else:
+            try:
+                response = self.handle_rpc(req['method'], params, request_id)
+            except Exception:
+                # An unexpected error must never take down the server thread, but it must
+                # not vanish silently either — log the traceback, return Internal error.
+                traceback.print_exc()
+                response = _error_str(request_id, -32603, 'Internal error')
+        return None if 'id' not in req else response
 
     def handle_rpc(self, method: str | None, params: dict[str, Any], request_id: Any = None) -> str:
         """Dispatch a single JSON-RPC call and return the response envelope as a JSON string.
@@ -188,6 +226,13 @@ def handle_rpc(self, method: str | None, params: dict[str, Any], request_id: Any
         _log.info('request: %s (id=%r)', method, request_id)
         self._archive_request(method, params, request_id)
 
+        # Reject an unsupported xdrFormat up front, before the request does anything —
+        # in particular before a sendTransaction executes and commits state.
+        if method in _XDR_FORMAT_METHODS:
+            format_error = _check_xdr_format(params, request_id)
+            if format_error is not None:
+                return format_error
+
         if method in _TX_METHODS:
             transaction = params.get('transaction')
             if not isinstance(transaction, str):
@@ -346,6 +391,22 @@ def _configure_logging() -> None:
     _log.setLevel(logging.INFO)
 
 
+def _check_xdr_format(params: dict[str, Any], request_id: Any) -> str | None:
+    """Validate the optional ``xdrFormat`` param; ``None`` means the request may proceed.
+
+    Only ``'base64'`` (the spec default) is supported. The spec's alternative ``'json'``
+    format is not implemented in komet-node, so it gets a dedicated error message; any
+    other value is rejected as invalid. Both cases are Invalid params (-32602), matching
+    real stellar-rpc's handling of a bad format value.
+    """
+    xdr_format = params.get('xdrFormat', 'base64')
+    if xdr_format == 'base64':
+        return None
+    if xdr_format == 'json':
+        return _error_str(request_id, -32602, "Invalid params: xdrFormat 'json' is not supported, use 'base64'")
+    return _error_str(request_id, -32602, "Invalid params: unknown xdrFormat, expected 'base64'")
+
+
 def _error_str(rpc_id: Any, code: int, message: str) -> str:
     return json.dumps({'jsonrpc': '2.0', 'id': rpc_id, 'error': {'code': code, 'message': message}})