NIP-67: EOSE Completeness Hint (#2317)

Author: mattn

67.md

@@ -0,0 +1,91 @@
+NIP-67
+======
+
+EOSE Completeness Hint
+----------------------
+
+`draft` `optional`
+
+This NIP extends the `EOSE` message defined in [NIP-01](01.md) with an optional third element that signals the client that the relay has sent every stored event matching the subscription's filters.
+
+## Motivation
+
+`EOSE` as defined in NIP-01 signals the boundary between stored and real-time events. It does not tell the client whether there are more stored events the relay chose not to send. Relays commonly enforce an internal cap (e.g. 500 events per filter) that is independent of the client's `limit`. Clients today guess completeness by comparing the number of events received to the `limit` they requested:
+
+- If fewer than `limit` events arrive, assume completion.
+- Otherwise, issue another `REQ` with `until` set to the oldest received event's `created_at`, and repeat.
+
+This heuristic has two well-known problems:
+
+1. When the relay's internal cap is lower than the client's `limit` (e.g. cap 300, client asks for 500), the client receives 300 events and incorrectly concludes the result is complete, silently missing data.
+2. When the total number of matching events happens to equal the relay's cap, the client cannot tell and must issue an additional `REQ` just to observe zero results. A relay that caps at 300 events will always be queried at least twice for any subscription that exhausts the cap.
+
+A single extra field on `EOSE` removes this ambiguity.
+
+## Specification
+
+A relay that implements this NIP MAY append a third element to the `EOSE` message: an array of hint strings.
+
+```
+["EOSE", <subscription_id>, [<hint>, ...]]
+```
+
+Currently two hint values are defined:
+
+- `"finish"`: the relay has sent every stored event matching the subscription's filters. The client SHOULD NOT paginate this subscription further.
+- `"more"`: the relay holds more matching stored events than it has sent. The client SHOULD paginate to retrieve them. A relay MAY send this hint, but is not required to, since detecting whether more events exist is not cheap on every storage backend.
+
+The array MAY carry multiple hints simultaneously, and MAY be empty. Clients MUST ignore unknown hint values without error.
+
+Their presence is definitive; their absence is not. When a relay omits the third element, or sends neither `"finish"` nor `"more"`, the client SHOULD paginate using `until = oldest received event's created_at` as usual.
+
+Clients that do not implement this NIP will ignore the extra element, as JSON arrays with trailing elements are accepted by conforming parsers and existing implementations index `EOSE` by position. Relays that do not implement this NIP continue to send `EOSE` with two elements and clients fall back to the legacy heuristic described above.
+
+The hint refers only to stored events. It has no effect on the real-time delivery of new events, which continues under the regular NIP-01 rules until the client sends `CLOSE` or the relay sends `CLOSED`.
+
+Future revisions of this NIP may define additional hint values. Implementations SHOULD accept unknown hint values without error and treat them as if absent.
+
+## Semantics around `created_at` ties
+
+When a relay does not send `"finish"`, multiple events may share the oldest `created_at` value in the response. Clients paginating with `until = oldest_created_at` risk missing events that share that timestamp. Relays SHOULD, when possible, advance the internal cursor so that all events with the boundary `created_at` are included in one response, emitting `"finish"` only when there are no older events remaining.
+
+## Relay advertisement
+
+Relays implementing this NIP SHOULD include `67` in the `supported_nips` field of their [NIP-11](11.md) document.
+
+## Examples
+
+Relay has more matching events than its internal cap (legacy-shaped response, client paginates):
+
+```
+["REQ", "sub1", {"authors": ["abcd..."], "kinds": [1]}]
+["EVENT", "sub1", {...}]
+... (300 events total)
+["EOSE", "sub1"]
+```
+
+Relay has sent every matching stored event:
+
+```
+["REQ", "sub2", {"ids": ["abcd..."]}]
+["EVENT", "sub2", {...}]
+["EOSE", "sub2", ["finish"]]
+```
+
+Relay has more matching events and says so explicitly:
+
+```
+["REQ", "sub2b", {"authors": ["abcd..."], "kinds": [1]}]
+["EVENT", "sub2b", {...}]
+... (300 events total)
+["EOSE", "sub2b", ["more"]]
+```
+
+Relay does not implement this NIP (legacy behavior — client falls back to heuristic):
+
+```
+["REQ", "sub3", {"kinds": [1], "limit": 500}]
+["EVENT", "sub3", {...}]
+... (N events)
+["EOSE", "sub3"]
+```

README.md

@@ -82,7 +82,8 @@ NIPs listed here are not a protocol checklist. Nothing forces any software to im
 - [NIP-62: Request to Vanish](62.md)
 - [NIP-64: Chess (PGN)](64.md)
 - [NIP-65: Relay List Metadata](65.md)
-- [NIP-66: Relay Liveness Monitoring](66.md)
+- [NIP-66: Relay Discovery and Liveness Monitoring](66.md)
+- [NIP-67: EOSE Completeness Hint](67.md)
 - [NIP-68: Picture-first feeds](68.md)
 - [NIP-69: Peer-to-peer Order events](69.md)
 - [NIP-70: Protected Events](70.md)