The ATProto loader pipeline currently runs parseRecord → filter → groupBy → transform per record (or group of records). This means any operation that needs to see the whole collection (e.g. sorting it) has to be written again every time the collection is used, which offends me personally.
Instead, we should add an optional afterAll hook to both defineAtProtoLiveCollection and defineAtProtoCollection.
This would run after all transforms have completed, get an array with all the entries, and return the final shape of the data. Obviously, callers can then do any type of re-ordering of re-filtering they’d like, but at least one could have e.g. a guarantee that their posts are always returned in chronological order.
The ATProto loader pipeline currently runs
parseRecord → filter → groupBy → transformper record (or group of records). This means any operation that needs to see the whole collection (e.g. sorting it) has to be written again every time the collection is used, which offends me personally.Instead, we should add an optional
afterAllhook to bothdefineAtProtoLiveCollectionanddefineAtProtoCollection.This would run after all transforms have completed, get an array with all the entries, and return the final shape of the data. Obviously, callers can then do any type of re-ordering of re-filtering they’d like, but at least one could have e.g. a guarantee that their posts are always returned in chronological order.