Add more comments

Author: fabioh8010

API-INTERNAL.md

@@ -17,9 +17,6 @@
 <dt><a href="#getDeferredInitTask">getDeferredInitTask()</a></dt>
 <dd><p>Getter - returns the deffered init task.</p>
 </dd>
-<dt><a href="#getEvictionBlocklist">getEvictionBlocklist()</a></dt>
-<dd><p>Getter - returns the eviction block list.</p>
-</dd>
 <dt><a href="#getSkippableCollectionMemberIDs">getSkippableCollectionMemberIDs()</a></dt>
 <dd><p>Getter - returns the skippable collection member IDs.</p>
 </dd>
@@ -71,9 +68,6 @@ is associated with a collection of keys.</p>
 <dd><p>Checks to see if a provided key is the exact configured key of our connected subscriber
 or if the provided key is a collection member key (in case our configured key is a &quot;collection key&quot;)</p>
 </dd>
-<dt><a href="#isEvictableKey">isEvictableKey()</a></dt>
-<dd><p>Checks to see if this key has been flagged as safe for removal.</p>
-</dd>
 <dt><a href="#getCollectionKey">getCollectionKey(key)</a> ⇒</dt>
 <dd><p>Extracts the collection identifier of a given collection member key.</p>
 <p>For example:</p>
@@ -88,20 +82,6 @@ or if the provided key is a collection member key (in case our configured key is
 <dd><p>Tries to get a value from the cache. If the value is not present in cache it will return the default value or undefined.
 If the requested key is a collection, it will return an object with all the collection members.</p>
 </dd>
-<dt><a href="#removeLastAccessedKey">removeLastAccessedKey()</a></dt>
-<dd><p>Remove a key from the recently accessed key list.</p>
-</dd>
-<dt><a href="#addLastAccessedKey">addLastAccessedKey()</a></dt>
-<dd><p>Add a key to the list of recently accessed keys. The least
-recently accessed key should be at the head and the most
-recently accessed key at the tail.</p>
-</dd>
-<dt><a href="#addEvictableKeysToRecentlyAccessedList">addEvictableKeysToRecentlyAccessedList()</a></dt>
-<dd><p>Take all the keys that are safe to evict and add them to
-the recently accessed list when initializing the app. This
-enables keys that have not recently been accessed to be
-removed.</p>
-</dd>
 <dt><a href="#keysChanged">keysChanged()</a></dt>
 <dd><p>When a collection of keys change, search for any callbacks matching the collection key and trigger those callbacks</p>
 </dd>
@@ -144,8 +124,15 @@ whatever it is we attempted to do.</p>
 This method transforms an object like {&#39;@MyApp_user&#39;: myUserValue, &#39;@MyApp_key&#39;: myKeyValue}
 to an array of key-value pairs in the above format and removes key-value pairs that are being set to null</p>
 </dd>
-<dt><a href="#mergeChanges">mergeChanges(changes)</a></dt>
-<dd><p>Merges an array of changes with an existing value or creates a single change</p>
+<dt><a href="#mergeChanges">mergeChanges(changes, existingValue)</a></dt>
+<dd><p>Merges an array of changes with an existing value or creates a single change.</p>
+</dd>
+<dt><a href="#mergeAndMarkChanges">mergeAndMarkChanges(changes, existingValue)</a></dt>
+<dd><p>Merges an array of changes with an existing value or creates a single change.
+It will also mark deep nested objects that need to be entirely replaced during the merge.</p>
+</dd>
+<dt><a href="#applyMerge">applyMerge(changes, existingValue)</a></dt>
+<dd><p>Merges an array of changes with an existing value or creates a single change.</p>
 </dd>
 <dt><a href="#initializeWithDefaultKeyStates">initializeWithDefaultKeyStates()</a></dt>
 <dd><p>Merge user provided default key value pairs.</p>
@@ -187,12 +174,6 @@ Getter - returns the default key states.
 ## getDeferredInitTask()
 Getter - returns the deffered init task.
 
-**Kind**: global function  
-<a name="getEvictionBlocklist"></a>
-
-## getEvictionBlocklist()
-Getter - returns the eviction block list.
-
 **Kind**: global function  
 <a name="getSkippableCollectionMemberIDs"></a>
 
@@ -217,7 +198,7 @@ Sets the initial values for the Onyx store
 | --- | --- |
 | keys | `ONYXKEYS` constants object from Onyx.init() |
 | initialKeyStates | initial data to set when `init()` and `clear()` are called |
-| evictableKeys | This is an array of keys (individual or collection patterns) that are eligible for automatic removal when storage limits are reached. |
+| evictableKeys | This is an array of keys (individual or collection patterns) that when provided to Onyx are flagged as "safe" for removal. |
 
 <a name="maybeFlushBatchUpdates"></a>
 
@@ -313,12 +294,6 @@ or throws an Error if the key is not a collection one.
 Checks to see if a provided key is the exact configured key of our connected subscriber
 or if the provided key is a collection member key (in case our configured key is a "collection key")
 
-**Kind**: global function  
-<a name="isEvictableKey"></a>
-
-## isEvictableKey()
-Checks to see if this key has been flagged as safe for removal.
-
 **Kind**: global function  
 <a name="getCollectionKey"></a>
 
@@ -344,29 +319,6 @@ For example:
 Tries to get a value from the cache. If the value is not present in cache it will return the default value or undefined.
 If the requested key is a collection, it will return an object with all the collection members.
 
-**Kind**: global function  
-<a name="removeLastAccessedKey"></a>
-
-## removeLastAccessedKey()
-Remove a key from the recently accessed key list.
-
-**Kind**: global function  
-<a name="addLastAccessedKey"></a>
-
-## addLastAccessedKey()
-Add a key to the list of recently accessed keys. The least
-recently accessed key should be at the head and the most
-recently accessed key at the tail.
-
-**Kind**: global function  
-<a name="addEvictableKeysToRecentlyAccessedList"></a>
-
-## addEvictableKeysToRecentlyAccessedList()
-Take all the keys that are safe to evict and add them to
-the recently accessed list when initializing the app. This
-enables keys that have not recently been accessed to be
-removed.
-
 **Kind**: global function  
 <a name="keysChanged"></a>
 
@@ -460,7 +412,6 @@ whatever it is we attempted to do.
 Notifies subscribers and writes current value to cache
 
 **Kind**: global function  
-**Returns**: The value without null values and a boolean "wasRemoved", which indicates if the key got removed completely  
 <a name="prepareKeyValuePairsForStorage"></a>
 
 ## prepareKeyValuePairsForStorage() ⇒
@@ -472,14 +423,40 @@ to an array of key-value pairs in the above format and removes key-value pairs t
 **Returns**: an array of key - value pairs <[key, value]>  
 <a name="mergeChanges"></a>
 
-## mergeChanges(changes)
-Merges an array of changes with an existing value or creates a single change
+## mergeChanges(changes, existingValue)
+Merges an array of changes with an existing value or creates a single change.
+
+**Kind**: global function  
+
+| Param | Description |
+| --- | --- |
+| changes | Array of changes that should be merged |
+| existingValue | The existing value that should be merged with the changes |
+
+<a name="mergeAndMarkChanges"></a>
+
+## mergeAndMarkChanges(changes, existingValue)
+Merges an array of changes with an existing value or creates a single change.
+It will also mark deep nested objects that need to be entirely replaced during the merge.
+
+**Kind**: global function  
+
+| Param | Description |
+| --- | --- |
+| changes | Array of changes that should be merged |
+| existingValue | The existing value that should be merged with the changes |
+
+<a name="applyMerge"></a>
+
+## applyMerge(changes, existingValue)
+Merges an array of changes with an existing value or creates a single change.
 
 **Kind**: global function  
 
 | Param | Description |
 | --- | --- |
-| changes | Array of changes that should be applied to the existing value |
+| changes | Array of changes that should be merged |
+| existingValue | The existing value that should be merged with the changes |
 
 <a name="initializeWithDefaultKeyStates"></a>
 

API.md

@@ -28,7 +28,7 @@ Values of type <code>Object</code> get merged with the old value, whilst for <co
 applied in the order they were called. Note: <code>Onyx.set()</code> calls do not work this way so use caution when mixing
 <code>Onyx.merge()</code> and <code>Onyx.set()</code>.</p>
 </dd>
-<dt><a href="#mergeCollection">mergeCollection(collectionKey, collection)</a></dt>
+<dt><a href="#mergeCollection">mergeCollection(collectionKey, collection, mergeReplaceNullPatches)</a></dt>
 <dd><p>Merges a collection based on their keys</p>
 </dd>
 <dt><a href="#clear">clear(keysToPreserve)</a></dt>
@@ -157,7 +157,7 @@ Onyx.merge(ONYXKEYS.POLICY, {name: 'My Workspace'}); // -> {id: 1, name: 'My Wor
 ```
 <a name="mergeCollection"></a>
 
-## mergeCollection(collectionKey, collection)
+## mergeCollection(collectionKey, collection, mergeReplaceNullPatches)
 Merges a collection based on their keys
 
 **Kind**: global function  
@@ -166,6 +166,7 @@ Merges a collection based on their keys
 | --- | --- |
 | collectionKey | e.g. `ONYXKEYS.COLLECTION.REPORT` |
 | collection | Object collection keyed by individual collection member keys and values |
+| mergeReplaceNullPatches | Record where the key is a collection member key and the value is a list of tuples that we'll use to replace the nested objects of that collection member record with something else. |
 
 **Example**  
 ```js

lib/Onyx.ts

@@ -356,6 +356,8 @@ function merge<TKey extends OnyxKey>(key: TKey, changes: OnyxMergeInput<TKey>):
  *
  * @param collectionKey e.g. `ONYXKEYS.COLLECTION.REPORT`
  * @param collection Object collection keyed by individual collection member keys and values
+ * @param mergeReplaceNullPatches Record where the key is a collection member key and the value is a list of
+ * tuples that we'll use to replace the nested objects of that collection member record with something else.
  */
 function mergeCollection<TKey extends CollectionKeyBase, TMap>(
     collectionKey: TKey,

lib/OnyxMerge/index.native.ts

@@ -31,6 +31,8 @@ const applyMerge: ApplyMerge = <TKey extends OnyxKey>(key: TKey, existingValue:
         return Promise.resolve({mergedValue, updatePromise});
     }
 
+    // For native platforms we use `mergeItem` that will take advantage of JSON_PATCH and JSON_REPLACE SQL operations to
+    // merge the object in a performant way.
     return Storage.mergeItem(key, batchedChanges as OnyxValue<TKey>, replaceNullPatches).then(() => ({
         mergedValue,
         updatePromise,

lib/OnyxMerge/index.ts

@@ -23,6 +23,7 @@ const applyMerge: ApplyMerge = <TKey extends OnyxKey>(key: TKey, existingValue:
         return Promise.resolve({mergedValue, updatePromise});
     }
 
+    // For web platforms we use `setItem` since the object was already merged with its changes before.
     return Storage.setItem(key, mergedValue as OnyxValue<TKey>).then(() => ({
         mergedValue,
         updatePromise,

lib/OnyxUtils.ts

@@ -1147,8 +1147,8 @@ function hasPendingMergeForKey(key: OnyxKey): boolean {
  * Storage expects array like: [["@MyApp_user", value_1], ["@MyApp_key", value_2]]
  * This method transforms an object like {'@MyApp_user': myUserValue, '@MyApp_key': myKeyValue}
  * to an array of key-value pairs in the above format and removes key-value pairs that are being set to null
-
-* @return an array of key - value pairs <[key, value]>
+ *
+ * @return an array of key - value pairs <[key, value]>
  */
 function prepareKeyValuePairsForStorage(
     data: Record<OnyxKey, OnyxInput<OnyxKey>>,
@@ -1173,10 +1173,23 @@ function prepareKeyValuePairsForStorage(
     return pairs;
 }
 
+/**
+ * Merges an array of changes with an existing value or creates a single change.
+ *
+ * @param changes Array of changes that should be merged
+ * @param existingValue The existing value that should be merged with the changes
+ */
 function mergeChanges<TValue extends OnyxInput<OnyxKey> | undefined, TChange extends OnyxInput<OnyxKey> | undefined>(changes: TChange[], existingValue?: TValue): FastMergeResult<TChange> {
     return applyMerge('merge', changes, existingValue);
 }
 
+/**
+ * Merges an array of changes with an existing value or creates a single change.
+ * It will also mark deep nested objects that need to be entirely replaced during the merge.
+ *
+ * @param changes Array of changes that should be merged
+ * @param existingValue The existing value that should be merged with the changes
+ */
 function mergeAndMarkChanges<TValue extends OnyxInput<OnyxKey> | undefined, TChange extends OnyxInput<OnyxKey> | undefined>(
     changes: TChange[],
     existingValue?: TValue,
@@ -1185,7 +1198,7 @@ function mergeAndMarkChanges<TValue extends OnyxInput<OnyxKey> | undefined, TCha
 }
 
 /**
- * Merges an array of changes with an existing value or creates a single change
+ * Merges an array of changes with an existing value or creates a single change.
  *
  * @param changes Array of changes that should be merged
  * @param existingValue The existing value that should be merged with the changes

lib/storage/providers/SQLiteProvider.ts

@@ -44,11 +44,17 @@ type PageCountResult = {
 const DB_NAME = 'OnyxDB';
 let db: NitroSQLiteConnection;
 
-function replacer(key: string, value: unknown) {
+/**
+ * Prevents the stringifying of the object markers.
+ */
+function objectMarkRemover(key: string, value: unknown) {
     if (key === utils.ONYX_INTERNALS__REPLACE_OBJECT_MARK) return undefined;
     return value;
 }
 
+/**
+ * Transforms the replace null patches into SQL queries to be passed to JSON_REPLACE.
+ */
 function generateJSONReplaceSQLQueries(key: string, patches: FastMergeReplaceNullPatch[]): string[][] {
     const queries = patches.map(([pathArray, value]) => {
         const jsonPath = `$.${pathArray.join('.')}`;
@@ -114,12 +120,15 @@ const provider: StorageProvider = {
     multiMerge(pairs) {
         const commands: BatchQueryCommand[] = [];
 
+        // Query to merge the change into the DB value.
         const patchQuery = `INSERT INTO keyvaluepairs (record_key, valueJSON)
             VALUES (:key, JSON(:value))
             ON CONFLICT DO UPDATE
             SET valueJSON = JSON_PATCH(valueJSON, JSON(:value));
         `;
         const patchQueryArguments: string[][] = [];
+
+        // Query to fully replace the nested objects of the DB value.
         const replaceQuery = `UPDATE keyvaluepairs
             SET valueJSON = JSON_REPLACE(valueJSON, ?, JSON(?))
             WHERE record_key = ?;
@@ -128,12 +137,9 @@ const provider: StorageProvider = {
 
         const nonNullishPairs = pairs.filter((pair) => pair[1] !== undefined);
 
-        // eslint-disable-next-line @typescript-eslint/prefer-for-of
-        for (let i = 0; i < nonNullishPairs.length; i++) {
-            const [key, value, replaceNullPatches] = nonNullishPairs[i];
-
-            const valueAfterReplace = JSON.stringify(value, replacer);
-            patchQueryArguments.push([key, valueAfterReplace]);
+        for (const [key, value, replaceNullPatches] of nonNullishPairs) {
+            const changeWithoutMarkers = JSON.stringify(value, objectMarkRemover);
+            patchQueryArguments.push([key, changeWithoutMarkers]);
 
             const patches = replaceNullPatches ?? [];
             if (patches.length > 0) {

lib/types.ts

@@ -486,7 +486,13 @@ type InitOptions = {
 // eslint-disable-next-line @typescript-eslint/no-explicit-any
 type GenericFunction = (...args: any[]) => any;
 
-type MultiMergeReplaceNullPatches = {[TKey in OnyxKey]: FastMergeReplaceNullPatch[]};
+/**
+ * Represents a record where the key is a collection member key and the value is a list of
+ * tuples that we'll use to replace the nested objects of that collection member record with something else.
+ */
+type MultiMergeReplaceNullPatches = {
+    [TKey in OnyxKey]: FastMergeReplaceNullPatch[];
+};
 
 /**
  * Represents a combination of Merge and Set operations that should be executed in Onyx

lib/utils.ts

@@ -3,6 +3,13 @@ import type {ConnectOptions, OnyxInput, OnyxKey} from './types';
 type EmptyObject = Record<string, never>;
 type EmptyValue = EmptyObject | null | undefined;
 
+/**
+ * A tuple where the first value is the path to the nested object that contains the
+ * internal `ONYX_INTERNALS__REPLACE_OBJECT_MARK` flag, and the second value is the data we want to replace
+ * in that path.
+ *
+ * This tuple will be used in SQLiteProvider to replace the nested object using `JSON_REPLACE`.
+ * */
 type FastMergeReplaceNullPatch = [string[], unknown];
 
 type FastMergeOptions = {
@@ -18,7 +25,7 @@ type FastMergeOptions = {
 };
 
 type FastMergeMetadata = {
-    /** The path to the object that contains the internal "ONYX_INTERNALS__REPLACE_OBJECT_MARK" flag. */
+    /** The list of tuples that will be used in SQLiteProvider to replace the nested objects using `JSON_REPLACE`. */
     replaceNullPatches: FastMergeReplaceNullPatch[];
 };
 

tests/unit/fastMergeTest.ts

@@ -138,7 +138,7 @@ describe('fastMerge', () => {
         expect(result.replaceNullPatches).toEqual([[['b', 'd'], {h: 'h'}]]);
     });
 
-    it('should completely replace the target object with its source when the source has the "ONYX_INTERNALS__REPLACE_OBJECT_MARK" flag and "shouldReplaceMarkedObjects" is true', () => {
+    it('should completely replace the target object with its source when the source has the "ONYX_INTERNALS__REPLACE_OBJECT_MARK" flag and "objectRemovalMode" is set to "replace"', () => {
         const result = utils.fastMerge(
             testObject,
             {