Merge pull request Expensify#761 from callstack-internal/feature/structural-sharing-cache-pr-1

[Structural Sharing] PR 1: Centralized key utilities with bidirectional indexes

Author: Julesssss

API-INTERNAL.md

@@ -17,6 +17,11 @@
 <dt><a href="#getDeferredInitTask">getDeferredInitTask()</a></dt>
 <dd><p>Getter - returns the deffered init task.</p>
 </dd>
+<dt><a href="#afterInit">afterInit(action)</a> ⇒</dt>
+<dd><p>Executes an action after Onyx has been initialized.
+If Onyx is already initialized, the action is executed immediately.
+Otherwise, it waits for initialization to complete before executing.</p>
+</dd>
 <dt><a href="#getSkippableCollectionMemberIDs">getSkippableCollectionMemberIDs()</a></dt>
 <dd><p>Getter - returns the skippable collection member IDs.</p>
 </dd>
@@ -54,45 +59,6 @@ to the values for those keys (correctly typed) such as <code>[OnyxCollection&lt;
 <dt><a href="#getAllKeys">getAllKeys()</a></dt>
 <dd><p>Returns current key names stored in persisted storage</p>
 </dd>
-<dt><a href="#getCollectionKeys">getCollectionKeys()</a></dt>
-<dd><p>Returns set of all registered collection keys</p>
-</dd>
-<dt><a href="#isCollectionKey">isCollectionKey()</a></dt>
-<dd><p>Checks to see if the subscriber&#39;s supplied key
-is associated with a collection of keys.</p>
-</dd>
-<dt><a href="#isCollectionMember">isCollectionMember(key)</a> ⇒</dt>
-<dd><p>Checks if a given key is a collection member key (not just a collection key).</p>
-</dd>
-<dt><a href="#isRamOnlyKey">isRamOnlyKey(key)</a> ⇒</dt>
-<dd><p>Checks if a given key is a RAM-only key, RAM-only collection key, or a RAM-only collection member</p>
-<p>For example:</p>
-<p>For the following Onyx setup</p>
-<p>ramOnlyKeys: [&quot;ramOnlyKey&quot;, &quot;ramOnlyCollection_&quot;]</p>
-<ul>
-<li><code>isRamOnlyKey(&quot;ramOnlyKey&quot;)</code> would return true</li>
-<li><code>isRamOnlyKey(&quot;ramOnlyCollection_&quot;)</code> would return true</li>
-<li><code>isRamOnlyKey(&quot;ramOnlyCollection_1&quot;)</code> would return true</li>
-<li><code>isRamOnlyKey(&quot;someOtherKey&quot;)</code> would return false</li>
-</ul>
-</dd>
-<dt><a href="#splitCollectionMemberKey">splitCollectionMemberKey(key, collectionKey)</a> ⇒</dt>
-<dd><p>Splits a collection member key into the collection key part and the ID part.</p>
-</dd>
-<dt><a href="#isKeyMatch">isKeyMatch()</a></dt>
-<dd><p>Checks to see if a provided key is the exact configured key of our connected subscriber
-or if the provided key is a collection member key (in case our configured key is a &quot;collection key&quot;)</p>
-</dd>
-<dt><a href="#getCollectionKey">getCollectionKey(key)</a> ⇒</dt>
-<dd><p>Extracts the collection identifier of a given collection member key.</p>
-<p>For example:</p>
-<ul>
-<li><code>getCollectionKey(&quot;report_123&quot;)</code> would return &quot;report_&quot;</li>
-<li><code>getCollectionKey(&quot;report_&quot;)</code> would return &quot;report_&quot;</li>
-<li><code>getCollectionKey(&quot;report_-1_something&quot;)</code> would return &quot;report_&quot;</li>
-<li><code>getCollectionKey(&quot;sharedNVP_user_-1_something&quot;)</code> would return &quot;sharedNVP_user_&quot;</li>
-</ul>
-</dd>
 <dt><a href="#tryGetCachedValue">tryGetCachedValue()</a></dt>
 <dd><p>Tries to get a value from the cache. If the value is not present in cache it will return the default value or undefined.
 If the requested key is a collection, it will return an object with all the collection members.</p>
@@ -223,6 +189,20 @@ Getter - returns the default key states.
 Getter - returns the deffered init task.
 
 **Kind**: global function  
+<a name="afterInit"></a>
+
+## afterInit(action) ⇒
+Executes an action after Onyx has been initialized.
+If Onyx is already initialized, the action is executed immediately.
+Otherwise, it waits for initialization to complete before executing.
+
+**Kind**: global function  
+**Returns**: The result of the action  
+
+| Param | Description |
+| --- | --- |
+| action | The action to execute after initialization |
+
 <a name="getSkippableCollectionMemberIDs"></a>
 
 ## getSkippableCollectionMemberIDs()
@@ -312,93 +292,6 @@ Deletes a subscription ID associated with its corresponding key.
 Returns current key names stored in persisted storage
 
 **Kind**: global function  
-<a name="getCollectionKeys"></a>
-
-## getCollectionKeys()
-Returns set of all registered collection keys
-
-**Kind**: global function  
-<a name="isCollectionKey"></a>
-
-## isCollectionKey()
-Checks to see if the subscriber's supplied key
-is associated with a collection of keys.
-
-**Kind**: global function  
-<a name="isCollectionMember"></a>
-
-## isCollectionMember(key) ⇒
-Checks if a given key is a collection member key (not just a collection key).
-
-**Kind**: global function  
-**Returns**: true if the key is a collection member, false otherwise  
-
-| Param | Description |
-| --- | --- |
-| key | The key to check |
-
-<a name="isRamOnlyKey"></a>
-
-## isRamOnlyKey(key) ⇒
-Checks if a given key is a RAM-only key, RAM-only collection key, or a RAM-only collection member
-
-For example:
-
-For the following Onyx setup
-
-ramOnlyKeys: ["ramOnlyKey", "ramOnlyCollection_"]
-
-- `isRamOnlyKey("ramOnlyKey")` would return true
-- `isRamOnlyKey("ramOnlyCollection_")` would return true
-- `isRamOnlyKey("ramOnlyCollection_1")` would return true
-- `isRamOnlyKey("someOtherKey")` would return false
-
-**Kind**: global function  
-**Returns**: true if key is a RAM-only key, RAM-only collection key, or a RAM-only collection member  
-
-| Param | Description |
-| --- | --- |
-| key | The key to check |
-
-<a name="splitCollectionMemberKey"></a>
-
-## splitCollectionMemberKey(key, collectionKey) ⇒
-Splits a collection member key into the collection key part and the ID part.
-
-**Kind**: global function  
-**Returns**: A tuple where the first element is the collection part and the second element is the ID part,
-or throws an Error if the key is not a collection one.  
-
-| Param | Description |
-| --- | --- |
-| key | The collection member key to split. |
-| collectionKey | The collection key of the `key` param that can be passed in advance to optimize the function. |
-
-<a name="isKeyMatch"></a>
-
-## isKeyMatch()
-Checks to see if a provided key is the exact configured key of our connected subscriber
-or if the provided key is a collection member key (in case our configured key is a "collection key")
-
-**Kind**: global function  
-<a name="getCollectionKey"></a>
-
-## getCollectionKey(key) ⇒
-Extracts the collection identifier of a given collection member key.
-
-For example:
-- `getCollectionKey("report_123")` would return "report_"
-- `getCollectionKey("report_")` would return "report_"
-- `getCollectionKey("report_-1_something")` would return "report_"
-- `getCollectionKey("sharedNVP_user_-1_something")` would return "sharedNVP_user_"
-
-**Kind**: global function  
-**Returns**: The plain collection key or throws an Error if the key is not a collection one.  
-
-| Param | Description |
-| --- | --- |
-| key | The collection key to process. |
-
 <a name="tryGetCachedValue"></a>
 
 ## tryGetCachedValue()

lib/DevTools/RealDevTools.ts

@@ -1,5 +1,5 @@
 import type {IDevTools, DevtoolsOptions, DevtoolsConnection, ReduxDevtools} from './types';
-import OnyxUtils from '../OnyxUtils';
+import OnyxKeys from '../OnyxKeys';
 
 const ERROR_LABEL = 'Onyx DevTools - Error: ';
 
@@ -77,7 +77,7 @@ class RealDevTools implements IDevTools {
     clearState(keysToPreserve: string[] = []): void {
         const newState = Object.entries(this.state).reduce((obj: Record<string, unknown>, [key, value]) => {
             // eslint-disable-next-line no-param-reassign
-            obj[key] = keysToPreserve.some((preserveKey) => OnyxUtils.isKeyMatch(preserveKey, key)) ? value : this.defaultState[key];
+            obj[key] = keysToPreserve.some((preserveKey) => OnyxKeys.isKeyMatch(preserveKey, key)) ? value : this.defaultState[key];
             return obj;
         }, {});
 

lib/Onyx.ts

@@ -23,6 +23,7 @@ import type {
     SetOptions,
 } from './types';
 import OnyxUtils from './OnyxUtils';
+import OnyxKeys from './OnyxKeys';
 import logMessages from './logMessages';
 import type {Connection} from './OnyxConnectionManager';
 import connectionManager from './OnyxConnectionManager';
@@ -55,13 +56,13 @@ function init({
     OnyxUtils.setSkippableCollectionMemberIDs(new Set(skippableCollectionMemberIDs));
     OnyxUtils.setSnapshotMergeKeys(new Set(snapshotMergeKeys));
 
-    cache.setRamOnlyKeys(new Set<OnyxKey>(ramOnlyKeys));
+    OnyxKeys.setRamOnlyKeys(new Set<OnyxKey>(ramOnlyKeys));
 
     if (shouldSyncMultipleInstances) {
         Storage.keepInstancesSync?.((key, value) => {
             // RAM-only keys should never sync from storage as they may have stale persisted data
             // from before the key was migrated to RAM-only.
-            if (OnyxUtils.isRamOnlyKey(key)) {
+            if (OnyxKeys.isRamOnlyKey(key)) {
                 return;
             }
 
@@ -70,7 +71,7 @@ function init({
             // Check if this is a collection member key to prevent duplicate callbacks
             // When a collection is updated, individual members sync separately to other tabs
             // Setting isProcessingCollectionUpdate=true prevents triggering collection callbacks for each individual update
-            const isKeyCollectionMember = OnyxUtils.isCollectionMember(key);
+            const isKeyCollectionMember = OnyxKeys.isCollectionMember(key);
 
             OnyxUtils.keyChanged(key, value as OnyxValue<typeof key>, undefined, isKeyCollectionMember);
         });
@@ -87,7 +88,7 @@ function init({
     // eager cache loading populates the key index (cache.setAllKeys) inside initializeWithDefaultKeyStates,
     // and the evictable keys list depends on that index being populated.
     OnyxUtils.initializeWithDefaultKeyStates()
-        .then(() => cache.addEvictableKeysToRecentlyAccessedList(OnyxUtils.isCollectionKey, OnyxUtils.getAllKeys))
+        .then(() => cache.addEvictableKeysToRecentlyAccessedList(OnyxKeys.isCollectionKey, OnyxUtils.getAllKeys))
         .then(OnyxUtils.getDeferredInitTask().resolve);
 }
 
@@ -204,7 +205,7 @@ function merge<TKey extends OnyxKey>(key: TKey, changes: OnyxMergeInput<TKey>):
         const skippableCollectionMemberIDs = OnyxUtils.getSkippableCollectionMemberIDs();
         if (skippableCollectionMemberIDs.size) {
             try {
-                const [, collectionMemberID] = OnyxUtils.splitCollectionMemberKey(key);
+                const [, collectionMemberID] = OnyxKeys.splitCollectionMemberKey(key);
                 if (skippableCollectionMemberIDs.has(collectionMemberID)) {
                     // The key is a skippable one, so we set the new changes to undefined.
                     // eslint-disable-next-line no-param-reassign
@@ -348,7 +349,7 @@ function clear(keysToPreserve: OnyxKey[] = []): Promise<void> {
                 //      to null would cause unknown behavior)
                 //   2.1 However, if a default key was explicitly set to null, we need to reset it to the default value
                 for (const key of allKeys) {
-                    const isKeyToPreserve = keysToPreserve.some((preserveKey) => OnyxUtils.isKeyMatch(preserveKey, key));
+                    const isKeyToPreserve = keysToPreserve.some((preserveKey) => OnyxKeys.isKeyMatch(preserveKey, key));
                     const isDefaultKey = key in defaultKeyStates;
 
                     // If the key is being removed or reset to default:
@@ -361,7 +362,7 @@ function clear(keysToPreserve: OnyxKey[] = []): Promise<void> {
                         if (newValue !== oldValue) {
                             cache.set(key, newValue);
 
-                            const collectionKey = OnyxUtils.getCollectionKey(key);
+                            const collectionKey = OnyxKeys.getCollectionKey(key);
 
                             if (collectionKey) {
                                 if (!keyValuesToResetAsCollection[collectionKey]) {
@@ -396,7 +397,7 @@ function clear(keysToPreserve: OnyxKey[] = []): Promise<void> {
                 // Exclude RAM-only keys to prevent them from being saved to storage
                 const defaultKeyValuePairs = Object.entries(
                     Object.keys(defaultKeyStates)
-                        .filter((key) => !keysToPreserve.some((preserveKey) => OnyxUtils.isKeyMatch(preserveKey, key)) && !OnyxUtils.isRamOnlyKey(key))
+                        .filter((key) => !keysToPreserve.some((preserveKey) => OnyxKeys.isKeyMatch(preserveKey, key)) && !OnyxKeys.isRamOnlyKey(key))
                         .reduce((obj: KeyValueMapping, key) => {
                             // eslint-disable-next-line no-param-reassign
                             obj[key] = defaultKeyStates[key];
@@ -499,8 +500,8 @@ function update<TKey extends OnyxKey>(data: Array<OnyxUpdate<TKey>>): Promise<vo
         // Group all the collection-related keys and update each collection in a single `mergeCollection` call.
         // This is needed to prevent multiple `mergeCollection` calls for the same collection and `merge` calls for the individual items of the said collection.
         // This way, we ensure there is no race condition in the queued updates of the same key.
-        for (const collectionKey of OnyxUtils.getCollectionKeys()) {
-            const collectionItemKeys = Object.keys(updateQueue).filter((key) => OnyxUtils.isKeyMatch(collectionKey, key));
+        for (const collectionKey of OnyxKeys.getCollectionKeys()) {
+            const collectionItemKeys = Object.keys(updateQueue).filter((key) => OnyxKeys.isKeyMatch(collectionKey, key));
             if (collectionItemKeys.length <= 1) {
                 // If there are no items of this collection in the updateQueue, we should skip it.
                 // If there is only one item, we should update it individually, therefore retain it in the updateQueue.