docs: read-cache-after-write consistency and event filtering (#3193)

Signed-off-by: Attila Mészáros <a_meszaros@apple.com>

Author: csviri

docs/content/en/blog/news/primary-cache-for-next-recon.md

@@ -5,6 +5,15 @@ author: >-
   [Attila Mészáros](https://github.com/csviri) and [Chris Laprun](https://github.com/metacosm)
 ---
 
+{{% alert title="Deprecated" %}}
+
+Read-cache-after-write consistency feature replaces this functionality. (since version 5.3.0)
+
+> It provides this functionality also for secondary resources and optimistic locking
+is not required anymore. See [details here](./../../docs/documentation/reconciler.md#read-cache-after-write-consistency-and-event-filtering).
+{{% /alert %}}
+
+
 We recently released v5.1 of Java Operator SDK (JOSDK). One of the highlights of this release is related to a topic of
 so-called
 [allocated values](https://github.com/kubernetes/community/blob/master/contributors/devel/sig-architecture/api-conventions.md#representing-allocated-values

docs/content/en/docs/documentation/reconciler.md

@@ -160,7 +160,84 @@ annotation. If you do not specify a finalizer name, one will be automatically ge
 
 From v5, by default, the finalizer is added using Server Side Apply. See also `UpdateControl` in docs.
 
-### Making sure the primary resource is up to date for the next reconciliation
+### Read-cache-after-write consistency and event filtering
+
+It is an inherent issue with Informers that their caches are eventually consistent even
+with the updates to Kubernetes API done from the controller. From version 5.3.0 the framework
+supports stronger guarantees, both for primary and secondary resources. If this feature is used:
+
+1. Reading from the cache after our update — even within the same reconciliation — returns a fresh resource.
+   "Fresh" means at least the version of the resource that is in the response from our update,
+   or a more recent version if some other party updated the resource after our update. In particular, this means that
+   you can safely store state (e.g. generated IDs) in the status sub-resource of your resources since it is now
+   guaranteed that the stored values will be observable during the next reconciliation.
+2. Filtering events for our updates. When a controller updates a resource an event is produced by the Kubernetes API and
+   propagated to Informer. This would normally trigger another reconciliation. This is, however, not optimal since we
+   already have that up-to-date resource in the current reconciliation cache. There is generally no need to reconcile
+   that resource again. This feature also makes sure that the reconciliation is not triggered from the event from our
+   writes.
+
+
+In order to benefit from these stronger guarantees, use [`ResourceOperations`](https://github.com/operator-framework/java-operator-sdk/blob/main/operator-framework-core/src/main/java/io/javaoperatorsdk/operator/api/reconciler/ResourceOperations.java)
+from the context of the reconciliation:   
+
+```java 
+
+public UpdateControl<WebPage> reconcile(WebPage webPage, Context<WebPage> context) {
+    
+    ConfigMap managedConfigMap = prepareConfigMap(webPage);
+    // filtering and caching update
+    context.resourceOperations().serverSideApply(managedConfigMap);
+    
+    // fresh resource instantly available from our update in the cache
+    var upToDateResource = context.getSecondaryResource(ConfigMap.class);
+    
+    makeStatusChanges(webPage);
+    
+    // built in update methods by default use this feature
+    return UpdateControl.patchStatus(webPage);
+}
+```
+
+`UpdateControl` and `ErrorStatusUpdateControl` by default use this functionality, but you can also update your primary resource at any time during the reconciliation using `ResourceOperations`:
+
+```java 
+
+public UpdateControl<WebPage> reconcile(WebPage webPage, Context<WebPage> context) {
+
+    makeStatusChanges(webPage);
+    // this is equivalent to UpdateControl.patchStatus(webpage)
+    context.resourceOperations().serverSideApplyPrimaryStatus(webPage);
+    return UpdateControl.noUpdate();
+}
+```
+
+If your reconciler is built around the assumption that new reconciliations would occur after its own updates, a new
+`reschedule` method is provided on `UpdateControl` to immediately reschedule a new reconciliation, to mimic the previous
+behavior.
+
+### Caveats
+
+- This feature is implemented on top of the Fabric8 client informers, using additional caches in `InformerEventSource`,
+  so it is safe to use `context.getSecondaryResources(..)` or `InformerEventSource.get(ResourceID)`methods. Listing
+  resources directly via `InformerEventSource.list(..)`, however, won't work since this method directly reads from the
+  underlying informer cache, thus bypassing the additional caches that make the feature possible.
+
+
+### Notes
+- This [talk](https://www.youtube.com/watch?v=HrwHh5Yh6AM&t=1387s) mentions this feature.
+- [Umbrella issue](https://github.com/operator-framework/java-operator-sdk/issues/2944) on GitHub.
+- We were able to implement this feature since Kubernetes introduces guideline to compare 
+  resource versions. See the details [here](https://github.com/kubernetes/enhancements/tree/master/keps/sig-api-machinery/5504-comparable-resource-version).
+
+### Making sure the primary resource is up to date for the next reconciliation (deprecated)
+
+{{% alert title="Deprecated" %}}
+
+Read-cache-after-write consistency feature replaces this functionality.
+
+> It provides this functionality also for secondary resources and optimistic locking is not required anymore. See details above.
+{{% /alert %}}
 
 It is typical to want to update the status subresource with the information that is available during the reconciliation.
 This is sometimes referred to as the last observed state. When the primary resource is updated, though, the framework
@@ -263,30 +340,30 @@ See also [sample](https://github.com/operator-framework/java-operator-sdk/blob/m
 ### Expectations
 
 Expectations are a pattern to ensure that, during reconciliation, your secondary resources are in a certain state.
-For a more detailed explanation see [this blogpost](https://ahmet.im/blog/controller-pitfalls/#expectations-pattern).
-You can find framework support for this pattern in [`io.javaoperatorsdk.operator.processing.expectation`](https://github.com/operator-framework/java-operator-sdk/blob/main/operator-framework-core/src/main/java/io/javaoperatorsdk/operator/processing/expectation/) 
-package. See also related [integration test](https://github.com/operator-framework/java-operator-sdk/blob/main/operator-framework/src/test/java/io/javaoperatorsdk/operator/baseapi/expectation/ExpectationReconciler.java).
-Note that this feature is marked as `@Experimental`, since based on feedback the API might be improved / changed, but we intend 
-to support it, later also might be integrated to Dependent Resources and/or Workflows.
+For a more detailed explanation, see [this blogpost](https://ahmet.im/blog/controller-pitfalls/#expectations-pattern).
+You can find framework support for this pattern in the [`io.javaoperatorsdk.operator.processing.expectation`](https://github.com/operator-framework/java-operator-sdk/blob/main/operator-framework-core/src/main/java/io/javaoperatorsdk/operator/processing/expectation/)
+package. See also the related [integration test](https://github.com/java-operator-sdk/java-operator-sdk/blob/main/operator-framework/src/test/java/io/javaoperatorsdk/operator/baseapi/expectation/onallevent/ExpectationReconciler.java).
+Note that this feature is marked as `@Experimental`: based on feedback the API may be improved or changed, but we intend
+to keep supporting it and may later integrate it into Dependent Resources and/or Workflows.
 
-The idea is the nutshell, is that you can track your expectations in the expectation manager in the reconciler
-which has an API that covers the common use cases. 
+The idea, in a nutshell, is that you can track your expectations in the expectation manager within the reconciler,
+which provides an API covering the common use cases.
 
-The following sample is the simplified version of the integration test that implements the logic that creates a 
-deployment and sets status message if there are the target three replicas ready:
+The following is a simplified version of the integration test that implements the logic to create a
+deployment and set a status message once the target three replicas are ready:
 
 ```java
 public class ExpectationReconciler implements Reconciler<ExpectationCustomResource> {
 
     // some code is omitted
-    
+
     private final ExpectationManager<ExpectationCustomResource> expectationManager =
             new ExpectationManager<>();
 
     @Override
     public UpdateControl<ExpectationCustomResource> reconcile(
             ExpectationCustomResource primary, Context<ExpectationCustomResource> context) {
-        // exiting asap if there is an expectation that is not timed out neither fulfilled yet
+        // exit early if there is an expectation that has not yet timed out or been fulfilled
         if (expectationManager.ongoingExpectationPresent(primary, context)) {
             return UpdateControl.noUpdate();
         }
@@ -299,7 +376,7 @@ public class ExpectationReconciler implements Reconciler<ExpectationCustomResour
             return UpdateControl.noUpdate();
         } else {
             // Checks the expectation and removes it once it is fulfilled.
-            // In your logic you might add a next expectation based on your workflow.
+            // In your logic, you might add a next expectation based on your workflow.
             // Expectations have a name, so you can easily distinguish multiple expectations.
             var res = expectationManager.checkExpectation("deploymentReadyExpectation", primary, context);
             if (res.isFulfilled()) {

docs/content/en/docs/documentation/working-with-es-caches.md

@@ -216,3 +216,9 @@ With this index in place, you can retrieve the target resources very efficiently
                         .collect(Collectors.toSet()))
             .withNamespacesInheritedFromController().build(), context);
 ```
+
+## Read-cache-after-write consistency and event filtering
+
+From version 5.3.0 we provide stronger consistency guarantees and
+other features on top of basic informers, see [this section](reconciler.md#read-cache-after-write-consistency-and-event-filtering)
+for details.